c522acec8b
J said "let's go" → "next" (option 3): actual flip via Bun mcp-server. Done. Real Bun-frontend traffic now reaches the Go substrate via /_go/* on Bun :3700, routed to the persistent Go gateway at :4110. Companion change in /home/profit/lakehouse (Rust legacy): mcp-server/index.ts: new /_go/* pass-through, opt-in via GO_LAKEHOUSE_URL env var. Off-by-default (returns 503 on /_go/* with rationale). Existing /api/* (Rust gateway) path unchanged. Committed locally on the demo/post-pr11 branch. System config: /etc/systemd/system/lakehouse-agent.service.d/go-cutover.conf adds Environment=GO_LAKEHOUSE_URL=http://127.0.0.1:4110 to the systemd-managed Bun service. Reversible via systemctl revert lakehouse-agent. Live verification (operator curl through Bun frontend): - /_go/health: gateway responds {"status":"ok","service":"gateway"} - /_go/v1/embed: nomic-embed-text-v2-moe vectors, dim=768 - /_go/v1/matrix/search vs persistent 200-worker corpus: rank=0 id=w-43 Brian Ramirez (Forklift Operator, Springfield IL) rank=1 id=w-102 Laura Long (Forklift Operator, Cleveland OH) rank=2 id=w-101 Terrence Gray (Forklift Operator, Champaign IL) 3/3 role match, top-1 in IL exactly - /api/health: lakehouse ok (Rust path unchanged — control verified) What this is NOT: - Not an nginx flip — devop.live/lakehouse/* still goes through /api/* → Rust :3100. /_go/* is parallel slice for opt-in. - Not a tool-level cutover — each /_go/<path> is a manual choice; no automatic mapping of Rust paths to Go equivalents. - Not a transformation layer — caller sends Go-shaped requests (e.g. /_go/v1/embed expects {texts, model}, not {text}). Three cutover unit properties verified: - ADDITIVE: zero modification to any existing Bun tool - REVERSIBLE: unset GO_LAKEHOUSE_URL → /_go/* → 503 - ISOLATED: Rust gateway state unaffected (different port, different binary, different MinIO bucket) This is the cutover slice operators can use to validate Go-side handlers under realistic frontend conditions before any production-traffic flip. Next step (deferred): pick a specific mcp-server tool to optionally route through Go with response- shape adapter — that's a product-visible flip rather than this infrastructure-visible slice. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
111 lines
4.5 KiB
Markdown
111 lines
4.5 KiB
Markdown
# G5 cutover slice live — first real Bun-frontend traffic to Go substrate
|
|
|
|
2026-05-01: the Bun mcp-server (which serves devop.live/lakehouse/
|
|
via nginx → :3700) now has a `/_go/*` pass-through that routes
|
|
requests to the Go gateway. Real frontend traffic can flow through
|
|
the Go substrate while the existing /api/* path (Rust gateway)
|
|
stays untouched.
|
|
|
|
## What changed
|
|
|
|
**Rust legacy repo** (`/home/profit/lakehouse/`):
|
|
|
|
- `mcp-server/index.ts`:
|
|
- New `GO_BASE = process.env.GO_LAKEHOUSE_URL || ""` (off-by-default)
|
|
- New `/_go/*` handler at the same shape as `/api/*` (forwarded
|
|
method+headers+body) but pointing at `${GO_BASE}${path}`
|
|
- Returns 503 with rationale when `GO_LAKEHOUSE_URL` is unset
|
|
- Returns 502 when Go gateway is unreachable (preserves operator
|
|
visibility into Go-side failures vs silent fallback)
|
|
|
|
**System config**:
|
|
|
|
- `/etc/systemd/system/lakehouse-agent.service.d/go-cutover.conf`:
|
|
```
|
|
[Service]
|
|
Environment=GO_LAKEHOUSE_URL=http://127.0.0.1:4110
|
|
```
|
|
Drop-in adds the env var to the systemd-managed Bun service.
|
|
Reversible via `systemctl revert lakehouse-agent.service` or by
|
|
removing the file.
|
|
|
|
**Go rewrite repo** (this commit): documentation + cross-runtime
|
|
parity log entry.
|
|
|
|
## Verification (all live, against persistent Go stack on :4110)
|
|
|
|
| Path | Routes to | Verdict |
|
|
|---|---|---|
|
|
| `GET /_go/health` | Go gateway `:4110/health` | `{"status":"ok","service":"gateway"}` |
|
|
| `POST /_go/v1/embed` | Go embedd via gateway | nomic-embed-text-v2-moe vectors, dim=768 |
|
|
| `POST /_go/v1/matrix/search` | Go matrixd via gateway | 3/3 Forklift Operators against the 200-worker persistent corpus |
|
|
| `GET /api/health` (control) | Rust gateway `:3100/health` | `lakehouse ok` (unchanged) |
|
|
|
|
Search results from the live cutover slice:
|
|
|
|
```
|
|
rank=0 id=w-43 dist=0.445 Brian Ramirez (Forklift Operator, Springfield, IL)
|
|
rank=1 id=w-102 dist=0.448 Laura Long (Forklift Operator, Cleveland, OH)
|
|
rank=2 id=w-101 dist=0.485 Terrence Gray (Forklift Operator, Champaign, IL)
|
|
```
|
|
|
|
3/3 role match. Top-1 in IL exactly. Real coordinator-shape query
|
|
served by Go substrate via Bun frontend.
|
|
|
|
## What this is
|
|
|
|
This is the **first time** real Bun-frontend traffic has reached the
|
|
Go substrate end-to-end. Production paths (devop.live/lakehouse/* →
|
|
nginx → Bun :3700 → /api/*) still go through Rust on :3100; the
|
|
new /_go/* path is a parallel slice that operators or external
|
|
tools can hit to validate Go under realistic frontend conditions.
|
|
|
|
The cutover unit is FULLY ADDITIVE:
|
|
- No existing tool modified
|
|
- /api/* unchanged
|
|
- All Bun mcp-server tools that currently call BASE (Rust) unchanged
|
|
- /_go/* fails-closed (503) when env var unset
|
|
|
|
The cutover unit is FULLY REVERSIBLE:
|
|
- Unset `GO_LAKEHOUSE_URL` → /_go/* returns 503
|
|
- `systemctl revert lakehouse-agent` → removes the drop-in
|
|
- `git revert` of the Bun source change → removes the handler
|
|
|
|
## What this is NOT
|
|
|
|
- **Not** an nginx flip. devop.live/lakehouse/* still goes
|
|
through Rust. Operators have to opt into /_go/*.
|
|
- **Not** a wholesale routing change. Each path under /_go/ is a
|
|
manual choice; no automatic routing of Rust paths to Go
|
|
equivalents.
|
|
- **Not** a transformation layer. /_go/v1/embed expects the Go
|
|
request shape (`{texts, model}`); /_go/* doesn't translate
|
|
Rust-shaped requests for the caller.
|
|
|
|
## Operational discipline
|
|
|
|
The systemd drop-in is at `/etc/systemd/system/lakehouse-agent.service.d/`.
|
|
Operators inspecting the system should know to look there for
|
|
cutover-related env overrides.
|
|
|
|
The persistent Go stack must be up at :4110 for /_go/* to succeed.
|
|
Tear down via `pkill -f 'bin/persistent-'`. Bring up via
|
|
`scripts/cutover/start_go_stack.sh`.
|
|
|
|
## What's next
|
|
|
|
- **Pick a real coordinator-facing tool to flip.** The current
|
|
/_go/* pass-through is operator-facing (manual curl). The next
|
|
step would be modifying ONE specific tool in mcp-server/index.ts
|
|
(e.g. `/search` → /v1/matrix/search) to optionally route through
|
|
Go when GO_BASE is set, with response shape adapter. That's a
|
|
product-visible flip; current state is infrastructure-visible.
|
|
- **Shadow-read mode.** A hybrid where /api/* ALSO calls /_go/*
|
|
in parallel and logs diffs to stderr — operators see Go-vs-Rust
|
|
divergences on real traffic without changing user-facing
|
|
behavior. Cleaner cutover discipline than direct flip.
|
|
- **Production load test.** This slice has been exercised on 3
|
|
manual curl requests. A traffic generator hitting /_go/v1/matrix/search
|
|
at sustained rate would expose latency / stability under load
|
|
that single-shot tests can't.
|