Per J's request: move the parallel-runtime comparison from
reports/cutover/ (where it lived as cutover-prep evidence) into
docs/ as the source-of-truth file. J will keep updating it as
fixes ship on either side.
Restructured for living-document use:
- Status header (last refresh date, owner, update triggers)
- 'How to update this doc' section with explicit dos and don'ts
- Decisions tracker at top — actioned items with commit refs
+ open backlog with LOC estimates
- Each comparison section now has 'Last verified' columns where
numbers are time-sensitive
- Change log section at bottom for one-line entries on every
meaningful refresh
The original at reports/cutover/architecture_comparison.md gains
a 'THIS IS A SNAPSHOT' header pointing at the docs/ source. Kept
as historical record but no longer the place to update.
Sister pointer file in /home/profit/lakehouse/docs/ARCHITECTURE_COMPARISON.md
so the doc is reachable from either repo side. That file explicitly
says the source lives in golangLAKEHOUSE and warns against
authoritative content in the pointer.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
J asked for the comparison before locking in primary line. This
report documents what's actually structurally different vs
implementation-level different, and what to do about each.
Key findings:
1. Python sidecar is the single biggest architectural lever
- Rust: gateway → HTTP → Python sidecar :3200 → HTTP → Ollama
- Go: gateway → HTTP → embedd → HTTP → Ollama (no Python)
- Sidecar adds zero compute over Ollama (just pydantic + httpx)
- 63× perf gap (8,119 vs 128 RPS) driven by sidecar + cache absence
2. Process model: Rust 1 mega-binary (14.9G RSS), Go 11 daemons
- Rust: simpler ops at small scale, panic blast radius = whole system
- Go: per-daemon scale + crash isolation, more config surface
3. Code volume: Go 15,128 lines vs Rust 35,447 + 1,237 sidecar
- Go is 43% the size doing similar work
- Gap concentrated in vectord (Rust 11k lines, Go 804 — Lance + benchmarking)
4. Distillation pipeline asymmetry
- Audit/observation: BOTH sides parallel-mature
- Production: Rust-only (materializer + replay + RAG/pref export)
- Go can READ everything but can't PRODUCE evidence
5. Production validators (FillValidator/EmailValidator/'/v1/validate')
- Rust has them (1,286 lines, 12 tests each)
- Go doesn't — matrix gate covers role bleed but not structural validation
Cross-cutting abstracts to address regardless of which wins:
- Drop Python sidecar from Rust (call Ollama directly)
- Add LRU embed cache to Rust aibridge
- Port materializer + replay + validators to Go
- Pin shared JSONL schemas as canonical (both runtimes consume same spec)
- Decide on Lance backend (defer until corpus > 5M rows)
If keeping Go primary: port materializer first, validators second,
skip Lance. If keeping Rust primary: drop Python + add cache,
port chatd 5-provider dispatcher + cross-role gate from Go.
Bottom line: substrate is parallel-mature on observation; producer
side is Rust-only; performance structurally favors Go ~60× on warm
workloads; operations favors Go on isolation; production deployment
favors Rust today.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
