29468b1413
Pre-Phase-G0 research sweep against current Go ecosystem state. Three upstream changes that the day-of SPEC missed: 1. DuckDB Go binding ownership transferred. marcboeker/go-duckdb is deprecated as of v2.5.0 — official maintainer is now github.com/duckdb/duckdb-go/v2 (DuckDB team + Marc Boeker joint hand-off). Current v2.10502.0 / DuckDB v1.5.2. SPEC §3.1 + component table updated. 2. Official Go MCP SDK exists. Switching from mark3labs/mcp-go (community) to github.com/modelcontextprotocol/go-sdk (official, Google collaboration, v1.5.0 stable, 4.4k stars, targets MCP spec 2025-11-25). Component table updated. 3. arrow-go is on v18, not v15. v18.5.2 (March 2026) has parquet encryption fixes relevant for PII-masked safe views. PRD locked stack + SPEC component table updated. Validated unchanged: coder/hnsw (220 stars, active), chi (still the clean-architecture pick over fiber/gin/echo). Surfaced for future use: anthropics/anthropic-sdk-go (official, available for direct Claude calls bypassing opencode if ever needed), duckdb-wasm (browser-side analytics future option), IVF as HNSW fallback if recall gate fails. See docs/RESEARCH_LOG_2026-04-28.md for full survey + sources. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
134 lines
5.0 KiB
Markdown
134 lines
5.0 KiB
Markdown
# Research Log — 2026-04-28
|
|
|
|
Survey of upstream Go ecosystem state before Phase G0 begins. Goal:
|
|
verify the SPEC's library choices reflect the *currently maintained*
|
|
upstream and surface anything that has shifted since the SPEC was
|
|
drafted earlier today.
|
|
|
|
## Sources used
|
|
|
|
- Context7 — `resolve-library-id` for `go-duckdb` and `arrow-go`
|
|
- WebSearch — six targeted queries on Go HNSW, MCP SDK, DuckDB
|
|
bindings, Anthropic SDK, HTTP frameworks, parquet maturity
|
|
- WebFetch — direct GitHub READMEs for `coder/hnsw`,
|
|
`modelcontextprotocol/go-sdk`, `duckdb/duckdb-go`
|
|
|
|
## Findings — three SPEC-changing pivots
|
|
|
|
### 1. DuckDB Go binding ownership transferred
|
|
|
|
The original SPEC named `marcboeker/go-duckdb` as the query-engine
|
|
library. The package has been formally **deprecated** in favor of the
|
|
official maintainer transfer.
|
|
|
|
| | Before (SPEC v1) | After |
|
|
|---|---|---|
|
|
| Repo | `github.com/marcboeker/go-duckdb` | `github.com/duckdb/duckdb-go/v2` |
|
|
| Maintainer | Marc Boeker (community) | DuckDB team + Marc Boeker (joint, official) |
|
|
| Latest | (varies) | v2.10502.0 (April 2026) |
|
|
| DuckDB engine | varies | v1.5.2 |
|
|
| Migration | n/a | `gofmt -w -r '"github.com/marcboeker/go-duckdb/v2" -> "github.com/duckdb/duckdb-go/v2"' .` |
|
|
|
|
**Why this matters:** the upstream migration happened at v2.5.0; using
|
|
the old import means we'd be on a deprecated branch from day one.
|
|
SPEC §3.1 now names the official path.
|
|
|
|
### 2. Official Go MCP SDK exists (Google collaboration)
|
|
|
|
The original SPEC named `mark3labs/mcp-go` (community implementation).
|
|
An official SDK now exists.
|
|
|
|
| | Before (SPEC v1) | After |
|
|
|---|---|---|
|
|
| Repo | `mark3labs/mcp-go` | `github.com/modelcontextprotocol/go-sdk` |
|
|
| Maintainer | mark3labs (community) | MCP org + Google (official) |
|
|
| Stars | (smaller) | 4.4k |
|
|
| Version | n/a | v1.5.0 stable |
|
|
| Spec compat | various | targets MCP 2025-11-25, backward-compat to 2024-11-05 |
|
|
| OSSF Scorecard | n/a | yes |
|
|
|
|
**Why this matters:** the MCP server (`cmd/mcp`) is one of the most
|
|
visible Go binaries we'll ship — it's what AI agents talk to. Using
|
|
the official SDK aligns with the MCP spec's evolution and gets us the
|
|
Google-tested code path.
|
|
|
|
### 3. arrow-go is on v18, not v15
|
|
|
|
The original SPEC referenced `arrow-go/v15`. Apache Arrow Go has
|
|
shipped through **v18.5.2** (March 2026).
|
|
|
|
| | Before (SPEC v1) | After |
|
|
|---|---|---|
|
|
| Module path | `apache/arrow-go/v15` | `apache/arrow-go/v18` |
|
|
| Latest | n/a | v18.5.2 |
|
|
| Recent fixes | n/a | parquet decryption, large string handling, complex type read |
|
|
|
|
**Why this matters:** v18 has parquet encryption fixes that are
|
|
relevant for our PII-masked safe views (per Rust ADR-017's federation
|
|
and the production cutover work). Skipping three major versions is
|
|
unnecessary risk.
|
|
|
|
## Findings — validations (SPEC unchanged)
|
|
|
|
### `coder/hnsw` — keep
|
|
|
|
220 stars, 45 commits, active CI, recent PRs/issues. Documented
|
|
import speed of 796.85 MB/s. No deprecation signals. In-memory
|
|
alternative to Pinecone/Weaviate, fits the no-external-vector-DB
|
|
constraint.
|
|
|
|
### `chi` for HTTP routing — keep
|
|
|
|
Confirmed as the "clean architecture, stdlib `net/http`, zero deps"
|
|
pick. Fiber is faster (36k req/s vs ~34k for chi/gin/echo) but uses
|
|
fasthttp, which is off the standard library path — wrong fit for our
|
|
"boring is good" Go ethos. Fiber stays as a documented alternative if
|
|
hot-path performance ever proves chi insufficient.
|
|
|
|
### `marcboeker/go-duckdb` → `duckdb/duckdb-go/v2`
|
|
|
|
Already covered above as pivot #1.
|
|
|
|
## New things worth noting (not SPEC-changing yet)
|
|
|
|
### Anthropic Go SDK is official
|
|
|
|
`github.com/anthropics/anthropic-sdk-go` is the official Anthropic Go
|
|
client library. We currently route Claude calls through OpenCode/Zen,
|
|
so this isn't on the Phase G0 critical path. **Worth knowing for**:
|
|
direct Claude API calls in `aibridge` if we ever want to bypass
|
|
opencode (e.g., for the overseer correction loop in dev mode without
|
|
a Zen subscription).
|
|
|
|
Added as a noted option in the SPEC `aibridge` row.
|
|
|
|
### DuckDB-Wasm exists
|
|
|
|
`github.com/duckdb/duckdb-wasm` brings DuckDB to the browser via
|
|
WebAssembly with Arrow / Parquet / CSV / JSON support. **Not in scope**
|
|
for Phase G0 but a future option if the UI ever needs client-side
|
|
analytical queries against fetched parquet (offline analytics over a
|
|
permit cache, etc.).
|
|
|
|
### IVF as an HNSW alternative
|
|
|
|
If during Phase G3 the HNSW recall validation gate (G3.2.C) shows
|
|
problems, IVF (Inverted File Index) is the next-best alternative:
|
|
faster index builds, lower memory, better filtered-search performance
|
|
than HNSW. No first-class Go IVF library was found — would need to
|
|
wrap FAISS via cgo or hand-roll. Documented as a fallback only.
|
|
|
|
## What I checked but found nothing actionable
|
|
|
|
- HTMX-specific Go libraries (`htmx-go`, `gotempl`, etc.) — none has
|
|
emerged as a clear standard. Sticking with `html/template` + raw
|
|
HTMX as the SPEC plans.
|
|
- Langfuse Go client — OSS support varies. SPEC's "hand-roll if
|
|
needed" stays.
|
|
|
|
## Outcome
|
|
|
|
SPEC §1 + §2 + §3.1 updated to reflect the three pivots. PRD's locked
|
|
stack table updated to match. No phase changes, no acceptance gate
|
|
changes, no new hard problems.
|