b1d52306ad
Last day of Phase G0. Gateway promotes the D1 stub endpoints into
real reverse-proxies on :3110 fronting storaged + catalogd + ingestd
+ queryd. /v1 prefix lives at the edge — internal services route on
/storage, /catalog, /ingest, /sql, with the prefix stripped by a
custom Director per Kimi K2's D1-plan finding.
Routes:
/v1/storage/* → storaged
/v1/catalog/* → catalogd
/v1/ingest → ingestd
/v1/sql → queryd
Acceptance smoke 6/6 PASS — every assertion goes through :3110, none
direct to backing services. Full ingest → storage → catalog → query
round-trip verified end-to-end. The smoke's "rows[0].name=Alice"
assertion is the architectural payoff: five binaries, six HTTP
routes, one round-trip through one edge.
Cross-lineage scrum on shipped code:
- Opus 4.7 (opencode): 1 BLOCK + 2 WARN + 2 INFO
- Kimi K2-0905 (openrouter): 1 BLOCK + 3 WARN + 1 INFO (3 false positives, all from one wrong TrimPrefix theory)
- Qwen3-coder (openrouter): 5 completion tokens — "No BLOCKs."
Fixed (2, both Opus single-reviewer):
O-BLOCK: Director path stripping fails if upstream URL has a
non-empty path. The default Director's singleJoiningSlash runs
BEFORE the custom code, so an upstream like http://host/api
produces /api/v1/storage/... after the join — then TrimPrefix("/v1")
is a no-op because the string starts with /api. Fix: strip /v1
BEFORE calling origDirector. New TestProxy_SubPathUpstream regression
locks this in. Today: bare-host URLs only, dormant — but moving
gateway behind a sub-path in prod would have silently 404'd.
O-WARN2: url.Parse is permissive — typo "127.0.0.1:3211" (no scheme)
parses fine, produces empty Host, every request 502s. mustParseUpstream
fail-fast at startup with a clear message naming the offending
config field.
Dismissed (3, all Kimi, same false TrimPrefix theory):
K-BLOCK "TrimPrefix loops forever on //v1storage" — false, single
check-and-trim, no loop
K-WARN "no upper bound on repeated // removal" — same false theory
K-WARN "goroutines leak if upstream parse fails while binaries
running" — confused scope; binaries are separate OS processes
launched by the smoke script
D1 smoke updated (post-D6): the 501 stub probes are gone (gateway no
longer stubs /v1/ingest and /v1/sql). Replaced with proxy probes that
verify gateway forwards malformed requests to ingestd and queryd. Launch
order changed from parallel to dep-ordered (storaged → catalogd →
ingestd → queryd → gateway) since catalogd's rehydrate now needs
storaged, queryd's initial Refresh needs catalogd.
All six G0 smokes (D1 through D6) PASS end-to-end after every fix
round. Phase G0 substrate is complete: 5 binaries, 6 routes, 25 fixes
applied across 6 days from cross-lineage review.
G1+ next: gRPC adapters, Lance/HNSW vector indices, Go MCP SDK port,
distillation rebuild, observer + Langfuse integration.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
92 lines
3.3 KiB
Bash
Executable File
92 lines
3.3 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# D1 smoke — proves the Day 1 acceptance gate end-to-end.
|
|
# Builds all 5 binaries, launches them, polls /health on each until
|
|
# ready, then runs the actual probes. Exits 0 on success.
|
|
#
|
|
# Per Opus + Qwen BLOCK #2 review: replaced the prior `sleep 0.5`
|
|
# liveness gate with a poll loop so cold-start CI boxes don't race
|
|
# the bind.
|
|
#
|
|
# Usage: ./scripts/d1_smoke.sh
|
|
|
|
set -euo pipefail
|
|
cd "$(dirname "$0")/.."
|
|
|
|
export PATH="$PATH:/usr/local/go/bin"
|
|
|
|
echo "[d1-smoke] building..."
|
|
go build -o bin/ ./cmd/...
|
|
|
|
PIDS=()
|
|
trap 'echo "[d1-smoke] cleanup"; kill ${PIDS[@]} 2>/dev/null || true; wait 2>/dev/null || true' EXIT INT TERM
|
|
|
|
# Poll /health on each service until it returns 200 or we hit the
|
|
# 5s budget. Cheaper than a fixed sleep AND deterministic — first
|
|
# bind error surfaces immediately, slow boxes wait as long as needed.
|
|
poll_health() {
|
|
local name="$1" port="$2" deadline=$(($(date +%s) + 5))
|
|
while [ "$(date +%s)" -lt "$deadline" ]; do
|
|
if curl -sS --max-time 1 "http://127.0.0.1:$port/health" >/dev/null 2>&1; then
|
|
return 0
|
|
fi
|
|
sleep 0.05
|
|
done
|
|
echo " [d1-smoke] $name (:$port) failed to bind within 5s — log:"
|
|
tail -5 "/tmp/${name}.log" | sed 's/^/ /'
|
|
return 1
|
|
}
|
|
|
|
# Launch services in dependency order — catalogd's rehydrate needs
|
|
# storaged up; queryd's initial Refresh needs catalogd up. Gateway is
|
|
# last so its proxy probes have all upstreams ready.
|
|
echo "[d1-smoke] launching in dep order..."
|
|
for SPEC in "storaged:3211" "catalogd:3212" "ingestd:3213" "queryd:3214" "gateway:3110"; do
|
|
NAME="${SPEC%:*}"; PORT="${SPEC#*:}"
|
|
./bin/$NAME > /tmp/${NAME}.log 2>&1 &
|
|
PIDS+=($!)
|
|
if ! poll_health "$NAME" "$PORT"; then
|
|
exit 1
|
|
fi
|
|
done
|
|
|
|
echo "[d1-smoke] /health probes:"
|
|
FAILED=0
|
|
for SPEC in "gateway:3110" "storaged:3211" "catalogd:3212" "ingestd:3213" "queryd:3214"; do
|
|
NAME="${SPEC%:*}"; PORT="${SPEC#*:}"
|
|
RESP="$(curl -sS --max-time 2 "http://127.0.0.1:$PORT/health" || echo FAIL)"
|
|
if echo "$RESP" | grep -q "\"service\":\"$NAME\""; then
|
|
echo " ✓ $NAME (:$PORT) → $RESP"
|
|
else
|
|
echo " ✗ $NAME (:$PORT) → $RESP"
|
|
FAILED=1
|
|
fi
|
|
done
|
|
|
|
# Post-D6: gateway proxies /v1/ingest → ingestd and /v1/sql → queryd.
|
|
# We verify the proxy is wired by sending malformed-but-reachable
|
|
# requests and asserting the BACKING service's 400 (not the gateway's
|
|
# 502 which would mean the proxy can't reach the upstream).
|
|
echo "[d1-smoke] gateway proxy probes (D6+):"
|
|
# /v1/ingest with no `name` query param → ingestd returns 400.
|
|
HTTP="$(curl -sS -o /dev/null -w '%{http_code}' --max-time 2 -X POST "http://127.0.0.1:3110/v1/ingest")"
|
|
if [ "$HTTP" = "400" ]; then
|
|
echo " ✓ POST /v1/ingest (no name) → 400 from ingestd (proxy wired)"
|
|
else
|
|
echo " ✗ POST /v1/ingest → $HTTP (want 400; 502 means proxy can't reach ingestd)"
|
|
FAILED=1
|
|
fi
|
|
# /v1/sql with empty body → queryd returns 400.
|
|
HTTP="$(curl -sS -o /dev/null -w '%{http_code}' --max-time 2 -X POST -H 'Content-Type: application/json' "http://127.0.0.1:3110/v1/sql")"
|
|
if [ "$HTTP" = "400" ]; then
|
|
echo " ✓ POST /v1/sql (no body) → 400 from queryd (proxy wired)"
|
|
else
|
|
echo " ✗ POST /v1/sql → $HTTP (want 400; 502 means proxy can't reach queryd)"
|
|
FAILED=1
|
|
fi
|
|
|
|
if [ "$FAILED" -ne 0 ]; then
|
|
echo "[d1-smoke] FAILED"
|
|
exit 1
|
|
fi
|
|
echo "[d1-smoke] D1 acceptance gate: PASSED"
|