diff --git a/STATE_OF_PLAY.md b/STATE_OF_PLAY.md index 4c805a1..0d9c5b8 100644 --- a/STATE_OF_PLAY.md +++ b/STATE_OF_PLAY.md @@ -1,6 +1,6 @@ # STATE OF PLAY — Lakehouse-Go -**Last verified:** 2026-05-02 ~08:00 CDT +**Last verified:** 2026-05-02 ~10:30 CDT **Verified by:** **production-readiness gauntlet** — 21/21 smoke chain green, per-component scrum across 4 bundles, **3 cross-runtime parity probes all green post-fix** (validator: **6/6 match** after wire-format alignment shipped; materializer: 2/2 after omitempty fix; extract_json: 12/12). All findings surfaced by the parity probes have been actioned. Disposition: `reports/cutover/gauntlet_2026-05-02/disposition.md`. > **Read this FIRST.** When the user says "we're working on lakehouse," default to the Go rewrite (this repo); the Rust legacy at `/home/profit/lakehouse/` is maintenance-only. If memory contradicts this file, this file wins. Update it when something is verified working — not when a phase finishes. diff --git a/docs/ARCHITECTURE_COMPARISON.md b/docs/ARCHITECTURE_COMPARISON.md index e3545da..65bfcce 100644 --- a/docs/ARCHITECTURE_COMPARISON.md +++ b/docs/ARCHITECTURE_COMPARISON.md @@ -49,6 +49,7 @@ Don't: | 2026-05-01 | **coder/hnsw v0.6.1 panic — REAL FIX landed** | Lifted source-of-truth out of coder/hnsw via `i.vectors map[string][]float32` side store + `safeGraphAdd`/`safeGraphDelete` recover wrappers + warm-path rebuild fallback. Re-run: 0 failures across 19,622 scenarios (was 96-98% on 2/6). **DONE.** Architecture invariant in STATE_OF_PLAY "DO NOT RELITIGATE". | | 2026-05-02 | **Substrate fix verified at original failure scale** | Re-ran multitier 5min @ conc=50 (the footprint that originally surfaced the bug at 96-98% fail). Result: 132,211 scenarios at 438/sec, **6/6 classes at 0% failure**. Throughput dropped 1,115/sec → 438/sec because broken scenarios now do real HNSW Add work. Tails healthy: surge_fill_validate p99=1.53s, playbook_record_replay p99=2.32s. **Fix scales — closing the open thread.** | | 2026-05-02 | **Drop Python sidecar from Rust aibridge — DONE** | `crates/aibridge/src/client.rs` rewrite (commit `ba928b1` in lakehouse). AiClient now talks Ollama directly: per-text `/api/embed` loop, `/api/generate` for chat + rerank-loop + admin (unload/preload), `/api/ps` + `std::process::Command nvidia-smi` for vram_snapshot. Public API unchanged — 0 callers updated. Verification: cargo test -p aibridge 32/32 PASS, live smoke `/ai/embed` returns 768-dim vector + `/v1/chat` returns "OK". Sidecar's `lab_ui.py` + `pipeline_lab.py` (~888 LOC dev-only UIs) keep running; only the hot-path embed/generate/rerank/admin routes are retired. Process count drops from "mega-binary + sidecar" to "mega-binary alone". | +| 2026-05-02 | **Rust observability parity (trace-id + session JSONL) — DONE** | Mirrors the Go-side wave (commits `d6d2fdf` + `1a3a82a` in golangLAKEHOUSE). `crates/gateway/src/v1/iterate.rs` now reads `X-Lakehouse-Trace-Id` header, forwards to `/v1/chat` + `/v1/validate` hops, emits per-attempt Langfuse spans (`iterate.attempt[N]`), and writes one `SessionRecord` JSONL row per session via the new `crates/gateway/src/v1/session_log.rs` writer. New `crates/gateway/src/bin/parity_session_log` helper enables the cross-runtime parity probe at `scripts/cutover/parity/session_log_parity.sh` — **4/4 fixtures byte-equal** (including unicode prompts) after normalizing the 2 fields that legitimately differ (timestamp, daemon name). Both runtimes can write to the same path and DuckDB queries see them as one stream. 90/90 Rust unit tests PASS, 33 Go packages PASS. | | 2026-05-02 | **Port Rust materializer to Go (transforms.ts) — DONE** | `internal/materializer` + `cmd/materializer` + `materializer_smoke.sh`. Ports `transforms.ts` (12 transforms) + `build_evidence_index.ts`. Idempotency, day-partition, receipt. 14 tests green; on-wire JSON matches TS so both runtimes interoperate. | | 2026-05-02 | **Port Rust replay tool to Go — DONE** | `internal/replay` + `cmd/replay` + `replay_smoke.sh`. Ports `replay.ts` retrieve → bundle → /v1/chat → validate → log. Closes audit-FULL phase 7 live invocation on Go side. 14 tests green; same `data/_kb/replay_runs.jsonl` shape (schema=replay_run.v1) as TS. | | 2026-05-02 | **`/v1/validate` + `/v1/iterate` HTTP surface — DONE** | `cmd/validatord` (port 3221) hosts both endpoints. `internal/validator` gains `PlaybookValidator` (3rd kind), JSONL roster loader, and the `Iterate` orchestrator + `ExtractJSON` helper. Gateway proxies `/v1/validate` + `/v1/iterate` to validatord. Closes the last "Go-primary" backlog item (architecture_comparison.md item #7). 30+ tests + `validatord_smoke.sh` 5/5 PASS. | diff --git a/reports/cutover/gauntlet_2026-05-02/parity/session_log_parity.md b/reports/cutover/gauntlet_2026-05-02/parity/session_log_parity.md new file mode 100644 index 0000000..8040702 --- /dev/null +++ b/reports/cutover/gauntlet_2026-05-02/parity/session_log_parity.md @@ -0,0 +1,15 @@ +# session_log parity probe — Rust gateway vs Go validatord + +**Date:** 2026-05-02T10:38:10Z +**Rust helper:** `/home/profit/lakehouse/target/release/parity_session_log` +**Go helper:** `./bin/parity_session_log_go` + +Identical fixture inputs through each runtime's +`SessionRecord` builder + JSON marshaler. Match = byte-equal +after stripping `timestamp` (per-run wall clock) + `daemon` +("gateway" on Rust side, "validatord" on Go side; both are +valid producers in the same longitudinal log). + +**Tally:** 4 match · 0 diff (out of 4 fixtures) + +_No divergences — schema parity holds across all fixtures._ diff --git a/scripts/cutover/parity/session_log_helper/main.go b/scripts/cutover/parity/session_log_helper/main.go new file mode 100644 index 0000000..30810bc --- /dev/null +++ b/scripts/cutover/parity/session_log_helper/main.go @@ -0,0 +1,69 @@ +// session_log_helper — Go-side counterpart for the cross-runtime +// session-log parity probe. Reads a fixture JSON on stdin, builds a +// SessionRecord, emits one JSONL row to stdout. +// +// Used by scripts/cutover/parity/session_log_parity.sh. +package main + +import ( + "encoding/json" + "fmt" + "io" + "os" + "time" + + "git.agentview.dev/profit/golangLAKEHOUSE/internal/validator" +) + +// fixtureInput shape mirrors the Rust helper's accepted shape so both +// helpers can read the same stdin payload. +type fixtureInput struct { + SessionID string `json:"session_id"` + Kind string `json:"kind"` + Model string `json:"model"` + Provider string `json:"provider"` + Prompt string `json:"prompt"` + Iterations int `json:"iterations"` + MaxIterations int `json:"max_iterations"` + FinalVerdict string `json:"final_verdict"` + Attempts []validator.SessionAttemptRecord `json:"attempts"` + Artifact map[string]any `json:"artifact"` + GroundedInRoster *bool `json:"grounded_in_roster"` + DurationMs int64 `json:"duration_ms"` +} + +func main() { + buf, err := io.ReadAll(os.Stdin) + if err != nil { + fmt.Fprintf(os.Stderr, "read stdin: %v\n", err) + os.Exit(1) + } + var in fixtureInput + if err := json.Unmarshal(buf, &in); err != nil { + fmt.Fprintf(os.Stderr, "parse stdin: %v\n", err) + os.Exit(1) + } + rec := validator.SessionRecord{ + Schema: validator.SessionRecordSchema, + SessionID: in.SessionID, + Timestamp: time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC).Format(time.RFC3339), + Daemon: "validatord", + Kind: in.Kind, + Model: in.Model, + Provider: in.Provider, + Prompt: in.Prompt, + Iterations: in.Iterations, + MaxIterations: in.MaxIterations, + FinalVerdict: in.FinalVerdict, + Attempts: in.Attempts, + Artifact: in.Artifact, + GroundedInRoster: in.GroundedInRoster, + DurationMs: in.DurationMs, + } + body, err := json.Marshal(rec) + if err != nil { + fmt.Fprintf(os.Stderr, "marshal: %v\n", err) + os.Exit(1) + } + fmt.Println(string(body)) +} diff --git a/scripts/cutover/parity/session_log_parity.sh b/scripts/cutover/parity/session_log_parity.sh new file mode 100755 index 0000000..aec1004 --- /dev/null +++ b/scripts/cutover/parity/session_log_parity.sh @@ -0,0 +1,127 @@ +#!/usr/bin/env bash +# session_log_parity — verify Rust gateway and Go validatord write +# byte-equivalent SessionRecord JSONL rows from identical input. +# +# Why: the longitudinal session log (one row per /v1/iterate session) +# is the offline-analysis layer. Cross-runtime drift means a DuckDB +# query that works against one log fails against the other — the +# whole "unified longitudinal view" theory unravels. This probe is +# the regression gate. +# +# Approach (pure schema check, no live daemon needed): +# 1. Build both runtimes' session_log writers as standalone helper +# binaries that take a fixture JSON on stdin and emit one row. +# 2. Feed identical fixture inputs through each. +# 3. Diff the resulting rows after normalizing the few fields that +# MUST differ (timestamp, daemon). +# +# Outputs: reports/cutover/gauntlet_2026-05-02/parity/session_log_parity.md +# +# Exit 0 on schema match; exit 1 on drift. + +set -uo pipefail +cd "$(dirname "$0")/../../.." + +RUST_REPO="${RUST_REPO:-/home/profit/lakehouse}" +RUST_BIN="${RUST_BIN:-$RUST_REPO/target/release/parity_session_log}" +GO_BIN="${GO_BIN:-./bin/parity_session_log_go}" +OUT_DIR="reports/cutover/gauntlet_2026-05-02/parity" +mkdir -p "$OUT_DIR" +OUT="$OUT_DIR/session_log_parity.md" + +export PATH="$PATH:/usr/local/go/bin" + +# ── Build both helpers ───────────────────────────────────────────── +go build -o "$GO_BIN" ./scripts/cutover/parity/session_log_helper + +if [ ! -x "$RUST_BIN" ]; then + echo "[session-log-parity] building Rust helper..." + (cd "$RUST_REPO" && cargo build -p gateway --bin parity_session_log --release 2>&1 | tail -3) +fi +if [ ! -x "$RUST_BIN" ]; then + echo "[session-log-parity] SKIP: $RUST_BIN missing" + exit 0 +fi + +# ── Fixtures ─────────────────────────────────────────────────────── +# Parallel arrays — labels and fixtures keyed by index. Avoids a +# separator-in-string scheme (the prior `||` form silently injected +# a `|` character into every fixture body, which both helpers then +# rejected with empty output → trivially-equal "match" — caught +# during initial probe authoring 2026-05-02). +LABELS=( + "accepted_grounded" + "max_iter_exhausted" + "infra_error" + "unicode_in_prompt" +) +FIXTURES=( + '{"session_id":"trace-1","kind":"fill","model":"qwen3.5:latest","provider":"ollama","prompt":"produce a fill","iterations":1,"max_iterations":3,"final_verdict":"accepted","attempts":[{"iteration":0,"verdict_kind":"accepted","span_id":"sp-0"}],"artifact":{"fills":[{"candidate_id":"W-1"}]},"grounded_in_roster":true,"duration_ms":50}' + '{"session_id":"trace-2","kind":"fill","model":"qwen3.5:latest","provider":"ollama","prompt":"P","iterations":3,"max_iterations":3,"final_verdict":"max_iter_exhausted","attempts":[{"iteration":0,"verdict_kind":"validation_failed","error":"phantom W-X","span_id":"sp-a"},{"iteration":1,"verdict_kind":"validation_failed","error":"phantom W-Y","span_id":"sp-b"},{"iteration":2,"verdict_kind":"no_json","span_id":"sp-c"}],"duration_ms":3200}' + '{"session_id":"trace-3","kind":"playbook","model":"qwen","provider":"ollama","prompt":"P","iterations":0,"max_iterations":3,"final_verdict":"infra_error","attempts":[{"iteration":0,"verdict_kind":"infra_error","error":"connection refused"}],"duration_ms":12}' + '{"session_id":"trace-4","kind":"playbook","model":"qwen","provider":"ollama","prompt":"Café résumé ⭐ 你好","iterations":1,"max_iterations":3,"final_verdict":"accepted","attempts":[{"iteration":0,"verdict_kind":"accepted","span_id":"sp-u"}],"artifact":{"endorsed_names":["W-1"],"operation":"fill: X x1 in A, B","fingerprint":"abc"},"duration_ms":80}' +) + +normalize() { + # Strip the few fields that MUST differ between Rust+Go runs + # (timestamp = now; daemon = producer name). Sort the keys so the + # JSON serialization order doesn't break a byte-equal diff. + jq -cS 'del(.timestamp) | del(.daemon)' <<<"$1" +} + +TOTAL=0; MATCH=0; DIFF=0 +DIFF_DETAIL="" + +for i in "${!FIXTURES[@]}"; do + label="${LABELS[$i]}" + fixture="${FIXTURES[$i]}" + TOTAL=$((TOTAL+1)) + rust_row=$(printf '%s' "$fixture" | "$RUST_BIN" 2>&1 || echo '{"error":"rust_helper_failed"}') + go_row=$(printf '%s' "$fixture" | "$GO_BIN" 2>&1 || echo '{"error":"go_helper_failed"}') + # Refuse trivial-equal-empty matches: if either side produced no + # parseable JSON at all, mark this as a diff rather than a match. + if ! echo "$rust_row" | jq -e . >/dev/null 2>&1; then + rust_norm="" + else + rust_norm=$(normalize "$rust_row") + fi + if ! echo "$go_row" | jq -e . >/dev/null 2>&1; then + go_norm="" + else + go_norm=$(normalize "$go_row") + fi + if [ "$rust_norm" = "$go_norm" ]; then + MATCH=$((MATCH+1)) + else + DIFF=$((DIFF+1)) + DIFF_DETAIL="$DIFF_DETAIL"$'\n\n'"### $label"$'\n\n''**Rust:**'$'\n''```json'$'\n'"$rust_norm"$'\n''```'$'\n\n''**Go:**'$'\n''```json'$'\n'"$go_norm"$'\n''```' + fi +done + +# ── Report ───────────────────────────────────────────────────────── +{ + echo "# session_log parity probe — Rust gateway vs Go validatord" + echo + echo "**Date:** $(date -u +%Y-%m-%dT%H:%M:%SZ)" + echo "**Rust helper:** \`$RUST_BIN\`" + echo "**Go helper:** \`$GO_BIN\`" + echo + echo "Identical fixture inputs through each runtime's" + echo "\`SessionRecord\` builder + JSON marshaler. Match = byte-equal" + echo "after stripping \`timestamp\` (per-run wall clock) + \`daemon\`" + echo "(\"gateway\" on Rust side, \"validatord\" on Go side; both are" + echo "valid producers in the same longitudinal log)." + echo + echo "**Tally:** $MATCH match · $DIFF diff (out of $TOTAL fixtures)" + if [ -n "$DIFF_DETAIL" ]; then + echo + echo "## Divergences" + echo "$DIFF_DETAIL" + else + echo + echo "_No divergences — schema parity holds across all fixtures._" + fi +} > "$OUT" + +echo "[parity] session_log: $MATCH match / $DIFF diff (out of $TOTAL) → $OUT" +[ "$DIFF" -eq 0 ]