89ca72d471
Two threads landing together — the doc edits interleave so they ship in a single commit. 1. **vectord substrate fix verified at original scale** (closes the 2026-05-01 thread). Re-ran multitier 5min @ conc=50: 132,211 scenarios at 438/sec, 6/6 classes at 0% failure (was 4/6 pre-fix). Throughput dropped 1,115 → 438/sec because previously-broken scenarios now do real HNSW Add work — honest cost of correctness. The fix (i.vectors side-store + safeGraphAdd recover wrappers + smallIndexRebuildThreshold=32 + saveTask coalescing) holds at the footprint that originally surfaced the bug. 2. **Materializer port** — internal/materializer + cmd/materializer + scripts/materializer_smoke.sh. Ports scripts/distillation/transforms.ts (12 transforms) + build_evidence_index.ts (idempotency, day-partition, receipt). On-wire JSON shape matches TS so Bun and Go runs are interchangeable. 14 tests green. 3. **Replay port** — internal/replay + cmd/replay + scripts/replay_smoke.sh. Ports scripts/distillation/replay.ts (retrieve → bundle → /v1/chat → validate → log). Closes audit-FULL phase 7 live invocation on the Go side. Both runtimes append to the same data/_kb/replay_runs.jsonl (schema=replay_run.v1). 14 tests green. Side effect on internal/distillation/types.go: EvidenceRecord gained prompt_tokens, completion_tokens, and metadata fields to mirror the TS shape the materializer transforms produce. STATE_OF_PLAY refreshed to 2026-05-02; ARCHITECTURE_COMPARISON decisions tracker moves the materializer + replay items from _open_ to DONE and adds the substrate-fix scale verification row. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
74 lines
3.1 KiB
Bash
Executable File
74 lines
3.1 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# materializer smoke — Go port of scripts/distillation/build_evidence_index.ts.
|
|
# Validates that the materializer:
|
|
# - Builds a minimal evidence partition from a synthetic source jsonl
|
|
# - Skips bad-JSON rows into distillation_skips.jsonl
|
|
# - Idempotently dedups identical rows on re-run (rows_deduped > 0)
|
|
# - Honors --dry-run (no files written, exit 0)
|
|
# - Emits a parseable receipt.json with validation_pass
|
|
|
|
set -euo pipefail
|
|
cd "$(dirname "$0")/.."
|
|
|
|
export PATH="$PATH:/usr/local/go/bin"
|
|
|
|
echo "[materializer-smoke] building bin/materializer..."
|
|
go build -o bin/materializer ./cmd/materializer
|
|
|
|
ROOT="$(mktemp -d)"
|
|
trap 'rm -rf "$ROOT"' EXIT INT TERM
|
|
|
|
mkdir -p "$ROOT/data/_kb"
|
|
cat > "$ROOT/data/_kb/distilled_facts.jsonl" <<EOF
|
|
{"run_id":"r1","source_label":"lab-a","created_at":"2026-04-26T00:00:00Z","extractor":"qwen3.5:latest","text":"first"}
|
|
{"run_id":"r2","source_label":"lab-b","created_at":"2026-04-26T01:00:00Z","extractor":"qwen3.5:latest","text":"second"}
|
|
not-json
|
|
EOF
|
|
|
|
cat > "$ROOT/data/_kb/observer_escalations.jsonl" <<EOF
|
|
{"ts":"2026-04-26T12:00:00Z","sig_hash":"abc","cluster_endpoint":"/v1/chat","prompt_tokens":42,"completion_tokens":11,"analysis":"esc"}
|
|
EOF
|
|
|
|
echo "[materializer-smoke] dry-run probe"
|
|
# Materializer exits 1 when validation_pass=false (bad-json row); set -e
|
|
# would kill the script on that. Run with || true and inspect stdout.
|
|
DRY_OUT="$(./bin/materializer -root "$ROOT" -dry-run 2>&1 || true)"
|
|
echo "$DRY_OUT" | grep -q "DRY RUN" || { echo "expected DRY RUN marker: $DRY_OUT"; exit 1; }
|
|
[ ! -d "$ROOT/data/evidence" ] || { echo "dry-run wrote evidence dir"; exit 1; }
|
|
|
|
echo "[materializer-smoke] first run"
|
|
# Same exit-1 path as dry-run when bad-json present; expect that.
|
|
./bin/materializer -root "$ROOT" || true
|
|
|
|
OUT_FACTS="$ROOT/data/evidence/$(date -u +'%Y/%m/%d')/distilled_facts.jsonl"
|
|
OUT_OBS="$ROOT/data/evidence/$(date -u +'%Y/%m/%d')/observer_escalations.jsonl"
|
|
SKIPS="$ROOT/data/_kb/distillation_skips.jsonl"
|
|
|
|
[ -s "$OUT_FACTS" ] || { echo "expected $OUT_FACTS"; exit 1; }
|
|
[ -s "$OUT_OBS" ] || { echo "expected $OUT_OBS"; exit 1; }
|
|
[ -s "$SKIPS" ] || { echo "expected $SKIPS to capture bad-json row"; exit 1; }
|
|
|
|
GOOD_ROWS=$(wc -l < "$OUT_FACTS")
|
|
[ "$GOOD_ROWS" -eq 2 ] || { echo "expected 2 good rows in $OUT_FACTS, got $GOOD_ROWS"; exit 1; }
|
|
|
|
# Receipt — find the most recent one and parse validation_pass.
|
|
RECEIPT="$(find "$ROOT/reports/distillation" -name 'receipt.json' -print0 | xargs -0 ls -t | head -1)"
|
|
[ -n "$RECEIPT" ] || { echo "no receipt produced"; exit 1; }
|
|
grep -q '"validation_pass": false' "$RECEIPT" || {
|
|
echo "expected validation_pass=false (1 row was bad JSON):";
|
|
cat "$RECEIPT";
|
|
exit 1;
|
|
}
|
|
|
|
echo "[materializer-smoke] idempotent re-run"
|
|
./bin/materializer -root "$ROOT" >/tmp/materializer_smoke_rerun.txt 2>&1 || true
|
|
# Rerun should fail validation again (the bad-JSON row is still there)
|
|
# but successful rows should have hit dedup not write.
|
|
grep -q "dedup=2" /tmp/materializer_smoke_rerun.txt || {
|
|
echo "expected dedup=2 on rerun, got:";
|
|
cat /tmp/materializer_smoke_rerun.txt;
|
|
exit 1;
|
|
}
|
|
|
|
echo "[materializer-smoke] PASS"
|