a81291e38c
Per docs/TEST_PROOF_SCOPE.md, building the claims-verification tier
above the smoke chain. This commit lays the scaffolding and proves
the orchestrator end-to-end with one canary case (00_health).
What landed:
tests/proof/
README.md how to read a report, layout, modes
claims.yaml 24 claims enumerated (GOLAKE-001..100)
run_proof.sh orchestrator with --mode {contract|integration|performance}
and --no-bootstrap / --regenerate-{rankings,baseline}
lib/
env.sh service URLs, report dir, mode, git context
http.sh curl wrappers writing per-probe JSON + body + headers
assert.sh proof_assert_{eq,ne,contains,lt,gt,status,json_eq} +
proof_skip — each emits one JSONL record per call
metrics.sh start/stop timers, value capture, RSS sampling,
percentile compute (for Phase D)
cases/
00_health.sh canary — gateway + 6 services /health → 200,
body identifies service, latency < 500ms (21 assertions)
fixtures/
csv/workers.csv spec's 5-row deterministic CSV
text/docs.txt 4 deterministic vector docs
expected/queries.json expected results for the 5 SQL assertions
Wired into the task runner:
just proof contract # canary only this commit
just proof integration # Phase C
just proof performance # Phase D
.gitignore: /tests/proof/reports/* with !.gitkeep — same pattern as
reports/scrum/_evidence/. Per-run output is a runtime artifact.
Specs landed alongside (J's drops):
docs/TEST_PROOF_SCOPE.md the harness contract this implements
docs/CLAUDE_REFACTOR_GUARDRAILS.md process discipline this harness obeys
Verified end-to-end (cached binaries):
just proof contract wall < 2s, 21 pass / 0 fail / 0 skip
just verify wall 31s, vet + test + 9 smokes still green
Two bugs fixed during canary run, both in run_proof.sh aggregation:
- grep -c exits 1 on zero matches; the `|| echo 0` form concatenated
"0\n0" and broke jq --argjson + integer comparison. Fixed via a
_count helper that captures count-or-zero cleanly.
- per-case table iterated case scripts (filename-based) but cases
write evidence under CASE_ID. Switched to JSONL-file iteration so
multi-case scripts work and the mapping is faithful.
Phase B (contract cases) lands next: 05_embedding, 06_vector_add,
08_gateway_contracts, 09_failure_modes. Each sourcing the same lib
helpers and writing to the same report shape.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
56 lines
1.2 KiB
Plaintext
56 lines
1.2 KiB
Plaintext
# Go
|
|
*.exe
|
|
*.exe~
|
|
*.dll
|
|
*.so
|
|
*.dylib
|
|
*.test
|
|
*.out
|
|
go.work
|
|
go.work.sum
|
|
vendor/
|
|
|
|
# Build artifacts
|
|
/bin/
|
|
/dist/
|
|
|
|
# Editor / OS
|
|
.DS_Store
|
|
.idea/
|
|
.vscode/
|
|
*.swp
|
|
*~
|
|
|
|
# Local data — these directories follow the Rust lakehouse pattern;
|
|
# regenerated by services on demand. Do not commit runtime artifacts.
|
|
/data/_auditor/
|
|
/data/_kb/
|
|
/data/_pathway_memory/
|
|
/data/_errors/
|
|
/data/_imagecache/
|
|
/data/datasets/
|
|
/data/vectors/
|
|
/data/headshots/
|
|
/data/lance/
|
|
/exports/
|
|
/logs/
|
|
# /reports/ holds runtime artifacts by default (matches Rust lakehouse
|
|
# convention) — but reports/scrum/ is intentional audit documentation.
|
|
# Use /reports/* + un-ignore so git can traverse into reports/.
|
|
/reports/*
|
|
!/reports/scrum/
|
|
# Inside the audit directory, the per-run _evidence/ dump (smoke logs,
|
|
# command output) IS runtime — track the dir, ignore its contents.
|
|
/reports/scrum/_evidence/*
|
|
!/reports/scrum/_evidence/.gitkeep
|
|
|
|
# Proof harness runtime output — same pattern as reports/scrum/_evidence.
|
|
# Track the directory but ignore per-run subdirs.
|
|
/tests/proof/reports/*
|
|
!/tests/proof/reports/.gitkeep
|
|
|
|
# Secrets — never commit. Resolved via SecretsProvider per ADR-001 §1.x.
|
|
*.env
|
|
secrets.toml
|
|
secrets-go.toml
|