golangLAKEHOUSE/justfile
root a81291e38c proof harness Phase A: scaffolding + canary case green
Per docs/TEST_PROOF_SCOPE.md, building the claims-verification tier
above the smoke chain. This commit lays the scaffolding and proves
the orchestrator end-to-end with one canary case (00_health).

What landed:

  tests/proof/
    README.md             how to read a report, layout, modes
    claims.yaml           24 claims enumerated (GOLAKE-001..100)
    run_proof.sh          orchestrator with --mode {contract|integration|performance}
                          and --no-bootstrap / --regenerate-{rankings,baseline}
    lib/
      env.sh              service URLs, report dir, mode, git context
      http.sh             curl wrappers writing per-probe JSON + body + headers
      assert.sh           proof_assert_{eq,ne,contains,lt,gt,status,json_eq} +
                          proof_skip — each emits one JSONL record per call
      metrics.sh          start/stop timers, value capture, RSS sampling,
                          percentile compute (for Phase D)
    cases/
      00_health.sh        canary — gateway + 6 services /health → 200,
                          body identifies service, latency < 500ms (21 assertions)
    fixtures/
      csv/workers.csv     spec's 5-row deterministic CSV
      text/docs.txt       4 deterministic vector docs
      expected/queries.json  expected results for the 5 SQL assertions

Wired into the task runner:

  just proof contract       # canary only this commit
  just proof integration    # Phase C
  just proof performance    # Phase D

.gitignore: /tests/proof/reports/* with !.gitkeep — same pattern as
reports/scrum/_evidence/. Per-run output is a runtime artifact.

Specs landed alongside (J's drops):
  docs/TEST_PROOF_SCOPE.md           the harness contract this implements
  docs/CLAUDE_REFACTOR_GUARDRAILS.md process discipline this harness obeys

Verified end-to-end (cached binaries):
  just proof contract        wall < 2s, 21 pass / 0 fail / 0 skip
  just verify                wall 31s, vet + test + 9 smokes still green

Two bugs fixed during canary run, both in run_proof.sh aggregation:
- grep -c exits 1 on zero matches; the `|| echo 0` form concatenated
  "0\n0" and broke jq --argjson + integer comparison. Fixed via a
  _count helper that captures count-or-zero cleanly.
- per-case table iterated case scripts (filename-based) but cases
  write evidence under CASE_ID. Switched to JSONL-file iteration so
  multi-case scripts work and the mapping is faithful.

Phase B (contract cases) lands next: 05_embedding, 06_vector_add,
08_gateway_contracts, 09_failure_modes. Each sourcing the same lib
helpers and writing to the same report shape.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 05:08:51 -05:00

112 lines
3.8 KiB
Makefile

# golangLAKEHOUSE — task runner.
#
# Sprint 0 acceptance gate (R-004): smokes are no longer documentation
# only — `just verify` is the single command that runs vet + tests +
# the 9 smokes. The pre-push hook calls this; CI calls this; reviewers
# call this. One source of truth.
#
# Usage:
# just # alias for `just --list`
# just verify # vet + test + all 9 smokes (full gate)
# just smoke <day> # single smoke (d1..d6, g1, g1p, g2)
# just smoke-all # all 9 smokes only
# just doctor # dependency probe
# just fmt / vet / test / build
# Go lives at /usr/local/go/bin per ADR-001 §1.x; prepend so every
# recipe sees it without depending on the parent shell's PATH.
export PATH := "/usr/local/go/bin:" + env('PATH', '')
# Default recipe shows the menu so `just` alone is a discoverable entry point.
default:
@just --list
# Full Sprint 0 gate: vet + tests + 9 smokes. Pre-push hook calls this.
verify: vet test smoke-all
@echo ""
@echo "[verify] PASS — go vet + go test + 9 smokes all green"
# Static analysis. Runs first so we fail fast on syntax / shape issues.
vet:
@echo "[vet] go vet ./..."
@go vet ./...
# Go unit tests, short mode. Excludes hardware-in-the-loop tags.
test:
@echo "[test] go test -short -count=1 ./..."
@go test -short -count=1 ./...
# Format Go source. Idempotent; CI can run with --check via `just fmt-check`.
fmt:
@gofmt -w cmd internal scripts
# Verify formatting without modifying. Non-zero exit means run `just fmt`.
fmt-check:
@diff -u <(echo -n) <(gofmt -d cmd internal scripts)
# Build every binary into bin/. Mirrors what each smoke does internally.
build:
@echo "[build] go build -o bin/ ./cmd/..."
@go build -o bin/ ./cmd/...
# Single smoke. Day is the suffix before _smoke.sh — d1, d2, …, g2.
smoke day:
@bash scripts/{{day}}_smoke.sh
# All 9 smokes in dependency order. Halts on first failure.
smoke-all:
#!/usr/bin/env bash
set -euo pipefail
for day in d1 d2 d3 d4 d5 d6 g1 g1p g2; do
printf "[smoke-all] %s ... " "$day"
SECONDS=0
if bash "scripts/${day}_smoke.sh" >/tmp/smoke_${day}.log 2>&1; then
printf "PASS (%ss)\n" "$SECONDS"
else
printf "FAIL (%ss)\n" "$SECONDS"
echo ""
echo " last 20 lines of /tmp/smoke_${day}.log:"
tail -20 "/tmp/smoke_${day}.log" | sed 's/^/ /'
exit 1
fi
done
# Dependency probe. Add --json for machine-readable output.
doctor *args:
@bash scripts/doctor.sh {{args}}
# Proof harness — claims-verification tier above the smoke chain.
# See tests/proof/README.md and docs/TEST_PROOF_SCOPE.md.
# just proof contract fast: APIs + status codes + dim/nonempty
# just proof integration full: CSV→Parquet→SQL, text→vector→search
# just proof performance measurements; runs only after contract+integration
proof mode *flags:
@bash tests/proof/run_proof.sh --mode {{mode}} {{flags}}
# Install pre-push hook so `git push` runs `just verify` first.
install-hooks:
#!/usr/bin/env bash
set -euo pipefail
HOOK=".git/hooks/pre-push"
cat > "$HOOK" <<'HOOK'
#!/usr/bin/env bash
# golangLAKEHOUSE pre-push hook (managed by `just install-hooks`).
# Runs the Sprint 0 gate before letting commits leave this machine.
set -e
cd "$(git rev-parse --show-toplevel)"
echo "[pre-push] running just verify ..."
if ! just verify; then
echo ""
echo "[pre-push] FAIL — push aborted. Fix the gate or use --no-verify (NOT recommended)."
exit 1
fi
HOOK
chmod +x "$HOOK"
echo "[install-hooks] $HOOK installed and executable"
# Clean built binaries + smoke logs. Does NOT touch reports/ or data/.
clean:
@rm -rf bin/
@rm -f /tmp/smoke_*.log
@echo "[clean] bin/ removed, smoke logs cleared"