profit 156dae6732 Auditor self-test branch: real-world pipelines + cohesion Phase C + KB index (PR #8 )

Bundles 12 commits validating the auditor + scrum_master architecture end-to-end:

- enrich_prd_pipeline / hard_task_escalation / scrum_master_pipeline stress tests
- Tree-split + scrum_reviews.jsonl + kb_query surfacing
- Verdict → audit_lessons feedback loop (closed)
- kb_index aggregator with confidence-based severity policy
- 9-run + 5-run empirical tests proved the predictive-compounding property
- Level 1 correction: temp=0 cloud inference for deterministic per-claim verdicts
- audit_one.ts dry-run CLI
- Fixes: static quoted-string guard, empirical-claim classification, symbol-resolver gate, repo-file size cap

See PR #8 for run-by-run commit history.

2026-04-23 03:28:32 +00:00

2.3 KiB

Raw Blame History

Lakehouse Claim Auditor

A Bun sub-agent that watches open PRs on Gitea, reads the ship-claims in commit messages and PR bodies, and hard-blocks merges when the code doesn't back the claim.

Rationale: when "compiles + one curl works" gets called "phase shipped," placeholder code accumulates. This auditor runs every 90s, fetches each open PR, and subjects it to four checks:

Static diff — grep/parse looking for placeholder patterns
Dynamic — runs the never-before-executed hybrid test fixture
Cloud inference — asks gpt-oss:120b via /v1/chat to identify gaps in the diff
KB query — looks up data/_kb/ + observer for prior failure patterns on similar claims

Verdict is assembled, posted to Gitea as:

A failing commit status (hard block — branch protection prevents merge)
A review comment explaining every finding

Run manually

cd /home/profit/lakehouse
bun run auditor/index.ts

Defaults: polls every 90s, stops on auditor.paused file present.

State

data/_auditor/state.json — last-audited head SHA per PR
data/_auditor/verdicts/{pr}-{sha}.json — per-run verdict record
data/_kb/audit_lessons.jsonl — one row per block/warn finding, path-agnostic signature for dedup. Tailed by kb_query on each audit to surface recurring patterns (2+ distinct PRs with same signature → info, 3-4 → warn, 5+ → block). This is how the auditor learns.
data/_kb/scrum_reviews.jsonl — scrum-master per-file reviews. If a file in the current PR has been scrum-reviewed, kb_query surfaces the review as a finding with the accepted model and attempt count.

Where YOU edit

auditor/policy.ts — the verdict assembler. Controls which findings block vs warn vs inform. All other code is mechanical: fetching, running checks, posting to Gitea.

Hard-block mechanism

Commit status is posted as failure with context lakehouse/auditor
If main branch protection requires lakehouse/auditor status to pass, Gitea prevents merge
When code is fixed and re-audit passes, status flips to success, merge unblocks

Enable branch protection (one-time, via Gitea UI or API):

POST /repos/profit/lakehouse/branch_protections
{"branch_name": "main", "required_status_checks": {"contexts": ["lakehouse/auditor"]}}

2.3 KiB Raw Blame History