ac01fffd9a
Architectural snapshot of the lakehouse codebase at the point where the
full matrix-driven agent loop with Mem0 versioning + deletion was
validated end-to-end.
WHAT THIS REPO IS
A clean single-commit snapshot of the lakehouse code. Heavy test data
(.parquet datasets, vector indexes) excluded — see REPLICATION.md for
regen path. Full lakehouse history at git.agentview.dev/profit/lakehouse.
WHAT WAS PROVEN
- Vector retrieval across multi-corpora matrix (chicago_permits + entity
briefs + sec_tickers + distilled procedural + llm_team runs)
- Observer hand-review (cloud + heuristic fallback) gating each candidate
- Local-model agent loop (qwen3.5:latest) with tool use + scratchpad
- Playbook seal on success → next-iter retrieval surfaces it as preamble
- Mem0 versioning + deletion in pathway_memory:
* UPSERT: ADD on new workflow, UPDATE bumps replay_count on identical
* REVISE: chains versions, parent.superseded_at + superseded_by stamped
* RETIRE: marks specific trace retired with reason, excluded from retrieval
* HISTORY: walks chain root→tip, cycle-safe
KEY DIRECTORIES
- crates/vectord/src/pathway_memory.rs — Mem0 ops live here
- crates/vectord/src/playbook_memory.rs — original Mem0 reference
- tests/agent_test/ — local-model agent harness + PRD + session archives
- scripts/dump_raw_corpus.sh — MinIO bucket dump (raw test corpus)
- scripts/vectorize_raw_corpus.ts — corpus → vector indexes
- scripts/analyze_chicago_contracts.ts — real inference pipeline
- scripts/seal_agent_playbook.ts — Mem0 upsert from agent traces
Replication: see REPLICATION.md for Debian 13 clean install + cloud-only
adaptation (no local Ollama).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
61 lines
2.3 KiB
Markdown
61 lines
2.3 KiB
Markdown
# Lakehouse Claim Auditor
|
|
|
|
A Bun sub-agent that watches open PRs on Gitea, reads the ship-claims
|
|
in commit messages and PR bodies, and **hard-blocks** merges when the
|
|
code doesn't back the claim.
|
|
|
|
Rationale: when "compiles + one curl works" gets called "phase shipped,"
|
|
placeholder code accumulates. This auditor runs every 90s, fetches
|
|
each open PR, and subjects it to four checks:
|
|
|
|
1. **Static diff** — grep/parse looking for placeholder patterns
|
|
2. **Dynamic** — runs the never-before-executed hybrid test fixture
|
|
3. **Cloud inference** — asks `gpt-oss:120b` via `/v1/chat` to
|
|
identify gaps in the diff
|
|
4. **KB query** — looks up `data/_kb/` + observer for prior failure
|
|
patterns on similar claims
|
|
|
|
Verdict is assembled, posted to Gitea as:
|
|
- A **failing commit status** (hard block — branch protection
|
|
prevents merge)
|
|
- A **review comment** explaining every finding
|
|
|
|
## Run manually
|
|
|
|
```bash
|
|
cd /home/profit/lakehouse
|
|
bun run auditor/index.ts
|
|
```
|
|
|
|
Defaults: polls every 90s, stops on `auditor.paused` file present.
|
|
|
|
## State
|
|
|
|
- `data/_auditor/state.json` — last-audited head SHA per PR
|
|
- `data/_auditor/verdicts/{pr}-{sha}.json` — per-run verdict record
|
|
- `data/_kb/audit_lessons.jsonl` — one row per block/warn finding,
|
|
path-agnostic signature for dedup. Tailed by kb_query on each audit
|
|
to surface recurring patterns (2+ distinct PRs with same signature
|
|
→ info, 3-4 → warn, 5+ → block). This is how the auditor learns.
|
|
- `data/_kb/scrum_reviews.jsonl` — scrum-master per-file reviews. If
|
|
a file in the current PR has been scrum-reviewed, kb_query surfaces
|
|
the review as a finding with the accepted model and attempt count.
|
|
|
|
## Where YOU edit
|
|
|
|
`auditor/policy.ts` — the verdict assembler. Controls which findings
|
|
block vs warn vs inform. All other code is mechanical: fetching,
|
|
running checks, posting to Gitea.
|
|
|
|
## Hard-block mechanism
|
|
|
|
1. Commit status is posted as `failure` with context `lakehouse/auditor`
|
|
2. If `main` branch protection requires `lakehouse/auditor` status
|
|
to pass, Gitea prevents merge
|
|
3. When code is fixed and re-audit passes, status flips to `success`,
|
|
merge unblocks
|
|
|
|
Enable branch protection (one-time, via Gitea UI or API):
|
|
- `POST /repos/profit/lakehouse/branch_protections`
|
|
- `{"branch_name": "main", "required_status_checks": {"contexts": ["lakehouse/auditor"]}}`
|