lakehouse/auditor 1 blocking issue: cloud: claim not backed — "the proven escalation ladder with learning context, collects"
Observed on PR #8 audit (de11ac4): 7 warn findings, all from the
cloud inference check. Investigation showed two distinct bug classes
that weren't "ship bad code", they were "auditor misreads the diff":
1. Cloud flagged "X not defined in this diff / missing implementation"
for symbols like `tailJsonl` and `stubFinding` that ARE defined —
just not in the added lines of this diff. Fix: extract candidate
symbols from the cloud's gap summary, grep the repo for their
definitions (function/const/let/def/class/struct/enum/trait/fn).
If every named symbol resolves, drop the finding; if some do,
demote to info with the resolution in evidence.
2. Cloud flagged runtime metrics like "58 cloud calls, 306s
end-to-end" as unbacked claims. These are empirical outputs
from running the test, not things a static diff can prove.
Fix: claim_parser now has an `empirical` strength class
matching iteration counts, cloud-call counts, duration metrics,
attempt counts, tier-count phrases. Inference drops empirical
claims from its cloud prompt (verifiable[] subset only) and
claim-index mapping uses verifiable[] so cloud responses still
line up.
Added `claims_empirical` to audit metrics so the verdict is
introspectable: how many claims WERE runtime-only vs how many
are diff-verifiable?
Verified: unit tests confirm empirical classification on 5
sample commit messages; symbol resolver found both false-positive
symbols (tailJsonl + stubFinding) and correctly skipped a known-
fake symbol.
auditor/claim_parser.ts — reads PR body + commit messages, extracts
ship-claims. Regex-based, intentionally not LLM-driven: the parser's
job is to surface claim substrates, not to judge them (that's the
inference check's job, runs later with cloud model).
Three strength tiers:
- strong — "verified end-to-end", "live-proven", "production-ready",
"phase N shipped", "proven"
- moderate — "shipped", "landed", "green", "passing", "works",
"complete", "done"
- weak — "should work", "expected to", "probably"
Live-proven against PR #1 (this PR): 4 claims extracted from
1 commit (2 strong, 2 moderate). "live-proven" correctly tagged as
strong (it IS a stronger claim than "shipped").
Next: static diff check consumes these claims + the PR diff to find
placeholder patterns — empty fns, TODO, unwired fields, etc.
