profit/lakehouse
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Fixture: unique-per-run nonce eliminates state-pollution false positive

Browse Source 
After the serde fix (PR #2, fix/upsert-outcome-serde) landed on main,
re-running this fixture STILL reported "doc_refs field is empty" —
but with a different root cause than the panic.

Root cause: pre-fix runs panicked on response serialization but had
already added entries to state (panic happened between upsert_entry
returning and the handler's serde_json::json! of the response). So
state.json was polluted with __auditor_test_worker__ entries from
those runs, WITHOUT doc_refs (doc_refs wasn't even wired at the time
those state rows were written).

The fixture's `find(endorsed_names.includes(TEST_WORKER_NAME))` was
picking the oldest polluted entry, not the fresh one.

Compounding: discovered a secondary bug while investigating —
upsert_entry's UPDATE branch only merges endorsed_names. doc_refs,
schema_fingerprint, valid_until on an UPDATE are silently dropped.
Filed as task #12, separate PR to follow.

Fix in this fixture: use a nonce suffix on both TEST_WORKER_NAME and
TEST_OPERATION so every run is guaranteed to hit the ADD path in
upsert_entry, sidestepping the UPDATE bug AND eliminating state
pollution entirely.

Live re-run after this edit:
  ✓ Phase 38    /v1/chat            449ms, 42 tokens
  ✓ Phase 40    Langfuse trace       20ms
  ✓ Phase 45.1  seed + doc_refs     239ms, doc_refs.length=1 persisted
  ✓ Phase 45.2  bridge diff           2ms, drifted=true
  ✗ Phase 45.3  drift-check           HONEST 404 (endpoint not built)

shipped_phases: [38, 40, 45.1, 45.2]  (was [38, 40, 45.2])
placeholder:    [45.3]                 (was [45.1, 45.3])

One fewer placeholder — exactly because the serde fix merged on
fix/upsert-outcome-serde and the fixture now cleanly exercises the
path. The loop is:
  fixture finds bug → PR fixes bug → fixture re-run confirms fix →
  one fewer placeholder.
This commit is contained in:
profit 2026-04-22 03:50:46 -05:00
parent 5bbcaf4c33
commit c5da680add
1 changed files with 11 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

15
auditor/fixtures/hybrid_38_40_45.ts
View File

@ -28,6 +28,13 @@ const BRIDGE = process.env.CONTEXT7_BRIDGE_URL ?? "http://localhost:3900";
const FIXTURE_ID = "auditor:hybrid_38_40_45:v1";
const TEST_WORKER_NAME = "__auditor_test_worker__";
const STALE_HASH = "stale-hash-for-drift-proof";
// Unique-per-run identifiers so each fixture invocation hits the ADD
// path in upsert_entry (not UPDATE/NOOP on a leftover from a prior
// run). Catches state pollution + avoids interaction with task #12
// (UPDATE branch silently drops doc_refs).
const RUN_NONCE = Date.now().toString(36);
const TEST_WORKER_NAME_VERSIONED = `__auditor_test_worker_${RUN_NONCE}__`;
const TEST_OPERATION_VERSIONED = `TEST: fill: __auditor_${RUN_NONCE}__ x1 in TestCity, TC`;
export interface LayerResult {
layer: string;
@ -190,10 +197,10 @@ export async function runHybridFixture(): Promise<FixtureResult> {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({
operation: `TEST: fill: __auditor__ x1 in TestCity, TC`,
operation: TEST_OPERATION_VERSIONED,
approach: "auditor hybrid fixture run",
context: "doc_refs integration test — retire after audit",
endorsed_names: [TEST_WORKER_NAME],
endorsed_names: [TEST_WORKER_NAME_VERSIONED],
append: true,
doc_refs: [
{
@ -215,8 +222,8 @@ export async function runHybridFixture(): Promise<FixtureResult> {
const state_raw = await readFile("/home/profit/lakehouse/data/_playbook_memory/state.json", "utf8");
const state = JSON.parse(state_raw);
const entries = state.entries ?? [];
const ours = entries.find((e: any) => (e.endorsed_names ?? []).includes(TEST_WORKER_NAME));
if (!ours) throw new Error(`seeded entry not found in state.json (worker ${TEST_WORKER_NAME} not present)`);
const ours = entries.find((e: any) => (e.endorsed_names ?? []).includes(TEST_WORKER_NAME_VERSIONED));
if (!ours) throw new Error(`seeded entry not found in state.json (worker ${TEST_WORKER_NAME_VERSIONED} not present)`);
const docRefs = ours.doc_refs ?? [];
if (docRefs.length === 0) {
throw new Error("entry saved but doc_refs field is empty — the field accepted by the API isn't being persisted");