diff --git a/STATE_OF_PLAY.md b/STATE_OF_PLAY.md index 97ad3f1..e9a865d 100644 --- a/STATE_OF_PLAY.md +++ b/STATE_OF_PLAY.md @@ -1,12 +1,52 @@ # STATE OF PLAY — Lakehouse -**Last verified:** 2026-05-05 morning CDT -**Verified by:** live probe (`/audit/health` 200, `/biometric/subject/{id}/erase` 21-test substrate + `/audit/subject/{id}` legal-tier endpoint live verified against WORKER-100; new `verify_biometric_erasure.sh` + `biometric_destruction_report.sh` + `bundle_counsel_packet.sh` smoke-tested clean against live data) — not memory. +**Last verified:** 2026-06-30 (search index-alignment fix live on :3700) · biometric substrate 2026-05-05 +**Verified by (2026-06-30):** live probe after mcp-server restart — `/match` + `/search` with `state='IL'` return only IL workers (new PID on :3700). Not memory. +**Verified by (2026-05-05):** live probe (`/audit/health` 200, `/biometric/subject/{id}/erase` 21-test substrate + `/audit/subject/{id}` legal-tier endpoint live verified against WORKER-100; new `verify_biometric_erasure.sh` + `biometric_destruction_report.sh` + `bundle_counsel_packet.sh` smoke-tested clean against live data) — not memory. > **Read this FIRST.** When the user says "we're working on lakehouse," they mean the working code captured below — NOT what `git log` framed as "the cutover" or what memory snapshots from 2 days ago suggest. If memory contradicts this file, this file wins. Update it when something is verified working — not when a phase finishes. --- +## WHAT LANDED 2026-06-30 (search index-alignment sweep + P1-already-done finding) + +**Fixed a systemic hybrid-search bug.** Every `/vectors/hybrid` call that paired the +`ethereal_workers` SQL dataset (10K rows) with the `workers_500k_v1` vector index (50K, +a *different* worker set) returned wrong-entity results: same `worker_id` = different +person in each index, so a `state='IL'` query surfaced a **KY** worker flagged +`sql_verified:true`. Root cause: the default index didn't share an id-space with the +default dataset. + +Swept **8 call sites** in `mcp-server/index.ts` to `ethereal_workers_v1` (10K, same +id-space as the SQL table). Aligned sites (`filter_dataset: workers_500k`) and pure-RAG +sites (`/vectors/rag`) were correctly left alone. + +| Commit | Sites | +|---|---| +| `423400e` | `/search` HTTP handler | +| `35a80a7` | `search_workers` tool · `match_contract` tool · `/match` HTTP handler | +| `1570306` | 4 intelligence routes (similar-workers, role-discovery, smart_search, scenario-find) | + +**Verified live** after an mcp-server restart (new PID on :3700): `/match` and `/search` +with `state='IL'` return only IL workers (Champaign/Springfield/Urbana); `state='WI'` +returns WI. Grep confirms no `workers_500k_v1` index remains paired with `ethereal_workers`. +Server boots clean. Only `mcp-server/index.ts` was touched. + +**Finding — demo-plan P1 was already done.** The "search box that actually filters" item +in this doc's CURRENT PLAN (P1, from the 2026-04-27 demo audit) is **stale**. The UI now +posts structured `state`/`role` to the evolved `/intelligence/chat`, which returns real +`sql_results` (IL Forklift → 167 matches, correct workers; WI → WI). The P1 section below +describes a bug that no longer exists — don't re-implement it. + +**⚠ Uncommitted WIP left untouched (pre-existing, not from this session).** +`mcp-server/search.html` + `proof.html` (edited 2026-05-30; proof.html restructured into +chapters) and a `biometric_endpoint.rs` comment cleanup remain uncommitted in the working +tree. Also untracked and newer than this doc: `RECOVERY_PLAN_2026-05-05.md`, +`mcp-server/UX_FIX_PLAN.md`, `docs/scope-sentinel/`. Someone should reconcile these — they +imply planning context this file doesn't yet capture. + +--- + ## WHAT LANDED 2026-05-05 (afternoon wave — full BIPA lifecycle endpoints + UIs + ops dashboard) After the morning's doc-reconciliation wave (described below), the afternoon wave shipped the full **operational lifecycle** for biometric data. Phase 1.6 now has the complete end-to-end runtime: candidate consent flow → photo upload → withdrawal → retention sweep flag → erase. Five commits, all on `demo/post-pr11-polish-2026-04-28`.