docs: STATE_OF_PLAY — record search index-alignment fix + P1-already-done
- New 2026-06-30 wave: the ethereal_workers↔workers_500k_v1 misalignment fix (8 sites, commits 423400e/35a80a7/1570306), verified live after mcp-server restart - Flag that demo-plan P1 (search-box filter) is already done — the P1 section is stale - Note the untouched 2026-05-30 uncommitted WIP + newer untracked planning docs Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
15703063ea
commit
7dddb98f4b
@ -1,12 +1,52 @@
|
|||||||
# STATE OF PLAY — Lakehouse
|
# STATE OF PLAY — Lakehouse
|
||||||
|
|
||||||
**Last verified:** 2026-05-05 morning CDT
|
**Last verified:** 2026-06-30 (search index-alignment fix live on :3700) · biometric substrate 2026-05-05
|
||||||
**Verified by:** live probe (`/audit/health` 200, `/biometric/subject/{id}/erase` 21-test substrate + `/audit/subject/{id}` legal-tier endpoint live verified against WORKER-100; new `verify_biometric_erasure.sh` + `biometric_destruction_report.sh` + `bundle_counsel_packet.sh` smoke-tested clean against live data) — not memory.
|
**Verified by (2026-06-30):** live probe after mcp-server restart — `/match` + `/search` with `state='IL'` return only IL workers (new PID on :3700). Not memory.
|
||||||
|
**Verified by (2026-05-05):** live probe (`/audit/health` 200, `/biometric/subject/{id}/erase` 21-test substrate + `/audit/subject/{id}` legal-tier endpoint live verified against WORKER-100; new `verify_biometric_erasure.sh` + `biometric_destruction_report.sh` + `bundle_counsel_packet.sh` smoke-tested clean against live data) — not memory.
|
||||||
|
|
||||||
> **Read this FIRST.** When the user says "we're working on lakehouse," they mean the working code captured below — NOT what `git log` framed as "the cutover" or what memory snapshots from 2 days ago suggest. If memory contradicts this file, this file wins. Update it when something is verified working — not when a phase finishes.
|
> **Read this FIRST.** When the user says "we're working on lakehouse," they mean the working code captured below — NOT what `git log` framed as "the cutover" or what memory snapshots from 2 days ago suggest. If memory contradicts this file, this file wins. Update it when something is verified working — not when a phase finishes.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## WHAT LANDED 2026-06-30 (search index-alignment sweep + P1-already-done finding)
|
||||||
|
|
||||||
|
**Fixed a systemic hybrid-search bug.** Every `/vectors/hybrid` call that paired the
|
||||||
|
`ethereal_workers` SQL dataset (10K rows) with the `workers_500k_v1` vector index (50K,
|
||||||
|
a *different* worker set) returned wrong-entity results: same `worker_id` = different
|
||||||
|
person in each index, so a `state='IL'` query surfaced a **KY** worker flagged
|
||||||
|
`sql_verified:true`. Root cause: the default index didn't share an id-space with the
|
||||||
|
default dataset.
|
||||||
|
|
||||||
|
Swept **8 call sites** in `mcp-server/index.ts` to `ethereal_workers_v1` (10K, same
|
||||||
|
id-space as the SQL table). Aligned sites (`filter_dataset: workers_500k`) and pure-RAG
|
||||||
|
sites (`/vectors/rag`) were correctly left alone.
|
||||||
|
|
||||||
|
| Commit | Sites |
|
||||||
|
|---|---|
|
||||||
|
| `423400e` | `/search` HTTP handler |
|
||||||
|
| `35a80a7` | `search_workers` tool · `match_contract` tool · `/match` HTTP handler |
|
||||||
|
| `1570306` | 4 intelligence routes (similar-workers, role-discovery, smart_search, scenario-find) |
|
||||||
|
|
||||||
|
**Verified live** after an mcp-server restart (new PID on :3700): `/match` and `/search`
|
||||||
|
with `state='IL'` return only IL workers (Champaign/Springfield/Urbana); `state='WI'`
|
||||||
|
returns WI. Grep confirms no `workers_500k_v1` index remains paired with `ethereal_workers`.
|
||||||
|
Server boots clean. Only `mcp-server/index.ts` was touched.
|
||||||
|
|
||||||
|
**Finding — demo-plan P1 was already done.** The "search box that actually filters" item
|
||||||
|
in this doc's CURRENT PLAN (P1, from the 2026-04-27 demo audit) is **stale**. The UI now
|
||||||
|
posts structured `state`/`role` to the evolved `/intelligence/chat`, which returns real
|
||||||
|
`sql_results` (IL Forklift → 167 matches, correct workers; WI → WI). The P1 section below
|
||||||
|
describes a bug that no longer exists — don't re-implement it.
|
||||||
|
|
||||||
|
**⚠ Uncommitted WIP left untouched (pre-existing, not from this session).**
|
||||||
|
`mcp-server/search.html` + `proof.html` (edited 2026-05-30; proof.html restructured into
|
||||||
|
chapters) and a `biometric_endpoint.rs` comment cleanup remain uncommitted in the working
|
||||||
|
tree. Also untracked and newer than this doc: `RECOVERY_PLAN_2026-05-05.md`,
|
||||||
|
`mcp-server/UX_FIX_PLAN.md`, `docs/scope-sentinel/`. Someone should reconcile these — they
|
||||||
|
imply planning context this file doesn't yet capture.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## WHAT LANDED 2026-05-05 (afternoon wave — full BIPA lifecycle endpoints + UIs + ops dashboard)
|
## WHAT LANDED 2026-05-05 (afternoon wave — full BIPA lifecycle endpoints + UIs + ops dashboard)
|
||||||
|
|
||||||
After the morning's doc-reconciliation wave (described below), the afternoon wave shipped the full **operational lifecycle** for biometric data. Phase 1.6 now has the complete end-to-end runtime: candidate consent flow → photo upload → withdrawal → retention sweep flag → erase. Five commits, all on `demo/post-pr11-polish-2026-04-28`.
|
After the morning's doc-reconciliation wave (described below), the afternoon wave shipped the full **operational lifecycle** for biometric data. Phase 1.6 now has the complete end-to-end runtime: candidate consent flow → photo upload → withdrawal → retention sweep flag → erase. Five commits, all on `demo/post-pr11-polish-2026-04-28`.
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user