lakehouse

History

root fd4b6836ae IVF_PQ recall harness — closes ADR-019's explicit measurement gap

POST /vectors/lance/recall/{index} runs an existing harness through
Lance IVF_PQ search and measures recall@k against brute-force ground
truth. Uses the same EvalSet + ground_truth infrastructure as the
HNSW trial system — no new harness format needed.

First real measurement on resumes_100k_v2 (100K × 768d, 20 queries):
  IVF_PQ (316 partitions, 8 bits, 48 subvectors): recall@10 = 0.805
  For comparison — HNSW ec=80 es=30: recall@10 = 1.000

ADR-019 predicted "likely 0.85-0.95" — actual is 0.805. Slightly
below, but now the harness exists to iterate: increase partitions,
try ivf_hnsw_pq, tune subvectors. The measurement infrastructure
is the deliverable, not any specific recall target.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-04-16 20:52:34 -05:00

agent.rs

Phases 16.2 + L2 + 17 VRAM gate + MySQL + 18 Lance hybrid milestone

2026-04-16 20:24:46 -05:00

autotune.rs

Phase 16: Hot-swap generations + autotune agent loop

2026-04-16 10:26:21 -05:00

chunker.rs

Fix chunker UTF-8 boundary panic (causes 120GB OOM in refresh path)

2026-04-16 03:27:17 -05:00

embedding_cache.rs

Federation foundation + HNSW trial system + Postgres streaming + PRD reframe