lakehouse

profit/lakehouse

Fork 0

Commit Graph

Author	SHA1	Message	Date
profit	5b1fcf6d27	Phase 28-36 body of work Accumulated since a6f12e2 (Phase 21 Rust port + Phase 27 versioning): - Phase 36: embed_semaphore on VectorState (permits=1) serializes seed embed calls — prevents sidecar socket collisions under concurrent /seed stress load - Phase 31+: run_stress.ts 6-task diverse stress scaffolding; run_e2e_rated.ts + orchestrator.ts tightening - Catalog dedupe cleanup: 16 duplicate manifests removed; canonical candidates.parquet (10.5MB -> 76KB) + placements.parquet (1.2MB -> 11KB) regenerated post-dedupe; fresh manifests for active datasets - vectord: harness EvalSet refinements (+181), agent portfolio rotation + ingest triggers (+158), autotune + rag adjustments - catalogd/storaged/ingestd/mcp-server: misc tightening - docs: Phase 28-36 PRD entries + DECISIONS ADR additions; control-plane pivot banner added to top of docs/PRD.md (pointing at docs/CONTROL_PLANE_PRD.md which lands in next commit) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-22 02:41:15 -05:00
root	bd8c30c7bd	Public URL: devop.live/lakehouse/proof — SSL, no IP needed Added nginx proxy: /lakehouse/* → localhost:3700 (agent gateway). Separate include file so the main llms3 config stays clean. https://devop.live/lakehouse/proof — styled proof page https://devop.live/lakehouse/proof.json — raw verification data https://devop.live/lakehouse/ — dashboard Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-17 12:41:53 -05:00
root	f9f92706f3	RAG reranker + manifest bucket fix — quality improvements from eval RAG pipeline now includes a cross-encoder rerank step between retrieval and generation. The LLM re-sorts top-K results by relevance before they become context. Falls back to original order if model output is unparseable (~5% with 7B models). Also improved the generation prompt to be domain-aware ("staffing database") and request specific citations. Fixed 4 catalog manifests with bucket="data" (pre-federation leftover) that poisoned the entire DataFusion query context on startup. The "users", "lab_trials", "meta_runs", and "new_candidates" datasets now correctly reference bucket="primary". This bug was surfaced by the quality evaluation pipeline — wouldn't have been found by structural tests alone. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-16 22:19:11 -05:00

Author

SHA1

Message

Date

profit

5b1fcf6d27

Phase 28-36 body of work

Accumulated since a6f12e2 (Phase 21 Rust port + Phase 27 versioning):

- Phase 36: embed_semaphore on VectorState (permits=1) serializes
  seed embed calls — prevents sidecar socket collisions under
  concurrent /seed stress load
- Phase 31+: run_stress.ts 6-task diverse stress scaffolding;
  run_e2e_rated.ts + orchestrator.ts tightening
- Catalog dedupe cleanup: 16 duplicate manifests removed; canonical
  candidates.parquet (10.5MB -> 76KB) + placements.parquet (1.2MB ->
  11KB) regenerated post-dedupe; fresh manifests for active datasets
- vectord: harness EvalSet refinements (+181), agent portfolio
  rotation + ingest triggers (+158), autotune + rag adjustments
- catalogd/storaged/ingestd/mcp-server: misc tightening
- docs: Phase 28-36 PRD entries + DECISIONS ADR additions;
  control-plane pivot banner added to top of docs/PRD.md (pointing
  at docs/CONTROL_PLANE_PRD.md which lands in next commit)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-04-22 02:41:15 -05:00

root

bd8c30c7bd

Public URL: devop.live/lakehouse/proof — SSL, no IP needed

Added nginx proxy: /lakehouse/* → localhost:3700 (agent gateway).
Separate include file so the main llms3 config stays clean.

https://devop.live/lakehouse/proof  — styled proof page
https://devop.live/lakehouse/proof.json — raw verification data
https://devop.live/lakehouse/ — dashboard

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-04-17 12:41:53 -05:00

root

f9f92706f3

RAG reranker + manifest bucket fix — quality improvements from eval

RAG pipeline now includes a cross-encoder rerank step between retrieval
and generation. The LLM re-sorts top-K results by relevance before
they become context. Falls back to original order if model output is
unparseable (~5% with 7B models). Also improved the generation prompt
to be domain-aware ("staffing database") and request specific citations.

Fixed 4 catalog manifests with bucket="data" (pre-federation leftover)
that poisoned the entire DataFusion query context on startup. The
"users", "lab_trials", "meta_runs", and "new_candidates" datasets
now correctly reference bucket="primary". This bug was surfaced by
the quality evaluation pipeline — wouldn't have been found by
structural tests alone.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-04-16 22:19:11 -05:00

3 Commits