2 Commits

Author	SHA1	Message	Date
root	21fd3b9c61	Scrum-driven fixes: P5-001 auth wired, P42-001 truth evaluator, P9-001 journal on ingest ... Apply the highest-confidence findings from the Phase 0→42 forensic sweep after four scrum-master iterations under the adversarial prompt. Each fix is independently validated by a later scrum iteration scoring the same file higher under the same bar. Code changes ──────────── P5-001 — crates/gateway/src/auth.rs + main.rs api_key_auth was marked #[allow(dead_code)] and never wrapped around the router, so `[auth] enabled=true` logged a green message and enforced nothing. Now wired via from_fn_with_state, with constant-time header compare and /health exempted for LB probes. P42-001 — crates/truth/src/lib.rs TruthStore::check() ignored RuleCondition entirely — signature looked like enforcement, body returned every action unconditionally. Added evaluate(task_class, ctx) that actually walks FieldEquals / FieldEmpty / FieldGreater / Always against a serde_json::Value via dot-path lookup. check() kept for back-compat. Tests 14 → 24 (10 new exercising real pass/fail semantics). serde_json moved to [dependencies]. P9-001 (partial) — crates/ingestd/src/service.rs Added Optional<Journal> to IngestState + a journal.record_ingest() call on /ingest/file success. Gateway wires it with `journal.clone()` before the /journal nest consumes the original. First-ever internal mutation journal event verified live (total_events_created 0→1 after probe). Iter-4 scrum scored these files higher under same prompt: ingestd/src/service.rs 3 → 6 (P9-001 visible) truth/src/lib.rs 3 → 4 (P42-001 visible) gateway/src/auth.rs 3 → 4 (P5-001 visible) gateway/src/execution_loop 4 → 6 (indirect) storaged/src/federation 3 → 4 (indirect) Infrastructure additions ──────────────────────── * tests/real-world/scrum_master_pipeline.ts - cloud-first ladder: kimi-k2:1t → deepseek-v3.1:671b → mistral-large-3:675b → gpt-oss:120b → devstral-2:123b → qwen3.5:397b (deep final thinker) - LH_SCRUM_FORENSIC env: injects SCRUM_FORENSIC_PROMPT.md as adversarial preamble - LH_SCRUM_PROPOSAL env: per-iter fix-wave doc override - Confidence extraction (markdown + JSON), schema v4 KB rows with: verdict, critical_failures_count, verified_components_count, missing_components_count, output_format, gradient_tier - Model trust profile written per file-accept to data/_kb/model_trust.jsonl - Fire-and-forget POST to observer /event so by_source.scrum appears in /stats * mcp-server/observer.ts — unchanged in shape, confirmed receiving scrum events * ui/ — new Visual Control Plane on :3950 - Bun.serve with /data/{services,reviews,metrics,trust,overrides,findings,file,refactor_signals,search,logs/:svc,scrum_log} - Views: MAP (D3 graph, 5 overlays) / TRACE (per-file iter timeline) / TRAJECTORY (refactor signals + reverse index search) / METRICS (explainers with SOURCE + GOOD lines) / KB (card grid with tooltips) / CONSOLE (per-service journalctl tail, tabs for gateway/sidecar/observer/mcp/ctx7/auditor/langfuse) - tryFetch always attempts JSON.parse (fix for observer returning JSON without content-type) - renderNodeContext primitive-vs-object guard (fix for gateway /health string) * docs/SCRUM_FIX_WAVE.md — iter-specific scope directing the scrum * docs/SCRUM_FORENSIC_PROMPT.md — adversarial audit prompt (verdict/critical/verified schema) * docs/SCRUM_LOOP_NOTES.md — iteration observations + fix-next-loop queue * docs/SYSTEM_EVOLUTION_LAYERS.md — Layers 1-10 roadmap (trust profiling, execution DNA, drift sentinel, etc) Measurements across iterations ────────────────────────────── iter 1 (soft prompt, gpt-oss:120b): mean score 5.00/10 iter 3 (forensic, kimi-k2:1t): mean score 3.56/10 (−1.44 — bar raised) iter 4 (same bar, post fixes): mean score 4.00/10 (+0.44 — fixes landed) Score movement iter3→iter4: ↑5 ↓1 =12 21/21 first-attempt accept by kimi-k2:1t in iter 4 20/21 emitted forensic JSON (richer signal than markdown) 16 verified_components captured (proof-of-life, new metric) Permission Gradient distribution: 0 auto · 16 dry_run · 4 sim · 1 block Observer loop: by_source {scrum: 21, langfuse: 1985, phase24_audit: 1} v1/usage: 224 requests, 477K tokens, all tracked Signal classes per file (iter 3 → iter 4): CONVERGING: 1 (ingestd/service.rs — fix clearly landed) LOOPING: 4 (catalogd/registry, main, queryd/service, vectord/index_registry) ORBITING: 1 (truth — novel findings surfacing as surface ones fix) PLATEAU: 9 (scores flat with high confidence — diminishing returns) MIXED: 6 Loop thesis status ────────────────── A file's score rises only when the scrum confirms a real fix landed. No false positives yet across 3 iterations. Fixes applied to 3 files all raised their independent scores under the same adversarial prompt. Loop is measurable, not hand-wavy. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-24 02:25:43 -05:00
profit	a6f12e2609	Phase 21 Rust port + Phase 27 playbook versioning + doc-sync ... Phase 21 — Rust port of scratchpad + tree-split primitives (companion to the 2026-04-21 TS shipment). New crates/aibridge modules: context.rs — estimate_tokens (chars/4 ceil), context_window_for, assert_context_budget returning a BudgetCheck with numeric diagnostics on both success and overflow. Windows table mirrors config/models.json. continuation.rs — generate_continuable<G: TextGenerator>. Handles the two failure modes: empty-response from thinking models (geometric 2x budget backoff up to budget_cap) and truncated-non-empty (continuation with partial as scratchpad). is_structurally_complete balances braces then JSON.parse-checks. Guards the degen case "all retries empty, don't loop on empty partial". tree_split.rs — generate_tree_split map->reduce with running scratchpad. Per-shard + reduce-prompt go through assert_context_budget first; loud-fails rather than silently truncating. Oldest-digest-first scratchpad truncation at scratchpad_budget (default 6000 t). TextGenerator trait (native async-fn-in-trait, edition 2024). AiClient implements it; ScriptedGenerator test double lets tests inject canned sequences without a live Ollama. GenerateRequest gained think: Option<bool> — forwards to sidecar for per-call hidden-reasoning opt-out on hot-path JSON emitters. Three existing callsites updated (rag.rs x2, service.rs hybrid answer). Phase 27 — Playbook versioning. PlaybookEntry gained four optional fields (all #[serde(default)] so pre-Phase-27 state loads as roots): version u32, default 1 parent_id Option<String>, previous version's playbook_id superseded_at Option<String>, set when newer version replaces superseded_by Option<String>, the playbook_id that replaced New methods: revise_entry(parent_id, new_entry) — appends new version, stamps superseded_at+superseded_by on parent, inherits parent_id and sets version = parent + 1 on the new entry. Rejects revising a retired or already-superseded parent (tip-of-chain is the only valid revise target). history(playbook_id) — returns full chain root->tip from any node. Walks parent_id back to root, then superseded_by forward to tip. Cycle-safe. Superseded entries excluded from boost (same rule as retired): filter in compute_boost_for_filtered_with_role (both active-entries prefilter and geo-filtered path), rebuild_geo_index, and upsert_entry's existing- idx search. status_counts returns (total, retired, superseded, failures); /status JSON reports active = total - retired - superseded. Endpoints: POST /vectors/playbook_memory/revise GET /vectors/playbook_memory/history/{id} Doc-sync — PHASES.md + PRD.md drifted from git after Phases 24-26 shipped. Fixes applied: - Phase 24 marked shipped (commit b95dd86) with detail of observer HTTP ingest + scenario outcome streaming. PRD "NOT YET WIRED" rewritten to reflect shipped state. - Phase 25 (validity windows, commit e0a843d) added to PHASES + PRD. - Phase 26 (Mem0 upsert + Letta hot cache, commit 640db8c) added. - Phase 27 entry added to both docs. - Phase 19.6 time decay corrected: was documented as "deferred", actually wired via BOOST_HALF_LIFE_DAYS = 30.0 in playbook_memory.rs. - Phase E/Phase 8 tombstone-at-compaction limit note updated — Phase E.2 closed it. Tests: 8 new version_tests in vectord (chain-metadata stamping, retired/superseded parent rejection, boost exclusion, history from root/tip/middle, legacy default round-trip, status counts). 25 new aibridge tests (context/continuation/tree_split). Workspace total 145 green (was 120). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-21 17:40:49 -05:00