124 Commits

Author	SHA1	Message	Date
root	bc698eb6da	gateway: OpenCode (Zen + Go) provider adapter ... Wires opencode.ai as a /v1/chat provider. One sk-* key reaches 40 models across Anthropic, OpenAI, Google, Moonshot, DeepSeek, Zhipu, Alibaba, Minimax — billed against either the user's Zen balance (pay-per-token premium models) or Go subscription (flat-rate Kimi/GLM/DeepSeek/etc.). The unified /zen/v1 endpoint routes both; upstream picks the billing tier based on model id. Notable adapter quirks: - Strip "opencode/" prefix on outbound (mirrors openrouter/kimi pattern). Caller can use {provider:"opencode", model:"X"} or {model:"opencode/X"}. - Drop temperature for claude-*, gpt-5*, o1/o3/o4 models. Anthropic and OpenAI's reasoning lineage rejects temperature with 400 "deprecated for this model". OCChatBody now serializes temperature as Option<f64> with skip_serializing_if so omitting it produces clean JSON. - max_tokens.filter(|&n| n > 0) catches Some(0) — defensive after the same trap bit kimi.rs (empty env -> Number("") -> 0 -> 503). - 600s default upstream timeout; reasoning models on big audit prompts legitimately take 3-5 min. Override OPENCODE_TIMEOUT_SECS. Key handling: - /etc/lakehouse/opencode.env (0600 root) loaded via systemd EnvironmentFile. Same pattern as kimi.env. - OPENCODE_API_KEY env first, file scrape as fallback. Verified end-to-end: opencode/claude-opus-4-7 -> "I'm Claude, made by Anthropic." opencode/kimi-k2.6 -> PONG-K26-GO opencode/deepseek-v4-pro -> PONG-DS-V4 opencode/glm-5.1 -> PONG-GLM opencode/minimax-m2.5-free -> PONG-FREE Pricing reference (per audit @ ~14k in / 6k out): claude-opus-4-7 ~$0.22 (Zen) claude-haiku-4-5 ~$0.04 (Zen) gpt-5.5-pro ~$1.50 (Zen) gemini-3-flash ~$0.03 (Zen) kimi-k2.6 / glm / deepseek / qwen / minimax / mimo: covered by Go subscription ($10/mo, $60/mo cap). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-27 06:40:55 -05:00
root	ff5de76241	auditor + gateway: 2 fixes from kimi_architect's first real run ... Acted on 2 of 10 findings Kimi caught when auditing its own integration on PR #11 head 8d02c7f. Skipped 8 (false positives or out-of-scope). 1. crates/gateway/src/v1/kimi.rs — flatten OpenAI multimodal content array to plain string before forwarding to api.kimi.com. The Kimi coding endpoint is text-only; passing a [{type,text},...] array returns 400. Use Message::text() to concat text-parts and drop non-text. Verified with curl using array-shape content: gateway now returns "PONG-ARRAY" instead of upstream error. 2. auditor/checks/kimi_architect.ts — computeGrounding switched from readFileSync to async readFile inside Promise.all. Doesn't matter at 10 findings; would matter at 100+. Removed unused readFileSync import. Skipped findings (with reason): - drift_report.ts:18 schema bump migration concern: the strict schema_version refusal IS the migration boundary (v1 readers explicitly fail on v2; not a silent corruption risk). - replay.ts:383 ISO timestamp precision: Date.toISOString always emits "YYYY-MM-DDTHH:mm:ss.sssZ" (ms precision). False positive. - mode.rs:1035 matrix_corpus deserializer compat: deserialize_string _or_vec at mode.rs:175 already accepts both shapes. Confabulation from not seeing the deserializer in the input bundle. - /etc/lakehouse/kimi.env world-readable: actually 0600 root. Real concern would be permission-drift; not a code bug. - callKimi response.json hang: obsolete; we use curl now. - parseFindings silent-drop: ergonomic concern, not a bug. - appendMetrics join with "..": works for current path; deferred. - stubFinding dead-type extension: cosmetic. Self-audit grounding rate at v1.0.0: 10/10 file:line citations verified by grep. 2 of 10 actionable bugs landed. The other 8 were correctly flagged as concerns but didn't earn a code change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-27 06:16:23 -05:00
root	643dd2d520	gateway: direct Kimi For Coding provider adapter (api.kimi.com) ... Wires kimi-for-coding (Kimi K2.6 underneath) as a first-class /v1/chat provider so consumers can target it via {provider:"kimi"} or model prefix kimi/<model>. Bypasses the upstream-broken kimi-k2:1t on Ollama Cloud and the rate-limited moonshotai/kimi-k2.6 path through OpenRouter. Adapter shape mirrors openrouter.rs (OpenAI-compatible Chat Completions). Differences from generic OpenAI providers: - api.kimi.com is a SEPARATE account system from api.moonshot.ai and api.moonshot.cn. sk-kimi-* keys are NOT interchangeable across them. - Endpoint is User-Agent-gated to "approved" coding agents (Kimi CLI, Claude Code, Roo Code, Kilo Code, ...). Requests from generic clients return 403 access_terminated_error. Adapter sends User-Agent: claude-code/1.0.0. Per Moonshot TOS this is a tampering-class action that may result in seat suspension; J authorized 2026-04-27 with awareness of the risk. - kimi-for-coding is a reasoning model — reasoning_content counts against max_tokens. Default 800-token budget yields empty visible content with finish_reason=length. Code-review workloads need max_tokens >= 1500. - Default 600s upstream timeout (vs 180s for openrouter.rs) — code audits with full file context legitimately take 3-5 minutes. Override via KIMI_TIMEOUT_SECS env. Key handling: - /etc/lakehouse/kimi.env (0600 root) loaded via systemd EnvironmentFile - KIMI_API_KEY env first, then file scrape as fallback - /etc/systemd/system/lakehouse.service NOT included in this commit (system file outside repo); operator must add EnvironmentFile=- /etc/lakehouse/kimi.env to the lakehouse.service unit NOT in scrum_master_pipeline LADDER. The 9-rung ladder is for unattended automatic recovery; placing Kimi there would hammer a TOS-gated endpoint with hostility-policy potential. Kimi is addressable via /v1/chat for explicit invocations only — auditor integration in a follow-up commit. Verification: cargo check -p gateway --tests compiles curl /v1/chat provider=kimi 200 OK, content="PONG" curl /v1/chat model="kimi/kimi-for-coding" 200 OK (prefix routing) Kimi audit on distillation last-week 7/7 grounded findings (reports/kimi/audit-last-week-full.md) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-27 05:35:58 -05:00
root	d77622fc6b	distillation: fix 7 grounding bugs found by Kimi audit ... Kimi For Coding (api.kimi.com, kimi-for-coding) ran a forensic audit on distillation v1.0.0 with full file content. 7/7 flags verified real on grep. Substrate now matches what v1.0.0 claimed: deterministic, no schema bypasses, Rust tests compile. Fixes: - mode.rs:1035,1042 matrix_corpus Some/None -> vec![..]/vec![]; cargo check --tests now compiles (was silently broken; only bun tests were running) - scorer.ts:30 SCORER_VERSION env override removed - identical input now produces identical version stamp, not env-dependent drift - transforms.ts:181 auto_apply wall-clock fallback (new Date()) -> deterministic recorded_at fallback - replay.ts:378 recorded_run_id Date.now() -> sha256(recorded_at); replay rows now reproducible given recorded_at - receipts.ts:454,495 input_hash_match hardcoded true was misleading telemetry; bumped DRIFT_REPORT_SCHEMA_VERSION 1->2, field is now boolean|null with honest null when not computed at this layer - score_runs.ts:89-100,159 dedup keyed only on sig_hash made scorer-version bumps invisible. Composite sig_hash:scorer_version forces re-scoring - export_sft.ts:126 (ev as any).contractor bypass emitted "<contractor>" placeholder for every contract_analyses SFT row. Added typed EvidenceRecord.metadata bucket; transforms.ts populates metadata.contractor; exporter reads typed value Verification (all green): cargo check -p gateway --tests compiles bun test tests/distillation/ 145 pass / 0 fail bun acceptance 22/22 invariants bun audit-full 16/16 required checks Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-27 05:34:31 -05:00
root	20a039c379	auditor: rebuild on mode runner + drop tree-split (use distillation substrate) ... Architectural simplification leveraging Phase 5 distillation work: the auditor no longer pre-extracts facts via per-shard summaries because lakehouse_answers_v1 (gold-standard prior PR audits + observer escalations corpus) supplies cross-PR context through the mode runner's matrix retrieval. Same signal, ~50× fewer cloud calls per audit. Per-audit cost: Before: 168 gpt-oss:120b shard summaries + 3 final inference calls After: 3 deepseek-v3.1:671b mode-runner calls (full retrieval included) Wall-clock on PR #11 (1.36MB diff): Before: ~25 minutes After: 88 seconds (3/3 consensus succeeded) Files: auditor/checks/inference.ts - Default MODEL kimi-k2:1t → deepseek-v3.1:671b. kimi-k2 is hitting sustained Ollama Cloud 500 ISE (verified via repeated trivial probes; multi-hour outage). deepseek is the proven drop-in from Phase 5 distillation acceptance testing. - Dropped treeSplitDiff invocation. Diff truncates to MAX_DIFF_CHARS and goes straight to /v1/mode/execute task_class=pr_audit; mode runner pulls cross-PR context from lakehouse_answers_v1 via matrix retrieval. SHARD_MODEL retained for legacy callCloud compatibility (default qwen3-coder:480b if it ever runs). - extractAndPersistFacts now reads from truncated diff (no scratchpad post-tree-split-removal). auditor/checks/static.ts - serde-derived struct exemption (commit 107a682 shipped this; this commit is the rest of the auditor rebuild it landed alongside) - multi-line template literal awareness in isInsideQuotedString — tracks backtick state across lines so todo!() inside docstrings doesn't trip BLOCK_PATTERNS. crates/gateway/src/v1/mode.rs - pr_audit native runner mode added to VALID_MODES + is_native_mode + flags_for_mode + framing_text. PrAudit framing produces strict JSON {claim_verdicts, unflagged_gaps} for the auditor to parse. config/modes.toml - pr_audit task class with default_model=deepseek-v3.1:671b and matrix_corpus=lakehouse_answers_v1. Documents kimi-k2 outage with link to the swap rationale. Real-data audit on PR #11 head 1b433a9 (which is the PR with all the distillation work + auditor rebuild itself): - Pipeline ran to completion (88s for inference; full audit ~3 min) - 3/3 consensus runs succeeded on deepseek-v3.1:671b - 156 findings: 12 block, 23 warn, 121 info - Block findings are legitimate signal: 12 reviewer claims like "Invariants enforced (proven by tests + real run):" that the truncated diff can't directly verify. The auditor is correctly flagging claim-vs-diff divergence — exactly its job. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-26 23:32:44 -05:00
root	d1d97a045b	v1: fire observer /event from /v1/chat alongside Langfuse trace ... Observer at :3800 already collects scrum + scenario events into a ring buffer that pathway-memory + KB consolidation read from. /v1/chat now posts a lightweight {endpoint, source:"v1.chat", input_summary, output_summary, success, duration_ms} event there too — fire-and-forget tokio::spawn, observer-down doesn't block the chat response. Now any tool routed through our gateway (Pi CLI, Archon, openai SDK clients, langchain-js) shows up in the same ring buffer the scrum loop reads, ready for the same KB-consolidation analysis. Independent of the existing langfuse-bridge that polls Langfuse — this path is immediate. Verified: GET /stats shows {by_source: {v1.chat: N}} grows by 1 per chat call, both for direct curl and for Pi CLI invocations. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-26 18:01:52 -05:00
root	540a9a27ee	v1: accept OpenAI multimodal content shape (array-of-parts) ... Modern OpenAI clients (pi-ai, openai SDK 6.x, langchain-js, the official agents) send `messages[].content` as an array of content parts: `[{type:"text", text:"..."}, {type:"image_url", ...}]`. Our gateway typed `content` as plain `String` and 422'd those calls. Fix: `Message.content` is now `serde_json::Value` so requests deserialize regardless of shape. `Message::text()` flattens content-parts arrays (concat'd `text` fields, non-text parts skipped) for places that need a plain string — Ollama prompt assembly, char counts, the assistant's own response synthesis. `Message::new_text()` constructs string-content messages without writing the wrapper at each call site. Forwarders (openrouter) clone content through verbatim so providers see exactly what the client sent. Verified end-to-end: Pi CLI (`pi --print --provider openrouter`) landed a clean 1902-token request through `/v1/chat/completions`, routed to OpenRouter as `openai/gpt-oss-120b:free`, response in 1.62s, Langfuse trace `v1.chat:openrouter` recorded with provider tag. Same path that any tool using the official openai SDK takes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-26 17:56:46 -05:00
root	3a0b37ed93	v1: OpenAI-compat alias + smart provider routing — gateway is now drop-in middleware ... /v1/chat/completions route alias (same handler as /chat) lets any tool using the official `openai` SDK adopt the gateway via OPENAI_BASE_URL alone — no custom provider field needed. resolve_provider() extended: - bare `vendor/model` (slash) → openrouter (catches x-ai/grok-4.1-fast, moonshotai/kimi-k2, deepseek/deepseek-v4-flash, openai/gpt-oss-120b:free) - bare vendor model names (no slash, no colon) get auto-prefixed: gpt-* / o1-* / o3-* / o4-* → openai/<name> (OpenRouter form) claude-* → anthropic/<name> grok-* → x-ai/<name> Then routed to openrouter. Ollama models (with colon, no slash) keep default routing. Tools like pi-ai validate against an OpenAI-style catalog and send bare names — this lets them flow through cleanly. Verified end-to-end: - curl POST /v1/chat/completions {model: "gpt-4o-mini", ...} → 200, routed to openrouter as openai/gpt-4o-mini - openai SDK with baseURL=http://localhost:3100/v1 → 3 model variants all succeed (openai/gpt-4o-mini, gpt-4o-mini, x-ai/grok-4.1-fast) - Langfuse traces fire automatically on every call (v1.chat:openrouter, provider tagged in metadata) scripts/mode_pass5_variance_paid.ts gains LH_CONDITIONS env so subset runs (e.g. just isolation vs composed) take half the latency. Archon-on-Lakehouse integration: gateway side is done. Pi-ai's openai-responses backend uses /v1/responses (not /chat/completions) and its openrouter backend appears to bail in client-side validation before sending. Patching Pi locally to override baseUrl works for arch but the harness still rejects — needs more work in a follow-up. Direct openai SDK path (langchain-js / agents / patched Pi) works today. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-26 17:49:37 -05:00
root	2dbc8dbc83	v1/mode: model-aware enrichment downgrade + 3 corpora + variance harness ... Pass 5 (5 reps × 4 conditions × 1 file on grok-4.1-fast) showed composing matrix corpora is anti-additive on strong models — composed lakehouse_arch + symbols LOST 5/5 head-to-head vs codereview_isolation (Δ −1.8 grounded findings, p=0.031). Default flips to isolation; matrix path now auto- downgrades when the resolved model is strong. Mode runner: - matrix_corpus is Vec<String> (string OR array via deserialize_string_or_vec) - top_k=6 from each corpus, merge by score, take top 8 globally - chunk tag prefers doc_id over source so reviewer sees [adr:009] vs [lakehouse_arch] - is_weak_model() gate auto-downgrades codereview_lakehouse → codereview_isolation for strong models (default-strong; weak = :free suffix or local last-resort) - LH_FORCE_FULL_ENRICHMENT=1 bypasses for diagnostic runs - EnrichmentSources.downgraded_from records when the gate fires Three corpora indexed via /vectors/index (5849 chunks total): - lakehouse_arch_v1 — ADRs + phases + PRD + scrum spec (93 docs, 2119 chunks) - scrum_findings_v1 — past scrum_reviews.jsonl (168 docs, 1260 chunks; EXCLUDED from defaults — 24% out-of-bounds line citations from cross-file drift) - lakehouse_symbols_v1 — regex-extracted pub items + /// docs (656 docs, 2470 chunks) Experiment infra: - scripts/build_*_corpus.ts — re-runnable when source content changes - scripts/mode_pass5_variance_paid.ts — N reps × M conditions on one file - scripts/mode_pass5_summarize.ts — mean ± σ + head-to-head, parser handles numbered + path-with-line + path-with-symbol finding tables - scripts/mode_compare.ts — groups by mode|corpus when sweeps span corpora - scripts/mode_experiment.ts — default model bumped to x-ai/grok-4.1-fast, --corpus flag for per-call override Decisions + open follow-ups: docs/MODE_RUNNER_TUNING_PLAN.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-26 17:29:17 -05:00
root	56bf30cfd8	v1/mode: override knobs + staffing native runner + pass 2/3/4 harnesses ... Setup for the corpus-tightening experiment sweep (J 2026-04-26 — "now is the only cheap window before the corpus gets large and refactoring costs go up"). Override params on /v1/mode/execute (additive — old callers unaffected): force_matrix_corpus — Pass 2: try alternate corpora per call force_relevance_threshold — Pass 2: sweep filter strictness force_temperature — Pass 3: variance test New native mode `staffing_inference_lakehouse` (Pass 4): - Same composer architecture as codereview_lakehouse - Staffing framing: coordinator producing fillable|contingent| unfillable verdict + ranked candidate list with playbook citations - matrix_corpus = workers_500k_v8 - Validates that modes-as-prompt-molders generalizes beyond code - Framing explicitly says "do NOT fabricate workers" — the staffing analog of the lakehouse mode's symbol-grounding requirement Three sweep harnesses: scripts/mode_pass2_corpus_sweep.ts — 4 corpora × 4 thresholds × 5 files scripts/mode_pass3_variance.ts — 3 files × 3 temps × 5 reps scripts/mode_pass4_staffing.ts — 5 fill requests through staffing mode Each appends per-call rows to data/_kb/mode_experiments.jsonl which mode_compare.ts already aggregates with grounding column. Pass 1 (10 files × 5 modes broad sweep) currently running via the existing scripts/mode_experiment.ts — gateway restart deferred until it completes so the new override knobs aren't enabled mid-experiment. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-26 01:55:12 -05:00
root	7c47734287	v1/mode: parameterized runner + 5 enrichment-experiment modes ... J's directive (2026-04-26): "Create different modes so we can really dial in the architecture before it gets further along — pinpoint the failures and strengths equally so I know what direction to go in. Loop theater happens when we don't pinpoint the most accurate path." Refactored execute() to switch on mode name → EnrichmentFlags preset. Five native modes designed as deliberate experiments — each isolates one architectural axis so the comparison matrix reads off what's doing work vs what's adding latency for nothing: codereview_lakehouse — all enrichment on (ceiling) codereview_null — raw file + generic prompt (baseline) codereview_isolation — file + pathway only (no matrix) codereview_matrix_only — file + matrix only (no pathway) codereview_playbook_only — pathway only, NO file content (lossy ceiling) Each call appends a row to data/_kb/mode_experiments.jsonl with full sources + response. LH_MODE_LOG_OFF=1 to suppress. scripts/mode_experiment.ts — sweeps files × modes serially, prints live progress with per-call enrichment stats. Defaults to OpenRouter free model so cloud quota doesn't gate experiments. scripts/mode_compare.ts — reads the JSONL, outputs per-file matrix + per-mode aggregate + mode-vs-baseline win/loss with avg finding delta. Heuristic finding-count from markdown table rows; pathway citation count from preamble references. scrum_master_pipeline.ts gets a mode-runner fast path gated by LH_USE_MODE_RUNNER=1: try /v1/mode/execute first, fall through to the existing ladder if response < LH_MODE_MIN_CHARS (default 2000) or anything errors. Off by default until A/B-validated. First experiment results (2 files × 5 modes via gpt-oss-120b:free): - codereview_null produces 12.6KB response with ZERO findings (proves adversarial framing is load-bearing) - codereview_playbook_only produces MORE findings than lakehouse on average (12 vs 9) at 73% the latency — pathway memory is the dominant signal driver - codereview_matrix_only underperforms isolation by ~0.5 findings while costing the same latency — matrix corpus likely underperforming for scrum_review task class Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-26 01:36:42 -05:00
root	86f63a083d	v1/mode: codereview_lakehouse native runner — modes are prompt-molders ... J's framing (2026-04-26): "Modes are how you ask ONCE and get BETTER information — they mold the data, hyperfocus the prompt on this codebase's needs, so the model gets it right the first time without the cascading retry ladder." Built the first concrete native enrichment runner (codereview_lakehouse) that composes every context primitive the gateway exposes: 1. Focus file content (read from disk OR caller-supplied) 2. Pathway memory bug_fingerprints for this file area (ADR-021 preamble — "📚 BUGS PREVIOUSLY FOUND IN THIS FILE AREA") 3. Matrix corpus search via the task_class's matrix_corpus 4. Relevance filter (observer /relevance) drops adjacency pollution 5. Assembles ONE precise prompt with system framing 6. Single call to /v1/chat with the recommended model POST /v1/mode/execute dispatches. Native mode → runs the composer. Non-native mode → 501 NOT_IMPLEMENTED with hint (proxy to LLM Team /api/run is queued). Provider hint logic auto-routes by model name shape: - vendor/model[:tag] → openrouter - kimi-*/qwen3-coder*/deepseek-v*/mistral-large* → ollama_cloud - everything else → local ollama Live test against crates/queryd/src/delta.rs (10593 bytes, 10 historical bug fingerprints, 2 matrix chunks dropped by relevance): - enriched_chars: 12876 - response_chars: 16346 (14 findings with confidence percentages) - Model literally cited the pathway memory preamble in finding #7 - One call to free-tier gpt-oss:120b produced what previously required the 9-rung escalation ladder Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-26 00:28:46 -05:00
root	d277efbfd2	v1/mode: task_class → mode/model router (decision-only, phase 1) ... HANDOVER §queued (2026-04-25): "Mode router — port LLM Team multi-model patterns. Pick the right TOOL/MODE for each task class via the matrix, not cascade through models." Two-stage architecture: 1. Decision (POST /v1/mode) — pure recommendation, no execution. Returns {mode, model, decision: {source, fallbacks, matrix_corpus, notes}} so callers see WHY this mode was picked. 2. Execution (future POST /v1/mode/execute) — proxy to LLM Team /api/run for modes not yet ported to native Rust runners. Not wired in this phase. Splitting decision from execution lets us A/B-test the routing logic without committing to running every recommendation. The decision function is pure enough for exhaustive unit tests (3 added). config/modes.toml — initial map for 5 task_classes (scrum_review, contract_analysis, staffing_inference, fact_extract, doc_drift_check) + a default. matrix_corpus per task is reserved for the future matrix-informed routing pass. VALID_MODES list (24 modes) is kept in sync manually with LLM Team's /api/run handler at /root/llm_team_ui.py:10581. Adding a mode here without adding it upstream returns 400 from a future proxy. GET /v1/mode/list — operator introspection so a UI can render the registry table without re-parsing TOML. Live-tested: 5 task classes match, unknown classes fall through to default, force_mode override works + validates, bogus modes return 400 with the valid_modes list. Updates reference_llm_team_modes.md memory — earlier note claiming "only extract is registered" was wrong (all 25 are registered). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-26 00:16:32 -05:00
root	626f18d491	pathway_memory: audit-consensus → retire wire ... When observer's hand-review explicitly rejects the output of a hot-swap-recommended model, the matrix's recommendation was wrong for this context. Auto-retire the trace so future agents don't get the same poisoned recommendation in their preamble. crates/vectord/src/pathway_memory.rs — add `trace_uid` to HotSwapCandidate response and populate from the matched trace. This gives consumers single-trace precision for /pathway/retire. tests/real-world/scrum_master_pipeline.ts: - HotSwapCandidate interface gains trace_uid - new retirePathwayTrace() helper (fire-and-forget, fall-open) - in the obsVerdict reject branch: if hotSwap was active AND the rejected model is the hot-swap-recommended one AND observer confidence ≥0.7, fire retire and null hotSwap so post-loop replay bookkeeping doesn't double-process. - hotSwap declared `let` (was const) so it can be nulled Cycle verdicts ("needs different angle") don't trigger retire — only outright rejects do. Confidence gate avoids retiring on heuristic-fallback verdicts that come back without a confidence number. Closes the "audit-consensus → retire" item from HANDOVER.md. Live-tested: insert synthetic trace → /pathway/retire by trace_uid → retired counter 1 → 2. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-26 00:01:20 -05:00
root	6ac7f61819	pathway_memory: Mem0 versioning + deletion (upsert/revise/retire/history) ... Per J 2026-04-25: pathway_memory was append-only — every agent run added a new trace, bad/failed runs polluted the matrix forever, no notion of "this is the canonical evolved playbook." Ported playbook_memory's Phase 25/27 patterns into pathway_memory so the agent loop's matrix converges on best-known approaches per task class instead of bloating. Fields added to PathwayTrace (all #[serde(default)] for back-compat): - trace_uid: stable UUID per individual trace within a bucket - version: u32 default 1 - parent_trace_uid, superseded_at, superseded_by_trace_uid - retirement_reason (paired with existing retired:bool) Methods added to PathwayMemory: - upsert(trace) → PathwayUpsertOutcome {Added|Updated|Noop} Workflow-fingerprint dedup: ladder_attempts + final_verdict hash. Identical workflow → bumps existing replay_count instead of duplicating. - revise(parent_uid, new_trace) → PathwayReviseOutcome Chains versions; rejects retired or already-superseded parents. - retire(trace_uid, reason) → bool Marks specific trace retired with reason. Idempotent. - history(trace_uid) → Vec<PathwayTrace> Walks parent_trace_uid back to root, then superseded_by forward to tip. Cycle-safe via visited set. Retrieval gates updated: - query_hot_swap skips superseded_at.is_some() - bug_fingerprints_for skips both retired AND superseded HTTP endpoints in service.rs: - POST /vectors/pathway/upsert - POST /vectors/pathway/retire - POST /vectors/pathway/revise - GET /vectors/pathway/history/{trace_uid} scripts/seal_agent_playbook.ts switched insert→upsert + accepts SESSION_DIR arg so it can seal any archived session, not just iter4. Verified live (4/4 ops): - UPSERT first run: Added trace_uid 542ae53f - UPSERT identical: Updated, replay_count bumped 0→1 (no duplicate) - REVISE 542ae53f→87a70a61: parent stamped superseded_at, v2 created - HISTORY of v2: chain_len=2, v1 superseded, v2 tip - RETIRE iter-6 broken trace: retired=true, retirement_reason preserved - pathway_memory.stats: total=79, retired=1, reuse_rate=0.0127 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-25 19:31:44 -05:00
root	4087dde780	execution_loop: update stale test assertion to match current prompt format ... Pre-existing failure I've been noting across this session — `executor_prompt_includes_surfaced_candidates` expected the substring "W-1 Alice Smith" but the prompt format was intentionally changed (probably in a Phase 38/39 commit) to separate doc_id from name so the executor doesn't conflate `doc_id` (vector-index key) with `workers_500k.worker_id` (integer PK). Current prompt format (line 1178 in build_executor_prompt): - name="Alice Smith" city="Toledo" state="OH" (vector doc_id=W-1) The prompt body explicitly instructs the model NOT to conflate the two IDs — the format separation is the mechanism enforcing that instruction. The OLD test assertion predated that separation. Assertion now checks the semantic contract (both tokens present, any order) instead of the exact old concatenation. Workspace test result after this commit: 343 passed, 0 failed, 0 warnings (both lib + tests). This is the last stale-test hole from the phase-audit sweep — it popped up during the 41-commit push but I was leaving it as pre-existing-unrelated. J called it: sitting broken for hours is worse than a one-line assertion update. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-24 14:06:24 -05:00
root	951c6014ec	gateway: boot-time probe of truth/ file-backed rules ... Phase 42 PRD deliverable de8fb10 landed the file loader + 2 rule files. This commit wires the loader into gateway startup so the rules actually get READ at boot — catches parse errors and duplicate-ID collisions before the first request hits, rather than "silently 0 rules loaded." Scope is deliberately narrow — a probe, not full plumbing: - Reads LAKEHOUSE_TRUTH_DIR env override, defaults to /home/profit/lakehouse/truth - Skips silently with a debug log if the dir is absent - Loads rules on top of default_truth_store() into a throwaway store, logs the count (or the error) - Does NOT yet replace the per-request default_truth_store() in execution_loop or v1/chat. That plumbing needs a V1State.truth field + passing it through the request context, which is a separate scope. Why the separation matters: this commit gives ops + me a visible boot-time signal ("truth: loaded 3 file-backed rule(s)") that the loader + files work end-to-end. The next commit can confidently swap per-request stores without wondering whether the parsing even succeeds. Workspace warnings still at 0. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-24 14:03:17 -05:00
root	fee094f653	gateway/access: wire get_role + is_enabled into HTTP routes ... Two of the four #[allow(dead_code)] methods in access.rs were dead because nothing exposed them externally. access.rs itself is fine — list_roles, set_role, can_access all have live callers. But get_role and is_enabled were shaped as public API with no surface to call them through. Fix adds two small routes under /access (where the rest of the access surface lives): GET /access/roles/{agent} Calls AccessControl::get_role(agent). Returns 404 with a clear message when the agent isn't registered so clients distinguish "unknown agent" from "access denied." Part of P13-001 (ops tooling needs per-agent role introspection). GET /access/enabled Calls AccessControl::is_enabled(). Returns {"enabled": bool}. Dashboards + ops tooling poll this to confirm auth posture of the running gateway — distinct from /health which answers "is the process up" vs "is access enforcement on." #[allow(dead_code)] removed from both methods — they have live callers now via these routes, the linter will enforce that going forward. Still #[allow(dead_code)] on access.rs: masked_fields + log_query. Both need cross-crate wiring: - masked_fields wants the agent's role + query response columns, called in response shaping (queryd returning to gateway path) - log_query wants post-execution audit, called after every SQL execution on the gateway boundary Both are P13-001 phase 2 work — need AgentIdentity plumbed through the /query nested router before the call sites make sense. Flagged for follow-up. Workspace warnings still at 0. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-24 14:02:01 -05:00
root	91a38dc20b	vectord/index_registry: add last_used + build_signature (scrum iter 11) ... Scrum iter 11 on crates/vectord/src/index_registry.rs flagged two concrete field gaps (90% confidence). Both were tagged UnitMismatch / missing-invariant. IndexMeta gains two Optional fields: last_used: Option<DateTime<Utc>> PRD 11.3 — when this index was last searched against. Callers were reading created_at as a liveness proxy, which conflated "built" with "used." IndexRegistry::touch_used(name) stamps the field on every hit; incremental re-embed can now skip cold indexes without misattributing "fresh build" to "recent use." build_signature: Option<String> PRD 11.3 — stable SHA-256 of (sorted source files + chunk_size + overlap + model_version). compute_build_signature() in the same module is deterministic: file-order-invariant, changes on chunk param, changes on model version. Lets incremental re-embed answer "has anything changed since last build?" without scanning the source Parquet. Both fields are #[serde(default)] — the ~40 existing .json meta files under vectors/meta/ load unchanged. Backward-compat verified by the explicit `index_meta_deserializes_without_new_fields_backcompat` test. 7 new tests: - build_signature_is_deterministic - build_signature_order_invariant (sorted internally) - build_signature_changes_on_chunk_param - build_signature_changes_on_model_version - touch_used_updates_last_used - touch_used_is_noop_on_missing_index - index_meta_deserializes_without_new_fields_backcompat Call-site fixes: crates/vectord/src/refresh.rs:294 and crates/vectord/src/service.rs:244 both construct IndexMeta with fully-literal init, default the new fields to None. One indentation cleanup on service.rs (a pre-existing visual issue on id_prefix: None). Workspace warnings still at 0. touch_used() isn't wired into search hot-path yet — follow-up commit when the search handlers can adopt it without a broader refactor. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-24 14:00:09 -05:00
root	6532938e85	gateway/tools: truth gate for model-provided SQL (iter 11 CF-1+CF-2) ... Scrum iter 11 flagged crates/gateway/src/tools/service.rs with two 95%-confidence critical failures: CF-1: "Direct SQL execution from model-provided parameters without explicit validation or sanitization" (line 68, 95% conf) CF-2: "No permission check performed before executing SQL query; access control is bypassed entirely" (line 102, 90% conf) CF-1 is the real one — same security gap as queryd /sql had before P42-002 (9cc0ceb). Tool invocations build SQL from a template + model-provided params, then state.query_fn.execute(&sql) runs it. No truth-gate check between build and execute meant an adversarial model could emit DROP TABLE / DELETE FROM / TRUNCATE inside a param and bypass queryd's gate by routing through the tool surface instead. Fix mirrors the queryd SQL gate exactly: - ToolState grows an Arc<TruthStore> field - main.rs constructs it via truth::sql_query_guard_store() (shared default — same destructive-verb block as queryd) - call_tool evaluates the built SQL against "sql_query" task class BEFORE executing - Any Reject/Block outcome → 403 FORBIDDEN + log_invocation row marked success=false with the rule message CF-2 (access control) is P13-001 territory — needs AccessControl wiring into queryd first, still open. Flagged in memory. Workspace warnings still at 0. Pattern is now: queryd /sql → truth::sql_query_guard_store (9cc0ceb) gateway /tools → truth::sql_query_guard_store (this commit) execution_loop → truth::default_truth_store (51a1aa3) All three surfaces that pipe SQL or spec-shaped data through to the substrate now gate it. Any new SQL-executing surface should follow the same pattern. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-24 13:52:29 -05:00
root	de8fb10f52	phase-42: truth/ repo-root dir + TOML rule loader ... Phase 42 PRD (docs/CONTROL_PLANE_PRD.md:144): "truth/ dir at repo root — rule files, versioned in git." Didn't exist. Landing both the dir + its loader. New files: truth/ README.md — documents file format, rule shape, composition model (file rules are additive on top of in-code default_ truth_store), explicit non-goals (no hot reload, no inheritance) staffing.fill.toml — 2 staffing.fill rules: endorsed-count-matches-target, city-required (both Reject via FieldEmpty) staffing.any.toml — 1 staffing.any rule: no-destructive-sql-in-context via FieldContainsAny (parallel to the queryd SQL gate we already ship) crates/truth/src/loader.rs — load_from_dir(store, dir) — 5 tests: happy path, duplicate-ID rejection within files, duplicate-ID rejection against in-code rules, non-toml files skipped, missing-dir error. Alphabetical file order for reproducible error messages. crates/truth/src/lib.rs — new pub fn all_rule_ids() helper on TruthStore so the loader can detect collisions without breaching the private `rules` field. crates/truth/Cargo.toml — adds `toml` workspace dep. Composition model: file rules are ADDITIVE on top of what default_truth_store() registers in code. Operators can tune thresholds/needles/descriptions at the file layer without a code deploy. Schema changes (new RuleCondition variants) still need a code bump. Integration hook (not in this commit, flagged for follow-up): main.rs should call loader::load_from_dir(&mut store, "truth/") after default_truth_store() so file-backed rules take effect on gateway boot. Deliberately separate: this commit lands the machinery; wiring it on happens when the team is ready to own the rule file lifecycle. Total: 37 truth tests green (was 32). Workspace warnings still 0. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-24 13:44:23 -05:00
root	0b3bd28cf8	phase-40: Gemini + Claude provider adapters ... Phase 40 PRD (docs/CONTROL_PLANE_PRD.md:82-83) listed: - crates/aibridge/src/providers/gemini.rs - crates/aibridge/src/providers/claude.rs Neither existed. Landing both now, in gateway/src/v1/ (matches the existing ollama.rs + openrouter.rs sibling pattern — aibridge's providers/ is for the adapter *trait* abstractions, v1/ holds the concrete /v1/chat dispatchers that know the wire format). gemini.rs: - POST https://generativelanguage.googleapis.com/v1beta/models/ {model}:generateContent?key=<API_KEY> - Auth: query-string key (not bearer) - Maps messages → contents+parts (Gemini's wire shape), extracts from candidates[0].content.parts[0].text - 3 tests: key resolution, body serialization (camelCase generationConfig + maxOutputTokens), prefix-strip claude.rs: - POST https://api.anthropic.com/v1/messages - Auth: x-api-key header + anthropic-version: 2023-06-01 - Carries system prompt in top-level `system` field (not messages[]). Extracts from content[0].text where type=="text" - 4 tests: key resolution, body serialization with/without system field, prefix-strip v1/mod.rs: + V1State.gemini_key + claude_key Option<String> + resolve_provider() strips "gemini/" and "claude/" prefixes + /v1/chat dispatcher handles "gemini" + "claude"/"anthropic" + 2 new resolve_provider tests (prefix + strip per adapter) main.rs: + Construct both keys at startup via resolve_*_key() helpers. Missing keys log at debug (not warn) since these are optional providers — unlike OpenRouter which is the rescue rung. Every /v1/chat error path mirrors the existing pattern: - 503 SERVICE_UNAVAILABLE when key isn't configured - 502 BAD_GATEWAY with the provider's error text when the upstream call fails - Response shape always the OpenAI-compatible ChatResponse Workspace warnings still at 0. 9 new tests pass. Pre-existing test failure `executor_prompt_includes_surfaced_ candidates` at execution_loop/mod.rs:1550 is unrelated (fails on pristine HEAD too — PR fixture divergence). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-24 13:41:31 -05:00
root	b5b0c00efe	phase-43: new crates/validator — trait, staffing impls, devops scaffold ... Phase 43 PRD (docs/CONTROL_PLANE_PRD.md:161) was the one audit finding truly unimplemented — no crate, no trait, no tests, no workspace entry. Neither PHASES.md nor the source tree had any Phase 43 presence. Genuine greenfield gap. Lands the scaffold as a real crate, registered in workspace Cargo.toml: crates/validator/ src/lib.rs — Validator trait, Artifact enum (5 variants: FillProposal, EmailDraft, Playbook, TerraformPlan, AnsiblePlaybook), Report, Finding, Severity, ValidationError src/staffing/mod.rs — staffing validators module root src/staffing/fill.rs — FillValidator (schema-level: fills array + per-fill {candidate_id, name}). 4 tests. Worker-existence + status + geo checks are TODO v2 (need catalog query handle). src/staffing/email.rs — EmailValidator (to/body schema + SMS ≤160 + email subject ≤78). 4 tests. PII scan + name-consistency TODO v2. src/staffing/playbook.rs — PlaybookValidator (operation prefix, endorsed_names non-empty + ≤ target×2, fingerprint present per Phase 25). 5 tests. src/devops.rs — TerraformValidator + AnsibleValidator scaffolds. Return Unimplemented — keeps dispatcher shape stable, surfaces a clear "phase 43 not wired" signal instead of silently passing or panicking. Total: 15 tests, all green. Covers the happy paths, the common failure modes (missing fields, overfull arrays, length violations), and the dispatch-error path (wrong artifact type into wrong validator). Still open from Phase 43 (v2 work, beyond scaffold): - FillValidator catalog integration (worker-existence, status, geo/role match) — needs catalog handle in constructor - EmailValidator PII scan (shared::pii::strip_pii integration) + name-consistency cross-check - Execution loop wiring: generate → validate → observer correction + retry (bounded by max_iterations=3) — spans crates, follow-up - Observer logging: validation results to data/_observer/ops.jsonl and data/_kb/outcomes.jsonl - Scenario fixture tests against tests/multi-agent/playbooks/* Workspace warnings still at 0. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-24 13:35:22 -05:00
root	2f1b9c9768	phase-39+41: land promised artifacts — providers.toml, activation.rs, profiles/ ... Three PRD gaps closed in one coherent batch — all were cosmetic or scaffold-shaped, now real files: Phase 39 (PRD:57): + config/providers.toml — provider registry (name/base_url/auth/ default_model) for ollama, ollama_cloud, openrouter. Commented stubs for gemini + claude pending adapter work. Secrets stay in /etc/lakehouse/secrets.toml or env, NEVER inline. Phase 41 (PRD:115): + crates/vectord/src/activation.rs — ActivationTracker with the PRD-named single-flight guard ("refuse new activation if one is pending/running"). Per-profile granularity — activating A doesn't block B. 5 tests cover the full state machine. Handler body stays in service.rs for now; tracker usage integration is a follow-up. Phase 41 (PRD:113): + crates/shared/src/profiles/ with 4 submodules: * execution.rs — `pub use crate::types::ModelProfile as ExecutionProfile` (backward-compat rename per PRD) * retrieval.rs — top_k, rerank_top_k, freshness cutoff, playbook boost, sensitivity-gate enforcement * memory.rs — playbook boost ceiling, history cap, doc staleness, auto-retire-on-failure * observer.rs — failure cluster size, alert cooldown, ring size, langfuse forwarding All fields `#[serde(default)]` so existing ModelProfile files load unchanged. Still open from the same phases: - Gemini + Claude provider adapters (Phase 40 — 100-200 LOC each) - Full activate_profile handler extraction into activation.rs (Phase 41 — module-structure refactor) - Catalogd CRUD endpoints for retrieval/memory/observer profiles (Phase 41 — exists at list level, no create/update/delete yet) - truth/ repo-root directory for file-backed rules (Phase 42 — TOML loader + schema) - crates/validator crate (Phase 43 — full greenfield) Workspace warnings still at 0. 5 new tests, all green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-24 13:32:40 -05:00
root	049a4b69fb	truth: split staffing + devops into dedicated modules (Phase 42 PRD) ... Phase 42 PRD (docs/CONTROL_PLANE_PRD.md:137) specified: - crates/truth/src/staffing.rs — staffing rule shapes - crates/truth/src/devops.rs — scaffold for DevOps long-horizon PHASES.md marked Phase 42 done, but the rule sets lived inline in default_truth_store() in lib.rs. Worked, but doesn't match the PRD's module separation — and that separation matters when the long-horizon phase fleshes out devops rules: "Keeps the dispatcher signature stable so no refactor needed later." Fix: extract staffing_rules() into staffing.rs (5 rules, unchanged behavior) + create devops.rs with an empty scaffold. default_truth_store becomes a one-line composition: devops::devops_rules(staffing::staffing_rules(TruthStore::new())) 4 new tests in the submodules cover: - staffing_rules registers expected count (regression guard) - blacklisted worker fails the client-not-blacklisted rule - missing deadline fires Reject via FieldEmpty condition - devops scaffold is a no-op for now Total truth tests: 28 → 32. Workspace warnings still at 0. Still open from Phase 42 (flagged, not in this commit): - `truth/` dir at repo root for file-backed rule loading (TOML/YAML). Rules are in-code today; loader work is a separate feature. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-24 13:25:54 -05:00
root	08cc960115	vectord: Phase 41 gate fixes — 202 ACCEPTED + /profile/jobs/{id} alias ... Phase 41 PRD (docs/CONTROL_PLANE_PRD.md:121) gate: "Activate a profile → returns 202 in <100ms → job completes in background → /vectors/profile/jobs/{id} shows progress" Two concrete mismatches to PRD: 1. activate_profile returned HTTP 200, not 202. Fix: wrap the Json return in (StatusCode::ACCEPTED, Json(...)) so the async semantics are visible at the status-code level. 2. The PRD quotes GET /vectors/profile/jobs/{id} but code only exposed /vectors/jobs/{id}. Fix: add an alias route — same get_job handler, second URL matches what the PRD's polling example documents. Still open from Phase 41 (flagged for follow-up, bigger scope): - crates/shared/src/profiles/ module with ExecutionProfile, RetrievalProfile, MemoryProfile, ObserverProfile types — PRD claims them, file doesn't exist; ModelProfile still does all four roles today. This is a real schema-refactor, not 6-line work. - crates/vectord/src/activation.rs with ActivationTracker — the activation logic lives inline in service.rs; extracting it is a module-structure change. - Phase 37 hot-swap stress test in tests/multi-agent/run_stress.ts Phase 3 — PRD says it must pass, current state unknown. Workspace warnings still at 0. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-24 13:21:49 -05:00
root	999abd6999	gateway/v1: model-prefix routing closes Phase 39 PRD gate ... Phase 39 PRD (docs/CONTROL_PLANE_PRD.md:62) promised: "/v1/chat routes by `model` field: prefix match (e.g. openrouter/anthropic/claude-3.5-sonnet → OpenRouter; bare names → Ollama)" Actual behavior required clients to pass `provider: "openrouter"` explicitly. Bare `model: "openrouter/..."` would fall through to the "unknown provider ''" error. PRD gate never actually passed. Fix: resolve_provider(&ChatRequest) picks (provider, effective_model): - explicit `req.provider` wins, model passes through unchanged - else strip "openrouter/" prefix → provider="openrouter", model without prefix (OpenRouter API expects "openai/gpt-4o-mini", not "openrouter/openai/gpt-4o-mini") - else strip "cloud/" prefix → provider="ollama_cloud" - else default provider="ollama" Adapter calls use Cow<ChatRequest>: borrowed when no strip needed (zero alloc), owned when we needed to build a new model string. Keeps the hot path allocation-free for the common case. ChatRequest gains #[derive(Clone)] — needed for the Owned variant. 5 new tests pin the resolution semantics including the "explicit provider + prefixed model" corner case (trust the caller, don't double-strip). Workspace warnings unchanged at 0. Still not shipped from Phase 39: config/providers.toml — hardcoded match arms work fine in practice, centralizing them is cosmetic. Flag as a follow-up if a 4th provider lands. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-24 13:16:36 -05:00
root	81bae108f4	gateway/tools: collapse ToolRegistry::new() and new_with_defaults() into one ... Two constructors existed with a subtle trap: - `new()` had `#[allow(dead_code)]` and called `register_defaults()` via `tokio::task::block_in_place(...)` — a sync wrapper hack around an async method, fragile and unused. - `new_with_defaults()` was misleadingly named — it created the empty registry WITHOUT registering defaults, despite the name. main.rs was doing the right thing: `new_with_defaults()` + explicit `.register_defaults().await`. The misleading name was a landmine for future callers. Fix: delete the dead `new()` with its block_in_place hack, rename `new_with_defaults()` → `new()` (Rust idiom — `new` is the canonical constructor), add a docstring that says what you need to do after. Single clear API. Update the one caller in main.rs. Workspace warnings still at 0. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-24 06:44:18 -05:00
root	5df4d48109	cleanup: drop two #[allow] attributes that were hiding real dead code ... - ingestd/src/service.rs: top-of-file `#[allow(unused_imports)]` was masking genuinely unused `delete` and `patch` routing constructors in an axum import block. Removed the attribute, trimmed the imports to only `get` and `post` (what's actually used). Any future over-import now trips the unused_imports lint immediately instead of being silently allowed. - gateway/src/v1/truth.rs: `truth_router()` was a 4-line stub wrapping a single `/context` route — carried `#[allow(dead_code)]` because v1/mod.rs wires `get(truth::context)` directly onto its own router, bypassing this helper. Zero callers across the workspace. Deleted the function + allow + now-unused Router import. Left a breadcrumb comment pointing to the real wiring. Workspace warnings: 0 (lib + tests). Each #[allow] removed raises the bar on future code entering these modules — the linter now catches the same classes of bugs at PR time. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-24 06:42:49 -05:00
root	ffdc842ec3	ingestd: scope test-only imports into the test module ... schema_evolution.rs had two `#[allow(unused_imports)]` attributes hiding over-broad top-level imports: - `Schema` was imported at crate level but only used in test code - `Arc` was imported at crate level but only used in test code - `DataType` and `SchemaRef` were actually used (28 references) — the allow on that line was cargo-culted. Fix: drop the allows, move Schema + Arc into the #[cfg(test)] block where they're actually used. The non-test build no longer imports symbols it doesn't need. Test build still works because the imports are now in the test module's scope. Workspace warnings still at 0 (lib + tests). Net: -3 import lines from crate scope, +2 into test scope. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-24 06:41:15 -05:00
root	12e615bb5d	ingestd/vectord: remove two fragile unwraps on Option paths ... Both were technically safe — guarded above by map_or(true, ...) and Some(entry) assignment respectively — but relied on multi-line invariants that a future refactor could easily break. - ingestd/watcher.rs:80: path.file_name().unwrap() on a path that was already checked via map_or(true, ...) two lines up. Fix: let-else binds filename once, no double lookup, no unwrap. - vectord/promotion.rs:145: file.current.as_ref().unwrap() called TWICE on the same line to log config + trial_id. Guard via `if let Some(cur) = &file.current` so the log gracefully skips if the invariant ever breaks instead of panicking at runtime. Both are drop-in semantically: happy path identical, error path now graceful-skip instead of panic. Workspace warnings still at 0. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-24 06:39:40 -05:00
root	a934a76988	aibridge: delete deprecated estimate_tokens wrapper — fully migrated ... cdc24d8 migrated all 5 call sites to shared::model_matrix::ModelMatrix. Grep across the workspace confirms zero remaining callers (only doc comments in the new module reference the old name). Wrapper was there to smooth the transition; transition is done. Leaves a 3-line breadcrumb comment pointing to the new location so anyone opening this file sees the migration history. The deprecated wrapper itself is 4 lines deleted. Workspace warnings still at 0 (both lib + tests). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-24 06:38:01 -05:00
root	cdc24d8bd0	shared: build ModelMatrix — migrate 5 call sites off deprecated estimate_tokens ... The `aibridge::context::estimate_tokens` deprecation has been pointing at `shared::model_matrix::ModelMatrix::estimate_tokens` for a while, but that module didn't exist — so the deprecation was aspirational noise, not actionable guidance. Built the minimal target: `shared::model_matrix::ModelMatrix` with an associated `estimate_tokens(text: &str) -> usize` method. Same chars/4 ceiling heuristic as the deprecated helper. 6 tests cover empty/3/4/5-char cases, multi-byte UTF-8 (emoji count as 1 char each), and linear scaling to 400-char inputs. Migrated 5 call sites: - aibridge/context.rs:88 — opts.system token count - aibridge/context.rs:89 — prompt token count - aibridge/tree_split.rs:22 — import (now uses ModelMatrix) - aibridge/tree_split.rs:84, 89 — truncate_scratchpad budget loop - aibridge/tree_split.rs:282 — scratchpad post-truncation assertion - aibridge/context.rs:183 — system-prompt budget test Also cleaned up two parallel test warnings: - aibridge/context.rs legacy estimate_tokens_ceiling_divides_by_four test deleted (ModelMatrix's tests cover the same behavior now). - vectord/playbook_memory.rs:1650 unused_mut on e_alive. Net workspace warning count: 11 → 0 (including --tests build). The deprecated `estimate_tokens` wrapper stays in aibridge/context.rs for external callers. Future commits can remove it entirely once no public API surface still references it. The applier's warning-count gate now has a floor of 0 — any future patch that introduces a single warning trips the gate automatically. Previously a floor of 11 tolerated noise. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-24 06:32:16 -05:00
root	fdc5123f6d	cleanup: drop workspace warnings from 11 to 6 ... Three trivial cleanups that pull the workspace baseline down by five: - vectord/trial.rs: removed unused ObjectStore import (not referenced anywhere in the file; cargo's unused_imports lint was flagging it on every check). Net: -2 warnings (cascade effect from one import). - ui/main.rs:1241: `Err(e)` with unused binding → `Err(_)`. - ui/main.rs:1247: `let mut import_table` never mutated → `let`. Matters because the scrum_applier's hardened warning-count gate uses this baseline as its reject threshold. Lower baseline = lower floor = any future patch that adds a warning trips the gate earlier. Remaining 6 warnings are all aibridge context::estimate_tokens deprecation notices pointing at a planned-but-unbuilt shared::model_matrix::ModelMatrix::estimate_tokens. Fix requires creating that type (next commit). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-24 06:28:36 -05:00
root	51a1aa3ddc	gateway/execution_loop: wire truth gate (Phase 42 step 6 — was TODO) ... Line 156 had `// --- (6) TRUTH GATE — PORT FROM Phase 42 (TODO) ---` sitting empty for weeks. The Blocked outcome variant existed but was marked #[allow(dead_code)] because nothing constructed it. Now: before the main turn loop, evaluate truth rules for the request's task_class against self.req.spec. Any rule whose condition holds AND whose action is Reject/Block short-circuits to RespondOutcome::Blocked with a reason citing the rule_id. Downstream finalize() already matched Blocked at line 848 (maps to truth_block category in kb row). Mirrors the queryd/service.rs SQL gate from 9cc0ceb — same truth::evaluate contract, same short-circuit pattern, same reason shape. For staffing.fill that means rules like deadline-required and budget-required now enforce at /v1/respond entry. Workspace warnings unchanged at 11. Blocked variant no longer needs #[allow(dead_code)] because it's now constructed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-24 06:24:38 -05:00
root	d122703e9a	vectord: delete _run_embedding_job_legacy — 44 lines of explicit dead code ... Function was labeled "Legacy single-pipeline embedding (replaced by supervisor)" with a #[allow(dead_code)] attribute. Zero callers across the workspace. This is exactly what `#[allow(dead_code)]` is supposed to silently flag as "I know this is dead but I'm not committing to removing it" — so let's commit to removing it. Iter memory grep for this pattern showed 5 remaining #[allow(dead_code)] attributes in the workspace (1 here, 4 in gateway/access.rs). The four in access.rs are waiting on P13-001 (queryd → AccessControl wiring) before removing — that's cross-crate work. This one was self-contained. Net: -44 lines of dead code + comment. Workspace warnings unchanged at 11. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-24 06:22:27 -05:00
root	3963b28b50	aibridge: fix glob_match — remove dead panic branch + add multi-* support ... Iter 9 scrum flagged routing.rs with OffByOne + NullableConfusion risks on the glob matcher. Two real bugs in one function: 1. The `else if parts.len() == 1` branch was dead AND panic-hazardous: split('*') on a string containing '*' always yields ≥2 parts, so the branch was unreachable — but if ever reached (via future caller or split-behavior change), `parts[1]` would index out of bounds and panic. 2. Multi-* patterns like `gpt-*-large*` fell through to exact-match because the `parts.len() == 2` branch only handled single-*. Result: a rule like `model_pattern: "gpt-*-oss-*"` would only match the literal string "gpt-*-oss-*", never an actual gpt-4-oss-120b. Fix walks parts left-to-right: prefix check, suffix check, each interior segment must appear in order. Cursor-advance logic ensures a mid-segment that appears before cursor (duplicate prefix) can't falsely match. 8 new tests cover: exact match, exact mismatch, leading/trailing/bare wildcards, multi-* in-order, multi-* wrong-order (regression guard), and the old panic-hazard case ("a*b*c" variants) as an explicit check. Workspace warnings unchanged at 11. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-24 06:21:11 -05:00
root	c47523e5bd	queryd: add latency_ms to QueryResponse (iter 9 finding #3, 88% conf) ... Scrum iter 9 flagged that gateway's audit row stores null for `latency_ms` — required for PRD audit-log parity. The field didn't exist; adding it now with a single Instant captured at handler entry, populated on both response paths (empty batches + non-empty result). No behavior change for existing clients — they read the JSON and ignore unknown fields. Audit-log consumers can now surface p50/p99 latency from the response body instead of inferring from tracing. Narrow fingerprint on crates/queryd already has this as a known BoundaryViolation pattern (`latency_ms-row_count` key) — iter 10 on any queryd file will see the preamble say "this was fixed in iter 10" when it runs. Workspace warnings unchanged at 11. 7 policy tests still pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-24 06:18:46 -05:00
root	0a0843b605	ADR-021: semantic-correctness layer lands in pathway_memory (A+B+C) ... Phase A — data model (vectord/src/pathway_memory.rs): + SemanticFlag enum (9 variants: UnitMismatch, TypeConfusion, NullableConfusion, OffByOne, StaleReference, PseudoImpl, DeadCode, WarningNoise, BoundaryViolation) as #[serde(tag = "kind")] + TypeHint { source, symbol, type_repr } + BugFingerprint { flag, pattern_key, example, occurrences } + PathwayTrace gains semantic_flags, type_hints_used, bug_fingerprints all #[serde(default)] for back-compat deserialization of pre-ADR-021 traces on disk + build_pathway_vec now tokenizes flag:{variant} + bug:{flag}:{key} so traces with different bug histories cluster separately in the similarity gate (proven by pathway_vec_differs_when_bug_fingerprint_added test) Phase B — producer (scrum_master_pipeline.ts): + Prompt addendum: each finding must carry `**Flag: <CATEGORY>**` tag alongside the existing Confidence: NN% tag. 9 category choices plus `None` for improvements that aren't bug-shaped. + Parser extracts tagged flags from reviewer markdown; falls back to bare-word match if reviewer omits the label. Deduplicated per trace. + PathwayTracePayload gains semantic_flags / type_hints_used / bug_fingerprints fields. Wire format matches Rust serde tagged enum so TS and Rust interop directly. Phase C — pre-review enrichment: + new `/vectors/pathway/bug_fingerprints` endpoint aggregates occurrences by (flag, pattern_key) across traces sharing a narrow fingerprint, sorts by frequency, returns top-K. + scrum calls it before the ladder and prepends a PATHWAY MEMORY preamble to the reviewer prompt ("these patterns appeared N times on this file area before — check for recurrences"). Empty on fresh install; grows as the matrix index learns. Tests: 27 pathway_memory tests green (was 18). New tests: - pathway_trace_deserializes_without_new_fields_backcompat - semantic_flag_serializes_as_tagged_enum - bug_fingerprint_roundtrips_through_serde - pathway_vec_differs_when_bug_fingerprint_added - semantic_flag_discriminates_by_variant - bug_fingerprints_aggregate_by_pattern_key (sums occurrences, sorts desc) - bug_fingerprints_empty_for_unseen_fingerprint - bug_fingerprints_respects_limit - insert_preserves_semantic_fields (roundtrip via persist + reload) Workspace warnings unchanged at 11. What's still queued (not this commit): - type_hints_used population from catalogd column types + Arrow schema - bug_fingerprint extraction from reviewer output (Phase D — for now semantic_flags populate but the fingerprint key requires parsing code-shape from the finding; next iteration's work) - auditor → pathway audit_consensus update wire (explicit-fail gate) Why this commit matters: the mechanical applier's gates are syntactic (warning count, patch size, rationale-token alignment). The queryd/delta.rs base_rows bug (86901f8) was found by human reading — unit mismatch between row counts and file counts. At 100 bugs this deep, humans can't catch them all; the matrix index has to learn the shapes. This commit gives it the fields to learn into and the surface to read from. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-24 05:49:10 -05:00
root	86901f8def	queryd/delta: fix CompactResult.base_rows unit mismatch (6-line fix) ... Before: `base_rows = pre_filter_rows - delta_count` subtracted a FILE count (delta_batches.len()) from a ROW count (pre_filter_rows), producing a meaningless "rough" approximation the comment acknowledged. Now: base_rows is captured directly from the pre-extend state. Same for delta_rows, which now reports actual delta row count instead of file count. Workspace baseline warnings unchanged at 11. Flagged by scrum iter 4-7 as a PRD §8.6 contract gap (upsert semantics); this closes the reporting half. Full dedup work remains queued. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-24 05:35:30 -05:00
root	2f8b347f37	pathway_memory: consensus-designed sidecar + hot-swap learning loop ... 10-probe N=3 consensus (kimi-k2:1t / gpt-oss:120b / qwen3.5:latest / deepseek-v3.1:671b / qwen3-coder:480b / mistral-large-3:675b / qwen3.5:397b + 2 stability re-probes; 2 openrouter probes 429'd) locked the design across three rounds. Full JSON responses in data/_kb/consensus_reducer_design_{mocq3akn,mocq6pi1,mocqatik}.json. What it does Preserves FULL backtrack context per reviewed file (ladder attempts + latencies + reject reasons, KB chunks with provenance + cosine + rank, observer signals, context7 bridge hits, sub-pipeline calls, audit consensus) and indexes them by narrow fingerprint for hot-swap of proven review pathways. When scrum reviews a file: 1. narrow fingerprint = task_class + file_prefix + signal_class 2. query_hot_swap checks pathway memory for a match that passes probation (≥3 replays @ ≥80% success) + audit gate + similarity (≥0.90 cosine on normalized-metadata-token embedding) 3. if hot-swap eligible, recommended model tried first in the ladder 4. replay outcome reported back, updating the pathway's success_rate 5. pathways below 0.80 after ≥3 replays retire permanently (sticky) 6. full PathwayTrace always inserted at end of review — hot-swap grows with use, it doesn't bootstrap from nothing Gate design is load-bearing: - narrow fingerprint (6 of 8 consensus models converged on the same 3-field composition; lock) — enables generalization within crate - probation ≥3 replays — binomial tail at 80% is ~5%, below is noise - success rate ≥0.80 — mistral + qwen3-coder independently proposed this exact threshold across two rounds - similarity ≥0.90 — middle of the 0.85/0.95 consensus spread - bootstrap: null audit_consensus ALLOWED (auditor → pathway update not wired yet; probation + success_rate gates alone enforce safety during bootstrap; explicit audit FAIL still blocks) - retirement is sticky — prevents oscillation on noise Files + crates/vectord/src/pathway_memory.rs (new, 600 lines + 18 tests) PathwayTrace, LadderAttempt, KbChunkRef, ObserverSignal, BridgeHit, SubPipelineCall, AuditConsensus, HotSwapCandidate, PathwayMemory, PathwayMemoryStats. 18/18 tests green. Cosine + 32-bucket L2-normalized embedding; mirror of TS impl. M crates/vectord/src/lib.rs pub mod pathway_memory; M crates/vectord/src/service.rs VectorState grows pathway_memory field; 4 HTTP handlers (/pathway/insert, /pathway/query, /pathway/record_replay, /pathway/stats). M crates/gateway/src/main.rs Construct PathwayMemory + load from storage on boot, wire into VectorState. M tests/real-world/scrum_master_pipeline.ts Byte-matching TS bucket-hash (verified same bucket indices as Rust); pre-ladder hot-swap query; ladder reorder on hit; per-attempt latency capture; post-accept trace insert (fire-and-forget); replay outcome recording; observer /event emits pathway_hot_swap_hit, pathway_similarity, rungs_saved per review for the VCP UI. M ui/server.ts /data/pathway_stats aggregates /vectors/pathway/stats + scrum_reviews.jsonl window for the value metric. M ui/ui.js Three new metric cards: · pathway reuse rate (activity: is it firing?) · avg rungs saved (value: is it earning its keep?) · pathways tracked (stability: retirement = learning) What's not in this commit (queued) - auditor → pathway audit_consensus update wire (explicit audit-fail block activates when this lands) - bridge_hits + sub_pipeline_calls population from context7 / LLM Team extract results (fields wired, callers not yet) - replay log (PathwayReplayOutcome {matched_id, succeeded, ts}) as a separate jsonl for forensic audit of why specific replays failed Why > summarization Summaries discard the causal chain. With this, auditor can verify citation provenance, applier can distinguish lucky from learned paths, and the matrix indexing actually stores end-to-end pathways instead of just RAG chunks — which is what J meant by "why aren't we using it for everything." Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-24 05:15:32 -05:00
root	9cc0ceb894	P42-002: wire truth gate into queryd /sql + /paged SQL paths ... The scrum master flagged crates/queryd/src/service.rs across iters 3-5 with the same finding: "raw SQL forwarded to DataFusion without schema or policy gate; violates PRD §42-002 truth enforcement." Confidence 79-95%, gradient tier auto/dry_run. Applier couldn't touch it — the fix is larger than 6 lines and crosses crate boundaries. Hand-fix lands the missing enforcement point: - truth: new RuleCondition::FieldContainsAny { field, needles } with case-insensitive substring matching. 4 new unit tests cover the positive, negative, missing-field, and empty-needles paths. - truth: sql_query_guard_store() helper returns a baseline store that rejects destructive verbs (DROP/TRUNCATE/DELETE FROM) and empty SQL. - queryd: QueryState grows an Arc<TruthStore>; default router() loads sql_query_guard_store; new router_with_truth(engine, store) lets tests inject a custom store. - queryd: sql_policy_check() runs truth.evaluate("sql_query", ctx) before hitting DataFusion. Reject/Block actions on matched conditions short-circuit to HTTP 403 with the rule's message. Both /sql and /paged gated. - queryd: 7 new tests cover block/allow/case-insensitive/false- positive scenarios. "SELECT deleted_at FROM t" must NOT be rejected (substring match is narrow: "delete from", not "delete"). Total: 28 truth tests green (was 24), 7 new queryd policy tests green. Workspace baseline warnings unchanged at 11. This is a signal-driven fix the mechanical pipeline couldn't produce but the scrum master kept asking for. Closes one of four LOOPING files. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-24 04:38:52 -05:00
root	5e8d87bf34	cleanup: remove unused HashSet import from 96b46cd + tighten applier gates ... 96b46cd ("first auto-applied commit") added `use tracing;` and `use std::collections::HashSet;` to queryd/service.rs under a commit message claiming to add a destructive SQL filter. HashSet was unused — cargo check passed (warnings aren't errors) but the workspace now carries a permanent `unused_imports` warning. `use tracing;` is redundant but not flagged by the compiler, leave it. This is an honest postmortem of the rationale-diff divergence problem: emitter claimed one thing, diffed another. The cargo-green gate alone can't catch that. Applier hardening in this commit addresses all three failure modes: - new-warning gate: reject patches that keep build green but add warnings (baseline → post-patch diff) - rationale-diff token alignment heuristic: reject patches whose rationale shares no vocabulary with the actual new_string - dry-run workspace revert: COMMIT=0 was silently leaving files modified between runs; now reverts after each cargo check - prompt additions: forbid unused-symbol imports; require rationale vocabulary to appear in the diff Next-iter applier runs should produce cleaner commits or none at all. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-24 04:25:53 -05:00
root	96b46cdb91	auto-apply: 1 high-confidence fix in crates/queryd/src/service.rs ... - Add basic destructive SQL filter to mitigate PRD §42-002 violation (conf 90%) 🤖 scrum_applier.ts	2026-04-24 04:13:39 -05:00
root	8b77d67c9c	OpenRouter rescue ladder + tree-split reduce fix + observer→LLM Team + scrum_applier + first auto-applied patch ... ## Infrastructure (scrum loop hardening) crates/gateway/src/v1/openrouter.rs — new OpenRouter provider Direct HTTPS to openrouter.ai/api/v1/chat/completions with OpenAI-compatible shape. Key resolution: OPENROUTER_API_KEY env → /home/profit/.env → /root/llm_team_config.json (shares LLM Team UI's quota). Added after iter 5 hit repeated Ollama Cloud 502s on kimi-k2:1t — different provider backbone as rescue rung. Unit tests pin the URL stripping and OpenAI wire shape. crates/gateway/src/v1/mod.rs + main.rs Added `"openrouter" | "openrouter_free"` arm to /v1/chat dispatch. V1State.openrouter_key loaded at startup via openrouter::resolve_openrouter_key() mirroring the Ollama Cloud pattern. Startup log: "v1: OpenRouter key loaded — /v1/chat provider=openrouter enabled" tests/real-world/scrum_master_pipeline.ts * 9-rung ladder — kimi-k2:1t → qwen3-coder:480b → deepseek-v3.1:671b → mistral-large-3:675b → gpt-oss:120b → qwen3.5:397b → openrouter/gpt-oss-120b:free → openrouter/gemma-3-27b-it:free → local qwen3.5:latest. Added qwen3-coder:480b as rung 2 after live probes confirmed it rescues kimi-k2:1t 502s cleanly (0.9s latency, substantive reviews). Dropped devstral-2 (displaced by qwen3-coder); dropped kimi-k2.6 (not available); dropped minimax-m2.7 (returned 0 chars / 400 thinking tokens). Local fallback promoted qwen3.5:latest per J's direction 2026-04-24. * MAX_ATTEMPTS bumped 6 → 9 to accommodate the rescue tier. * Tree-split scratchpad fixed — was concatenating shard markers directly into the reviewer input, causing kimi-k2:1t to write titles like "Forensic Audit Report – file.rs (shard 3)". Now uses internal §N§ markers during accumulation and runs a proper reduce step that collapses per-shard digests into ONE coherent file-level synthesis with markers stripped. Matches the Phase 21 aibridge::tree_split map→reduce design. Fallback to stripped scratchpad if reducer returns thin. tests/real-world/scrum_applier.ts — NEW (737 lines) The auto-apply pipeline. Reads scrum_reviews.jsonl, filters rows where gradient_tier ∈ {auto, dry_run} AND confidence_avg ≥ MIN_CONF (default 90), asks the reviewer model for concrete old_string/new_string patch JSON, applies via text replacement, runs cargo check after each file, commits if green and reverts if red. Deny-list: /etc/, config/, ops/, auditor/, docs/, data/, mcp-server/, ui/, sidecar/, scripts/. Hard caps: per-patch confidence ≥ MIN_CONF, old_string must be exactly unique, max 20 lines per patch. Never runs on main without explicit LH_APPLIER_BRANCH override. Audit trail in data/_kb/auto_apply.jsonl. Empirical behavior (dry-run over iter 4 reviews): 5 eligible files → 1 green commit-ready, 2 build-red reverts, 2 all-rejected The build-green gate caught 2 bad patches before they'd have merged. mcp-server/observer.ts — LLM Team code_review escalation When a sig_hash accumulates ≥3 failures (ESCALATION_THRESHOLD), fire-and-forget POST /api/run?mode=code_review at localhost:5000 with the failure cluster context. Parses facts/entities/relationships/file_hints from the response. Writes to a new data/_kb/observer_escalations.jsonl surface. Answers J's vision of the observer triggering richer LLM Team calls when failures pile up. Non-blocking: runs parallel to existing qwen2.5 analyzer, never replaces it. Tracks escalated sig_hashes in a session-local Set to avoid re-hammering LLM Team when a cluster persists across observer cycles. crates/aibridge/src/context.rs First auto-applied patch produced by scrum_applier.ts (dry-run path — applier writes files in dry-run mode but doesn't commit; bug noted for iter 6 fix). Adds #[deprecated] annotation to the inline estimate_tokens helper pointing callers to the centralized shared::model_matrix::ModelMatrix entry point (P21-002 — duplicate token-estimator surfaces). Cargo check passes with the annotation (verified by applier's own build gate). ## Visual Control Plane (UI) ui/server.ts — Bun.serve on :3950 with /data/* fan-out: /data/services, /data/reviews, /data/metrics, /data/trust, /data/overrides, /data/findings, /data/outcomes, /data/audit_facts, /data/file/:path, /data/refactor_signals, /data/search?q=, /data/signal_classes, /data/logs/:svc (journalctl tail per systemd unit), /data/scrum_log. Bug fix: tryFetch always attempts JSON.parse before falling back to text — observer's Bun.serve returns JSON without application/json content-type, which was displaying stats as a raw string ("0 ops" on map) before. ui/index.html + ui.css — dark neo-brutalist shell. 6 views: MAP (D3 force-graph + overlays) / TRACE (per-file iter history) / TRAJECTORY (signal-class cards + refactor-signals table + reverse-index search box) / METRICS (every card has SOURCE + GOOD lines explaining where the number comes from and what target trajectory means) / KB (card grid with tooltips on every field) / CONSOLE (per-service journalctl tabs). ui/ui.js — polling client, D3 wiring, signal-class panel, refactor-signals table, reverse-index search, per-service console tabs. Bug fix: renderNodeContext had Object.entries() iterating string characters when /health returned a plain string — now guards with typeof check so "lakehouse ok" renders as one row instead of "0 l / 1 a / 2 k / ...". 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-24 03:45:35 -05:00
root	bb4a8dff34	test: committed verification for P9-001 journal-on-ingest behavior ... Responds to PR #10 auditor block (2/2 blocking: "claim not backed"): the auditor's N=3 cloud consensus flagged the "verified live" language in the description as unbacked by the diff. That was fair — the verification was a manual curl probe, not committed code. Committed verification now lives in the diff: * journal_record_ingest_increments_counter - mirrors the /ingest/file success path against an in-memory store - asserts total_events_created: 0 → 1 after record_ingest - asserts the event is retrievable by entity_id with correct fields * optional_journal_field_none_is_valid_back_compat - pins IngestState.journal as Option<Journal> - forces explicit reconsideration if a refactor makes it mandatory * journal_record_event_fields_match_adr_012_schema - pins the 11-field ADR-012 event schema against field-rot 3/3 pass. Resolves block 2. Block 1 ("no changes to ingestd/service.rs appear in the diff") was a tree-split shard-leakage false positive — the diff at lines 37-40 + 149-163 clearly adds the journal wiring; this commit moves those lines into direct test-exercised contact so the next audit cycle has fewer shards to stitch together. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-24 02:40:07 -05:00
root	21fd3b9c61	Scrum-driven fixes: P5-001 auth wired, P42-001 truth evaluator, P9-001 journal on ingest ... Apply the highest-confidence findings from the Phase 0→42 forensic sweep after four scrum-master iterations under the adversarial prompt. Each fix is independently validated by a later scrum iteration scoring the same file higher under the same bar. Code changes ──────────── P5-001 — crates/gateway/src/auth.rs + main.rs api_key_auth was marked #[allow(dead_code)] and never wrapped around the router, so `[auth] enabled=true` logged a green message and enforced nothing. Now wired via from_fn_with_state, with constant-time header compare and /health exempted for LB probes. P42-001 — crates/truth/src/lib.rs TruthStore::check() ignored RuleCondition entirely — signature looked like enforcement, body returned every action unconditionally. Added evaluate(task_class, ctx) that actually walks FieldEquals / FieldEmpty / FieldGreater / Always against a serde_json::Value via dot-path lookup. check() kept for back-compat. Tests 14 → 24 (10 new exercising real pass/fail semantics). serde_json moved to [dependencies]. P9-001 (partial) — crates/ingestd/src/service.rs Added Optional<Journal> to IngestState + a journal.record_ingest() call on /ingest/file success. Gateway wires it with `journal.clone()` before the /journal nest consumes the original. First-ever internal mutation journal event verified live (total_events_created 0→1 after probe). Iter-4 scrum scored these files higher under same prompt: ingestd/src/service.rs 3 → 6 (P9-001 visible) truth/src/lib.rs 3 → 4 (P42-001 visible) gateway/src/auth.rs 3 → 4 (P5-001 visible) gateway/src/execution_loop 4 → 6 (indirect) storaged/src/federation 3 → 4 (indirect) Infrastructure additions ──────────────────────── * tests/real-world/scrum_master_pipeline.ts - cloud-first ladder: kimi-k2:1t → deepseek-v3.1:671b → mistral-large-3:675b → gpt-oss:120b → devstral-2:123b → qwen3.5:397b (deep final thinker) - LH_SCRUM_FORENSIC env: injects SCRUM_FORENSIC_PROMPT.md as adversarial preamble - LH_SCRUM_PROPOSAL env: per-iter fix-wave doc override - Confidence extraction (markdown + JSON), schema v4 KB rows with: verdict, critical_failures_count, verified_components_count, missing_components_count, output_format, gradient_tier - Model trust profile written per file-accept to data/_kb/model_trust.jsonl - Fire-and-forget POST to observer /event so by_source.scrum appears in /stats * mcp-server/observer.ts — unchanged in shape, confirmed receiving scrum events * ui/ — new Visual Control Plane on :3950 - Bun.serve with /data/{services,reviews,metrics,trust,overrides,findings,file,refactor_signals,search,logs/:svc,scrum_log} - Views: MAP (D3 graph, 5 overlays) / TRACE (per-file iter timeline) / TRAJECTORY (refactor signals + reverse index search) / METRICS (explainers with SOURCE + GOOD lines) / KB (card grid with tooltips) / CONSOLE (per-service journalctl tail, tabs for gateway/sidecar/observer/mcp/ctx7/auditor/langfuse) - tryFetch always attempts JSON.parse (fix for observer returning JSON without content-type) - renderNodeContext primitive-vs-object guard (fix for gateway /health string) * docs/SCRUM_FIX_WAVE.md — iter-specific scope directing the scrum * docs/SCRUM_FORENSIC_PROMPT.md — adversarial audit prompt (verdict/critical/verified schema) * docs/SCRUM_LOOP_NOTES.md — iteration observations + fix-next-loop queue * docs/SYSTEM_EVOLUTION_LAYERS.md — Layers 1-10 roadmap (trust profiling, execution DNA, drift sentinel, etc) Measurements across iterations ────────────────────────────── iter 1 (soft prompt, gpt-oss:120b): mean score 5.00/10 iter 3 (forensic, kimi-k2:1t): mean score 3.56/10 (−1.44 — bar raised) iter 4 (same bar, post fixes): mean score 4.00/10 (+0.44 — fixes landed) Score movement iter3→iter4: ↑5 ↓1 =12 21/21 first-attempt accept by kimi-k2:1t in iter 4 20/21 emitted forensic JSON (richer signal than markdown) 16 verified_components captured (proof-of-life, new metric) Permission Gradient distribution: 0 auto · 16 dry_run · 4 sim · 1 block Observer loop: by_source {scrum: 21, langfuse: 1985, phase24_audit: 1} v1/usage: 224 requests, 477K tokens, all tracked Signal classes per file (iter 3 → iter 4): CONVERGING: 1 (ingestd/service.rs — fix clearly landed) LOOPING: 4 (catalogd/registry, main, queryd/service, vectord/index_registry) ORBITING: 1 (truth — novel findings surfacing as surface ones fix) PLATEAU: 9 (scores flat with high confidence — diminishing returns) MIXED: 6 Loop thesis status ────────────────── A file's score rises only when the scrum confirms a real fix landed. No false positives yet across 3 iterations. Fixes applied to 3 files all raised their independent scores under the same adversarial prompt. Loop is measurable, not hand-wavy. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-24 02:25:43 -05:00
root	f59ddbebd4	Phase 41: Profile System Expansion ... - ProfileType enum: Execution, Retrieval, Memory, Observer - Per-type endpoints: /profiles/retrieval, /profiles/memory, /profiles/observer - profile_type field on ModelProfile - All tests pass	2026-04-23 03:07:22 -05:00
root	55f8e0fe6e	Phase 40: Routing Engine + Policy ... - RoutingEngine with RouteDecision (model_pattern → provider) - config/routing.toml: rules, fallback chain, cost gating - Per-provider Usage tracking in /v1/usage response - 12 gateway tests green	2026-04-23 02:36:45 -05:00
root	e27a17e950	Phase 39: Provider Adapter Refactor ... - ProviderAdapter trait with chat(), embed(), unload(), health() - OllamaAdapter wrapping existing AiClient - OpenRouterAdapter for openrouter.ai API integration - provider_key() routing by model prefix (openrouter/*, etc)	2026-04-23 02:24:15 -05:00