golangLAKEHOUSE/reports/cutover/g5_first_loop_live.md
root 6507dff26d g5 cutover: first 5-loop end-to-end through Bun frontend
Companion to c522ace (cutover slice live). That commit proved
infrastructure (Bun /_go/* → Go gateway). This commit proves the
SUBSTRATE'S CORE LEARNING BEHAVIOR through the same path.

Two tests against persistent Go stack on :4110 with the 200-worker
corpus, all traffic via Bun frontend on :3700:

TEST 1: same-role boost fires with exact math
  Q1: Need 3 Forklift Operators in Aurora IL for Parallel Machining
  query_role: "Forklift Operator"

  cold (use_playbook=false):
    rank=0 id=w-43  dist=0.4449  Brian Ramirez, Springfield IL

  POST /_go/v1/matrix/playbooks/record:
    query_text=Q1, role=Forklift Operator, answer_id=w-43, score=1.0
    → playbook_id=pb-1126c52bd106df6b

  warm (use_playbook=true):
    rank=0 id=w-43  dist=0.2224  ← halved
    boosted=1, injected=0

  Math check: BoostFactor = 1 - 0.5*score = 0.5 (for score=1.0).
  Expected warm_dist = 0.4449 * 0.5 = 0.22245.
  Observed: 0.2224. 4-decimal exact through 3 HTTP hops.

TEST 2: cross-role gate prevents bleed
  Q2: Need 1 CNC Operator in Detroit MI for Beacon Freight
  query_role: "CNC Operator"
  use_playbook: true (Forklift recording from Test 1 in playbook corpus)

  result:
    rank=0 id=w-175  Kevin Ruiz       (Machine Operator, Detroit MI)
    rank=2 id=w-102  Laura Long       (Forklift Operator, Cleveland OH)
    boosted=0, injected=0  ← role gate fired correctly

  w-102 (Forklift Operator) appears at rank 2 organically via
  cosine retrieval — but boosted=0 confirms the Forklift PLAYBOOK
  did NOT influence this query. Surgical: gate suppresses
  playbook-driven boosts from cross-role recordings, leaves
  organic retrieval untouched.

What this confirms about the substrate:
1. Learning works — single recording → measurable, math-exact boost
2. Bleed protection works — role gate (real_001 fix) holds through
   cutover slice
3. Math holds across HTTP hops — Bun → gateway → matrixd → vectord
   with no drift
4. Substrate works through real production-shape framing — CORS,
   content-type, body forwarding, all transparent

The substrate's reason-for-being (5-loop learning) is now
demonstrably executing on persistent daemons under
production-shape frontend traffic.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 04:14:21 -05:00

152 lines
6.2 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# G5 cutover slice — first 5-loop end-to-end through Bun frontend
Companion to `g5_first_slice_live.md`. That report proved the
infrastructure (Bun /_go/* → Go gateway → Go upstreams). This
report proves the SUBSTRATE'S CORE BEHAVIOR through the same path:
playbook learning loop fires correctly, cross-role gate prevents
bleed, distance math hits the designed value exactly.
## What was tested
Real Bun-frontend traffic (`localhost:3700/_go/v1/...`) through
the persistent Go stack on `:4110+:4211-:4219`. Workers corpus =
200 rows from `workers_500k.parquet`.
Two tests:
1. **Same-role learning loop** — record a playbook entry on Q1's
top-1 result, re-query Q1 with `use_playbook=true`, verify
Shape A boost fires.
2. **Cross-role gate** — with the Forklift playbook recorded,
query for CNC Operator and verify the Forklift entry does
NOT bleed.
## Test 1: same-role boost fires (distance halved exactly)
```
Q1: Need 3 Forklift Operators in Aurora IL for Parallel Machining
query_role: "Forklift Operator"
corpora: ["workers"]
cold (use_playbook=false):
rank=0 id=w-43 dist=0.4449 Brian Ramirez (Forklift Operator, Springfield, IL)
rank=1 id=w-102 dist=0.4483 Laura Long (Forklift Operator, Cleveland, OH)
rank=2 id=w-19 dist=0.4548 Kimberly Williams (Forklift Operator, Peoria, IL)
cold_top1_distance = 0.4449
→ POST /_go/v1/matrix/playbooks/record:
query_text = "Need 3 Forklift Operators in Aurora IL for Parallel Machining"
role = "Forklift Operator"
answer_id = "w-43"
score = 1.0
→ playbook_id = "pb-1126c52bd106df6b"
warm (use_playbook=true):
rank=0 id=w-43 dist=0.2224 ← halved (BoostFactor=0.5 for score=1.0)
rank=1 id=w-102 dist=0.4483 (unchanged — not in playbook)
rank=2 id=w-19 dist=0.4548 (unchanged)
boosted=1, injected=0
warm_top1_distance = 0.2224
```
**Math check:** `BoostFactor = 1 - 0.5 × score = 1 - 0.5 × 1.0 = 0.5`.
Expected warm_dist = 0.4449 × 0.5 = 0.22245. Observed: 0.2224.
The substrate fired through 3 HTTP hops (Bun proxy → gateway →
matrixd) with the boost math hitting the design value to 4
decimal places.
## Test 2: cross-role gate prevents bleed
```
Q2: Need 1 CNC Operator in Detroit MI for Beacon Freight
query_role: "CNC Operator"
corpora: ["workers"]
use_playbook: true (the Forklift recording from Test 1 is in playbook corpus)
result:
rank=0 id=w-175 dist=0.4922 Kevin Ruiz (Machine Operator, Detroit, MI)
rank=1 id=w-178 dist=0.6177 Christine Jenkins (Maintenance Tech, Grand Rapids, MI)
rank=2 id=w-102 dist=0.6432 Laura Long (Forklift Operator, Cleveland, OH)
boosted=0, injected=0
```
**Read:**
- `boosted=0, injected=0`: the Forklift playbook entry did NOT
fire on this CNC query. The role gate (real_001 fix) rejected
it before BoostFactor or judge-gate logic ran.
- `w-102` (Forklift Operator) DOES appear at rank 2 — but that's
the regular cosine retrieval pulling it in via semantic
similarity (Forklift Operator and CNC Operator share some
embedder-space proximity). NOT a playbook injection.
The cross-role gate's job is surgical: prevent PLAYBOOK-DRIVEN
boosts from cross-role recordings, while leaving organic cosine
retrieval untouched. Both halves verified.
## What this confirms about the substrate
1. **Learning works**: a single recorded playbook entry produces a
measurable, mathematically-exact boost on the same-role
re-query. The 5-loop's "matrix learns from outcomes" property is
not aspirational — it's executing on persistent daemons through
production-shape frontend traffic.
2. **Bleed protection works**: the same recording does NOT
contaminate cross-role queries. The role gate (added 2026-04-30,
verified via real_002 reality test) holds through the cutover
slice.
3. **Math holds across HTTP hops**: cold 0.4449 × 0.5 = 0.2224
warm. Three intermediate hops (Bun frontend → gateway proxy →
matrixd retrieve → vectord search) and the score-driven boost
factor produces the exact designed value. No silent drift in
any layer.
4. **Substrate works through real production-shape framing**:
prior reality tests (`real_001..real_005`) ran direct against
matrixd. This test ran through Bun's CORS layer, content-type
negotiation, request-body forwarding, response-body
re-streaming. All transparent.
## What this is NOT
- Not a load test (one cold + one record + one warm + one cross-role
query, not sustained traffic).
- Not a multi-paraphrase test (single verbatim query repeated, not
the harder paraphrase-recovery property real_002+ tested).
- Not a multi-coordinator test (one synthetic operator, no
alice/bob/carol-style namespace separation).
## Repro
```bash
# Stack must be up first:
./scripts/cutover/start_go_stack.sh
# Workers corpus must be populated:
./bin/staffing_workers -limit 200 -gateway http://127.0.0.1:4110 -drop=true
# Bun mcp-server must have GO_LAKEHOUSE_URL set (via systemd
# drop-in at /etc/systemd/system/lakehouse-agent.service.d/go-cutover.conf).
# Cold:
curl -sS -X POST http://localhost:3700/_go/v1/matrix/search \
-H 'content-type: application/json' \
-d '{"query_text":"Need 3 Forklift Operators in Aurora IL for Parallel Machining","query_role":"Forklift Operator","corpora":["workers"],"k":5,"per_corpus_k":5,"use_playbook":false}' | jq
# Record:
curl -sS -X POST http://localhost:3700/_go/v1/matrix/playbooks/record \
-H 'content-type: application/json' \
-d '{"query_text":"Need 3 Forklift Operators in Aurora IL for Parallel Machining","role":"Forklift Operator","answer_id":"w-43","answer_corpus":"workers","score":1.0,"tags":["g5-cutover-slice"]}' | jq
# Warm:
curl -sS -X POST http://localhost:3700/_go/v1/matrix/search \
-H 'content-type: application/json' \
-d '{"query_text":"Need 3 Forklift Operators in Aurora IL for Parallel Machining","query_role":"Forklift Operator","corpora":["workers"],"k":5,"per_corpus_k":5,"use_playbook":true}' | jq
# Cross-role check:
curl -sS -X POST http://localhost:3700/_go/v1/matrix/search \
-H 'content-type: application/json' \
-d '{"query_text":"Need 1 CNC Operator in Detroit MI for Beacon Freight","query_role":"CNC Operator","corpora":["workers"],"k":3,"per_corpus_k":3,"use_playbook":true}' | jq
```