Companion to c522ace (cutover slice live). That commit proved
infrastructure (Bun /_go/* → Go gateway). This commit proves the
SUBSTRATE'S CORE LEARNING BEHAVIOR through the same path.
Two tests against persistent Go stack on :4110 with the 200-worker
corpus, all traffic via Bun frontend on :3700:
TEST 1: same-role boost fires with exact math
Q1: Need 3 Forklift Operators in Aurora IL for Parallel Machining
query_role: "Forklift Operator"
cold (use_playbook=false):
rank=0 id=w-43 dist=0.4449 Brian Ramirez, Springfield IL
POST /_go/v1/matrix/playbooks/record:
query_text=Q1, role=Forklift Operator, answer_id=w-43, score=1.0
→ playbook_id=pb-1126c52bd106df6b
warm (use_playbook=true):
rank=0 id=w-43 dist=0.2224 ← halved
boosted=1, injected=0
Math check: BoostFactor = 1 - 0.5*score = 0.5 (for score=1.0).
Expected warm_dist = 0.4449 * 0.5 = 0.22245.
Observed: 0.2224. 4-decimal exact through 3 HTTP hops.
TEST 2: cross-role gate prevents bleed
Q2: Need 1 CNC Operator in Detroit MI for Beacon Freight
query_role: "CNC Operator"
use_playbook: true (Forklift recording from Test 1 in playbook corpus)
result:
rank=0 id=w-175 Kevin Ruiz (Machine Operator, Detroit MI)
rank=2 id=w-102 Laura Long (Forklift Operator, Cleveland OH)
boosted=0, injected=0 ← role gate fired correctly
w-102 (Forklift Operator) appears at rank 2 organically via
cosine retrieval — but boosted=0 confirms the Forklift PLAYBOOK
did NOT influence this query. Surgical: gate suppresses
playbook-driven boosts from cross-role recordings, leaves
organic retrieval untouched.
What this confirms about the substrate:
1. Learning works — single recording → measurable, math-exact boost
2. Bleed protection works — role gate (real_001 fix) holds through
cutover slice
3. Math holds across HTTP hops — Bun → gateway → matrixd → vectord
with no drift
4. Substrate works through real production-shape framing — CORS,
content-type, body forwarding, all transparent
The substrate's reason-for-being (5-loop learning) is now
demonstrably executing on persistent daemons under
production-shape frontend traffic.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
152 lines
6.2 KiB
Markdown
152 lines
6.2 KiB
Markdown
# G5 cutover slice — first 5-loop end-to-end through Bun frontend
|
||
|
||
Companion to `g5_first_slice_live.md`. That report proved the
|
||
infrastructure (Bun /_go/* → Go gateway → Go upstreams). This
|
||
report proves the SUBSTRATE'S CORE BEHAVIOR through the same path:
|
||
playbook learning loop fires correctly, cross-role gate prevents
|
||
bleed, distance math hits the designed value exactly.
|
||
|
||
## What was tested
|
||
|
||
Real Bun-frontend traffic (`localhost:3700/_go/v1/...`) through
|
||
the persistent Go stack on `:4110+:4211-:4219`. Workers corpus =
|
||
200 rows from `workers_500k.parquet`.
|
||
|
||
Two tests:
|
||
|
||
1. **Same-role learning loop** — record a playbook entry on Q1's
|
||
top-1 result, re-query Q1 with `use_playbook=true`, verify
|
||
Shape A boost fires.
|
||
2. **Cross-role gate** — with the Forklift playbook recorded,
|
||
query for CNC Operator and verify the Forklift entry does
|
||
NOT bleed.
|
||
|
||
## Test 1: same-role boost fires (distance halved exactly)
|
||
|
||
```
|
||
Q1: Need 3 Forklift Operators in Aurora IL for Parallel Machining
|
||
query_role: "Forklift Operator"
|
||
corpora: ["workers"]
|
||
|
||
cold (use_playbook=false):
|
||
rank=0 id=w-43 dist=0.4449 Brian Ramirez (Forklift Operator, Springfield, IL)
|
||
rank=1 id=w-102 dist=0.4483 Laura Long (Forklift Operator, Cleveland, OH)
|
||
rank=2 id=w-19 dist=0.4548 Kimberly Williams (Forklift Operator, Peoria, IL)
|
||
cold_top1_distance = 0.4449
|
||
|
||
→ POST /_go/v1/matrix/playbooks/record:
|
||
query_text = "Need 3 Forklift Operators in Aurora IL for Parallel Machining"
|
||
role = "Forklift Operator"
|
||
answer_id = "w-43"
|
||
score = 1.0
|
||
→ playbook_id = "pb-1126c52bd106df6b"
|
||
|
||
warm (use_playbook=true):
|
||
rank=0 id=w-43 dist=0.2224 ← halved (BoostFactor=0.5 for score=1.0)
|
||
rank=1 id=w-102 dist=0.4483 (unchanged — not in playbook)
|
||
rank=2 id=w-19 dist=0.4548 (unchanged)
|
||
boosted=1, injected=0
|
||
warm_top1_distance = 0.2224
|
||
```
|
||
|
||
**Math check:** `BoostFactor = 1 - 0.5 × score = 1 - 0.5 × 1.0 = 0.5`.
|
||
Expected warm_dist = 0.4449 × 0.5 = 0.22245. Observed: 0.2224.
|
||
The substrate fired through 3 HTTP hops (Bun proxy → gateway →
|
||
matrixd) with the boost math hitting the design value to 4
|
||
decimal places.
|
||
|
||
## Test 2: cross-role gate prevents bleed
|
||
|
||
```
|
||
Q2: Need 1 CNC Operator in Detroit MI for Beacon Freight
|
||
query_role: "CNC Operator"
|
||
corpora: ["workers"]
|
||
use_playbook: true (the Forklift recording from Test 1 is in playbook corpus)
|
||
|
||
result:
|
||
rank=0 id=w-175 dist=0.4922 Kevin Ruiz (Machine Operator, Detroit, MI)
|
||
rank=1 id=w-178 dist=0.6177 Christine Jenkins (Maintenance Tech, Grand Rapids, MI)
|
||
rank=2 id=w-102 dist=0.6432 Laura Long (Forklift Operator, Cleveland, OH)
|
||
boosted=0, injected=0
|
||
```
|
||
|
||
**Read:**
|
||
- `boosted=0, injected=0`: the Forklift playbook entry did NOT
|
||
fire on this CNC query. The role gate (real_001 fix) rejected
|
||
it before BoostFactor or judge-gate logic ran.
|
||
- `w-102` (Forklift Operator) DOES appear at rank 2 — but that's
|
||
the regular cosine retrieval pulling it in via semantic
|
||
similarity (Forklift Operator and CNC Operator share some
|
||
embedder-space proximity). NOT a playbook injection.
|
||
|
||
The cross-role gate's job is surgical: prevent PLAYBOOK-DRIVEN
|
||
boosts from cross-role recordings, while leaving organic cosine
|
||
retrieval untouched. Both halves verified.
|
||
|
||
## What this confirms about the substrate
|
||
|
||
1. **Learning works**: a single recorded playbook entry produces a
|
||
measurable, mathematically-exact boost on the same-role
|
||
re-query. The 5-loop's "matrix learns from outcomes" property is
|
||
not aspirational — it's executing on persistent daemons through
|
||
production-shape frontend traffic.
|
||
|
||
2. **Bleed protection works**: the same recording does NOT
|
||
contaminate cross-role queries. The role gate (added 2026-04-30,
|
||
verified via real_002 reality test) holds through the cutover
|
||
slice.
|
||
|
||
3. **Math holds across HTTP hops**: cold 0.4449 × 0.5 = 0.2224
|
||
warm. Three intermediate hops (Bun frontend → gateway proxy →
|
||
matrixd retrieve → vectord search) and the score-driven boost
|
||
factor produces the exact designed value. No silent drift in
|
||
any layer.
|
||
|
||
4. **Substrate works through real production-shape framing**:
|
||
prior reality tests (`real_001..real_005`) ran direct against
|
||
matrixd. This test ran through Bun's CORS layer, content-type
|
||
negotiation, request-body forwarding, response-body
|
||
re-streaming. All transparent.
|
||
|
||
## What this is NOT
|
||
|
||
- Not a load test (one cold + one record + one warm + one cross-role
|
||
query, not sustained traffic).
|
||
- Not a multi-paraphrase test (single verbatim query repeated, not
|
||
the harder paraphrase-recovery property real_002+ tested).
|
||
- Not a multi-coordinator test (one synthetic operator, no
|
||
alice/bob/carol-style namespace separation).
|
||
|
||
## Repro
|
||
|
||
```bash
|
||
# Stack must be up first:
|
||
./scripts/cutover/start_go_stack.sh
|
||
|
||
# Workers corpus must be populated:
|
||
./bin/staffing_workers -limit 200 -gateway http://127.0.0.1:4110 -drop=true
|
||
|
||
# Bun mcp-server must have GO_LAKEHOUSE_URL set (via systemd
|
||
# drop-in at /etc/systemd/system/lakehouse-agent.service.d/go-cutover.conf).
|
||
|
||
# Cold:
|
||
curl -sS -X POST http://localhost:3700/_go/v1/matrix/search \
|
||
-H 'content-type: application/json' \
|
||
-d '{"query_text":"Need 3 Forklift Operators in Aurora IL for Parallel Machining","query_role":"Forklift Operator","corpora":["workers"],"k":5,"per_corpus_k":5,"use_playbook":false}' | jq
|
||
|
||
# Record:
|
||
curl -sS -X POST http://localhost:3700/_go/v1/matrix/playbooks/record \
|
||
-H 'content-type: application/json' \
|
||
-d '{"query_text":"Need 3 Forklift Operators in Aurora IL for Parallel Machining","role":"Forklift Operator","answer_id":"w-43","answer_corpus":"workers","score":1.0,"tags":["g5-cutover-slice"]}' | jq
|
||
|
||
# Warm:
|
||
curl -sS -X POST http://localhost:3700/_go/v1/matrix/search \
|
||
-H 'content-type: application/json' \
|
||
-d '{"query_text":"Need 3 Forklift Operators in Aurora IL for Parallel Machining","query_role":"Forklift Operator","corpora":["workers"],"k":5,"per_corpus_k":5,"use_playbook":true}' | jq
|
||
|
||
# Cross-role check:
|
||
curl -sS -X POST http://localhost:3700/_go/v1/matrix/search \
|
||
-H 'content-type: application/json' \
|
||
-d '{"query_text":"Need 1 CNC Operator in Detroit MI for Beacon Freight","query_role":"CNC Operator","corpora":["workers"],"k":3,"per_corpus_k":3,"use_playbook":true}' | jq
|
||
```
|