profit/lakehouse
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
lakehouse/tests/multi-agent/playbooks/scenario-2026-04-21T03-15-34/report.md
root 41b0a99ed2 chore: add real content that was sitting untracked 
Surfaced by today's untracked-files audit. None of these are accidents —
multiple are referenced by name in CLAUDE.md and memory files but were
never added.

Categories:
- docs/PHASE_AUDIT_GUIDE.md (106 LOC) — Claude Code phase audit guidance
- ops/systemd/lakehouse-langfuse-bridge.service — Langfuse bridge unit
- package.json — top-level npm manifest
- scripts/e2e_pipeline_check.sh + production_smoke.sh — real test scripts
- reports/kimi/audit-last-week*.md — the "Two reports live" CLAUDE.md cites
- tests/multi-agent/scenarios/ — 44 staffing scenarios (cutover decision A)
- tests/multi-agent/playbooks/ — 102 playbook records
- tests/battery/, tests/agent_test/PRD.md, tests/real-world/* — real tests
- sidecar/sidecar/{lab_ui,pipeline_lab}.py — 888 LOC dev-only UIs that
  remain in service post-sidecar-drop (commit ba928b1 explicitly kept them)

Sensitivity check: scenarios use synthetic company names ("Heritage Foods",
"Cornerstone Fabrication"); audit reports describe code findings only;
no PII or secrets surfaced.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 22:22:10 -05:00

97 lines
5.8 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Scenario retrospective — Riverline Logistics — Nashville Downtown Build-Out, 2026-04-22
Executor: `qwen3.5:latest` Reviewer: `qwen3:latest` Draft: `qwen2.5:latest` Overview(T3): `gpt-oss:120b (cloud)`
Prior lessons loaded into executor context: **1** (from 2026-05-05)
## Events
| At | Kind | Role / Count | Pool | Fills | Turns | Dur(s) | Cites | Gaps |
|---|---|---|---|---|---|---|---|---|
| 07:00 | baseline_fill | Welder × 4 | 298 | ✓ 4 | 2 | 15.2 | 2 | 3 |
| 08:30 | expansion | Packaging Operator × 6 | 189 | ✓ 6 | 2 | 17.2 | 2 | 6 |
| 09:00 | baseline_fill | Shipping Clerk × 2 | 579 | ✓ 2 | 2 | 12.7 | 1 | 2 |
| 13:00 | emergency | Welder × 2 | 211 | ✓ 2 | 2 | 14.1 | 3 | 2 |
| 15:30 | misplacement | Packaging Operator × 1 | - | ✗ 0 | 0 | 32.9 | 0 | 1 |
## Final roster
| Worker | Booked | Role | City, ST | Status |
|---|---|---|---|---|
| undefined Rachel D. Lewis | 07:00 | Welder | Nashville, TN | no_show |
| undefined Kevin N. Watson | 07:00 | Welder | Nashville, TN | confirmed |
| undefined Melissa K. Rivera | 07:00 | Welder | Nashville, TN | confirmed |
| undefined Lisa F. Wood | 07:00 | Welder | Nashville, TN | confirmed |
| undefined Jamal Ruiz | 08:30 | Packaging Operator | Nashville, TN | confirmed |
| undefined Adam M. Reyes | 08:30 | Packaging Operator | Nashville, TN | confirmed |
| undefined Kenneth L. Diaz | 08:30 | Packaging Operator | Nashville, TN | confirmed |
| undefined Joshua J. Phillips | 08:30 | Packaging Operator | Nashville, TN | confirmed |
| undefined Aisha Nguyen | 08:30 | Packaging Operator | Nashville, TN | confirmed |
| undefined Joyce E. Peterson | 08:30 | Packaging Operator | Nashville, TN | confirmed |
| undefined Brenda M. Hernandez | 09:00 | Shipping Clerk | Nashville, TN | confirmed |
| undefined Rachel S. Gonzalez | 09:00 | Shipping Clerk | Nashville, TN | confirmed |
| undefined Rachel D. Lewis | 13:00 | Welder | Nashville, TN | confirmed |
| undefined Melissa K. Rivera | 13:00 | Welder | Nashville, TN | confirmed |
## Gap signals
### double_book
- **07:00** — undefined Kevin N. Watson already booked for 07:00
- **07:00** — undefined Melissa K. Rivera already booked for 07:00
- **07:00** — undefined Lisa F. Wood already booked for 07:00
- **08:30** — undefined Jamal Ruiz already booked for 07:00
- **08:30** — undefined Adam M. Reyes already booked for 07:00
- **08:30** — undefined Kenneth L. Diaz already booked for 07:00
- **08:30** — undefined Joshua J. Phillips already booked for 07:00
- **08:30** — undefined Aisha Nguyen already booked for 07:00
- **08:30** — undefined Joyce E. Peterson already booked for 07:00
- **09:00** — undefined Brenda M. Hernandez already booked for 07:00
- **09:00** — undefined Rachel S. Gonzalez already booked for 07:00
- **13:00** — undefined Rachel D. Lewis already booked for 07:00
- **13:00** — undefined Melissa K. Rivera already booked for 07:00
### drift_or_tool
- **15:30** — invalid JSON from executor: JSON Parse error: Expected '}' | raw: {"kind":"propose_done","fills":[{"candidate_id":"W500K-4654","name":"Jamal Ruiz"}],"rationale":"hybrid_search returned 20 candidates for Packaging Operator in Nashville, TN with availability > 0.5. W500K-4654 (Jamal Ruiz) has the highest score (0.91) and matches the target role/location. Per strateg
### fairness
- _cross-event_ — Rachel D. Lewis (undefined) booked 13 times today
### write_through_audit
- _post-run_ — playbook_memory has 1579 entries (ran 5 events, expected ≥ 4 new entries from this run)
## Workers touched across the week
13 distinct workers made it through to a decision. Every one is accounted for below — no-shows flagged, rebookings noted, everyone visible.
| Worker ID | Name | Events | Outcome |
|---|---|---|---|
| W500K-17215 | Rachel D. Lewis | 07:00 baseline_fill + 13:00 emergency | booked |
| W500K-16627 | Kevin N. Watson | 07:00 baseline_fill | booked |
| W500K-29052 | Melissa K. Rivera | 07:00 baseline_fill + 13:00 emergency | booked |
| W500K-40747 | Lisa F. Wood | 07:00 baseline_fill | booked |
| W500K-4654 | Jamal Ruiz | 08:30 expansion | booked |
| W500K-21124 | Adam M. Reyes | 08:30 expansion | booked |
| W500K-21175 | Kenneth L. Diaz | 08:30 expansion | booked |
| W500K-22863 | Joshua J. Phillips | 08:30 expansion | booked |
| W500K-1911 | Aisha Nguyen | 08:30 expansion | booked |
| W500K-36638 | Joyce E. Peterson | 08:30 expansion | booked |
| W500K-49412 | Brenda M. Hernandez | 09:00 baseline_fill | booked |
| W500K-18660 | Rachel S. Gonzalez | 09:00 baseline_fill | booked |
| undefined | Rachel D. Lewis | 07:00 | no_show |
## Discovered patterns (meta-index)
What the system identified across semantically-similar past fills as each event ran:
- **07:00 baseline_fill** (Welder): Across 25 similar past playbooks (28 workers examined) · recurring certifications: OSHA-10 (46%) · archetype mostly: communicator · reliability median 0.80 (range 0.19–1.00)
- **08:30 expansion** (Packaging Operator): Across 25 similar past playbooks (29 workers examined) · recurring certifications: OSHA-10 (45%) · archetype mostly: communicator · reliability median 0.77 (range 0.19–1.00)
- **09:00 baseline_fill** (Shipping Clerk): Across 25 similar past playbooks (29 workers examined) · recurring certifications: OSHA-10 (45%) · archetype mostly: communicator · reliability median 0.77 (range 0.19–1.00)
- **13:00 emergency** (Welder): Across 25 similar past playbooks (28 workers examined) · recurring certifications: OSHA-10 (46%) · archetype mostly: communicator · reliability median 0.80 (range 0.19–1.00)
- **15:30 misplacement** (Packaging Operator): —
## Narrative
- 4/5 events reached consensus.
- Final roster: 14 bookings across 1 distinct workers.
- Workers touched (booked, failed, or otherwise decided): 13.
- Playbook citations across the day: 8 (proof the feedback loop fired across events).
- Dropped events: 15:30 misplacement.
Reference in New Issue View Git Blame Copy Permalink