J reported https://devop.live/contractor?name=3115%20W%20POLK%20ST.%20LLC
returned 404. Cause: the anchor href was a bare /contractor, which on
devop.live routes to the LLM Team UI (port 5000) at the main site root,
not the lakehouse mcp-server (which lives under /lakehouse/*).
Every page that renders a contractor link now uses the same prefix
detector the dashboard already had:
var P = location.pathname.indexOf('/lakehouse') >= 0 ? '/lakehouse' : '';
Files updated:
- search.html: entity-brief anchor + preview anchor → P+/contractor
- console.html: permit-card contractor list → P+/contractor
- contractor.html: history.replaceState + back-link + the
/intelligence/contractor_profile fetch all use P prefix. The page
is reachable at /lakehouse/contractor on the public URL and bare
/contractor on localhost; both work without further config.
Verified:
https://devop.live/lakehouse/contractor?name=3115%20W%20POLK%20ST.%20LLC
→ 200, 29.9 KB, full profile renders. Contractor has 1 permit on
file (a small LLC), 1 geocoded so the heat map plots one marker.
The contractor.html click-target J asked for: a separate page (not a
modal, not a fall-through search) showing every angle on a contractor.
Reachable from the Co-Pilot dashboard, the staffers console, and the
search box — all anchor-wrap contractor names to /contractor?name=...
What's new on the page:
1. PROJECT INDEX — build-signal score
Single 0-100 number with the drivers laid out beneath. Driver list
is staffer-readable: "59 Chicago permits in 180d (+30) · OSHA 20
inspections (-25) · federal contractor (+15)". Score weights are
placeholders to be replaced by an ML model once the 12 awaiting
sources ship — the current 6 wired signals would not give a real
model enough features.
2. HEAT MAP — every Chicago permit they've been contact_1 or contact_2
on, last 24 months, plotted on a leaflet dark map. Color by cost
(green <$100K, amber $100K-$1M, red ≥$1M), radius proportional to
cost so the staffer sees where money + activity concentrates. Click
a marker for permit detail (cost, date, work type, address, permit
ID). All 50 of Turner Construction's geocoded recent permits in
Chicago plot end-to-end.
3. ACTIVITY TIMELINE — monthly permit count, bar chart, with the
first/last month labels so the staffer sees momentum. Tooltip on
each bar gives the count and total cost for that month.
4. 12 AWAITING SOURCES — placeholder cards for the public datasets
that would 3× the build-signal feature count. Each card has:
- source name (real, e.g. DOL Wage & Hour, EPA ECHO, MSHA, BBB)
- one-liner in coordinator language ("Has this contractor stiffed
workers? Will they pay our staffing invoices?")
- "Would show:" sample shape so the engineering scope is concrete
Order is staffing-decision relevance:
1. DOL Wage & Hour (WHD violations)
2. State Licensure Boards (active license + expiry)
3. Surety Bond Capacity (bonding ceiling)
4. EPA ECHO Compliance (env violations at sites)
5. DOT/FMCSA Carrier Safety (crash + OOS rates)
6. BBB Complaints + Rating
7. PACER Civil Suits (FLSA / Title VII / ADA)
8. UCC Lien Filings (cash flow distress)
9. D&B / Credit Bureau (PAYDEX, payment behavior)
10. State UI Employer Claims (workforce stability)
11. MSHA Mine Safety (excavation / aggregate / heavy)
12. Registered Apprenticeships (DOL RAPIDS pipeline)
Server-side: entity.ts fetchContractorHistory now pulls the 50 most
recent permits with id + lat/lng + work_description, so the heat map
and timeline have what they need without a second SQL hop. The
ContractorHistory.recent_permits type gained the optional fields.
Front-end: contractor.html got 4 new render sections, leaflet wiring
(stylesheet + script in head), placeholder grid CSS, and a PLACEHOLDERS
const at the bottom with the 12 sources. All popup HTML is built via
DOM construction (textContent + appendChild) — no innerHTML, no XSS.
console.html: contractor names from /intelligence/permit_contracts now
anchor-wrapped to /contractor?name=... so the click-through J described
works from the staffers console too. Click stops propagation so the
permit details element doesn't toggle on the same click.
Verified end-to-end via playwright — Turner Construction profile shows:
PIX score "Mixed signals — review drivers below"
Heat map: "50 permits plotted · green/amber/red"
4 section labels in order
12 placeholder cards in the documented order
Phase 8.5 was fully built on the Rust side (WorkspaceManager with
create/handoff/search/shortlist/activity/get/list, persisted to
object storage, zero-copy handoff between agents). Nothing surfaced
it in the recruiter UI. This page closes that gap.
/workspaces — split-pane UI:
Left: scrollable list of all workspaces, sorted by updated_at.
Each card shows name, tier pill (daily/weekly/monthly/pinned),
current owner, count of shortlisted candidates + activity events.
Right: selected workspace detail with five sections:
1. Header — name, tier, owner, created/updated dates, description,
previous-owners audit trail (each handoff is preserved)
2. Actions row — Hand off, Shortlist candidate, Save search, Log activity
3. Shortlist — candidates flagged with dataset + record_id + notes
4. Saved searches — named SQL queries the staffer wants to rerun
5. Activity — chronological (newest first) log of what happened
Four modals for the add/edit actions (create, handoff, shortlist,
save-search, log-activity). All forms POST through the existing
/api/* passthrough to the gateway's /workspaces/* routes.
End-to-end verified live:
1. Sarah creates 'Demo: Toledo Week 17' workspace
2. Shortlists Helen Sanchez (W500K-4661) with notes about prior endorsements
3. Logs activity: 'called — Helen confirmed Tuesday 7am shift'
4. Hands off to Kim with reason 'end of shift'
5. Kim opens the workspace: owner=kim, previous_owners=[{sarah→kim}],
sees all 3 prior events + the shortlisted Helen
— no data copy, pointer swap only (Phase 8.5 design)
Security: all dynamic content built via el(tag,cls,text) DOM helper.
Zero innerHTML on API-derived strings. Modal close-on-backdrop-click
is guarded to the backdrop element.
Nav updated across all 7 pages. Workspaces is the 7th tab.
Dashboard · Walkthrough · Architecture · Spec · Onboard · Alerts · Workspaces.
Converts the app from 'dashboard you visit' to 'system that finds you.'
Critical for the phone-first staffing shop that won't open a URL —
the system reaches out when something matters.
Daemon:
- Starts once per Bun process (guarded via globalThis sentinel)
- Default interval 15 min (configurable, min 1, max 1440)
- On each cycle, buildDigest() compares current state against prior
snapshot persisted in mcp-server/data/notification_state.json
- Events detected:
- risk_escalation: role moved to tight or critical (was ok/watch)
- deadline_approaching: staffing window falls within warn window
(default 7 days) AND deadline date differs from prior
- memory_growth: playbook_memory entries grew by >= 5 since last run
Channels (all opt-out individually via config):
- console: always on, logged to journalctl -u lakehouse-agent
- file: always on, appends JSONL to mcp-server/data/notifications.jsonl
- webhook: optional, POSTs {text, digest} to configured URL
(Slack incoming-webhook / Discord webhook / any custom endpoint)
Digest format (human-readable, fits in a Slack message):
LAKEHOUSE DIGEST — 2026-04-20 23:24
3 staffing deadlines within window:
• Production Worker — 2d to 2026-04-23 · demand 724
• Maintenance Tech — 4d to 2026-04-25 · demand 32
• Electrician — 5d to 2026-04-26 · demand 34
+779 new playbooks (total 779, 2204 endorsed names)
snapshot: 0 critical · 0 tight · $275,599,326 pipeline
/alerts page:
- Current status table (daemon state, interval, webhook, last run)
- Config form: enable toggle, interval, deadline warn window, webhook
URL + label (saved to data/notification_config.json)
- 'Fire a test digest now' button — force a cycle without waiting
- Recent digests panel shows the last 10 dispatches with full text
End-to-end verified live:
- Daemon armed successfully on startup
- First-run digest dispatched to console + file in <1s
- Events detected correctly: 3 deadlines within 7 days from real
Chicago permit data; 779 playbook entries surfaced as memory growth
- Digest text format is Slack-pastable
- Dispatch records appear in /alerts recent list
TDZ caveat: startAlertsDaemon() invocation moved to end of module so
all const/let in the alerts block evaluate before daemon reads them.
Previously failed with 'Cannot access X before initialization' when
the call lived near the top of the file. Nav added to all 6 pages:
Dashboard · Walkthrough · Architecture · Spec · Onboard · Alerts.
New /onboard page. Client-facing wizard for getting real data into
the system without engineering help.
Flow:
1. Drop a CSV (or click 'Use the sample as my data' — ships a 25-row
realistic staffing roster under /samples/staffing_roster_sample.csv)
2. Browser parses client-side. Columns auto-typed (text/int/decimal/
date). PII flagged by name hint AND content regex (emails, phones).
First rows previewed. Read-only — nothing written yet.
3. Name the dataset (lowercase+underscores). Commit.
4. Post-commit: dataset is live. Shows 4 next steps the operator can
take (SQL query, vector index, dashboard search, playbook training).
Backend:
- /onboard serves onboard.html
- /samples/*.csv serves CSV files from mcp-server/samples/ with
filename validation (only [a-zA-Z0-9_-.]+.csv, prevents path traversal)
- /onboard/ingest forwards multipart/form-data to gateway /ingest/file
preserving the boundary. The generic /api/* passthrough breaks
multipart because it reads as text and forwards as JSON; this route
uses arrayBuffer + original Content-Type.
Verified end-to-end: upload sample roster (25 rows, 12 columns) →
parse in browser → show columns + PII flags + preview → commit →
gateway writes Parquet, registers in catalog → immediately queryable:
SELECT * FROM onboard_demo2 LIMIT 3
→ Sarah Johnson, Forklift Operator, Chicago, IL, 0.92
Round-trip <1 second.
Nav updated on all pages to link Onboard. Shipped with a sample CSV
so the full flow is demonstrable without real client data.
When a real client shows up, same path — they upload their CSV.
No engineering ticket, no code change, no schema pre-definition.
Security: sample filename regex prevents path traversal. CSV parse
is client-side pure JS (no DOM injection). Commit uses existing
/ingest/file validation (schema fingerprint, PII server-side,
content-hash dedup).
J's ask: explain the full architecture so someone reading a README
can dispute it or recreate it. The repo isn't public yet; this page
IS the spec until it is.
Ch1 Repository layout — 13 crates + tests/multi-agent + docs + data,
with owned responsibility and file path per crate.
Ch2 Data ingest pipeline (8 steps) — sources (file/inbox/DB/cron),
parse+normalize with ADR-010 conservative typing, PII auto-tag,
dedup, Parquet write, catalog register with fingerprint gate,
mark embeddings stale, queryable immediately.
Ch3 Measurement & indexing — row count / fingerprint / owner /
sensitivity / freshness / lineage per dataset. HNSW vs Lance
tradeoff table with measured numbers (ADR-019). Autotune loop.
Per-profile scoping (Phase 17).
Ch4 Contract inference from external signal — Chicago permit feed
→ role mapping → worker count heuristic → timeline → hybrid
search with boost → pattern discovery → rendered card. All
pre-computed before staffer opens UI.
Ch5 What a CRM can't do — 11-row comparison table of capabilities.
Ch6 How it gets better over time — three paths:
- Phase 19 playbook boost (full math)
- Pattern discovery meta-index
- Autotune agent
Ch7 Scale story: 20 staffers, 300 contracts, midday +20/+1M surge
- Async gateway + per-staffer profile isolation + client blacklists
- 7-step surge handling flow (ingest, stale-mark, incremental refresh,
degradation, hot-swap, autotune re-enter)
- Known pain points: Ollama inference serial, RAM ceiling ~5M on
HNSW (mitigated by Lance), VRAM 1-2 models sequential,
playbook_memory unbounded.
Ch8 Error surfaces & recovery — 10-row table covering ingest schema
conflicts, bucket failures, ghost names, dual-agent drift,
empty searches, Ollama down, gateway restart, schema fingerprint
divergence. Every failure has a named surface and recovery path.
Ch9 Per-staffer context — active profile, workspace, client blacklist,
audit trail, daily summary. How 20 staffers don't see the same UI.
Ch10 Day in the life — 07:00 housekeeping → 07:30 refresh → 08:00
staffer opens → 08:15 drill down → 08:30 Call click → 09:00
second staffer shares memory → 12:30 surge → 14:00 no-show →
15:00 new embeddings live → 17:00 retrospective → 22:00
overnight trials.
Ch11 Known limits & non-goals — deferred (rate/margin, push, confidence
calibration, neural re-ranker, pm compaction, call_log cross-ref)
and explicitly out-of-scope (cloud, ACID, streaming, CRM replace,
proprietary formats, hard multi-tenant).
Also: nav updated on /dashboard, /console, /proof to link /spec.
Every architectural claim in the spec cites either a code path, an
ADR number, or a phase reference so someone skeptical can target
the specific artifact.
Old console was a chat playground. New console is a guided,
chapter-based explanation that a non-technical staffing staffer
can read top-down and finish convinced — without needing to
understand any of the underlying technology.
Six chapters, each loading live data:
1. Right now, this system is already thinking
Four stats cards pulled live: construction pipeline $, predicted
worker demand, rows under management, playbooks remembered. Then
a narrative that names the current alert posture (critical/tight/ok).
2. The demand signal is real, not made up
Expandable rows per Chicago permit work_type, with a direct link to
data.cityofchicago.org for verification. Pill labeled LIVE ·
DATA.CITYOFCHICAGO.ORG leaves no ambiguity.
3. Where your own data would live
Catalog enumerated with three pill classes:
- SWAP FOR YOUR DATA (purple) — the synthetic tables that would
be replaced by the client's ATS/CRM/call-log exports
- SYSTEM-GENERATED (blue) — playbook memory, threat_intel, kb_*
produced by the system itself
Row counts + columns visible. Names it honestly.
4. Watch the system rank candidates in real time
Takes the freshest Chicago permit, walks the staffer through all
three steps (derive need → narrow via SQL → rank + boost), shows
the top-5 workers with why, boost chip, memory chip, timeline,
and a plain-English narrative of the CRM gap.
5. Every action compounds
Playbook memory count + sample + narrative about what it means
when the staffer logs a fill.
6. Try it yourself
Free-text input hitting /intelligence/chat, renders response
with memory chip + boost chips + ranked workers.
Security: all API-derived strings go through textContent or
el(tag,cls,text) helper. Zero innerHTML usage on dynamic content.
Passes security reminder hook.
File size: 419 → ~500 lines. Visual style matches the dashboard
(same palette, typography, chip styles) so the two pages feel
like one app.
"find me a warehouse worker available today near Nashville" now:
- Parses: role=warehouse, city=Nashville, available=true
- Builds SQL: role LIKE '%warehouse%' AND city='Nashville' AND availability>0.5
- Returns: 12 Nashville warehouse workers with ZIP codes, availability %,
reliability %, skills, certs, and archetype
- Shows understanding tags so user sees what the system parsed
- 414ms, 12 records — not a generic search, a targeted answer
Recognizes 20 role keywords, 40+ cities, 10 states, availability/reliability
signals from natural language. Falls through to vector search for anything
the parser doesn't catch.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New page at /lakehouse/console — a $200/hr consultant's intelligence product:
Morning Brief (auto-loads in ~120ms across 500K profiles):
- Workforce Pulse: total, reliable %, elite %, archetype breakdown
- Geographic Bench: state-by-state reliable % with weakest-state alert
- Comeback Watch: 15K improving workers who crossed 80% reliability
- Risk Watch: 5K erratic + 5K silent workers flagged automatically
- Ready & Waiting: available + reliable workers to call first
- Role Supply: 20 roles with supply/available/reliability
Conversational Chat with 5 intelligent routes:
- "Find someone like [Name] but in OH" → vector similarity search
- "Who could handle industrial electrical work?" → semantic role discovery
(finds workers for roles that DON'T EXIST in the database)
- "What if we lose our top 5 forklift operators?" → scenario analysis
with risk rating, bench depth, state-by-state breakdown
- "Which workers should we stop placing?" → risk flagging
- Default: hybrid SQL+vector search with LLM summary
Every response shows: query steps, records scanned, response time.
Transparency kills the "AI is making it up" argument.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
root