ab1f12296d
Replaces the single-source quote feed with a 3-source consensus system,
the differentiator for the index: pull each ticker from independent
feeds, take the median, and score confidence by how many corroborate it.
Disagreement becomes a signal instead of a silently-wrong number.
Context: Stooq retired its free q/l/ CSV quote API (404s server-side;
q/d/l behind a JS challenge), which broke the profiler basket. The
interim Yahoo-only fix worked but had a hidden wrong-baseline bug —
Yahoo's chartPreviousClose over a 5d window references the pre-window
close, so day-change read +11.65% for CAT when the real figure is
+1.11%. Cross-checking against Nasdaq + CNBC exposes exactly that class
of quiet error, which a single source can never self-report.
- quotes.ts — Yahoo + Nasdaq + CNBC fetchers (all keyless, browser UA;
Yahoo prev-close now derived from the daily candle, not the buggy
meta field), median consensus, confidence = corroborating sources / 3,
outlier flagging, and a bun:sqlite hourly time-series cache.
- quote_ingest.ts + ops/systemd/lakehouse-quote-ingest.{service,timer} —
hourly snapshot into the cache (real-time not required per product
call). Runs as root to match the mcp-server that co-owns the db;
busy_timeout serializes the two writers.
- /intelligence/ticker_quotes — serves cache-first (fast), aggregates
cold tickers on-demand once and caches them. New /quote_ingest admin
trigger.
- profiler.html — per-card confidence badge (green/amber/red) + a
per-source breakdown tooltip; "3-source consensus" labeling.
- entity.ts — the earlier Stooq->Yahoo swap for the single-ticker brief
(kept fetchStooqQuote dead-but-reversible).
Verified live on devop.live/lakehouse/profiler: liquid names read
100% / 3-source consensus; thin microcaps (BLPG, ESHSF — Yahoo-only)
honestly read 33% / 1-source. Full basket (17 tickers) seeded; hourly
timer armed. Cache db (data/_quote_cache/) is gitignored.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Lakehouse systemd units
Service definitions for long-running Lakehouse sidecars that aren't the
Rust gateway itself. The gateway has its own pre-existing unit
(lakehouse.service) that was configured at initial deploy time and
isn't tracked here.
Units
| File | Service | Port | Purpose |
|---|---|---|---|
lakehouse-auditor.service |
lakehouse-auditor |
n/a | Polls Gitea for open PRs, runs four checks (static / dynamic / inference / KB query), posts commit-status + review comment. Hard-blocks merges when claims aren't backed. |
lakehouse-context7-bridge.service |
lakehouse-context7-bridge |
:3900 |
HTTP wrapper around context7's public API for Phase 45 doc-drift detection. |
Install
sudo bash ops/systemd/install.sh
Idempotent. Copies units to /etc/systemd/system/, reloads, enables + (re)starts both services.
Operate
# Status
systemctl status lakehouse-auditor
systemctl status lakehouse-context7-bridge
# Live logs
journalctl -u lakehouse-auditor -f
# Restart
systemctl restart lakehouse-auditor
# Stop (won't restart until enable + start again)
systemctl stop lakehouse-auditor
Pause the auditor without stopping
touch /home/profit/lakehouse/auditor.paused # skip cycles until removed
rm /home/profit/lakehouse/auditor.paused # resume
Env toggles on the auditor (edit the unit file, systemctl daemon-reload, restart)
LH_AUDITOR_RUN_DYNAMIC=1 # include the hybrid fixture on every audit
# default off — fixture mutates live playbook state
LH_AUDITOR_SKIP_INFERENCE=1 # skip cloud inference for fast/cheap runs
Why both services run as root
To match the existing lakehouse.service + mcp-server + observer conventions on this host. Hardening to a dedicated unprivileged user is a follow-up: would need PATH adjustment for bun, credential file accessibility (the auditor reads /home/profit/.git-credentials which is 0600 profit:profit — root reads fine, a non-profit non-root user wouldn't).