root ab1f12296d profiler: multi-source quote aggregation with confidence scoring
Replaces the single-source quote feed with a 3-source consensus system,
the differentiator for the index: pull each ticker from independent
feeds, take the median, and score confidence by how many corroborate it.
Disagreement becomes a signal instead of a silently-wrong number.

Context: Stooq retired its free q/l/ CSV quote API (404s server-side;
q/d/l behind a JS challenge), which broke the profiler basket. The
interim Yahoo-only fix worked but had a hidden wrong-baseline bug —
Yahoo's chartPreviousClose over a 5d window references the pre-window
close, so day-change read +11.65% for CAT when the real figure is
+1.11%. Cross-checking against Nasdaq + CNBC exposes exactly that class
of quiet error, which a single source can never self-report.

- quotes.ts — Yahoo + Nasdaq + CNBC fetchers (all keyless, browser UA;
  Yahoo prev-close now derived from the daily candle, not the buggy
  meta field), median consensus, confidence = corroborating sources / 3,
  outlier flagging, and a bun:sqlite hourly time-series cache.
- quote_ingest.ts + ops/systemd/lakehouse-quote-ingest.{service,timer} —
  hourly snapshot into the cache (real-time not required per product
  call). Runs as root to match the mcp-server that co-owns the db;
  busy_timeout serializes the two writers.
- /intelligence/ticker_quotes — serves cache-first (fast), aggregates
  cold tickers on-demand once and caches them. New /quote_ingest admin
  trigger.
- profiler.html — per-card confidence badge (green/amber/red) + a
  per-source breakdown tooltip; "3-source consensus" labeling.
- entity.ts — the earlier Stooq->Yahoo swap for the single-ticker brief
  (kept fetchStooqQuote dead-but-reversible).

Verified live on devop.live/lakehouse/profiler: liquid names read
100% / 3-source consensus; thin microcaps (BLPG, ESHSF — Yahoo-only)
honestly read 33% / 1-source. Full basket (17 tickers) seeded; hourly
timer armed. Cache db (data/_quote_cache/) is gitignored.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 01:28:28 -05:00
..