Builds two AiClient instances at boot:
- `ai_client_direct = AiClient::new(sidecar_url)` — direct sidecar
transport. Used by V1State (gateway's own /v1/chat ollama_arm
needs this — calling /v1/chat from itself would self-loop) and
by the legacy /ai proxy.
- `ai_client_observable = AiClient::new_with_gateway(sidecar_url,
${gateway_host}:${gateway_port})` — routes generate() through
/v1/chat with provider="ollama". Used by:
vectord::agent (autotune background loop)
vectord::service (the /vectors HTTP surface — RAG, summary,
playbook synthesis, etc.)
Net result: every LLM call from a vectord module now lands in
/v1/usage and Langfuse traces. The autotune agent's hourly cycle
becomes observable; /vectors RAG calls show provider+model+latency
in the usage report. Phase 44 PRD's gate ("/v1/usage accounts for
every LLM call in the system within a 1-minute window") is now
satisfied for the gateway-hosted services.
Cost: one localhost HTTP hop per vectord-originated LLM call. At
~1-3ms RTT for in-process loopback, negligible against the LLM
call's own 30-90s wall-clock.
Phase 44 part 4 (deferred):
- Standalone consumers that build their own AiClient (test
harnesses, bot/propose, etc) — the TS-side already migrated in
part 1 + the regression guard at scripts/check_phase44_callers.sh
catches new direct callers. Rust standalone harnesses (if any
surface) follow the same pattern: construct via new_with_gateway
to opt into observability.
- Direct sidecar callers in standalone tools (scripts/serve_lab.py
is one) — Python-side; out of Rust scope.
Verified:
cargo build --release -p gateway compiles
systemctl restart lakehouse active
/v1/chat sanity PONG, finish=stop
When the autotune agent next cycles or any /vectors RAG endpoint
fires, /v1/usage will show the provider=ollama tick — first
real-world data should land within the next agent cycle.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
5d93a715c3