lakehouse/data/_catalog/manifests/03b65605-7cce-4a49-b338-4f19b0ff2ed5.json
root 6a532cb248 Background job system for embedding — fixes 100K timeout
- JobTracker: create/update/complete/fail jobs with progress tracking
- POST /vectors/index now returns immediately with job_id (HTTP 202)
- Embedding runs in tokio::spawn background task
- GET /vectors/jobs/{id} returns live progress (chunks embedded, rate, ETA)
- GET /vectors/jobs lists all jobs
- Progress logged every 100 batches with chunks/sec and ETA
- 100K embedding job running successfully at 44 chunks/sec
- System stays responsive during embedding (queries in 23ms)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 09:03:07 -05:00

15 lines
385 B
JSON

{
"id": "03b65605-7cce-4a49-b338-4f19b0ff2ed5",
"name": "call_log",
"schema_fingerprint": "auto",
"objects": [
{
"bucket": "data",
"key": "datasets/call_log.parquet",
"size_bytes": 35951077,
"created_at": "2026-03-27T14:00:44.377704982Z"
}
],
"created_at": "2026-03-27T14:00:44.377712082Z",
"updated_at": "2026-03-27T14:00:44.377712082Z"
}