Two-tier PDF extraction: lopdf text layer first (fast, digital PDFs), Tesseract OCR fallback when text extraction yields zero pages (scanned documents, image-only PDFs). Falls back gracefully if Tesseract isn't installed — returns an actionable error directing the operator to `apt install tesseract-ocr tesseract-ocr-eng`. OCR path: extract embedded XObject /Image streams from each page via lopdf, detect format from magic bytes (JPEG/PNG/TIFF), write to temp file, shell out to tesseract with --oem 3 --psm 6 (LSTM + uniform text block), read output, clean up. Temp files cleaned even on error. Schema unchanged — both paths produce (source_file, page_number, text_content) so downstream consumers (chunker, vectord, queryd) work identically regardless of how text was produced. Verified: created a synthetic scanned PDF (PIL → image → PDF with no text layer), ingested via POST /ingest/file. Tesseract recovered the text with expected OCR artifacts. Queryable via DataFusion SQL. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Description
Rust-first object storage system
Languages
TypeScript
38.4%
Rust
35.8%
HTML
13.9%
Python
7.8%
Shell
2.1%
Other
2%