lakehouse/docs/DECISIONS.md
root a52ca841c6 Phase 0: bootstrap Rust workspace
- Cargo workspace with 6 crates: shared, storaged, catalogd, queryd, aibridge, gateway
- shared: types (DatasetId, ObjectRef, SchemaFingerprint, DatasetManifest) + error enum
- gateway: Axum HTTP entrypoint with nested service routers + tracing
- All services expose /health stubs
- justfile with build/test/run recipes
- PRD, phase tracker, and ADR docs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 04:59:05 -05:00

1.5 KiB

Architecture Decision Records

ADR-001: Object storage as source of truth

Date: 2026-03-27 Decision: All data lives in S3-compatible object storage. No traditional database. Rationale: Eliminates DB operational overhead, enables infinite scale at storage tier, forces clean separation of data and metadata.

ADR-002: Catalog metadata persistence

Date: 2026-03-27 Decision: catalogd persists manifests as Parquet files in object storage. In-memory index rebuilt on startup. Rationale: No external DB dependency. Storage is already the source of truth. Write-ahead pattern ensures consistency.

ADR-003: Real models only (no mocks)

Date: 2026-03-27 Decision: AI sidecar hits Ollama with real models from Phase 3 onward. No stub/mock endpoints. Rationale: Local Ollama instance available with nomic-embed-text, qwen2.5, mistral, gemma2, llama3.2. Mocks hide integration bugs.

ADR-004: Python sidecar as Ollama adapter

Date: 2026-03-27 Decision: Python FastAPI sidecar is a thin HTTP adapter over Ollama's API. No model loading in Python. Rationale: Ollama handles model lifecycle, GPU scheduling, caching. Sidecar stays stateless and lightweight — no torch/transformers deps.

ADR-005: HTTP-first, gRPC later

Date: 2026-03-27 Decision: All inter-service communication uses HTTP through Phase 4. gRPC migration in Phase 5. Rationale: HTTP is simpler to debug, test, and iterate on. gRPC adds protobuf compilation and streaming complexity before APIs stabilize.