50a8c8013f
- ui: Dioxus WASM app with dataset sidebar, SQL editor (Ctrl+Enter), results table - ui: dynamic API base URL (same-origin for nginx, port-based for local dev) - gateway: CORS enabled for cross-origin requests - nginx: lakehouse.devop.live proxies UI (:3300) + API (:3100) on same origin - justfile: ui-build, ui-serve, sidecar, up commands added Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
59 lines
2.9 KiB
Markdown
59 lines
2.9 KiB
Markdown
# Phase Tracker
|
|
|
|
## Phase 0: Bootstrap
|
|
- [x] 0.1 — Cargo workspace with all crate stubs compiling
|
|
- [x] 0.2 — `shared` crate: error types, ObjectRef, DatasetId
|
|
- [x] 0.3 — `gateway` with Axum: GET /health → 200
|
|
- [x] 0.4 — tracing + tracing-subscriber wired in gateway
|
|
- [x] 0.5 — justfile with build, test, run recipes
|
|
- [x] 0.6 — docs committed to git
|
|
|
|
**Gate: PASSED** — All crates compile. Gateway runs. Logs emit. Docs committed.
|
|
|
|
## Phase 1: Storage + Catalog
|
|
- [x] 1.1 — storaged: object_store backend init (LocalFileSystem)
|
|
- [x] 1.2 — storaged: Axum endpoints (PUT/GET/DELETE/LIST /objects/{key})
|
|
- [x] 1.3 — shared/arrow_helpers.rs: RecordBatch ↔ Parquet + schema fingerprinting
|
|
- [x] 1.4 — catalogd/registry.rs: in-memory index + manifest persistence to object storage
|
|
- [x] 1.5 — catalogd/schema.rs: schema fingerprinting (merged into shared/arrow_helpers.rs)
|
|
- [x] 1.6 — catalogd service: POST/GET /datasets + GET /datasets/by-name/{name}
|
|
- [x] 1.7 — gateway routes to storaged + catalogd with shared state
|
|
|
|
**Gate: PASSED** — PUT object → register dataset → list → get by name. All via gateway HTTP.
|
|
|
|
## Phase 2: Query Engine
|
|
- [x] 2.1 — queryd: SessionContext + object_store config (custom scheme to avoid path doubling)
|
|
- [x] 2.2 — queryd: ListingTable from catalog ObjectRefs with schema inference
|
|
- [x] 2.3 — queryd service: POST /query/sql → JSON (columns + rows + row_count)
|
|
- [x] 2.4 — queryd → catalogd wiring (reads dataset list, registers as tables)
|
|
- [x] 2.5 — gateway routes /query with QueryEngine state
|
|
|
|
**Gate: PASSED** — SELECT *, WHERE/ORDER BY, COUNT/AVG all return correct results via catalog.
|
|
|
|
## Phase 3: AI Integration
|
|
- [x] 3.1 — Python sidecar: FastAPI + Ollama (embed/generate/rerank) — real models, no mocks
|
|
- [x] 3.2 — Dockerfile for sidecar
|
|
- [x] 3.3 — aibridge/client.rs: reqwest HTTP client with 120s timeout
|
|
- [x] 3.4 — aibridge service: Axum proxy endpoints (POST /ai/embed, /ai/generate, /ai/rerank)
|
|
- [x] 3.5 — Model config via env vars (EMBED_MODEL, GEN_MODEL, RERANK_MODEL, SIDECAR_URL)
|
|
|
|
**Gate: PASSED** — Gateway → aibridge → sidecar → Ollama → real 768d embeddings + generation.
|
|
|
|
## Phase 4: Frontend
|
|
- [x] 4.1 — Dioxus scaffold, WASM build (dx build --platform web)
|
|
- [x] 4.2 — Dataset browser (sidebar, click to select, refresh)
|
|
- [x] 4.3 — Query editor + results table (Ctrl+Enter to run, column types, row count)
|
|
- [x] 4.4 — Error display + loading states
|
|
- [x] 4.5 — Nginx proxy (lakehouse.devop.live), same-origin API detection
|
|
|
|
**Gate: PASSED** — Browse datasets and query from browser at lakehouse.devop.live.
|
|
|
|
## Phase 5: Hardening
|
|
- [ ] 5.1 — Proto definitions
|
|
- [ ] 5.2 — Internal gRPC migration
|
|
- [ ] 5.3 — OpenTelemetry tracing
|
|
- [ ] 5.4 — Auth middleware
|
|
- [ ] 5.5 — Config-driven startup
|
|
|
|
**Gate:** gRPC internals, traces, auth, restartable from repo + config.
|