1142f54f23
Phase G0 Day 1 executed end-to-end after a third-pass review by
qwen3-coder:480b consolidated all findings across Opus/Kimi/Qwen
lineages.
Cross-lineage review consolidation (3 model passes + 1 runtime pass):
- Opus 4.7: 9 findings · 7 fixed inline · 2 deferred
- Kimi K2.6: 2 BLOCKs (introduced by Opus fixes) · 2 fixed
- Qwen3-coder:480b: 2 WARNs · 1 fixed (D2.4 256 MiB cap + 4-slot
semaphore on PUTs) · 1 deferred (Q2 view refresh batching)
- Runtime smoke: 1 finding (port 3100 collision with live Rust
lakehouse) · fixed (Go dev ports shifted to 3110+)
- Total: 14 findings · 11 fixed · 3 deferred to G2
What landed in code:
- internal/shared/server.go — chi factory, slog JSON, /health,
graceful shutdown via signal.NotifyContext
- internal/shared/config.go — TOML loader, DefaultConfig, -config flag
- cmd/{gateway,storaged,catalogd,ingestd,queryd}/main.go — five
binaries, each ~30 lines using the shared factory
- lakehouse.toml — G0 dev defaults (3110-3214)
- scripts/d1_smoke.sh — repeatable smoke that exits 0 on PASS
- go.mod / go.sum — chi v5.2.5, pelletier/go-toml/v2 v2.3.0
Verified end-to-end via scripts/d1_smoke.sh:
- All 5 /health endpoints return 200 with correct service name
- Gateway /v1/ingest + /v1/sql stubs return 501 with X-Lakehouse-Stub
- Graceful shutdown logs cleanly on SIGTERM
- DuckDB cgo path verified separately (sql.Open("duckdb","") + ping)
D1 ACCEPTANCE GATE: PASSED.
Next: D2 — storaged S3 GET/PUT/LIST against MinIO.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
91 lines
2.8 KiB
Go
91 lines
2.8 KiB
Go
// Package shared also provides the TOML config loader. Per ADR
|
|
// equivalent of Rust ADR-006 (TOML config over env vars), every
|
|
// service reads `lakehouse.toml` with sane defaults and env
|
|
// overrides. Config is hot-reload-unaware in G0; reload-on-SIGHUP
|
|
// is a G1+ concern.
|
|
package shared
|
|
|
|
import (
|
|
"fmt"
|
|
"os"
|
|
|
|
"github.com/pelletier/go-toml/v2"
|
|
)
|
|
|
|
// Config is the unified Lakehouse config. Each service reads only
|
|
// the section it cares about, but they all share the same file so
|
|
// operators have one place to look.
|
|
type Config struct {
|
|
Gateway ServiceConfig `toml:"gateway"`
|
|
Storaged ServiceConfig `toml:"storaged"`
|
|
Catalogd ServiceConfig `toml:"catalogd"`
|
|
Ingestd ServiceConfig `toml:"ingestd"`
|
|
Queryd ServiceConfig `toml:"queryd"`
|
|
S3 S3Config `toml:"s3"`
|
|
Log LogConfig `toml:"log"`
|
|
}
|
|
|
|
// ServiceConfig is the per-binary bind config. Default Bind ""
|
|
// means "use the service's hardcoded G0 default" — see DefaultConfig.
|
|
type ServiceConfig struct {
|
|
Bind string `toml:"bind"`
|
|
}
|
|
|
|
// S3Config holds S3-compatible storage settings. Endpoint blank →
|
|
// AWS default. Bucket "" → "lakehouse-primary".
|
|
type S3Config struct {
|
|
Endpoint string `toml:"endpoint"`
|
|
Region string `toml:"region"`
|
|
Bucket string `toml:"bucket"`
|
|
AccessKeyID string `toml:"access_key_id"`
|
|
SecretAccessKey string `toml:"secret_access_key"`
|
|
UsePathStyle bool `toml:"use_path_style"`
|
|
}
|
|
|
|
// LogConfig — slog level for now; structured fields land G1+.
|
|
type LogConfig struct {
|
|
Level string `toml:"level"`
|
|
}
|
|
|
|
// DefaultConfig returns the G0 dev defaults. Ports are shifted to
|
|
// 3110+ to coexist with the live Rust lakehouse on 3100/3201-3204
|
|
// during the migration. G5 cutover flips gateway back to 3100.
|
|
func DefaultConfig() Config {
|
|
return Config{
|
|
Gateway: ServiceConfig{Bind: "127.0.0.1:3110"},
|
|
Storaged: ServiceConfig{Bind: "127.0.0.1:3211"},
|
|
Catalogd: ServiceConfig{Bind: "127.0.0.1:3212"},
|
|
Ingestd: ServiceConfig{Bind: "127.0.0.1:3213"},
|
|
Queryd: ServiceConfig{Bind: "127.0.0.1:3214"},
|
|
S3: S3Config{
|
|
Endpoint: "http://localhost:9000",
|
|
Region: "us-east-1",
|
|
Bucket: "lakehouse-primary",
|
|
UsePathStyle: true,
|
|
},
|
|
Log: LogConfig{Level: "info"},
|
|
}
|
|
}
|
|
|
|
// LoadConfig reads `lakehouse.toml` from path; if path is empty or
|
|
// the file doesn't exist, returns DefaultConfig. Any decode error is
|
|
// fatal (we don't want a misconfigured service silently falling back
|
|
// to defaults — that's the kind of bug you find at 2am).
|
|
func LoadConfig(path string) (Config, error) {
|
|
cfg := DefaultConfig()
|
|
if path == "" {
|
|
return cfg, nil
|
|
}
|
|
b, err := os.ReadFile(path)
|
|
if os.IsNotExist(err) {
|
|
return cfg, nil
|
|
}
|
|
if err != nil {
|
|
return cfg, fmt.Errorf("read config: %w", err)
|
|
}
|
|
if err := toml.Unmarshal(b, &cfg); err != nil {
|
|
return cfg, fmt.Errorf("parse config: %w", err)
|
|
}
|
|
return cfg, nil
|
|
}
|