9e9e4c26a4
Phase G0 Day 5 ships queryd: in-memory DuckDB with custom Connector
that runs INSTALL httpfs / LOAD httpfs / CREATE OR REPLACE SECRET
(TYPE S3) on every new connection, sourced from SecretsProvider +
shared.S3Config. SetMaxOpenConns(1) so registrar's CREATE VIEWs and
handler's SELECTs serialize through one connection (avoids cross-
connection MVCC visibility edge cases).
Registrar.Refresh reads catalogd /catalog/list, runs CREATE OR
REPLACE VIEW "name" AS SELECT * FROM read_parquet('s3://bucket/key')
per manifest, drops views for removed manifests, skips on unchanged
updated_at (the implicit etag). Drop pass runs BEFORE create pass so
a poison manifest can't block other manifest refreshes (post-scrum
C1 fix).
POST /sql with JSON body {"sql":"…"} returns
{"columns":[{"name":"id","type":"BIGINT"},…], "rows":[[…]],
"row_count":N}. []byte → string conversion so VARCHAR rows
JSON-encode as text. 30s default refresh ticker, configurable via
[queryd].refresh_every.
Cross-lineage scrum on shipped code:
- Opus 4.7 (opencode): 1 BLOCK + 4 WARN + 4 INFO
- Kimi K2-0905 (openrouter): 2 BLOCK + 2 WARN + 1 INFO
- Qwen3-coder (openrouter): 2 BLOCK + 1 WARN + 1 INFO
Fixed (4):
C1 (Opus + Kimi convergent): Refresh aborts on first per-view error
→ drop pass first, collect errors, errors.Join. Poison manifest
no longer blocks the rest of the catalog from re-syncing.
B-CTX (Opus BLOCK): bootstrap closure captured OpenDB's ctx →
cancelled-ctx silently fails every reconnect. context.Background()
inside closure; passed ctx only for initial Ping.
B-LEAK (Kimi BLOCK): firstLine(stmt) truncated CREATE SECRET to 80
chars but those 80 chars contained KEY_ID + SECRET prefix → log
aggregator captures credentials. Stable per-statement labels +
redactCreds() filter on wrapped DuckDB errors.
JSON-ERR (Opus WARN): swallowed json.Encode error → silent
truncated 200 on unsupported column types. slog.Warn the failure.
Dismissed (4 false positives):
Qwen BLOCK "bootstrap not transactional" — DuckDB DDL is auto-commit
Qwen BLOCK "MaxBytesReader after Decode" — false, applied before
Kimi BLOCK "concurrent Refresh + user SELECT deadlock" — not a
deadlock, just serialization, by design with 10s timeout retry
Kimi WARN "dropView leaves r.known inconsistent" — current code
returns before the delete; the entry persists for retry
Critical reviewer behavior: 1 convergent BLOCK between Opus + Kimi
on the per-view error blocking, plus two independent single-reviewer
BLOCKs (B-CTX, B-LEAK) that smoke could never have caught. The
B-LEAK fix uses defense-in-depth: never pass SQL into the error
path AND redact known cred values from DuckDB's own error message.
DuckDB cgo path: github.com/duckdb/duckdb-go/v2 v2.10502.0 (per
ADR-001 §1) on Go 1.25 + arrow-go. Smoke 6/6 PASS after every
fix round.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
58 lines
2.8 KiB
Modula-2
58 lines
2.8 KiB
Modula-2
module git.agentview.dev/profit/golangLAKEHOUSE
|
|
|
|
go 1.25.0
|
|
|
|
require (
|
|
github.com/apache/arrow-go/v18 v18.6.0
|
|
github.com/aws/aws-sdk-go-v2 v1.41.6
|
|
github.com/aws/aws-sdk-go-v2/config v1.32.16
|
|
github.com/aws/aws-sdk-go-v2/credentials v1.19.15
|
|
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.22.16
|
|
github.com/aws/aws-sdk-go-v2/service/s3 v1.100.0
|
|
github.com/aws/smithy-go v1.25.0
|
|
github.com/go-chi/chi/v5 v5.2.5
|
|
github.com/google/uuid v1.6.0
|
|
github.com/pelletier/go-toml/v2 v2.3.0
|
|
)
|
|
|
|
require (
|
|
github.com/andybalholm/brotli v1.2.1 // indirect
|
|
github.com/apache/thrift v0.22.0 // indirect
|
|
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.9 // indirect
|
|
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.22 // indirect
|
|
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.22 // indirect
|
|
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.22 // indirect
|
|
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.23 // indirect
|
|
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.8 // indirect
|
|
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.14 // indirect
|
|
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.22 // indirect
|
|
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.22 // indirect
|
|
github.com/aws/aws-sdk-go-v2/service/signin v1.0.10 // indirect
|
|
github.com/aws/aws-sdk-go-v2/service/sso v1.30.16 // indirect
|
|
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.20 // indirect
|
|
github.com/aws/aws-sdk-go-v2/service/sts v1.42.0 // indirect
|
|
github.com/cespare/xxhash/v2 v2.3.0 // indirect
|
|
github.com/duckdb/duckdb-go-bindings v0.10502.0 // indirect
|
|
github.com/duckdb/duckdb-go-bindings/lib/darwin-amd64 v0.10502.0 // indirect
|
|
github.com/duckdb/duckdb-go-bindings/lib/darwin-arm64 v0.10502.0 // indirect
|
|
github.com/duckdb/duckdb-go-bindings/lib/linux-amd64 v0.10502.0 // indirect
|
|
github.com/duckdb/duckdb-go-bindings/lib/linux-arm64 v0.10502.0 // indirect
|
|
github.com/duckdb/duckdb-go-bindings/lib/windows-amd64 v0.10502.0 // indirect
|
|
github.com/duckdb/duckdb-go/v2 v2.10502.0 // indirect
|
|
github.com/go-viper/mapstructure/v2 v2.5.0 // indirect
|
|
github.com/goccy/go-json v0.10.6 // indirect
|
|
github.com/google/flatbuffers v25.12.19+incompatible // indirect
|
|
github.com/klauspost/compress v1.18.5 // indirect
|
|
github.com/klauspost/cpuid/v2 v2.3.0 // indirect
|
|
github.com/pierrec/lz4/v4 v4.1.26 // indirect
|
|
github.com/zeebo/xxh3 v1.1.0 // indirect
|
|
golang.org/x/exp v0.0.0-20260112195511-716be5621a96 // indirect
|
|
golang.org/x/net v0.52.0 // indirect
|
|
golang.org/x/sync v0.20.0 // indirect
|
|
golang.org/x/sys v0.43.0 // indirect
|
|
golang.org/x/text v0.35.0 // indirect
|
|
google.golang.org/genproto/googleapis/rpc v0.0.0-20260120221211-b8f7ae30c516 // indirect
|
|
google.golang.org/grpc v1.80.0 // indirect
|
|
google.golang.org/protobuf v1.36.11 // indirect
|
|
)
|