ac7c996596
Four findings deferred from the 2026-05-02 scrum, all 1-5 line fixes: W1 (kimi WARN @ scrum_master_pipeline.ts:1143) — `gemini-3-flash-preview` hardcoded twice in MAP and REDUCE phases. Extracted TREE_SPLIT_MODEL + TREE_SPLIT_PROVIDER constants near the existing config block. Diverging the two would break tree-split coherence (per-shard digests must come from the same model the reducer collapses). W2 (qwen WARN @ providers.toml:30) — stale `kimi-k2:1t` reference in operator-facing comments after PR #13 noted it's upstream-broken. Reframed as historical context ("was X here pre-2026-05-03 — that model is broken") so future operators don't paste-route from the comment. W3 (opus WARN @ vectord-lance/src/lib.rs:622) — temp_path() entropy was only pid+nanos, which collide under tokio scheduling when multiple tests in the same cargo process create temp dirs back-to-back. Added per-process AtomicU64 sequence counter — guarantees uniqueness regardless of clock. W4 (opus INFO @ scripts/lance_smoke.sh:38) — `|| echo '{}'` swallowed curl transport failures (gateway down, network broken, timeout), surfacing as misleading "no method field" jq errors at the next probe. Now captures $? separately, gates a "curl reachable" probe, and only falls back to empty body for the dependent jq parse. Smoke went 9 → 10 probes. Verified: vectord-lance 7/7 tests PASS, gateway cargo check clean, lance_smoke.sh 10/10 PASS against live gateway. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
105 lines
5.0 KiB
Bash
Executable File
105 lines
5.0 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# lance smoke — gates the 5 /vectors/lance/* HTTP routes (search, doc,
|
|
# index, append, migrate). Only the read paths are exercised here so a
|
|
# CI run doesn't mutate state. Migrate + index + append have shape
|
|
# probes (request bodies are well-formed) but ride the not-found path
|
|
# that the 2026-05-02 audit added.
|
|
#
|
|
# Targets the live gateway at $LH_GATEWAY (default :3100). Uses an
|
|
# existing on-disk Lance dataset — `workers_500k_v1` — so no
|
|
# migration setup is needed. If the dataset is missing the smoke
|
|
# fails loudly with a clear message.
|
|
#
|
|
# Surfaced 2026-05-02: the lance crates had zero tests + no smoke;
|
|
# substrate change to lance_backend.rs would silently break the live
|
|
# surface. This smoke is the regression gate.
|
|
#
|
|
# Usage:
|
|
# ./scripts/lance_smoke.sh
|
|
# LH_GATEWAY=http://127.0.0.1:3100 ./scripts/lance_smoke.sh
|
|
|
|
set -euo pipefail
|
|
|
|
GATEWAY="${LH_GATEWAY:-http://127.0.0.1:3100}"
|
|
DATASET="${LH_LANCE_DATASET:-workers_500k_v1}"
|
|
PREFIX="$GATEWAY/vectors/lance"
|
|
PASS=0; FAIL=0
|
|
PROBE() { local label="$1"; shift; "$@" && { echo " ✓ $label"; PASS=$((PASS+1)); } || { echo " ✗ $label"; FAIL=$((FAIL+1)); }; }
|
|
|
|
echo "[lance-smoke] gateway=$GATEWAY dataset=$DATASET"
|
|
|
|
# ── 0. Gateway alive ─────────────────────────────────────────────
|
|
PROBE "gateway /v1/health responds" \
|
|
bash -c "curl -sf -m 3 $GATEWAY/v1/health -o /dev/null"
|
|
|
|
# ── 1. Search returns IVF_PQ results on existing dataset ────────
|
|
# Capture curl status separately so a transport-level failure (gateway
|
|
# down, network broken, timeout) shows up as its own probe — instead of
|
|
# being swallowed by `|| echo '{}'` which would surface as the next jq
|
|
# probe failing with a misleading "no method field" message. Per opus
|
|
# INFO at lance_smoke.sh:38 from the 2026-05-02 scrum.
|
|
RESP=$(curl -sS -m 30 -X POST "$PREFIX/search/$DATASET" \
|
|
-H 'Content-Type: application/json' \
|
|
-d '{"query":"forklift operator","top_k":3}' 2>/dev/null)
|
|
CURL_RC=$?
|
|
PROBE "search/$DATASET curl reachable (exit 0)" \
|
|
test "$CURL_RC" = "0"
|
|
[ "$CURL_RC" != "0" ] && RESP='{}'
|
|
PROBE "search/$DATASET returns top-3 lance_ivf_pq results" \
|
|
bash -c "echo '$RESP' | jq -e '.method == \"lance_ivf_pq\" and (.results | length) == 3' >/dev/null"
|
|
|
|
# Capture one doc_id from those results so the next probe has something real to fetch.
|
|
DOC_ID=$(echo "$RESP" | jq -r '.results[0].doc_id // ""')
|
|
|
|
# ── 2. get_doc by id returns the row ────────────────────────────
|
|
PROBE "doc/$DATASET/<known-id> returns full row" \
|
|
bash -c "[ -n '$DOC_ID' ] && curl -sf -m 5 '$PREFIX/doc/$DATASET/$DOC_ID' | jq -e '.row.doc_id == \"$DOC_ID\"' >/dev/null"
|
|
|
|
# ── 3. get_doc with bogus id returns 404 (not 500) ──────────────
|
|
STATUS=$(curl -sS -m 5 -o /tmp/lance_smoke_404.json -w '%{http_code}' \
|
|
"$PREFIX/doc/$DATASET/W500K-NOT-A-REAL-ID-00000")
|
|
PROBE "doc/$DATASET/<missing-id> → 404" \
|
|
test "$STATUS" = "404"
|
|
|
|
# ── 4. search on missing dataset returns 404 + sanitized message ─
|
|
STATUS=$(curl -sS -m 5 -o /tmp/lance_smoke_500.json -w '%{http_code}' \
|
|
-X POST "$PREFIX/search/no-such-dataset-${RANDOM}" \
|
|
-H 'Content-Type: application/json' \
|
|
-d '{"query":"x","top_k":1}')
|
|
BODY=$(cat /tmp/lance_smoke_500.json)
|
|
PROBE "search/<missing> → 404 (was 500 pre-2026-05-02)" \
|
|
test "$STATUS" = "404"
|
|
# Assert "pattern absent" — `! grep -qE` (NOT `grep -qvE` which is unsound:
|
|
# -v -q exits 0 if ANY line lacks the pattern, so a multi-line body containing
|
|
# both a leak line AND any clean line would false-PASS. Caught 2026-05-02 by
|
|
# opus scrum on the lance backend wave.)
|
|
PROBE "search/<missing> body sanitized — no filesystem leak" \
|
|
bash -c "! echo '$BODY' | grep -qE '/home/|/root/\.cargo/|/var/|/tmp/'"
|
|
|
|
# ── 5. build_index on missing dataset also sanitized ────────────
|
|
STATUS=$(curl -sS -m 5 -o /tmp/lance_smoke_idx.json -w '%{http_code}' \
|
|
-X POST "$PREFIX/index/no-such-dataset-${RANDOM}" \
|
|
-H 'Content-Type: application/json' \
|
|
-d '{}')
|
|
BODY=$(cat /tmp/lance_smoke_idx.json)
|
|
PROBE "index/<missing> body sanitized" \
|
|
bash -c "! echo '$BODY' | grep -qE '/home/|/root/\.cargo/|/var/|/tmp/'"
|
|
|
|
# ── 6. append validates input shape (rejects empty rows array) ──
|
|
STATUS=$(curl -sS -m 5 -o /dev/null -w '%{http_code}' \
|
|
-X POST "$PREFIX/append/$DATASET" \
|
|
-H 'Content-Type: application/json' \
|
|
-d '{"rows":[]}')
|
|
PROBE "append with empty rows[] → 400" \
|
|
test "$STATUS" = "400"
|
|
|
|
# ── 7. migrate route is reachable (POST without body returns a real error, not 404) ──
|
|
STATUS=$(curl -sS -m 5 -o /dev/null -w '%{http_code}' \
|
|
-X POST "$PREFIX/migrate/probe-not-real-${RANDOM}?bucket=primary" 2>/dev/null)
|
|
# Should be 4xx (bad request shape), NOT 404 (route registered) and NOT 200.
|
|
PROBE "migrate route registered (non-404, non-200 on empty body)" \
|
|
bash -c "[ '$STATUS' != '404' ] && [ '$STATUS' != '200' ]"
|
|
|
|
echo "[lance-smoke] $PASS PASS / $FAIL FAIL"
|
|
[ "$FAIL" -eq 0 ]
|