Adds cmd/fake_ollama, a minimal Ollama-API-compatible fake that
implements just enough surface for embedd to drive end-to-end
without a real Ollama install:
GET /api/tags — fixed model list including nomic-embed-text
POST /api/embeddings — deterministic dim-D vector from sha256(prompt)
GET /health — for the smoke's poll_health helper
Same prompt → bit-identical vector across runs, machines, and CI
nodes. Vectors are NOT semantically meaningful; the fake validates
the embed CONTRACT (dimension echo, response shape, status codes,
deterministic round-trip), not real semantic ranking. Real ranking
still requires real Ollama and lives in scripts/g2_smoke.sh + the
integration tier of the proof harness.
scripts/g2_smoke_fixtures.sh — full chain smoke against the fake:
- Build fake_ollama + embedd + vectord + gateway
- Start fake on :11435 (distinct from real Ollama at :11434)
- Generate temp lakehouse.toml with provider_url override
- Boot embedd/vectord/gateway with --config <override>
- 4 assertions: dim=768, deterministic same-text, different-text
divergence, bad-model → 4xx/5xx (fake 404 → embedd 502)
- Trap-cleanup tears down all 4 binaries + tmp config
Wired into the task runner:
just smoke-g2-fixtures
Closes R-006 partially:
- Embed half: ✓ — CI / fresh-clone reviewers without Ollama can
now run the embed contract smoke
- Storage half: deferred — mocking S3 protocol is non-trivial
(multipart, signed URLs, etc.) and MinIO itself is lightweight
enough to install via Docker in any CI environment. Documented
as Sprint 0 follow-up if a CI system without Docker shows up.
What this DOESN'T cover:
- Real semantic similarity (use scripts/g2_smoke.sh + real Ollama)
- Real Ollama API quirks (timeouts, version-specific shapes,
/api/embed batch endpoint that newer versions support)
Verified:
bash scripts/g2_smoke_fixtures.sh — 4/4 assertions PASS, ~3s wall
just verify — vet + test + 9 smokes still green
Doesn't replace the existing g2_smoke.sh (which still requires real
Ollama and exercises the actual embed semantics). Adds an alternate
mode for portability.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
147 lines
5.1 KiB
Bash
Executable File
147 lines
5.1 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# G2 smoke — fixtures variant. Same shape as g2_smoke.sh but points
|
|
# embedd at the Go fake Ollama (cmd/fake_ollama) instead of a real
|
|
# Ollama install. Useful for CI / fresh-clone reviewers who don't
|
|
# have Ollama set up.
|
|
#
|
|
# Validates the embed contract end-to-end:
|
|
# - POST /v1/embed → 200, dim=768
|
|
# - Same text twice → byte-identical vector (fake is deterministic)
|
|
# - Different texts → different vectors
|
|
# - Bad model → 4xx (fake rejects unknown models with 404 → embedd
|
|
# maps to 502)
|
|
#
|
|
# What this DOESN'T cover:
|
|
# - Real semantic similarity (fake vectors are sha256-derived; not
|
|
# semantically meaningful)
|
|
# - Real Ollama API quirks (timeouts, version-specific shapes)
|
|
#
|
|
# Closes R-006 partial: embedd no longer needs real Ollama for the
|
|
# CI / fresh-clone path. MinIO mocking is a separate Sprint 0
|
|
# follow-up.
|
|
#
|
|
# Usage: ./scripts/g2_smoke_fixtures.sh
|
|
|
|
set -euo pipefail
|
|
cd "$(dirname "$0")/.."
|
|
|
|
export PATH="$PATH:/usr/local/go/bin"
|
|
|
|
FAKE_PORT=11435 # distinct from real Ollama at 11434
|
|
EMBEDD_PORT=3216
|
|
GATEWAY_PORT=3110
|
|
VECTORD_PORT=3215
|
|
|
|
echo "[g2-fixtures] building fake_ollama + embedd + vectord + gateway..."
|
|
go build -o bin/ ./cmd/fake_ollama ./cmd/embedd ./cmd/vectord ./cmd/gateway
|
|
|
|
pkill -f "bin/fake_ollama" 2>/dev/null || true
|
|
pkill -f "bin/(embedd|vectord|gateway)" 2>/dev/null || true
|
|
sleep 0.3
|
|
|
|
PIDS=()
|
|
TMP="$(mktemp -d)"
|
|
cleanup() {
|
|
echo "[g2-fixtures] cleanup"
|
|
for p in "${PIDS[@]}"; do [ -n "$p" ] && kill "$p" 2>/dev/null || true; done
|
|
rm -rf "$TMP"
|
|
}
|
|
trap cleanup EXIT INT TERM
|
|
|
|
poll_health() {
|
|
local port="$1" deadline=$(($(date +%s) + 5))
|
|
while [ "$(date +%s)" -lt "$deadline" ]; do
|
|
if curl -sS --max-time 1 "http://127.0.0.1:$port/health" >/dev/null 2>&1; then
|
|
return 0
|
|
fi
|
|
sleep 0.05
|
|
done
|
|
return 1
|
|
}
|
|
|
|
# 1. Start fake_ollama on port 11435
|
|
echo "[g2-fixtures] launching fake_ollama on :${FAKE_PORT}..."
|
|
./bin/fake_ollama --bind "127.0.0.1:${FAKE_PORT}" --dim 768 \
|
|
> "$TMP/fake_ollama.log" 2>&1 &
|
|
PIDS+=($!)
|
|
poll_health "$FAKE_PORT" || { echo "fake_ollama failed"; cat "$TMP/fake_ollama.log"; exit 1; }
|
|
|
|
# 2. Write override config pointing embedd at fake_ollama
|
|
CFG="$TMP/lakehouse_fixtures.toml"
|
|
sed "s|provider_url *= *\".*\"|provider_url = \"http://127.0.0.1:${FAKE_PORT}\"|" \
|
|
lakehouse.toml > "$CFG"
|
|
|
|
# 3. Start embedd, vectord, gateway with the override config
|
|
echo "[g2-fixtures] launching embedd/vectord/gateway with fixture config..."
|
|
for SPEC in "vectord:${VECTORD_PORT}" "embedd:${EMBEDD_PORT}" "gateway:${GATEWAY_PORT}"; do
|
|
NAME="${SPEC%:*}"; PORT="${SPEC#*:}"
|
|
./bin/"$NAME" --config "$CFG" > "$TMP/${NAME}.log" 2>&1 &
|
|
PIDS+=($!)
|
|
if ! poll_health "$PORT"; then
|
|
echo "[g2-fixtures] $NAME failed to bind on :$PORT"
|
|
tail -10 "$TMP/${NAME}.log"
|
|
exit 1
|
|
fi
|
|
done
|
|
|
|
# 4. Run the assertions
|
|
FAILED=0
|
|
|
|
echo "[g2-fixtures] /v1/embed with one text → 200 + dim=768"
|
|
RESP=$(curl -sS -X POST "http://127.0.0.1:${GATEWAY_PORT}/v1/embed" \
|
|
-H 'Content-Type: application/json' \
|
|
-d '{"texts":["hello world"]}')
|
|
DIM=$(echo "$RESP" | jq -r '.dimension // empty')
|
|
N=$(echo "$RESP" | jq -r '.vectors | length')
|
|
MODEL=$(echo "$RESP" | jq -r '.model // empty')
|
|
if [ "$DIM" = "768" ] && [ "$N" = "1" ] && [ "$MODEL" = "nomic-embed-text" ]; then
|
|
echo " ✓ dim=768, model=nomic-embed-text"
|
|
else
|
|
echo " ✗ dim=$DIM n=$N model=$MODEL"; FAILED=1
|
|
fi
|
|
|
|
echo "[g2-fixtures] same text twice → byte-identical vector (deterministic)"
|
|
V1=$(curl -sS -X POST "http://127.0.0.1:${GATEWAY_PORT}/v1/embed" \
|
|
-H 'Content-Type: application/json' \
|
|
-d '{"texts":["test"]}' | jq -c '.vectors[0]')
|
|
V2=$(curl -sS -X POST "http://127.0.0.1:${GATEWAY_PORT}/v1/embed" \
|
|
-H 'Content-Type: application/json' \
|
|
-d '{"texts":["test"]}' | jq -c '.vectors[0]')
|
|
if [ "$V1" = "$V2" ]; then
|
|
echo " ✓ deterministic"
|
|
else
|
|
echo " ✗ same input → different vectors (fake should be deterministic)"; FAILED=1
|
|
fi
|
|
|
|
echo "[g2-fixtures] different texts → different vectors"
|
|
VA=$(curl -sS -X POST "http://127.0.0.1:${GATEWAY_PORT}/v1/embed" \
|
|
-H 'Content-Type: application/json' \
|
|
-d '{"texts":["alpha"]}' | jq -c '.vectors[0]')
|
|
VB=$(curl -sS -X POST "http://127.0.0.1:${GATEWAY_PORT}/v1/embed" \
|
|
-H 'Content-Type: application/json' \
|
|
-d '{"texts":["beta"]}' | jq -c '.vectors[0]')
|
|
if [ "$VA" != "$VB" ]; then
|
|
echo " ✓ different texts diverge"
|
|
else
|
|
echo " ✗ different texts produced identical vectors"; FAILED=1
|
|
fi
|
|
|
|
echo "[g2-fixtures] bad model → 4xx/5xx (fake returns 404, embedd maps to 502)"
|
|
HTTP=$(curl -s -o /dev/null -w "%{http_code}" \
|
|
-X POST "http://127.0.0.1:${GATEWAY_PORT}/v1/embed" \
|
|
-H 'Content-Type: application/json' \
|
|
-d '{"texts":["x"],"model":"definitely-not-loaded"}')
|
|
if [ "$HTTP" -ge 400 ] && [ "$HTTP" -lt 600 ]; then
|
|
echo " ✓ unknown model → $HTTP"
|
|
else
|
|
echo " ✗ unknown model → $HTTP"; FAILED=1
|
|
fi
|
|
|
|
if [ "$FAILED" = "0" ]; then
|
|
echo "[g2-fixtures] ✓ G2 fixture-mode acceptance: PASSED"
|
|
exit 0
|
|
else
|
|
echo "[g2-fixtures] ✗ G2 fixture-mode acceptance: FAILED"
|
|
exit 1
|
|
fi
|