b8c072cf0b
First G1+ piece. Standalone vectord service with in-memory HNSW
indexes keyed by string IDs and optional opaque JSON metadata.
Wraps github.com/coder/hnsw v0.6.1 (pure Go, no cgo). New port
:3215 with /v1/vectors/* routed through gateway.
API:
POST /v1/vectors/index create
GET /v1/vectors/index list
GET /v1/vectors/index/{name} get info
DELETE /v1/vectors/index/{name}
POST /v1/vectors/index/{name}/add (batch)
POST /v1/vectors/index/{name}/search
Acceptance smoke 7/7 PASS — including recall=1 on inserted vector
w-042 (cosine distance 5.96e-8, float32 precision noise), 200-
vector batch round-trip, dim mismatch → 400, missing index → 404,
duplicate create → 409.
Two upstream library quirks worked around in the wrapper:
1. coder/hnsw.Add panics with "node not added" on re-adding an
existing key (length-invariant fires because internal
delete+re-add doesn't change Len). Pre-Delete fixes for n>1.
2. Delete of the LAST node leaves layers[0] non-empty but
entryless; next Add SIGSEGVs in Dims(). Workaround: when
re-adding to a 1-node graph, recreate the underlying graph
fresh via resetGraphLocked().
Cross-lineage scrum on shipped code:
- Opus 4.7 (opencode): 0 BLOCK + 4 WARN + 3 INFO
- Kimi K2-0905 (openrouter): 2 BLOCK + 2 WARN + 1 INFO
- Qwen3-coder (openrouter): "No BLOCKs" (4 tokens)
Fixed (4 real + 2 cleanup):
O-W1: Lookup returned the raw []float32 from coder/hnsw — caller
mutation would corrupt index. Now copies before return.
O-W3: NaN/Inf vectors poison HNSW (distance comparisons return
false for both < and >, breaking heap invariants). Zero-norm
under cosine produces NaN. Now validated at Add time.
K-B1: Re-adding with nil metadata silently cleared the existing
entry — JSON-omitted "metadata" field deserializes as nil,
making upsert non-idempotent. Now nil = "leave alone"; explicit
{} or Delete to clear.
O-W4: Batch Add with mid-batch failure left items 0..N-1
committed and item N rejected. Now pre-validates all IDs+dims
before any Add.
O-I1: jsonItoa hand-roll replaced with strconv.Itoa — no
measured allocation win.
O-I2: distanceFn re-resolved per Search → use stored i.g.Distance.
Dismissed (2 false positives):
K-B2 "MaxBytesReader applied after full read" — false, applied
BEFORE Decode in decodeJSON
K-W1 "Search distances under read lock might see invalidated
slices from concurrent Add" — false, RWMutex serializes
write-lock during Add against read-lock during Search
Deferred (3): HTTP server timeouts (consistent G0 punt),
Content-Type validation (internal service behind gateway), Lookup
dim assertion (in-memory state can't drift).
The K-B1 finding is worth pausing on: nil metadata on re-add is
the kind of API ergonomics bug only a code-reading reviewer
catches — smoke would never detect it because the smoke always
sends explicit metadata. Three lines changed in Add; the resulting
API matches what callers actually expect.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
86 lines
2.4 KiB
Go
86 lines
2.4 KiB
Go
// registry.go — multi-index manager. Mirrors the catalogd Registry
|
|
// shape: a thread-safe map[name]*Index with Create / Get / Delete.
|
|
// Per-index operations (Add, Search) go through each Index's own
|
|
// RWMutex so registry-wide locking only fires on lifecycle events.
|
|
package vectord
|
|
|
|
import (
|
|
"errors"
|
|
"fmt"
|
|
"sort"
|
|
"sync"
|
|
)
|
|
|
|
// ErrIndexNotFound is returned by Get / Delete when the requested
|
|
// name has no registered index.
|
|
var ErrIndexNotFound = errors.New("vectord: index not found")
|
|
|
|
// ErrIndexAlreadyExists is returned by Create when the name is
|
|
// taken. Callers can treat this as a 409 Conflict — paralleling
|
|
// catalogd's ADR-020 idempotency contract, but stricter (no
|
|
// "same params reuses index" semantics yet).
|
|
var ErrIndexAlreadyExists = errors.New("vectord: index already exists")
|
|
|
|
// Registry holds the live indexes by name.
|
|
type Registry struct {
|
|
mu sync.RWMutex
|
|
indexes map[string]*Index
|
|
}
|
|
|
|
func NewRegistry() *Registry {
|
|
return &Registry{indexes: make(map[string]*Index)}
|
|
}
|
|
|
|
// Create builds a new Index from params and registers it under
|
|
// params.Name. Returns ErrIndexAlreadyExists if the name is taken.
|
|
func (r *Registry) Create(p IndexParams) (*Index, error) {
|
|
idx, err := NewIndex(p)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
r.mu.Lock()
|
|
defer r.mu.Unlock()
|
|
if _, exists := r.indexes[p.Name]; exists {
|
|
return nil, fmt.Errorf("%w: %q", ErrIndexAlreadyExists, p.Name)
|
|
}
|
|
r.indexes[p.Name] = idx
|
|
return idx, nil
|
|
}
|
|
|
|
// Get returns the index for name, or ErrIndexNotFound.
|
|
func (r *Registry) Get(name string) (*Index, error) {
|
|
r.mu.RLock()
|
|
defer r.mu.RUnlock()
|
|
idx, ok := r.indexes[name]
|
|
if !ok {
|
|
return nil, fmt.Errorf("%w: %q", ErrIndexNotFound, name)
|
|
}
|
|
return idx, nil
|
|
}
|
|
|
|
// Delete removes the index for name. Returns ErrIndexNotFound if
|
|
// not present (so callers see explicit no-op vs success on the
|
|
// idempotent path).
|
|
func (r *Registry) Delete(name string) error {
|
|
r.mu.Lock()
|
|
defer r.mu.Unlock()
|
|
if _, ok := r.indexes[name]; !ok {
|
|
return fmt.Errorf("%w: %q", ErrIndexNotFound, name)
|
|
}
|
|
delete(r.indexes, name)
|
|
return nil
|
|
}
|
|
|
|
// Names returns the registered index names sorted ascending —
|
|
// stable enumeration for /v1/vectors GET listings.
|
|
func (r *Registry) Names() []string {
|
|
r.mu.RLock()
|
|
defer r.mu.RUnlock()
|
|
out := make([]string, 0, len(r.indexes))
|
|
for name := range r.indexes {
|
|
out = append(out, name)
|
|
}
|
|
sort.Strings(out)
|
|
return out
|
|
}
|