steve
cacce61ddc
- NEW musicgen leaf package: blocking Generate over ACE-Step's async job queue (release_task -> poll query_result -> fetch file, all via /upstream); tolerant envelope parsing, double-encoded result handled - NEW embeddings leaf package: EmbedModel + RerankModel as separate mints (two server instances on the host, llama.cpp #20085); InstructedQuery helper for Qwen3-style query/document asymmetry - provider/llamaswap: /v1/embeddings + /v1/rerank clients with strict validation (index-ordered vectors, count mismatch and out-of-range index are hard errors; rerank sorted descending, minimal parser) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1.9 KiB
1.9 KiB
ADR-0022: embeddings + rerank interface
Status: Accepted (2026-07-12)
Context
majordomo had no embedding or reranking surface at all. The llama-swap host
now runs two persistent CPU-only llama-server members (Qwen3-Embedding-0.6B
via /v1/embeddings, bge-reranker-v2-m3 via /v1/rerank), and mort wants a
reranking stage in memory retrieval with embedding-backed retrieval as a
later step.
Decision
- New
embeddingsleaf package with TWO half-surfaces, split like audio's Speech/Transcription:EmbedModel/EmbedProviderandRerankModel/RerankProvider. They are separate mints because on the reference host they are two DIFFERENT server instances — llama-server with--embeddingsand--reranktogether returns all-zero embeddings (llama.cpp #20085) — and because rerankers are cross-encoders, not embedders. EmbedResult.Vectors [][]float32in input order (provider must order by the response'sindex, never trust wire order). Nodimensionsparam: llama-server doesn't implement it; Matryoshka truncation is caller-side.InstructedQuery(task, query)helper encodes the instruction-aware asymmetry (queries wrapped, documents bare) so call sites can't silently degrade retrieval by forgetting the prefix.RerankResultsorted by descending score; parser reads ONLYresults[].indexandresults[].relevance_scorebecause llama-server documents the shape as subject to change. Scores are model-specific — comparable within one response only.
Consequences
- Callers get vectors/scores with strict validation (count mismatch, index out of range, empty vector are hard errors — a silently missing vector is a retrieval bug factory).
- llama-server's rerank scoring has open correctness issues for some models (llama.cpp #16407); consumers must validate against a fixture before trusting scores in production (mort gates its memory-rerank convar on exactly that).