steve
cacce61ddc
- NEW musicgen leaf package: blocking Generate over ACE-Step's async job queue (release_task -> poll query_result -> fetch file, all via /upstream); tolerant envelope parsing, double-encoded result handled - NEW embeddings leaf package: EmbedModel + RerankModel as separate mints (two server instances on the host, llama.cpp #20085); InstructedQuery helper for Qwen3-style query/document asymmetry - provider/llamaswap: /v1/embeddings + /v1/rerank clients with strict validation (index-ordered vectors, count mismatch and out-of-range index are hard errors; rerank sorted descending, minimal parser) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
42 lines
1.9 KiB
Markdown
42 lines
1.9 KiB
Markdown
# ADR-0022: embeddings + rerank interface
|
|
|
|
Status: Accepted (2026-07-12)
|
|
|
|
## Context
|
|
|
|
majordomo had no embedding or reranking surface at all. The llama-swap host
|
|
now runs two persistent CPU-only llama-server members (Qwen3-Embedding-0.6B
|
|
via `/v1/embeddings`, bge-reranker-v2-m3 via `/v1/rerank`), and mort wants a
|
|
reranking stage in memory retrieval with embedding-backed retrieval as a
|
|
later step.
|
|
|
|
## Decision
|
|
|
|
- New `embeddings` leaf package with TWO half-surfaces, split like audio's
|
|
Speech/Transcription: `EmbedModel`/`EmbedProvider` and
|
|
`RerankModel`/`RerankProvider`. They are separate mints because on the
|
|
reference host they are two DIFFERENT server instances — llama-server with
|
|
`--embeddings` and `--rerank` together returns all-zero embeddings
|
|
(llama.cpp #20085) — and because rerankers are cross-encoders, not
|
|
embedders.
|
|
- `EmbedResult.Vectors [][]float32` in input order (provider must order by
|
|
the response's `index`, never trust wire order). No `dimensions` param:
|
|
llama-server doesn't implement it; Matryoshka truncation is caller-side.
|
|
- `InstructedQuery(task, query)` helper encodes the instruction-aware
|
|
asymmetry (queries wrapped, documents bare) so call sites can't silently
|
|
degrade retrieval by forgetting the prefix.
|
|
- `RerankResult` sorted by descending score; parser reads ONLY
|
|
`results[].index` and `results[].relevance_score` because llama-server
|
|
documents the shape as subject to change. Scores are model-specific —
|
|
comparable within one response only.
|
|
|
|
## Consequences
|
|
|
|
- Callers get vectors/scores with strict validation (count mismatch, index
|
|
out of range, empty vector are hard errors — a silently missing vector is
|
|
a retrieval bug factory).
|
|
- llama-server's rerank scoring has open correctness issues for some models
|
|
(llama.cpp #16407); consumers must validate against a fixture before
|
|
trusting scores in production (mort gates its memory-rerank convar on
|
|
exactly that).
|