# ADR-0015: llama-swap provider

**Status:** Accepted — 2026-06-27

## Context

llama-swap (https://github.com/mostlygeek/llama-swap) is an on-demand
model-swapping proxy in front of llama.cpp (and stable-diffusion.cpp) servers:
it extracts the `model` from each request, loads/hot-swaps the matching
upstream, and serves it. It is what foreman reached for, but more robust
(groups, TTL unload, health checks, a management API). We want it as a
first-class majordomo target — `llama-swap://token@host:port` in the DSN — and
the user explicitly asked for a *tailored* provider, not a bare alias of the
OpenAI client.

The tension: llama-swap's **chat** API is byte-for-byte OpenAI Chat
Completions. A new hand-rolled chat wire client would duplicate
`provider/openai` for zero behavioral gain, which ADR-0007 forbids. But the
"more robust" surface (model discovery, running list, unload) does not fit the
canonical `llm.Provider`/`llm.Model` interface (anti-creep: no provider-specific
features leak into the canonical API).

## Decision

- A dedicated `provider/llamaswap` package, but its chat path **delegates to
  `provider/openai`** pointed at `{baseURL}/v1` — no duplicated wire client.
  `Provider.Model` returns `openai.New(...).Model(id)`.
- Chat construction specifics: `WithLegacyMaxTokens()` (llama.cpp's OpenAI shim
  honors `max_tokens`, not `max_completion_tokens`); a placeholder `Bearer
  no-key` when no token is set (the openai client treats a blank key as a
  synthetic 401, but a local keyless llama-swap ignores a bearer it didn't ask
  for); the injected HTTP client carries **no timeout** — a cold model swap
  blocks up to llama-swap's `healthCheckTimeout` (≥15s), so callers bound work
  with a context deadline, never a client timeout.
- The "tailored" surface lives as **concrete methods** on `*llamaswap.Provider`,
  outside the canonical interface: `ListModels` (GET `/v1/models`), `Running`
  (GET `/running`, returned as raw JSON — its shape is not a stable contract),
  `Unload` (POST `/api/models/unload[/:model]`). A small `doJSON` helper shares
  bearer auth + error mapping; non-2xx → `*llm.APIError` (so `llm.Classify`
  applies), transport errors wrapped raw.
- DSN: two schemes share one factory. `llama-swap` builds an **http://** base
  URL from the host (llama-swap is local-first), deliberately *not* the DSN's
  https-always `BaseURL()`; `llama-swaps` builds **https://** for a TLS-fronted
  instance (mirrors redis/rediss). Why a second scheme rather than auto-detect:
  a DSN carries no reliable http-vs-https signal, so the choice stays explicit.
  Only `llama-swap` registers a no-DSN built-in provider (errors on use, mirrors
  foreman); `llama-swaps` is a scheme only.
- Image generation is implemented here too, against the new `imagegen`
  interface (see ADR-0016).

## Consequences

- No new dependency, no duplicated chat client; the chat path inherits every
  openai feature/fix automatically.
- Management methods are reachable only by holding the concrete
  `*llamaswap.Provider` (e.g. mort), not through `Parse`/`llm.Provider` — the
  correct boundary for non-canonical features.
- `Running`'s raw-JSON return is honest about llama-swap not publishing a stable
  schema; a typed shape can be added later without breaking callers that ignore
  it.