13 Commits
Author SHA1 Message Date
steveandClaude Opus 5 85561ab477 chore: ignore .claude/ (agent worktrees)
CI / Tidy (push) Successful in 9m23s
CI / Build & Test (push) Successful in 9m48s
A `git add -A` in a checkout with a worktree under .claude/worktrees/ records
it as a gitlink — a submodule pointer no clone can resolve. That happened in
gadfly today; this is the same one-line prevention, applied before it happens
here.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-12 18:58:35 -04:00
steve f837115a55 Merge pull request 'feat(qwen): Alibaba Qwen built-in over Model Studio's OpenAI-compatible mode' (#27)
CI / Tidy (push) Successful in 9m24s
CI / Build & Test (push) Successful in 11m10s
2026-08-12 21:03:34 +00:00
steveandClaude Opus 5 f8ced9c629 docs(progress): describe the shape this PR actually landed in
CI / Tidy (pull_request) Successful in 9m22s
CI / Build & Test (pull_request) Successful in 10m28s
The progress entry was written before four review rounds reshaped the change:
it credited openaiCompatScheme alone, listed the tests as six per-provider
cases, and mentioned a captureRT detail that has since moved. Rewritten to
match what merges — registerOpenAICompatBuiltin owning both halves,
envKeyForProvider as the single LLM_<NAME> definition, and the shared table
that every OpenAI-compat built-in is now checked against.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-12 16:57:33 -04:00
steveandClaude Opus 5 0760cf96d4 docs: gadfly round 4 — two doc-comment fixes
CI / Tidy (pull_request) Successful in 9m35s
CI / Build & Test (pull_request) Successful in 10m17s
DSN.Scheme's list named kimi but not qwen — the same sibling drift this PR
keeps finding, in a doc comment this time (I updated the README's scheme list
and not this one). Added qwen, and llama-swap/llama-swaps while there, since
they were missing too.

envKeyForProvider's example ran backwards: "LLM_M1 → m1" describes registry
naming, not what the function does. Now reads name → variable in one
direction.

Not taking the third: the const block's alignment is gofmt's own output
(gofmt -l is empty), and the uneven padding is forced by the doc comments that
split the block into alignment groups. glm-5.2 reached that same conclusion in
round 2 before flagging it here.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-12 16:53:05 -04:00
steveandClaude Opus 5 8670ed22be refactor: gadfly round 3 — one table owns the OpenAI-compat contract
CI / Tidy (pull_request) Successful in 9m21s
CI / Build & Test (pull_request) Successful in 10m23s
Three findings, and the first two are the same recurring shape.

envKeyForProvider (env.go) is now the single definition of the LLM_<NAME>
form. It lived in two places — lazy resolution in registry.go and the
missing-key hint in openaiCompatScheme — with a comment on the second asserting
it matched the first. A comment is not enforcement: if either had drifted, a
keyless DSN target would have named a variable that does nothing, and nothing
would have failed.

The kimi and qwen test files had become near-identical, which is round 1's
finding at the level above it: I deduped the fixtures, then left two parallel
suites asserting the same four things. They are now ONE table in
builtin_openaicompat_test.go — endpoint + credential, missing key fails closed
naming its own variable and never reaching the network, the name:// DSN
reaching another host, and a keyless DSN naming LLM_<NAME> instead of the
built-in's key. Adding an OpenAI-compat built-in is a table row that
immediately owes all four; builtin_kimi_test.go is deleted because the table
covers it. Only genuinely qwen-specific tests remain in the qwen file: the
reverse credential leak and the reasoning_effort wire claim ADR-0027 rests on.

Also trimmed ProviderQwen's doc comment, which restated the ADR-0027 rationale
already given at the registration site.

The break-check suite caught its own rot again — two mutations went stale when
these tests were renamed, and the landed-check reported them loudly instead of
passing them off as green. Now 9 cases, including one that drifts
envKeyForProvider to prove the shared helper is load-bearing. 9/9 apply and are
caught.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-12 16:40:53 -04:00
steveandClaude Opus 5 f1f2b653c3 refactor: gadfly round 2 — both halves of an OpenAI-compat built-in register together
CI / Tidy (pull_request) Successful in 9m25s
CI / Build & Test (pull_request) Successful in 9m50s
Same class of finding as round 1, one level in: I factored the DSN-scheme half
of the kimi/qwen duplication into openaiCompatScheme and left the eager
provider half copy-pasted, so a third built-in still had six lines to clone —
including both credential rules, which is exactly the pair you do not want
re-typed.

registerOpenAICompatBuiltin now installs both halves from one call. The rules
that matter hold by construction for every future caller: WithAPIKey passed
unconditionally (an unset key must not fall through to OPENAI_API_KEY), and
WithAPIKeyName naming that same variable in the 401 hint. Registering kimi and
qwen is now one line each.

Also fixed a cross-reference the ADR got wrong: Qwen's image-input caveat is
README matrix footnote ⁴, not ³ — ³ is kimi's. I wrote "³, shared with kimi"
in the ADR and then gave Qwen its own footnote in the README.

The break-check harness needed fixing before any of this could be trusted:
three of its mutations targeted lines this refactor moved, so they matched
nothing, the code was never broken, and the suite reported "test still passed"
— identical output to a test that genuinely misses the bug. Mutations are now
verified to have landed (sha before/after) and the suite fails loudly if one
doesn't. Two new cases cover the helper: dropping the unconditional WithAPIKey,
and dropping the scheme-half registration. 8/8 apply and are caught.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-12 16:25:38 -04:00
steveandClaude Opus 5 31d6b59356 refactor(test): gadfly round 1 — share the OpenAI-compat test fixtures
CI / Tidy (pull_request) Successful in 9m25s
CI / Build & Test (pull_request) Successful in 9m50s
Both findings were the same one, and both were fair: the PR that retires two
byte-identical DSN factories into openaiCompatScheme then copy-pasted the test
fixtures. qwenResponse was byte-identical to kimiResponse, and the single-key
env-lookup closure appeared three times in the new file (plus a fourth in the
kimi file, which neither reviewer was looking at).

Fixed for the class rather than for qwen: captureRT, the canned Chat
Completions body (now chatCompletionOK), and a new singleKeyEnv helper move to
builtin_openaicompat_test.go, owned by no single provider. The kimi tests adopt
them too, so the next OpenAI-compat built-in has nothing left to copy — the
same argument the production helper makes.

Also aligned the test model ids to the current Model Studio names
(qwen3.8-max / qwen3.7-plus), which the docs already cited. One reviewer called
those ids fictional and named the 2025 ones instead; they shipped 2026-08-03
and 2026-05-21 respectively, so that finding is stale model knowledge, not a
defect — but having tests and prose name the same models removes the smell that
prompted it. A dotted id also now proves it passes through verbatim.

Break-checked again after the refactor: all six mutations still fail their test.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-12 16:16:54 -04:00
steveandClaude Opus 5 02cd561eaf feat(qwen): Alibaba Qwen built-in over Model Studio's OpenAI-compatible mode
Gadfly review (reusable) / review (pull_request) Successful in 5m14s
Adversarial Review (Gadfly) / review (pull_request) Successful in 5m14s
CI / Tidy (pull_request) Successful in 9m24s
CI / Build & Test (pull_request) Successful in 9m53s
Adds the `qwen` built-in provider and the `qwen://` DSN scheme, keyed by
QWEN_API_KEY and defaulting to Model Studio's international host. Like kimi
(ADR-0026) it is `provider/openai` pointed elsewhere — no new client.

Model Studio serves the same models over two protocols, so the real decision
was which wire format to speak. ADR-0027 records why it is the OpenAI one:
down the anthropic client `ReasoningEffort` is ignored by design, structured
output rides the first-party `output_config.format` mechanism the shim does
not implement, and cached-token accounting reads Anthropic-only usage fields.
Each of those fails silently rather than loudly, which is what makes the
choice worth writing down. The shim stays reachable ad hoc via an
`anthropic://` DSN.

The kimi and qwen DSN factories were byte-identical, so they now share one
`openaiCompatScheme` helper: the "credential comes from the DSN token, and
the missing-key hint names LLM_<NAME>" rules hold by construction instead of
by copy.

Tests are hermetic and break-checked (all six fail on a deliberate mutation),
including the reverse credential leak — a visible QWEN_API_KEY must not
authenticate the openai built-in — and reasoning_effort asserted on the wire
body, which is the ADR's load-bearing claim.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-12 16:07:41 -04:00
steve e779169416 Merge pull request 'feat(videogen): LastImage — pin the trailing keyframe (first-last-frame-to-video)' (#26) from feat/videogen-last-frame into main
CI / Tidy (push) Successful in 9m23s
CI / Build & Test (push) Successful in 9m47s
2026-08-08 07:10:26 +00:00
steveandClaude Opus 5 5994d96921 refactor(llamaswap): drop initImageFilename — one caller left, and it was a rename of imageFilename
CI / Tidy (pull_request) Successful in 9m40s
CI / Build & Test (pull_request) Successful in 11m38s
The 2/4 finding is right: after writeImagePart started passing an explicit
filename stem, initImageFilename had no caller in video.go, and its
"conditioning frame" doc no longer described its one remaining user
(lipsync.go's avatar image). A one-line wrapper that survives only to be
misdescribed is not indirection worth keeping.

lipsync now calls imageFilename(mime, "frame") directly, and imageFilename's
doc lists the real bases — including WHY the video keyframes need distinct
ones: a backend that stages uploads by filename would otherwise have the
second overwrite the first.

Not taken: consolidating the first/last-frame rationale to a single canonical
site. The copies address different readers — the wire encoding (provider), the
contract's undetectable-support caveat (videogen), and the mode table (README)
— and last round's finding was a doc pointing at a note that did not exist.
Trading duplication for cross-references is what produced that.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01PLjgrxvHjm1sJgUu9zBPH9
2026-08-08 03:06:21 -04:00
steveandClaude Opus 5 588e092465 docs(videogen): gadfly — README FL2V section, and stop pointing at a note that does not exist
CI / Tidy (pull_request) Successful in 9m25s
CI / Build & Test (pull_request) Successful in 9m50s
- README documented only t2v/i2v. Now a table of the four keyframe
  combinations, plus the undetectable-support caveat, which is the one thing a
  caller cannot work out for itself.
- The LastImage doc comment said "see the note on LastImage support in
  provider/llamaswap" — there was no such note. A pointer to something that
  does not exist is worse than no pointer; the comment is now self-contained.
- Generate's doc described only input_reference; it now names
  input_reference_last and explains why an unsupporting backend returns a clip
  rather than an error.

The 2/4 finding (writeImagePart reusing the "frame" base for both parts) was
already fixed in dbc9689 — from the receiving end, where the consequence is
concrete rather than stylistic: ComfyUI stages uploads by FILENAME with
overwrite=true, so a shared name means the second clobbers the first and both
keyframes resolve to one image.

Not taken: initImageFilename's name is no longer misleading (writeImagePart
stopped calling it), and it is still used by lipsync.go so it is not dead.
The empty-LastImage test stays standalone — it mirrors the existing standalone
empty-InitImage coverage rather than a table this file does not have.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01PLjgrxvHjm1sJgUu9zBPH9
2026-08-08 02:59:12 -04:00
steveandClaude Opus 5 dbc96898ab fix(videogen): distinct FILENAMES for the two keyframes, not just distinct field names
CI / Tidy (pull_request) Successful in 9m25s
CI / Build & Test (pull_request) Successful in 10m28s
Caught while writing the receiving end. Distinct multipart field names are not
sufficient: backends stage an uploaded frame under a name derived from the
FILENAME, and our own ComfyUI shim posts to /upload/image with overwrite=true.
Both parts were sending initImageFilename(mime) — literally "frame.png" for
each — so the second upload would have clobbered the first and BOTH keyframe
inputs would have resolved to the same stored image.

The failure mode is the worst kind: a clip pinned at both ends to the same
frame renders cleanly, returns 200, and looks like the feature not working
rather than like a bug. Nothing upstream or downstream would report a fault.

writeImagePart now takes the filename stem (frame / frame_last), and the test
asserts the two arrive under different filenames.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01PLjgrxvHjm1sJgUu9zBPH9
2026-08-08 02:50:43 -04:00
steveandClaude Opus 5 44fcfbb273 feat(videogen): LastImage — pin the trailing keyframe (first-last-frame-to-video)
Gadfly review (reusable) / review (pull_request) Successful in 3m41s
Adversarial Review (Gadfly) / review (pull_request) Successful in 3m41s
CI / Tidy (pull_request) Successful in 9m26s
CI / Build & Test (pull_request) Successful in 9m48s
videogen.Request gains LastImage alongside InitImage, so one Request covers
t2v, i2v and FL2V without a mode flag. With InitImage it pins both ends of the
clip; alone it pins the destination and lets the backend invent the approach.

The llamaswap provider sends it as a SEPARATE `input_reference_last` part
rather than a second `input_reference`. Multipart permits repeated names, but
then which frame is first and which is last depends on part ORDER — an
ordering contract invisible in the payload, that nothing notices breaking. A
backend that does not know the new name ignores the part, the same degradation
as any other unknown field.

Both parts go through one writeImagePart helper so their encoding cannot
drift, and an empty LastImage is rejected up front exactly as InitImage
already is.

Support is per-model and deliberately NOT advertised in this contract: a
backend that ignores a trailing keyframe returns an ordinary clip, which is
indistinguishable from success. The doc comment says so, because a caller that
needs to know whether the pin took effect has to establish that out of band —
and the mort side gates on a convar for exactly this reason.

Motivated by mort's #1567 (long-form video): with both ends pinned, drift
becomes structurally bounded inside each shot instead of compounding across an
autoregressive chain.

Tests break-checked: sending the last frame under the shared name fails both
the distinct-name assertion and the last-alone case.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01PLjgrxvHjm1sJgUu9zBPH9
2026-08-08 02:47:39 -04:00
17 changed files with 794 additions and 225 deletions
+1
View File
@@ -7,6 +7,7 @@ OLLAMA_API_KEY=your-ollama-cloud-key-here
# Built-in provider keys (each optional; only needed for the providers you use). # Built-in provider keys (each optional; only needed for the providers you use).
#OPENAI_API_KEY=sk-... #OPENAI_API_KEY=sk-...
#KIMI_API_KEY=sk-... # Moonshot AI (Kimi); provider name "kimi" #KIMI_API_KEY=sk-... # Moonshot AI (Kimi); provider name "kimi"
#QWEN_API_KEY=sk-... # Alibaba Model Studio (Qwen); provider name "qwen"
#ANTHROPIC_API_KEY=sk-ant-... #ANTHROPIC_API_KEY=sk-ant-...
#GOOGLE_API_KEY=... #GOOGLE_API_KEY=...
+3
View File
@@ -28,3 +28,6 @@ go.work.sum
# macOS # macOS
.DS_Store .DS_Store
# Local worktrees created for agent work — never part of the repo.
.claude/
+41 -6
View File
@@ -122,6 +122,7 @@ Chains are health-tracked per target:
|----------|-----------|-------------|------------------| |----------|-----------|-------------|------------------|
| OpenAI (+compatible) | `openai` | `OPENAI_API_KEY` | https://api.openai.com/v1 | | OpenAI (+compatible) | `openai` | `OPENAI_API_KEY` | https://api.openai.com/v1 |
| Kimi (Moonshot AI) | `kimi` | `KIMI_API_KEY` | https://api.moonshot.ai/v1 | | Kimi (Moonshot AI) | `kimi` | `KIMI_API_KEY` | https://api.moonshot.ai/v1 |
| Qwen (Alibaba) | `qwen` | `QWEN_API_KEY` | https://dashscope-intl.aliyuncs.com/compatible-mode/v1 |
| Anthropic (+compatible) | `anthropic` | `ANTHROPIC_API_KEY` | https://api.anthropic.com | | Anthropic (+compatible) | `anthropic` | `ANTHROPIC_API_KEY` | https://api.anthropic.com |
| Google (Gemini) | `google` | `GOOGLE_API_KEY` / `GEMINI_API_KEY` | Gemini API (official SDK) | | Google (Gemini) | `google` | `GOOGLE_API_KEY` / `GEMINI_API_KEY` | Gemini API (official SDK) |
| Ollama Cloud | `ollama-cloud` | `OLLAMA_API_KEY` | https://ollama.com | | Ollama Cloud | `ollama-cloud` | `OLLAMA_API_KEY` | https://ollama.com |
@@ -134,6 +135,19 @@ the openai client (like llama-swap). The `kimi` built-in defaults to the
international endpoint; reach the China endpoint (or any other host) with a international endpoint; reach the China endpoint (or any other host) with a
`kimi://` DSN, e.g. `LLM_KCN=kimi://[email protected]/v1`. `kimi://` DSN, e.g. `LLM_KCN=kimi://[email protected]/v1`.
Qwen is the same shape: Alibaba Model Studio's OpenAI-compatible mode, reusing
the openai client. The `qwen` built-in defaults to the international
(Singapore) host; reach the China host or a workspace-scoped regional one with
a `qwen://` DSN, e.g.
`LLM_QCN=qwen://[email protected]/compatible-mode/v1`. Model Studio
also fronts the same models with an Anthropic-compatible `/v1/messages` shim —
majordomo does **not** use it, because on that surface `reasoning_effort` is
dropped, `Request.Schema` stops being enforced, and cached-token accounting
disappears; see [ADR-0027](docs/adr/0027-qwen-builtin.md). Two Alibaba-side
quirks are worth knowing: thinking is on by default for some models (e.g.
`qwen3.7-plus`), and the Qwen3 open-source models require streaming while
thinking, so buffered `Generate` calls want a Max/Plus model.
OpenAI-compatible / Anthropic-compatible endpoints: construct the provider OpenAI-compatible / Anthropic-compatible endpoints: construct the provider
with a name and base URL and register it — with a name and base URL and register it —
@@ -165,7 +179,7 @@ m, _ := reg.Parse("m5/qwen3:30b,m1/qwen3:30b,thinking")
``` ```
DSN format: `scheme://[token@]host[/path]`, scheme ∈ `foreman`, `ollama`, DSN format: `scheme://[token@]host[/path]`, scheme ∈ `foreman`, `ollama`,
`ollama-cloud`, `openai`, `kimi`, `anthropic`, `google`/`gemini`, `llama-swap`, `ollama-cloud`, `openai`, `kimi`, `qwen`, `anthropic`, `google`/`gemini`, `llama-swap`,
`llama-swaps`, or any scheme you add with `RegisterScheme`. The token is the `llama-swaps`, or any scheme you add with `RegisterScheme`. The token is the
credential (bearer token / API key); the base URL is always `https://host[/path]` credential (bearer token / API key); the base URL is always `https://host[/path]`
— except `llama-swap`, which builds `http://host[:port]` since it's local-first — except `llama-swap`, which builds `http://host[:port]` since it's local-first
@@ -272,17 +286,30 @@ tr, err := tm.Transcribe(ctx, audio.TranscriptionRequest{
voices, err := ls.ListVoices(ctx, "kokoro") // []string of voice ids voices, err := ls.ListVoices(ctx, "kokoro") // []string of voice ids
``` ```
## Video: text-to-video + image-to-video ## Video: text-to-video, image-to-video, first-last-frame
Video generation lives in the `videogen` package (ADR-0019), mirroring Video generation lives in the `videogen` package (ADR-0019), mirroring
imagegen/audio: one small `Model` contract, zero values mean backend imagegen/audio: one small `Model` contract, zero values mean backend
defaults, bytes in/out. Text-to-video and image-to-video are one surface defaults, bytes in/out. All modes are one surface, selected by which
a nil `InitImage` is a pure text prompt; setting it conditions generation keyframes are set rather than by a mode flag:
on that frame (hybrid checkpoints like Wan 2.2 TI2V serve both). First
backend: llama-swap (blocking `/v1/videos/sync`, vLLM-Omni style — the | `InitImage` | `LastImage` | mode |
|---|---|---|
| nil | nil | text-to-video |
| set | nil | image-to-video (hybrid checkpoints like Wan 2.2 TI2V serve both) |
| set | set | first-last-frame — both ends pinned |
| nil | set | pin the destination, model invents the approach |
First backend: llama-swap (blocking `/v1/videos/sync`, vLLM-Omni style — the
response body is the encoded clip, so `Result` carries a single `Video`). response body is the encoded clip, so `Result` carries a single `Video`).
Generation runs for minutes; bound the call with a context deadline. Generation runs for minutes; bound the call with a context deadline.
**`LastImage` support is per-model and cannot be detected.** A backend that
does not understand a trailing keyframe ignores the part and returns an
ordinary clip — indistinguishable from success. There is no capability bit,
because the contract has no way to learn one, so a caller depending on the
pin must establish support out of band.
```go ```go
vm, _ := ls.VideoModel("videogen-wan22-5b") vm, _ := ls.VideoModel("videogen-wan22-5b")
res, err := vm.Generate(ctx, videogen.Request{Prompt: "a cat surfing"}, res, err := vm.Generate(ctx, videogen.Request{Prompt: "a cat surfing"},
@@ -407,6 +434,7 @@ to build one.
|----------------------|:---:|:---:|:---:|:---:|:---:|:---:|:---:| |----------------------|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
| OpenAI (+compatible) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | OpenAI (+compatible) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Kimi (Moonshot AI) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅³ | ✅ | | Kimi (Moonshot AI) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅³ | ✅ |
| Qwen (Alibaba) | ✅ | ✅ | ✅ | ✅ | ✅⁴ | ✅⁴ | ✅ |
| Anthropic (+compat) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | Anthropic (+compat) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Google (Gemini) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | Google (Gemini) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Ollama Cloud | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | Ollama Cloud | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
@@ -431,6 +459,13 @@ probe and management methods on `*llamaswap.Provider`.
level; whether a call succeeds depends on the Moonshot model — only the vision level; whether a call succeeds depends on the Moonshot model — only the vision
variants (e.g. `moonshot-v1-8k-vision-preview`) accept images. variants (e.g. `moonshot-v1-8k-vision-preview`) accept images.
⁴ Qwen also reuses the openai client (ADR-0027), so both columns are present at
the client level and gated by the Model Studio model you name: `json_schema`
structured output is on the Max/Plus families, image inputs on the `qwen-vl-*`
/ `qwen3-vl-*` models. `reasoning_effort` rides through as a top-level field —
one reason the built-in speaks OpenAI-compat rather than Model Studio's
Anthropic-compat shim.
Notes: Ollama has no native tool_choice — `"none"` drops the tools; Notes: Ollama has no native tool_choice — `"none"` drops the tools;
`"required"`/named choices are best-effort ignored there. Ollama Cloud `"required"`/named choices are best-effort ignored there. Ollama Cloud
ignores the `format` field (verified live), so the provider also states ignores the `format` field (verified live), so the provider also states
+68 -27
View File
@@ -2,7 +2,6 @@ package majordomo
import ( import (
"net/http" "net/http"
"strings"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm" "gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/majordomo/provider/anthropic" "gitea.stevedudenhoeffer.com/steve/majordomo/provider/anthropic"
@@ -19,6 +18,12 @@ const (
// Chat Completions endpoint. Reuses the openai client (like llama-swap); // Chat Completions endpoint. Reuses the openai client (like llama-swap);
// keyed by KIMI_API_KEY, default base URL kimiBaseURL. // keyed by KIMI_API_KEY, default base URL kimiBaseURL.
ProviderKimi = "kimi" ProviderKimi = "kimi"
// ProviderQwen is Alibaba's Qwen models over Model Studio's
// OpenAI-compatible Chat Completions endpoint. Reuses the openai client
// (like kimi and llama-swap); keyed by QWEN_API_KEY, default base URL
// qwenBaseURL. ADR-0027 records why the OpenAI surface and not the
// Anthropic-compatible one Model Studio also exposes.
ProviderQwen = "qwen"
ProviderAnthropic = "anthropic" ProviderAnthropic = "anthropic"
ProviderGoogle = "google" ProviderGoogle = "google"
ProviderOllama = "ollama" ProviderOllama = "ollama"
@@ -37,6 +42,55 @@ const (
// China endpoint (api.moonshot.cn/v1) is reachable via a kimi:// LLM_* DSN. // China endpoint (api.moonshot.cn/v1) is reachable via a kimi:// LLM_* DSN.
const kimiBaseURL = "https://api.moonshot.ai/v1" const kimiBaseURL = "https://api.moonshot.ai/v1"
// qwenBaseURL is Alibaba Model Studio's international (Singapore) endpoint in
// OpenAI-compatible mode. The China endpoint
// (dashscope.aliyuncs.com/compatible-mode/v1) and any regional host are
// reachable via a qwen:// LLM_* DSN.
const qwenBaseURL = "https://dashscope-intl.aliyuncs.com/compatible-mode/v1"
// openaiCompatScheme builds the DSN factory shared by every built-in that is
// "the openai client pointed somewhere else" (kimi, qwen, ...). The provider
// is named after the LLM_<NAME> var that defined it, takes its credential from
// the DSN token — not the built-in's own env var, which does nothing for a
// DSN-defined provider — and so names that same LLM_<NAME> var in the
// missing-key hint, matching the lazy-resolution key form in providerFor.
//
// wrap is the caller's option-decorator (it injects the registry's HTTP
// client), so a DSN provider is built exactly like the eager built-ins.
func openaiCompatScheme(wrap func(...openai.Option) []openai.Option) SchemeFactory {
return func(name string, dsn DSN) (llm.Provider, error) {
return openai.New(wrap(
openai.WithName(name),
openai.WithBaseURL(dsn.BaseURL()),
openai.WithAPIKey(dsn.Token),
openai.WithAPIKeyName(envKeyForProvider(name)),
)...), nil
}
}
// registerOpenAICompatBuiltin installs BOTH halves of an OpenAI-compat
// built-in: the eager provider under name (credential from keyEnv) and the
// matching name:// DSN scheme. Why both in one call: the two halves are a pair
// — a built-in whose scheme is missing resolves as a spec but not from an
// LLM_* DSN, and the credential rules below have to hold identically in each.
// Adding the next one is a single line rather than six lines to copy.
//
// The two credential rules, holding by construction for every caller:
// - WithAPIKey is passed UNCONDITIONALLY, even when the lookup comes back
// empty. openai.New defaults its key to OPENAI_API_KEY, so anything less
// lets an unset keyEnv silently authenticate as OpenAI.
// - WithAPIKeyName makes the synthetic-401 hint name keyEnv, so a keyless
// call tells the operator the variable that actually fixes it.
func registerOpenAICompatBuiltin(r *Registry, wrap func(...openai.Option) []openai.Option, name, baseURL, keyEnv string) {
r.providers[name] = openai.New(wrap(
openai.WithName(name),
openai.WithBaseURL(baseURL),
openai.WithAPIKey(r.envLookup(keyEnv)),
openai.WithAPIKeyName(keyEnv),
)...)
r.schemes[name] = openaiCompatScheme(wrap)
}
// registerBuiltins installs the built-in providers and env-DSN scheme // registerBuiltins installs the built-in providers and env-DSN scheme
// factories into a fresh registry. httpClient, when non-nil, is used by // factories into a fresh registry. httpClient, when non-nil, is used by
// every provider and factory the registry itself constructs. // every provider and factory the registry itself constructs.
@@ -83,32 +137,19 @@ func registerBuiltins(r *Registry, httpClient *http.Client) {
)...), nil )...), nil
} }
// Kimi (Moonshot AI): OpenAI-compatible Chat Completions, so it reuses the // Third-party endpoints that ARE the openai client at another base URL —
// openai client (like llama-swap). Defaults to Moonshot's international // no new package, mirroring llama-swap's chat path. Each gets the eager
// endpoint and the KIMI_API_KEY credential. WithAPIKey is passed // built-in plus its name:// DSN scheme, and the credential rules hold by
// unconditionally — even empty — so an unset KIMI_API_KEY can never fall // construction (see registerOpenAICompatBuiltin).
// through to the openai client's OPENAI_API_KEY default; WithAPIKeyName //
// makes the missing-key error name KIMI_API_KEY. // kimi (ADR-0026): Moonshot's international endpoint; China host via
r.providers[ProviderKimi] = openai.New(openaiOpts( // kimi://[email protected]/v1.
openai.WithName(ProviderKimi), registerOpenAICompatBuiltin(r, openaiOpts, ProviderKimi, kimiBaseURL, "KIMI_API_KEY")
openai.WithBaseURL(kimiBaseURL), // qwen (ADR-0027): Alibaba Model Studio's international host. Model Studio
openai.WithAPIKey(r.envLookup("KIMI_API_KEY")), // also exposes an Anthropic-compatible endpoint; the ADR records why the
openai.WithAPIKeyName("KIMI_API_KEY"), // OpenAI one is the built-in. China / workspace-scoped regional hosts via
)...) // qwen://[email protected]/compatible-mode/v1.
// kimi:// DSN scheme: an OpenAI-compatible target labeled kimi, base URL registerOpenAICompatBuiltin(r, openaiOpts, ProviderQwen, qwenBaseURL, "QWEN_API_KEY")
// from the DSN host (e.g. kimi://[email protected]/v1 for China). Its
// credential is the DSN token, not KIMI_API_KEY, so the missing-key hint
// names the LLM_<NAME> env var that defines this provider (matching the
// lazy-resolution key form in providerFor) — the fix for a keyless target
// here is adding a token to that DSN.
r.schemes[ProviderKimi] = func(name string, dsn DSN) (llm.Provider, error) {
return openai.New(openaiOpts(
openai.WithName(name),
openai.WithBaseURL(dsn.BaseURL()),
openai.WithAPIKey(dsn.Token),
openai.WithAPIKeyName("LLM_"+strings.ToUpper(strings.ReplaceAll(name, "-", "_"))),
)...), nil
}
// llama-swap: OpenAI-compatible chat + image generation + management // llama-swap: OpenAI-compatible chat + image generation + management
// endpoints over a model-swapping proxy. Chat reuses the openai client // endpoints over a model-swapping proxy. Chat reuses the openai client
-171
View File
@@ -1,171 +0,0 @@
package majordomo
import (
"context"
"errors"
"io"
"net/http"
"strings"
"testing"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
)
// kimiResponse is a minimal valid Chat Completions body so Generate returns a
// non-empty response (an empty one would trigger failover, not a clean pass).
const kimiResponse = `{"id":"c1","object":"chat.completion","choices":[` +
`{"index":0,"message":{"role":"assistant","content":"ok"},"finish_reason":"stop"}]}`
// captureRT records the last request and returns a canned response without
// touching the network, so these tests stay hermetic while still exercising
// the real openai client the kimi built-in reuses (base URL + auth header).
type captureRT struct {
req *http.Request
body string
}
func (c *captureRT) RoundTrip(r *http.Request) (*http.Response, error) {
c.req = r
return &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(strings.NewReader(c.body)),
Header: make(http.Header),
Request: r,
}, nil
}
// TestKimiBuiltin: the built-in "kimi" provider resolves in Parse, targets
// Moonshot's default endpoint, and authenticates with KIMI_API_KEY.
func TestKimiBuiltin(t *testing.T) {
rt := &captureRT{body: kimiResponse}
r := newTestRegistry(t,
WithEnvLookup(func(k string) string {
if k == "KIMI_API_KEY" {
return "kimi-secret"
}
return ""
}),
WithHTTPClient(&http.Client{Transport: rt}),
)
if p, ok := r.Provider(ProviderKimi); !ok {
t.Fatal("built-in kimi provider not registered")
} else if p.Name() != ProviderKimi {
t.Errorf("name = %q, want %q", p.Name(), ProviderKimi)
}
m, err := r.Parse("kimi/kimi-k2-0711-preview")
if err != nil {
t.Fatalf("Parse: %v", err)
}
if got := targetsOf(t, m); len(got) != 1 || got[0] != "kimi/kimi-k2-0711-preview" {
t.Fatalf("targets = %v", got)
}
if _, err := m.Generate(context.Background(), llm.Request{Messages: []llm.Message{llm.UserText("hi")}}); err != nil {
t.Fatalf("Generate: %v", err)
}
if rt.req == nil {
t.Fatal("no request captured")
}
if want := "https://api.moonshot.ai/v1/chat/completions"; rt.req.URL.String() != want {
t.Errorf("URL = %q, want %q", rt.req.URL.String(), want)
}
if want := "Bearer kimi-secret"; rt.req.Header.Get("Authorization") != want {
t.Errorf("Authorization = %q, want %q", rt.req.Header.Get("Authorization"), want)
}
}
// TestKimiBuiltinMissingKey: with no KIMI_API_KEY the built-in fails fast with a
// synthetic 401 whose hint names KIMI_API_KEY — never OPENAI_API_KEY (proving
// the credential does not fall through to the openai client's default), and
// without hitting the network.
func TestKimiBuiltinMissingKey(t *testing.T) {
rt := &captureRT{body: kimiResponse}
r := newTestRegistry(t, WithHTTPClient(&http.Client{Transport: rt}))
m, err := r.Parse("kimi/kimi-k2-0711-preview")
if err != nil {
t.Fatalf("Parse: %v", err)
}
_, err = m.Generate(context.Background(), llm.Request{Messages: []llm.Message{llm.UserText("hi")}})
apiErr, ok := errors.AsType[*llm.APIError](err)
if !ok {
t.Fatalf("err = %v (%T), want *llm.APIError", err, err)
}
if apiErr.Status != http.StatusUnauthorized || apiErr.Code != "missing_api_key" {
t.Errorf("Status/Code = %d/%q, want 401/missing_api_key", apiErr.Status, apiErr.Code)
}
if !strings.Contains(apiErr.Message, "KIMI_API_KEY") {
t.Errorf("message = %q, want it to name KIMI_API_KEY", apiErr.Message)
}
if strings.Contains(apiErr.Message, "OPENAI_API_KEY") {
t.Errorf("message = %q, must not name OPENAI_API_KEY", apiErr.Message)
}
if rt.req != nil {
t.Error("network was hit despite missing key")
}
}
// TestKimiScheme: a kimi:// LLM_* DSN defines a named provider on any Moonshot
// host (here the China endpoint) that is first-class in Parse and carries the
// DSN token as its bearer credential.
func TestKimiScheme(t *testing.T) {
rt := &captureRT{body: kimiResponse}
r := newTestRegistry(t, WithHTTPClient(&http.Client{Transport: rt}))
if err := r.LoadEnv(map[string]string{
"LLM_KCN": "kimi://[email protected]/v1",
}); err != nil {
t.Fatalf("LoadEnv: %v", err)
}
m, err := r.Parse("kcn/moonshot-v1-8k")
if err != nil {
t.Fatalf("Parse: %v", err)
}
if _, err := m.Generate(context.Background(), llm.Request{Messages: []llm.Message{llm.UserText("hi")}}); err != nil {
t.Fatalf("Generate: %v", err)
}
if rt.req == nil {
t.Fatal("no request captured")
}
if want := "https://api.moonshot.cn/v1/chat/completions"; rt.req.URL.String() != want {
t.Errorf("URL = %q, want %q", rt.req.URL.String(), want)
}
if want := "Bearer tok"; rt.req.Header.Get("Authorization") != want {
t.Errorf("Authorization = %q, want %q", rt.req.Header.Get("Authorization"), want)
}
}
// TestKimiSchemeMissingToken: a kimi:// DSN with no token is fixed by adding one
// to the DSN, not by setting KIMI_API_KEY — so the missing-key hint names the
// defining LLM_<NAME> env var, never KIMI_API_KEY (which does nothing for a
// DSN-defined provider).
func TestKimiSchemeMissingToken(t *testing.T) {
rt := &captureRT{body: kimiResponse}
r := newTestRegistry(t, WithHTTPClient(&http.Client{Transport: rt}))
if err := r.LoadEnv(map[string]string{
"LLM_KCN": "kimi://api.moonshot.cn/v1", // no token
}); err != nil {
t.Fatalf("LoadEnv: %v", err)
}
m, err := r.Parse("kcn/moonshot-v1-8k")
if err != nil {
t.Fatalf("Parse: %v", err)
}
_, err = m.Generate(context.Background(), llm.Request{Messages: []llm.Message{llm.UserText("hi")}})
apiErr, ok := errors.AsType[*llm.APIError](err)
if !ok {
t.Fatalf("err = %v (%T), want *llm.APIError", err, err)
}
if !strings.Contains(apiErr.Message, "LLM_KCN") {
t.Errorf("message = %q, want it to name LLM_KCN", apiErr.Message)
}
if strings.Contains(apiErr.Message, "KIMI_API_KEY") {
t.Errorf("message = %q, must not name KIMI_API_KEY for a DSN provider", apiErr.Message)
}
if rt.req != nil {
t.Error("network was hit despite missing token")
}
}
+240
View File
@@ -0,0 +1,240 @@
package majordomo
import (
"context"
"errors"
"io"
"net/http"
"strings"
"testing"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
)
// Shared fixtures and the shared contract for the built-ins that are "the
// openai client pointed somewhere else" (kimi, qwen, ...). They live here
// rather than in any one provider's test file so a new OpenAI-compat built-in
// has nothing to copy — the same reason registerOpenAICompatBuiltin exists on
// the production side.
// chatCompletionOK is a minimal valid Chat Completions body, so Generate
// returns a non-empty response (an empty one would trigger failover, not a
// clean pass).
const chatCompletionOK = `{"id":"c1","object":"chat.completion","choices":[` +
`{"index":0,"message":{"role":"assistant","content":"ok"},"finish_reason":"stop"}]}`
// captureRT records the last request (and the bytes of its body) and returns a
// canned response without touching the network, so these tests stay hermetic
// while still exercising the real openai client the built-ins reuse: base URL,
// auth header, and the JSON actually put on the wire.
type captureRT struct {
req *http.Request
reqBody []byte
body string
}
func (c *captureRT) RoundTrip(r *http.Request) (*http.Response, error) {
c.req = r
// Drain and close the request body: a RoundTripper owns it, and those
// bytes are what wire-shape assertions read.
c.reqBody = nil
if r.Body != nil {
c.reqBody, _ = io.ReadAll(r.Body)
_ = r.Body.Close()
}
return &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(strings.NewReader(c.body)),
Header: make(http.Header),
Request: r,
}, nil
}
// singleKeyEnv builds a WithEnvLookup function that knows exactly one variable
// and returns "" for everything else. The empty default has teeth: a built-in
// that reached for any other variable name gets nothing, so the request 401s
// and the test fails rather than quietly authenticating off the wrong key.
func singleKeyEnv(key, value string) func(string) string {
return func(k string) string {
if k == key {
return value
}
return ""
}
}
// openAICompatBuiltin describes one built-in for the shared contract below.
// Adding an OpenAI-compat built-in means adding a row here — not copying a
// test file, which is how kimi's and qwen's suites became near-identical.
type openAICompatBuiltin struct {
name string // registry name and spec prefix
keyEnv string // the credential variable this built-in reads
model string // a current model id for that endpoint
wantURL string // chat-completions URL the default endpoint must produce
// The name:// DSN case: an alternate host (regional/China endpoint)
// reached through an LLM_<dsnVar> definition.
dsnVar string
dsnHost string
wantDSNURL string
}
var openAICompatBuiltins = []openAICompatBuiltin{
{
name: ProviderKimi,
keyEnv: "KIMI_API_KEY",
model: "kimi-k2-0711-preview",
wantURL: "https://api.moonshot.ai/v1/chat/completions",
dsnVar: "LLM_KCN",
dsnHost: "api.moonshot.cn/v1",
wantDSNURL: "https://api.moonshot.cn/v1/chat/completions",
},
{
name: ProviderQwen,
keyEnv: "QWEN_API_KEY",
model: "qwen3.8-max",
wantURL: "https://dashscope-intl.aliyuncs.com/compatible-mode/v1/chat/completions",
dsnVar: "LLM_QCN",
dsnHost: "dashscope.aliyuncs.com/compatible-mode/v1",
wantDSNURL: "https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions",
},
}
// TestOpenAICompatBuiltins is the whole contract an OpenAI-compat built-in
// owes, asserted identically for every one of them: it resolves in Parse and
// targets its own endpoint with its own key; a missing key fails closed naming
// the right variable and never reaching the network; its name:// DSN reaches
// any other host on the DSN token; and a keyless DSN names the LLM_<NAME> that
// actually fixes it rather than the built-in's variable, which does nothing
// for a DSN-defined provider.
func TestOpenAICompatBuiltins(t *testing.T) {
for _, tc := range openAICompatBuiltins {
t.Run(tc.name+"/builtin", func(t *testing.T) {
rt := &captureRT{body: chatCompletionOK}
secret := tc.name + "-secret"
r := newTestRegistry(t,
WithEnvLookup(singleKeyEnv(tc.keyEnv, secret)),
WithHTTPClient(&http.Client{Transport: rt}),
)
if p, ok := r.Provider(tc.name); !ok {
t.Fatalf("built-in %q not registered", tc.name)
} else if p.Name() != tc.name {
t.Errorf("name = %q, want %q", p.Name(), tc.name)
}
spec := tc.name + "/" + tc.model
m, err := r.Parse(spec)
if err != nil {
t.Fatalf("Parse(%q): %v", spec, err)
}
if got := targetsOf(t, m); len(got) != 1 || got[0] != spec {
t.Fatalf("targets = %v, want [%q]", got, spec)
}
if _, err := m.Generate(context.Background(), llm.Request{Messages: []llm.Message{llm.UserText("hi")}}); err != nil {
t.Fatalf("Generate: %v", err)
}
if rt.req == nil {
t.Fatal("no request captured")
}
if rt.req.URL.String() != tc.wantURL {
t.Errorf("URL = %q, want %q", rt.req.URL.String(), tc.wantURL)
}
if want := "Bearer " + secret; rt.req.Header.Get("Authorization") != want {
t.Errorf("Authorization = %q, want %q", rt.req.Header.Get("Authorization"), want)
}
})
t.Run(tc.name+"/builtin missing key", func(t *testing.T) {
rt := &captureRT{body: chatCompletionOK}
r := newTestRegistry(t, WithHTTPClient(&http.Client{Transport: rt}))
m, err := r.Parse(tc.name + "/" + tc.model)
if err != nil {
t.Fatalf("Parse: %v", err)
}
_, err = m.Generate(context.Background(), llm.Request{Messages: []llm.Message{llm.UserText("hi")}})
apiErr, ok := errors.AsType[*llm.APIError](err)
if !ok {
t.Fatalf("err = %v (%T), want *llm.APIError", err, err)
}
if apiErr.Status != http.StatusUnauthorized || apiErr.Code != "missing_api_key" {
t.Errorf("Status/Code = %d/%q, want 401/missing_api_key", apiErr.Status, apiErr.Code)
}
if !strings.Contains(apiErr.Message, tc.keyEnv) {
t.Errorf("message = %q, want it to name %s", apiErr.Message, tc.keyEnv)
}
// The load-bearing half: openai.New defaults its key to
// OPENAI_API_KEY, so a built-in that stopped passing WithAPIKey
// unconditionally would authenticate as OpenAI instead of failing.
if strings.Contains(apiErr.Message, "OPENAI_API_KEY") {
t.Errorf("message = %q, must not name OPENAI_API_KEY", apiErr.Message)
}
if rt.req != nil {
t.Error("network was hit despite missing key")
}
})
t.Run(tc.name+"/dsn scheme", func(t *testing.T) {
rt := &captureRT{body: chatCompletionOK}
r := newTestRegistry(t, WithHTTPClient(&http.Client{Transport: rt}))
if err := r.LoadEnv(map[string]string{
tc.dsnVar: tc.name + "://tok@" + tc.dsnHost,
}); err != nil {
t.Fatalf("LoadEnv: %v", err)
}
dsnName := strings.ToLower(strings.TrimPrefix(tc.dsnVar, "LLM_"))
m, err := r.Parse(dsnName + "/" + tc.model)
if err != nil {
t.Fatalf("Parse: %v", err)
}
if _, err := m.Generate(context.Background(), llm.Request{Messages: []llm.Message{llm.UserText("hi")}}); err != nil {
t.Fatalf("Generate: %v", err)
}
if rt.req == nil {
t.Fatal("no request captured")
}
if rt.req.URL.String() != tc.wantDSNURL {
t.Errorf("URL = %q, want %q", rt.req.URL.String(), tc.wantDSNURL)
}
if want := "Bearer tok"; rt.req.Header.Get("Authorization") != want {
t.Errorf("Authorization = %q, want %q", rt.req.Header.Get("Authorization"), want)
}
})
t.Run(tc.name+"/dsn scheme missing token", func(t *testing.T) {
rt := &captureRT{body: chatCompletionOK}
r := newTestRegistry(t, WithHTTPClient(&http.Client{Transport: rt}))
if err := r.LoadEnv(map[string]string{
tc.dsnVar: tc.name + "://" + tc.dsnHost, // no token
}); err != nil {
t.Fatalf("LoadEnv: %v", err)
}
dsnName := strings.ToLower(strings.TrimPrefix(tc.dsnVar, "LLM_"))
m, err := r.Parse(dsnName + "/" + tc.model)
if err != nil {
t.Fatalf("Parse: %v", err)
}
_, err = m.Generate(context.Background(), llm.Request{Messages: []llm.Message{llm.UserText("hi")}})
apiErr, ok := errors.AsType[*llm.APIError](err)
if !ok {
t.Fatalf("err = %v (%T), want *llm.APIError", err, err)
}
// A keyless DSN is fixed by adding a token to that DSN, so the
// hint must name the defining variable — never the built-in's own
// key, which does nothing for a DSN-defined provider.
if !strings.Contains(apiErr.Message, tc.dsnVar) {
t.Errorf("message = %q, want it to name %s", apiErr.Message, tc.dsnVar)
}
if strings.Contains(apiErr.Message, tc.keyEnv) {
t.Errorf("message = %q, must not name %s for a DSN provider", apiErr.Message, tc.keyEnv)
}
if rt.req != nil {
t.Error("network was hit despite missing token")
}
})
}
}
+86
View File
@@ -0,0 +1,86 @@
package majordomo
import (
"context"
"encoding/json"
"net/http"
"testing"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
)
// The contract qwen shares with every other OpenAI-compat built-in (endpoint,
// credential isolation, its qwen:// DSN) is asserted by the table in
// builtin_openaicompat_test.go. What remains here is qwen-specific: the
// reverse-leak direction, and the wire claim ADR-0027 turns on.
// TestQwenBuiltinKeyDoesNotLeakToOpenAI: QWEN_API_KEY is the qwen built-in's
// credential and nothing else's. Why this direction too: the shared table's
// missing-key case only proves qwen never borrows OPENAI_API_KEY; this proves
// the reverse — a registry that can see QWEN_API_KEY must not hand it to the
// openai built-in, which would send an Alibaba key to api.openai.com.
func TestQwenBuiltinKeyDoesNotLeakToOpenAI(t *testing.T) {
// Set before newTestRegistry: the openai built-in reads OPENAI_API_KEY at
// construction. Giving it a real key is what keeps this test honest — a
// keyless openai target would 401 before any request, and the assertion
// below would pass without a single byte reaching the wire.
t.Setenv("OPENAI_API_KEY", "openai-secret")
rt := &captureRT{body: chatCompletionOK}
r := newTestRegistry(t,
WithEnvLookup(singleKeyEnv("QWEN_API_KEY", "qwen-secret")),
WithHTTPClient(&http.Client{Transport: rt}),
)
m, err := r.Parse("openai/gpt-4o-mini")
if err != nil {
t.Fatalf("Parse: %v", err)
}
if _, err := m.Generate(context.Background(), llm.Request{Messages: []llm.Message{llm.UserText("hi")}}); err != nil {
t.Fatalf("Generate: %v", err)
}
if rt.req == nil {
t.Fatal("no request captured")
}
if want := "Bearer openai-secret"; rt.req.Header.Get("Authorization") != want {
t.Errorf("Authorization = %q, want %q — the qwen credential must not reach the openai built-in",
rt.req.Header.Get("Authorization"), want)
}
}
// TestQwenReasoningEffortReachesWire is the load-bearing test for ADR-0027's
// central claim: Model Studio's OpenAI-compatible surface takes reasoning as a
// top-level "reasoning_effort" body field, which the openai client already
// sends — so llm.WithReasoningEffort survives the trip on qwen with no
// qwen-specific code. Routing qwen through the anthropic client instead would
// drop it silently (provider/anthropic ignores ReasoningEffort by design), and
// that difference would be invisible without asserting on the wire body.
func TestQwenReasoningEffortReachesWire(t *testing.T) {
rt := &captureRT{body: chatCompletionOK}
r := newTestRegistry(t,
WithEnvLookup(singleKeyEnv("QWEN_API_KEY", "qwen-secret")),
WithHTTPClient(&http.Client{Transport: rt}),
)
m, err := r.Parse("qwen/qwen3.8-max")
if err != nil {
t.Fatalf("Parse: %v", err)
}
_, err = m.Generate(context.Background(), llm.Request{
Messages: []llm.Message{llm.UserText("hi")},
ReasoningEffort: "high",
})
if err != nil {
t.Fatalf("Generate: %v", err)
}
if rt.reqBody == nil {
t.Fatal("no request body captured")
}
var sent map[string]any
if err := json.Unmarshal(rt.reqBody, &sent); err != nil {
t.Fatalf("decode request body: %v", err)
}
if got := sent["reasoning_effort"]; got != "high" {
t.Errorf("reasoning_effort = %v, want %q (body: %s)", got, "high", rt.reqBody)
}
}
+102
View File
@@ -0,0 +1,102 @@
# ADR-0027: Qwen (Alibaba) built-in provider — OpenAI-compat, not Anthropic-compat
**Status:** Accepted — 2026-08-12
## Context
Alibaba's Qwen models (`qwen3.8-max`, `qwen3.7-plus`, the `qwen3-vl-*` vision
variants, …) are served from Model Studio / DashScope, and mort wants them as a
first-class failover tier with a dedicated `QWEN_API_KEY` — the same ergonomics
ADR-0026 gave Kimi.
Unlike Kimi, Model Studio exposes the same models over **two** protocols:
| | OpenAI-compatible | Anthropic-compatible |
|---|---|---|
| Base URL | `https://dashscope-intl.aliyuncs.com/compatible-mode/v1` | `https://dashscope-intl.aliyuncs.com/apps/anthropic` |
| Endpoints | full Chat Completions surface | `/v1/messages` only (no `/v1/models`) |
| Purpose | the documented developer API | a shim, documented around hosting Claude Code |
So the question this ADR answers is not "which client do we reuse" but
"which of Alibaba's two wire protocols does the built-in speak".
## Decision
**The `qwen` built-in and the `qwen://` DSN scheme speak OpenAI-compat**, over
`provider/openai` — no new package, mirroring ADR-0026 (kimi) and ADR-0015
(llama-swap chat). Default base URL is the international host; the China host
(`dashscope.aliyuncs.com/compatible-mode/v1`) and workspace-scoped regional
hosts are reachable with a `qwen://` DSN.
Credential handling is copied from kimi verbatim, because both of its rules
are load-bearing: `WithAPIKey` is passed unconditionally (even empty) so an
unset `QWEN_API_KEY` can never fall through to `openai.New`'s `OPENAI_API_KEY`
default, and `WithAPIKeyName("QWEN_API_KEY")` makes the synthetic-401 hint name
the variable the operator actually has to set.
The kimi and qwen DSN factories were identical, so they now share one
`openaiCompatScheme` helper — the next OpenAI-compat built-in gets the
credential and key-hint rules by construction rather than by copy.
### Why not the Anthropic-compatible endpoint
Every concrete difference favors OpenAI-compat *for this codebase*:
- **Reasoning survives the trip.** Model Studio takes `reasoning_effort` as a
top-level field on the OpenAI surface, which `provider/openai` already sends
`llm.WithReasoningEffort` works on qwen with zero qwen-specific code
(`TestQwenReasoningEffortReachesWire` asserts it on the wire). Down the
anthropic client it would be dropped in silence: `provider/anthropic`
deliberately ignores `Request.ReasoningEffort`, because first-party Claude
has no such knob.
- **Structured output would regress.** `provider/anthropic` implements
`Request.Schema` with the first-party GA `output_config.format` mechanism.
Alibaba's shim does not implement it; a compat endpoint that ignores an
unknown field returns unconstrained prose while still reporting success.
The OpenAI path sends `response_format: json_schema`, which Model Studio
supports natively on the Max/Plus families.
- **Cache accounting already lands.** Model Studio's implicit prefix cache
reports hits in `usage.prompt_tokens_details.cached_tokens`, which the openai
client already maps to `llm.Usage.CacheReadTokens`. The anthropic client
reads `cache_read_input_tokens`, a field the shim has no reason to emit.
- **Thinking content is discarded on the anthropic path anyway.**
`provider/anthropic` skips `thinking` blocks in both the buffered and
streaming decoders, so the shim's headline feature — first-class
`thinking: {type: "enabled", budget_tokens: N}` — buys majordomo nothing
today.
- **Smaller blast radius.** The anthropic client has no `WithAPIKeyName`
option, so a keyless qwen would tell the operator to set `ANTHROPIC_API_KEY`;
fixing that means changing the first-party Anthropic client to serve a
third-party shim.
- **It is the less-exercised surface.** The Anthropic endpoint is documented as
Messages-only, with a temperature range that differs from Anthropic's own
([0, 2) vs [0.0, 1.0]) — i.e. it is Qwen semantics wearing an Anthropic
envelope, not an Anthropic-equivalent target.
The one thing the Anthropic surface offers that OpenAI-compat does not is
explicit `cache_control` breakpoints reached through `Request.PromptCache`.
That is not a reason to route Qwen through it: Model Studio's implicit cache is
automatic and already metered, and if explicit breakpoints ever matter they
belong in `provider/openai` (Model Studio accepts `cache_control` on content
blocks there too), where every OpenAI-compat target would get them.
## Consequences
- `qwen/<model>` is first-class in Parse, chains, aliases, and health/failover
with no consumer wiring; model ids pass through verbatim (no catalog).
- Chat, streaming, tools, structured output, reasoning effort, and cached-token
accounting all ride the openai client and inherit its fixes.
- Image *inputs* work at the client level, but only the `qwen-vl-*` /
`qwen3-vl-*` models accept them (matrix footnote ⁴; ³ is kimi's).
- Two model-side quirks are Alibaba's, not majordomo's, and are left to the
caller rather than papered over: thinking is **on by default** on some models
(e.g. `qwen3.7-plus`), and Qwen3 *open-source* models require streaming when
thinking is enabled — a buffered `Generate` against one of those needs a
model that supports non-streaming thinking (the Max/Plus families do).
- If a future consumer genuinely needs the Anthropic surface, it is reachable
today without library changes:
`LLM_QWEN_ANTHROPIC=anthropic://[email protected]/apps/anthropic`
— with the reasoning/structured-output caveats above.
- Second third-party built-in after kimi. The ADR-0026 bar still holds: a named
consumer needs it in-config. `RegisterProvider`/`LLM_*` remain the path for
everything else.
+1
View File
@@ -30,3 +30,4 @@ One decision per file, append-only; supersede rather than rewrite.
| [0024](0024-audio-wave3-surfaces.md) | Wave-3 audio surfaces (stems, SFX, speech enhance, voice clone, translate) | Accepted | | [0024](0024-audio-wave3-surfaces.md) | Wave-3 audio surfaces (stems, SFX, speech enhance, voice clone, translate) | Accepted |
| [0025](0025-videogen-wave3-surfaces.md) | Wave-3 video surfaces (lipsync, video matte, video upscale, chain jobs) | Accepted | | [0025](0025-videogen-wave3-surfaces.md) | Wave-3 video surfaces (lipsync, video matte, video upscale, chain jobs) | Accepted |
| [0026](0026-kimi-builtin.md) | Kimi (Moonshot AI) built-in provider — reuse openai client, KIMI_API_KEY | Accepted | | [0026](0026-kimi-builtin.md) | Kimi (Moonshot AI) built-in provider — reuse openai client, KIMI_API_KEY | Accepted |
| [0027](0027-qwen-builtin.md) | Qwen (Alibaba) built-in provider — OpenAI-compat, not Model Studio's Anthropic-compat endpoint | Accepted |
+16 -2
View File
@@ -26,8 +26,9 @@ var ErrUnknownProvider = errors.New("unknown provider")
// authenticated with the bearer token "test-token". // authenticated with the bearer token "test-token".
type DSN struct { type DSN struct {
// Scheme selects the provider implementation: "foreman", "ollama", // Scheme selects the provider implementation: "foreman", "ollama",
// "ollama-cloud", "openai", "kimi", "anthropic", "google"/"gemini", or // "ollama-cloud", "openai", "kimi", "qwen", "anthropic",
// any custom scheme registered with RegisterScheme. // "google"/"gemini", "llama-swap"/"llama-swaps", or any custom scheme
// registered with RegisterScheme.
Scheme string Scheme string
// Token is the provider secret (bearer token or API key); empty = none. // Token is the provider secret (bearer token or API key); empty = none.
Token string Token string
@@ -40,6 +41,19 @@ type DSN struct {
// env-defined providers always speak TLS). // env-defined providers always speak TLS).
func (d DSN) BaseURL() string { return "https://" + d.Host } func (d DSN) BaseURL() string { return "https://" + d.Host }
// envKeyForProvider returns the LLM_* variable that defines the provider named
// name: "m1" → LLM_M1, "my-prov" → LLM_MY_PROV.
//
// This is the single definition on purpose. Two call sites need byte-identical
// output and would drift apart in silence: lazy resolution reads this variable
// to find an unregistered provider, and openaiCompatScheme names it in the
// missing-key hint so a keyless DSN target tells the operator which variable to
// set. Those two were separate copies with a comment asserting they matched —
// a comment is not enforcement, this function is.
func envKeyForProvider(name string) string {
return "LLM_" + strings.ToUpper(strings.ReplaceAll(name, "-", "_"))
}
// ParseDSN parses a raw DSN string. The algorithm matches go-llm exactly: // ParseDSN parses a raw DSN string. The algorithm matches go-llm exactly:
// split on "://", then an optional "@" separates the token from the host; // split on "://", then an optional "@" separates the token from the host;
// trailing slashes on the host are trimmed. // trailing slashes on the host are trimmed.
+36
View File
@@ -285,3 +285,39 @@ tests flush out.
(footnote ³), `.env.example`, ADR-0026 (+ index; also backfilled the missing (footnote ³), `.env.example`, ADR-0026 (+ index; also backfilled the missing
0024/0025 index rows). 0024/0025 index rows).
- Consumer: mort names Kimi as a failover tier. - Consumer: mort names Kimi as a failover tier.
## 2026-08-12 — Qwen (Alibaba) built-in provider (ADR-0027)
- New built-in `qwen` provider + `qwen://` DSN scheme over Alibaba Model
Studio's OpenAI-compatible mode, reusing `provider/openai` (no new client,
mirrors kimi/llama-swap). Default base URL
`https://dashscope-intl.aliyuncs.com/compatible-mode/v1`; China/regional
hosts via `LLM_QCN=qwen://[email protected]/compatible-mode/v1`.
- Credential is `QWEN_API_KEY` (via the registry's injected envLookup).
`WithAPIKey` passed unconditionally so an unset key cannot fall through to
`OPENAI_API_KEY`; `WithAPIKeyName` names `QWEN_API_KEY` in the 401 hint.
- **Chose OpenAI-compat over Model Studio's Anthropic-compatible
`/apps/anthropic` shim** (ADR-0027): on the anthropic client
`ReasoningEffort` is ignored by design, `Request.Schema` rides
`output_config.format` (which the shim does not implement), and cached-token
accounting reads Anthropic-only usage fields. The shim is still reachable
ad hoc via an `anthropic://` DSN.
- `registerOpenAICompatBuiltin` installs BOTH halves of an OpenAI-compat
built-in (eager provider + `name://` DSN scheme via the shared
`openaiCompatScheme`), so the two credential rules — unconditional
`WithAPIKey`, and `WithAPIKeyName` naming that same variable — hold by
construction. kimi and qwen are one line each.
- `envKeyForProvider` is the single definition of the `LLM_<NAME>` form,
shared by lazy resolution (`registry.go`) and the DSN missing-key hint. They
were separate copies with a comment asserting they matched.
- The shared contract is ONE table (`builtin_openaicompat_test.go`), run
identically for every OpenAI-compat built-in: endpoint + bearer, missing key
fails closed naming its own variable with no network hit, the `name://` DSN
reaching another host, and a keyless DSN naming `LLM_<NAME>` rather than the
built-in's key. Adding a built-in is a table row that immediately owes all
four; `builtin_kimi_test.go` was retired into it. Qwen-only tests: the
reverse credential leak, and `reasoning_effort` asserted on the wire body
(the ADR's load-bearing claim).
- Docs in sync: README built-in table + Qwen paragraph + DSN scheme list +
support matrix (footnote ⁴), `.env.example`, ADR-0027 (+ index).
- Consumer: mort wants Qwen as a failover tier.
+5 -3
View File
@@ -207,10 +207,12 @@ func parseSwapReport(header string) []imagegen.SwappedFace {
// imageFilename picks a multipart filename for an image part. The shim reads // imageFilename picks a multipart filename for an image part. The shim reads
// bytes, not names, but a plausible extension keeps server-side sniffing and // bytes, not names, but a plausible extension keeps server-side sniffing and
// request logs honest. base distinguishes the parts of a multi-file form // request logs honest. base distinguishes the parts of a multi-file form
// ("target"/"source") so a log line says which one was malformed. // ("target"/"source", "frame"/"frame_last") so a log line says which one was
// malformed — and, for the video keyframes, so a backend that stages uploads
// by filename cannot have the second overwrite the first.
// //
// initImageFilename (video.go) is this function with base fixed to "frame" // Every caller routes through here: two copies of one extension table is how
// and delegates here — two copies of one extension table is how they drift. // they drift.
func imageFilename(mimeType, base string) string { func imageFilename(mimeType, base string) string {
if base == "" { if base == "" {
base = "image" base = "image"
+1 -1
View File
@@ -56,7 +56,7 @@ func (m *lipsyncModel) Lipsync(ctx context.Context, req videogen.LipsyncRequest,
// hand (mirrors videoModel.Generate). // hand (mirrors videoModel.Generate).
var buf bytes.Buffer var buf bytes.Buffer
w := multipart.NewWriter(&buf) w := multipart.NewWriter(&buf)
fw, err := w.CreateFormFile("image", initImageFilename(req.Image.MIME)) fw, err := w.CreateFormFile("image", imageFilename(req.Image.MIME, "frame"))
if err != nil { if err != nil {
return nil, fmt.Errorf("llama-swap: build lipsync form: %w", err) return nil, fmt.Errorf("llama-swap: build lipsync form: %w", err)
} }
+42 -13
View File
@@ -38,10 +38,13 @@ type videoModel struct {
// bound the call with a context deadline. // bound the call with a context deadline.
// //
// Parameter names follow vLLM-Omni's videos API (num_frames, fps, // Parameter names follow vLLM-Omni's videos API (num_frames, fps,
// num_inference_steps, guidance_scale); the conditioning frame is sent as an // num_inference_steps, guidance_scale); the leading conditioning frame is sent
// `input_reference` file part, following OpenAI's videos API. Upstreams // as an `input_reference` file part, following OpenAI's videos API, and a
// trailing keyframe (Request.LastImage) as `input_reference_last`. Upstreams
// ignore fields they don't understand, and optional fields stay off the wire // ignore fields they don't understand, and optional fields stay off the wire
// entirely so the model's own defaults apply. // entirely so the model's own defaults apply — which is also why a backend
// without first-last-frame support returns an ordinary clip here rather than
// an error.
func (m *videoModel) Generate(ctx context.Context, req videogen.Request, opts ...videogen.Option) (*videogen.Result, error) { func (m *videoModel) Generate(ctx context.Context, req videogen.Request, opts ...videogen.Option) (*videogen.Result, error) {
req = req.Apply(opts...) req = req.Apply(opts...)
if strings.TrimSpace(req.Prompt) == "" { if strings.TrimSpace(req.Prompt) == "" {
@@ -56,6 +59,9 @@ func (m *videoModel) Generate(ctx context.Context, req videogen.Request, opts ..
if req.InitImage != nil && len(req.InitImage.Data) == 0 { if req.InitImage != nil && len(req.InitImage.Data) == 0 {
return nil, fmt.Errorf("%w: video init image has no bytes", llm.ErrUnsupported) return nil, fmt.Errorf("%w: video init image has no bytes", llm.ErrUnsupported)
} }
if req.LastImage != nil && len(req.LastImage.Data) == 0 {
return nil, fmt.Errorf("%w: video last image has no bytes", llm.ErrUnsupported)
}
width, height, err := parseSize(req.Size) width, height, err := parseSize(req.Size)
if err != nil { if err != nil {
return nil, fmt.Errorf("%w: %v", llm.ErrUnsupported, err) return nil, fmt.Errorf("%w: %v", llm.ErrUnsupported, err)
@@ -83,12 +89,19 @@ func (m *videoModel) Generate(ctx context.Context, req videogen.Request, opts ..
return nil, err return nil, err
} }
if req.InitImage != nil { if req.InitImage != nil {
fw, err := w.CreateFormFile("input_reference", initImageFilename(req.InitImage.MIME)) if err := writeImagePart(w, "input_reference", "frame", req.InitImage); err != nil {
if err != nil { return nil, err
return nil, fmt.Errorf("llama-swap: build video form: %w", err)
} }
if _, err := fw.Write(req.InitImage.Data); err != nil { }
return nil, fmt.Errorf("llama-swap: build video form: %w", err) // The trailing keyframe rides a SEPARATE part rather than a second
// `input_reference`: multipart permits repeated names, but the receiving
// end would then have to rely on part ORDER to tell first from last, and
// an ordering contract that is invisible in the field name is one nobody
// can see they have broken. A backend that does not know the name ignores
// the part, which is the same degradation as any other unknown field.
if req.LastImage != nil {
if err := writeImagePart(w, "input_reference_last", "frame_last", req.LastImage); err != nil {
return nil, err
} }
} }
if err := w.Close(); err != nil { if err := w.Close(); err != nil {
@@ -134,11 +147,27 @@ func singleVideoResult(provider, model, verb string, raw []byte, contentType str
return &videogen.Result{Video: videogen.Video{Data: raw, MIME: mimeType}}, nil return &videogen.Result{Video: videogen.Video{Data: raw, MIME: mimeType}}, nil
} }
// initImageFilename picks the multipart filename hint for the conditioning // writeImagePart attaches one conditioning frame under the given field name,
// frame from its MIME subtype. The name is provider-chosen (never // with a filename derived from nameStem. Shared by the first- and last-frame
// caller-supplied), so no sanitization is needed. // parts so the two cannot drift in how they encode, which is the usual way a
func initImageFilename(mimeType string) string { // second copy of a block goes wrong.
return imageFilename(mimeType, "frame") //
// The two frames MUST carry DISTINCT filenames, not merely distinct field
// names. Backends commonly stage an uploaded frame under a name derived from
// the filename — our own ComfyUI shim posts to /upload/image with
// overwrite=true — so two parts sharing "frame.png" would have the second
// clobber the first, and BOTH keyframe inputs would then resolve to the same
// stored image. The clip would render clean, pinned at both ends to the same
// frame, with nothing anywhere reporting a problem.
func writeImagePart(w *multipart.Writer, field, nameStem string, img *videogen.Image) error {
fw, err := w.CreateFormFile(field, imageFilename(img.MIME, nameStem))
if err != nil {
return fmt.Errorf("llama-swap: build video form: %w", err)
}
if _, err := fw.Write(img.Data); err != nil {
return fmt.Errorf("llama-swap: build video form: %w", err)
}
return nil
} }
// formatInt renders an optional int pointer for a form field; nil = "" (omit). // formatInt renders an optional int pointer for a form field; nil = "" (omit).
+131
View File
@@ -223,3 +223,134 @@ func TestVideoGenerateNonVideoBodyErrors(t *testing.T) {
t.Errorf("message = %q, want mention of non-video body", apiErr.Message) t.Errorf("message = %q, want mention of non-video body", apiErr.Message)
} }
} }
// Both keyframes reach the wire, under DISTINCT field names.
//
// The distinct-name property is the actual contract with the backend shim: the
// two frames could have shared one repeated `input_reference` name, and then
// which is first and which is last would depend on multipart part ORDER — an
// ordering contract invisible in the payload, that nothing would notice
// breaking. Asserting the names is what pins it.
func TestVideoGenerateSendsBothKeyframes(t *testing.T) {
var gotFirst, gotLast []byte
var firstName, lastName string
var sawLastPart bool
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if err := r.ParseMultipartForm(32 << 20); err != nil {
t.Errorf("parse form: %v", err)
return
}
if f, hdr, err := r.FormFile("input_reference"); err == nil {
gotFirst, _ = io.ReadAll(f)
firstName = hdr.Filename
f.Close()
}
if f, hdr, err := r.FormFile("input_reference_last"); err == nil {
sawLastPart = true
gotLast, _ = io.ReadAll(f)
lastName = hdr.Filename
f.Close()
}
w.Header().Set("Content-Type", "video/mp4")
_, _ = w.Write([]byte("fake-mp4-bytes"))
}))
defer srv.Close()
p := New(WithBaseURL(srv.URL), WithHTTPClient(srv.Client()))
vm, err := p.VideoModel("videogen-minimax-h3")
if err != nil {
t.Fatalf("VideoModel: %v", err)
}
first, _ := base64.StdEncoding.DecodeString(onePixelPNG)
last := append(append([]byte{}, first...), 0x00) // distinguishable from first
if _, err := vm.Generate(context.Background(), videogen.Request{
Prompt: "a cat surfing",
InitImage: &videogen.Image{MIME: "image/png", Data: first},
LastImage: &videogen.Image{MIME: "image/png", Data: last},
}); err != nil {
t.Fatalf("Generate: %v", err)
}
if !sawLastPart {
t.Fatal("input_reference_last was not sent — a pinned end frame would be silently dropped")
}
if string(gotFirst) != string(first) {
t.Errorf("input_reference = %d bytes, want %d", len(gotFirst), len(first))
}
if string(gotLast) != string(last) {
t.Errorf("input_reference_last = %d bytes, want %d", len(gotLast), len(last))
}
// The two must not be the same bytes, or a swap/aliasing bug reads as a pass.
if string(gotFirst) == string(gotLast) {
t.Error("both parts carry identical bytes — the frames are being aliased")
}
// DISTINCT FILENAMES, not just distinct field names. Backends stage an
// uploaded frame under a name derived from the filename (our ComfyUI shim
// posts to /upload/image with overwrite=true), so two parts sharing
// "frame.png" would have the second clobber the first and BOTH keyframes
// would resolve to the same stored image — a clip pinned at both ends to
// the same frame, rendering cleanly with nothing reporting a fault.
if firstName == "" || lastName == "" {
t.Fatalf("filenames = %q / %q, want both set", firstName, lastName)
}
if firstName == lastName {
t.Errorf("both parts use filename %q — the second upload would clobber the first", firstName)
}
}
// LastImage alone (no InitImage) is a legitimate request: pin the destination
// and let the model invent the approach. It must not require a first frame.
func TestVideoGenerateLastImageAloneIsAllowed(t *testing.T) {
var sawFirst, sawLast bool
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if err := r.ParseMultipartForm(32 << 20); err != nil {
t.Errorf("parse form: %v", err)
return
}
if f, _, err := r.FormFile("input_reference"); err == nil {
sawFirst = true
f.Close()
}
if f, _, err := r.FormFile("input_reference_last"); err == nil {
sawLast = true
f.Close()
}
w.Header().Set("Content-Type", "video/mp4")
_, _ = w.Write([]byte("fake-mp4-bytes"))
}))
defer srv.Close()
p := New(WithBaseURL(srv.URL), WithHTTPClient(srv.Client()))
vm, _ := p.VideoModel("videogen-minimax-h3")
frame, _ := base64.StdEncoding.DecodeString(onePixelPNG)
if _, err := vm.Generate(context.Background(),
videogen.Request{Prompt: "arrive here"},
videogen.WithLastImage(videogen.Image{MIME: "image/png", Data: frame}),
); err != nil {
t.Fatalf("Generate: %v", err)
}
if sawFirst {
t.Error("input_reference sent, want omitted")
}
if !sawLast {
t.Error("input_reference_last omitted, want sent")
}
}
// An empty LastImage is rejected before the request is built, matching
// InitImage's existing contract — a zero-byte frame reaching the backend is a
// confusing upstream error instead of a clear local one.
func TestVideoGenerateRejectsEmptyLastImage(t *testing.T) {
p := New(WithBaseURL("http://unused"))
vm, _ := p.VideoModel("videogen-minimax-h3")
_, err := vm.Generate(context.Background(), videogen.Request{
Prompt: "x",
LastImage: &videogen.Image{MIME: "image/png"},
})
if !errors.Is(err, llm.ErrUnsupported) {
t.Fatalf("err = %v, want llm.ErrUnsupported", err)
}
}
+1 -1
View File
@@ -263,7 +263,7 @@ func (r *Registry) providerFor(name string) (llm.Provider, error) {
return nil, envErr return nil, envErr
} }
envKey := "LLM_" + strings.ToUpper(strings.ReplaceAll(name, "-", "_")) envKey := envKeyForProvider(name)
envVal := r.envLookup(envKey) envVal := r.envLookup(envKey)
if envVal == "" { if envVal == "" {
return nil, fmt.Errorf("%w: %q (checked registry and %s env var)", ErrUnknownProvider, name, envKey) return nil, fmt.Errorf("%w: %q (checked registry and %s env var)", ErrUnknownProvider, name, envKey)
+19
View File
@@ -12,6 +12,8 @@
// InitImage is a pure text prompt, a non-nil InitImage conditions generation // InitImage is a pure text prompt, a non-nil InitImage conditions generation
// on that frame. Hybrid models (e.g. Wan 2.2 TI2V) serve both from the same // on that frame. Hybrid models (e.g. Wan 2.2 TI2V) serve both from the same
// checkpoint, so unlike imagegen there is no separate Editor-style interface. // checkpoint, so unlike imagegen there is no separate Editor-style interface.
// LastImage extends the same surface to the other end of the clip, so one
// Request covers t2v, i2v, and first-last-frame-to-video without a mode flag.
// //
// The first implementation is provider/llamaswap, which targets the blocking // The first implementation is provider/llamaswap, which targets the blocking
// OpenAI/vLLM-Omni-style POST /v1/videos/sync endpoint: the response body is // OpenAI/vLLM-Omni-style POST /v1/videos/sync endpoint: the response body is
@@ -51,6 +53,19 @@ type Request struct {
// nil = pure text-to-video. // nil = pure text-to-video.
InitImage *Image InitImage *Image
// LastImage conditions generation on an ENDING frame. With InitImage it
// pins both ends (first-last-frame-to-video); alone it pins only the
// destination and lets the backend invent the approach.
//
// Support is per-model and NOT advertised anywhere in this contract: a
// backend that does not understand a trailing keyframe ignores it and
// returns an ordinary clip, which is indistinguishable from success.
// There is no capability bit to consult, because the contract has no way
// to learn one. A caller that needs to know whether the pin actually took
// effect must establish that out of band — by configuration it controls,
// not by inspecting the result.
LastImage *Image
// Size is the requested resolution, e.g. "1280x704"; "" = backend default. // Size is the requested resolution, e.g. "1280x704"; "" = backend default.
Size string Size string
@@ -92,6 +107,10 @@ type Option func(*Request)
// WithInitImage conditions generation on a starting frame (image-to-video). // WithInitImage conditions generation on a starting frame (image-to-video).
func WithInitImage(img Image) Option { return func(r *Request) { r.InitImage = &img } } func WithInitImage(img Image) Option { return func(r *Request) { r.InitImage = &img } }
// WithLastImage conditions generation on an ending frame. Combined with
// WithInitImage this pins both ends of the clip.
func WithLastImage(img Image) Option { return func(r *Request) { r.LastImage = &img } }
// WithSize sets the requested resolution (e.g. "1280x704"). // WithSize sets the requested resolution (e.g. "1280x704").
func WithSize(size string) Option { return func(r *Request) { r.Size = size } } func WithSize(size string) Option { return func(r *Request) { r.Size = size } }