Compare commits
13
Commits
203895696c
..
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
85561ab477 | ||
|
|
f837115a55 | ||
|
|
f8ced9c629 | ||
|
|
0760cf96d4 | ||
|
|
8670ed22be | ||
|
|
f1f2b653c3 | ||
|
|
31d6b59356 | ||
|
|
02cd561eaf | ||
|
|
e779169416 | ||
|
|
5994d96921 | ||
|
|
588e092465 | ||
|
|
dbc96898ab | ||
|
|
44fcfbb273 |
@@ -7,6 +7,7 @@ OLLAMA_API_KEY=your-ollama-cloud-key-here
|
|||||||
# Built-in provider keys (each optional; only needed for the providers you use).
|
# Built-in provider keys (each optional; only needed for the providers you use).
|
||||||
#OPENAI_API_KEY=sk-...
|
#OPENAI_API_KEY=sk-...
|
||||||
#KIMI_API_KEY=sk-... # Moonshot AI (Kimi); provider name "kimi"
|
#KIMI_API_KEY=sk-... # Moonshot AI (Kimi); provider name "kimi"
|
||||||
|
#QWEN_API_KEY=sk-... # Alibaba Model Studio (Qwen); provider name "qwen"
|
||||||
#ANTHROPIC_API_KEY=sk-ant-...
|
#ANTHROPIC_API_KEY=sk-ant-...
|
||||||
#GOOGLE_API_KEY=...
|
#GOOGLE_API_KEY=...
|
||||||
|
|
||||||
|
|||||||
@@ -28,3 +28,6 @@ go.work.sum
|
|||||||
|
|
||||||
# macOS
|
# macOS
|
||||||
.DS_Store
|
.DS_Store
|
||||||
|
|
||||||
|
# Local worktrees created for agent work — never part of the repo.
|
||||||
|
.claude/
|
||||||
|
|||||||
@@ -122,6 +122,7 @@ Chains are health-tracked per target:
|
|||||||
|----------|-----------|-------------|------------------|
|
|----------|-----------|-------------|------------------|
|
||||||
| OpenAI (+compatible) | `openai` | `OPENAI_API_KEY` | https://api.openai.com/v1 |
|
| OpenAI (+compatible) | `openai` | `OPENAI_API_KEY` | https://api.openai.com/v1 |
|
||||||
| Kimi (Moonshot AI) | `kimi` | `KIMI_API_KEY` | https://api.moonshot.ai/v1 |
|
| Kimi (Moonshot AI) | `kimi` | `KIMI_API_KEY` | https://api.moonshot.ai/v1 |
|
||||||
|
| Qwen (Alibaba) | `qwen` | `QWEN_API_KEY` | https://dashscope-intl.aliyuncs.com/compatible-mode/v1 |
|
||||||
| Anthropic (+compatible) | `anthropic` | `ANTHROPIC_API_KEY` | https://api.anthropic.com |
|
| Anthropic (+compatible) | `anthropic` | `ANTHROPIC_API_KEY` | https://api.anthropic.com |
|
||||||
| Google (Gemini) | `google` | `GOOGLE_API_KEY` / `GEMINI_API_KEY` | Gemini API (official SDK) |
|
| Google (Gemini) | `google` | `GOOGLE_API_KEY` / `GEMINI_API_KEY` | Gemini API (official SDK) |
|
||||||
| Ollama Cloud | `ollama-cloud` | `OLLAMA_API_KEY` | https://ollama.com |
|
| Ollama Cloud | `ollama-cloud` | `OLLAMA_API_KEY` | https://ollama.com |
|
||||||
@@ -134,6 +135,19 @@ the openai client (like llama-swap). The `kimi` built-in defaults to the
|
|||||||
international endpoint; reach the China endpoint (or any other host) with a
|
international endpoint; reach the China endpoint (or any other host) with a
|
||||||
`kimi://` DSN, e.g. `LLM_KCN=kimi://[email protected]/v1`.
|
`kimi://` DSN, e.g. `LLM_KCN=kimi://[email protected]/v1`.
|
||||||
|
|
||||||
|
Qwen is the same shape: Alibaba Model Studio's OpenAI-compatible mode, reusing
|
||||||
|
the openai client. The `qwen` built-in defaults to the international
|
||||||
|
(Singapore) host; reach the China host or a workspace-scoped regional one with
|
||||||
|
a `qwen://` DSN, e.g.
|
||||||
|
`LLM_QCN=qwen://[email protected]/compatible-mode/v1`. Model Studio
|
||||||
|
also fronts the same models with an Anthropic-compatible `/v1/messages` shim —
|
||||||
|
majordomo does **not** use it, because on that surface `reasoning_effort` is
|
||||||
|
dropped, `Request.Schema` stops being enforced, and cached-token accounting
|
||||||
|
disappears; see [ADR-0027](docs/adr/0027-qwen-builtin.md). Two Alibaba-side
|
||||||
|
quirks are worth knowing: thinking is on by default for some models (e.g.
|
||||||
|
`qwen3.7-plus`), and the Qwen3 open-source models require streaming while
|
||||||
|
thinking, so buffered `Generate` calls want a Max/Plus model.
|
||||||
|
|
||||||
OpenAI-compatible / Anthropic-compatible endpoints: construct the provider
|
OpenAI-compatible / Anthropic-compatible endpoints: construct the provider
|
||||||
with a name and base URL and register it —
|
with a name and base URL and register it —
|
||||||
|
|
||||||
@@ -165,7 +179,7 @@ m, _ := reg.Parse("m5/qwen3:30b,m1/qwen3:30b,thinking")
|
|||||||
```
|
```
|
||||||
|
|
||||||
DSN format: `scheme://[token@]host[/path]`, scheme ∈ `foreman`, `ollama`,
|
DSN format: `scheme://[token@]host[/path]`, scheme ∈ `foreman`, `ollama`,
|
||||||
`ollama-cloud`, `openai`, `kimi`, `anthropic`, `google`/`gemini`, `llama-swap`,
|
`ollama-cloud`, `openai`, `kimi`, `qwen`, `anthropic`, `google`/`gemini`, `llama-swap`,
|
||||||
`llama-swaps`, or any scheme you add with `RegisterScheme`. The token is the
|
`llama-swaps`, or any scheme you add with `RegisterScheme`. The token is the
|
||||||
credential (bearer token / API key); the base URL is always `https://host[/path]`
|
credential (bearer token / API key); the base URL is always `https://host[/path]`
|
||||||
— except `llama-swap`, which builds `http://host[:port]` since it's local-first
|
— except `llama-swap`, which builds `http://host[:port]` since it's local-first
|
||||||
@@ -272,17 +286,30 @@ tr, err := tm.Transcribe(ctx, audio.TranscriptionRequest{
|
|||||||
voices, err := ls.ListVoices(ctx, "kokoro") // []string of voice ids
|
voices, err := ls.ListVoices(ctx, "kokoro") // []string of voice ids
|
||||||
```
|
```
|
||||||
|
|
||||||
## Video: text-to-video + image-to-video
|
## Video: text-to-video, image-to-video, first-last-frame
|
||||||
|
|
||||||
Video generation lives in the `videogen` package (ADR-0019), mirroring
|
Video generation lives in the `videogen` package (ADR-0019), mirroring
|
||||||
imagegen/audio: one small `Model` contract, zero values mean backend
|
imagegen/audio: one small `Model` contract, zero values mean backend
|
||||||
defaults, bytes in/out. Text-to-video and image-to-video are one surface —
|
defaults, bytes in/out. All modes are one surface, selected by which
|
||||||
a nil `InitImage` is a pure text prompt; setting it conditions generation
|
keyframes are set rather than by a mode flag:
|
||||||
on that frame (hybrid checkpoints like Wan 2.2 TI2V serve both). First
|
|
||||||
backend: llama-swap (blocking `/v1/videos/sync`, vLLM-Omni style — the
|
| `InitImage` | `LastImage` | mode |
|
||||||
|
|---|---|---|
|
||||||
|
| nil | nil | text-to-video |
|
||||||
|
| set | nil | image-to-video (hybrid checkpoints like Wan 2.2 TI2V serve both) |
|
||||||
|
| set | set | first-last-frame — both ends pinned |
|
||||||
|
| nil | set | pin the destination, model invents the approach |
|
||||||
|
|
||||||
|
First backend: llama-swap (blocking `/v1/videos/sync`, vLLM-Omni style — the
|
||||||
response body is the encoded clip, so `Result` carries a single `Video`).
|
response body is the encoded clip, so `Result` carries a single `Video`).
|
||||||
Generation runs for minutes; bound the call with a context deadline.
|
Generation runs for minutes; bound the call with a context deadline.
|
||||||
|
|
||||||
|
**`LastImage` support is per-model and cannot be detected.** A backend that
|
||||||
|
does not understand a trailing keyframe ignores the part and returns an
|
||||||
|
ordinary clip — indistinguishable from success. There is no capability bit,
|
||||||
|
because the contract has no way to learn one, so a caller depending on the
|
||||||
|
pin must establish support out of band.
|
||||||
|
|
||||||
```go
|
```go
|
||||||
vm, _ := ls.VideoModel("videogen-wan22-5b")
|
vm, _ := ls.VideoModel("videogen-wan22-5b")
|
||||||
res, err := vm.Generate(ctx, videogen.Request{Prompt: "a cat surfing"},
|
res, err := vm.Generate(ctx, videogen.Request{Prompt: "a cat surfing"},
|
||||||
@@ -407,6 +434,7 @@ to build one.
|
|||||||
|----------------------|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
|
|----------------------|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
|
||||||
| OpenAI (+compatible) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
|
| OpenAI (+compatible) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||||||
| Kimi (Moonshot AI) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅³ | ✅ |
|
| Kimi (Moonshot AI) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅³ | ✅ |
|
||||||
|
| Qwen (Alibaba) | ✅ | ✅ | ✅ | ✅ | ✅⁴ | ✅⁴ | ✅ |
|
||||||
| Anthropic (+compat) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
|
| Anthropic (+compat) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||||||
| Google (Gemini) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
|
| Google (Gemini) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||||||
| Ollama Cloud | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
|
| Ollama Cloud | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||||||
@@ -431,6 +459,13 @@ probe and management methods on `*llamaswap.Provider`.
|
|||||||
level; whether a call succeeds depends on the Moonshot model — only the vision
|
level; whether a call succeeds depends on the Moonshot model — only the vision
|
||||||
variants (e.g. `moonshot-v1-8k-vision-preview`) accept images.
|
variants (e.g. `moonshot-v1-8k-vision-preview`) accept images.
|
||||||
|
|
||||||
|
⁴ Qwen also reuses the openai client (ADR-0027), so both columns are present at
|
||||||
|
the client level and gated by the Model Studio model you name: `json_schema`
|
||||||
|
structured output is on the Max/Plus families, image inputs on the `qwen-vl-*`
|
||||||
|
/ `qwen3-vl-*` models. `reasoning_effort` rides through as a top-level field —
|
||||||
|
one reason the built-in speaks OpenAI-compat rather than Model Studio's
|
||||||
|
Anthropic-compat shim.
|
||||||
|
|
||||||
Notes: Ollama has no native tool_choice — `"none"` drops the tools;
|
Notes: Ollama has no native tool_choice — `"none"` drops the tools;
|
||||||
`"required"`/named choices are best-effort ignored there. Ollama Cloud
|
`"required"`/named choices are best-effort ignored there. Ollama Cloud
|
||||||
ignores the `format` field (verified live), so the provider also states
|
ignores the `format` field (verified live), so the provider also states
|
||||||
|
|||||||
+68
-27
@@ -2,7 +2,6 @@ package majordomo
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"net/http"
|
"net/http"
|
||||||
"strings"
|
|
||||||
|
|
||||||
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
|
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
|
||||||
"gitea.stevedudenhoeffer.com/steve/majordomo/provider/anthropic"
|
"gitea.stevedudenhoeffer.com/steve/majordomo/provider/anthropic"
|
||||||
@@ -19,6 +18,12 @@ const (
|
|||||||
// Chat Completions endpoint. Reuses the openai client (like llama-swap);
|
// Chat Completions endpoint. Reuses the openai client (like llama-swap);
|
||||||
// keyed by KIMI_API_KEY, default base URL kimiBaseURL.
|
// keyed by KIMI_API_KEY, default base URL kimiBaseURL.
|
||||||
ProviderKimi = "kimi"
|
ProviderKimi = "kimi"
|
||||||
|
// ProviderQwen is Alibaba's Qwen models over Model Studio's
|
||||||
|
// OpenAI-compatible Chat Completions endpoint. Reuses the openai client
|
||||||
|
// (like kimi and llama-swap); keyed by QWEN_API_KEY, default base URL
|
||||||
|
// qwenBaseURL. ADR-0027 records why the OpenAI surface and not the
|
||||||
|
// Anthropic-compatible one Model Studio also exposes.
|
||||||
|
ProviderQwen = "qwen"
|
||||||
ProviderAnthropic = "anthropic"
|
ProviderAnthropic = "anthropic"
|
||||||
ProviderGoogle = "google"
|
ProviderGoogle = "google"
|
||||||
ProviderOllama = "ollama"
|
ProviderOllama = "ollama"
|
||||||
@@ -37,6 +42,55 @@ const (
|
|||||||
// China endpoint (api.moonshot.cn/v1) is reachable via a kimi:// LLM_* DSN.
|
// China endpoint (api.moonshot.cn/v1) is reachable via a kimi:// LLM_* DSN.
|
||||||
const kimiBaseURL = "https://api.moonshot.ai/v1"
|
const kimiBaseURL = "https://api.moonshot.ai/v1"
|
||||||
|
|
||||||
|
// qwenBaseURL is Alibaba Model Studio's international (Singapore) endpoint in
|
||||||
|
// OpenAI-compatible mode. The China endpoint
|
||||||
|
// (dashscope.aliyuncs.com/compatible-mode/v1) and any regional host are
|
||||||
|
// reachable via a qwen:// LLM_* DSN.
|
||||||
|
const qwenBaseURL = "https://dashscope-intl.aliyuncs.com/compatible-mode/v1"
|
||||||
|
|
||||||
|
// openaiCompatScheme builds the DSN factory shared by every built-in that is
|
||||||
|
// "the openai client pointed somewhere else" (kimi, qwen, ...). The provider
|
||||||
|
// is named after the LLM_<NAME> var that defined it, takes its credential from
|
||||||
|
// the DSN token — not the built-in's own env var, which does nothing for a
|
||||||
|
// DSN-defined provider — and so names that same LLM_<NAME> var in the
|
||||||
|
// missing-key hint, matching the lazy-resolution key form in providerFor.
|
||||||
|
//
|
||||||
|
// wrap is the caller's option-decorator (it injects the registry's HTTP
|
||||||
|
// client), so a DSN provider is built exactly like the eager built-ins.
|
||||||
|
func openaiCompatScheme(wrap func(...openai.Option) []openai.Option) SchemeFactory {
|
||||||
|
return func(name string, dsn DSN) (llm.Provider, error) {
|
||||||
|
return openai.New(wrap(
|
||||||
|
openai.WithName(name),
|
||||||
|
openai.WithBaseURL(dsn.BaseURL()),
|
||||||
|
openai.WithAPIKey(dsn.Token),
|
||||||
|
openai.WithAPIKeyName(envKeyForProvider(name)),
|
||||||
|
)...), nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// registerOpenAICompatBuiltin installs BOTH halves of an OpenAI-compat
|
||||||
|
// built-in: the eager provider under name (credential from keyEnv) and the
|
||||||
|
// matching name:// DSN scheme. Why both in one call: the two halves are a pair
|
||||||
|
// — a built-in whose scheme is missing resolves as a spec but not from an
|
||||||
|
// LLM_* DSN, and the credential rules below have to hold identically in each.
|
||||||
|
// Adding the next one is a single line rather than six lines to copy.
|
||||||
|
//
|
||||||
|
// The two credential rules, holding by construction for every caller:
|
||||||
|
// - WithAPIKey is passed UNCONDITIONALLY, even when the lookup comes back
|
||||||
|
// empty. openai.New defaults its key to OPENAI_API_KEY, so anything less
|
||||||
|
// lets an unset keyEnv silently authenticate as OpenAI.
|
||||||
|
// - WithAPIKeyName makes the synthetic-401 hint name keyEnv, so a keyless
|
||||||
|
// call tells the operator the variable that actually fixes it.
|
||||||
|
func registerOpenAICompatBuiltin(r *Registry, wrap func(...openai.Option) []openai.Option, name, baseURL, keyEnv string) {
|
||||||
|
r.providers[name] = openai.New(wrap(
|
||||||
|
openai.WithName(name),
|
||||||
|
openai.WithBaseURL(baseURL),
|
||||||
|
openai.WithAPIKey(r.envLookup(keyEnv)),
|
||||||
|
openai.WithAPIKeyName(keyEnv),
|
||||||
|
)...)
|
||||||
|
r.schemes[name] = openaiCompatScheme(wrap)
|
||||||
|
}
|
||||||
|
|
||||||
// registerBuiltins installs the built-in providers and env-DSN scheme
|
// registerBuiltins installs the built-in providers and env-DSN scheme
|
||||||
// factories into a fresh registry. httpClient, when non-nil, is used by
|
// factories into a fresh registry. httpClient, when non-nil, is used by
|
||||||
// every provider and factory the registry itself constructs.
|
// every provider and factory the registry itself constructs.
|
||||||
@@ -83,32 +137,19 @@ func registerBuiltins(r *Registry, httpClient *http.Client) {
|
|||||||
)...), nil
|
)...), nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// Kimi (Moonshot AI): OpenAI-compatible Chat Completions, so it reuses the
|
// Third-party endpoints that ARE the openai client at another base URL —
|
||||||
// openai client (like llama-swap). Defaults to Moonshot's international
|
// no new package, mirroring llama-swap's chat path. Each gets the eager
|
||||||
// endpoint and the KIMI_API_KEY credential. WithAPIKey is passed
|
// built-in plus its name:// DSN scheme, and the credential rules hold by
|
||||||
// unconditionally — even empty — so an unset KIMI_API_KEY can never fall
|
// construction (see registerOpenAICompatBuiltin).
|
||||||
// through to the openai client's OPENAI_API_KEY default; WithAPIKeyName
|
//
|
||||||
// makes the missing-key error name KIMI_API_KEY.
|
// kimi (ADR-0026): Moonshot's international endpoint; China host via
|
||||||
r.providers[ProviderKimi] = openai.New(openaiOpts(
|
// kimi://[email protected]/v1.
|
||||||
openai.WithName(ProviderKimi),
|
registerOpenAICompatBuiltin(r, openaiOpts, ProviderKimi, kimiBaseURL, "KIMI_API_KEY")
|
||||||
openai.WithBaseURL(kimiBaseURL),
|
// qwen (ADR-0027): Alibaba Model Studio's international host. Model Studio
|
||||||
openai.WithAPIKey(r.envLookup("KIMI_API_KEY")),
|
// also exposes an Anthropic-compatible endpoint; the ADR records why the
|
||||||
openai.WithAPIKeyName("KIMI_API_KEY"),
|
// OpenAI one is the built-in. China / workspace-scoped regional hosts via
|
||||||
)...)
|
// qwen://[email protected]/compatible-mode/v1.
|
||||||
// kimi:// DSN scheme: an OpenAI-compatible target labeled kimi, base URL
|
registerOpenAICompatBuiltin(r, openaiOpts, ProviderQwen, qwenBaseURL, "QWEN_API_KEY")
|
||||||
// from the DSN host (e.g. kimi://[email protected]/v1 for China). Its
|
|
||||||
// credential is the DSN token, not KIMI_API_KEY, so the missing-key hint
|
|
||||||
// names the LLM_<NAME> env var that defines this provider (matching the
|
|
||||||
// lazy-resolution key form in providerFor) — the fix for a keyless target
|
|
||||||
// here is adding a token to that DSN.
|
|
||||||
r.schemes[ProviderKimi] = func(name string, dsn DSN) (llm.Provider, error) {
|
|
||||||
return openai.New(openaiOpts(
|
|
||||||
openai.WithName(name),
|
|
||||||
openai.WithBaseURL(dsn.BaseURL()),
|
|
||||||
openai.WithAPIKey(dsn.Token),
|
|
||||||
openai.WithAPIKeyName("LLM_"+strings.ToUpper(strings.ReplaceAll(name, "-", "_"))),
|
|
||||||
)...), nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// llama-swap: OpenAI-compatible chat + image generation + management
|
// llama-swap: OpenAI-compatible chat + image generation + management
|
||||||
// endpoints over a model-swapping proxy. Chat reuses the openai client
|
// endpoints over a model-swapping proxy. Chat reuses the openai client
|
||||||
|
|||||||
@@ -1,171 +0,0 @@
|
|||||||
package majordomo
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"errors"
|
|
||||||
"io"
|
|
||||||
"net/http"
|
|
||||||
"strings"
|
|
||||||
"testing"
|
|
||||||
|
|
||||||
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
|
|
||||||
)
|
|
||||||
|
|
||||||
// kimiResponse is a minimal valid Chat Completions body so Generate returns a
|
|
||||||
// non-empty response (an empty one would trigger failover, not a clean pass).
|
|
||||||
const kimiResponse = `{"id":"c1","object":"chat.completion","choices":[` +
|
|
||||||
`{"index":0,"message":{"role":"assistant","content":"ok"},"finish_reason":"stop"}]}`
|
|
||||||
|
|
||||||
// captureRT records the last request and returns a canned response without
|
|
||||||
// touching the network, so these tests stay hermetic while still exercising
|
|
||||||
// the real openai client the kimi built-in reuses (base URL + auth header).
|
|
||||||
type captureRT struct {
|
|
||||||
req *http.Request
|
|
||||||
body string
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *captureRT) RoundTrip(r *http.Request) (*http.Response, error) {
|
|
||||||
c.req = r
|
|
||||||
return &http.Response{
|
|
||||||
StatusCode: http.StatusOK,
|
|
||||||
Body: io.NopCloser(strings.NewReader(c.body)),
|
|
||||||
Header: make(http.Header),
|
|
||||||
Request: r,
|
|
||||||
}, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestKimiBuiltin: the built-in "kimi" provider resolves in Parse, targets
|
|
||||||
// Moonshot's default endpoint, and authenticates with KIMI_API_KEY.
|
|
||||||
func TestKimiBuiltin(t *testing.T) {
|
|
||||||
rt := &captureRT{body: kimiResponse}
|
|
||||||
r := newTestRegistry(t,
|
|
||||||
WithEnvLookup(func(k string) string {
|
|
||||||
if k == "KIMI_API_KEY" {
|
|
||||||
return "kimi-secret"
|
|
||||||
}
|
|
||||||
return ""
|
|
||||||
}),
|
|
||||||
WithHTTPClient(&http.Client{Transport: rt}),
|
|
||||||
)
|
|
||||||
|
|
||||||
if p, ok := r.Provider(ProviderKimi); !ok {
|
|
||||||
t.Fatal("built-in kimi provider not registered")
|
|
||||||
} else if p.Name() != ProviderKimi {
|
|
||||||
t.Errorf("name = %q, want %q", p.Name(), ProviderKimi)
|
|
||||||
}
|
|
||||||
|
|
||||||
m, err := r.Parse("kimi/kimi-k2-0711-preview")
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("Parse: %v", err)
|
|
||||||
}
|
|
||||||
if got := targetsOf(t, m); len(got) != 1 || got[0] != "kimi/kimi-k2-0711-preview" {
|
|
||||||
t.Fatalf("targets = %v", got)
|
|
||||||
}
|
|
||||||
|
|
||||||
if _, err := m.Generate(context.Background(), llm.Request{Messages: []llm.Message{llm.UserText("hi")}}); err != nil {
|
|
||||||
t.Fatalf("Generate: %v", err)
|
|
||||||
}
|
|
||||||
if rt.req == nil {
|
|
||||||
t.Fatal("no request captured")
|
|
||||||
}
|
|
||||||
if want := "https://api.moonshot.ai/v1/chat/completions"; rt.req.URL.String() != want {
|
|
||||||
t.Errorf("URL = %q, want %q", rt.req.URL.String(), want)
|
|
||||||
}
|
|
||||||
if want := "Bearer kimi-secret"; rt.req.Header.Get("Authorization") != want {
|
|
||||||
t.Errorf("Authorization = %q, want %q", rt.req.Header.Get("Authorization"), want)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestKimiBuiltinMissingKey: with no KIMI_API_KEY the built-in fails fast with a
|
|
||||||
// synthetic 401 whose hint names KIMI_API_KEY — never OPENAI_API_KEY (proving
|
|
||||||
// the credential does not fall through to the openai client's default), and
|
|
||||||
// without hitting the network.
|
|
||||||
func TestKimiBuiltinMissingKey(t *testing.T) {
|
|
||||||
rt := &captureRT{body: kimiResponse}
|
|
||||||
r := newTestRegistry(t, WithHTTPClient(&http.Client{Transport: rt}))
|
|
||||||
|
|
||||||
m, err := r.Parse("kimi/kimi-k2-0711-preview")
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("Parse: %v", err)
|
|
||||||
}
|
|
||||||
_, err = m.Generate(context.Background(), llm.Request{Messages: []llm.Message{llm.UserText("hi")}})
|
|
||||||
apiErr, ok := errors.AsType[*llm.APIError](err)
|
|
||||||
if !ok {
|
|
||||||
t.Fatalf("err = %v (%T), want *llm.APIError", err, err)
|
|
||||||
}
|
|
||||||
if apiErr.Status != http.StatusUnauthorized || apiErr.Code != "missing_api_key" {
|
|
||||||
t.Errorf("Status/Code = %d/%q, want 401/missing_api_key", apiErr.Status, apiErr.Code)
|
|
||||||
}
|
|
||||||
if !strings.Contains(apiErr.Message, "KIMI_API_KEY") {
|
|
||||||
t.Errorf("message = %q, want it to name KIMI_API_KEY", apiErr.Message)
|
|
||||||
}
|
|
||||||
if strings.Contains(apiErr.Message, "OPENAI_API_KEY") {
|
|
||||||
t.Errorf("message = %q, must not name OPENAI_API_KEY", apiErr.Message)
|
|
||||||
}
|
|
||||||
if rt.req != nil {
|
|
||||||
t.Error("network was hit despite missing key")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestKimiScheme: a kimi:// LLM_* DSN defines a named provider on any Moonshot
|
|
||||||
// host (here the China endpoint) that is first-class in Parse and carries the
|
|
||||||
// DSN token as its bearer credential.
|
|
||||||
func TestKimiScheme(t *testing.T) {
|
|
||||||
rt := &captureRT{body: kimiResponse}
|
|
||||||
r := newTestRegistry(t, WithHTTPClient(&http.Client{Transport: rt}))
|
|
||||||
if err := r.LoadEnv(map[string]string{
|
|
||||||
"LLM_KCN": "kimi://[email protected]/v1",
|
|
||||||
}); err != nil {
|
|
||||||
t.Fatalf("LoadEnv: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
m, err := r.Parse("kcn/moonshot-v1-8k")
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("Parse: %v", err)
|
|
||||||
}
|
|
||||||
if _, err := m.Generate(context.Background(), llm.Request{Messages: []llm.Message{llm.UserText("hi")}}); err != nil {
|
|
||||||
t.Fatalf("Generate: %v", err)
|
|
||||||
}
|
|
||||||
if rt.req == nil {
|
|
||||||
t.Fatal("no request captured")
|
|
||||||
}
|
|
||||||
if want := "https://api.moonshot.cn/v1/chat/completions"; rt.req.URL.String() != want {
|
|
||||||
t.Errorf("URL = %q, want %q", rt.req.URL.String(), want)
|
|
||||||
}
|
|
||||||
if want := "Bearer tok"; rt.req.Header.Get("Authorization") != want {
|
|
||||||
t.Errorf("Authorization = %q, want %q", rt.req.Header.Get("Authorization"), want)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestKimiSchemeMissingToken: a kimi:// DSN with no token is fixed by adding one
|
|
||||||
// to the DSN, not by setting KIMI_API_KEY — so the missing-key hint names the
|
|
||||||
// defining LLM_<NAME> env var, never KIMI_API_KEY (which does nothing for a
|
|
||||||
// DSN-defined provider).
|
|
||||||
func TestKimiSchemeMissingToken(t *testing.T) {
|
|
||||||
rt := &captureRT{body: kimiResponse}
|
|
||||||
r := newTestRegistry(t, WithHTTPClient(&http.Client{Transport: rt}))
|
|
||||||
if err := r.LoadEnv(map[string]string{
|
|
||||||
"LLM_KCN": "kimi://api.moonshot.cn/v1", // no token
|
|
||||||
}); err != nil {
|
|
||||||
t.Fatalf("LoadEnv: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
m, err := r.Parse("kcn/moonshot-v1-8k")
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("Parse: %v", err)
|
|
||||||
}
|
|
||||||
_, err = m.Generate(context.Background(), llm.Request{Messages: []llm.Message{llm.UserText("hi")}})
|
|
||||||
apiErr, ok := errors.AsType[*llm.APIError](err)
|
|
||||||
if !ok {
|
|
||||||
t.Fatalf("err = %v (%T), want *llm.APIError", err, err)
|
|
||||||
}
|
|
||||||
if !strings.Contains(apiErr.Message, "LLM_KCN") {
|
|
||||||
t.Errorf("message = %q, want it to name LLM_KCN", apiErr.Message)
|
|
||||||
}
|
|
||||||
if strings.Contains(apiErr.Message, "KIMI_API_KEY") {
|
|
||||||
t.Errorf("message = %q, must not name KIMI_API_KEY for a DSN provider", apiErr.Message)
|
|
||||||
}
|
|
||||||
if rt.req != nil {
|
|
||||||
t.Error("network was hit despite missing token")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -0,0 +1,240 @@
|
|||||||
|
package majordomo
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"io"
|
||||||
|
"net/http"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Shared fixtures and the shared contract for the built-ins that are "the
|
||||||
|
// openai client pointed somewhere else" (kimi, qwen, ...). They live here
|
||||||
|
// rather than in any one provider's test file so a new OpenAI-compat built-in
|
||||||
|
// has nothing to copy — the same reason registerOpenAICompatBuiltin exists on
|
||||||
|
// the production side.
|
||||||
|
|
||||||
|
// chatCompletionOK is a minimal valid Chat Completions body, so Generate
|
||||||
|
// returns a non-empty response (an empty one would trigger failover, not a
|
||||||
|
// clean pass).
|
||||||
|
const chatCompletionOK = `{"id":"c1","object":"chat.completion","choices":[` +
|
||||||
|
`{"index":0,"message":{"role":"assistant","content":"ok"},"finish_reason":"stop"}]}`
|
||||||
|
|
||||||
|
// captureRT records the last request (and the bytes of its body) and returns a
|
||||||
|
// canned response without touching the network, so these tests stay hermetic
|
||||||
|
// while still exercising the real openai client the built-ins reuse: base URL,
|
||||||
|
// auth header, and the JSON actually put on the wire.
|
||||||
|
type captureRT struct {
|
||||||
|
req *http.Request
|
||||||
|
reqBody []byte
|
||||||
|
body string
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *captureRT) RoundTrip(r *http.Request) (*http.Response, error) {
|
||||||
|
c.req = r
|
||||||
|
// Drain and close the request body: a RoundTripper owns it, and those
|
||||||
|
// bytes are what wire-shape assertions read.
|
||||||
|
c.reqBody = nil
|
||||||
|
if r.Body != nil {
|
||||||
|
c.reqBody, _ = io.ReadAll(r.Body)
|
||||||
|
_ = r.Body.Close()
|
||||||
|
}
|
||||||
|
return &http.Response{
|
||||||
|
StatusCode: http.StatusOK,
|
||||||
|
Body: io.NopCloser(strings.NewReader(c.body)),
|
||||||
|
Header: make(http.Header),
|
||||||
|
Request: r,
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// singleKeyEnv builds a WithEnvLookup function that knows exactly one variable
|
||||||
|
// and returns "" for everything else. The empty default has teeth: a built-in
|
||||||
|
// that reached for any other variable name gets nothing, so the request 401s
|
||||||
|
// and the test fails rather than quietly authenticating off the wrong key.
|
||||||
|
func singleKeyEnv(key, value string) func(string) string {
|
||||||
|
return func(k string) string {
|
||||||
|
if k == key {
|
||||||
|
return value
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// openAICompatBuiltin describes one built-in for the shared contract below.
|
||||||
|
// Adding an OpenAI-compat built-in means adding a row here — not copying a
|
||||||
|
// test file, which is how kimi's and qwen's suites became near-identical.
|
||||||
|
type openAICompatBuiltin struct {
|
||||||
|
name string // registry name and spec prefix
|
||||||
|
keyEnv string // the credential variable this built-in reads
|
||||||
|
model string // a current model id for that endpoint
|
||||||
|
wantURL string // chat-completions URL the default endpoint must produce
|
||||||
|
|
||||||
|
// The name:// DSN case: an alternate host (regional/China endpoint)
|
||||||
|
// reached through an LLM_<dsnVar> definition.
|
||||||
|
dsnVar string
|
||||||
|
dsnHost string
|
||||||
|
wantDSNURL string
|
||||||
|
}
|
||||||
|
|
||||||
|
var openAICompatBuiltins = []openAICompatBuiltin{
|
||||||
|
{
|
||||||
|
name: ProviderKimi,
|
||||||
|
keyEnv: "KIMI_API_KEY",
|
||||||
|
model: "kimi-k2-0711-preview",
|
||||||
|
wantURL: "https://api.moonshot.ai/v1/chat/completions",
|
||||||
|
dsnVar: "LLM_KCN",
|
||||||
|
dsnHost: "api.moonshot.cn/v1",
|
||||||
|
wantDSNURL: "https://api.moonshot.cn/v1/chat/completions",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: ProviderQwen,
|
||||||
|
keyEnv: "QWEN_API_KEY",
|
||||||
|
model: "qwen3.8-max",
|
||||||
|
wantURL: "https://dashscope-intl.aliyuncs.com/compatible-mode/v1/chat/completions",
|
||||||
|
dsnVar: "LLM_QCN",
|
||||||
|
dsnHost: "dashscope.aliyuncs.com/compatible-mode/v1",
|
||||||
|
wantDSNURL: "https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestOpenAICompatBuiltins is the whole contract an OpenAI-compat built-in
|
||||||
|
// owes, asserted identically for every one of them: it resolves in Parse and
|
||||||
|
// targets its own endpoint with its own key; a missing key fails closed naming
|
||||||
|
// the right variable and never reaching the network; its name:// DSN reaches
|
||||||
|
// any other host on the DSN token; and a keyless DSN names the LLM_<NAME> that
|
||||||
|
// actually fixes it rather than the built-in's variable, which does nothing
|
||||||
|
// for a DSN-defined provider.
|
||||||
|
func TestOpenAICompatBuiltins(t *testing.T) {
|
||||||
|
for _, tc := range openAICompatBuiltins {
|
||||||
|
t.Run(tc.name+"/builtin", func(t *testing.T) {
|
||||||
|
rt := &captureRT{body: chatCompletionOK}
|
||||||
|
secret := tc.name + "-secret"
|
||||||
|
r := newTestRegistry(t,
|
||||||
|
WithEnvLookup(singleKeyEnv(tc.keyEnv, secret)),
|
||||||
|
WithHTTPClient(&http.Client{Transport: rt}),
|
||||||
|
)
|
||||||
|
|
||||||
|
if p, ok := r.Provider(tc.name); !ok {
|
||||||
|
t.Fatalf("built-in %q not registered", tc.name)
|
||||||
|
} else if p.Name() != tc.name {
|
||||||
|
t.Errorf("name = %q, want %q", p.Name(), tc.name)
|
||||||
|
}
|
||||||
|
|
||||||
|
spec := tc.name + "/" + tc.model
|
||||||
|
m, err := r.Parse(spec)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Parse(%q): %v", spec, err)
|
||||||
|
}
|
||||||
|
if got := targetsOf(t, m); len(got) != 1 || got[0] != spec {
|
||||||
|
t.Fatalf("targets = %v, want [%q]", got, spec)
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, err := m.Generate(context.Background(), llm.Request{Messages: []llm.Message{llm.UserText("hi")}}); err != nil {
|
||||||
|
t.Fatalf("Generate: %v", err)
|
||||||
|
}
|
||||||
|
if rt.req == nil {
|
||||||
|
t.Fatal("no request captured")
|
||||||
|
}
|
||||||
|
if rt.req.URL.String() != tc.wantURL {
|
||||||
|
t.Errorf("URL = %q, want %q", rt.req.URL.String(), tc.wantURL)
|
||||||
|
}
|
||||||
|
if want := "Bearer " + secret; rt.req.Header.Get("Authorization") != want {
|
||||||
|
t.Errorf("Authorization = %q, want %q", rt.req.Header.Get("Authorization"), want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run(tc.name+"/builtin missing key", func(t *testing.T) {
|
||||||
|
rt := &captureRT{body: chatCompletionOK}
|
||||||
|
r := newTestRegistry(t, WithHTTPClient(&http.Client{Transport: rt}))
|
||||||
|
|
||||||
|
m, err := r.Parse(tc.name + "/" + tc.model)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Parse: %v", err)
|
||||||
|
}
|
||||||
|
_, err = m.Generate(context.Background(), llm.Request{Messages: []llm.Message{llm.UserText("hi")}})
|
||||||
|
apiErr, ok := errors.AsType[*llm.APIError](err)
|
||||||
|
if !ok {
|
||||||
|
t.Fatalf("err = %v (%T), want *llm.APIError", err, err)
|
||||||
|
}
|
||||||
|
if apiErr.Status != http.StatusUnauthorized || apiErr.Code != "missing_api_key" {
|
||||||
|
t.Errorf("Status/Code = %d/%q, want 401/missing_api_key", apiErr.Status, apiErr.Code)
|
||||||
|
}
|
||||||
|
if !strings.Contains(apiErr.Message, tc.keyEnv) {
|
||||||
|
t.Errorf("message = %q, want it to name %s", apiErr.Message, tc.keyEnv)
|
||||||
|
}
|
||||||
|
// The load-bearing half: openai.New defaults its key to
|
||||||
|
// OPENAI_API_KEY, so a built-in that stopped passing WithAPIKey
|
||||||
|
// unconditionally would authenticate as OpenAI instead of failing.
|
||||||
|
if strings.Contains(apiErr.Message, "OPENAI_API_KEY") {
|
||||||
|
t.Errorf("message = %q, must not name OPENAI_API_KEY", apiErr.Message)
|
||||||
|
}
|
||||||
|
if rt.req != nil {
|
||||||
|
t.Error("network was hit despite missing key")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run(tc.name+"/dsn scheme", func(t *testing.T) {
|
||||||
|
rt := &captureRT{body: chatCompletionOK}
|
||||||
|
r := newTestRegistry(t, WithHTTPClient(&http.Client{Transport: rt}))
|
||||||
|
if err := r.LoadEnv(map[string]string{
|
||||||
|
tc.dsnVar: tc.name + "://tok@" + tc.dsnHost,
|
||||||
|
}); err != nil {
|
||||||
|
t.Fatalf("LoadEnv: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
dsnName := strings.ToLower(strings.TrimPrefix(tc.dsnVar, "LLM_"))
|
||||||
|
m, err := r.Parse(dsnName + "/" + tc.model)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Parse: %v", err)
|
||||||
|
}
|
||||||
|
if _, err := m.Generate(context.Background(), llm.Request{Messages: []llm.Message{llm.UserText("hi")}}); err != nil {
|
||||||
|
t.Fatalf("Generate: %v", err)
|
||||||
|
}
|
||||||
|
if rt.req == nil {
|
||||||
|
t.Fatal("no request captured")
|
||||||
|
}
|
||||||
|
if rt.req.URL.String() != tc.wantDSNURL {
|
||||||
|
t.Errorf("URL = %q, want %q", rt.req.URL.String(), tc.wantDSNURL)
|
||||||
|
}
|
||||||
|
if want := "Bearer tok"; rt.req.Header.Get("Authorization") != want {
|
||||||
|
t.Errorf("Authorization = %q, want %q", rt.req.Header.Get("Authorization"), want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run(tc.name+"/dsn scheme missing token", func(t *testing.T) {
|
||||||
|
rt := &captureRT{body: chatCompletionOK}
|
||||||
|
r := newTestRegistry(t, WithHTTPClient(&http.Client{Transport: rt}))
|
||||||
|
if err := r.LoadEnv(map[string]string{
|
||||||
|
tc.dsnVar: tc.name + "://" + tc.dsnHost, // no token
|
||||||
|
}); err != nil {
|
||||||
|
t.Fatalf("LoadEnv: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
dsnName := strings.ToLower(strings.TrimPrefix(tc.dsnVar, "LLM_"))
|
||||||
|
m, err := r.Parse(dsnName + "/" + tc.model)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Parse: %v", err)
|
||||||
|
}
|
||||||
|
_, err = m.Generate(context.Background(), llm.Request{Messages: []llm.Message{llm.UserText("hi")}})
|
||||||
|
apiErr, ok := errors.AsType[*llm.APIError](err)
|
||||||
|
if !ok {
|
||||||
|
t.Fatalf("err = %v (%T), want *llm.APIError", err, err)
|
||||||
|
}
|
||||||
|
// A keyless DSN is fixed by adding a token to that DSN, so the
|
||||||
|
// hint must name the defining variable — never the built-in's own
|
||||||
|
// key, which does nothing for a DSN-defined provider.
|
||||||
|
if !strings.Contains(apiErr.Message, tc.dsnVar) {
|
||||||
|
t.Errorf("message = %q, want it to name %s", apiErr.Message, tc.dsnVar)
|
||||||
|
}
|
||||||
|
if strings.Contains(apiErr.Message, tc.keyEnv) {
|
||||||
|
t.Errorf("message = %q, must not name %s for a DSN provider", apiErr.Message, tc.keyEnv)
|
||||||
|
}
|
||||||
|
if rt.req != nil {
|
||||||
|
t.Error("network was hit despite missing token")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,86 @@
|
|||||||
|
package majordomo
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"net/http"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
|
||||||
|
)
|
||||||
|
|
||||||
|
// The contract qwen shares with every other OpenAI-compat built-in (endpoint,
|
||||||
|
// credential isolation, its qwen:// DSN) is asserted by the table in
|
||||||
|
// builtin_openaicompat_test.go. What remains here is qwen-specific: the
|
||||||
|
// reverse-leak direction, and the wire claim ADR-0027 turns on.
|
||||||
|
|
||||||
|
// TestQwenBuiltinKeyDoesNotLeakToOpenAI: QWEN_API_KEY is the qwen built-in's
|
||||||
|
// credential and nothing else's. Why this direction too: the shared table's
|
||||||
|
// missing-key case only proves qwen never borrows OPENAI_API_KEY; this proves
|
||||||
|
// the reverse — a registry that can see QWEN_API_KEY must not hand it to the
|
||||||
|
// openai built-in, which would send an Alibaba key to api.openai.com.
|
||||||
|
func TestQwenBuiltinKeyDoesNotLeakToOpenAI(t *testing.T) {
|
||||||
|
// Set before newTestRegistry: the openai built-in reads OPENAI_API_KEY at
|
||||||
|
// construction. Giving it a real key is what keeps this test honest — a
|
||||||
|
// keyless openai target would 401 before any request, and the assertion
|
||||||
|
// below would pass without a single byte reaching the wire.
|
||||||
|
t.Setenv("OPENAI_API_KEY", "openai-secret")
|
||||||
|
|
||||||
|
rt := &captureRT{body: chatCompletionOK}
|
||||||
|
r := newTestRegistry(t,
|
||||||
|
WithEnvLookup(singleKeyEnv("QWEN_API_KEY", "qwen-secret")),
|
||||||
|
WithHTTPClient(&http.Client{Transport: rt}),
|
||||||
|
)
|
||||||
|
|
||||||
|
m, err := r.Parse("openai/gpt-4o-mini")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Parse: %v", err)
|
||||||
|
}
|
||||||
|
if _, err := m.Generate(context.Background(), llm.Request{Messages: []llm.Message{llm.UserText("hi")}}); err != nil {
|
||||||
|
t.Fatalf("Generate: %v", err)
|
||||||
|
}
|
||||||
|
if rt.req == nil {
|
||||||
|
t.Fatal("no request captured")
|
||||||
|
}
|
||||||
|
if want := "Bearer openai-secret"; rt.req.Header.Get("Authorization") != want {
|
||||||
|
t.Errorf("Authorization = %q, want %q — the qwen credential must not reach the openai built-in",
|
||||||
|
rt.req.Header.Get("Authorization"), want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestQwenReasoningEffortReachesWire is the load-bearing test for ADR-0027's
|
||||||
|
// central claim: Model Studio's OpenAI-compatible surface takes reasoning as a
|
||||||
|
// top-level "reasoning_effort" body field, which the openai client already
|
||||||
|
// sends — so llm.WithReasoningEffort survives the trip on qwen with no
|
||||||
|
// qwen-specific code. Routing qwen through the anthropic client instead would
|
||||||
|
// drop it silently (provider/anthropic ignores ReasoningEffort by design), and
|
||||||
|
// that difference would be invisible without asserting on the wire body.
|
||||||
|
func TestQwenReasoningEffortReachesWire(t *testing.T) {
|
||||||
|
rt := &captureRT{body: chatCompletionOK}
|
||||||
|
r := newTestRegistry(t,
|
||||||
|
WithEnvLookup(singleKeyEnv("QWEN_API_KEY", "qwen-secret")),
|
||||||
|
WithHTTPClient(&http.Client{Transport: rt}),
|
||||||
|
)
|
||||||
|
|
||||||
|
m, err := r.Parse("qwen/qwen3.8-max")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Parse: %v", err)
|
||||||
|
}
|
||||||
|
_, err = m.Generate(context.Background(), llm.Request{
|
||||||
|
Messages: []llm.Message{llm.UserText("hi")},
|
||||||
|
ReasoningEffort: "high",
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Generate: %v", err)
|
||||||
|
}
|
||||||
|
if rt.reqBody == nil {
|
||||||
|
t.Fatal("no request body captured")
|
||||||
|
}
|
||||||
|
var sent map[string]any
|
||||||
|
if err := json.Unmarshal(rt.reqBody, &sent); err != nil {
|
||||||
|
t.Fatalf("decode request body: %v", err)
|
||||||
|
}
|
||||||
|
if got := sent["reasoning_effort"]; got != "high" {
|
||||||
|
t.Errorf("reasoning_effort = %v, want %q (body: %s)", got, "high", rt.reqBody)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,102 @@
|
|||||||
|
# ADR-0027: Qwen (Alibaba) built-in provider — OpenAI-compat, not Anthropic-compat
|
||||||
|
|
||||||
|
**Status:** Accepted — 2026-08-12
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
Alibaba's Qwen models (`qwen3.8-max`, `qwen3.7-plus`, the `qwen3-vl-*` vision
|
||||||
|
variants, …) are served from Model Studio / DashScope, and mort wants them as a
|
||||||
|
first-class failover tier with a dedicated `QWEN_API_KEY` — the same ergonomics
|
||||||
|
ADR-0026 gave Kimi.
|
||||||
|
|
||||||
|
Unlike Kimi, Model Studio exposes the same models over **two** protocols:
|
||||||
|
|
||||||
|
| | OpenAI-compatible | Anthropic-compatible |
|
||||||
|
|---|---|---|
|
||||||
|
| Base URL | `https://dashscope-intl.aliyuncs.com/compatible-mode/v1` | `https://dashscope-intl.aliyuncs.com/apps/anthropic` |
|
||||||
|
| Endpoints | full Chat Completions surface | `/v1/messages` only (no `/v1/models`) |
|
||||||
|
| Purpose | the documented developer API | a shim, documented around hosting Claude Code |
|
||||||
|
|
||||||
|
So the question this ADR answers is not "which client do we reuse" but
|
||||||
|
"which of Alibaba's two wire protocols does the built-in speak".
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
**The `qwen` built-in and the `qwen://` DSN scheme speak OpenAI-compat**, over
|
||||||
|
`provider/openai` — no new package, mirroring ADR-0026 (kimi) and ADR-0015
|
||||||
|
(llama-swap chat). Default base URL is the international host; the China host
|
||||||
|
(`dashscope.aliyuncs.com/compatible-mode/v1`) and workspace-scoped regional
|
||||||
|
hosts are reachable with a `qwen://` DSN.
|
||||||
|
|
||||||
|
Credential handling is copied from kimi verbatim, because both of its rules
|
||||||
|
are load-bearing: `WithAPIKey` is passed unconditionally (even empty) so an
|
||||||
|
unset `QWEN_API_KEY` can never fall through to `openai.New`'s `OPENAI_API_KEY`
|
||||||
|
default, and `WithAPIKeyName("QWEN_API_KEY")` makes the synthetic-401 hint name
|
||||||
|
the variable the operator actually has to set.
|
||||||
|
|
||||||
|
The kimi and qwen DSN factories were identical, so they now share one
|
||||||
|
`openaiCompatScheme` helper — the next OpenAI-compat built-in gets the
|
||||||
|
credential and key-hint rules by construction rather than by copy.
|
||||||
|
|
||||||
|
### Why not the Anthropic-compatible endpoint
|
||||||
|
|
||||||
|
Every concrete difference favors OpenAI-compat *for this codebase*:
|
||||||
|
|
||||||
|
- **Reasoning survives the trip.** Model Studio takes `reasoning_effort` as a
|
||||||
|
top-level field on the OpenAI surface, which `provider/openai` already sends
|
||||||
|
— `llm.WithReasoningEffort` works on qwen with zero qwen-specific code
|
||||||
|
(`TestQwenReasoningEffortReachesWire` asserts it on the wire). Down the
|
||||||
|
anthropic client it would be dropped in silence: `provider/anthropic`
|
||||||
|
deliberately ignores `Request.ReasoningEffort`, because first-party Claude
|
||||||
|
has no such knob.
|
||||||
|
- **Structured output would regress.** `provider/anthropic` implements
|
||||||
|
`Request.Schema` with the first-party GA `output_config.format` mechanism.
|
||||||
|
Alibaba's shim does not implement it; a compat endpoint that ignores an
|
||||||
|
unknown field returns unconstrained prose while still reporting success.
|
||||||
|
The OpenAI path sends `response_format: json_schema`, which Model Studio
|
||||||
|
supports natively on the Max/Plus families.
|
||||||
|
- **Cache accounting already lands.** Model Studio's implicit prefix cache
|
||||||
|
reports hits in `usage.prompt_tokens_details.cached_tokens`, which the openai
|
||||||
|
client already maps to `llm.Usage.CacheReadTokens`. The anthropic client
|
||||||
|
reads `cache_read_input_tokens`, a field the shim has no reason to emit.
|
||||||
|
- **Thinking content is discarded on the anthropic path anyway.**
|
||||||
|
`provider/anthropic` skips `thinking` blocks in both the buffered and
|
||||||
|
streaming decoders, so the shim's headline feature — first-class
|
||||||
|
`thinking: {type: "enabled", budget_tokens: N}` — buys majordomo nothing
|
||||||
|
today.
|
||||||
|
- **Smaller blast radius.** The anthropic client has no `WithAPIKeyName`
|
||||||
|
option, so a keyless qwen would tell the operator to set `ANTHROPIC_API_KEY`;
|
||||||
|
fixing that means changing the first-party Anthropic client to serve a
|
||||||
|
third-party shim.
|
||||||
|
- **It is the less-exercised surface.** The Anthropic endpoint is documented as
|
||||||
|
Messages-only, with a temperature range that differs from Anthropic's own
|
||||||
|
([0, 2) vs [0.0, 1.0]) — i.e. it is Qwen semantics wearing an Anthropic
|
||||||
|
envelope, not an Anthropic-equivalent target.
|
||||||
|
|
||||||
|
The one thing the Anthropic surface offers that OpenAI-compat does not is
|
||||||
|
explicit `cache_control` breakpoints reached through `Request.PromptCache`.
|
||||||
|
That is not a reason to route Qwen through it: Model Studio's implicit cache is
|
||||||
|
automatic and already metered, and if explicit breakpoints ever matter they
|
||||||
|
belong in `provider/openai` (Model Studio accepts `cache_control` on content
|
||||||
|
blocks there too), where every OpenAI-compat target would get them.
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
- `qwen/<model>` is first-class in Parse, chains, aliases, and health/failover
|
||||||
|
with no consumer wiring; model ids pass through verbatim (no catalog).
|
||||||
|
- Chat, streaming, tools, structured output, reasoning effort, and cached-token
|
||||||
|
accounting all ride the openai client and inherit its fixes.
|
||||||
|
- Image *inputs* work at the client level, but only the `qwen-vl-*` /
|
||||||
|
`qwen3-vl-*` models accept them (matrix footnote ⁴; ³ is kimi's).
|
||||||
|
- Two model-side quirks are Alibaba's, not majordomo's, and are left to the
|
||||||
|
caller rather than papered over: thinking is **on by default** on some models
|
||||||
|
(e.g. `qwen3.7-plus`), and Qwen3 *open-source* models require streaming when
|
||||||
|
thinking is enabled — a buffered `Generate` against one of those needs a
|
||||||
|
model that supports non-streaming thinking (the Max/Plus families do).
|
||||||
|
- If a future consumer genuinely needs the Anthropic surface, it is reachable
|
||||||
|
today without library changes:
|
||||||
|
`LLM_QWEN_ANTHROPIC=anthropic://[email protected]/apps/anthropic`
|
||||||
|
— with the reasoning/structured-output caveats above.
|
||||||
|
- Second third-party built-in after kimi. The ADR-0026 bar still holds: a named
|
||||||
|
consumer needs it in-config. `RegisterProvider`/`LLM_*` remain the path for
|
||||||
|
everything else.
|
||||||
@@ -30,3 +30,4 @@ One decision per file, append-only; supersede rather than rewrite.
|
|||||||
| [0024](0024-audio-wave3-surfaces.md) | Wave-3 audio surfaces (stems, SFX, speech enhance, voice clone, translate) | Accepted |
|
| [0024](0024-audio-wave3-surfaces.md) | Wave-3 audio surfaces (stems, SFX, speech enhance, voice clone, translate) | Accepted |
|
||||||
| [0025](0025-videogen-wave3-surfaces.md) | Wave-3 video surfaces (lipsync, video matte, video upscale, chain jobs) | Accepted |
|
| [0025](0025-videogen-wave3-surfaces.md) | Wave-3 video surfaces (lipsync, video matte, video upscale, chain jobs) | Accepted |
|
||||||
| [0026](0026-kimi-builtin.md) | Kimi (Moonshot AI) built-in provider — reuse openai client, KIMI_API_KEY | Accepted |
|
| [0026](0026-kimi-builtin.md) | Kimi (Moonshot AI) built-in provider — reuse openai client, KIMI_API_KEY | Accepted |
|
||||||
|
| [0027](0027-qwen-builtin.md) | Qwen (Alibaba) built-in provider — OpenAI-compat, not Model Studio's Anthropic-compat endpoint | Accepted |
|
||||||
|
|||||||
@@ -26,8 +26,9 @@ var ErrUnknownProvider = errors.New("unknown provider")
|
|||||||
// authenticated with the bearer token "test-token".
|
// authenticated with the bearer token "test-token".
|
||||||
type DSN struct {
|
type DSN struct {
|
||||||
// Scheme selects the provider implementation: "foreman", "ollama",
|
// Scheme selects the provider implementation: "foreman", "ollama",
|
||||||
// "ollama-cloud", "openai", "kimi", "anthropic", "google"/"gemini", or
|
// "ollama-cloud", "openai", "kimi", "qwen", "anthropic",
|
||||||
// any custom scheme registered with RegisterScheme.
|
// "google"/"gemini", "llama-swap"/"llama-swaps", or any custom scheme
|
||||||
|
// registered with RegisterScheme.
|
||||||
Scheme string
|
Scheme string
|
||||||
// Token is the provider secret (bearer token or API key); empty = none.
|
// Token is the provider secret (bearer token or API key); empty = none.
|
||||||
Token string
|
Token string
|
||||||
@@ -40,6 +41,19 @@ type DSN struct {
|
|||||||
// env-defined providers always speak TLS).
|
// env-defined providers always speak TLS).
|
||||||
func (d DSN) BaseURL() string { return "https://" + d.Host }
|
func (d DSN) BaseURL() string { return "https://" + d.Host }
|
||||||
|
|
||||||
|
// envKeyForProvider returns the LLM_* variable that defines the provider named
|
||||||
|
// name: "m1" → LLM_M1, "my-prov" → LLM_MY_PROV.
|
||||||
|
//
|
||||||
|
// This is the single definition on purpose. Two call sites need byte-identical
|
||||||
|
// output and would drift apart in silence: lazy resolution reads this variable
|
||||||
|
// to find an unregistered provider, and openaiCompatScheme names it in the
|
||||||
|
// missing-key hint so a keyless DSN target tells the operator which variable to
|
||||||
|
// set. Those two were separate copies with a comment asserting they matched —
|
||||||
|
// a comment is not enforcement, this function is.
|
||||||
|
func envKeyForProvider(name string) string {
|
||||||
|
return "LLM_" + strings.ToUpper(strings.ReplaceAll(name, "-", "_"))
|
||||||
|
}
|
||||||
|
|
||||||
// ParseDSN parses a raw DSN string. The algorithm matches go-llm exactly:
|
// ParseDSN parses a raw DSN string. The algorithm matches go-llm exactly:
|
||||||
// split on "://", then an optional "@" separates the token from the host;
|
// split on "://", then an optional "@" separates the token from the host;
|
||||||
// trailing slashes on the host are trimmed.
|
// trailing slashes on the host are trimmed.
|
||||||
|
|||||||
+36
@@ -285,3 +285,39 @@ tests flush out.
|
|||||||
(footnote ³), `.env.example`, ADR-0026 (+ index; also backfilled the missing
|
(footnote ³), `.env.example`, ADR-0026 (+ index; also backfilled the missing
|
||||||
0024/0025 index rows).
|
0024/0025 index rows).
|
||||||
- Consumer: mort names Kimi as a failover tier.
|
- Consumer: mort names Kimi as a failover tier.
|
||||||
|
|
||||||
|
## 2026-08-12 — Qwen (Alibaba) built-in provider (ADR-0027)
|
||||||
|
|
||||||
|
- New built-in `qwen` provider + `qwen://` DSN scheme over Alibaba Model
|
||||||
|
Studio's OpenAI-compatible mode, reusing `provider/openai` (no new client,
|
||||||
|
mirrors kimi/llama-swap). Default base URL
|
||||||
|
`https://dashscope-intl.aliyuncs.com/compatible-mode/v1`; China/regional
|
||||||
|
hosts via `LLM_QCN=qwen://[email protected]/compatible-mode/v1`.
|
||||||
|
- Credential is `QWEN_API_KEY` (via the registry's injected envLookup).
|
||||||
|
`WithAPIKey` passed unconditionally so an unset key cannot fall through to
|
||||||
|
`OPENAI_API_KEY`; `WithAPIKeyName` names `QWEN_API_KEY` in the 401 hint.
|
||||||
|
- **Chose OpenAI-compat over Model Studio's Anthropic-compatible
|
||||||
|
`/apps/anthropic` shim** (ADR-0027): on the anthropic client
|
||||||
|
`ReasoningEffort` is ignored by design, `Request.Schema` rides
|
||||||
|
`output_config.format` (which the shim does not implement), and cached-token
|
||||||
|
accounting reads Anthropic-only usage fields. The shim is still reachable
|
||||||
|
ad hoc via an `anthropic://` DSN.
|
||||||
|
- `registerOpenAICompatBuiltin` installs BOTH halves of an OpenAI-compat
|
||||||
|
built-in (eager provider + `name://` DSN scheme via the shared
|
||||||
|
`openaiCompatScheme`), so the two credential rules — unconditional
|
||||||
|
`WithAPIKey`, and `WithAPIKeyName` naming that same variable — hold by
|
||||||
|
construction. kimi and qwen are one line each.
|
||||||
|
- `envKeyForProvider` is the single definition of the `LLM_<NAME>` form,
|
||||||
|
shared by lazy resolution (`registry.go`) and the DSN missing-key hint. They
|
||||||
|
were separate copies with a comment asserting they matched.
|
||||||
|
- The shared contract is ONE table (`builtin_openaicompat_test.go`), run
|
||||||
|
identically for every OpenAI-compat built-in: endpoint + bearer, missing key
|
||||||
|
fails closed naming its own variable with no network hit, the `name://` DSN
|
||||||
|
reaching another host, and a keyless DSN naming `LLM_<NAME>` rather than the
|
||||||
|
built-in's key. Adding a built-in is a table row that immediately owes all
|
||||||
|
four; `builtin_kimi_test.go` was retired into it. Qwen-only tests: the
|
||||||
|
reverse credential leak, and `reasoning_effort` asserted on the wire body
|
||||||
|
(the ADR's load-bearing claim).
|
||||||
|
- Docs in sync: README built-in table + Qwen paragraph + DSN scheme list +
|
||||||
|
support matrix (footnote ⁴), `.env.example`, ADR-0027 (+ index).
|
||||||
|
- Consumer: mort wants Qwen as a failover tier.
|
||||||
|
|||||||
@@ -207,10 +207,12 @@ func parseSwapReport(header string) []imagegen.SwappedFace {
|
|||||||
// imageFilename picks a multipart filename for an image part. The shim reads
|
// imageFilename picks a multipart filename for an image part. The shim reads
|
||||||
// bytes, not names, but a plausible extension keeps server-side sniffing and
|
// bytes, not names, but a plausible extension keeps server-side sniffing and
|
||||||
// request logs honest. base distinguishes the parts of a multi-file form
|
// request logs honest. base distinguishes the parts of a multi-file form
|
||||||
// ("target"/"source") so a log line says which one was malformed.
|
// ("target"/"source", "frame"/"frame_last") so a log line says which one was
|
||||||
|
// malformed — and, for the video keyframes, so a backend that stages uploads
|
||||||
|
// by filename cannot have the second overwrite the first.
|
||||||
//
|
//
|
||||||
// initImageFilename (video.go) is this function with base fixed to "frame"
|
// Every caller routes through here: two copies of one extension table is how
|
||||||
// and delegates here — two copies of one extension table is how they drift.
|
// they drift.
|
||||||
func imageFilename(mimeType, base string) string {
|
func imageFilename(mimeType, base string) string {
|
||||||
if base == "" {
|
if base == "" {
|
||||||
base = "image"
|
base = "image"
|
||||||
|
|||||||
@@ -56,7 +56,7 @@ func (m *lipsyncModel) Lipsync(ctx context.Context, req videogen.LipsyncRequest,
|
|||||||
// hand (mirrors videoModel.Generate).
|
// hand (mirrors videoModel.Generate).
|
||||||
var buf bytes.Buffer
|
var buf bytes.Buffer
|
||||||
w := multipart.NewWriter(&buf)
|
w := multipart.NewWriter(&buf)
|
||||||
fw, err := w.CreateFormFile("image", initImageFilename(req.Image.MIME))
|
fw, err := w.CreateFormFile("image", imageFilename(req.Image.MIME, "frame"))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("llama-swap: build lipsync form: %w", err)
|
return nil, fmt.Errorf("llama-swap: build lipsync form: %w", err)
|
||||||
}
|
}
|
||||||
|
|||||||
+42
-13
@@ -38,10 +38,13 @@ type videoModel struct {
|
|||||||
// bound the call with a context deadline.
|
// bound the call with a context deadline.
|
||||||
//
|
//
|
||||||
// Parameter names follow vLLM-Omni's videos API (num_frames, fps,
|
// Parameter names follow vLLM-Omni's videos API (num_frames, fps,
|
||||||
// num_inference_steps, guidance_scale); the conditioning frame is sent as an
|
// num_inference_steps, guidance_scale); the leading conditioning frame is sent
|
||||||
// `input_reference` file part, following OpenAI's videos API. Upstreams
|
// as an `input_reference` file part, following OpenAI's videos API, and a
|
||||||
|
// trailing keyframe (Request.LastImage) as `input_reference_last`. Upstreams
|
||||||
// ignore fields they don't understand, and optional fields stay off the wire
|
// ignore fields they don't understand, and optional fields stay off the wire
|
||||||
// entirely so the model's own defaults apply.
|
// entirely so the model's own defaults apply — which is also why a backend
|
||||||
|
// without first-last-frame support returns an ordinary clip here rather than
|
||||||
|
// an error.
|
||||||
func (m *videoModel) Generate(ctx context.Context, req videogen.Request, opts ...videogen.Option) (*videogen.Result, error) {
|
func (m *videoModel) Generate(ctx context.Context, req videogen.Request, opts ...videogen.Option) (*videogen.Result, error) {
|
||||||
req = req.Apply(opts...)
|
req = req.Apply(opts...)
|
||||||
if strings.TrimSpace(req.Prompt) == "" {
|
if strings.TrimSpace(req.Prompt) == "" {
|
||||||
@@ -56,6 +59,9 @@ func (m *videoModel) Generate(ctx context.Context, req videogen.Request, opts ..
|
|||||||
if req.InitImage != nil && len(req.InitImage.Data) == 0 {
|
if req.InitImage != nil && len(req.InitImage.Data) == 0 {
|
||||||
return nil, fmt.Errorf("%w: video init image has no bytes", llm.ErrUnsupported)
|
return nil, fmt.Errorf("%w: video init image has no bytes", llm.ErrUnsupported)
|
||||||
}
|
}
|
||||||
|
if req.LastImage != nil && len(req.LastImage.Data) == 0 {
|
||||||
|
return nil, fmt.Errorf("%w: video last image has no bytes", llm.ErrUnsupported)
|
||||||
|
}
|
||||||
width, height, err := parseSize(req.Size)
|
width, height, err := parseSize(req.Size)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("%w: %v", llm.ErrUnsupported, err)
|
return nil, fmt.Errorf("%w: %v", llm.ErrUnsupported, err)
|
||||||
@@ -83,12 +89,19 @@ func (m *videoModel) Generate(ctx context.Context, req videogen.Request, opts ..
|
|||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
if req.InitImage != nil {
|
if req.InitImage != nil {
|
||||||
fw, err := w.CreateFormFile("input_reference", initImageFilename(req.InitImage.MIME))
|
if err := writeImagePart(w, "input_reference", "frame", req.InitImage); err != nil {
|
||||||
if err != nil {
|
return nil, err
|
||||||
return nil, fmt.Errorf("llama-swap: build video form: %w", err)
|
|
||||||
}
|
}
|
||||||
if _, err := fw.Write(req.InitImage.Data); err != nil {
|
}
|
||||||
return nil, fmt.Errorf("llama-swap: build video form: %w", err)
|
// The trailing keyframe rides a SEPARATE part rather than a second
|
||||||
|
// `input_reference`: multipart permits repeated names, but the receiving
|
||||||
|
// end would then have to rely on part ORDER to tell first from last, and
|
||||||
|
// an ordering contract that is invisible in the field name is one nobody
|
||||||
|
// can see they have broken. A backend that does not know the name ignores
|
||||||
|
// the part, which is the same degradation as any other unknown field.
|
||||||
|
if req.LastImage != nil {
|
||||||
|
if err := writeImagePart(w, "input_reference_last", "frame_last", req.LastImage); err != nil {
|
||||||
|
return nil, err
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if err := w.Close(); err != nil {
|
if err := w.Close(); err != nil {
|
||||||
@@ -134,11 +147,27 @@ func singleVideoResult(provider, model, verb string, raw []byte, contentType str
|
|||||||
return &videogen.Result{Video: videogen.Video{Data: raw, MIME: mimeType}}, nil
|
return &videogen.Result{Video: videogen.Video{Data: raw, MIME: mimeType}}, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// initImageFilename picks the multipart filename hint for the conditioning
|
// writeImagePart attaches one conditioning frame under the given field name,
|
||||||
// frame from its MIME subtype. The name is provider-chosen (never
|
// with a filename derived from nameStem. Shared by the first- and last-frame
|
||||||
// caller-supplied), so no sanitization is needed.
|
// parts so the two cannot drift in how they encode, which is the usual way a
|
||||||
func initImageFilename(mimeType string) string {
|
// second copy of a block goes wrong.
|
||||||
return imageFilename(mimeType, "frame")
|
//
|
||||||
|
// The two frames MUST carry DISTINCT filenames, not merely distinct field
|
||||||
|
// names. Backends commonly stage an uploaded frame under a name derived from
|
||||||
|
// the filename — our own ComfyUI shim posts to /upload/image with
|
||||||
|
// overwrite=true — so two parts sharing "frame.png" would have the second
|
||||||
|
// clobber the first, and BOTH keyframe inputs would then resolve to the same
|
||||||
|
// stored image. The clip would render clean, pinned at both ends to the same
|
||||||
|
// frame, with nothing anywhere reporting a problem.
|
||||||
|
func writeImagePart(w *multipart.Writer, field, nameStem string, img *videogen.Image) error {
|
||||||
|
fw, err := w.CreateFormFile(field, imageFilename(img.MIME, nameStem))
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("llama-swap: build video form: %w", err)
|
||||||
|
}
|
||||||
|
if _, err := fw.Write(img.Data); err != nil {
|
||||||
|
return fmt.Errorf("llama-swap: build video form: %w", err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// formatInt renders an optional int pointer for a form field; nil = "" (omit).
|
// formatInt renders an optional int pointer for a form field; nil = "" (omit).
|
||||||
|
|||||||
@@ -223,3 +223,134 @@ func TestVideoGenerateNonVideoBodyErrors(t *testing.T) {
|
|||||||
t.Errorf("message = %q, want mention of non-video body", apiErr.Message)
|
t.Errorf("message = %q, want mention of non-video body", apiErr.Message)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Both keyframes reach the wire, under DISTINCT field names.
|
||||||
|
//
|
||||||
|
// The distinct-name property is the actual contract with the backend shim: the
|
||||||
|
// two frames could have shared one repeated `input_reference` name, and then
|
||||||
|
// which is first and which is last would depend on multipart part ORDER — an
|
||||||
|
// ordering contract invisible in the payload, that nothing would notice
|
||||||
|
// breaking. Asserting the names is what pins it.
|
||||||
|
func TestVideoGenerateSendsBothKeyframes(t *testing.T) {
|
||||||
|
var gotFirst, gotLast []byte
|
||||||
|
var firstName, lastName string
|
||||||
|
var sawLastPart bool
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if err := r.ParseMultipartForm(32 << 20); err != nil {
|
||||||
|
t.Errorf("parse form: %v", err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if f, hdr, err := r.FormFile("input_reference"); err == nil {
|
||||||
|
gotFirst, _ = io.ReadAll(f)
|
||||||
|
firstName = hdr.Filename
|
||||||
|
f.Close()
|
||||||
|
}
|
||||||
|
if f, hdr, err := r.FormFile("input_reference_last"); err == nil {
|
||||||
|
sawLastPart = true
|
||||||
|
gotLast, _ = io.ReadAll(f)
|
||||||
|
lastName = hdr.Filename
|
||||||
|
f.Close()
|
||||||
|
}
|
||||||
|
w.Header().Set("Content-Type", "video/mp4")
|
||||||
|
_, _ = w.Write([]byte("fake-mp4-bytes"))
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
p := New(WithBaseURL(srv.URL), WithHTTPClient(srv.Client()))
|
||||||
|
vm, err := p.VideoModel("videogen-minimax-h3")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("VideoModel: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
first, _ := base64.StdEncoding.DecodeString(onePixelPNG)
|
||||||
|
last := append(append([]byte{}, first...), 0x00) // distinguishable from first
|
||||||
|
|
||||||
|
if _, err := vm.Generate(context.Background(), videogen.Request{
|
||||||
|
Prompt: "a cat surfing",
|
||||||
|
InitImage: &videogen.Image{MIME: "image/png", Data: first},
|
||||||
|
LastImage: &videogen.Image{MIME: "image/png", Data: last},
|
||||||
|
}); err != nil {
|
||||||
|
t.Fatalf("Generate: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if !sawLastPart {
|
||||||
|
t.Fatal("input_reference_last was not sent — a pinned end frame would be silently dropped")
|
||||||
|
}
|
||||||
|
if string(gotFirst) != string(first) {
|
||||||
|
t.Errorf("input_reference = %d bytes, want %d", len(gotFirst), len(first))
|
||||||
|
}
|
||||||
|
if string(gotLast) != string(last) {
|
||||||
|
t.Errorf("input_reference_last = %d bytes, want %d", len(gotLast), len(last))
|
||||||
|
}
|
||||||
|
// The two must not be the same bytes, or a swap/aliasing bug reads as a pass.
|
||||||
|
if string(gotFirst) == string(gotLast) {
|
||||||
|
t.Error("both parts carry identical bytes — the frames are being aliased")
|
||||||
|
}
|
||||||
|
// DISTINCT FILENAMES, not just distinct field names. Backends stage an
|
||||||
|
// uploaded frame under a name derived from the filename (our ComfyUI shim
|
||||||
|
// posts to /upload/image with overwrite=true), so two parts sharing
|
||||||
|
// "frame.png" would have the second clobber the first and BOTH keyframes
|
||||||
|
// would resolve to the same stored image — a clip pinned at both ends to
|
||||||
|
// the same frame, rendering cleanly with nothing reporting a fault.
|
||||||
|
if firstName == "" || lastName == "" {
|
||||||
|
t.Fatalf("filenames = %q / %q, want both set", firstName, lastName)
|
||||||
|
}
|
||||||
|
if firstName == lastName {
|
||||||
|
t.Errorf("both parts use filename %q — the second upload would clobber the first", firstName)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// LastImage alone (no InitImage) is a legitimate request: pin the destination
|
||||||
|
// and let the model invent the approach. It must not require a first frame.
|
||||||
|
func TestVideoGenerateLastImageAloneIsAllowed(t *testing.T) {
|
||||||
|
var sawFirst, sawLast bool
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if err := r.ParseMultipartForm(32 << 20); err != nil {
|
||||||
|
t.Errorf("parse form: %v", err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if f, _, err := r.FormFile("input_reference"); err == nil {
|
||||||
|
sawFirst = true
|
||||||
|
f.Close()
|
||||||
|
}
|
||||||
|
if f, _, err := r.FormFile("input_reference_last"); err == nil {
|
||||||
|
sawLast = true
|
||||||
|
f.Close()
|
||||||
|
}
|
||||||
|
w.Header().Set("Content-Type", "video/mp4")
|
||||||
|
_, _ = w.Write([]byte("fake-mp4-bytes"))
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
p := New(WithBaseURL(srv.URL), WithHTTPClient(srv.Client()))
|
||||||
|
vm, _ := p.VideoModel("videogen-minimax-h3")
|
||||||
|
frame, _ := base64.StdEncoding.DecodeString(onePixelPNG)
|
||||||
|
|
||||||
|
if _, err := vm.Generate(context.Background(),
|
||||||
|
videogen.Request{Prompt: "arrive here"},
|
||||||
|
videogen.WithLastImage(videogen.Image{MIME: "image/png", Data: frame}),
|
||||||
|
); err != nil {
|
||||||
|
t.Fatalf("Generate: %v", err)
|
||||||
|
}
|
||||||
|
if sawFirst {
|
||||||
|
t.Error("input_reference sent, want omitted")
|
||||||
|
}
|
||||||
|
if !sawLast {
|
||||||
|
t.Error("input_reference_last omitted, want sent")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// An empty LastImage is rejected before the request is built, matching
|
||||||
|
// InitImage's existing contract — a zero-byte frame reaching the backend is a
|
||||||
|
// confusing upstream error instead of a clear local one.
|
||||||
|
func TestVideoGenerateRejectsEmptyLastImage(t *testing.T) {
|
||||||
|
p := New(WithBaseURL("http://unused"))
|
||||||
|
vm, _ := p.VideoModel("videogen-minimax-h3")
|
||||||
|
_, err := vm.Generate(context.Background(), videogen.Request{
|
||||||
|
Prompt: "x",
|
||||||
|
LastImage: &videogen.Image{MIME: "image/png"},
|
||||||
|
})
|
||||||
|
if !errors.Is(err, llm.ErrUnsupported) {
|
||||||
|
t.Fatalf("err = %v, want llm.ErrUnsupported", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
+1
-1
@@ -263,7 +263,7 @@ func (r *Registry) providerFor(name string) (llm.Provider, error) {
|
|||||||
return nil, envErr
|
return nil, envErr
|
||||||
}
|
}
|
||||||
|
|
||||||
envKey := "LLM_" + strings.ToUpper(strings.ReplaceAll(name, "-", "_"))
|
envKey := envKeyForProvider(name)
|
||||||
envVal := r.envLookup(envKey)
|
envVal := r.envLookup(envKey)
|
||||||
if envVal == "" {
|
if envVal == "" {
|
||||||
return nil, fmt.Errorf("%w: %q (checked registry and %s env var)", ErrUnknownProvider, name, envKey)
|
return nil, fmt.Errorf("%w: %q (checked registry and %s env var)", ErrUnknownProvider, name, envKey)
|
||||||
|
|||||||
@@ -12,6 +12,8 @@
|
|||||||
// InitImage is a pure text prompt, a non-nil InitImage conditions generation
|
// InitImage is a pure text prompt, a non-nil InitImage conditions generation
|
||||||
// on that frame. Hybrid models (e.g. Wan 2.2 TI2V) serve both from the same
|
// on that frame. Hybrid models (e.g. Wan 2.2 TI2V) serve both from the same
|
||||||
// checkpoint, so unlike imagegen there is no separate Editor-style interface.
|
// checkpoint, so unlike imagegen there is no separate Editor-style interface.
|
||||||
|
// LastImage extends the same surface to the other end of the clip, so one
|
||||||
|
// Request covers t2v, i2v, and first-last-frame-to-video without a mode flag.
|
||||||
//
|
//
|
||||||
// The first implementation is provider/llamaswap, which targets the blocking
|
// The first implementation is provider/llamaswap, which targets the blocking
|
||||||
// OpenAI/vLLM-Omni-style POST /v1/videos/sync endpoint: the response body is
|
// OpenAI/vLLM-Omni-style POST /v1/videos/sync endpoint: the response body is
|
||||||
@@ -51,6 +53,19 @@ type Request struct {
|
|||||||
// nil = pure text-to-video.
|
// nil = pure text-to-video.
|
||||||
InitImage *Image
|
InitImage *Image
|
||||||
|
|
||||||
|
// LastImage conditions generation on an ENDING frame. With InitImage it
|
||||||
|
// pins both ends (first-last-frame-to-video); alone it pins only the
|
||||||
|
// destination and lets the backend invent the approach.
|
||||||
|
//
|
||||||
|
// Support is per-model and NOT advertised anywhere in this contract: a
|
||||||
|
// backend that does not understand a trailing keyframe ignores it and
|
||||||
|
// returns an ordinary clip, which is indistinguishable from success.
|
||||||
|
// There is no capability bit to consult, because the contract has no way
|
||||||
|
// to learn one. A caller that needs to know whether the pin actually took
|
||||||
|
// effect must establish that out of band — by configuration it controls,
|
||||||
|
// not by inspecting the result.
|
||||||
|
LastImage *Image
|
||||||
|
|
||||||
// Size is the requested resolution, e.g. "1280x704"; "" = backend default.
|
// Size is the requested resolution, e.g. "1280x704"; "" = backend default.
|
||||||
Size string
|
Size string
|
||||||
|
|
||||||
@@ -92,6 +107,10 @@ type Option func(*Request)
|
|||||||
// WithInitImage conditions generation on a starting frame (image-to-video).
|
// WithInitImage conditions generation on a starting frame (image-to-video).
|
||||||
func WithInitImage(img Image) Option { return func(r *Request) { r.InitImage = &img } }
|
func WithInitImage(img Image) Option { return func(r *Request) { r.InitImage = &img } }
|
||||||
|
|
||||||
|
// WithLastImage conditions generation on an ending frame. Combined with
|
||||||
|
// WithInitImage this pins both ends of the clip.
|
||||||
|
func WithLastImage(img Image) Option { return func(r *Request) { r.LastImage = &img } }
|
||||||
|
|
||||||
// WithSize sets the requested resolution (e.g. "1280x704").
|
// WithSize sets the requested resolution (e.g. "1280x704").
|
||||||
func WithSize(size string) Option { return func(r *Request) { r.Size = size } }
|
func WithSize(size string) Option { return func(r *Request) { r.Size = size } }
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user