steveandClaude Opus 5
f1f2b653c3
Same class of finding as round 1, one level in: I factored the DSN-scheme half of the kimi/qwen duplication into openaiCompatScheme and left the eager provider half copy-pasted, so a third built-in still had six lines to clone — including both credential rules, which is exactly the pair you do not want re-typed. registerOpenAICompatBuiltin now installs both halves from one call. The rules that matter hold by construction for every future caller: WithAPIKey passed unconditionally (an unset key must not fall through to OPENAI_API_KEY), and WithAPIKeyName naming that same variable in the 401 hint. Registering kimi and qwen is now one line each. Also fixed a cross-reference the ADR got wrong: Qwen's image-input caveat is README matrix footnote ⁴, not ³ — ³ is kimi's. I wrote "³, shared with kimi" in the ADR and then gave Qwen its own footnote in the README. The break-check harness needed fixing before any of this could be trusted: three of its mutations targeted lines this refactor moved, so they matched nothing, the code was never broken, and the suite reported "test still passed" — identical output to a test that genuinely misses the bug. Mutations are now verified to have landed (sha before/after) and the suite fails loudly if one doesn't. Two new cases cover the helper: dropping the unconditional WithAPIKey, and dropping the scheme-half registration. 8/8 apply and are caught. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
103 lines
5.6 KiB
Markdown
103 lines
5.6 KiB
Markdown
# ADR-0027: Qwen (Alibaba) built-in provider — OpenAI-compat, not Anthropic-compat
|
|
|
|
**Status:** Accepted — 2026-08-12
|
|
|
|
## Context
|
|
|
|
Alibaba's Qwen models (`qwen3.8-max`, `qwen3.7-plus`, the `qwen3-vl-*` vision
|
|
variants, …) are served from Model Studio / DashScope, and mort wants them as a
|
|
first-class failover tier with a dedicated `QWEN_API_KEY` — the same ergonomics
|
|
ADR-0026 gave Kimi.
|
|
|
|
Unlike Kimi, Model Studio exposes the same models over **two** protocols:
|
|
|
|
| | OpenAI-compatible | Anthropic-compatible |
|
|
|---|---|---|
|
|
| Base URL | `https://dashscope-intl.aliyuncs.com/compatible-mode/v1` | `https://dashscope-intl.aliyuncs.com/apps/anthropic` |
|
|
| Endpoints | full Chat Completions surface | `/v1/messages` only (no `/v1/models`) |
|
|
| Purpose | the documented developer API | a shim, documented around hosting Claude Code |
|
|
|
|
So the question this ADR answers is not "which client do we reuse" but
|
|
"which of Alibaba's two wire protocols does the built-in speak".
|
|
|
|
## Decision
|
|
|
|
**The `qwen` built-in and the `qwen://` DSN scheme speak OpenAI-compat**, over
|
|
`provider/openai` — no new package, mirroring ADR-0026 (kimi) and ADR-0015
|
|
(llama-swap chat). Default base URL is the international host; the China host
|
|
(`dashscope.aliyuncs.com/compatible-mode/v1`) and workspace-scoped regional
|
|
hosts are reachable with a `qwen://` DSN.
|
|
|
|
Credential handling is copied from kimi verbatim, because both of its rules
|
|
are load-bearing: `WithAPIKey` is passed unconditionally (even empty) so an
|
|
unset `QWEN_API_KEY` can never fall through to `openai.New`'s `OPENAI_API_KEY`
|
|
default, and `WithAPIKeyName("QWEN_API_KEY")` makes the synthetic-401 hint name
|
|
the variable the operator actually has to set.
|
|
|
|
The kimi and qwen DSN factories were identical, so they now share one
|
|
`openaiCompatScheme` helper — the next OpenAI-compat built-in gets the
|
|
credential and key-hint rules by construction rather than by copy.
|
|
|
|
### Why not the Anthropic-compatible endpoint
|
|
|
|
Every concrete difference favors OpenAI-compat *for this codebase*:
|
|
|
|
- **Reasoning survives the trip.** Model Studio takes `reasoning_effort` as a
|
|
top-level field on the OpenAI surface, which `provider/openai` already sends
|
|
— `llm.WithReasoningEffort` works on qwen with zero qwen-specific code
|
|
(`TestQwenReasoningEffortReachesWire` asserts it on the wire). Down the
|
|
anthropic client it would be dropped in silence: `provider/anthropic`
|
|
deliberately ignores `Request.ReasoningEffort`, because first-party Claude
|
|
has no such knob.
|
|
- **Structured output would regress.** `provider/anthropic` implements
|
|
`Request.Schema` with the first-party GA `output_config.format` mechanism.
|
|
Alibaba's shim does not implement it; a compat endpoint that ignores an
|
|
unknown field returns unconstrained prose while still reporting success.
|
|
The OpenAI path sends `response_format: json_schema`, which Model Studio
|
|
supports natively on the Max/Plus families.
|
|
- **Cache accounting already lands.** Model Studio's implicit prefix cache
|
|
reports hits in `usage.prompt_tokens_details.cached_tokens`, which the openai
|
|
client already maps to `llm.Usage.CacheReadTokens`. The anthropic client
|
|
reads `cache_read_input_tokens`, a field the shim has no reason to emit.
|
|
- **Thinking content is discarded on the anthropic path anyway.**
|
|
`provider/anthropic` skips `thinking` blocks in both the buffered and
|
|
streaming decoders, so the shim's headline feature — first-class
|
|
`thinking: {type: "enabled", budget_tokens: N}` — buys majordomo nothing
|
|
today.
|
|
- **Smaller blast radius.** The anthropic client has no `WithAPIKeyName`
|
|
option, so a keyless qwen would tell the operator to set `ANTHROPIC_API_KEY`;
|
|
fixing that means changing the first-party Anthropic client to serve a
|
|
third-party shim.
|
|
- **It is the less-exercised surface.** The Anthropic endpoint is documented as
|
|
Messages-only, with a temperature range that differs from Anthropic's own
|
|
([0, 2) vs [0.0, 1.0]) — i.e. it is Qwen semantics wearing an Anthropic
|
|
envelope, not an Anthropic-equivalent target.
|
|
|
|
The one thing the Anthropic surface offers that OpenAI-compat does not is
|
|
explicit `cache_control` breakpoints reached through `Request.PromptCache`.
|
|
That is not a reason to route Qwen through it: Model Studio's implicit cache is
|
|
automatic and already metered, and if explicit breakpoints ever matter they
|
|
belong in `provider/openai` (Model Studio accepts `cache_control` on content
|
|
blocks there too), where every OpenAI-compat target would get them.
|
|
|
|
## Consequences
|
|
|
|
- `qwen/<model>` is first-class in Parse, chains, aliases, and health/failover
|
|
with no consumer wiring; model ids pass through verbatim (no catalog).
|
|
- Chat, streaming, tools, structured output, reasoning effort, and cached-token
|
|
accounting all ride the openai client and inherit its fixes.
|
|
- Image *inputs* work at the client level, but only the `qwen-vl-*` /
|
|
`qwen3-vl-*` models accept them (matrix footnote ⁴; ³ is kimi's).
|
|
- Two model-side quirks are Alibaba's, not majordomo's, and are left to the
|
|
caller rather than papered over: thinking is **on by default** on some models
|
|
(e.g. `qwen3.7-plus`), and Qwen3 *open-source* models require streaming when
|
|
thinking is enabled — a buffered `Generate` against one of those needs a
|
|
model that supports non-streaming thinking (the Max/Plus families do).
|
|
- If a future consumer genuinely needs the Anthropic surface, it is reachable
|
|
today without library changes:
|
|
`LLM_QWEN_ANTHROPIC=anthropic://[email protected]/apps/anthropic`
|
|
— with the reasoning/structured-output caveats above.
|
|
- Second third-party built-in after kimi. The ADR-0026 bar still holds: a named
|
|
consumer needs it in-config. `RegisterProvider`/`LLM_*` remain the path for
|
|
everything else.
|