steve/majordomo

Fork 0

Files

T

steveandClaude Opus 5 f1f2b653c3

CI / Tidy (pull_request) Successful in 9m25s

Details

CI / Build & Test (pull_request) Successful in 9m50s

Details

refactor: gadfly round 2 — both halves of an OpenAI-compat built-in register together

Same class of finding as round 1, one level in: I factored the DSN-scheme half
of the kimi/qwen duplication into openaiCompatScheme and left the eager
provider half copy-pasted, so a third built-in still had six lines to clone —
including both credential rules, which is exactly the pair you do not want
re-typed.

registerOpenAICompatBuiltin now installs both halves from one call. The rules
that matter hold by construction for every future caller: WithAPIKey passed
unconditionally (an unset key must not fall through to OPENAI_API_KEY), and
WithAPIKeyName naming that same variable in the 401 hint. Registering kimi and
qwen is now one line each.

Also fixed a cross-reference the ADR got wrong: Qwen's image-input caveat is
README matrix footnote ⁴, not ³ — ³ is kimi's. I wrote "³, shared with kimi"
in the ADR and then gave Qwen its own footnote in the README.

The break-check harness needed fixing before any of this could be trusted:
three of its mutations targeted lines this refactor moved, so they matched
nothing, the code was never broken, and the suite reported "test still passed"
— identical output to a test that genuinely misses the bug. Mutations are now
verified to have landed (sha before/after) and the suite fails loudly if one
doesn't. Two new cases cover the helper: dropping the unconditional WithAPIKey,
and dropping the scheme-half registration. 8/8 apply and are caught.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

2026-08-12 16:25:38 -04:00

5.6 KiB

Raw Blame History

ADR-0027: Qwen (Alibaba) built-in provider — OpenAI-compat, not Anthropic-compat

Status: Accepted — 2026-08-12

Context

Alibaba's Qwen models (qwen3.8-max, qwen3.7-plus, the qwen3-vl-* vision variants, …) are served from Model Studio / DashScope, and mort wants them as a first-class failover tier with a dedicated QWEN_API_KEY — the same ergonomics ADR-0026 gave Kimi.

Unlike Kimi, Model Studio exposes the same models over two protocols:

	OpenAI-compatible	Anthropic-compatible
Base URL	`https://dashscope-intl.aliyuncs.com/compatible-mode/v1`	`https://dashscope-intl.aliyuncs.com/apps/anthropic`
Endpoints	full Chat Completions surface	`/v1/messages` only (no `/v1/models`)
Purpose	the documented developer API	a shim, documented around hosting Claude Code

So the question this ADR answers is not "which client do we reuse" but "which of Alibaba's two wire protocols does the built-in speak".

Decision

The qwen built-in and the qwen:// DSN scheme speak OpenAI-compat, over provider/openai — no new package, mirroring ADR-0026 (kimi) and ADR-0015 (llama-swap chat). Default base URL is the international host; the China host (dashscope.aliyuncs.com/compatible-mode/v1) and workspace-scoped regional hosts are reachable with a qwen:// DSN.

Credential handling is copied from kimi verbatim, because both of its rules are load-bearing: WithAPIKey is passed unconditionally (even empty) so an unset QWEN_API_KEY can never fall through to openai.New's OPENAI_API_KEY default, and WithAPIKeyName("QWEN_API_KEY") makes the synthetic-401 hint name the variable the operator actually has to set.

The kimi and qwen DSN factories were identical, so they now share one openaiCompatScheme helper — the next OpenAI-compat built-in gets the credential and key-hint rules by construction rather than by copy.

Why not the Anthropic-compatible endpoint

Every concrete difference favors OpenAI-compat for this codebase:

Reasoning survives the trip. Model Studio takes reasoning_effort as a top-level field on the OpenAI surface, which provider/openai already sends — llm.WithReasoningEffort works on qwen with zero qwen-specific code (TestQwenReasoningEffortReachesWire asserts it on the wire). Down the anthropic client it would be dropped in silence: provider/anthropic deliberately ignores Request.ReasoningEffort, because first-party Claude has no such knob.
Structured output would regress. provider/anthropic implements Request.Schema with the first-party GA output_config.format mechanism. Alibaba's shim does not implement it; a compat endpoint that ignores an unknown field returns unconstrained prose while still reporting success. The OpenAI path sends response_format: json_schema, which Model Studio supports natively on the Max/Plus families.
Cache accounting already lands. Model Studio's implicit prefix cache reports hits in usage.prompt_tokens_details.cached_tokens, which the openai client already maps to llm.Usage.CacheReadTokens. The anthropic client reads cache_read_input_tokens, a field the shim has no reason to emit.
Thinking content is discarded on the anthropic path anyway. provider/anthropic skips thinking blocks in both the buffered and streaming decoders, so the shim's headline feature — first-class thinking: {type: "enabled", budget_tokens: N} — buys majordomo nothing today.
Smaller blast radius. The anthropic client has no WithAPIKeyName option, so a keyless qwen would tell the operator to set ANTHROPIC_API_KEY; fixing that means changing the first-party Anthropic client to serve a third-party shim.
It is the less-exercised surface. The Anthropic endpoint is documented as Messages-only, with a temperature range that differs from Anthropic's own ([0, 2) vs [0.0, 1.0]) — i.e. it is Qwen semantics wearing an Anthropic envelope, not an Anthropic-equivalent target.

The one thing the Anthropic surface offers that OpenAI-compat does not is explicit cache_control breakpoints reached through Request.PromptCache. That is not a reason to route Qwen through it: Model Studio's implicit cache is automatic and already metered, and if explicit breakpoints ever matter they belong in provider/openai (Model Studio accepts cache_control on content blocks there too), where every OpenAI-compat target would get them.

Consequences

qwen/<model> is first-class in Parse, chains, aliases, and health/failover with no consumer wiring; model ids pass through verbatim (no catalog).
Chat, streaming, tools, structured output, reasoning effort, and cached-token accounting all ride the openai client and inherit its fixes.
Image inputs work at the client level, but only the qwen-vl-* / qwen3-vl-* models accept them (matrix footnote ⁴; ³ is kimi's).
Two model-side quirks are Alibaba's, not majordomo's, and are left to the caller rather than papered over: thinking is on by default on some models (e.g. qwen3.7-plus), and Qwen3 open-source models require streaming when thinking is enabled — a buffered Generate against one of those needs a model that supports non-streaming thinking (the Max/Plus families do).
If a future consumer genuinely needs the Anthropic surface, it is reachable today without library changes: LLM_QWEN_ANTHROPIC=anthropic://[email protected]/apps/anthropic — with the reasoning/structured-output caveats above.
Second third-party built-in after kimi. The ADR-0026 bar still holds: a named consumer needs it in-config. RegisterProvider/LLM_* remain the path for everything else.

5.6 KiB Raw Blame History