steve/gadfly
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request 'feat(qwen): let Qwen (and Kimi) join the swarm' (#30)
Build & push image / build-and-push (push) Successful in 39s
Details
Build & push image / test (push) Successful in 9m42s
Details

Browse Source
This commit was merged in pull request #30.
This commit is contained in:
steve
2026-08-12 22:46:04 +00:00
parent d272695c16 d8b023efd6
commit b850e35b35
12 changed files with 842 additions and 30 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+33 -1
View File
@@ -73,11 +73,43 @@ majordomo failover chain / alias) is used verbatim.
| **[llama-swap](https://github.com/mostlygeek/llama-swap)** (model-swapping proxy) | `llama-swap`/`llama-swaps` (un-hyphenated `llamaswap`/`llamaswaps` also accepted) + `GADFLY_BASE_URL` or a `GADFLY_ENDPOINT_*` entry, or an `LLM_*` `llama-swap://` / `llama-swaps://` DSN | optional bearer | ⚠️ wired, **untested** |
| **OpenAI-compatible** (incl. local Ollama's `/v1`) | `openai` + `GADFLY_BASE_URL` | `OPENAI_API_KEY` (any non-empty for Ollama) | ✅ tested against Ollama |
| **OpenAI** | `openai` | `OPENAI_API_KEY` | ⚠️ wired, **untested** |
| **Qwen** (Alibaba Model Studio) | `qwen` | `QWEN_API_KEY` | ⚠️ wired, **untested** |
| **Kimi** (Moonshot) | `kimi` | `KIMI_API_KEY` | ⚠️ wired, **untested** |
| **Anthropic** | `anthropic` | `ANTHROPIC_API_KEY` | ⚠️ wired, **untested** |
| **Google (Gemini)** | `google` | `GOOGLE_API_KEY` / `GEMINI_API_KEY` | ⚠️ wired, **untested** |
Qwen and Kimi are majordomo built-ins that speak the OpenAI protocol at their own
endpoints, so `qwen/qwen3.8-max` or `kimi/kimi-k2-0711-preview` work as
`GADFLY_MODELS` entries with only the matching key set. Each reads **only** its own
variable — no cross-provider fallback — so forgetting to forward `QWEN_API_KEY`
gets you a skip notice naming it, not a mis-keyed call. Note `kimi/<model>` (Moonshot's
API, `KIMI_API_KEY`) is a different route than the `kimi-k2.6:cloud` entry in the
default swarm, which is Ollama Cloud and keyed by `OLLAMA_CLOUD_API_KEY`.
> **Qwen keys are endpoint-scoped, and the failure looks like a bad key.**
> Alibaba Model Studio issues *workspace-scoped* endpoints of the form
> `https://<workspace>.<region>.maas.aliyuncs.com/compatible-mode/v1`. A key
> issued for one host is rejected by another with a genuine
> `401 Incorrect API key provided` — so a perfectly good key reads as invalid if
> the endpoint doesn't match. The built-in defaults to the shared international
> host; point at your own with a named endpoint, which needs no code change:
>
> ```
> GADFLY_ENDPOINT_QWENWS = "qwen|https://<workspace>.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"
> GADFLY_MODELS = "qwenws/qwen3.8-max,..."
> QWEN_API_KEY = <secret>
> ```
>
> **Leave the key out of the endpoint var.** `GADFLY_ENDPOINT_*` are Gitea
> *variables*, which are not masked in logs; the third `|<key>` field would put
> a credential there. Omit it and a `qwen`/`kimi` endpoint falls back to its own
> `QWEN_API_KEY` / `KIMI_API_KEY` secret — its own vendor's key, never another's.
>
> (Verified the hard way against a live deployment.)
> ### 🧪 Honest status
> Only the **Ollama** paths above are actually exercised. The OpenAI / Anthropic / Google
> Only the **Ollama** paths above are actually exercised. The OpenAI / Qwen / Kimi /
> Anthropic / Google
> providers come "for free" from majordomo's abstraction and *should* work, but I haven't
> spent money verifying them — treat them as untested. The OpenAI-**compatible** path **is**
> tested, because you can point it at a local Ollama (`GADFLY_BASE_URL=http://localhost:11434/v1`)