A failover chain previously treated a successful-but-empty completion (no content parts and no tool calls — a "stop with nothing") as a valid result and returned it. The agent loop then ended the run with empty output, and the configured backup models were never tried because no error was raised. This let a single flaky model silently terminate an agent/skill run with no answer (observed in the wild with ollama-cloud/glm-5.2 returning empty completions right after a large tool/think turn). - Add llm.ErrEmptyResponse (classified transient) and Response.IsEmpty(): true only when there are no tool calls and no meaningful content (no parts, or whitespace-only text). A media/image part counts as content, so image-only responses are NOT empty. - chain.Generate converts an empty completion into ErrEmptyResponse so the chain fails over to the next target. Unlike an ordinary transient it is NOT retried on the same target (the model just produced it; these calls are expensive) — the chain penalizes health (so a persistently-empty target benches) and advances immediately. - When every target returns empty the call fails with ErrChainExhausted joined to ErrEmptyResponse — a visible error instead of a hollow success. Single-element chains therefore also surface empties as errors. Stream path is unchanged (can't inspect content before the consumer reads it). Tests: Response.IsEmpty table; chain fails over past an empty head; all-empty chain returns ErrChainExhausted/ErrEmptyResponse; repeated empties bench the target across requests. Full suite green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
6.9 KiB
CLAUDE.md — majordomo operating manual
majordomo is a clean-slate Go substrate for LLM-backed agents: target-agnostic model access, a parseable model naming / failover / tiering system with health tracking, multimodality, tool calls, structured output, and agents composed from model + system prompt + toolboxes + skills.
Public, vibe-coded project. This is built almost entirely by an AI agent (Claude Code) and is public. Keep that framing honest in the README — don't oversell it — and keep the README/support-matrix/examples updated in the same commit as the behavior they describe (that in-sync promise is part of the project's credibility).
North star: majordomo exists to re-architect mort's agentic layer. mort is the first consumer and the design's acceptance test — when a choice is a toss-up, pick what makes mort's tiers, failover chains, toolboxes, and skills cleanest to express. But majordomo itself stays general-purpose and mort-agnostic: no mort types, no Discord, no mort config.
Module & stack
- Module:
gitea.stevedudenhoeffer.com/steve/majordomo, Go 1.26. - Stdlib-first (ADR-0007): hand-rolled
net/httpclients for OpenAI(+compat), Anthropic(+compat), Ollama (cloud+local), foreman. The one approved dependency isgoogle.golang.org/genai(Google provider). Anything else needs an ADR. Nogo-llm, nogo-agentkit— importing either is an automatic failure.
Package map (ADR-0001)
majordomo Registry, Parse, env-DSN loading, chain executor, re-exports
llm/ canonical contract: Message/Part/Request/Response/Option,
Tool/Toolbox, Capabilities, Stream, Model, Provider, errors
health/ clock-injected health tracker (bench/backoff)
media/ image normalization to target capabilities (sniff real
format, downscale, transcode, byte ladder; ErrUnsupported
for what can't fit) — chains normalize PER TARGET
provider/fake/ scriptable in-memory provider for hermetic tests
provider/openai/ Chat Completions client (+ all OpenAI-compat targets)
provider/anthropic/ Messages API client (+ Anthropic-compat targets)
provider/ollama/ one native /api/chat client serving the ollama,
ollama-cloud, and foreman built-ins via presets
provider/google/ Gemini on google.golang.org/genai (the one approved
dependency; lazy client, raw-JSON-schema tools,
ThinkingLevel reasoning, iter.Pull2 streaming)
agent/ Agent run loop (Phase 5)
skill/ Skill interface + composition (Phase 6)
examples/ one runnable example per hard requirement (Phase 7-8)
Canonical types live in leaf package llm; the root re-exports them via
type aliases. Providers import llm, never each other, never the root.
Parse grammar (ADR-0003)
spec := element ("," element)* # ordered failover chain
element := target | alias
target := provider "/" model # model id VERBATIM after first "/"
alias := bare token (no slash), expands INLINE, recursively, cycle-checked
Parse("ollama-cloud/minimax-m3:cloud,ollama-cloud/kimi-k2.6:cloud,anthropic/opus-4.8")→ try head-to-tail. Appending,thinkingexpands the registered alias in place at the tail.- Provider resolution: registry (built-ins, RegisterProvider, eager env) →
lazy
LLM_{UPPER(name)}env DSN → error. - Single element ≡ chain of one; same Model interface, same semantics.
- No reasoning suffixes (
:highetc. are NOT stripped — model ids are verbatim). Reasoning effort becomes a request option (provider phases).
LLM_* env-DSN providers (ADR-0004, go-llm parity)
LLM_<NAME>=scheme://[token@]host[/path] — e.g.
LLM_M5=foreman://token@foreman-m5.example defines provider m5; then
m5/qwen3:30b works in Parse, chains, and aliases. Scheme ∈ {foreman,
ollama, ollama-cloud, openai, anthropic, google, gemini} ∪ RegisterScheme.
Token = credential; base URL = https://host always. New() scans the
process env eagerly; unknown names also resolve lazily at Parse time
(my-prov → LLM_MY_PROV). Malformed entries fail on use, not at startup.
Health & failover (ADR-0006, ADR-0008)
- Transient (408/429/5xx, timeouts, conn refused/reset, DNS, deadline) vs permanent (400/401/403/404/405/422, model-not-found, ctx.Canceled). Unknown → transient. Classifier overridable.
- One transient error → retry same target (default 1 retry). Every failed attempt counts; at threshold (default 2 consecutive) the target is benched for base 5s × 2^n, capped 5m. Success fully resets. Chains skip benched targets; 404 advances penalty-free; auth/malformed fail fast (configurable); exhaustion returns a joined error naming every target.
- Empty response = failover. A target that returns without error but
with no usable output — no content parts and no tool calls (
Response.IsEmpty; a media/image part counts as content) — is treated as a per-target failure (llm.ErrEmptyResponse, classified transient). Unlike an ordinary transient it is not retried on the same target (the model just did this; the call is expensive): the chain penalizes health and advances immediately. If every target comes back empty the call fails withErrChainExhaustedrather than a hollow "successful" empty completion, so a single flaky model can't silently end an agent run with nothing. - Tracker is in-memory, process-local, clock-injected. No persistence.
House conventions (mirror foreman)
- gofmt; check errors immediately and wrap with
fmt.Errorf("%w: ..."); imports stdlib → third-party → internal;// Why:doc comments where rationale isn't obvious. - ADRs in
docs/adr/, one decision each, append-only, indexed in its README. progress.md gets a dated entry per phase. - Conventional commits (
feat:,test:,docs:,chore:,refactor:). - Tests are hermetic: fake provider + fake clock; provider clients test
against
httptest; no network or credentials in the default suite. Live tests sit behind//go:build live/examples/live/and skip without their env vars. .envholds live keys (gitignored, never committed/printed/quoted);.env.examplecarries placeholders.
Gates (every phase; what CI runs)
go build ./...
go vet ./...
go test -race -count=1 ./...
go mod tidy && git diff --exit-code go.mod go.sum
CI: .gitea/workflows/ci.yaml (Gitea Actions, mirrors foreman). README.md
must match reality in the same commit that changes behavior — no
aspirational docs; unbuilt features are marked pending in the matrix.
Out of scope (anti-creep)
No persistent store (health is in-memory behind the registry), no observability/metrics stack, no config-file framework beyond LLM_* env DSNs, no CLI beyond examples, no provider-specific features leaking into the canonical API, nothing mort-specific in the library.