Files
majordomo/CLAUDE.md
T
steve 74474c6da0
CI / Tidy (push) Successful in 9m26s
CI / Build & Test (push) Successful in 10m29s
feat(chain): fail over on empty/degenerate responses
A failover chain previously treated a successful-but-empty completion (no
content parts and no tool calls — a "stop with nothing") as a valid result
and returned it. The agent loop then ended the run with empty output, and
the configured backup models were never tried because no error was raised.
This let a single flaky model silently terminate an agent/skill run with
no answer (observed in the wild with ollama-cloud/glm-5.2 returning empty
completions right after a large tool/think turn).

- Add llm.ErrEmptyResponse (classified transient) and Response.IsEmpty():
  true only when there are no tool calls and no meaningful content (no
  parts, or whitespace-only text). A media/image part counts as content,
  so image-only responses are NOT empty.
- chain.Generate converts an empty completion into ErrEmptyResponse so the
  chain fails over to the next target. Unlike an ordinary transient it is
  NOT retried on the same target (the model just produced it; these calls
  are expensive) — the chain penalizes health (so a persistently-empty
  target benches) and advances immediately.
- When every target returns empty the call fails with ErrChainExhausted
  joined to ErrEmptyResponse — a visible error instead of a hollow success.
  Single-element chains therefore also surface empties as errors.

Stream path is unchanged (can't inspect content before the consumer reads
it). Tests: Response.IsEmpty table; chain fails over past an empty head;
all-empty chain returns ErrChainExhausted/ErrEmptyResponse; repeated
empties bench the target across requests. Full suite green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 10:35:07 -04:00

6.9 KiB
Raw Blame History

CLAUDE.md — majordomo operating manual

majordomo is a clean-slate Go substrate for LLM-backed agents: target-agnostic model access, a parseable model naming / failover / tiering system with health tracking, multimodality, tool calls, structured output, and agents composed from model + system prompt + toolboxes + skills.

Public, vibe-coded project. This is built almost entirely by an AI agent (Claude Code) and is public. Keep that framing honest in the README — don't oversell it — and keep the README/support-matrix/examples updated in the same commit as the behavior they describe (that in-sync promise is part of the project's credibility).

North star: majordomo exists to re-architect mort's agentic layer. mort is the first consumer and the design's acceptance test — when a choice is a toss-up, pick what makes mort's tiers, failover chains, toolboxes, and skills cleanest to express. But majordomo itself stays general-purpose and mort-agnostic: no mort types, no Discord, no mort config.

Module & stack

  • Module: gitea.stevedudenhoeffer.com/steve/majordomo, Go 1.26.
  • Stdlib-first (ADR-0007): hand-rolled net/http clients for OpenAI(+compat), Anthropic(+compat), Ollama (cloud+local), foreman. The one approved dependency is google.golang.org/genai (Google provider). Anything else needs an ADR. No go-llm, no go-agentkit — importing either is an automatic failure.

Package map (ADR-0001)

majordomo        Registry, Parse, env-DSN loading, chain executor, re-exports
  llm/           canonical contract: Message/Part/Request/Response/Option,
                 Tool/Toolbox, Capabilities, Stream, Model, Provider, errors
  health/        clock-injected health tracker (bench/backoff)
  media/         image normalization to target capabilities (sniff real
                 format, downscale, transcode, byte ladder; ErrUnsupported
                 for what can't fit) — chains normalize PER TARGET
  provider/fake/ scriptable in-memory provider for hermetic tests
  provider/openai/    Chat Completions client (+ all OpenAI-compat targets)
  provider/anthropic/ Messages API client (+ Anthropic-compat targets)
  provider/ollama/    one native /api/chat client serving the ollama,
                      ollama-cloud, and foreman built-ins via presets
  provider/google/    Gemini on google.golang.org/genai (the one approved
                      dependency; lazy client, raw-JSON-schema tools,
                      ThinkingLevel reasoning, iter.Pull2 streaming)
  agent/         Agent run loop                               (Phase 5)
  skill/         Skill interface + composition                (Phase 6)
  examples/      one runnable example per hard requirement    (Phase 7-8)

Canonical types live in leaf package llm; the root re-exports them via type aliases. Providers import llm, never each other, never the root.

Parse grammar (ADR-0003)

spec    := element ("," element)*       # ordered failover chain
element := target | alias
target  := provider "/" model           # model id VERBATIM after first "/"
alias   := bare token (no slash), expands INLINE, recursively, cycle-checked
  • Parse("ollama-cloud/minimax-m3:cloud,ollama-cloud/kimi-k2.6:cloud,anthropic/opus-4.8") → try head-to-tail. Appending ,thinking expands the registered alias in place at the tail.
  • Provider resolution: registry (built-ins, RegisterProvider, eager env) → lazy LLM_{UPPER(name)} env DSN → error.
  • Single element ≡ chain of one; same Model interface, same semantics.
  • No reasoning suffixes (:high etc. are NOT stripped — model ids are verbatim). Reasoning effort becomes a request option (provider phases).

LLM_* env-DSN providers (ADR-0004, go-llm parity)

LLM_<NAME>=scheme://[token@]host[/path] — e.g. LLM_M5=foreman://token@foreman-m5.example defines provider m5; then m5/qwen3:30b works in Parse, chains, and aliases. Scheme ∈ {foreman, ollama, ollama-cloud, openai, anthropic, google, gemini} RegisterScheme. Token = credential; base URL = https://host always. New() scans the process env eagerly; unknown names also resolve lazily at Parse time (my-provLLM_MY_PROV). Malformed entries fail on use, not at startup.

Health & failover (ADR-0006, ADR-0008)

  • Transient (408/429/5xx, timeouts, conn refused/reset, DNS, deadline) vs permanent (400/401/403/404/405/422, model-not-found, ctx.Canceled). Unknown → transient. Classifier overridable.
  • One transient error → retry same target (default 1 retry). Every failed attempt counts; at threshold (default 2 consecutive) the target is benched for base 5s × 2^n, capped 5m. Success fully resets. Chains skip benched targets; 404 advances penalty-free; auth/malformed fail fast (configurable); exhaustion returns a joined error naming every target.
  • Empty response = failover. A target that returns without error but with no usable output — no content parts and no tool calls (Response.IsEmpty; a media/image part counts as content) — is treated as a per-target failure (llm.ErrEmptyResponse, classified transient). Unlike an ordinary transient it is not retried on the same target (the model just did this; the call is expensive): the chain penalizes health and advances immediately. If every target comes back empty the call fails with ErrChainExhausted rather than a hollow "successful" empty completion, so a single flaky model can't silently end an agent run with nothing.
  • Tracker is in-memory, process-local, clock-injected. No persistence.

House conventions (mirror foreman)

  • gofmt; check errors immediately and wrap with fmt.Errorf("%w: ..."); imports stdlib → third-party → internal; // Why: doc comments where rationale isn't obvious.
  • ADRs in docs/adr/, one decision each, append-only, indexed in its README. progress.md gets a dated entry per phase.
  • Conventional commits (feat:, test:, docs:, chore:, refactor:).
  • Tests are hermetic: fake provider + fake clock; provider clients test against httptest; no network or credentials in the default suite. Live tests sit behind //go:build live / examples/live/ and skip without their env vars.
  • .env holds live keys (gitignored, never committed/printed/quoted); .env.example carries placeholders.

Gates (every phase; what CI runs)

go build ./...
go vet ./...
go test -race -count=1 ./...
go mod tidy && git diff --exit-code go.mod go.sum

CI: .gitea/workflows/ci.yaml (Gitea Actions, mirrors foreman). README.md must match reality in the same commit that changes behavior — no aspirational docs; unbuilt features are marked pending in the matrix.

Out of scope (anti-creep)

No persistent store (health is in-memory behind the registry), no observability/metrics stack, no config-file framework beyond LLM_* env DSNs, no CLI beyond examples, no provider-specific features leaking into the canonical API, nothing mort-specific in the library.