70 Commits

Author SHA1 Message Date
steve 8ecdadf8b8 feat: first-class skill packs on agents + ship gifsmith builtin
executus CI / test (push) Successful in 3m21s
Lifts the 'an agent uses a SKILL.md pack' concept out of a host and into the
harness:
- run.Ports.SkillPacks (SkillPackActivator) — nil-safe port; the executor folds
  a loaded agent's pack catalog into the system prompt and adds a skill_use
  loader tool to the toolbox (uses the existing ra.SystemPrompt + toolbox seams)
- run.RunnableAgent.SkillPacks + persona.Agent.SkillPacks (+ skill_packs YAML,
  extends-inherit, ToRunnable) — the Agent noun is now pack-aware
- skillpack.Activator — the battery's default port impl (resolve names → packs →
  catalog + skill_use), with a per-run BundleStager factory the host plumbs;
  satisfies the port structurally (no import of run)
- agentbuiltins: ships gifsmith, a portable focused GIF/MP4 render agent that
  uses the gif pack — references tool/tier/pack NAMES only, no host coupling

A host now wires run.Ports.SkillPacks instead of carrying its own activation
glue. Tests: Activator resolution + gifsmith loads through persona→RunnableAgent.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 01:05:58 -04:00
steve d5ea9b6e5e Merge pull request 'feat(skillpack): SKILL.md-subscription battery' (#22) from feat/skillpack-battery into main
executus CI / test (push) Successful in 3m3s
2026-07-05 01:28:57 +00:00
steve 29598df814 feat(skillpack): lazy BundleStager for bundled files in skill_use
executus CI / test (pull_request) Successful in 2m19s
Replace Activate's stagedDir string with a BundleStager callback invoked
lazily inside skill_use: when the model loads a pack with bundled files, the
host stages them (mort: into run-scoped file storage) and the returned note is
appended to the body so the model knows how to reach them. A nil stager (or a
stager error) degrades gracefully to just listing the file names.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 20:56:05 -04:00
steve 9bb5d143f7 fix(skillpack): address review — symlink read, git arg-injection, dup-subscribe, nil panics
executus CI / test (pull_request) Successful in 3m30s
Real issues from the PR review:
- security: readTree now skips symlinks (a pack with SKILL.md -> /etc/passwd
  or scripts/x -> ../../.ssh/id_rsa could read host files); covers file and
  dir symlinks, incl. within a git subpath
- security: GitSource rejects url/ref beginning with '-' (git arg injection)
  and clones with '--' separator; --filter=blob:none (blobless partial clone)
  instead of full-history clone
- correctness: Subscribe no longer swallows a non-ErrNotFound store error from
  GetByName (would create a duplicate subscription); handles *GitSource as well
  as GitSource in the URL/subpath extraction
- correctness: pinTo no longer renames a subscription, so Apply can't silently
  collide two subscriptions when an upstream pack changes its name
- validation: isKebab rejects leading/trailing/consecutive hyphens; BOM-
  prefixed SKILL.md now parses (matches the doc comment)
- robustness: Catalog/Activate/renderPackBody/Stage guard nil/malformed packs
- test cleanup: Syncer.Store field renamed Cache (collided with the Store
  interface); test NewID returns distinct ids
- tests: symlink-skip, BOM, strict-kebab, nil-pack-safety

Deferred (advisory perf, documented): PackCache stores raw trees so activation
re-parses; CheckAll is serial. Both fine at expected scale.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 20:41:44 -04:00
steve bf0b67f9af feat(skillpack): SKILL.md-subscription battery
executus CI / test (pull_request) Successful in 1m6s
Adversarial Review (Gadfly) / review (pull_request) Successful in 15m50s
New additive, nil-safe battery for subscribing to skill packages in the
Anthropic agent-skills format (SKILL.md manifest + bundled files):

- Manifest/ParseManifest: SKILL.md frontmatter+body parse & validation
  (name/description required, allowed-tools passthrough, kebab/length limits)
- Tree/Pack/LoadPack: self-contained file set, order-independent content
  digest (the pin identity + change signal), bundled-file listing, traversal-
  safe staging
- Source (DirSource, GitSource): Fetch returns tree + resolved ref; git clones
  to temp, reads subpath into memory, cleans up (self-contained tree)
- Subscription + Store + content-addressed PackCache, with Memory defaults
- Syncer: Subscribe pins; Check records a PENDING update but never moves the
  pin; Apply is the only re-pin (supply-chain guard — upstream can't silently
  change what an agent runs)
- Activate: resolved packs -> majordomo agent.Skill (catalog instructions +
  one skill_use tool) for progressive disclosure; Stage materializes files

Third distinct 'skill' concept, deliberately separate from executus/skill
(saved-agent noun) and majordomo/skill (eager capability bundle). Mort-side
wiring (convars, .skillpack commands, Agent.SkillPacks, allowed-tools shim)
is a later, separate step. Full unit + hermetic local-git tests; gofmt/vet
clean; race-tested.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 16:46:37 -04:00
steve 2a43210f38 Merge pull request 'feat(run): critic owns the deadline — MaxRuntime becomes the soft trigger' (#21) from feat/critic-owns-deadline into main
executus CI / test (push) Successful in 1m45s
2026-06-30 15:56:31 +00:00
steve 79ce833dd7 fix(run): address round-2 gadfly nits (max(), drop dead soft fallback, decouple doc)
executus CI / test (pull_request) Successful in 49s
All low-severity follow-ups on the critic-deadline change:
- hardCap uses max(CriticAbsoluteMax, maxRuntime) instead of a nested if (723193a7).
- Drop the now-dead 90s soft-trigger fallback + its bare literal: the sole caller
  passes the resolved MaxRuntime (>0), and Run's unsupervised-run failsafe bounds
  even an impossible 0 (8d377051, 2f86bf58).
- Decouple the kernel doc from a named downstream convar ("a 6h host convar")
  (730c67fc).

Graded false-positive: agent.go BackstopMultiplier validation (handled in the host;
not in this diff), the 24h default "magic number" (matches every withFallbacks
default), and the defer-in-conditional pattern (idiomatic). Kept: the thorough
two-tier comment (this logic regressed once) and the rare-path nested timer.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Jo75sqmeVPgFUWZQBn179X
2026-06-30 11:54:38 -04:00
steve cb4c612461 fix(run): address gadfly review of the critic-deadline PR
executus CI / test (pull_request) Successful in 1m45s
All 11 findings were real (3 clusters):

- Failsafe ceiling could pre-empt the critic's backstop (e9c9483f, 9109317b,
  d5a9bf0d, 76ad171e): CriticAbsoluteMax was 6h, but the host's backstop
  (MaxRuntime × multiplier, or its own absolute max) can reach 6h+, so the
  ceiling fired first and reintroduced a premature hard cap. Now CriticAbsoluteMax
  is a 24h RUNAWAY guard set far beyond any realistic backstop (the host clamps
  its own backstop to a much smaller absolute max, e.g. mort's 6h convar), so it
  never pre-empts a healthy supervised run. Comments corrected.

- nil Monitor handle lost the MaxRuntime cap (df016a6f, 9dd42827): a critic-enabled
  run whose host Monitor returned no handle had no deadline-watch and was bounded
  only by the generous ceiling. Added an unsupervised-run failsafe that re-wraps
  runCtx to the nominal MaxRuntime when the critic is enabled but didn't arm.
  New test TestCriticOwnsDeadline_NilHandleFallsBackToMaxRuntime.

- CriticSoftTimeout vestigial / dead fallback (f7764919, 9805bebe, 6864086f,
  b2b11721): the soft trigger is now always the resolved MaxRuntime (> 0), so the
  CriticSoftTimeout field + its startCritic fallback were unreachable. Removed the
  field entirely; the remaining 90s floor is documented as defensive-only.

- DRY (f30ce827): extracted e.criticOwnsDeadline(ra), now the single predicate used
  by both Run and startCritic so they can't drift.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Jo75sqmeVPgFUWZQBn179X
2026-06-30 11:32:46 -04:00
steve 5b5ee4148e feat(run): critic owns the deadline — MaxRuntime becomes the soft trigger
executus CI / test (pull_request) Successful in 47s
Adversarial Review (Gadfly) / review (pull_request) Successful in 23m4s
When a run enables the critic (Ports.Critic set + RunnableAgent.Critic.Enabled),
the kernel no longer hard-caps it at MaxRuntime. MaxRuntime becomes the SOFT
trigger (passed to startCritic, used by the host critic as its wake + the base
for its extendable backstop); the critic's deadline-watch is the real hard
cancel. This restores mort's old agentexec two-tier timeout semantics — a
slow-but-progressing run (e.g. a parent agent blocked on a 30-min animate render)
is given room up to the critic's backstop instead of being killed at the nominal
MaxRuntime.

Specifics:
- run/executor.go: the WithTimeout(MaxRuntime) is now conditional. Non-critic
  runs keep the literal MaxRuntime kill (→ "timeout"). Critic-owned runs get a
  GENEROUS WithTimeout at the new Defaults.CriticAbsoluteMax (default 6h) as a
  failsafe ceiling only — it never fires before the critic's backstop, and it
  guarantees a broken/nil host handle can't run unbounded.
- run/critic.go: startCritic takes the resolved MaxRuntime as the soft trigger
  (falling back to Defaults.CriticSoftTimeout, then 90s), instead of always using
  the global CriticSoftTimeout.
- Defaults.CriticAbsoluteMax added (withFallbacks default 6h).
- Tests: non-critic dies at MaxRuntime; critic-owned survives past it; soft
  trigger == MaxRuntime.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Jo75sqmeVPgFUWZQBn179X
2026-06-30 11:03:40 -04:00
steve 31f9078915 Merge pull request 'feat(run): durable checkpoint + resume (wire Ports.Checkpointer)' (#20) from feat/kernel-checkpoint into main
executus CI / test (push) Successful in 45s
2026-06-29 20:44:17 +00:00
steve 38d656ec71 fix(run): address gadfly review of the checkpoint PR
executus CI / test (pull_request) Successful in 45s
Real findings from the consensus review (44 raw; heavy devstral noise):

- finalizeCheckpoint is now fired from the top-of-Run defer, so it runs on
  EVERY exit: a panic, an early build-error return (before the run loop), AND
  normal completion. Previously an early return on a recovered run left its
  durable record unfinalized → boot recovery would retry it forever on a
  persistent build error. (opus + glm)
- Removed the dead ActivePhase field from run.RunCheckpointState +
  run.ResumeState (and the battery RunCheckpoint) — phase recovery is
  boundary-granular (skip completed phases; the interrupted phase re-runs from
  its start), so ActivePhase was never written nor read. Docs across
  ports/checkpoint/phases now state this plainly (5-model consensus that the
  field + docs over-promised mid-phase resume).
- CheckpointerFactory.Begin error is now logged (WARN) before degrading to
  non-durable, per the documented contract (was silently swallowed). (4 models)
- finalizeCheckpoint logs Complete/Fail errors (was silent).
- Resume phase-skip now keys off a SEPARATE resumeSkip set, not the live
  outputs map — a fresh run with two same-named phases no longer skips the
  second (the outputs map fills as phases run). (opus:max) + regression test.
- Removed the dead checkpoint.factory.now field (never set). (opus + glm)
- Fixed the stale phaseDeps doc (the step observer moved out of sharedOpts to
  per-path). Hoisted the resume guard to a local; dropped the wasted acc
  allocation on the resume path; documented that Save throttling is the
  Checkpointer's responsibility and the accumulated transcript is pre-compaction
  (host size-caps it).

Note (carried from the PR): classifyCheckpointOutcome keys shutdown on
run.ErrShutdown; mort stamps its own runengine.ErrShutdown — the mort wiring PR
aliases them so errors.Is matches.

New test: duplicate phase names both run on a fresh run. Full ./... green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 16:34:42 -04:00
steve 899059a791 feat(run): durable checkpoint + resume (wire Ports.Checkpointer)
executus CI / test (pull_request) Successful in 46s
Adversarial Review (Gadfly) / review (pull_request) Successful in 17m25s
The kernel defined run.Ports.Checkpointer + the checkpoint battery but never
drove them (the documented "P2 follow-up"). This wires durable recovery into
the run loop so a run interrupted by shutdown can resume on the next boot
instead of being lost — the executus-side half of mort's durable-agent-recovery
parity (mort #1355).

Kernel (run/):
- Ports.Checkpointer is now a CheckpointerFactory (Begin per run → a per-run
  Checkpointer, or nil for a non-durable run). The single per-instance
  Checkpointer couldn't distinguish runs; a factory mints one per run, matching
  mort's agentexec.CheckpointerFactory.
- RunInfo gains GuildID + ModelTier (so the factory can build resume meta);
  RunCheckpointState gains CompletedPhases + ActivePhase (+ PhaseOutput).
- run/checkpoint.go: ResumeState + WithResumeState / WithExistingCheckpointer
  context carriers, classifyCheckpointOutcome (success→Complete, shutdown→leave
  for boot recovery, else→Fail using run.ErrShutdown), and finalizeCheckpoint.
- run/executor.go: resolve the per-run checkpointer (existing-from-ctx on a
  recovery re-run, else factory.Begin); single-loop wraps the step observer to
  accumulate the transcript + Save each step (host throttles), and a recovered
  run seeds the saved transcript via WithHistory and continues with no new
  input; finalize on exit.
- run/phases.go: phase-boundary checkpointing — record completed phases after
  each phase; a resumed run skips already-completed phases (the interrupted
  phase re-runs from its start — boundary-granular, documented; only the
  single-loop path resumes mid-loop).

Battery (checkpoint/): NewFactory wires the battery into the factory port
(per-run handle, meta derived from RunInfo); RunCheckpoint + handle.Save carry
the phase fields.

Tests (run/checkpoint_test.go): the finalize decision matrix; single-loop
Save+Complete; terminal-error Fail; resume seeds history; phase-boundary Saves
completed phases; resume skips completed phases. Full ./... green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 16:04:06 -04:00
steve c071ed4996 Merge pull request 'feat(run): execute multi-phase pipelines (RunnableAgent.Phases)' (#19) from feat/kernel-phases into main
executus CI / test (push) Successful in 48s
2026-06-29 19:52:51 +00:00
steve 0dd2ced717 fix(run): address gadfly review of the phases PR
executus CI / test (pull_request) Successful in 48s
Real findings from the consensus review (37 raw; many devstral dups/noise):

- Optional/budget-salvage branches no longer swallow a context
  cancellation / deadline / critic-kill: such errors return immediately so
  the run is classified cancelled/timeout/killed, not "ok" with a fallback.
  (the most serious finding — an Optional final phase could mask a killed run)
- IsRunFunc bare phase now feeds the SHARED step observer (not just the
  audit recorder), so the critic's activity clock + Result.Steps see it —
  a long synthesize phase no longer looks idle to the critic.
- phaseModel returns the resolver's enriched (usage-attribution) context and
  the phase's calls use it, mirroring the single-loop path (non-base-tier
  phases were mis-attributed).
- salvagePhaseTranscript trims the tail on a rune boundary (was a raw byte
  slice that could split a UTF-8 rune); maxSalvage is now a named const with
  rationale.
- expandPhaseTemplate logs a WARN on parse/execute failure instead of
  silently returning the unexpanded template; documented the phase-name
  identifier requirement + the "Query" shadow.
- removed the dead phaseDeps.baseTier field.
- extracted multimodalUserMessage, shared by runAgent + the phase runner
  (was duplicated image-folding).
- aggregated phase usage is stamped onto the result even on a hard-error
  return; TrimSpace computed once; filterToolbox returns the base toolbox
  as-is for the empty-names (full-palette) case instead of copying;
  phaseModel WARN no longer prints error=<nil>.

New test: Optional phase does not swallow a cancellation. Full ./... green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 15:44:04 -04:00
steve 30b79a330f feat(run): execute multi-phase pipelines (RunnableAgent.Phases)
executus CI / test (pull_request) Successful in 1m49s
Adversarial Review (Gadfly) / review (pull_request) Successful in 13m59s
The kernel carried RunnableAgent.Phases as a DTO but never executed it —
Run always ran a single agent loop with ra.SystemPrompt, so a phased agent
(mort's deepresearch/research) silently ran one loop with the base prompt
instead of its pipeline. This implements the phase loop, ported from mort's
agentexec pipeline but reusing the kernel's own machinery.

- run/phases.go: runPhases / runOnePhase. Phases run sequentially; each is a
  fresh agent loop (or a bare LLM call for IsRunFunc phases) with its own
  template-expanded system prompt ({{.Query}} + {{.<PhaseName>}}), model
  tier, step cap, and tool subset. Outputs thread into later phases; the
  final phase's output is the run output. Optional phases swallow errors and
  substitute FallbackMessage; a non-optional phase that merely exhausts its
  step/tool budget salvages its partial transcript and continues (a hard
  error still aborts); per-phase tier-resolve failures fall back with a WARN.
- run/agent.go: Phase gains IsRunFunc + FallbackMessage (the kernel Phase
  struct previously omitted them).
- run/executor.go: Run factors the shared agent options (tool-error limits,
  step observer, compactor) and branches — single loop (critic's dynamic
  step ceiling) vs the phase runner (fixed per-phase caps; the run-level
  critic's steer + hard deadline still apply across phases). systemPrompt
  now delegates to systemPromptWithBody so each phase keeps the platform
  header. The same step observer feeds audit/steps/critic across all phases.

Tests (run/phases_test.go): sequential output threading + template
expansion, Optional-failure → FallbackMessage continues, hard-error abort,
IsRunFunc bare call, per-phase SystemHeader, filterToolbox subset, template
expansion. Full ./... suite green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 15:14:45 -04:00
Steve Dudenhoeffer b25a13ed4f chore: repin gadfly reusable to @5007597 (structured findings + consensus + inline review)
Adopts gadfly's review-representation overhaul: one ranked consensus comment
across the swarm + an advisory COMMENT-state inline PR review, on image
sha-3095ebf. Swarm config still rides the owner variables.

[skip ci]
2026-06-28 22:13:24 -04:00
steve add8f847a4 Merge pull request 'feat(run): InputFileStager seam — stage non-image attachments into the prompt' (#18) from feat/input-file-stager into main
executus CI / test (push) Successful in 1m51s
2026-06-28 18:19:28 +00:00
steve df4033f42e fix(run): harden input-file staging per gadfly #18 validation pass
executus CI / test (pull_request) Successful in 48s
Second-pass findings on the security fix:

- Mime sanitized ONCE and passed to BOTH StageInputFile and the descriptor (was
  passing raw f.MimeType to the host store while only the descriptor sanitized) —
  3 models.
- sanitizeField now also strips Unicode format chars (category Cf, incl. the bidi
  overrides U+202A–U+202E that can reorder how the descriptor renders); IsControl
  already covers \n\r\t so the explicit checks are dropped.
- fileID is sanitized before inlining + an empty file_id drops the file (defense
  vs a misbehaving stager).
- humanizeBytes clamps the prefix index so an absurd size (≥1024^6) can't index
  past "KMGTPE" and panic — a no-panic guarantee independent of the per-file cap.
- Docs sync: README Ports list gains InputFiles; tool.InputFile.Name doc now says
  the executor reduces an untrusted name to a safe base name (was claiming the
  field is already safe).

Tests: bidi/control stripping; mime sanitized in staged value + descriptor; empty
file_id drop; humanizeBytes no-panic across sizes up to 1<<62. Suite green (-race).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 14:08:57 -04:00
steve 1e65f4b6e5 fix(run): sanitize input-file names — path-traversal + prompt-injection hardening (gadfly #18)
executus CI / test (pull_request) Successful in 48s
The full swarm (5-6 models) flagged that stageInputFiles passed the untrusted
attachment filename straight to StageInputFile and inlined it into the
[ATTACHED FILES]/`/workspace/<name>` descriptor with no sanitization — a path
the byte-cap already treats as a trust boundary. A name like ../../etc/passwd or
an absolute/drive path could escape the host store or the sandbox workspace, and
newlines in the name/mime could inject text into the prompt block.

- sanitizeName: strips control chars/newlines, then reduces to a base name
  (path.Base after backslash-normalization) so ../, nested dirs, and absolute /
  drive paths all collapse to their last element; "attachment" fallback for
  empty/"."/"..". Applied BEFORE staging AND inlining.
- sanitizeField: strips control chars from MimeType (also inlined verbatim).
- maxInputFiles (32) count cap — defense-in-depth vs a flood of tiny files,
  independent of the per-file byte cap.

Tests: sanitizeName table (traversal/absolute/backslash/control/fallback, +
no-separator invariant); traversal staged+described under the base name only;
oversize skip; count-cap truncation. Full suite green (-race).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 13:29:45 -04:00
steve 2ef88f2a73 feat(run): InputFileStager seam — stage non-image attachments into the prompt
Adversarial Review (Gadfly) / review (pull_request) Has been cancelled
executus CI / test (pull_request) Successful in 2m21s
executus's tool.Invocation already carried InputFiles (audio/PDF/binary), but the
executor never staged them — only Images were folded into the run. This adds the
host seam mort's chat/chatbot surfaces need for audio-input parity with agentexec.

- run.Ports gains InputFiles InputFileStager (nil-safe; nil = input files silently
  ignored, run still proceeds text-only). The interface mirrors mort's skill
  FileStorage: StageInputFile(ctx, runID, agentID, name, mime, content) → file_id.
- run/input_files.go (ported from mort agentexec/input_files.go): stageInputFiles
  persists each file under run scope and appends an [ATTACHED FILES] descriptor
  block to the prompt so the agent can reach them by file_id (e.g. code_exec
  files_in → /workspace/<name>). Bytes are NEVER inlined into model context.
  Best-effort: empty/oversized(>50MB)/save-error files are skipped; colliding
  base names are disambiguated (name-2, name-3) so they don't clobber at
  /workspace/<name>.
- Executor.Run calls it after the model/toolbox build, before the loop, so the
  descriptor rides the first user turn (alongside the existing Images folding).

Tests: stages + builds the block; nil stager / no files leave the prompt intact;
dedup; empty/save-error skipping. Full suite green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 13:02:55 -04:00
Steve Dudenhoeffer 7a5eebc468 fix(ci): restore valid adversarial-review.yml + pin gadfly reusable @7bc3c98 [skip ci]
The reusable now reads swarm config from user-scope vars (GADFLY_DEFAULT_* +
GADFLY_ENDPOINT_*); this immutable @sha bumps past the long-lived-runner ref
cache so the vars-config reusable is adopted. Direct to main + [skip ci] to
avoid triggering the review swarm.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 02:05:28 -04:00
steve 7211ce227c ci: pin gadfly reusable workflow to immutable sha (cache-bust @v1)
executus CI / test (push) Successful in 48s
Long-lived act_runners cache the reusable-workflow ref, so a moved @v1 tag
keeps resolving to a stale cached copy and a newly-added reviewer never runs.
Pinning to a unique immutable sha forces a cache miss → fresh fetch.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 05:44:52 +00:00
steve f367796244 Merge pull request 'run: fold inv.Images into the initial user message (multimodal opening turn)' (#16) from feat/initial-images into main
executus CI / test (push) Failing after 18m3s
Adversarial Review (Gadfly) / review (pull_request) Successful in 2s
2026-06-28 05:16:46 +00:00
steve 0acaa8c9a5 run: guard empty text part in runAgent + drop cross-repo doc ref (gadfly #16)
executus CI / test (pull_request) Successful in 1m46s
Every reviewer flagged that runAgent appended llm.Text(input) unconditionally, so
an image-only run (blank prompt) emitted an empty TextPart — inconsistent with the
sibling runSession.AttachImages which guards it. Mirror that guard
(strings.TrimSpace(input) != ""). Also:
- copy opts before appending (variadic backing array can have spare capacity; avoid
  aliasing a caller's slice).
- reword the doc comment to drop the mort-agentexec reference (executus is a
  standalone lib; a consumer name doesn't belong in its godoc).

Tests: image+text are co-located in ONE user message; an image-only run emits no
blank TextPart.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 01:11:15 -04:00
steve a35c176b42 run: fold inv.Images into the initial user message (multimodal opening turn)
executus CI / test (pull_request) Successful in 46s
Adversarial Review (Gadfly) / review (pull_request) Successful in 6m5s
The executor passed only the text `input` to majordomo's agent.Run, silently
dropping inv.Images — so a multimodal run (vision: chatbot @mention, chat API)
lost its images on the executus path. majordomo's Run input arg is text-only, so
fold the images into the first user message (text + image parts) via WithHistory
and call Run with empty input, mirroring mort agentexec's multimodal seeding. The
image-less path is unchanged (prompt passes straight through).

Tests: a run with Images carries the image bytes + prompt into the first model
request; the text-only path still reaches the model.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 00:37:53 -04:00
steve 1cf46c9954 ci: track gadfly's v1 release tag instead of a pinned sha (#15)
executus CI / test (push) Successful in 52s
2026-06-28 04:08:30 +00:00
steve 56baac758d ci: inherit gadfly's default swarm (slim caller, re-pin @b02b11d) (#14)
executus CI / test (push) Successful in 50s
2026-06-28 02:48:25 +00:00
steve 5779035722 Merge pull request 'ci: subscribe to gadfly's reusable review workflow (cloud + Claude Code, no local)' (#13) from ci/gadfly-reusable into main
executus CI / test (push) Successful in 3m56s
2026-06-28 01:43:42 +00:00
Steve Dudenhoeffer 1a2a2364ec security: scope forwarded secrets + pin gadfly reusable to an immutable sha
executus CI / test (pull_request) Successful in 2m13s
Adversarial Review (Gadfly) / review (pull_request) Successful in 10m31s
Address the swarm's findings on this rollout:
- Replace `secrets: inherit` (which forwarded ALL repo secrets — registry/
  Komodo/Discord/DB creds the reviewer never uses) with explicit forwarding of
  only OLLAMA_CLOUD_API_KEY / CLAUDE_CODE_OAUTH_TOKEN / findings tokens.
  GITEA_TOKEN is the automatic job token (github.token in the reusable).
- Pin uses: ...@main -> @20a5c43 (immutable) so a push to gadfly can't change
  the code that runs with our forwarded secrets.

Requires gadfly's review-reusable.yml secrets contract (steve/gadfly#9, merged).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 21:18:59 -04:00
Steve Dudenhoeffer c08ce47fa6 ci: subscribe to gadfly's reusable review workflow (cloud + Claude Code, no local)
executus CI / test (pull_request) Successful in 47s
Adversarial Review (Gadfly) / review (pull_request) Successful in 12m29s
Replace the full self-contained stub with a thin caller of steve/gadfly's
reusable workflow, using gadfly's own dogfood config: 6 cloud models +
the Claude Code engine (sonnet, opus, opus:max). No local Macs / foreman.
Advisory only.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 20:03:00 -04:00
steve 784d5d7ce4 run: PostRun detached ctx + panic-isolated Cleanup (gadfly #12)
executus CI / test (pull_request) Successful in 45s
executus CI / test (push) Successful in 1m47s
Two convergent gadfly refinements on the PostRun wiring:
- PostRun now runs on detach(ctx), not the caller's ctx — a finished/cancelled
  caller no longer aborts artifact production (3-model: glm-5.2/minimax/deepseek).
- Cleanup is panic-isolated via safeCleanup (recover+log), matching runPostRun, so
  a misbehaving teardown can't clobber an otherwise-successful run (deepseek).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 18:33:41 -04:00
steve 4e179259de run: wire SessionToolFactory + PostRun artifacts + AttachImages
executus CI / test (pull_request) Successful in 1m49s
Adversarial Review (Gadfly) / review (pull_request) Successful in 5m19s
The session-tool TYPES already lived in tool/ (P4 move) but the executor never
used them. This wires them, unblocking artifact-producing host surfaces (mort's
chat API / chatbot / .skill / scaddy) to run on executus:

- run/session.go: steerMailbox (thread-safe message queue) + runSession
  (tool.AgentSession over it: AttachImages → a user-role multimodal message
  injected before the agent's next step) + runPostRun (panic-isolated hook call).
- executor: create the mailbox + set inv.AttachImages BEFORE the toolbox build;
  add inv.ExtraTools + a SessionToolFactory's per-run Tools to the toolbox; defer
  its Cleanup; merge the session mailbox with the critic's nudges into ONE
  WithSteer; after the run, call PostRun with the full transcript
  (runRes.Messages) → Result.PostRunResult (best-effort, never fails the run).
- run.Result += PostRunResult *tool.PostRunResult.
- dropped the now-dead criticBinding.steerOptions (superseded by drainSteer).

Tests: a factory whose PostRun emits an artifact from the output+transcript +
Cleanup lands on Result.PostRunResult; a factory-added tool is callable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 18:13:16 -04:00
steve 82a816ae29 ci(gadfly): trim pool to the strong 6 (drop m5/qwen3.6, gemma4, gpt-oss, kimi-k2.7)
executus CI / test (push) Successful in 46s
Pool now: minimax-m3, glm-5.2, glm-5.1, deepseek-v4-pro, nemotron-3-super,
qwen3-coder:480b (all cloud, ollama-cloud=3). Removed the low-value reviewers +
the last local endpoint (m5).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 18:06:36 -04:00
steve be4bbbcad5 run: fix statusFor — don't relabel a generic error / caller-cancel as timeout (gadfly #11)
executus CI / test (pull_request) Successful in 47s
executus CI / test (push) Successful in 45s
The WithCancelCause+timer rewrite made MaxRuntime surface as Canceled (not
DeadlineExceeded), so statusFor's context.Cause(DeadlineExceeded) check could
relabel (a) a genuine run error as 'timeout' and (b) a caller cancel/deadline as
'timeout' (was 'cancelled'). Convergent gadfly finding (glm-5.2 + cluster).

Fix: keep MaxRuntime as WithTimeout (its DeadlineExceeded propagates → 'timeout',
preserving own-timeout vs caller-cancel), add a NESTED WithCancelCause layer only
for the kill. statusFor consults context.Cause ONLY for ErrCriticKill; everything
else is classified by the run error itself. Tests: generic-error-not-relabeled +
caller-cancel-stays-cancelled.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 17:00:26 -04:00
steve 390e6cf905 run: critic parity — fuller RecordStep + cause-carrying Kill (distinct status)
executus CI / test (pull_request) Successful in 46s
Adversarial Review (Gadfly) / review (pull_request) Successful in 22m30s
Completes the run-critic seam so a host adapter (mort's agentcritic) has full
fidelity, closing the two limitations gadfly surfaced on mort #1334.

- RecordStep(iter int, resp *llm.Response): the completed step's model response
  is now passed to the critic (was index-only), so a host that records a trace
  (mort's ProgressRecorder) can show what the agent actually produced, not just
  an iteration count. The executor forwards s.Response; the battery ignores it
  (its Progress is count-based).
- CriticHandle.KillCause() error + ErrCriticKill: the executor now distinguishes
  an explicit critic KILL from a natural backstop expiry. runCtx uses a
  cause-carrying cancel (WithCancelCause + a MaxRuntime timer cancelling with
  DeadlineExceeded); the deadline-watch cancels with ErrCriticKill when
  KillCause()!=nil, else DeadlineExceeded. statusFor reads context.Cause →
  killed / timeout / cancelled are now distinct (were all "cancelled"). The
  battery sets killCause from Decision.KillReason on a Kill.

Tests: statusFor "killed" case (cause=ErrCriticKill, err=Canceled); fake handle
+ battery RecordStep/KillCause signatures. Core stays battery-free.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 16:35:13 -04:00
steve 1a1d5e417b chore: go mod tidy (add missing go.sum entry; CI tidiness gate)
executus CI / test (pull_request) Successful in 2m8s
executus CI / test (push) Successful in 1m45s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 14:53:58 -04:00
steve f3bd43b726 ci(gadfly): drop the m1 reviewer (dead weight; keep m5)
executus CI / test (pull_request) Failing after 1m1s
m1/qwen3:14b proved consistently low-value + slowest in the pool over multiple
PRs. Removed from GADFLY_MODELS + GADFLY_PROVIDER_CONCURRENCY + its endpoint so it
never fires again. m5 retained.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 14:41:14 -04:00
steve 306d575c31 critic: overflow-guard maxSteps += RaiseStepsBy (gadfly 5-model convergence)
executus CI / test (pull_request) Has been cancelled
A buggy/hostile Escalator returning a huge RaiseStepsBy could wrap handle.maxSteps
negative (which the executor reads as defer-to-base). Clamp at math.MaxInt.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 14:38:48 -04:00
steve 4ba83ab905 run: critic can raise a run's step ceiling mid-flight (CriticHandle.MaxSteps)
executus CI / test (pull_request) Failing after 1m1s
Adversarial Review (Gadfly) / review (pull_request) Successful in 21m8s
Prerequisite for a full-fidelity mort agentcritic adapter (which adjusts a
healthy-but-long run's iteration budget, not just its deadline). executus's
CriticHandle was deadline+steer only; this adds the dynamic step ceiling above
an unchanged majordomo (which already exposes WithMaxStepsFunc).

- run.RunInfo += MaxIterations (the run's base ceiling, so a critic can raise it
  relative to the baseline).
- run.CriticHandle += MaxSteps() int — polled by the executor each step via
  agent.WithMaxStepsFunc; <=0 defers to the base. The executor uses
  WithMaxStepsFunc(critic.MaxSteps) when a critic is active, else WithMaxSteps.
- critic battery: handle.maxSteps (initialised from RunInfo.MaxIterations) +
  MaxSteps(); Decision gains RaiseStepsBy so an Escalator can raise the ceiling
  alongside ExtendBy. ExtendOnce default is unchanged (time-only).

Test: a critic returning MaxSteps=5 lets a base-MaxIterations=1 run complete two
tool-dispatch steps past the base ceiling. Core stays battery-free (run doesn't
import critic).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 14:16:03 -04:00
steve a103cc5e9f ci(gadfly): 9-cloud panel @ 3 models x 3 lenses (9 concurrent)
executus CI / test (push) Failing after 1m57s
Match mort: minimax-m3, glm-5.2, glm-5.1 (SWE-Bench Pro SOTA), kimi-k2.7-code,
deepseek-v4-pro, nemotron-3-super, gpt-oss:120b, qwen3-coder:480b, gemma4 (8
families) + m1/m5 locals. ollama-cloud=3 x lens=3 = 9 concurrent (10 budget).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 12:17:24 -04:00
steve 4d28cd6e2c ci(gadfly): 4-cloud pool — add kimi-k2.7-code + deepseek-v4-pro, drop v4-flash
executus CI / test (push) Failing after 1m2s
Match mort's new cloud panel: minimax-m3, glm-5.2, kimi-k2.7-code (Moonshot),
deepseek-v4-pro (frontier, replaces v4-flash). Keeps m1/m5 locals + the existing
ollama-cloud=1 + lens-concurrency=3 serial-model style.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 11:59:13 -04:00
steve dcaefff756 ci(gadfly): add M1/M5 Macs back to the reviewer pool (full fleet)
executus CI / test (push) Failing after 1m23s
Re-adds the local Macs (m1/qwen3:14b, m5/qwen3.6:35b-mlx) via their foreman endpoints alongside the 3 cloud models. Cloud keeps lens fan-out (ollama-cloud=1 model + lens=3); each Mac runs one model with lenses serial (foreman serializes anyway); all provider lanes parallel. Bumps the job timeout 30->90m for the slow local lanes. With findings telemetry now on, gadfly-reports can quantify whether the Macs earn their keep.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 10:44:22 -04:00
steve 97154395e6 C0b: document recordToolStart post-iteration timing (gadfly glm finding)
executus CI / test (pull_request) Failing after 59s
executus CI / test (push) Failing after 1m1s
majordomo's step observer fires post-iteration, so the critic's activity clock
refreshes per-iteration, not mid-tool — a single long tool call won't refresh it
until it returns. Documented + the host-progress-bridge mitigation (mort's
pattern). A true pre-dispatch hook needs majordomo support (follow-up).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 10:10:56 -04:00
steve 4aa06f652e C0b: address verified gadfly findings (panic-safety + test honesty)
executus CI / test (pull_request) Failing after 58s
From PR #9 (minimax + deepseek):
- Run now has a top-level recover() — the "never propagates a panic" promise was
  unenforced; a panicking host Port (Critic/Audit/Palette) on the run goroutine
  now becomes Result.Err instead of unwinding into the caller.
- The critic deadline-watch goroutine recovers panics from a host Deadline()
  (it's a separate goroutine, so Run's recover can't catch it) — a buggy
  CriticHandle can't crash the process.
- CriticHandle interface documents its concurrency contract (Record*/Steer on the
  run goroutine vs Deadline()/Stop() from the watch goroutine — impls must be
  concurrent-safe; the critic battery already is).
- startCritic's dead `soft <= 0 -> noop` guard (withFallbacks already coerces to
  90s) replaced with a defensive inline 90s default, so a bypass of withFallbacks
  still gets a working critic instead of silently none.
- Delivery tests made honest: the old "error path" test only checked the
  early-return (no delivery); added TestDeliverErrorOnRunFailure (in-loop model
  error -> DeliverError to the target) + renamed the early-return test.

Graded all #9 findings in the gadfly MCP.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 10:09:22 -04:00
steve 43b2471737 C0b: wire Critic + Delivery into run.Executor
executus CI / test (pull_request) Failing after 1m0s
Adversarial Review (Gadfly) / review (pull_request) Successful in 5m9s
Continues finishing the executor's run.Ports wiring (after C0's Palette).

Critic (run/critic.go): when Ports.Critic is set and the agent enables it, the
executor calls Monitor at run start, feeds RecordStep/RecordToolStart from the
step observer, drains the critic's Steer messages into the loop via
agent.WithSteer, and binds the run's hard cancellation to the critic's
(extendable) Deadline through a watch goroutine — a healthy-but-slow run gets
room while a hung one is killed. Stop() on run end. Soft timeout from
Defaults.CriticSoftTimeout (default 90s). nil-safe: no critic / not-enabled =
no-op.

Delivery (run/executor.go deliver): after the run, when Ports.Delivery is set
and inv.DeliveryID is non-empty, the executor posts Result.Output (or
DeliverError on failure) to a host-interpreted deliver.Target
{inv.DeliveryKind, inv.DeliveryID}. Empty target = caller reads Result.Output
itself (the synchronous default; the `.agent run` canary). Best-effort +
detached.

tool.Invocation gains DeliveryKind/DeliveryID (host-set egress target).

Tests: critic monitored/fed/steered/stopped when enabled, untouched when not;
delivery posts on a target, skips without one. Deferred: Checkpointer (needs a
majordomo hook to snapshot the running message history).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 10:00:05 -04:00
steve 0c80679719 C0: address verified gadfly findings (trivial fixes)
executus CI / test (pull_request) Failing after 1m31s
executus CI / test (push) Failing after 1m31s
From the PR #8 review (all graded in the gadfly MCP):
- skip empty palette names + dedupe by final tool name, instead of producing a
  "skill__" tool or an opaque box.Add duplicate error.
- delegationResult: no trailing blank line when a non-ok child produced no output.
- delegationErr: fold a child's partial output into the hard-failure error so it
  isn't silently dropped.

Deferred to C0b (design-level, not trivial): route delegation through the
tool.Registry gate/audit wrappers; expose the skill's real input schema to the
LLM instead of a generic inputs map. typed-nil PaletteSource is left as a caller
contract (the == nil guard catches the untyped-nil interface).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 09:53:11 -04:00
steve 9d41987b0e C0: wire Palette delegation into run.Executor (skill__/agent__ tools)
executus CI / test (pull_request) Failing after 1m2s
Adversarial Review (Gadfly) / review (pull_request) Successful in 3m47s
The first cutover prerequisite: the executor now turns an agent's SkillPalette /
SubAgentPalette into delegation tools so a mort agent that delegates works
through run.Executor (the piece the `.agent run` canary needs beyond the
already-wired audit/budget).

- run/palette.go: addDelegationTools builds a skill__<name> tool (structured
  inputs) per SkillPalette entry and an agent__<name> tool (prompt) per
  SubAgentPalette entry, each invoking run.Ports.Palette as a CHILD of the
  current run (parentRunID = inv.RunID, inheriting caller + channel). A non-ok
  child status is surfaced to the parent with the partial output. nil-safe: no
  PaletteSource or empty palette → no delegation tools (unchanged behavior).
- executor.go: call it right after building the low-level toolbox.

Tests: the model calls skill__helper → routed through Palette with the right
name/caller/inputs/parent; nil palette → run still works.

Deferred to C0b (the remaining run.Ports executor wiring): Critic (soft-timeout
monitor + deadline binding + steer), Delivery (output egress for surfaces that
need executor-side delivery), Checkpointer (needs a majordomo message-history
hook to snapshot resumable state). The `.agent run` canary delivers its returned
Result.Output itself, so these aren't on its critical path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 09:28:01 -04:00
steve e37cf415de ci(gadfly): emit findings to gadfly-reports + bump image to sha-d7f364d
executus CI / test (push) Failing after 2m40s
Adds GADFLY_FINDINGS_URL / GADFLY_FINDINGS_TOKEN (user-scope secrets) so each review POSTs its run + findings to the gadfly-reports store, and bumps the pinned gadfly image to sha-d7f364d (the build carrying the findings-emit). Advisory only — emit failures never affect the review.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 09:12:46 -04:00
steve a87e7d2c72 fix: address verified gadfly P5 findings (canary robustness)
executus CI / test (pull_request) Failing after 3s
executus CI / test (push) Failing after 1m9s
All 3 cloud models converged (all "minor" — example code, no blocking):
- Consolidate: a model whose every lens errored now reads "review incomplete",
  not a misleading "no issues found" (all 3 models). + test.
- Consolidate: swarm-cancelled (unattributed) cells now surface a "swarm
  cancelled — N cell(s) did not run" banner instead of vanishing (all 3). + test.
- main: io.ReadAll(os.Stdin) error is surfaced (all 3); a TTY stdin no longer
  hangs forever (TTY guard, minimax).
- providerOf: a bare tier name now keys its own PerKey bucket instead of all
  bare tiers collapsing onto "tier" (minimax, glm-5.2) — distinct tiers throttle
  independently.
- Review doc reworded (the closure, not fanout, carries per-cell errors).

Left as documented example-scope behavior: no per-cell timeout (caller supplies
ctx), unknown-severity → lowest rank (no crash).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 00:34:01 -04:00
steve ea9475da54 P5: light-tier canary — gadfly-shaped reviewer on executus core
executus CI / test (pull_request) Failing after 1m5s
Adversarial Review (Gadfly) / review (pull_request) Successful in 8m18s
examples/reviewer proves the core is sufficient for a static-binary light host
(gadfly's shape) with NO batteries:
- config.Env + model.Configure  -> env-driven model fleet + tier overrides
- model.ParseModelForContext    -> tier resolution + failover
- fanout.Run (PerKey caps)      -> N models x M lenses swarm, per-provider bound
- model.GenerateWith[T]         -> structured findings per (model, lens) cell
- Consolidate                   -> one verdict-led report section per model

Hermetic test runs the full 2x3 swarm against majordomo's fake provider and
asserts the consolidated verdicts. A go list -deps CI check asserts the canary
imports ZERO batteries (the light-tier invariant) — gadfly's go.sum stays free
of gorm/redis/discordgo/sqlite. README + docs updated.

This is the canary; migrating the LIVE gadfly repo onto executus core is a
follow-up (kept separate to not destabilize the active reviewer).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 00:22:02 -04:00
steve dc2d4ec425 P4c: remaining batteries — checkpoint + schedule + critic
executus CI / test (push) Failing after 1m6s
Completes the P4 battery set (squashed onto main from phase-4c-batteries).
- checkpoint/: run.Checkpointer durable-resume (CheckpointStore + throttled
  handle + Memory).
- schedule/: generic cron Runner (Tick/Loop; no cron grammar of its own).
- critic/: two-tier timeout watchdog (run.Critic) + Escalator policy seam +
  ExtendOnce default.
Includes the verified gadfly #6 fixes (ExtendOnce per-run, Kill-sticky, watch
panic-recovery; checkpoint throttle-after-success; schedule Next-before-Run +
nil-guard + Loop recovery).

P4 battery set complete: audit, budget, persona, skill, checkpoint, schedule,
critic — each nil-safe, each with a default, each core-import-clean. Executor
wiring for Critic/Checkpointer remains a P2 follow-up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 00:15:32 -04:00
steve c8559676ed P4b: skill noun + contrib/store (SQLite for budget/persona/skill/audit)
executus CI / test (push) Has been cancelled
Merges the skill half of the persona/skill pair plus the second nested module.
(Squashed onto main from phase-4b-skill; the audit/budget/persona batteries it
was stacked on already landed via the P4 merge.)

- skill/: clean-redesign Skill noun + LEAN SkillStore (lifecycle/versions/
  schedule only) + ToRunnable + Memory default.
- contrib/store/: separate go.mod carrying modernc.org/sqlite, so the driver
  never enters the core go.sum. db.Budget()/Personas()/Skills()/Audit() back
  all four store seams (JSON-blob + indexed columns; round-trip tested).
  Includes the verified gadfly #5 fixes (AppendVersion tx+UNIQUE+error,
  Mark*ScheduledRun atomic json_set, busy_timeout, NaN guard).
- CI: builds + tests the nested module and asserts it owns the sqlite driver.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 00:15:00 -04:00
steve d82cef46b4 fix: address verified gadfly P4/#4 findings (audit/budget/persona)
executus CI / test (push) Failing after 1m4s
Security (all 3 models — HIGH): audit OnTool persisted raw tool args + results
verbatim for the very tools the OnStep narration-redaction flags as secret
(mcp_call/email_send/http_*) — the args/results are what CARRY the secret, so
they landed in skill_run_logs unredacted. Factored the predicate into
isSecretTool() (single source of truth) and OnTool now emits
args_redacted/result_redacted (+ lengths) for secret tools. Test asserts no
secret reaches the log. (persona) webhook_ip_allowlist entries are now
CIDR/IP-validated at load (malformed dropped + warned) instead of accepted raw.

Contract correctness (glm-5.2 + deepseek) — audit Memory now honors its
documented Storage contract: ListChildrenByParent/ListFinishedRunsBefore return
oldest-first; WalkParentChain returns root-first and honors MaxParentChainDepth;
ListRunsFiltered clamps limit (<=0 or >500 -> 50); ListFinishedRunsBefore with
limit<=0 returns none; an explicit RunFilter.Status (incl. "dry_run") matches
regardless of IncludeDryRun; LastRunBySkills counts only status=="ok" unless
includeFailed. (PurgeOlderThan's FinishedAt key is the SAFE behavior — in-flight
runs retained — so the doc was aligned to it, not the impl.)

Error-handling: appendLog now uses a bounded context (auditAppendTimeout=3s) so
a hung backend can't block the run goroutine on the hot path; Sink.StartRun
logs its (still best-effort) failure instead of swallowing it; budget Memory.Get
uses RLock (RWMutex); budget package doc fixed (was skillexec's); Check uses the
budgetWindow constant, not a duplicated literal.

Triaged false-positive: NewNoOpBudget returning BudgetTracker is assignable to
run.Budget (identical method sets) — no change needed.

Core go.sum still free of host/DB deps.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 00:12:19 -04:00
steve 2260480c81 P4: persona noun — Agent + ToRunnable bridge + Memory store
The headline P4 piece (clean redesign): the Agent persona noun, decoupled from
its Discord shell.
- agent.go/storage.go/builtin_loader.go moved from mort's pkg/logic/agents; the
  Storage seam drops the Discord CommandBindingStorage embedding (a host
  concern). The host-entangled files (commands, chatbot_provider, command-
  binding dispatcher, personalization, system) stay in mort.
- runnable.go: Agent.ToRunnable() lowers a persona into run.RunnableAgent — the
  bridge that lets run.Executor run a persona without importing this battery
  (the inversion of agentexec.Run(*agents.Agent)).
- memory.go: NewMemory() — zero-dep in-process persona Storage (all 11 CRUD +
  trigger-query methods).

Tests: ToRunnable field/phase mapping; Memory round-trip. CI invariant: core
imports ZERO from persona.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 00:12:19 -04:00
steve 9116abcae2 P4: budget battery — DBBudget (rolling 7-day) over run.Budget
Second Tier-2 battery, plugging into run.Ports.Budget:
- budget.go: skillexec's BudgetTracker / NoOpBudget / DBBudget moved clean
  (stdlib only). Check/Commit match run.Budget exactly (compile-time proof in
  run.go: NoOpBudget and *DBBudget are run.Budget).
- storage.go: the BudgetStorage seam + SkillBudget domain, split out of mort's
  GORM file (the GORM impl stays in mort).
- memory.go: NewMemory() — zero-dependency in-process BudgetStorage with the
  7-day rolling-window rollover in Add.

Tests: per-user cap enforced, window rolls over after 7 days, NoOp always
allows. CI invariant: core imports ZERO from the budget battery.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 00:12:19 -04:00
steve 4d2f85d139 P4: audit battery — run.Audit Sink + Writer + queryable Memory store
First Tier-2 battery, plugging into run.Ports.Audit:
- storage.go/writer.go: skillaudit's Storage interface + per-run Writer moved
  clean (only utils->fmt); the Writer already matches run.RunRecorder's shape.
- sink.go: Sink adapts a Storage to run.Audit (StartRun -> a run row + a Writer
  wrapped as run.RunRecorder, converting run.RunStats on Close). NewSink(nil) is
  equivalent to no audit. Compile-time proofs: Sink is run.Audit, recorder is
  run.RunRecorder.
- memory.go: NewMemory() — a zero-dependency, queryable in-process Storage
  (retains runs + logs; all 17 read/filter/purge/walk methods) so a light host
  gets run history with no setup. Mort keeps its GORM Storage; contrib/store
  adds durable SQLite at P4.

End-to-end test: wire audit.NewSink(audit.NewMemory()) into the executor, run an
agent, and the run is recorded with terminal status/output and queryable by
caller. CI invariant verified: core imports ZERO from the audit battery (proper
battery direction; battery imports core, never the reverse).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 00:12:19 -04:00
steve d0bd3ec3d9 fix: address verified gadfly P3 review (3-cloud fleet)
executus CI / test (push) Has been cancelled
All 3 cloud models converged on a real access-control bug; fixed it + the
other genuine findings (the false-positives were dropped):

Security (HIGH — all 3 models):
- create_file_url skipped ValidateScope: a same-skill caller could mint a
  PUBLIC url for a file scoped to another user/run. Now runs ValidateScope
  (admin-aware), skipped only for the descendant-grant case — mirroring the
  read tools.

Other real fixes:
- ValidateScope hard-coded `false` at every call site (admin branch dead) ->
  pass inv.CallerIsAdmin (the executor sets it via the host AdminPolicy; still
  false/fail-closed when no admin). Stale "no admin flag" comment corrected.
- create_file_url: ExpiresInSeconds clamped BEFORE the *time.Second multiply
  (huge values overflowed to a negative duration that slipped under the cap,
  minting already-expired tokens); swallowed json.Marshal error now returned.
- RegisterMeta: build the default budget WITH the configured MaxPerRun (was
  NewInMemorySearchBudget(nil) -> hardcoded 10, ignoring MetaDeps.MaxPerRun).
- classify: all-zero scores no longer return a false-positive top-1 winner;
  coerceClassifyScore uses strconv.ParseFloat (rejects trailing garbage like
  "50extra" that fmt.Sscanf silently accepted).
- file_delete: honor the descendant grant (parent can clean up a worker's
  artifacts) — was the lone cross-skill-reject-outright file tool.
- meta tools: input caps truncate at a UTF-8 rune boundary (truncateUTF8), not
  mid-rune.
- think: removed the dead `var _ = fmt.Errorf` import-keeper; file_save default
  aligned to 16 MiB (matched RegisterStore).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 00:11:54 -04:00
steve 78e6858751 P3: store group — kv_* + file_* tools (agent memory)
RegisterStore(reg, StoreDeps) registers the persistent-memory tools over the
host's KV and/or File backends:
- kv_get/set/list/delete (KVStorage seam)
- file_save/get/get_text/get_metadata/list/delete (FileStorage seam), plus
  file_search (FileSearcher) and create_file_url (FileTokenMinter) when wired.

Near-zero-config: Quota defaults to a generous static cap (staticQuota), the
per-value/per-file caps default, and the kv vs file groups register
independently (a host can take just one). Seams moved clean (interface-only):
kv_storage.go, quota_provider.go, file_descendant_grant.go. The default
in-memory KV/File backends come with contrib/store at P4.

Core go.sum still free of gorm/redis/discordgo/sqlite.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 00:11:54 -04:00
steve 1e201550b3 P3: meta + primitive tool group (think/now/cite + classify/extract/summarize)
Grow executus/tools into a real generic tool library:

- Register(reg): the always-available, zero-config tools — think, now (UTC
  unless a CurrentTimeProvider is wired), cite (inert unless a CitationStorage
  is wired). All nil-safe; a light host calls Register and is useful.
- RegisterMeta(reg, MetaDeps): the LLM-backed meta tools — classify,
  extract_entities, summarize — over the llmmeta helper. Budget defaults to the
  shipped in-memory per-run cap; Files optional; caps default.
- Seams moved (interface/type-only, no host coupling): research_providers.go
  (CurrentTimeProvider/CitationStorage/SearchBudget/PageExtractor/PDFFetcher/…)
  and file_storage.go (FileStorage + FileDomainMeta). Plus the in-memory budget
  default (research_defaults.go) and scope_validate.go.

calculate deferred (drags github.com/Krognol/go-wolfram + a module-path replace
— not worth it in the lean core for one tool). Core go.sum still free of
gorm/redis/discordgo/sqlite/wolfram.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 00:11:54 -04:00
steve df95425bb5 P3 (kickoff): generic tools/ library + end-to-end tool-using-agent test
Stand up executus/tools — the generic, host-agnostic tool library — and prove
the full pattern end to end:

- tools/tools.go: Register(reg) adds the always-available zero-dependency tools
  (currently `think`). A light host calls it and is immediately useful; backed
  tools (web/store/meta groups) will register via grouped registrars with
  nil-safe Deps as they land.
- tools/think.go: the `think` tool moved from mort (imports only executus/tool).
- tools/integration_test.go: end-to-end proof that the executor runs an agent
  which CALLS a registered tool — the fake model emits a `think` tool call, the
  executor dispatches it through the registry, the model finalises, and the step
  instrumentation captures the `think` step. Exercises the full tool-dispatch
  loop through run.Executor.

Stacked on phase-2-run-kernel (P3 needs run.Executor). Remaining P3: the
meta/web/net/store/compose groups + their Deps + default backends (splitting
mort's default.go grab-bag).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 00:11:54 -04:00
steve 16ddd90914 ci(gadfly): new build sha-d0de034 + per-lens concurrency
executus CI / test (push) Successful in 59s
Bump the gadfly image to sha-d0de034 (adds GADFLY_PROVIDER_LENS_CONCURRENCY)
and move ollama-cloud's concurrency from the MODEL axis to the LENS axis:
- GADFLY_PROVIDER_CONCURRENCY: ollama-cloud=1 (one model at a time)
- GADFLY_PROVIDER_LENS_CONCURRENCY: ollama-cloud=3 (its 3 lenses concurrent)

Net: still 3 models, but reviewed serially — the first model's consolidated
comment lands sooner and each model finishes faster, while the other two
models' comments arrive in series after it (instead of all 3 in parallel).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 22:57:14 -04:00
steve b35514dfaa ci(gadfly): cloud-only fleet (3 models, drop local Macs)
executus CI / test (push) Successful in 57s
Measured on the P2 review: the local Macs (m1/m5) took 26–29 min with lens
timeouts and found ZERO real bugs, while the two cloud models found every
genuine finding in 6–12 min. Drop the Macs; add glm-5.2:cloud as a third
cloud reviewer. Net: faster (~29→~12 min) and higher signal.

Models: minimax-m3:cloud, deepseek-v4-flash:cloud, glm-5.2:cloud
(ollama-cloud=3 concurrency). timeout-minutes 90→30.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 02:02:21 +00:00
steve 7b3da87c08 fix: address verified gadfly P2 findings (9 real of 18)
Independently verified all 18 gadfly findings against the code (18-agent
fan-out). Fixed the 9 real ones; the other 9 were false-positive /
hallucinated / valid-tradeoff (no change).

High:
- F1 nil model: a Models resolver returning (ctx,nil,nil) flowed into the
  agent loop and nil-panicked. Now a clean error (Run never panics). +test.
- F9 compactor data-leak: renderTranscript sent tool-call args verbatim to
  the summarizer (a possibly-different provider/tier); secret-bearing tool
  args (mcp_call/email_send/http_*/webhook_*) are now redacted, with a doc
  note that result bodies still flow (summary needs them).

Medium/minor:
- F2 compactor error path returned the folded slice, not the original msgs
  (contradicting the documented non-fatal contract) -> return msgs.
- F3 RunStats.Status only ok/error; now timeout (DeadlineExceeded) /
  cancelled (Canceled) via statusFor. +test.
- F4 step-zip emitted empty-name "ghost" steps when results>calls; now pairs
  min(calls,results) only.
- F5 SetIteration was never called -> RunState.Iteration always 0; the step
  observer now updates it each loop.
- F6 matchPending fallback was LIFO; now FIFO (matches the per-key queue).
- F7 estimateTokens had no default arm (future Part kinds counted as 0);
  unknown parts now counted conservatively.
- F8 cloud_sync silently truncated >1MiB responses -> opaque JSON error; now
  a clear "response exceeded N bytes" via readCapped.
- F12 step observer captured the caller ctx; now the merged runCtx.
- F13 compaction onFire was nil (doc claimed it logged); now wired to
  audit LogEvent("compaction_fired").
- F11 (no pre-dispatch hook in majordomo) documented honestly as a known
  limitation; F18 UsageSink doc clarified cache tokens are subsets of input.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 02:02:21 +00:00
steve dfbc5a42b9 P2: run.Executor — executus is runnable
The capstone of the run kernel: run.Executor.Run(ctx, RunnableAgent, inv)
ties model resolution + the tool registry + majordomo's agent loop +
context compaction + run-bounding + step/audit instrumentation into one
path, with every host concern behind the nil-safe run.Ports.

- run/executor.go: New(Config{Registry, Models, Defaults, Ports, Compactor,
  ContextTokens, SystemHeader}) + Run -> Result{RunID, Output, Steps, Usage,
  Err}. Budget gate (pre-run), model resolve, Audit StartRun/recorder
  (satisfies RunTally, stamped on inv.RunState), toolbox build, step observer
  (zips tool calls/results -> emitter + recorder.OnStep/OnTool), V10
  detached-MaxRuntime context with caller-cancel merged back, compaction wired
  from ContextTokens×ratio, audit Close + Budget Commit on a detached cleanup
  ctx. Zero Ports = a bounded in-memory run (gadfly's case).
- run/executor_test.go: hermetic end-to-end run against majordomo's fake
  provider (hello-world), Budget-rejection (no model call), Audit-port wiring
  (StartRun + Close with terminal status/output). All green under -race.
- examples/minimal upgraded to the real "hello, agentic world" (~15 lines:
  Configure tiers -> run.New -> Run -> print). README/CLAUDE.md updated.

Remaining P2 follow-ups (incremental): wire Critic/Checkpointer/PaletteSource/
Delivery into the loop, multi-phase Pipelines, and the no-tools direct path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 02:02:21 +00:00
steve 130c2bdfab P2: move compactor -> compact/ + step instrumentation -> run/steps.go
- compact/compactor.go: the per-run stateful context compactor (token-threshold
  gate, fast-tier middle summarisation, fold memory) lifted from mort's
  skillexec/compactor.go. Self-contained; its only dependency is a ModelResolver
  func (model.ParseModelForContext satisfies it) + a token threshold.
- run/steps.go: the step-emission/instrumentation (stepEmitter, tool->kind/
  summary mapping with redaction, Result.Steps accumulation) from agentexec,
  repointed onto executus/tool.

Both build green. With the run-loop mechanics, RunnableAgent DTO, run.Ports,
compactor, and step instrumentation now all in place, the remaining P2 work is
the run.Executor itself (wiring these + majordomo's agent loop), which makes
executus runnable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 02:02:21 +00:00
steve d9b44387f5 fix: address gadfly P1 review (3 low-risk findings)
Triaged gadfly's P1 review (advisory). Fixed the three clearly-correct,
low-risk items; the rest were pre-existing mort behavior or theoretical:

- model/call.go: recordUsage dropped fully-cached responses (input==0 &&
  output==0 early-return missed CacheRead/CacheWrite-only usage, which
  Anthropic/OpenAI prompt-caching bills). Guard now also checks cache tokens.
- llmmeta/helper.go: recordLedger swallowed Storage.RecordMetaCall errors;
  now logs them (slog.Warn) so a non-logging Storage impl can't silently drop
  audit rows.
- model/cloud_sync.go: the ollama.com limit-cache used unbounded io.ReadAll;
  wrapped both reads in io.LimitReader(1 MiB) so a misbehaving endpoint can't
  exhaust memory before the 15s timeout.

Noted-not-fixed (follow-ups / pre-existing mort semantics): tier_not_allowed
ledger label on resolution failure, unknown-model usage attribution, the
cloud_sync https scheme allowlist, and several theoretical/cosmetic items.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 02:02:21 +00:00
steve e0e2c0451a ci: sync gadfly review config to mort's foreman-provider setup
Mirror mort's updated adversarial-review.yml: m1/m5 pulled in via the
GADFLY_ENDPOINT_M1/_M5 secrets using gadfly's "foreman" provider type
(providers m1/m5; models m1/qwen3:14b, m5/qwen3.6:35b-mlx), 2 cloud models,
3-lens suite, pinned to the gadfly :sha-6e3a83c image. Header adjusted for
executus; functional config identical to mort's tested version.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 02:02:21 +00:00
steve c732a677df P2: define nil-safe run.Ports (the inversion spine)
Add run/ports.go: the host seams the executor will consume, every one
nil-safe so a light host runs with the zero Ports (no persistence/audit/
budget/critic/delegation/delivery) and a heavy host wires each to a battery.

Ports mirror mort's existing interfaces so the batteries implement them
directly:
- Audit + RunRecorder (mort skillaudit.Storage/Writer): StartRun -> per-run
  recorder (OnStep/OnTool/LogEvent/Close), recorder satisfies RunTally.
- Budget (mort skillexec.BudgetTracker): Check / Commit.
- Critic + CriticHandle (mort agentcritic): Monitor -> handle with
  RecordStep/RecordToolStart/Steer/Deadline/Stop (the loop wiring finalizes
  with the executor merge).
- Checkpointer (mort agentexec.RunCheckpointer): Save/Complete/Fail.
- PaletteSource (mort SkillInvokerForPalette + AgentInvokerForPalette):
  Resolve/Invoke skill + agent delegation.
Plus host-neutral RunInfo / RunStats.

This completes the P2 inversion DESIGN; the agentexec+skillexec ->
run.Executor merge that consumes these Ports is the remaining P2 work.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 02:02:21 +00:00
steve fa644f1826 P2 (foundation): run-loop mechanics + RunnableAgent DTO
Stand up the executus/run kernel foundation, decoupled from mort:

- runengine.go: the shared run-loop scaffolding (MergeCancellation,
  CleanupContextTimeout, RunFinalizer/FireFinalizers, RunStateAccessor) moved
  from mort. The accessor's *skillaudit.Writer dependency is inverted to a
  narrow run.RunTally interface (TokenStats + ToolCallsCount) — the kernel
  reads live tallies without importing the audit battery.
- submit.go: the legacy submit-capture compat tool (stdlib + majordomo/llm).
- agent.go: RunnableAgent DTO — the kernel's view of "a thing to run" (tier,
  prompt, caps, palette, phases, critic config). The persona Agent and saved
  Skill will LOWER into this DTO so the kernel never imports a noun battery.
  This is the spine of the agentexec.Run(*agents.Agent) inversion.

run/ builds with only majordomo + executus/tool. The executor merge
(agentexec+skillexec -> run.Executor) and the nil-safe run.Ports
(Audit/Critic/Budget/Checkpointer/PaletteSource) are the next P2 block.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 02:02:21 +00:00
steve b424261aca P1: model layer (convar->config inversion) + llmmeta
executus CI / test (pull_request) Successful in 58s
Adversarial Review (Gadfly) / review (pull_request) Successful in 26m27s
executus CI / test (push) Successful in 1m2s
Lifts mort's pkg/logic/llms into executus/model, decoupled from mort:

- tiers.go: the tier resolver now reads a host-supplied config.Source under
  "model.tier.<name>" with host-supplied fallbacks (Configure(cfg, defaults,
  ttl)), instead of convar.Manager. Tier NAMES + specs are host config; the
  resolution mechanism (cache, reasoning-suffix dialect, chain validation) is
  generic. No tier names hard-coded in the harness.
- sink.go: usage/trace recording inverted off mort's llmusage/llmtrace into
  UsageSink / TraceSink seams + a model-owned Span, with nil-safe context
  attribution helpers (WithModel/WithTraceID/WithUsageTool/WithUsageUser).
  Both sinks optional (nil = off) so a light host records nothing.
- lane decoration repointed to executus/lane; utils.Errorf -> fmt.Errorf.
- call.go keeps GenerateWith[T] (instrumented structured output) — this is the
  structured-output primitive; no separate structured/ package.
- llmmeta moved over model/ (the meta-LLM helper: tier allowlist + JSON retry
  + ledger). Its tests configure a minimal tier table via TestMain.

New tests cover the inversion: config overrides fallback, tier registration,
reasoning-suffix survival, nested-tier rejection, nil-sink no-ops.

Full module: go build/vet/test -race green; core go.sum still free of
gorm/redis/discordgo/sqlite.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 19:47:13 -04:00
144 changed files with 21176 additions and 82 deletions
+23 -44
View File
@@ -1,10 +1,8 @@
# Gadfly — agentic adversarial PR reviewer (https://gitea.stevedudenhoeffer.com/steve/gadfly).
#
# Runs the published Gadfly image (pinned to :v1) as a specialist swarm and posts
# ONE consolidated review comment as gitea-actions. Advisory only — never blocks a
# merge. This reviews executus PRs (same setup as mort: m1/m5 locals + 2 cloud,
# 3-lens suite). Gadfly is a simple system — treat its findings as advisory and
# double-check before acting.
# Gadfly adversarial review — subscribes to steve/gadfly's reusable workflow and
# INHERITS its default swarm. This stub holds only the triggers, the actor gate,
# secret forwarding, and the allow-list; the swarm config (models, lenses,
# concurrency, timeouts) lives centrally in gadfly's review-reusable.yml so it is
# tuned in ONE place. Advisory only — never blocks a merge.
name: Adversarial Review (Gadfly)
@@ -31,46 +29,27 @@ concurrency:
jobs:
review:
# Security: only trusted users may trigger a secret-bearing run via a PR
# comment (pull_request + workflow_dispatch are already trusted). Mirrors
# GADFLY_ALLOWED_USERS, the in-container belt-and-suspenders check.
# comment (pull_request + workflow_dispatch are already trusted). Mirrors the
# allowed_users input below (the in-container belt-and-suspenders check) — both
# lists must stay in sync; a workflow if: can't read a workflow_call input.
if: >-
github.event_name != 'issue_comment'
|| (github.event.issue.pull_request
&& (github.actor == 'steve'
|| github.actor == 'fizi'
|| github.actor == 'dazed'))
runs-on: ubuntu-latest
# Full fleet (2 cloud + 2 local Macs, all running concurrently) reviewing
# every PR with the 3-lens suite — the slow local lanes dominate wall time.
timeout-minutes: 90
steps:
- uses: docker://gitea.stevedudenhoeffer.com/steve/gadfly:v1
env:
GITEA_API: ${{ github.server_url }}/api/v1/repos/${{ github.repository }}
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
OLLAMA_CLOUD_API_KEY: ${{ secrets.OLLAMA_CLOUD_API_KEY }}
# Local Ollama boxes (each its own lane, cap 1). NOTE: both Macs must be
# awake/reachable for their reviews to run; if a box is offline, that
# model's comment shows an error and the others still post.
GADFLY_ENDPOINT_M1PRO: "ollama|http://192.168.0.175:11434"
GADFLY_ENDPOINT_M5MAX: "ollama|http://192.168.0.173:11434"
# 2 cloud (parallel) + M1 Pro + M5 Max — one consolidated comment each.
GADFLY_MODELS: "minimax-m3:cloud,deepseek-v4-flash:cloud,m1pro/qwen3:14b,m5max/qwen3.6:35b-mlx"
# cloud runs 2 at once; each Mac one at a time; all three lanes parallel.
GADFLY_PROVIDER_CONCURRENCY: "ollama-cloud=2,m1pro=1,m5max=1"
# Default => the 3-lens suite (security, correctness, error-handling).
# Set the repo var GADFLY_SPECIALISTS to override (csv / "all" / "auto").
GADFLY_SPECIALISTS: ${{ vars.GADFLY_SPECIALISTS || 'security,correctness,error-handling' }}
# Per-lens deadline + bounded steps so the slow local models stay sane.
GADFLY_TIMEOUT_SECS: "600"
GADFLY_MAX_STEPS: "14"
# Allow-list for the comment trigger (mirrors the job-level if: guard).
GADFLY_ALLOWED_USERS: "steve,fizi,dazed"
# --- event context (leave as-is) ---
EVENT_NAME: ${{ github.event_name }}
PR: ${{ github.event.pull_request.number || github.event.issue.number || github.event.inputs.pr_number }}
PR_BRANCH: ${{ github.head_ref }}
IS_DRAFT: ${{ github.event.pull_request.draft }}
COMMENT_BODY: ${{ github.event.comment.body }}
COMMENT_ID: ${{ github.event.comment.id }}
ACTOR: ${{ github.actor }}
# Pinned to an immutable gadfly commit (not @v1): our act_runners are long-lived
# and cache the reusable-workflow ref, so a moved v1 tag keeps resolving to the
# stale cached copy. A unique sha forces a cache miss → fresh fetch. Bump this
# sha to adopt central swarm changes.
uses: steve/gadfly/.gitea/workflows/review-reusable.yml@5007597cf921dc3f0a83c708878facfe65fd8e8b
# Least privilege: forward only the review secrets (not `secrets: inherit`,
# which would expose every repo secret). GITEA_TOKEN is the automatic token.
secrets:
OLLAMA_CLOUD_API_KEY: ${{ secrets.OLLAMA_CLOUD_API_KEY }}
CLAUDE_CODE_OAUTH_TOKEN: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
GADFLY_FINDINGS_URL: ${{ secrets.GADFLY_FINDINGS_URL }}
GADFLY_FINDINGS_TOKEN: ${{ secrets.GADFLY_FINDINGS_TOKEN }}
with:
# Consumer-specific allow-list; everything else is inherited.
allowed_users: "steve,fizi,dazed"
+23
View File
@@ -103,3 +103,26 @@ jobs:
exit 1
fi
echo "OK: core go.sum is free of host/DB dependencies."
- name: Light-tier canary imports no battery
run: |
# examples/reviewer is gadfly's shape on the CORE only. If it ever
# pulls in a battery (audit/budget/persona/skill/critic/schedule/
# checkpoint/contrib), the light path has regressed.
LEAK=$(go list -deps ./examples/reviewer/... | grep -E 'executus/(audit|budget|persona|skill|critic|schedule|checkpoint|contrib)' || true)
if [ -n "$LEAK" ]; then
echo "ERROR: light-tier canary pulled in a battery:"; echo "$LEAK"; exit 1
fi
echo "OK: examples/reviewer is core-only."
- name: contrib/store (nested SQLite module — isolated from core)
run: |
# contrib/store is a SEPARATE module carrying modernc.org/sqlite; the
# core's `go test ./...` doesn't reach it. Build + test it on its own,
# and confirm it DOES carry the driver the core forbids (proof the
# split works: persistence lives here, not in the core go.sum).
cd contrib/store
go build ./...
go test -race -count=1 -timeout 5m ./...
grep -qE 'modernc.org/sqlite' go.sum || { echo "ERROR: contrib/store should carry the sqlite driver"; exit 1; }
echo "OK: contrib/store builds, tests pass, and owns the SQLite dep."
+55 -17
View File
@@ -43,28 +43,66 @@ CORE (majordomo + stdlib):
fanout/ programmatic N×M swarm [P0 ✓]
deliver/ output egress seam (+ Discard/Stdout) [P0 ✓]
identity/ caller identity seams [P0 ✓]
run/ progress bridge now; the executor kernel + [P0 partial]
nil-safe Ports + RunnableAgent later [P2]
run/ run.Executor is RUNNABLE: model-resolve + [P2 core ✓]
toolbox + majordomo loop + compaction +
run-bounding (V10 detached timeout) + step/
audit observers + Budget gate; RunnableAgent
DTO + nil-safe run.Ports. Palette delegation +
Critic (monitor/deadline/steer) + Delivery
WIRED. Follow-ups: Checkpointer (needs a
majordomo msg-history hook), Phases [C0c]
dispatchguard/ loop/depth/fan-out caps [P0 ✓]
pendingattach/ attachment dedupe [P0 ✓]
tool/ registry + 3-stage permissions + ssrf [P1 ✓]
model/ config-driven tier resolution over majordomo [P1]
llmmeta/ shared meta-LLM helper (moves with model/) [P1]
compact/ context compactor (WithCompactor hook) [P2]
tools/{web,net,store,compose,meta,comms} generic tools [P3]
structured/ Generate[T] convenience over majordomo [P1]
model/ config-driven tier resolution over majordomo [P1]
(convar->config.Source; UsageSink/TraceSink seams; GenerateWith[T]
structured output — no separate structured/ pkg)
llmmeta/ shared meta-LLM helper over model/ [P1 ✓]
compact/ context compactor (WithCompactor hook) [P2 ✓]
tools/ generic tool library: Register (think/now/ [P3 ✓]
cite, zero-config) + RegisterMeta (classify/
extract_entities/summarize) + RegisterStore
(kv_*/file_*, default static quota); seams in
research_providers.go/file_storage.go/
kv_storage.go/quota_provider.go. End-to-end
"agent calls a tool" test green. Remaining
(deferred): web/net/compose groups + backends
BATTERIES (opt-in siblings, each nil-safe + a default):
persona/ Agent noun + AgentStore seam + yml loader [P4]
skill/ rich Skill + SkillStore seam + toml loader [P4]
audit/ run-trace Sink (+ Noop/Slog) [P4]
critic/ two-tier timeout state machine + Escalator [P4]
schedule/ cron runner cores [P4]
checkpoint/ durable resume seam [P4]
budget/ rolling-window tracker (+ NoOp) [P4]
persona/ Agent noun + Storage seam + builtin loader [P4]
+ ToRunnable() bridge to run.RunnableAgent +
Memory default (host: chatbot/commands/personalization)
skill/ Skill noun + LEAN SkillStore (lifecycle/ [P4]
versions/schedule, NOT mort's 60-method
monster) + ToRunnable + Memory default
audit/ run.Audit Sink + Writer + queryable Memory [P4]
default (skillaudit Storage iface; GORM stays in mort)
critic/ two-tier timeout watchdog (run.Critic) + [P4 ✓]
Escalator policy seam + ExtendOnce default
schedule/ generic cron Runner (Tick/Loop over a wired [P4 ✓]
Due/Run/Mark/Next; no cron grammar of its own)
checkpoint/ CheckpointStore + run.Checkpointer handle [P4 ✓]
(throttled Save/Complete/Fail) + Memory
budget/ DBBudget rolling-7d + NoOp (run.Budget); [P4 ✓]
BudgetStorage iface + Memory default
skillpack/ SKILL.md-subscription battery: Manifest + [P5 ✓]
Source (Dir/Git) + Subscription/Store +
content-addressed PackCache + Syncer
(pending-only; Apply re-pins) + Activate →
majordomo agent.Skill (catalog + skill_use,
progressive disclosure) + Memory defaults.
NOT executus/skill (saved-agent noun) nor
majordomo/skill (eager capability bundle).
contrib/store/ SECOND module (+ modernc.org/sqlite): [P4]
in-memory + pure-Go SQLite impls of every *Store seam
contrib/store/ SECOND module (+ modernc.org/sqlite): [P4]
pure-Go SQLite impls of ALL store seams: budget +
persona + skill + audit (JSON-blob+indexed cols,
round-trip tested). CI proves the driver lands HERE,
not in the core go.sum.
NOTE: critic/checkpoint executor wiring (run.Ports.Critic /
.Checkpointer call sites) is a P2 follow-up — the batteries +
defaults exist ahead of that wiring.
```
### The one architectural move
@@ -86,7 +124,7 @@ repackaging.
P0 module + zero-coupling moves + core seams (this) → P1 tool registry + model →
P2 run kernel + Ports inversion → P3 generic tools + defaults → P4 persona/skill
redesign + batteries + SQLite store → P5 gadfly on core (light-tier canary) → P6
redesign + batteries + SQLite store → P5 gadfly-on-core canary (examples/reviewer ✓) → P6
rewire mort + tag v0.1.0. The mort-side rewrite reuses mort's existing
`mort_*_adapters.go` wall as the host adapter layer.
+16 -5
View File
@@ -31,15 +31,26 @@ bot) — mort and gadfly are the first two consumers (heavy and light). See
[mort]: https://gitea.stevedudenhoeffer.com/steve/mort
**Available today (P0):**
**Available today:**
- `run/`**executus is runnable.** `run.Executor` ties model resolution, the
tool registry, majordomo's agent loop, context compaction, run-bounding, and
step/audit instrumentation into one `Run(ctx, RunnableAgent, inv) Result`, with
every host concern behind a nil-safe `run.Ports` (Audit/Budget/Critic/
Checkpointer/PaletteSource/Delivery/InputFiles). See `examples/minimal`.
- `model/` — config-driven tier resolution + failover over majordomo, with
pluggable `UsageSink`/`TraceSink` and `GenerateWith[T]` structured output.
- `tool/` — the tool registry + 3-stage permission model + SSRF guard.
- `compact/` — the per-run context compactor.
- `lane/` — bounded worker pool with fair-share queueing (run- and
provider-concurrency).
- `fanout/` — programmatic N×M swarm with bounded global + per-key concurrency.
- `config/` — the host config seam (`Source`) with an env-var default.
- `deliver/` — the output-egress seam with `Discard`/`Stdout` defaults.
- `identity/` — caller-identity seams (`AdminPolicy`, `MemberResolver`).
- `dispatchguard/`, `pendingattach/`, `run/progress.go` — run-safety primitives.
- `config/`, `deliver/`, `identity/` — host seams (config / output / identity),
each with a shipped default.
- `dispatchguard/`, `pendingattach/` — run-safety primitives.
- `examples/reviewer` — a **gadfly-shaped PR reviewer on the core only** (env-config
model fleet → `fanout` N×M swarm → `model.GenerateWith[T]` structured findings →
consolidation), the light-tier canary; CI asserts it pulls in no battery.
## Design
+49
View File
@@ -0,0 +1,49 @@
# gifsmith — a portable, focused render agent that makes animated GIFs/MP4s via
# the `gif` skill pack. Shipped by executus (agentbuiltins), run by any host that
# provides tools with these names, a `thinking` model tier, and the `gif` pack.
# Nothing here is host-specific — the names are the contract the host binds.
name: gifsmith
description: >-
Makes a funny animated GIF (or an MP4 when the piece is long or a GIF is too
big) from a description, via the gif skill pack. A single-purpose render agent
— use it for any request to draw/animate/gif something, including multi-minute
bits about people or things that happened.
model_tier: thinking
system_prompt: |
You make funny animated GIFs and MP4s from a description — often caricatures of
the people in the channel or a bit about something that happened. Work by
calling tools; do NOT introduce yourself or list capabilities.
Load the `gif` skill FIRST: call skill_use with name `gif` to get the full
recipe (scene/cast planning, the code_exec workspace rules, the bundled encode
helper, and the GIF-vs-MP4 size/length decision), then follow it exactly to
render and deliver the result. The skill also bundles an encode helper that
picks GIF vs MP4 and guarantees a Discord-playable MP4 — use it, don't hand-roll
the encode.
Reference images: the render is blind to attachments, so YOU are the eyes —
study any attached/linked image and weave its visual details into the frames.
If you can't make it out, proceed from the words.
low_level_tools:
- code_exec
- image_describe
- send_attachments
- file_get_metadata
- file_save
- think
skill_packs:
- gif
execution_lane: animate
max_iterations: 50
max_tool_calls: 80
max_runtime_seconds: 1800
critic_enabled: true
default_emoji: "🎬"
state_react:
__start__: "🎬"
code_exec: "🐍"
image_describe: "🖼️"
think: "🧠"
send_attachments: "📎"
__end__: "✅"
__error__: "❌"
+24
View File
@@ -0,0 +1,24 @@
// Package agentbuiltins ships executus's canonical builtin agent definitions as
// an embedded filesystem. They are portable persona manifests
// (agents/<name>/agent.yml): each references tool NAMES, a model-tier NAME, and
// skill-pack names — the host binds those to implementations. Nothing here
// imports a host or a battery, so any executus consumer can seed these via
// persona.LoadBuiltinAgents (or its own loader that reads the same schema):
//
// persona.LoadBuiltinAgents(ctx, store, agentbuiltins.FS(), skillChecker)
//
// Ships:
// - gifsmith — a focused GIF/MP4 render agent that uses the `gif` skill pack.
package agentbuiltins
import (
"embed"
"io/fs"
)
//go:embed agents
var embedded embed.FS
// FS returns the builtin agents tree, rooted so that a loader finds each
// definition at agents/<name>/agent.yml (the layout LoadBuiltinAgents expects).
func FS() fs.FS { return embedded }
+42
View File
@@ -0,0 +1,42 @@
package agentbuiltins_test
import (
"context"
"slices"
"testing"
"gitea.stevedudenhoeffer.com/steve/executus/agentbuiltins"
"gitea.stevedudenhoeffer.com/steve/executus/persona"
)
// TestGifsmithLoads proves executus's shipped gifsmith manifest flows through
// the persona loader and lowers into a RunnableAgent carrying the gif pack — the
// path a host uses to dogfood it.
func TestGifsmithLoads(t *testing.T) {
ctx := context.Background()
store := persona.NewMemory()
n, err := persona.LoadBuiltinAgents(ctx, store, agentbuiltins.FS(), nil)
if err != nil {
t.Fatal(err)
}
if n < 1 {
t.Fatalf("expected gifsmith seeded, got %d", n)
}
a, err := store.GetAgentByName(ctx, persona.BuiltinAgentOwnerID, "gifsmith")
if err != nil {
t.Fatal(err)
}
if len(a.SkillPacks) != 1 || a.SkillPacks[0] != "gif" {
t.Errorf("skill_packs = %v", a.SkillPacks)
}
if a.ModelTier != "thinking" {
t.Errorf("model_tier = %q (want a portable tier name)", a.ModelTier)
}
if !slices.Contains(a.LowLevelTools, "code_exec") || !slices.Contains(a.LowLevelTools, "send_attachments") {
t.Errorf("low_level_tools missing render/deliver tools: %v", a.LowLevelTools)
}
// The pack must survive the lowering the executor consumes.
if ra := a.ToRunnable(); len(ra.SkillPacks) != 1 || ra.SkillPacks[0] != "gif" {
t.Errorf("RunnableAgent.SkillPacks = %v", ra.SkillPacks)
}
}
+78
View File
@@ -0,0 +1,78 @@
package audit_test
import (
"context"
"testing"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/majordomo/provider/fake"
"gitea.stevedudenhoeffer.com/steve/executus/audit"
"gitea.stevedudenhoeffer.com/steve/executus/run"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
// TestAuditBatteryEndToEnd wires the audit battery (Memory storage) into
// run.Ports.Audit, runs an agent, and verifies the run was recorded and is
// queryable — proving Sink/Writer/Memory satisfy the core seams end to end.
func TestAuditBatteryEndToEnd(t *testing.T) {
mem := audit.NewMemory()
fp := fake.New("fake")
fp.Enqueue("m", fake.Reply("the answer"))
m, err := fp.Model("m")
if err != nil {
t.Fatal(err)
}
ex := run.New(run.Config{
Registry: tool.NewRegistry(),
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) {
return ctx, m, nil
},
Ports: run.Ports{Audit: audit.NewSink(mem)},
})
res := ex.Run(context.Background(),
run.RunnableAgent{ID: "agent-1", Name: "a", ModelTier: "m"},
tool.Invocation{RunID: "run-xyz", CallerID: "caller-1"},
"question")
if res.Err != nil {
t.Fatalf("run error: %v", res.Err)
}
// The run was recorded with a terminal status + output.
got, err := mem.GetRun(context.Background(), "run-xyz")
if err != nil {
t.Fatalf("GetRun: %v", err)
}
if got.Status != "ok" {
t.Errorf("status = %q, want ok", got.Status)
}
if got.Output != "the answer" {
t.Errorf("output = %q, want %q", got.Output, "the answer")
}
if got.FinishedAt == nil {
t.Error("FinishedAt should be set after the run")
}
if got.SkillID != "agent-1" {
t.Errorf("SkillID = %q, want agent-1 (the subject id)", got.SkillID)
}
// And it is queryable by caller.
runs, err := mem.ListRunsByCaller(context.Background(), "caller-1", 10)
if err != nil {
t.Fatalf("ListRunsByCaller: %v", err)
}
if len(runs) != 1 || runs[0].ID != "run-xyz" {
t.Errorf("ListRunsByCaller = %+v, want [run-xyz]", runs)
}
}
// TestNilSinkRecordsNothing: NewSink(nil) is equivalent to no audit.
func TestNilSinkRecordsNothing(t *testing.T) {
s := audit.NewSink(nil)
if rec := s.StartRun(context.Background(), run.RunInfo{RunID: "r"}); rec != nil {
t.Error("NewSink(nil).StartRun should return a nil recorder")
}
}
+280
View File
@@ -0,0 +1,280 @@
package audit
import (
"context"
"sort"
"sync"
"time"
)
// Memory is an in-process Storage: it retains runs + logs in memory so a light
// host (or a test) gets queryable run history with zero setup. It is bounded
// only by process memory — a host that runs forever should PurgeOlderThan
// periodically, or use a persistent Storage. Construct with NewMemory.
//
// Mort uses its GORM/MySQL Storage; contrib/store adds a durable SQLite one.
// Memory is the zero-dependency default behind audit.NewSink(audit.NewMemory()).
type Memory struct {
mu sync.RWMutex
order []string // run ids in insertion order
runs map[string]SkillRun // by run id
logs map[string][]SkillRunLog // by run id
}
// NewMemory returns an empty in-memory Storage.
func NewMemory() *Memory {
return &Memory{runs: map[string]SkillRun{}, logs: map[string][]SkillRunLog{}}
}
var _ Storage = (*Memory)(nil)
func (m *Memory) Initialize(context.Context) error { return nil }
func (m *Memory) StartRun(_ context.Context, run SkillRun) error {
m.mu.Lock()
defer m.mu.Unlock()
if _, ok := m.runs[run.ID]; !ok {
m.order = append(m.order, run.ID)
}
m.runs[run.ID] = run
return nil
}
func (m *Memory) FinishRun(_ context.Context, runID string, s RunStats) error {
m.mu.Lock()
defer m.mu.Unlock()
r, ok := m.runs[runID]
if !ok {
return ErrNotFound
}
now := time.Now()
r.FinishedAt = &now
r.Status = s.Status
r.Output = s.Output
r.Error = s.Error
r.ToolCallsCount = s.ToolCalls
r.RuntimeSeconds = s.RuntimeSeconds
r.TotalInputTokens = s.InputTokens
r.TotalOutputTokens = s.OutputTokens
r.TotalThinkingTokens = s.ThinkingTokens
m.runs[runID] = r
return nil
}
func (m *Memory) AppendLog(_ context.Context, log SkillRunLog) error {
m.mu.Lock()
defer m.mu.Unlock()
m.logs[log.RunID] = append(m.logs[log.RunID], log)
return nil
}
func (m *Memory) GetRun(_ context.Context, runID string) (*SkillRun, error) {
m.mu.RLock()
defer m.mu.RUnlock()
r, ok := m.runs[runID]
if !ok {
return nil, ErrNotFound
}
return &r, nil
}
func (m *Memory) ListLogsByRun(_ context.Context, runID string) ([]SkillRunLog, error) {
m.mu.RLock()
defer m.mu.RUnlock()
ls := append([]SkillRunLog(nil), m.logs[runID]...)
sort.SliceStable(ls, func(i, j int) bool { return ls[i].Sequence < ls[j].Sequence })
return ls, nil
}
// newestFirst returns the retained runs in reverse insertion order, optionally
// filtered. Caller holds at least RLock.
func (m *Memory) newestFirst(keep func(SkillRun) bool) []SkillRun {
out := make([]SkillRun, 0, len(m.order))
for i := len(m.order) - 1; i >= 0; i-- {
r := m.runs[m.order[i]]
if keep == nil || keep(r) {
out = append(out, r)
}
}
return out
}
// oldestFirst returns the retained runs in insertion (oldest-first) order,
// optionally filtered. Caller holds at least RLock.
func (m *Memory) oldestFirst(keep func(SkillRun) bool) []SkillRun {
out := make([]SkillRun, 0, len(m.order))
for _, id := range m.order {
r := m.runs[id]
if keep == nil || keep(r) {
out = append(out, r)
}
}
return out
}
func page(rs []SkillRun, offset, limit int) []SkillRun {
if offset < 0 {
offset = 0
}
if offset >= len(rs) {
return nil
}
rs = rs[offset:]
if limit > 0 && limit < len(rs) {
rs = rs[:limit]
}
return rs
}
func (m *Memory) ListRunsBySkill(ctx context.Context, skillID string, limit int) ([]SkillRun, error) {
return m.ListRunsBySkillPaginated(ctx, skillID, 0, limit, false)
}
func (m *Memory) ListRunsBySkillPaginated(_ context.Context, skillID string, offset, limit int, includeDryRun bool) ([]SkillRun, error) {
m.mu.RLock()
defer m.mu.RUnlock()
return page(m.newestFirst(func(r SkillRun) bool {
return r.SkillID == skillID && (includeDryRun || r.Status != "dry_run")
}), offset, limit), nil
}
func (m *Memory) CountRunsBySkill(_ context.Context, skillID string, includeDryRun bool) (int64, error) {
m.mu.RLock()
defer m.mu.RUnlock()
return int64(len(m.newestFirst(func(r SkillRun) bool {
return r.SkillID == skillID && (includeDryRun || r.Status != "dry_run")
}))), nil
}
func (m *Memory) ListRunsByCaller(_ context.Context, callerID string, limit int) ([]SkillRun, error) {
m.mu.RLock()
defer m.mu.RUnlock()
return page(m.newestFirst(func(r SkillRun) bool {
return r.CallerID == callerID && r.Status != "dry_run"
}), 0, limit), nil
}
func (m *Memory) matchesFilter(r SkillRun, f RunFilter) bool {
if f.Status != "" {
if r.Status != f.Status {
return false
}
// An explicit Status (even "dry_run") matches regardless of IncludeDryRun.
} else if !f.IncludeDryRun && r.Status == "dry_run" {
return false
}
if f.SkillID != "" && r.SkillID != f.SkillID {
return false
}
if f.CallerID != "" && r.CallerID != f.CallerID {
return false
}
if f.ChannelID != "" && r.ChannelID != f.ChannelID {
return false
}
if f.TopLevelOnly && r.ParentRunID != "" {
return false
}
if !f.Since.IsZero() && r.StartedAt.Before(f.Since) {
return false
}
if !f.Until.IsZero() && r.StartedAt.After(f.Until) {
return false
}
return true
}
func (m *Memory) ListRunsFiltered(_ context.Context, f RunFilter, offset, limit int) ([]SkillRun, error) {
if limit <= 0 || limit > 500 {
limit = 50 // bound admin scans, per the Storage contract
}
m.mu.RLock()
defer m.mu.RUnlock()
return page(m.newestFirst(func(r SkillRun) bool { return m.matchesFilter(r, f) }), offset, limit), nil
}
func (m *Memory) CountRunsFiltered(_ context.Context, f RunFilter) (int64, error) {
m.mu.RLock()
defer m.mu.RUnlock()
return int64(len(m.newestFirst(func(r SkillRun) bool { return m.matchesFilter(r, f) }))), nil
}
func (m *Memory) PurgeOlderThan(_ context.Context, t time.Time) (int64, error) {
m.mu.Lock()
defer m.mu.Unlock()
var purged int64
kept := m.order[:0:0]
for _, id := range m.order {
r := m.runs[id]
if r.FinishedAt != nil && r.FinishedAt.Before(t) {
delete(m.runs, id)
delete(m.logs, id)
purged++
continue
}
kept = append(kept, id)
}
m.order = kept
return purged, nil
}
func (m *Memory) ListChildrenByParent(_ context.Context, parentRunID string) ([]SkillRun, error) {
m.mu.RLock()
defer m.mu.RUnlock()
return m.oldestFirst(func(r SkillRun) bool { return r.ParentRunID == parentRunID }), nil
}
func (m *Memory) WalkParentChain(_ context.Context, runID string) ([]SkillRun, error) {
m.mu.RLock()
defer m.mu.RUnlock()
var chain []SkillRun
seen := map[string]bool{}
for id := runID; id != "" && len(chain) < MaxParentChainDepth; {
r, ok := m.runs[id]
if !ok || seen[id] {
break
}
seen[id] = true
chain = append(chain, r)
id = r.ParentRunID
}
// Contract: root first, the queried run last. We walked child→root, so reverse.
for i, j := 0, len(chain)-1; i < j; i, j = i+1, j-1 {
chain[i], chain[j] = chain[j], chain[i]
}
return chain, nil
}
func (m *Memory) ListFinishedRunsBefore(_ context.Context, cutoff time.Time, limit int) ([]SkillRun, error) {
if limit <= 0 {
return nil, nil // contract: a real bound is required
}
m.mu.RLock()
defer m.mu.RUnlock()
return page(m.oldestFirst(func(r SkillRun) bool {
return r.FinishedAt != nil && r.FinishedAt.Before(cutoff)
}), 0, limit), nil
}
func (m *Memory) LastRunBySkills(_ context.Context, skillIDs []string, includeFailed bool) (map[string]time.Time, error) {
m.mu.RLock()
defer m.mu.RUnlock()
want := map[string]bool{}
for _, id := range skillIDs {
want[id] = true
}
out := map[string]time.Time{}
for _, id := range m.order {
r := m.runs[id]
if !want[r.SkillID] {
continue
}
if !includeFailed && r.Status != "ok" {
continue // contract: only status=="ok" counts unless includeFailed
}
if r.StartedAt.After(out[r.SkillID]) {
out[r.SkillID] = r.StartedAt
}
}
return out, nil
}
+58
View File
@@ -0,0 +1,58 @@
package audit
import (
"context"
"strings"
"testing"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
)
// TestOnToolRedactsSecretTools: a secret-bearing tool's args/result must NOT be
// persisted verbatim in the audit log.
func TestOnToolRedactsSecretTools(t *testing.T) {
ctx := context.Background()
mem := NewMemory()
mem.StartRun(ctx, SkillRun{ID: "r1"})
w := NewWriter(mem, "r1")
secret := `{"url":"https://x","headers":{"Authorization":"Bearer SUPERSECRET"}}`
w.OnTool(llm.ToolCall{Name: "http_get", ID: "1", Arguments: []byte(secret)}, "TOPSECRETBODY")
// a non-secret tool is logged verbatim
w.OnTool(llm.ToolCall{Name: "think", ID: "2", Arguments: []byte(`{"thought":"hi"}`)}, "ok")
logs, _ := mem.ListLogsByRun(ctx, "r1")
var dump strings.Builder
for _, l := range logs {
for k, v := range l.Payload {
dump.WriteString(k)
dump.WriteString("=")
if s, ok := v.(string); ok {
dump.WriteString(s)
}
dump.WriteString(" ")
}
}
all := dump.String()
if strings.Contains(all, "SUPERSECRET") || strings.Contains(all, "TOPSECRETBODY") {
t.Fatalf("secret leaked into audit log: %s", all)
}
// the redaction marker is present, and the non-secret tool's args survive
foundRedacted, foundThink := false, false
for _, l := range logs {
if l.EventType == "tool_call" {
if r, _ := l.Payload["args_redacted"].(bool); r {
foundRedacted = true
}
if a, _ := l.Payload["args"].(string); strings.Contains(a, "thought") {
foundThink = true
}
}
}
if !foundRedacted {
t.Error("secret tool_call should carry args_redacted=true")
}
if !foundThink {
t.Error("non-secret tool args should be logged verbatim")
}
}
+81
View File
@@ -0,0 +1,81 @@
package audit
import (
"context"
"log/slog"
"time"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/executus/run"
)
// Sink adapts an audit Storage to the run.Audit port: StartRun opens a run row
// and returns a per-run recorder (a Writer) that the executor feeds with steps,
// tool calls, and the terminal roll-up. This is what plugs the audit battery
// into run.Ports.Audit — mort backs it with its GORM Storage, a light host with
// Memory() (or omits it entirely).
type Sink struct{ storage Storage }
// NewSink wraps a Storage as a run.Audit. A nil Storage yields a Sink whose
// StartRun returns nil (the executor then records nothing) — so NewSink(nil) is
// equivalent to leaving run.Ports.Audit unset.
func NewSink(storage Storage) *Sink { return &Sink{storage: storage} }
// compile-time proof the adapter satisfies the core seams.
var (
_ run.Audit = (*Sink)(nil)
_ run.RunRecorder = (*recorder)(nil)
)
// StartRun records the run start and returns a recorder. Implements run.Audit.
func (s *Sink) StartRun(ctx context.Context, info run.RunInfo) run.RunRecorder {
if s == nil || s.storage == nil {
return nil
}
started := info.StartedAt
if started.IsZero() {
started = time.Now()
}
// Best-effort: a failed StartRun must not break the user-visible run, but we
// surface it (a swallowed failure leaves orphan log events with no run row).
if err := s.storage.StartRun(ctx, SkillRun{
ID: info.RunID,
SkillID: info.SubjectID,
CallerID: info.CallerID,
ChannelID: info.ChannelID,
ParentRunID: info.ParentRunID,
Inputs: info.Inputs,
StartedAt: started,
Status: "running",
}); err != nil {
slog.Warn("audit: StartRun failed; the run row is missing so its log events will orphan",
"run_id", info.RunID, "error", err)
}
return &recorder{w: NewWriter(s.storage, info.RunID)}
}
// recorder adapts a *Writer to run.RunRecorder, converting run.RunStats to the
// audit RunStats on Close (the two have identical fields).
type recorder struct{ w *Writer }
func (r *recorder) TokenStats() (in, out, thinking int64) { return r.w.TokenStats() }
func (r *recorder) ToolCallsCount() int { return r.w.ToolCallsCount() }
func (r *recorder) OnStep(iter int, resp *llm.Response) { r.w.OnStep(iter, resp) }
func (r *recorder) OnTool(call llm.ToolCall, result string) { r.w.OnTool(call, result) }
func (r *recorder) LogEvent(eventType string, payload map[string]any) {
r.w.LogEvent(eventType, payload)
}
func (r *recorder) LogError(msg string) { r.w.LogError(msg) }
func (r *recorder) Close(ctx context.Context, s run.RunStats) {
r.w.Close(ctx, RunStats{
Status: s.Status,
Output: s.Output,
Error: s.Error,
ToolCalls: s.ToolCalls,
RuntimeSeconds: s.RuntimeSeconds,
InputTokens: s.InputTokens,
OutputTokens: s.OutputTokens,
ThinkingTokens: s.ThinkingTokens,
})
}
+245
View File
@@ -0,0 +1,245 @@
// Package skillaudit persists skill execution traces: per-run summary rows
// (skill_runs) and per-step event logs (skill_run_logs). The executor in
// pkg/logic/skillexec emits events through a Writer; the storage layer is
// kept separate so tests can mock it and so retention pruning has a clear
// home.
//
// Why: agentic runs can be long, multi-tool affairs. Without a structured
// audit trail, debugging "why did the LLM do that?" is impossible. The
// log table is keyed by (run_id, sequence) so insert order is preserved.
package audit
import (
"context"
"errors"
"time"
)
// ErrNotFound is returned when a run lookup fails.
var ErrNotFound = errors.New("skill run not found")
// SkillRun is the per-invocation summary row. One per call to
// Executor.Run. Status transitions through running → ok / error /
// timeout / budget_exceeded / dry_run.
type SkillRun struct {
ID string
SkillID string
CallerID string
ChannelID string
Inputs map[string]any
StartedAt time.Time
FinishedAt *time.Time
Status string // running|ok|error|timeout|budget_exceeded|dry_run
Output string
Error string
ToolCallsCount int
RuntimeSeconds float64
// ParentRunID is the run_id of the parent skill that invoked this
// run via skill_invoke. Empty for top-level invocations. Indexed
// in the gorm model so call-tree queries (ListChildrenByParent +
// WalkParentChain) are cheap.
ParentRunID string
// Token roll-ups, summed across all model completions in this run
// (one Usage per OnStep). All default to 0 when the provider did
// not expose token usage.
TotalInputTokens int64
TotalOutputTokens int64
TotalThinkingTokens int64
}
// RunStats captures the terminal state of a run for FinishRun. Bundling
// these into one struct (vs a long positional argument list) keeps
// callers readable; future fields slot in here without touching every
// call site.
//
// Why: FinishRun originally took six positional args; adding token
// columns would push it higher. A struct is the idiomatic Go way to
// avoid the positional-arg explosion.
type RunStats struct {
Status string // ok|error|timeout|budget_exceeded|dry_run
Output string // final agent output (empty on error)
Error string // error message (empty on success)
ToolCalls int // total OnTool count
RuntimeSeconds float64 // wall-clock duration
// Token roll-ups (all default to 0 when token usage was not
// exposed by the provider).
InputTokens int64
OutputTokens int64
ThinkingTokens int64
}
// SkillRunLog is one event recorded during a run. EventType ∈
// step|tool_call|tool_result|error. Payload is opaque JSON the writer
// emits.
type SkillRunLog struct {
RunID string
Sequence int
EventType string
Payload map[string]any
CreatedAt time.Time
}
// RunFilter is the predicate bundle for the cross-surface "recent runs"
// query (ListRunsFiltered / CountRunsFiltered). Every field is optional;
// the zero value matches the most recent runs across ALL audited surfaces
// (agents + skills). This powers the admin agent-trace debug view and the
// Claude debug API's /runs list.
//
// Why a struct (vs positional args): the debug list filters along several
// independent axes and more will be added; bundling avoids a positional
// explosion and keeps call sites readable.
type RunFilter struct {
Status string // exact status match; "" = all (dry_run excluded unless IncludeDryRun)
SkillID string // exact skill_id (holds the agent UUID for agent runs)
CallerID string // exact caller (Discord member id)
ChannelID string // exact channel id
// TopLevelOnly restricts to root runs (parent_run_id = ''), hiding
// nested sub-agent / sub-skill runs from the firehose. The debug list
// defaults this on; an "include nested" toggle clears it.
TopLevelOnly bool
// IncludeDryRun surfaces status="dry_run" sandbox rows, which are
// excluded by default. Ignored when Status is set explicitly (an
// explicit Status=="dry_run" still matches).
IncludeDryRun bool
// Since / Until bound started_at: started_at >= Since (zero = no lower
// bound) and started_at < Until (zero = no upper bound).
Since time.Time
Until time.Time
}
// Storage is the persistence interface for skill runs and per-step logs.
//
// Why: tests substitute fake implementations; production wires
// NewGormStorage. Keep the interface narrow — the system only needs CRUD
// plus the retention prune helper.
type Storage interface {
Initialize(ctx context.Context) error
// StartRun inserts the run with status=running. The caller MUST
// invoke FinishRun later (or the row stays in running indefinitely
// — operationally that signals a crash mid-run, which is useful
// signal).
StartRun(ctx context.Context, run SkillRun) error
// FinishRun updates the running row with terminal status, output
// and stats. Idempotent on second call (last write wins).
//
// V5: takes a RunStats struct so token + cost columns can be
// written alongside the legacy fields without changing the
// signature for every future addition.
FinishRun(ctx context.Context, runID string, stats RunStats) error
// AppendLog adds one event to the run's log. Sequence numbers must
// be unique per run; the writer is responsible for monotonic
// ordering.
AppendLog(ctx context.Context, log SkillRunLog) error
// GetRun returns the run summary, or ErrNotFound.
GetRun(ctx context.Context, runID string) (*SkillRun, error)
// ListLogsByRun returns all logs for a run in sequence order.
ListLogsByRun(ctx context.Context, runID string) ([]SkillRunLog, error)
// ListRunsBySkill returns recent runs for a skill, newest first,
// capped at limit. Excludes dry-run rows by default — use
// ListRunsBySkillPaginated with includeDryRun=true to see them.
ListRunsBySkill(ctx context.Context, skillID string, limit int) ([]SkillRun, error)
// ListRunsBySkillPaginated returns recent runs for a skill, newest
// first, with offset+limit. When includeDryRun is false, rows with
// status="dry_run" are excluded (matches the wizard's sandbox
// status; see skillaudit.Writer / wizardtools docs).
//
// Why a separate paginated method vs. expanding ListRunsBySkill:
// callers that need the legacy "last N" view (Discord .skill runs,
// chatbot tool result) want the simpler signature; the paginated
// view is webui-specific.
ListRunsBySkillPaginated(ctx context.Context, skillID string,
offset, limit int, includeDryRun bool) ([]SkillRun, error)
// CountRunsBySkill returns the total number of runs for a skill.
// When includeDryRun is false, dry-run rows are excluded so the
// count matches the default ListRunsBySkillPaginated result.
CountRunsBySkill(ctx context.Context, skillID string, includeDryRun bool) (int64, error)
// ListRunsByCaller returns recent runs by a caller, newest first,
// capped at limit.
ListRunsByCaller(ctx context.Context, callerID string, limit int) ([]SkillRun, error)
// ListRunsFiltered returns runs matching f, newest first
// (started_at DESC), with offset+limit. With an all-zero filter it
// returns the most recent runs across EVERY audited surface (agents +
// skills) — the cross-surface feed behind the admin agent-trace debug
// view and the Claude debug API. dry_run rows are excluded unless
// f.IncludeDryRun or f.Status=="dry_run". limit is clamped (<=0 or
// >500 → 50) to bound admin scans.
ListRunsFiltered(ctx context.Context, f RunFilter, offset, limit int) ([]SkillRun, error)
// CountRunsFiltered returns the total rows matching f (ignoring
// offset/limit), for pagination math.
CountRunsFiltered(ctx context.Context, f RunFilter) (int64, error)
// PurgeOlderThan deletes runs (and their logs) whose StartedAt is
// strictly before t. Returns the number of runs deleted.
PurgeOlderThan(ctx context.Context, t time.Time) (int64, error)
// ListChildrenByParent returns all SkillRun rows where
// parent_run_id == parentRunID, oldest first. Used for the
// call-tree view (skill_invoke trace section) and as a building
// block for WalkParentChain.
//
// Returns an empty slice when parentRunID has no children. An
// empty parentRunID never matches anything (no row stores ""
// as a parent — that's the top-level sentinel).
ListChildrenByParent(ctx context.Context, parentRunID string) ([]SkillRun, error)
// WalkParentChain walks from runID up via parent_run_id, returning
// the chain of SkillRun summaries (oldest = root first, newest =
// runID last). Used by the loop guard in skill_invoke.
//
// Cap walk depth at 32 to prevent pathological loops in the data
// itself: if the parent_run_id chain has been corrupted (e.g. by
// a bad migration) and forms a cycle, we want a bounded result
// rather than an infinite loop.
WalkParentChain(ctx context.Context, runID string) ([]SkillRun, error)
// ListFinishedRunsBefore returns runs whose FinishedAt is strictly
// before cutoff, oldest first, capped at limit. limit <= 0 yields
// no rows (the caller is expected to specify a real bound).
//
// Why: skills.StorageSweeper drives the run-scope storage purge from
// this query. The sweeper picks up only finished runs so an
// in-flight run's run-scope KV/files cannot be deleted out from
// under it.
//
// Test: storage_test.go covers the include/exclude boundaries
// (running rows excluded; finished-after-cutoff excluded; finished-
// before-cutoff included).
ListFinishedRunsBefore(ctx context.Context, cutoff time.Time, limit int) ([]SkillRun, error)
// LastRunBySkills returns the most recent StartedAt timestamp per
// skill in the input ID list. Skills with no rows simply have no
// entry in the result map (caller distinguishes "never run" from
// "run but no timestamp" by map key presence).
//
// When includeFailed is true, all non-dry-run statuses count
// (ok / error / timeout / budget_exceeded / preempted / lane_busy).
// When false, only status="ok" rows count — useful for "last
// successful run" semantics on dashboards where errored runs
// shouldn't surface as recent activity.
//
// Empty skillIDs short-circuits to an empty map without touching
// the DB.
LastRunBySkills(ctx context.Context, skillIDs []string, includeFailed bool) (map[string]time.Time, error)
}
// MaxParentChainDepth is the safety cap for WalkParentChain. The loop
// guard in skill_invoke enforces a separate (smaller) MaxInvokeDepth
// at the tool layer; this cap exists only to bound the walk in the
// presence of corrupted data.
const MaxParentChainDepth = 32
+359
View File
@@ -0,0 +1,359 @@
package audit
import (
"context"
"fmt"
"log/slog"
"strings"
"sync"
"sync/atomic"
"time"
llm "gitea.stevedudenhoeffer.com/steve/majordomo/llm"
)
// stepTextMax caps the per-step assistant-text preview persisted on a
// "step" event. Large enough to capture the model's reasoning around a
// (mis)fired tool call — the single best clue to WHY a model emitted a
// malformed call — but bounded so the longtext payload can't balloon.
const stepTextMax = 2000
// Writer wraps a Storage with the OnStep / OnTool callbacks suitable for
// wiring into the majordomo agent loop's step observer, tracking sequence
// numbers and tool-call counts internally.
//
// Why: the agent loop's observer hooks are unaware of run identity; the
// writer captures the runID + skill metadata at construction so the
// per-event callbacks stay simple. AppendLog failures are logged but
// never fatal — audit must not break user-visible execution.
//
// What: NewWriter(storage, runID) → use OnStep / OnTool / Close. Close
// records the final FinishRun. The executors translate each agent.Step
// into one OnStep call (1-indexed iteration, the step's *llm.Response)
// plus one OnTool call per executed tool.
//
// Test: see writer_test.go for sequence ordering and finish semantics.
type Writer struct {
storage Storage
runID string
sequence atomic.Int32
calls atomic.Int32
mu sync.Mutex // guards Close idempotency + token tally
closed bool
// V5 token accumulator — summed across each OnStep's resp.Usage.
// Reads come from TokenStats() so the executor can pass them to
// FinishRun. atomics-on-Int64 would also work, but mu already
// guards Close + we need consistent multi-field reads anyway
// (input + output + thinking). The mutex hot-path overhead is
// negligible vs the LLM call latency that dominates step time.
inputTokens int64
outputTokens int64
thinkingTokens int64
// Per-step wall-clock + run-level model attribution (guarded by mu).
// startedAt anchors the first step's duration; lastStepAt is the
// previous step's observation time; resolvedModelLogged ensures the
// one-shot "resolved_model" run-level event fires at most once.
startedAt time.Time
lastStepAt time.Time
resolvedModelLogged bool
}
// NewWriter constructs a Writer. The caller is expected to have already
// called Storage.StartRun.
func NewWriter(storage Storage, runID string) *Writer {
return &Writer{storage: storage, runID: runID, startedAt: time.Now()}
}
// OnStep records one agent-loop step: a "step" event with the iteration
// number and the response's text size.
//
// V5: also tallies per-step token usage. majordomo populates
// resp.Usage when the provider reports it; for providers that don't,
// the fields stay 0 and the tally stays at 0 — the formatter then
// renders "—" rather than a misleading "$0.00".
//
// Why we tally here vs in the agent loop: the loop's Result.Usage is a
// run total; the audit row needs the same numbers, but the writer also
// serves the live RunState accessor mid-run, so a per-step running sum
// is the right shape. Global usage attribution is handled by the llms
// package's instrumented models — the writer tally is strictly the
// per-run audit roll-up.
func (w *Writer) OnStep(iter int, resp *llm.Response) {
if w == nil || w.storage == nil {
return
}
now := time.Now()
payload := map[string]any{"iter": iter}
w.mu.Lock()
// Per-step wall-clock: time since the previous observed step, or since
// run start for the first step. A long gap localises a slow/hung model
// call — the signal that was missing when an animate step-0 call hung
// ~5 min. NOTE: this is step-to-step wall time (model call + the prior
// step's tool execution), not pure model latency.
prev := w.lastStepAt
if prev.IsZero() {
prev = w.startedAt
}
if !prev.IsZero() {
payload["step_ms"] = now.Sub(prev).Milliseconds()
}
w.lastStepAt = now
if resp != nil {
w.inputTokens += int64(resp.Usage.InputTokens)
w.outputTokens += int64(resp.Usage.OutputTokens)
// Thinking/reasoning tokens are a first-class Usage field in
// majordomo (populated by the providers that report them).
w.thinkingTokens += int64(resp.Usage.ReasoningTokens)
}
// One-shot run-level served-model attribution: the FIRST step with a
// resolved model name emits a "resolved_model" event so a run that
// errors before producing a useful step still records which model
// served it. resp.Model is failover-aware ("provider/model-id" of the
// element that actually served), unlike the static configured head.
logResolvedModel := ""
if resp != nil && resp.Model != "" && !w.resolvedModelLogged {
w.resolvedModelLogged = true
logResolvedModel = resp.Model
}
w.mu.Unlock()
if resp != nil {
payload["text_len"] = len(resp.Text())
// Served model + why generation stopped — the two scalars that turn
// a "model misbehaved" guess into a fact. finish_reason on an
// empty-tool-call step disambiguates truncation (length) from a
// deliberate empty emission (tool_calls).
if resp.Model != "" {
payload["model"] = resp.Model
}
if resp.FinishReason != "" {
payload["finish_reason"] = string(resp.FinishReason)
}
// Per-step token breakdown (OnStep already reads these into the run
// total above; persisting the per-step slice costs nothing more).
payload["in_tokens"] = resp.Usage.InputTokens
payload["out_tokens"] = resp.Usage.OutputTokens
if resp.Usage.ReasoningTokens > 0 {
payload["thinking_tokens"] = resp.Usage.ReasoningTokens
}
if resp.Usage.CacheReadTokens > 0 {
payload["cache_read_tokens"] = resp.Usage.CacheReadTokens
}
// The model's own narration accompanying this step — the smoking gun
// for WHY a malformed tool call was emitted. Capped; suppressed when
// the step fired a secret-bearing tool (mcp_call/email_send/http_*)
// whose narration could echo the secret it's about to send.
if t := strings.TrimSpace(resp.Text()); t != "" {
if stepHasSecretTool(resp) {
payload["text_redacted"] = true
} else {
payload["text"] = truncate(t, stepTextMax)
}
}
} else {
payload["text_len"] = 0
}
w.appendLog("step", payload)
if logResolvedModel != "" {
w.appendLog("resolved_model", map[string]any{"model": logResolvedModel})
}
}
// stepHasSecretTool reports whether a step's response fired a tool whose
// surrounding narration could leak a secret (MCP args, email body/
// recipients, raw HTTP request). Mirrors the steps.go redaction list so
// the audit trace never persists secret-adjacent assistant text.
// isSecretTool reports whether a tool's arguments/results may carry secrets
// (MCP args, email bodies/recipients, HTTP auth headers/bodies) and so must be
// redacted from the persisted audit log. Single source of truth for both the
// step-narration redaction and the OnTool arg/result redaction. NOTE: this is
// a name-prefix allowlist — a NEW secret-bearing tool must be added here or its
// args/results will be logged verbatim.
func isSecretTool(name string) bool {
switch name {
case "mcp_call", "email_send":
return true
}
return strings.HasPrefix(name, "http_")
}
func stepHasSecretTool(resp *llm.Response) bool {
if resp == nil {
return false
}
for _, c := range resp.ToolCalls {
if isSecretTool(c.Name) {
return true
}
}
return false
}
// TokenStats returns the running totals tallied from OnStep.
// Safe to call concurrently. Returned values are a snapshot at call
// time. Used by the executors to populate RunStats before Close
// finalises the audit row.
//
// Why: the executor needs the totals AND a model name to compute cost,
// but cost calculation is a different concern from audit persistence.
// Exposing this getter lets the cost calculation live in the executor
// where the model is known.
func (w *Writer) TokenStats() (input, output, thinking int64) {
if w == nil {
return 0, 0, 0
}
w.mu.Lock()
defer w.mu.Unlock()
return w.inputTokens, w.outputTokens, w.thinkingTokens
}
// OnTool records a "tool_call" event with the tool name and a
// "tool_result" event with the result length. Tool count is incremented
// for each call. The executors call this once per executed tool call
// from their step observers (call + matching result content).
func (w *Writer) OnTool(call llm.ToolCall, result string) {
if w == nil || w.storage == nil {
return
}
w.calls.Add(1)
// Redact the args/result of secret-bearing tools — these fields actually
// CARRY the secret (MCP args, email body/recipients, HTTP auth/body), so
// logging them verbatim would defeat the OnStep narration redaction.
if isSecretTool(call.Name) {
w.appendLog("tool_call", map[string]any{
"name": call.Name,
"id": call.ID,
"args_redacted": true,
"args_len": len(call.Arguments),
})
w.appendLog("tool_result", map[string]any{
"name": call.Name,
"id": call.ID,
"result_redacted": true,
"result_len": len(result),
})
return
}
w.appendLog("tool_call", map[string]any{
"name": call.Name,
"args": string(call.Arguments),
"id": call.ID,
})
w.appendLog("tool_result", map[string]any{
"name": call.Name,
"id": call.ID,
"result": truncate(result, 4000),
"truncated": len(result) > 4000,
})
}
// LogEvent records a custom event mid-run. The executor uses this for
// diagnostic events (e.g. "compaction_setup" / "compaction_fired")
// outside the canonical step / tool_call / tool_result / error set.
// Nil-safe: no-op when receiver or storage is nil.
//
// Why: skill_run_logs is the only sink Steve can read from SQL, so
// diagnostics intended for post-hoc debugging belong here. slog goes
// to mort.log which is harder to reach from outside the host.
func (w *Writer) LogEvent(eventType string, payload map[string]any) {
if w == nil || w.storage == nil {
return
}
w.appendLog(eventType, payload)
}
// LogError records an "error" event mid-run. Distinct from the terminal
// status set by Close.
func (w *Writer) LogError(msg string) {
if w == nil || w.storage == nil {
return
}
w.appendLog("error", map[string]any{"message": msg})
}
// Close finishes the run. The caller assembles a RunStats; the writer
// fills in ToolCalls (which is bookkept on the writer itself) and
// hands the full record to FinishRun.
//
// Idempotent: subsequent calls are no-ops.
//
// Why a struct vs the old positional form: v5 adds four token + cost
// fields on top of the legacy six. The struct keeps call sites readable
// and lets future fields slot in without churning every caller.
//
// Why context.WithoutCancel: the run's terminal status MUST land in
// the audit row regardless of the run ctx's state. Pre-fix, child
// skill runs invoked via skill_invoke / skill_invoke_parallel inherited
// the parent agent's runCtx as their outer ctx; when the parent
// timed out at MaxRuntime, every in-flight child's FinishRun fired
// with that already-cancelled ctx and the row was left in
// status=running forever. Detaching here is defence in depth — the
// caller (skillexec.runInner / agentexec.runInner) ALSO detaches at
// the call site, but a cancelled ctx in the writer's hands MUST NOT
// drop the audit write. The short timeout (auditFinishTimeout) bounds
// the write so a hung DB doesn't pin the run goroutine indefinitely.
func (w *Writer) Close(ctx context.Context, stats RunStats) {
if w == nil || w.storage == nil {
return
}
w.mu.Lock()
defer w.mu.Unlock()
if w.closed {
return
}
w.closed = true
stats.ToolCalls = int(w.calls.Load())
// Detach from the caller's deadline + cancellation. Run cleanup
// must complete even when the run ctx is dead. The fresh
// auditFinishTimeout caps how long we'll wait on the storage.
finishCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), auditFinishTimeout)
defer cancel()
if err := w.storage.FinishRun(finishCtx, w.runID, stats); err != nil {
slog.Warn("skillaudit: FinishRun failed", "run_id", w.runID, "error", err)
}
}
// auditFinishTimeout caps how long Close will wait on the storage's
// FinishRun call after detaching from the caller's ctx. 10s is generous
// for a single-row UPDATE against MySQL — anything longer suggests a
// hung connection that the run goroutine shouldn't keep waiting on.
const auditFinishTimeout = 10 * time.Second
// auditAppendTimeout bounds each per-event AppendLog on the hot path so a hung
// storage backend can't block the run goroutine.
const auditAppendTimeout = 3 * time.Second
// ToolCallsCount returns how many tool invocations OnTool has seen so
// far. Useful for budget enforcement.
func (w *Writer) ToolCallsCount() int { return int(w.calls.Load()) }
func (w *Writer) appendLog(eventType string, payload map[string]any) {
seq := int(w.sequence.Add(1))
log := SkillRunLog{
RunID: w.runID,
Sequence: seq,
EventType: eventType,
Payload: payload,
CreatedAt: time.Now(),
}
// Bound the write: a hung storage backend must not block the run goroutine
// on the hot path (every step/tool event flows through here). Detached from
// any caller deadline — the log write is independent of the run's context.
ctx, cancel := context.WithTimeout(context.Background(), auditAppendTimeout)
defer cancel()
if err := w.storage.AppendLog(ctx, log); err != nil {
slog.Warn("skillaudit: AppendLog failed", "run_id", w.runID, "seq", seq, "type", eventType, "error", err)
}
}
func truncate(s string, max int) string {
if len(s) <= max {
return s
}
return s[:max] + fmt.Sprintf("…[+%d bytes]", len(s)-max)
}
+169
View File
@@ -0,0 +1,169 @@
// Package budget gates and meters per-caller resource use over a rolling
// 7-day window (run.Ports.Budget). DBBudget is the durable tracker; NoOpBudget
// disables metering; the BudgetStorage seam backs it (Memory / contrib SQLite).
// loop (gitea.stevedudenhoeffer.com/steve/majordomo/agent).
//
// Why: a Skill is data; the executor turns data into a running agent
// (resolve model, build toolbox, start audit, run the agent loop,
// finish audit, deliver).
package budget
import (
"context"
"errors"
"fmt"
"log/slog"
"time"
)
// BudgetTracker enforces per-user GPU budgets in v2. v1 ships
// NoOpBudget which always allows. The interface exists now so the v2
// migration is a single line in the executor.
//
// Why interface now: the executor's Check/Commit calls would need to
// be added in v2 anyway; doing it now means v2 only swaps NoOp for
// DBBudget without touching call sites.
type BudgetTracker interface {
// Check reports whether the caller has remaining budget. Returns
// nil for "yes" or an error describing the exhaustion.
Check(ctx context.Context, callerID string) error
// Commit records that the caller spent runtimeSeconds of budget on
// this run. Called after the agent completes (success or error).
Commit(ctx context.Context, callerID string, runtimeSeconds float64)
}
// NoOpBudget always allows and never records. v1 default.
type NoOpBudget struct{}
// NewNoOpBudget constructs the no-op tracker.
func NewNoOpBudget() BudgetTracker { return NoOpBudget{} }
// Check always returns nil.
func (NoOpBudget) Check(_ context.Context, _ string) error { return nil }
// Commit is a no-op.
func (NoOpBudget) Commit(_ context.Context, _ string, _ float64) {}
// ErrBudgetExceeded is returned by DBBudget.Check when the caller's
// 7-day rolling window has hit the convar-configured cap.
//
// Why a sentinel: callers (executor, audit writer) need to distinguish
// budget rejection from generic errors so they can record
// status="budget_exceeded" instead of "error" and skip user-visible
// delivery side-effects.
var ErrBudgetExceeded = errors.New("weekly skill budget exceeded")
// BudgetNotifier is the optional callback DBBudget invokes when a
// Check rejects a caller. Production wires a Discord-DM hook so the
// user knows why their skill failed; tests inject a recorder.
//
// nil is allowed and is silently skipped.
type BudgetNotifier func(ctx context.Context, userID string, secondsUsed, cap float64)
// DBBudget enforces per-user weekly GPU budgets via the BudgetStorage
// interface. The "weekly" cap is a rolling 7-day window — see
// SkillBudget for the rollover semantics.
//
// Why a closure for the limit instead of an int field: the cap comes
// from a runtime convar. Reading it on every Check means a `.convar
// set skills.user_budget_seconds_per_week 7200` takes effect on the
// next call without restarting the bot or rewiring the executor.
type DBBudget struct {
storage BudgetStorage
// weeklyLimit returns the current cap in seconds. Reads convar at
// every Check so a runtime convar bump takes effect on the next
// call.
weeklyLimit func() float64
// notify is called when a Check rejects a caller. Optional —
// production wires a Discord-DM hook so the user knows why their
// skill failed. nil-safe.
notify BudgetNotifier
// now is the time source. Test injects a fake clock; production
// uses time.Now.
now func() time.Time
}
// NewDBBudget constructs a DBBudget. now may be nil — defaults to
// time.Now.
//
// Why time injection: budget rollover is time-sensitive; tests need to
// fast-forward past the 7-day boundary deterministically. now=nil
// means production callers (mort.go) don't have to think about it.
//
// Test: pass a closure that returns a fixed instant; assert rollover
// only happens when (now - WindowStart) >= 7 days.
func NewDBBudget(storage BudgetStorage, weeklyLimit func() float64, notify BudgetNotifier, now func() time.Time) *DBBudget {
if now == nil {
now = time.Now
}
return &DBBudget{
storage: storage,
weeklyLimit: weeklyLimit,
notify: notify,
now: now,
}
}
// Check returns ErrBudgetExceeded if the caller has spent at least
// weeklyLimit seconds in the current rolling 7-day window.
//
// Why anonymous callerID="" is unbudgeted: scheduler-driven and
// system-initiated runs don't have a Discord user to bill; charging
// "system" would conflate them with a real user. The scheduler sets
// CallerID to the skill owner where applicable, so cron-loop
// abusiveness still consumes the owner's budget.
//
// Why cap<=0 means "disabled": operator wants a runtime kill-switch.
// Setting the convar to "0" turns enforcement off without restart.
//
// Test: Get returns nil → Check returns nil; Get returns row with
// SecondsUsed >= cap → Check returns ErrBudgetExceeded and notify is
// invoked; window expired (>=7d) → Check returns nil regardless of
// SecondsUsed.
func (b *DBBudget) Check(ctx context.Context, callerID string) error {
if callerID == "" {
return nil
}
bud, err := b.storage.Get(ctx, callerID)
if err != nil {
return fmt.Errorf("budget: %w", err)
}
if bud != nil {
if b.now().Sub(bud.WindowStart) < budgetWindow {
cap := b.weeklyLimit()
if cap > 0 && bud.SecondsUsed >= cap {
if b.notify != nil {
b.notify(ctx, callerID, bud.SecondsUsed, cap)
}
return ErrBudgetExceeded
}
}
}
return nil
}
// Commit records the run's runtime against the caller's budget.
// Failures are logged but never returned — budget accounting must
// not break user-visible execution.
//
// Why callerID="" is a no-op: matches Check's anonymous-caller
// shortcut; system runs don't get billed.
//
// Why runtimeSeconds<=0 is a no-op: a run that errored before
// resolving a model has wallSecs near 0 in floating-point terms but
// can also be exactly 0 (synthetic test fixtures). Skipping avoids
// spurious 0-runs rows from short-lived failures.
//
// Test: Commit(50) → Get reports SecondsUsed=50; storage failure
// surfaces only as a slog.Warn (no panic, no return).
func (b *DBBudget) Commit(ctx context.Context, callerID string, runtimeSeconds float64) {
if callerID == "" || runtimeSeconds <= 0 {
return
}
if err := b.storage.Add(ctx, callerID, runtimeSeconds, b.now()); err != nil {
slog.Warn("skills budget: commit failed", "user", callerID, "error", err)
}
}
+44
View File
@@ -0,0 +1,44 @@
package budget
import (
"context"
"errors"
"testing"
"time"
)
func TestDBBudgetRollingWindow(t *testing.T) {
ctx := context.Background()
mem := NewMemory()
now := time.Now()
clock := func() time.Time { return now }
b := NewDBBudget(mem, func() float64 { return 100 }, nil, clock)
// Under cap: allowed.
if err := b.Check(ctx, "u"); err != nil {
t.Fatalf("fresh caller should pass: %v", err)
}
b.Commit(ctx, "u", 60)
if err := b.Check(ctx, "u"); err != nil {
t.Fatalf("60/100 should pass: %v", err)
}
// Over cap: rejected.
b.Commit(ctx, "u", 50) // 110 total
if err := b.Check(ctx, "u"); !errors.Is(err, ErrBudgetExceeded) {
t.Fatalf("110/100 should be ErrBudgetExceeded, got %v", err)
}
// Window rolls over after 7 days: allowed again.
now = now.Add(8 * 24 * time.Hour)
b.Commit(ctx, "u", 1) // triggers rollover inside Add
if err := b.Check(ctx, "u"); err != nil {
t.Fatalf("after window rollover should pass: %v", err)
}
}
func TestNoOpBudgetAlwaysAllows(t *testing.T) {
b := NewNoOpBudget()
if err := b.Check(context.Background(), "anyone"); err != nil {
t.Fatalf("NoOp must always allow: %v", err)
}
b.Commit(context.Background(), "anyone", 1e9) // no-op
}
+56
View File
@@ -0,0 +1,56 @@
package budget
import (
"context"
"sync"
"time"
)
// Memory is a zero-dependency in-process BudgetStorage: per-user rolling-window
// usage held in memory (lost on restart). The default behind DBBudget for a
// light host or tests; mort uses its GORM Storage, contrib/store adds SQLite.
type Memory struct {
mu sync.RWMutex
rows map[string]*SkillBudget
}
// NewMemory returns an empty in-memory BudgetStorage.
func NewMemory() *Memory { return &Memory{rows: map[string]*SkillBudget{}} }
var _ BudgetStorage = (*Memory)(nil)
func (m *Memory) Initialize(context.Context) error { return nil }
func (m *Memory) Get(_ context.Context, userID string) (*SkillBudget, error) {
m.mu.RLock()
defer m.mu.RUnlock()
r, ok := m.rows[userID]
if !ok {
return nil, nil
}
cp := *r // copy out so callers can't mutate our row
return &cp, nil
}
func (m *Memory) Add(_ context.Context, userID string, secondsUsed float64, now time.Time) error {
m.mu.Lock()
defer m.mu.Unlock()
r, ok := m.rows[userID]
if !ok {
m.rows[userID] = &SkillBudget{
UserID: userID, WindowStart: now,
SecondsUsed: secondsUsed, RunsCount: 1, UpdatedAt: now,
}
return nil
}
// Roll the window over if it's older than the window length.
if now.Sub(r.WindowStart) >= budgetWindow {
r.WindowStart = now
r.SecondsUsed = 0
r.RunsCount = 0
}
r.SecondsUsed += secondsUsed
r.RunsCount++
r.UpdatedAt = now
return nil
}
+9
View File
@@ -0,0 +1,9 @@
package budget
import "gitea.stevedudenhoeffer.com/steve/executus/run"
// The budget trackers plug directly into run.Ports.Budget (Check/Commit match).
var (
_ run.Budget = NoOpBudget{}
_ run.Budget = (*DBBudget)(nil)
)
+33
View File
@@ -0,0 +1,33 @@
package budget
import (
"context"
"time"
)
// BudgetStorage is the persistence seam behind DBBudget: one budget row per
// user, with an atomic Add that rolls the 7-day window over transparently. Mort
// backs this with GORM/MySQL (the skill_budgets table); Memory() is the
// zero-dependency default; contrib/store adds a durable SQLite one.
type BudgetStorage interface {
// Initialize runs any schema setup. Safe to call repeatedly.
Initialize(ctx context.Context) error
// Get returns the user's current budget row, or (nil, nil) if none exists.
Get(ctx context.Context, userID string) (*SkillBudget, error)
// Add increments seconds_used + runs_count atomically, rolling the window
// over when WindowStart is older than 7 days (reset to now, fresh count).
// Creates the row if absent.
Add(ctx context.Context, userID string, secondsUsed float64, now time.Time) error
}
// SkillBudget is one user's rolling-window usage row.
type SkillBudget struct {
UserID string
WindowStart time.Time
SecondsUsed float64
RunsCount int
UpdatedAt time.Time
}
// budgetWindow is the rolling window length the storage rolls over at.
const budgetWindow = 7 * 24 * time.Hour
+52
View File
@@ -0,0 +1,52 @@
// Package checkpoint is the durable-resume battery: it persists a run's
// resumable progress so a run interrupted by a shutdown can be recovered and
// continued on the next boot, rather than silently lost. It plugs into
// run.Ports.Checkpointer.
//
// Mort backs CheckpointStore with its durable-job table; Memory() is the
// zero-dependency default; contrib/store can add a SQLite one. The executor calls
// run.Ports.Checkpointer (a CheckpointerFactory) during the run loop; NewFactory
// wires this battery into that seam.
package checkpoint
import (
"context"
"time"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/executus/run"
)
// RunCheckpointMeta is the run attribution needed to resume a run from scratch
// (mirrors mort's agentexec.RunCheckpointMeta).
type RunCheckpointMeta struct {
RunID string
AgentID string
AgentName string
CallerID string
ChannelID string
GuildID string
Prompt string
ModelTier string
ParentRunID string
}
// RunCheckpoint is one persisted snapshot of a run's resumable progress.
type RunCheckpoint struct {
Meta RunCheckpointMeta
Messages []llm.Message // conversation so far (single-loop runs)
Iteration int // completed agent-loop iterations
CompletedPhases []run.PhaseOutput // finished phases, in order (multi-phase agents)
UpdatedAt time.Time
}
// CheckpointStore persists run checkpoints keyed by run id. A live checkpoint
// means "this run was in flight and not cleanly finished"; Complete/Fail delete
// it. ListInterrupted returns every surviving checkpoint at boot for recovery.
type CheckpointStore interface {
Save(ctx context.Context, cp RunCheckpoint) error
Load(ctx context.Context, runID string) (*RunCheckpoint, error)
Delete(ctx context.Context, runID string) error
ListInterrupted(ctx context.Context) ([]RunCheckpoint, error)
}
+64
View File
@@ -0,0 +1,64 @@
package checkpoint
import (
"context"
"testing"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/run"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
)
func TestHandleSaveCompleteDelete(t *testing.T) {
ctx := context.Background()
mem := NewMemory()
meta := RunCheckpointMeta{RunID: "r1", AgentID: "a1", CallerID: "c1"}
cp := New(mem, meta, 0, nil) // throttle 0 = save every call
if err := cp.Save(ctx, run.RunCheckpointState{Messages: []llm.Message{{Role: "user"}}, Iteration: 2}); err != nil {
t.Fatal(err)
}
got, _ := mem.Load(ctx, "r1")
if got == nil || got.Iteration != 2 || got.Meta.AgentID != "a1" {
t.Fatalf("checkpoint not persisted: %+v", got)
}
if il, _ := mem.ListInterrupted(ctx); len(il) != 1 {
t.Errorf("ListInterrupted = %d, want 1 (in-flight)", len(il))
}
// Complete clears it (no longer a recovery candidate).
if err := cp.Complete(ctx); err != nil {
t.Fatal(err)
}
if il, _ := mem.ListInterrupted(ctx); len(il) != 0 {
t.Errorf("after Complete, ListInterrupted = %d, want 0", len(il))
}
}
func TestHandleThrottle(t *testing.T) {
ctx := context.Background()
mem := NewMemory()
now := time.Now()
cp := New(mem, RunCheckpointMeta{RunID: "r"}, time.Minute, func() time.Time { return now })
cp.Save(ctx, run.RunCheckpointState{Iteration: 1})
now = now.Add(10 * time.Second) // within throttle window
cp.Save(ctx, run.RunCheckpointState{Iteration: 2})
if got, _ := mem.Load(ctx, "r"); got.Iteration != 1 {
t.Errorf("throttled save should keep iteration 1, got %d", got.Iteration)
}
now = now.Add(time.Minute) // past throttle
cp.Save(ctx, run.RunCheckpointState{Iteration: 3})
if got, _ := mem.Load(ctx, "r"); got.Iteration != 3 {
t.Errorf("post-throttle save should land iteration 3, got %d", got.Iteration)
}
}
func TestNilStoreNoop(t *testing.T) {
cp := New(nil, RunCheckpointMeta{RunID: "r"}, 0, nil)
if err := cp.Save(context.Background(), run.RunCheckpointState{}); err != nil {
t.Errorf("nil-store Save should be a no-op, got %v", err)
}
if err := cp.Complete(context.Background()); err != nil {
t.Error(err)
}
}
+121
View File
@@ -0,0 +1,121 @@
package checkpoint
import (
"context"
"sync"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/run"
)
// handle is a per-run run.Checkpointer bound to one run's id + meta. Save writes
// a fresh snapshot (throttled), Complete/Fail delete the checkpoint (a cleanly
// finished or terminally failed run is NOT a recovery candidate). A run
// interrupted by shutdown never calls Complete/Fail, so its checkpoint survives
// for ListInterrupted at boot.
type handle struct {
store CheckpointStore
meta RunCheckpointMeta
throttle time.Duration
now func() time.Time
mu sync.Mutex
lastSave time.Time
}
var _ run.Checkpointer = (*handle)(nil)
// New returns a run.Checkpointer that persists snapshots of the run identified
// by meta.RunID to store, no more often than throttle (Save calls inside the
// window are skipped). A nil store yields a no-op Checkpointer. throttle <= 0
// saves every call; now defaults to time.Now.
func New(store CheckpointStore, meta RunCheckpointMeta, throttle time.Duration, now func() time.Time) run.Checkpointer {
if store == nil {
return noop{}
}
if now == nil {
now = time.Now
}
return &handle{store: store, meta: meta, throttle: throttle, now: now}
}
func (h *handle) Save(ctx context.Context, st run.RunCheckpointState) error {
h.mu.Lock()
now := h.now()
if h.throttle > 0 && !h.lastSave.IsZero() && now.Sub(h.lastSave) < h.throttle {
h.mu.Unlock()
return nil // throttled — a more recent snapshot will land shortly
}
h.mu.Unlock()
// Advance the throttle clock only AFTER a successful persist. If the store
// write fails, lastSave stays put so the next Save isn't throttled away —
// otherwise a transient store error would silently drop the snapshot the
// caller believes was saved. (A run drives one Save goroutine, so the brief
// unguarded window here can't double-write.)
if err := h.store.Save(ctx, RunCheckpoint{
Meta: h.meta,
Messages: st.Messages,
Iteration: st.Iteration,
CompletedPhases: st.CompletedPhases,
UpdatedAt: now,
}); err != nil {
return err
}
h.mu.Lock()
if now.After(h.lastSave) {
h.lastSave = now
}
h.mu.Unlock()
return nil
}
func (h *handle) Complete(ctx context.Context) error { return h.store.Delete(ctx, h.meta.RunID) }
func (h *handle) Fail(ctx context.Context, _ error) error { return h.store.Delete(ctx, h.meta.RunID) }
// noop is the nil-store Checkpointer: every method is a successful no-op.
type noop struct{}
var _ run.Checkpointer = noop{}
func (noop) Save(context.Context, run.RunCheckpointState) error { return nil }
func (noop) Complete(context.Context) error { return nil }
func (noop) Fail(context.Context, error) error { return nil }
// factory is a run.CheckpointerFactory that mints a per-run handle over store,
// deriving the per-run meta from the kernel's RunInfo. It is the battery's glue
// for the Ports.Checkpointer (factory) seam: every run becomes durable (the
// store persists snapshots; a host wanting lazy/short-run skipping uses its own
// factory, as mort does over its durable-job table).
type factory struct {
store CheckpointStore
throttle time.Duration
}
var _ run.CheckpointerFactory = (*factory)(nil)
// NewFactory returns a run.CheckpointerFactory backed by store: each run gets a
// per-run Checkpointer (throttled to at most once per throttle). A nil store
// yields factory.Begin returning a no-op Checkpointer.
func NewFactory(store CheckpointStore, throttle time.Duration) run.CheckpointerFactory {
return &factory{store: store, throttle: throttle}
}
// Begin mints the per-run Checkpointer. The prompt is read from
// info.Inputs["prompt"] when present so a recovered run can re-dispatch.
func (f *factory) Begin(_ context.Context, info run.RunInfo) (run.Checkpointer, error) {
prompt, _ := info.Inputs["prompt"].(string)
meta := RunCheckpointMeta{
RunID: info.RunID,
AgentID: info.SubjectID,
AgentName: info.Name,
CallerID: info.CallerID,
ChannelID: info.ChannelID,
GuildID: info.GuildID,
Prompt: prompt,
ModelTier: info.ModelTier,
ParentRunID: info.ParentRunID,
}
return New(f.store, meta, f.throttle, nil /* now defaults to time.Now */), nil
}
+55
View File
@@ -0,0 +1,55 @@
package checkpoint
import (
"context"
"sync"
)
// Memory is a zero-dependency in-process CheckpointStore. NOTE: an in-memory
// checkpoint store does NOT survive the process restart it exists to recover
// from — it is the test/light-host default and makes ListInterrupted meaningful
// only within a single process lifetime. A host that wants real
// crash-recovery wires a durable CheckpointStore (mort's durable-job table).
type Memory struct {
mu sync.RWMutex
cps map[string]RunCheckpoint // by run id
}
// NewMemory returns an empty in-memory CheckpointStore.
func NewMemory() *Memory { return &Memory{cps: map[string]RunCheckpoint{}} }
var _ CheckpointStore = (*Memory)(nil)
func (m *Memory) Save(_ context.Context, cp RunCheckpoint) error {
m.mu.Lock()
defer m.mu.Unlock()
m.cps[cp.Meta.RunID] = cp
return nil
}
func (m *Memory) Load(_ context.Context, runID string) (*RunCheckpoint, error) {
m.mu.RLock()
defer m.mu.RUnlock()
cp, ok := m.cps[runID]
if !ok {
return nil, nil // no checkpoint (not an error — the run finished cleanly or never started)
}
return &cp, nil
}
func (m *Memory) Delete(_ context.Context, runID string) error {
m.mu.Lock()
defer m.mu.Unlock()
delete(m.cps, runID)
return nil
}
func (m *Memory) ListInterrupted(_ context.Context) ([]RunCheckpoint, error) {
m.mu.RLock()
defer m.mu.RUnlock()
out := make([]RunCheckpoint, 0, len(m.cps))
for _, cp := range m.cps {
out = append(out, cp)
}
return out, nil
}
+369
View File
@@ -0,0 +1,369 @@
// V15.2 context compactor (re-based on majordomo).
//
// Why: the agent loop accumulates tool results indefinitely. A
// research-heavy run with many web_search / read_page / http_get
// results easily crosses 200K tokens and trips the model's HTTP-400
// "prompt too long" rejection mid-run (observed at 410K tokens on
// qwen3-coder:480b which has a 262K cap). majordomo's agent loop calls
// the compactor with the full message slice before every model call;
// the compactor returns a shorter slice that preserves the system
// prompt + recent messages, with the middle range replaced by a
// synthetic summary.
//
// Strategy (unchanged from the agentkit era):
// - Keep any leading system message verbatim. (Under majordomo the
// system prompt normally travels in Request.System, not in the
// message slice, so this is defensive.)
// - Keep the last KeepRecent messages verbatim. This ensures the
// agent has fresh tool state to act on; compacting too aggressively
// would strip the in-flight context it needs to make the next
// decision.
// - Compress the middle range via a single fast-tier LLM call that
// receives the middle messages as raw text and produces a paragraph
// summary (URLs visited, key findings, file_ids created, what the
// agent is trying to accomplish).
// - Replace the middle range with one synthetic user-role message
// containing the summary. (user-role chosen because tool-result-role
// would be ambiguous without a matching tool_call_id.)
//
// What moved in the majordomo conversion:
// - The token-threshold gate lives HERE now. agentkit estimated
// tokens and only invoked the compactor past a configured
// threshold; majordomo's hook fires before every model call, so
// the threshold check (estimateTokens vs the per-run threshold the
// executor computes from the model's context limit) is the
// compactor's first step.
// - The compactor is per-run STATEFUL: majordomo does not replace
// the loop's internal transcript with the compacted slice (the
// hook shapes only what is SENT), so without memory the middle
// would be re-summarised from scratch on every step past the
// threshold. The state remembers how far the transcript has been
// folded into the running summary and folds the previous summary
// into the next one instead of re-paying for it.
//
// Failure path: any error (LLM unavailable, malformed response, etc.)
// is returned to the agent loop, which treats compactor errors as
// non-fatal and sends the original slice — if the next model call hits
// the provider's limit, the existing HTTP-400 error path takes over.
package compact
import (
"context"
"errors"
"fmt"
"strings"
"sync"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
)
// ModelResolver resolves a tier/spec to a usable llm.Model (and an enriched
// context for usage attribution). model.ParseModelForContext satisfies it.
type ModelResolver func(ctx context.Context, tier string) (context.Context, llm.Model, error)
// Compactor is the per-run compaction hook handed to the agent loop
// (matches the signature agent.WithCompactor expects).
type Compactor = func(ctx context.Context, msgs []llm.Message) ([]llm.Message, error)
// CompactionEvent describes one fired compaction so the executor can
// log it to skill_run_logs ("compaction_fired").
type CompactionEvent struct {
// MessagesBefore/After count the messages that would have been
// sent without/with this compaction.
MessagesBefore int
MessagesAfter int
// TokensBefore/After are the estimateTokens values for the same
// two slices.
TokensBefore int
TokensAfter int
}
// CompactorFactory mints a fresh per-run Compactor bound to a token
// threshold. onFire (nil-safe) observes every compaction that actually
// fires. A non-positive threshold yields a pass-through compactor.
type CompactorFactory func(thresholdTokens int, onFire func(CompactionEvent)) Compactor
// CompactorConfig controls the v15.2 compactor's behaviour. Construct
// in mort.go from convars + the executor's ModelResolver.
type CompactorConfig struct {
// Models resolves a model spec (typically "fast" or a specific tier)
// to an llm.Model. Required; nil disables compaction.
Models ModelResolver
// SummarizerTier is the model tier used for the compression LLM call.
// Production default "fast"; an admin may set "haiku" or a specific
// model spec. Empty falls back to "fast".
SummarizerTier string
// KeepRecent is the number of trailing messages preserved verbatim.
// Default 8. Lower values compact more aggressively (the next loop
// iteration sees less recent context); higher values keep more.
KeepRecent int
// SummaryWordCap bounds the LLM-generated summary length. Default
// 200 words ≈ 800 chars ≈ 200 tokens — small enough that the
// compaction always shrinks the slice meaningfully.
SummaryWordCap int
}
// NewCompactor returns a CompactorFactory implementing the middle-range
// summarisation strategy. nil cfg.Models returns a factory of no-op
// compactors that always return the input unchanged (degrades to
// v15.1 behaviour).
func NewCompactor(cfg CompactorConfig) CompactorFactory {
if cfg.Models == nil {
return func(int, func(CompactionEvent)) Compactor {
return func(_ context.Context, msgs []llm.Message) ([]llm.Message, error) {
return msgs, nil
}
}
}
if cfg.SummarizerTier == "" {
cfg.SummarizerTier = "fast"
}
if cfg.KeepRecent <= 0 {
cfg.KeepRecent = 8
}
if cfg.SummaryWordCap <= 0 {
cfg.SummaryWordCap = 200
}
return func(threshold int, onFire func(CompactionEvent)) Compactor {
st := &compactionState{}
return func(ctx context.Context, msgs []llm.Message) ([]llm.Message, error) {
return compactIfNeeded(ctx, cfg, st, threshold, onFire, msgs)
}
}
}
// compactionState is the per-run fold memory: msgs[:prefixEnd]
// (excluding a leading system message) are represented by summaryText
// in the rendered slice. Guarded by mu for safety although the agent
// loop invokes the hook from a single goroutine.
type compactionState struct {
mu sync.Mutex
prefixEnd int
summaryText string
}
// compactIfNeeded is the workhorse: render the transcript with any
// existing fold applied, check the threshold, and fold more of the
// middle into the summary when the rendered size still exceeds it.
func compactIfNeeded(ctx context.Context, cfg CompactorConfig, st *compactionState,
threshold int, onFire func(CompactionEvent), msgs []llm.Message) ([]llm.Message, error) {
st.mu.Lock()
defer st.mu.Unlock()
rendered := renderCompacted(st, msgs)
if threshold <= 0 {
return rendered, nil
}
tokensBefore := estimateTokens(rendered)
if tokensBefore < threshold {
return rendered, nil
}
// Determine the new middle range to fold: everything between what
// is already summarised (or the optional leading system message)
// and the KeepRecent tail.
startMiddle := st.prefixEnd
if startMiddle == 0 && len(msgs) > 0 && msgs[0].Role == llm.RoleSystem {
startMiddle = 1
}
endMiddle := len(msgs) - cfg.KeepRecent
if endMiddle <= startMiddle {
// Nothing new to fold (the tail alone exceeds the threshold).
// Return the rendered slice; the model call may still succeed.
return rendered, nil
}
middle := msgs[startMiddle:endMiddle]
summary, err := summariseMiddle(ctx, cfg, st.summaryText, middle)
if err != nil {
// Non-fatal upstream: the agent loop sends the ORIGINAL slice. Return
// msgs, not `rendered` — on a second+ compaction `rendered` already
// carries a prior synthetic summary, which is not the documented
// "original slice" the loop expects on a compactor error.
return msgs, fmt.Errorf("compactor: summarise middle: %w", err)
}
st.summaryText = summary
st.prefixEnd = endMiddle
out := renderCompacted(st, msgs)
if onFire != nil {
onFire(CompactionEvent{
MessagesBefore: len(rendered),
MessagesAfter: len(out),
TokensBefore: tokensBefore,
TokensAfter: estimateTokens(out),
})
}
return out, nil
}
// renderCompacted applies the fold state to msgs: [optional system] +
// [synthetic summary] + msgs[prefixEnd:]. With no fold yet, msgs is
// returned unchanged.
func renderCompacted(st *compactionState, msgs []llm.Message) []llm.Message {
if st.prefixEnd <= 0 || st.prefixEnd > len(msgs) {
return msgs
}
tail := msgs[st.prefixEnd:]
out := make([]llm.Message, 0, len(tail)+2)
if msgs[0].Role == llm.RoleSystem {
out = append(out, msgs[0])
}
out = append(out, llm.UserText(
"[CONTEXT COMPACTED] The earlier portion of this conversation was summarised "+
"to fit the model's context window. Summary:\n\n"+st.summaryText+
"\n\nResume from the recent messages below."))
out = append(out, tail...)
return out
}
// summariseMiddle composes a "compress this transcript" prompt and
// fires one fast-tier LLM call. prevSummary (may be empty) is the
// running summary from earlier compactions; it is folded into the new
// summary so prior context is not lost.
func summariseMiddle(ctx context.Context, cfg CompactorConfig, prevSummary string, middle []llm.Message) (string, error) {
if len(middle) == 0 {
return "", errors.New("compactor: empty middle range")
}
modelCtx, model, err := cfg.Models(ctx, cfg.SummarizerTier)
if err != nil {
return "", fmt.Errorf("compactor: resolve summarizer model %q: %w", cfg.SummarizerTier, err)
}
if model == nil {
return "", errors.New("compactor: summarizer model resolved to nil")
}
if modelCtx != nil {
ctx = modelCtx
}
var prior string
if strings.TrimSpace(prevSummary) != "" {
prior = "AN EARLIER PORTION WAS ALREADY SUMMARISED AS:\n" + prevSummary +
"\n\nFold that summary into your new one — its facts must survive.\n\n"
}
transcript := renderTranscript(middle)
prompt := fmt.Sprintf(
"You are compressing an in-flight agent's conversation transcript so the agent "+
"can continue working without blowing its model context. The transcript below is "+
"a sequence of tool calls and their results. Produce a single paragraph (under %d words) "+
"that captures:\n"+
" - WHAT the agent has been trying to accomplish.\n"+
" - WHICH URLs were visited / fetched (list inline, comma-separated).\n"+
" - KEY findings or decisions (factual results the agent will need later).\n"+
" - ANY file_ids or KV keys the agent created — these are persistent state references the agent must keep.\n"+
" - ANY errors or dead-ends that the agent should not re-try.\n"+
"DO NOT include verbose HTTP headers, tool-call metadata, error stack traces, or repetitive content. "+
"DO NOT add commentary or markdown headers. Output prose only.\n\n"+
"%sTRANSCRIPT TO COMPRESS:\n%s",
cfg.SummaryWordCap,
prior,
transcript,
)
resp, err := model.Generate(ctx, llm.Request{Messages: []llm.Message{llm.UserText(prompt)}})
if err != nil {
return "", fmt.Errorf("compactor: summarise LLM call: %w", err)
}
text := strings.TrimSpace(resp.Text())
if text == "" {
return "", errors.New("compactor: summarizer returned empty text")
}
return text, nil
}
// estimateTokens is the chars/4 heuristic over a message slice's text,
// tool calls, and tool results. Images count a flat ~1K tokens each.
// It intentionally matches the coarse estimator the old agentkit loop
// used — the 0.7 threshold ratio provides the safety margin.
func estimateTokens(msgs []llm.Message) int {
chars := 0
for _, m := range msgs {
for _, p := range m.Parts {
switch v := p.(type) {
case llm.TextPart:
chars += len(v.Text)
case llm.ImagePart:
chars += 4096
default:
// llm.Part is a sealed-but-extensible interface (future media
// kinds). Count an unknown part conservatively (like an image)
// rather than 0, so a transcript of unrecognised content can't
// silently slip under the compaction threshold and 400 the
// model. Bump this if a large new part kind lands.
_ = v
chars += 4096
}
}
for _, tc := range m.ToolCalls {
chars += len(tc.Name) + len(tc.Arguments)
}
for _, tr := range m.ToolResults {
chars += len(tr.Content)
}
}
return chars / 4
}
// transcriptMessageCap bounds individual message bodies at ~2KB so a
// single ultra-long tool result can't dominate the prompt sent to the
// summarizer.
const transcriptMessageCap = 2048
// secretBearingTools name tools whose ARGUMENTS routinely carry credentials or
// message bodies (bearer tokens, API keys, recipients, request bodies). Their
// args are dropped before the transcript reaches the summarizer model — which
// may be a different provider/tier than the run model — mirroring the redaction
// run/steps.go applies to user-facing step summaries. http_* and webhook_* are
// matched by prefix below.
var secretBearingTools = map[string]bool{
"mcp_call": true,
"email_send": true,
}
// redactToolArgs returns a summariser-safe rendering of a tool call's args:
// "[redacted]" for known secret-bearing tools, the args verbatim otherwise.
func redactToolArgs(name, args string) string {
if secretBearingTools[name] ||
strings.HasPrefix(name, "http_") ||
strings.HasPrefix(name, "webhook_") {
return "[redacted]"
}
return args
}
// renderTranscript flattens a message slice to a plain-text transcript
// suitable for the summarisation prompt. Tool calls show name + (redacted) args,
// tool results show name + body. Empty fields are skipped.
//
// NOTE: tool-RESULT bodies are forwarded to the summarizer (the summary needs
// the findings). A host whose tool results may contain secrets and whose
// summarizer tier resolves to an untrusted provider should ensure that tier is
// trusted, or pre-sanitise results before they reach the agent loop.
func renderTranscript(msgs []llm.Message) string {
var sb strings.Builder
for i, m := range msgs {
fmt.Fprintf(&sb, "---\n[%d] role=%s\n", i+1, m.Role)
if text := m.Text(); text != "" {
sb.WriteString(truncate(text, transcriptMessageCap))
sb.WriteString("\n")
}
for _, tc := range m.ToolCalls {
fmt.Fprintf(&sb, "tool_call name=%s args=%s\n", tc.Name, truncate(redactToolArgs(tc.Name, string(tc.Arguments)), transcriptMessageCap))
}
for _, tr := range m.ToolResults {
fmt.Fprintf(&sb, "tool_result name=%s body=%s\n", tr.Name, truncate(tr.Content, transcriptMessageCap))
}
}
return sb.String()
}
func truncate(s string, n int) string {
if len(s) <= n {
return s
}
return s[:n] + "...(truncated)"
}
+356
View File
@@ -0,0 +1,356 @@
package store
import (
"context"
"database/sql"
"encoding/json"
"errors"
"fmt"
"strings"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/audit"
)
// auditStore is the SQLite-backed audit.Storage: one row per run (+ a JSON
// `inputs` blob), one row per log event. The run-list/filter/walk queries are
// indexed on the columns they filter; the log payload is a JSON blob.
type auditStore struct{ db *sql.DB }
// Audit returns a durable audit.Storage backed by this database.
func (d *DB) Audit() audit.Storage { return &auditStore{db: d.sql} }
var _ audit.Storage = (*auditStore)(nil)
func (s *auditStore) Initialize(ctx context.Context) error {
_, err := s.db.ExecContext(ctx, `
CREATE TABLE IF NOT EXISTS skill_runs (
id TEXT PRIMARY KEY,
skill_id TEXT NOT NULL DEFAULT '',
caller_id TEXT NOT NULL DEFAULT '',
channel_id TEXT NOT NULL DEFAULT '',
parent_run_id TEXT NOT NULL DEFAULT '',
inputs TEXT NOT NULL DEFAULT '{}',
started_at INTEGER NOT NULL DEFAULT 0,
finished_at INTEGER NOT NULL DEFAULT 0, -- 0 = still running
status TEXT NOT NULL DEFAULT 'running',
output TEXT NOT NULL DEFAULT '',
error TEXT NOT NULL DEFAULT '',
tool_calls INTEGER NOT NULL DEFAULT 0,
runtime_seconds REAL NOT NULL DEFAULT 0,
total_input_tokens INTEGER NOT NULL DEFAULT 0,
total_output_tokens INTEGER NOT NULL DEFAULT 0,
total_thinking_tokens INTEGER NOT NULL DEFAULT 0
);
CREATE INDEX IF NOT EXISTS idx_runs_skill ON skill_runs(skill_id, started_at);
CREATE INDEX IF NOT EXISTS idx_runs_caller ON skill_runs(caller_id, started_at);
CREATE INDEX IF NOT EXISTS idx_runs_parent ON skill_runs(parent_run_id);
CREATE INDEX IF NOT EXISTS idx_runs_started ON skill_runs(started_at);
CREATE TABLE IF NOT EXISTS skill_run_logs (
run_id TEXT NOT NULL,
seq INTEGER NOT NULL,
event_type TEXT NOT NULL,
payload TEXT NOT NULL DEFAULT '{}',
created_at INTEGER NOT NULL,
PRIMARY KEY (run_id, seq)
);`)
if err != nil {
return fmt.Errorf("auditStore.Initialize: %w", err)
}
return nil
}
func unixOrZero(t time.Time) int64 {
if t.IsZero() {
return 0
}
return t.Unix()
}
func (s *auditStore) StartRun(ctx context.Context, r audit.SkillRun) error {
inputs, _ := json.Marshal(r.Inputs)
var fin int64
if r.FinishedAt != nil {
fin = unixOrZero(*r.FinishedAt)
}
status := r.Status
if status == "" {
status = "running"
}
_, err := s.db.ExecContext(ctx, `
INSERT INTO skill_runs (id, skill_id, caller_id, channel_id, parent_run_id, inputs, started_at, finished_at,
status, output, error, tool_calls, runtime_seconds, total_input_tokens, total_output_tokens, total_thinking_tokens)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(id) DO UPDATE SET
skill_id=excluded.skill_id, caller_id=excluded.caller_id, channel_id=excluded.channel_id,
parent_run_id=excluded.parent_run_id, inputs=excluded.inputs, started_at=excluded.started_at`,
r.ID, r.SkillID, r.CallerID, r.ChannelID, r.ParentRunID, string(inputs), unixOrZero(r.StartedAt), fin,
status, r.Output, r.Error, r.ToolCallsCount, r.RuntimeSeconds,
r.TotalInputTokens, r.TotalOutputTokens, r.TotalThinkingTokens)
if err != nil {
return fmt.Errorf("auditStore.StartRun: %w", err)
}
return nil
}
func (s *auditStore) FinishRun(ctx context.Context, runID string, st audit.RunStats) error {
res, err := s.db.ExecContext(ctx, `
UPDATE skill_runs SET finished_at=?, status=?, output=?, error=?, tool_calls=?, runtime_seconds=?,
total_input_tokens=?, total_output_tokens=?, total_thinking_tokens=? WHERE id=?`,
time.Now().Unix(), st.Status, st.Output, st.Error, st.ToolCalls, st.RuntimeSeconds,
st.InputTokens, st.OutputTokens, st.ThinkingTokens, runID)
if err != nil {
return fmt.Errorf("auditStore.FinishRun: %w", err)
}
if n, _ := res.RowsAffected(); n == 0 {
return audit.ErrNotFound
}
return nil
}
func (s *auditStore) AppendLog(ctx context.Context, l audit.SkillRunLog) error {
payload, _ := json.Marshal(l.Payload)
created := unixOrZero(l.CreatedAt)
if created == 0 {
created = time.Now().Unix()
}
_, err := s.db.ExecContext(ctx,
`INSERT OR REPLACE INTO skill_run_logs (run_id, seq, event_type, payload, created_at) VALUES (?, ?, ?, ?, ?)`,
l.RunID, l.Sequence, l.EventType, string(payload), created)
if err != nil {
return fmt.Errorf("auditStore.AppendLog: %w", err)
}
return nil
}
// runCols is the SELECT column list matching scanRun.
const runCols = `id, skill_id, caller_id, channel_id, parent_run_id, inputs, started_at, finished_at,
status, output, error, tool_calls, runtime_seconds, total_input_tokens, total_output_tokens, total_thinking_tokens`
func scanRun(sc interface{ Scan(...any) error }) (*audit.SkillRun, error) {
var r audit.SkillRun
var inputs string
var started, finished int64
if err := sc.Scan(&r.ID, &r.SkillID, &r.CallerID, &r.ChannelID, &r.ParentRunID, &inputs,
&started, &finished, &r.Status, &r.Output, &r.Error, &r.ToolCallsCount, &r.RuntimeSeconds,
&r.TotalInputTokens, &r.TotalOutputTokens, &r.TotalThinkingTokens); err != nil {
return nil, err
}
_ = json.Unmarshal([]byte(inputs), &r.Inputs)
r.StartedAt = time.Unix(started, 0).UTC()
if finished > 0 {
t := time.Unix(finished, 0).UTC()
r.FinishedAt = &t
}
return &r, nil
}
func (s *auditStore) GetRun(ctx context.Context, runID string) (*audit.SkillRun, error) {
row := s.db.QueryRowContext(ctx, `SELECT `+runCols+` FROM skill_runs WHERE id = ?`, runID)
r, err := scanRun(row)
if errors.Is(err, sql.ErrNoRows) {
return nil, audit.ErrNotFound
}
return r, err
}
func (s *auditStore) queryRuns(ctx context.Context, tail string, args ...any) ([]audit.SkillRun, error) {
rows, err := s.db.QueryContext(ctx, `SELECT `+runCols+` FROM skill_runs `+tail, args...)
if err != nil {
return nil, err
}
defer rows.Close()
var out []audit.SkillRun
for rows.Next() {
r, err := scanRun(rows)
if err != nil {
return nil, err
}
out = append(out, *r)
}
return out, rows.Err()
}
func (s *auditStore) ListLogsByRun(ctx context.Context, runID string) ([]audit.SkillRunLog, error) {
rows, err := s.db.QueryContext(ctx,
`SELECT run_id, seq, event_type, payload, created_at FROM skill_run_logs WHERE run_id = ? ORDER BY seq`, runID)
if err != nil {
return nil, fmt.Errorf("auditStore.ListLogsByRun: %w", err)
}
defer rows.Close()
var out []audit.SkillRunLog
for rows.Next() {
var l audit.SkillRunLog
var payload string
var created int64
if err := rows.Scan(&l.RunID, &l.Sequence, &l.EventType, &payload, &created); err != nil {
return nil, err
}
_ = json.Unmarshal([]byte(payload), &l.Payload)
l.CreatedAt = time.Unix(created, 0).UTC()
out = append(out, l)
}
return out, rows.Err()
}
func (s *auditStore) ListRunsBySkill(ctx context.Context, skillID string, limit int) ([]audit.SkillRun, error) {
return s.ListRunsBySkillPaginated(ctx, skillID, 0, limit, false)
}
func (s *auditStore) ListRunsBySkillPaginated(ctx context.Context, skillID string, offset, limit int, includeDryRun bool) ([]audit.SkillRun, error) {
w := `WHERE skill_id = ?`
args := []any{skillID}
if !includeDryRun {
w += ` AND status != 'dry_run'`
}
return s.queryRuns(ctx, w+` ORDER BY started_at DESC `+limitOffset(limit, offset), args...)
}
func (s *auditStore) CountRunsBySkill(ctx context.Context, skillID string, includeDryRun bool) (int64, error) {
q := `SELECT COUNT(*) FROM skill_runs WHERE skill_id = ?`
if !includeDryRun {
q += ` AND status != 'dry_run'`
}
var n int64
err := s.db.QueryRowContext(ctx, q, skillID).Scan(&n)
return n, err
}
func (s *auditStore) ListRunsByCaller(ctx context.Context, callerID string, limit int) ([]audit.SkillRun, error) {
return s.queryRuns(ctx, `WHERE caller_id = ? AND status != 'dry_run' ORDER BY started_at DESC `+limitOffset(limit, 0), callerID)
}
func (s *auditStore) buildFilter(f audit.RunFilter) (string, []any) {
var conds []string
var args []any
if !f.IncludeDryRun {
conds = append(conds, `status != 'dry_run'`)
}
if f.Status != "" {
conds = append(conds, `status = ?`)
args = append(args, f.Status)
}
if f.SkillID != "" {
conds = append(conds, `skill_id = ?`)
args = append(args, f.SkillID)
}
if f.CallerID != "" {
conds = append(conds, `caller_id = ?`)
args = append(args, f.CallerID)
}
if f.ChannelID != "" {
conds = append(conds, `channel_id = ?`)
args = append(args, f.ChannelID)
}
if f.TopLevelOnly {
conds = append(conds, `parent_run_id = ''`)
}
if !f.Since.IsZero() {
conds = append(conds, `started_at >= ?`)
args = append(args, f.Since.Unix())
}
if !f.Until.IsZero() {
conds = append(conds, `started_at <= ?`)
args = append(args, f.Until.Unix())
}
where := ""
if len(conds) > 0 {
where = `WHERE ` + strings.Join(conds, " AND ")
}
return where, args
}
func (s *auditStore) ListRunsFiltered(ctx context.Context, f audit.RunFilter, offset, limit int) ([]audit.SkillRun, error) {
where, args := s.buildFilter(f)
return s.queryRuns(ctx, where+` ORDER BY started_at DESC `+limitOffset(limit, offset), args...)
}
func (s *auditStore) CountRunsFiltered(ctx context.Context, f audit.RunFilter) (int64, error) {
where, args := s.buildFilter(f)
var n int64
err := s.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM skill_runs `+where, args...).Scan(&n)
return n, err
}
func (s *auditStore) PurgeOlderThan(ctx context.Context, t time.Time) (int64, error) {
res, err := s.db.ExecContext(ctx, `DELETE FROM skill_runs WHERE finished_at > 0 AND finished_at < ?`, t.Unix())
if err != nil {
return 0, fmt.Errorf("auditStore.PurgeOlderThan: %w", err)
}
n, _ := res.RowsAffected()
// Best-effort orphan-log cleanup.
_, _ = s.db.ExecContext(ctx, `DELETE FROM skill_run_logs WHERE run_id NOT IN (SELECT id FROM skill_runs)`)
return n, nil
}
func (s *auditStore) ListChildrenByParent(ctx context.Context, parentRunID string) ([]audit.SkillRun, error) {
return s.queryRuns(ctx, `WHERE parent_run_id = ? ORDER BY started_at DESC`, parentRunID)
}
func (s *auditStore) WalkParentChain(ctx context.Context, runID string) ([]audit.SkillRun, error) {
var chain []audit.SkillRun
seen := map[string]bool{}
for id := runID; id != ""; {
if seen[id] {
break
}
seen[id] = true
r, err := s.GetRun(ctx, id)
if errors.Is(err, audit.ErrNotFound) {
break
}
if err != nil {
return nil, err
}
chain = append(chain, *r)
id = r.ParentRunID
}
return chain, nil
}
func (s *auditStore) ListFinishedRunsBefore(ctx context.Context, cutoff time.Time, limit int) ([]audit.SkillRun, error) {
return s.queryRuns(ctx,
`WHERE finished_at > 0 AND finished_at < ? ORDER BY started_at DESC `+limitOffset(limit, 0), cutoff.Unix())
}
func (s *auditStore) LastRunBySkills(ctx context.Context, skillIDs []string, includeFailed bool) (map[string]time.Time, error) {
out := map[string]time.Time{}
if len(skillIDs) == 0 {
return out, nil
}
q := `SELECT skill_id, MAX(started_at) FROM skill_runs WHERE skill_id IN (` +
strings.TrimSuffix(strings.Repeat("?,", len(skillIDs)), ",") + `)`
args := make([]any, 0, len(skillIDs))
for _, id := range skillIDs {
args = append(args, id)
}
if !includeFailed {
q += ` AND status NOT IN ('error','timeout')`
}
q += ` GROUP BY skill_id`
rows, err := s.db.QueryContext(ctx, q, args...)
if err != nil {
return nil, fmt.Errorf("auditStore.LastRunBySkills: %w", err)
}
defer rows.Close()
for rows.Next() {
var id string
var ts int64
if err := rows.Scan(&id, &ts); err != nil {
return nil, err
}
out[id] = time.Unix(ts, 0).UTC()
}
return out, rows.Err()
}
// limitOffset renders an optional LIMIT/OFFSET clause (limit<=0 = no limit).
func limitOffset(limit, offset int) string {
if limit <= 0 {
return ""
}
if offset > 0 {
return fmt.Sprintf("LIMIT %d OFFSET %d", limit, offset)
}
return fmt.Sprintf("LIMIT %d", limit)
}
+67
View File
@@ -0,0 +1,67 @@
package store
import (
"context"
"testing"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/audit"
)
func TestSQLiteAuditStore(t *testing.T) {
ctx := context.Background()
db, err := Open(":memory:")
if err != nil {
t.Fatal(err)
}
defer db.Close()
st := db.Audit()
if err := st.Initialize(ctx); err != nil {
t.Fatal(err)
}
now := time.Now().UTC()
// parent run
if err := st.StartRun(ctx, audit.SkillRun{ID: "r1", SkillID: "agent-x", CallerID: "c1",
Inputs: map[string]any{"q": "hi"}, StartedAt: now}); err != nil {
t.Fatal(err)
}
// child run
st.StartRun(ctx, audit.SkillRun{ID: "r2", SkillID: "skill-y", CallerID: "c1", ParentRunID: "r1", StartedAt: now.Add(time.Second)})
st.AppendLog(ctx, audit.SkillRunLog{RunID: "r1", Sequence: 1, EventType: "step", Payload: map[string]any{"i": 1}, CreatedAt: now})
if err := st.FinishRun(ctx, "r1", audit.RunStats{Status: "ok", Output: "done", ToolCalls: 2, InputTokens: 10, OutputTokens: 5}); err != nil {
t.Fatal(err)
}
got, err := st.GetRun(ctx, "r1")
if err != nil || got.Status != "ok" || got.Output != "done" || got.FinishedAt == nil ||
got.Inputs["q"] != "hi" || got.TotalInputTokens != 10 {
t.Fatalf("GetRun: %v %+v", err, got)
}
if logs, _ := st.ListLogsByRun(ctx, "r1"); len(logs) != 1 || logs[0].EventType != "step" {
t.Errorf("ListLogsByRun = %+v", logs)
}
if kids, _ := st.ListChildrenByParent(ctx, "r1"); len(kids) != 1 || kids[0].ID != "r2" {
t.Errorf("ListChildrenByParent = %+v", kids)
}
if chain, _ := st.WalkParentChain(ctx, "r2"); len(chain) != 2 || chain[1].ID != "r1" {
t.Errorf("WalkParentChain = %+v", chain)
}
if byCaller, _ := st.ListRunsByCaller(ctx, "c1", 10); len(byCaller) != 2 {
t.Errorf("ListRunsByCaller = %d, want 2", len(byCaller))
}
// filter: top-level only
tl, _ := st.ListRunsFiltered(ctx, audit.RunFilter{TopLevelOnly: true}, 0, 10)
if len(tl) != 1 || tl[0].ID != "r1" {
t.Errorf("TopLevelOnly filter = %+v", tl)
}
// last-run map
last, _ := st.LastRunBySkills(ctx, []string{"agent-x", "skill-y"}, true)
if _, ok := last["agent-x"]; !ok {
t.Errorf("LastRunBySkills missing agent-x: %+v", last)
}
if n, _ := st.CountRunsBySkill(ctx, "agent-x", false); n != 1 {
t.Errorf("CountRunsBySkill = %d, want 1", n)
}
}
+105
View File
@@ -0,0 +1,105 @@
package store
import (
"context"
"database/sql"
"errors"
"fmt"
"math"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/budget"
)
// budgetStore is the SQLite-backed budget.BudgetStorage.
type budgetStore struct{ db *sql.DB }
// Budget returns a durable budget.BudgetStorage backed by this database.
func (d *DB) Budget() budget.BudgetStorage { return &budgetStore{db: d.sql} }
var _ budget.BudgetStorage = (*budgetStore)(nil)
func (s *budgetStore) Initialize(ctx context.Context) error {
_, err := s.db.ExecContext(ctx, `
CREATE TABLE IF NOT EXISTS skill_budgets (
user_id TEXT PRIMARY KEY,
window_start INTEGER NOT NULL, -- unix seconds
seconds_used REAL NOT NULL,
runs_count INTEGER NOT NULL,
updated_at INTEGER NOT NULL
)`)
if err != nil {
return fmt.Errorf("budgetStore.Initialize: %w", err)
}
return nil
}
func (s *budgetStore) Get(ctx context.Context, userID string) (*budget.SkillBudget, error) {
row := s.db.QueryRowContext(ctx,
`SELECT window_start, seconds_used, runs_count, updated_at FROM skill_budgets WHERE user_id = ?`, userID)
var ws, ua int64
var used float64
var runs int
switch err := row.Scan(&ws, &used, &runs, &ua); {
case errors.Is(err, sql.ErrNoRows):
return nil, nil // no row yet — documented (nil, nil)
case err != nil:
return nil, fmt.Errorf("budgetStore.Get: %w", err)
}
return &budget.SkillBudget{
UserID: userID,
WindowStart: time.Unix(ws, 0).UTC(),
SecondsUsed: used,
RunsCount: runs,
UpdatedAt: time.Unix(ua, 0).UTC(),
}, nil
}
// Add increments usage atomically, rolling the 7-day window over inside one
// transaction so concurrent Adds can't race the read-modify-write.
func (s *budgetStore) Add(ctx context.Context, userID string, secondsUsed float64, now time.Time) error {
// A NaN/Inf would poison the seconds_used column irrecoverably (NaN
// propagates through every later add), so reject it at the boundary.
if math.IsNaN(secondsUsed) || math.IsInf(secondsUsed, 0) {
return fmt.Errorf("budgetStore.Add: invalid secondsUsed %v", secondsUsed)
}
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return fmt.Errorf("budgetStore.Add: begin: %w", err)
}
defer tx.Rollback() //nolint:errcheck // no-op after Commit
var ws int64
var used float64
var runs int
err = tx.QueryRowContext(ctx,
`SELECT window_start, seconds_used, runs_count FROM skill_budgets WHERE user_id = ?`, userID).
Scan(&ws, &used, &runs)
switch {
case errors.Is(err, sql.ErrNoRows):
ws, used, runs = now.Unix(), 0, 0
case err != nil:
return fmt.Errorf("budgetStore.Add: select: %w", err)
}
// Roll the window over if older than 7 days.
if now.Sub(time.Unix(ws, 0)) >= 7*24*time.Hour {
ws, used, runs = now.Unix(), 0, 0
}
used += secondsUsed
runs++
if _, err := tx.ExecContext(ctx, `
INSERT INTO skill_budgets (user_id, window_start, seconds_used, runs_count, updated_at)
VALUES (?, ?, ?, ?, ?)
ON CONFLICT(user_id) DO UPDATE SET
window_start = excluded.window_start,
seconds_used = excluded.seconds_used,
runs_count = excluded.runs_count,
updated_at = excluded.updated_at`,
userID, ws, used, runs, now.Unix()); err != nil {
return fmt.Errorf("budgetStore.Add: upsert: %w", err)
}
if err := tx.Commit(); err != nil {
return fmt.Errorf("budgetStore.Add: commit: %w", err)
}
return nil
}
+65
View File
@@ -0,0 +1,65 @@
package store
import (
"context"
"errors"
"testing"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/budget"
)
// TestSQLiteBudgetConformance runs the budget battery over the SQLite store and
// asserts the same rolling-window contract the in-memory store must satisfy.
func TestSQLiteBudgetConformance(t *testing.T) {
ctx := context.Background()
db, err := Open(":memory:")
if err != nil {
t.Fatal(err)
}
defer db.Close()
st := db.Budget()
if err := st.Initialize(ctx); err != nil {
t.Fatal(err)
}
now := time.Now().UTC()
b := budget.NewDBBudget(st, func() float64 { return 100 }, nil, func() time.Time { return now })
if err := b.Check(ctx, "u"); err != nil {
t.Fatalf("fresh caller should pass: %v", err)
}
b.Commit(ctx, "u", 60)
if err := b.Check(ctx, "u"); err != nil {
t.Fatalf("60/100 should pass: %v", err)
}
b.Commit(ctx, "u", 50) // 110 total
if err := b.Check(ctx, "u"); !errors.Is(err, budget.ErrBudgetExceeded) {
t.Fatalf("110/100 should be ErrBudgetExceeded, got %v", err)
}
// Direct Get reflects the persisted row.
row, err := st.Get(ctx, "u")
if err != nil || row == nil {
t.Fatalf("Get: %v %+v", err, row)
}
if row.SecondsUsed != 110 || row.RunsCount != 2 {
t.Errorf("row = %+v, want seconds=110 runs=2", row)
}
// Window rolls over after 7 days.
now = now.Add(8 * 24 * time.Hour)
b.Commit(ctx, "u", 1)
if err := b.Check(ctx, "u"); err != nil {
t.Fatalf("after rollover should pass: %v", err)
}
row, _ = st.Get(ctx, "u")
if row.SecondsUsed != 1 || row.RunsCount != 1 {
t.Errorf("post-rollover row = %+v, want seconds=1 runs=1", row)
}
// Unknown user -> (nil, nil).
if r, err := st.Get(ctx, "nobody"); err != nil || r != nil {
t.Errorf("Get(unknown) = %+v %v, want nil,nil", r, err)
}
}
+54
View File
@@ -0,0 +1,54 @@
module gitea.stevedudenhoeffer.com/steve/executus/contrib/store
go 1.26.2
require (
gitea.stevedudenhoeffer.com/steve/executus v0.0.0
modernc.org/sqlite v1.34.4
)
require (
cloud.google.com/go v0.123.0 // indirect
cloud.google.com/go/auth v0.18.1 // indirect
cloud.google.com/go/compute/metadata v0.9.0 // indirect
gitea.stevedudenhoeffer.com/steve/majordomo v0.0.0-20260626223738-1fd7109a42f3 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/google/go-cmp v0.7.0 // indirect
github.com/google/s2a-go v0.1.9 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.11 // indirect
github.com/googleapis/gax-go/v2 v2.17.0 // indirect
github.com/gorilla/websocket v1.5.3 // indirect
github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/ncruces/go-strftime v0.1.9 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
github.com/robfig/cron/v3 v3.0.1 // indirect
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.65.0 // indirect
go.opentelemetry.io/otel v1.40.0 // indirect
go.opentelemetry.io/otel/metric v1.40.0 // indirect
go.opentelemetry.io/otel/trace v1.40.0 // indirect
golang.org/x/crypto v0.53.0 // indirect
golang.org/x/net v0.55.0 // indirect
golang.org/x/sys v0.46.0 // indirect
golang.org/x/text v0.38.0 // indirect
google.golang.org/genai v1.59.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20260203192932-546029d2fa20 // indirect
google.golang.org/grpc v1.78.0 // indirect
google.golang.org/protobuf v1.36.11 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6 // indirect
modernc.org/libc v1.55.3 // indirect
modernc.org/mathutil v1.6.0 // indirect
modernc.org/memory v1.8.0 // indirect
modernc.org/strutil v1.2.0 // indirect
modernc.org/token v1.1.0 // indirect
)
// Co-developed against the local checkout; dropped (pinned) at executus v0.1.0.
replace gitea.stevedudenhoeffer.com/steve/executus => ../../
+105
View File
@@ -0,0 +1,105 @@
cloud.google.com/go v0.123.0 h1:2NAUJwPR47q+E35uaJeYoNhuNEM9kM8SjgRgdeOJUSE=
cloud.google.com/go v0.123.0/go.mod h1:xBoMV08QcqUGuPW65Qfm1o9Y4zKZBpGS+7bImXLTAZU=
cloud.google.com/go/auth v0.18.1 h1:IwTEx92GFUo2pJ6Qea0EU3zYvKnTAeRCODxfA/G5UWs=
cloud.google.com/go/auth v0.18.1/go.mod h1:GfTYoS9G3CWpRA3Va9doKN9mjPGRS+v41jmZAhBzbrA=
cloud.google.com/go/compute/metadata v0.9.0 h1:pDUj4QMoPejqq20dK0Pg2N4yG9zIkYGdBtwLoEkH9Zs=
cloud.google.com/go/compute/metadata v0.9.0/go.mod h1:E0bWwX5wTnLPedCKqk3pJmVgCBSM6qQI1yTBdEb3C10=
gitea.stevedudenhoeffer.com/steve/majordomo v0.0.0-20260626223738-1fd7109a42f3 h1:KYKIFFRsXzbbBJVDa99+Fhy0zxl9G0xV/MCrLipsLL4=
gitea.stevedudenhoeffer.com/steve/majordomo v0.0.0-20260626223738-1fd7109a42f3/go.mod h1:UZLveG17SmENt4sne2RSLIbioix30RZbRIQUzBAnOyY=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=
github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/pprof v0.0.0-20240409012703-83162a5b38cd h1:gbpYu9NMq8jhDVbvlGkMFWCjLFlqqEZjEmObmhUy6Vo=
github.com/google/pprof v0.0.0-20240409012703-83162a5b38cd/go.mod h1:kf6iHlnVGwgKolg33glAes7Yg/8iWP8ukqeldJSO7jw=
github.com/google/s2a-go v0.1.9 h1:LGD7gtMgezd8a/Xak7mEWL0PjoTQFvpRudN895yqKW0=
github.com/google/s2a-go v0.1.9/go.mod h1:YA0Ei2ZQL3acow2O62kdp9UlnvMmU7kA6Eutn0dXayM=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/googleapis/enterprise-certificate-proxy v0.3.11 h1:vAe81Msw+8tKUxi2Dqh/NZMz7475yUvmRIkXr4oN2ao=
github.com/googleapis/enterprise-certificate-proxy v0.3.11/go.mod h1:RFV7MUdlb7AgEq2v7FmMCfeSMCllAzWxFgRdusoGks8=
github.com/googleapis/gax-go/v2 v2.17.0 h1:RksgfBpxqff0EZkDWYuz9q/uWsTVz+kf43LsZ1J6SMc=
github.com/googleapis/gax-go/v2 v2.17.0/go.mod h1:mzaqghpQp4JDh3HvADwrat+6M3MOIDp5YKHhb9PAgDY=
github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg=
github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/ncruces/go-strftime v0.1.9 h1:bY0MQC28UADQmHmaF5dgpLmImcShSi2kHU9XLdhx/f4=
github.com/ncruces/go-strftime v0.1.9/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
github.com/robfig/cron/v3 v3.0.1 h1:WdRxkvbJztn8LMz/QEvLN5sBU+xKpSqwwUO1Pjr4qDs=
github.com/robfig/cron/v3 v3.0.1/go.mod h1:eQICP3HwyT7UooqI/z+Ov+PtYAWygg1TEWWzGIFLtro=
go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=
go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.65.0 h1:7iP2uCb7sGddAr30RRS6xjKy7AZ2JtTOPA3oolgVSw8=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.65.0/go.mod h1:c7hN3ddxs/z6q9xwvfLPk+UHlWRQyaeR1LdgfL/66l0=
go.opentelemetry.io/otel v1.40.0 h1:oA5YeOcpRTXq6NN7frwmwFR0Cn3RhTVZvXsP4duvCms=
go.opentelemetry.io/otel v1.40.0/go.mod h1:IMb+uXZUKkMXdPddhwAHm6UfOwJyh4ct1ybIlV14J0g=
go.opentelemetry.io/otel/metric v1.40.0 h1:rcZe317KPftE2rstWIBitCdVp89A2HqjkxR3c11+p9g=
go.opentelemetry.io/otel/metric v1.40.0/go.mod h1:ib/crwQH7N3r5kfiBZQbwrTge743UDc7DTFVZrrXnqc=
go.opentelemetry.io/otel/trace v1.40.0 h1:WA4etStDttCSYuhwvEa8OP8I5EWu24lkOzp+ZYblVjw=
go.opentelemetry.io/otel/trace v1.40.0/go.mod h1:zeAhriXecNGP/s2SEG3+Y8X9ujcJOTqQ5RgdEJcawiA=
golang.org/x/crypto v0.53.0 h1:QZ4Muo8THX6CizN2vPPd5fBGHyogrdK9fG4wLPFUsto=
golang.org/x/crypto v0.53.0/go.mod h1:DNLU434OwVakk9PzuwV8w62mAJpRJL3vsgcfp4Qnsio=
golang.org/x/mod v0.16.0 h1:QX4fJ0Rr5cPQCF7O9lh9Se4pmwfwskqZfq5moyldzic=
golang.org/x/mod v0.16.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/net v0.55.0 h1:bcvxaJn3e1U6InsFWt1JUq1aSjnRxLzT2rtD2KfkDF8=
golang.org/x/net v0.55.0/go.mod h1:L5U2KuzuOe1lY7Z+aWVIKK6qEeJXnXV9yzGA+WCHJww=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.46.0 h1:noSf2Fq6F8DBgS+LysIkx7rIExoNHJsxOAtPp4rthXw=
golang.org/x/sys v0.46.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/text v0.38.0 h1:sXmwo9DwP3OK9EZ7PqAdaooSGozfl/3a6/xJcbzPRhE=
golang.org/x/text v0.38.0/go.mod h1:YXZt3QhHUKYT53r2lLKFIVi6Ao1jdzrTR/KQ09qyxF4=
golang.org/x/tools v0.19.0 h1:tfGCXNR1OsFG+sVdLAitlpjAvD/I6dHDKnYrpEZUHkw=
golang.org/x/tools v0.19.0/go.mod h1:qoJWxmGSIBmAeriMx19ogtrEPrGtDbPK634QFIcLAhc=
google.golang.org/genai v1.59.0 h1:xp+ydkJFW8hO0hTUaAkr8TrLM9HFP3NYAwFhPd0nDqA=
google.golang.org/genai v1.59.0/go.mod h1:mDdPDFXo1Ats7f1WXVyZgWb/CkMzFWTWJruIMy7hGIU=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260203192932-546029d2fa20 h1:Jr5R2J6F6qWyzINc+4AM8t5pfUz6beZpHp678GNrMbE=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260203192932-546029d2fa20/go.mod h1:j9x/tPzZkyxcgEFkiKEEGxfvyumM01BEtsW8xzOahRQ=
google.golang.org/grpc v1.78.0 h1:K1XZG/yGDJnzMdd/uZHAkVqJE+xIDOcmdSFZkBUicNc=
google.golang.org/grpc v1.78.0/go.mod h1:I47qjTo4OKbMkjA/aOOwxDIiPSBofUtQUI5EfpWvW7U=
google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
modernc.org/cc/v4 v4.21.4 h1:3Be/Rdo1fpr8GrQ7IVw9OHtplU4gWbb+wNgeoBMmGLQ=
modernc.org/cc/v4 v4.21.4/go.mod h1:HM7VJTZbUCR3rV8EYBi9wxnJ0ZBRiGE5OeGXNA0IsLQ=
modernc.org/ccgo/v4 v4.19.2 h1:lwQZgvboKD0jBwdaeVCTouxhxAyN6iawF3STraAal8Y=
modernc.org/ccgo/v4 v4.19.2/go.mod h1:ysS3mxiMV38XGRTTcgo0DQTeTmAO4oCmJl1nX9VFI3s=
modernc.org/fileutil v1.3.0 h1:gQ5SIzK3H9kdfai/5x41oQiKValumqNTDXMvKo62HvE=
modernc.org/fileutil v1.3.0/go.mod h1:XatxS8fZi3pS8/hKG2GH/ArUogfxjpEKs3Ku3aK4JyQ=
modernc.org/gc/v2 v2.4.1 h1:9cNzOqPyMJBvrUipmynX0ZohMhcxPtMccYgGOJdOiBw=
modernc.org/gc/v2 v2.4.1/go.mod h1:wzN5dK1AzVGoH6XOzc3YZ+ey/jPgYHLuVckd62P0GYU=
modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6 h1:5D53IMaUuA5InSeMu9eJtlQXS2NxAhyWQvkKEgXZhHI=
modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6/go.mod h1:Qz0X07sNOR1jWYCrJMEnbW/X55x206Q7Vt4mz6/wHp4=
modernc.org/libc v1.55.3 h1:AzcW1mhlPNrRtjS5sS+eW2ISCgSOLLNyFzRh/V3Qj/U=
modernc.org/libc v1.55.3/go.mod h1:qFXepLhz+JjFThQ4kzwzOjA/y/artDeg+pcYnY+Q83w=
modernc.org/mathutil v1.6.0 h1:fRe9+AmYlaej+64JsEEhoWuAYBkOtQiMEU7n/XgfYi4=
modernc.org/mathutil v1.6.0/go.mod h1:Ui5Q9q1TR2gFm0AQRqQUaBWFLAhQpCwNcuhBOSedWPo=
modernc.org/memory v1.8.0 h1:IqGTL6eFMaDZZhEWwcREgeMXYwmW83LYW8cROZYkg+E=
modernc.org/memory v1.8.0/go.mod h1:XPZ936zp5OMKGWPqbD3JShgd/ZoQ7899TUuQqxY+peU=
modernc.org/opt v0.1.3 h1:3XOZf2yznlhC+ibLltsDGzABUGVx8J6pnFMS3E4dcq4=
modernc.org/opt v0.1.3/go.mod h1:WdSiB5evDcignE70guQKxYUl14mgWtbClRi5wmkkTX0=
modernc.org/sortutil v1.2.0 h1:jQiD3PfS2REGJNzNCMMaLSp/wdMNieTbKX920Cqdgqc=
modernc.org/sortutil v1.2.0/go.mod h1:TKU2s7kJMf1AE84OoiGppNHJwvB753OYfNl2WRb++Ss=
modernc.org/sqlite v1.34.4 h1:sjdARozcL5KJBvYQvLlZEmctRgW9xqIZc2ncN7PU0P8=
modernc.org/sqlite v1.34.4/go.mod h1:3QQFCG2SEMtc2nv+Wq4cQCH7Hjcg+p/RMlS1XK+zwbk=
modernc.org/strutil v1.2.0 h1:agBi9dp1I+eOnxXeiZawM8F4LawKv4NzGWSaLfyeNZA=
modernc.org/strutil v1.2.0/go.mod h1:/mdcBmfOibveCTBxUl5B5l6W+TTH1FXPLHZE6bTosX0=
modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=
+174
View File
@@ -0,0 +1,174 @@
package store
import (
"context"
"database/sql"
"encoding/json"
"errors"
"fmt"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/persona"
)
// personaStore is the SQLite-backed persona.Storage. It stores each Agent as a
// JSON blob in `data` with a handful of extracted, indexed columns for the
// query methods — so the FULL struct round-trips (no domain↔GORM↔DB field-loss
// footgun) while owner/name/webhook/schedule lookups stay indexable.
type personaStore struct{ db *sql.DB }
// Personas returns a durable persona.Storage backed by this database.
func (d *DB) Personas() persona.Storage { return &personaStore{db: d.sql} }
var _ persona.Storage = (*personaStore)(nil)
func (s *personaStore) InitializeAgentStorage(ctx context.Context) error {
_, err := s.db.ExecContext(ctx, `
CREATE TABLE IF NOT EXISTS agents (
id TEXT PRIMARY KEY,
owner_id TEXT NOT NULL DEFAULT '',
name TEXT NOT NULL DEFAULT '',
webhook_secret TEXT NOT NULL DEFAULT '',
chatbot_channel_filter TEXT NOT NULL DEFAULT '',
schedule TEXT NOT NULL DEFAULT '',
next_run_at INTEGER NOT NULL DEFAULT 0, -- unix seconds; 0 = unset
data TEXT NOT NULL -- full Agent as JSON
);
CREATE INDEX IF NOT EXISTS idx_agents_owner ON agents(owner_id);
CREATE UNIQUE INDEX IF NOT EXISTS idx_agents_owner_name ON agents(owner_id, name);
CREATE INDEX IF NOT EXISTS idx_agents_sched ON agents(schedule, next_run_at);`)
if err != nil {
return fmt.Errorf("personaStore.Initialize: %w", err)
}
return nil
}
func (s *personaStore) SaveAgent(ctx context.Context, a *persona.Agent) error {
blob, err := json.Marshal(a)
if err != nil {
return fmt.Errorf("personaStore.SaveAgent: marshal: %w", err)
}
var next int64
if a.NextRunAt != nil && !a.NextRunAt.IsZero() {
next = a.NextRunAt.Unix()
}
_, err = s.db.ExecContext(ctx, `
INSERT INTO agents (id, owner_id, name, webhook_secret, chatbot_channel_filter, schedule, next_run_at, data)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(id) DO UPDATE SET
owner_id=excluded.owner_id, name=excluded.name, webhook_secret=excluded.webhook_secret,
chatbot_channel_filter=excluded.chatbot_channel_filter, schedule=excluded.schedule,
next_run_at=excluded.next_run_at, data=excluded.data`,
a.ID, a.OwnerID, a.Name, a.WebhookSecret, a.ChatbotChannelFilter, a.Schedule, next, string(blob))
if err != nil {
return fmt.Errorf("personaStore.SaveAgent: %w", err)
}
return nil
}
// scanAgents unmarshals the `data` column of every row in rows.
func scanAgents(rows *sql.Rows) ([]*persona.Agent, error) {
defer rows.Close()
var out []*persona.Agent
for rows.Next() {
var blob string
if err := rows.Scan(&blob); err != nil {
return nil, err
}
var a persona.Agent
if err := json.Unmarshal([]byte(blob), &a); err != nil {
return nil, err
}
out = append(out, &a)
}
return out, rows.Err()
}
func (s *personaStore) getOne(ctx context.Context, where string, arg ...any) (*persona.Agent, error) {
var blob string
err := s.db.QueryRowContext(ctx, `SELECT data FROM agents WHERE `+where, arg...).Scan(&blob)
switch {
case errors.Is(err, sql.ErrNoRows):
return nil, persona.ErrNotFound
case err != nil:
return nil, err
}
var a persona.Agent
if err := json.Unmarshal([]byte(blob), &a); err != nil {
return nil, err
}
return &a, nil
}
func (s *personaStore) GetAgent(ctx context.Context, id string) (*persona.Agent, error) {
return s.getOne(ctx, "id = ?", id)
}
func (s *personaStore) GetAgentByName(ctx context.Context, ownerID, name string) (*persona.Agent, error) {
return s.getOne(ctx, "owner_id = ? AND name = ?", ownerID, name)
}
func (s *personaStore) GetAgentByWebhookSecret(ctx context.Context, secret string) (*persona.Agent, error) {
if secret == "" {
return nil, persona.ErrNotFound
}
return s.getOne(ctx, "webhook_secret = ?", secret)
}
func (s *personaStore) ListAgents(ctx context.Context, ownerID string) ([]*persona.Agent, error) {
rows, err := s.db.QueryContext(ctx, `SELECT data FROM agents WHERE owner_id = ? ORDER BY name`, ownerID)
if err != nil {
return nil, fmt.Errorf("personaStore.ListAgents: %w", err)
}
return scanAgents(rows)
}
func (s *personaStore) ListAllAgents(ctx context.Context) ([]*persona.Agent, error) {
rows, err := s.db.QueryContext(ctx, `SELECT data FROM agents ORDER BY name`)
if err != nil {
return nil, fmt.Errorf("personaStore.ListAllAgents: %w", err)
}
return scanAgents(rows)
}
func (s *personaStore) DeleteAgent(ctx context.Context, id string) error {
if _, err := s.db.ExecContext(ctx, `DELETE FROM agents WHERE id = ?`, id); err != nil {
return fmt.Errorf("personaStore.DeleteAgent: %w", err)
}
return nil
}
func (s *personaStore) ListAgentsByChatbotChannelFilter(ctx context.Context) ([]*persona.Agent, error) {
rows, err := s.db.QueryContext(ctx, `SELECT data FROM agents WHERE chatbot_channel_filter != '' ORDER BY name`)
if err != nil {
return nil, fmt.Errorf("personaStore.ListAgentsByChatbotChannelFilter: %w", err)
}
return scanAgents(rows)
}
func (s *personaStore) ListScheduledAgents(ctx context.Context, dueBefore time.Time) ([]*persona.Agent, error) {
rows, err := s.db.QueryContext(ctx,
`SELECT data FROM agents WHERE schedule != '' AND next_run_at > 0 AND next_run_at <= ? ORDER BY next_run_at`,
dueBefore.Unix())
if err != nil {
return nil, fmt.Errorf("personaStore.ListScheduledAgents: %w", err)
}
return scanAgents(rows)
}
func (s *personaStore) MarkAgentScheduledRun(ctx context.Context, agentID string, ranAt, nextAt time.Time) error {
// Single atomic statement, not Get→mutate→Save: closes the lost-update
// window a concurrent Mark/edit would otherwise open. json_set keeps the
// blob's *time.Time fields consistent with the next_run_at column (Go
// encodes time.Time as RFC3339Nano, so it round-trips through GetAgent).
res, err := s.db.ExecContext(ctx,
`UPDATE agents SET next_run_at=?, data=json_set(data,'$.NextRunAt',?,'$.LastScheduledRunAt',?) WHERE id=?`,
nextAt.Unix(), nextAt.Format(time.RFC3339Nano), ranAt.Format(time.RFC3339Nano), agentID)
if err != nil {
return fmt.Errorf("personaStore.MarkAgentScheduledRun: %w", err)
}
if n, _ := res.RowsAffected(); n == 0 {
return persona.ErrNotFound
}
return nil
}
+106
View File
@@ -0,0 +1,106 @@
package store
import (
"context"
"testing"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/persona"
)
func TestSQLitePersonaStore(t *testing.T) {
ctx := context.Background()
db, err := Open(":memory:")
if err != nil {
t.Fatal(err)
}
defer db.Close()
st := db.Personas()
if err := st.InitializeAgentStorage(ctx); err != nil {
t.Fatal(err)
}
// Full struct round-trips through the JSON blob (incl. nested + map fields).
a := &persona.Agent{
ID: "a1", Name: "helper", OwnerID: "o1", SystemPrompt: "be nice",
ModelTier: "fast", SkillPalette: []string{"animate"},
StateReactEmoji: map[string]string{"running": "⏳"},
ChatbotChannelFilter: "general",
}
if err := st.SaveAgent(ctx, a); err != nil {
t.Fatal(err)
}
got, err := st.GetAgent(ctx, "a1")
if err != nil || got.SystemPrompt != "be nice" || len(got.SkillPalette) != 1 ||
got.StateReactEmoji["running"] != "⏳" {
t.Fatalf("round-trip lost fields: %+v (err %v)", got, err)
}
if byName, err := st.GetAgentByName(ctx, "o1", "helper"); err != nil || byName.ID != "a1" {
t.Fatalf("GetAgentByName: %v %+v", err, byName)
}
if cf, _ := st.ListAgentsByChatbotChannelFilter(ctx); len(cf) != 1 {
t.Errorf("ListAgentsByChatbotChannelFilter = %d, want 1", len(cf))
}
// Scheduling: due query + MarkAgentScheduledRun round-trip.
now := time.Now().UTC()
sched := &persona.Agent{ID: "s1", Name: "cron", OwnerID: "o1", Schedule: "0 * * * *"}
due := now.Add(-time.Minute)
sched.NextRunAt = &due
if err := st.SaveAgent(ctx, sched); err != nil {
t.Fatal(err)
}
dueList, _ := st.ListScheduledAgents(ctx, now)
if len(dueList) != 1 || dueList[0].ID != "s1" {
t.Fatalf("ListScheduledAgents = %+v", dueList)
}
next := now.Add(time.Hour)
if err := st.MarkAgentScheduledRun(ctx, "s1", now, next); err != nil {
t.Fatal(err)
}
if again, _ := st.ListScheduledAgents(ctx, now); len(again) != 0 {
t.Errorf("after MarkAgentScheduledRun, nothing should be due before now: %+v", again)
}
if err := st.DeleteAgent(ctx, "a1"); err != nil {
t.Fatal(err)
}
if _, err := st.GetAgent(ctx, "a1"); err != persona.ErrNotFound {
t.Errorf("GetAgent after delete = %v, want ErrNotFound", err)
}
}
// TestMarkAgentScheduledRunBlobRoundTrips guards the json_set atomic update:
// the JSON blob must stay parseable and reflect the new scheduled times.
func TestMarkAgentScheduledRunBlobRoundTrips(t *testing.T) {
ctx := context.Background()
db, _ := Open(":memory:")
defer db.Close()
st := db.Personas()
st.InitializeAgentStorage(ctx)
start := time.Now().UTC()
a := &persona.Agent{ID: "m1", Name: "n", OwnerID: "o", Schedule: "0 * * * *"}
a.NextRunAt = &start
if err := st.SaveAgent(ctx, a); err != nil {
t.Fatal(err)
}
ran := start
next := start.Add(time.Hour)
if err := st.MarkAgentScheduledRun(ctx, "m1", ran, next); err != nil {
t.Fatal(err)
}
got, err := st.GetAgent(ctx, "m1") // blob must still unmarshal
if err != nil {
t.Fatalf("GetAgent after json_set Mark failed (blob corrupt?): %v", err)
}
if got.NextRunAt == nil || !got.NextRunAt.Equal(next) {
t.Errorf("blob NextRunAt = %v, want %v", got.NextRunAt, next)
}
if got.LastScheduledRunAt == nil || !got.LastScheduledRunAt.Equal(ran) {
t.Errorf("blob LastScheduledRunAt = %v, want %v", got.LastScheduledRunAt, ran)
}
// Unknown id -> ErrNotFound.
if err := st.MarkAgentScheduledRun(ctx, "nope", ran, next); err != persona.ErrNotFound {
t.Errorf("Mark(unknown) = %v, want ErrNotFound", err)
}
}
+280
View File
@@ -0,0 +1,280 @@
package store
import (
"context"
"database/sql"
"encoding/json"
"errors"
"fmt"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/skill"
)
// skillStore is the SQLite-backed skill.SkillStore. Same JSON-blob + indexed
// columns approach as personaStore: the full Skill round-trips, lookups stay
// indexed. Versions live in their own table (each SkillVersion embeds a full
// Skill snapshot, stored as a JSON blob).
type skillStore struct{ db *sql.DB }
// Skills returns a durable skill.SkillStore backed by this database.
func (d *DB) Skills() skill.SkillStore { return &skillStore{db: d.sql} }
var _ skill.SkillStore = (*skillStore)(nil)
func (s *skillStore) Initialize(ctx context.Context) error {
_, err := s.db.ExecContext(ctx, `
CREATE TABLE IF NOT EXISTS skills (
id TEXT PRIMARY KEY,
owner_id TEXT NOT NULL DEFAULT '',
name TEXT NOT NULL DEFAULT '',
source TEXT NOT NULL DEFAULT '',
visibility TEXT NOT NULL DEFAULT '',
chatbot INTEGER NOT NULL DEFAULT 0, -- ExposeAsChatbotTool
schedule TEXT NOT NULL DEFAULT '',
next_run_at INTEGER NOT NULL DEFAULT 0,
data TEXT NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_skills_owner ON skills(owner_id);
CREATE INDEX IF NOT EXISTS idx_skills_vis ON skills(visibility);
CREATE INDEX IF NOT EXISTS idx_skills_sched ON skills(schedule, next_run_at);
CREATE TABLE IF NOT EXISTS skill_versions (
id TEXT PRIMARY KEY,
skill_id TEXT NOT NULL,
version TEXT NOT NULL DEFAULT '',
seq INTEGER NOT NULL, -- append order, for newest-first
data TEXT NOT NULL
);
CREATE UNIQUE INDEX IF NOT EXISTS idx_skill_versions_skill ON skill_versions(skill_id, seq);`)
if err != nil {
return fmt.Errorf("skillStore.Initialize: %w", err)
}
return nil
}
func (s *skillStore) Save(ctx context.Context, sk *skill.Skill) error {
blob, err := json.Marshal(sk)
if err != nil {
return fmt.Errorf("skillStore.Save: marshal: %w", err)
}
var next int64
if !sk.NextRunAt.IsZero() {
next = sk.NextRunAt.Unix()
}
chatbot := 0
if sk.ExposeAsChatbotTool {
chatbot = 1
}
_, err = s.db.ExecContext(ctx, `
INSERT INTO skills (id, owner_id, name, source, visibility, chatbot, schedule, next_run_at, data)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(id) DO UPDATE SET
owner_id=excluded.owner_id, name=excluded.name, source=excluded.source,
visibility=excluded.visibility, chatbot=excluded.chatbot, schedule=excluded.schedule,
next_run_at=excluded.next_run_at, data=excluded.data`,
sk.ID, sk.OwnerID, sk.Name, string(sk.Source), string(sk.Visibility), chatbot,
sk.Schedule, next, string(blob))
if err != nil {
return fmt.Errorf("skillStore.Save: %w", err)
}
return nil
}
func scanSkills(rows *sql.Rows) ([]skill.Skill, error) {
defer rows.Close()
var out []skill.Skill
for rows.Next() {
var blob string
if err := rows.Scan(&blob); err != nil {
return nil, err
}
var sk skill.Skill
if err := json.Unmarshal([]byte(blob), &sk); err != nil {
return nil, err
}
out = append(out, sk)
}
return out, rows.Err()
}
func (s *skillStore) getOne(ctx context.Context, where string, arg ...any) (*skill.Skill, error) {
var blob string
err := s.db.QueryRowContext(ctx, `SELECT data FROM skills WHERE `+where, arg...).Scan(&blob)
switch {
case errors.Is(err, sql.ErrNoRows):
return nil, skill.ErrNotFound
case err != nil:
return nil, err
}
var sk skill.Skill
if err := json.Unmarshal([]byte(blob), &sk); err != nil {
return nil, err
}
return &sk, nil
}
func (s *skillStore) Get(ctx context.Context, id string) (*skill.Skill, error) {
return s.getOne(ctx, "id = ?", id)
}
func (s *skillStore) GetByName(ctx context.Context, ownerID, name string) (*skill.Skill, error) {
return s.getOne(ctx, "owner_id = ? AND name = ?", ownerID, name)
}
func (s *skillStore) ListBuiltinByName(ctx context.Context, name string) (*skill.Skill, error) {
return s.getOne(ctx, "source = ? AND name = ?", string(skill.SourceBuiltin), name)
}
func (s *skillStore) Delete(ctx context.Context, id string) error {
if _, err := s.db.ExecContext(ctx, `DELETE FROM skills WHERE id = ?`, id); err != nil {
return fmt.Errorf("skillStore.Delete: %w", err)
}
return nil
}
func (s *skillStore) query(ctx context.Context, where string, arg ...any) ([]skill.Skill, error) {
rows, err := s.db.QueryContext(ctx, `SELECT data FROM skills WHERE `+where+` ORDER BY name`, arg...)
if err != nil {
return nil, err
}
return scanSkills(rows)
}
func (s *skillStore) ListByOwner(ctx context.Context, ownerID string) ([]skill.Skill, error) {
return s.query(ctx, "owner_id = ?", ownerID)
}
func (s *skillStore) ListPublic(ctx context.Context) ([]skill.Skill, error) {
return s.query(ctx, "visibility = ?", string(skill.VisibilityPublic))
}
func (s *skillStore) ListChatbotExposed(ctx context.Context) ([]skill.Skill, error) {
return s.query(ctx, "chatbot = 1")
}
// ListSharedWith loads visibility=shared rows and filters SharedWith in Go (the
// shared set per skill is small; avoids a JSON-array query).
func (s *skillStore) ListSharedWith(ctx context.Context, memberID string) ([]skill.Skill, error) {
shared, err := s.query(ctx, "visibility = ?", string(skill.VisibilityShared))
if err != nil {
return nil, err
}
out := shared[:0]
for _, sk := range shared {
for _, id := range sk.SharedWith {
if id == memberID {
out = append(out, sk)
break
}
}
}
return out, nil
}
func (s *skillStore) ListDueScheduled(ctx context.Context, now time.Time) ([]skill.Skill, error) {
rows, err := s.db.QueryContext(ctx,
`SELECT data FROM skills WHERE schedule != '' AND next_run_at > 0 AND next_run_at <= ? ORDER BY next_run_at`,
now.Unix())
if err != nil {
return nil, fmt.Errorf("skillStore.ListDueScheduled: %w", err)
}
return scanSkills(rows)
}
func (s *skillStore) MarkScheduledRun(ctx context.Context, skillID string, ranAt, nextAt time.Time) error {
// Single atomic statement instead of Get→mutate→Save: a concurrent Mark or
// admin edit can't lose this update (no read-modify-write window). json_set
// keeps the JSON blob's NextRunAt/LastScheduledRunAt consistent with the
// indexed next_run_at column; RFC3339Nano matches Go's time JSON encoding so
// the blob still round-trips through Get.
var next int64
if !nextAt.IsZero() {
next = nextAt.Unix()
}
res, err := s.db.ExecContext(ctx,
`UPDATE skills SET next_run_at=?, data=json_set(data,'$.NextRunAt',?,'$.LastScheduledRunAt',?) WHERE id=?`,
next, nextAt.Format(time.RFC3339Nano), ranAt.Format(time.RFC3339Nano), skillID)
if err != nil {
return fmt.Errorf("skillStore.MarkScheduledRun: %w", err)
}
if n, _ := res.RowsAffected(); n == 0 {
return skill.ErrNotFound
}
return nil
}
func (s *skillStore) AppendVersion(ctx context.Context, sv skill.SkillVersion) error {
if sv.SkillID == "" {
return fmt.Errorf("skillStore.AppendVersion: skill_id is required")
}
blob, err := json.Marshal(sv)
if err != nil {
return fmt.Errorf("skillStore.AppendVersion: marshal: %w", err)
}
// seq = current max+1 for this skill (newest-first ordering key). The
// MAX-then-INSERT runs in ONE transaction and the (skill_id, seq) index is
// UNIQUE, so two concurrent appends can't both land the same seq: the loser
// fails loudly on commit instead of silently corrupting the ordering. The
// Scan error is propagated (was swallowed, leaving seq=0 on failure).
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return fmt.Errorf("skillStore.AppendVersion: begin: %w", err)
}
defer tx.Rollback() //nolint:errcheck // no-op after Commit
var seq int64
if err := tx.QueryRowContext(ctx, `SELECT COALESCE(MAX(seq),0)+1 FROM skill_versions WHERE skill_id = ?`, sv.SkillID).Scan(&seq); err != nil {
return fmt.Errorf("skillStore.AppendVersion: seq: %w", err)
}
if _, err := tx.ExecContext(ctx,
`INSERT INTO skill_versions (id, skill_id, version, seq, data) VALUES (?, ?, ?, ?, ?)`,
sv.ID, sv.SkillID, sv.Version, seq, string(blob)); err != nil {
return fmt.Errorf("skillStore.AppendVersion: insert: %w", err)
}
if err := tx.Commit(); err != nil {
return fmt.Errorf("skillStore.AppendVersion: commit: %w", err)
}
return nil
}
func (s *skillStore) ListVersionsBySkill(ctx context.Context, skillID string, limit int) ([]skill.SkillVersion, error) {
q := `SELECT data FROM skill_versions WHERE skill_id = ? ORDER BY seq DESC`
args := []any{skillID}
if limit > 0 {
q += ` LIMIT ?`
args = append(args, limit)
}
rows, err := s.db.QueryContext(ctx, q, args...)
if err != nil {
return nil, fmt.Errorf("skillStore.ListVersionsBySkill: %w", err)
}
defer rows.Close()
var out []skill.SkillVersion
for rows.Next() {
var blob string
if err := rows.Scan(&blob); err != nil {
return nil, err
}
var sv skill.SkillVersion
if err := json.Unmarshal([]byte(blob), &sv); err != nil {
return nil, err
}
out = append(out, sv)
}
return out, rows.Err()
}
func (s *skillStore) GetVersionByID(ctx context.Context, versionID string) (*skill.SkillVersion, error) {
var blob string
err := s.db.QueryRowContext(ctx, `SELECT data FROM skill_versions WHERE id = ?`, versionID).Scan(&blob)
switch {
case errors.Is(err, sql.ErrNoRows):
return nil, skill.ErrNotFound
case err != nil:
return nil, err
}
var sv skill.SkillVersion
if err := json.Unmarshal([]byte(blob), &sv); err != nil {
return nil, err
}
return &sv, nil
}
+72
View File
@@ -0,0 +1,72 @@
package store
import (
"context"
"testing"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/skill"
)
func TestSQLiteSkillStore(t *testing.T) {
ctx := context.Background()
db, err := Open(":memory:")
if err != nil {
t.Fatal(err)
}
defer db.Close()
st := db.Skills()
if err := st.Initialize(ctx); err != nil {
t.Fatal(err)
}
pub := &skill.Skill{ID: "a", Name: "pub", OwnerID: "o1", Visibility: skill.VisibilityPublic,
Tools: []string{"summarize"}, ExposeAsChatbotTool: true}
shared := &skill.Skill{ID: "b", Name: "shr", OwnerID: "o1", Visibility: skill.VisibilityShared, SharedWith: []string{"bob"}}
if err := st.Save(ctx, pub); err != nil {
t.Fatal(err)
}
if err := st.Save(ctx, shared); err != nil {
t.Fatal(err)
}
got, err := st.Get(ctx, "a")
if err != nil || len(got.Tools) != 1 || !got.ExposeAsChatbotTool {
t.Fatalf("round-trip: %v %+v", err, got)
}
if ps, _ := st.ListPublic(ctx); len(ps) != 1 || ps[0].ID != "a" {
t.Errorf("ListPublic = %+v", ps)
}
if ss, _ := st.ListSharedWith(ctx, "bob"); len(ss) != 1 || ss[0].ID != "b" {
t.Errorf("ListSharedWith(bob) = %+v", ss)
}
if ss, _ := st.ListSharedWith(ctx, "carol"); len(ss) != 0 {
t.Errorf("ListSharedWith(carol) should be empty: %+v", ss)
}
if ce, _ := st.ListChatbotExposed(ctx); len(ce) != 1 {
t.Errorf("ListChatbotExposed = %d, want 1", len(ce))
}
// Versions newest-first + by id.
st.AppendVersion(ctx, skill.SkillVersion{ID: "v1", SkillID: "a", Version: "1.0.0"})
st.AppendVersion(ctx, skill.SkillVersion{ID: "v2", SkillID: "a", Version: "1.1.0"})
vs, _ := st.ListVersionsBySkill(ctx, "a", 10)
if len(vs) != 2 || vs[0].ID != "v2" {
t.Errorf("versions newest-first: %+v", vs)
}
if gv, err := st.GetVersionByID(ctx, "v1"); err != nil || gv.Version != "1.0.0" {
t.Errorf("GetVersionByID: %v %+v", err, gv)
}
// Scheduling.
now := time.Now().UTC()
cron := &skill.Skill{ID: "c", Name: "cron", OwnerID: "o1", Schedule: "0 * * * *", NextRunAt: now.Add(-time.Minute)}
st.Save(ctx, cron)
if due, _ := st.ListDueScheduled(ctx, now); len(due) != 1 || due[0].ID != "c" {
t.Fatalf("ListDueScheduled = %+v", due)
}
st.MarkScheduledRun(ctx, "c", now, now.Add(time.Hour))
if due, _ := st.ListDueScheduled(ctx, now); len(due) != 0 {
t.Errorf("after MarkScheduledRun nothing due: %+v", due)
}
}
+54
View File
@@ -0,0 +1,54 @@
// Package store provides durable, pure-Go SQLite implementations of executus's
// battery store seams (audit, budget, persona, skill). It is a SEPARATE nested
// module so the SQLite driver (modernc.org/sqlite — pure Go, no cgo) never
// enters the executus core go.sum: a static-binary host (gadfly) that imports
// only the core stays static, while a host that wants turnkey persistence
// imports this module and gets every *Store seam backed by one SQLite file.
//
// db, _ := store.Open("file:executus.db?_pragma=busy_timeout(5000)")
// defer db.Close()
// budgetStore := db.Budget() // satisfies budget.BudgetStorage
package store
import (
"database/sql"
"fmt"
_ "modernc.org/sqlite" // pure-Go driver, registered as "sqlite"
)
// DB is a handle to one SQLite database backing the executus store seams. Each
// accessor (Budget(), …) returns a seam implementation sharing this connection.
// Safe for concurrent use (SQLite serializes writes; busy_timeout handles
// contention). Construct with Open; close with Close.
type DB struct {
sql *sql.DB
}
// Open opens (creating if absent) a SQLite database at dsn and returns a DB. A
// dsn of ":memory:" yields an ephemeral in-memory database. The caller owns the
// returned DB and must Close it.
func Open(dsn string) (*DB, error) {
sqldb, err := sql.Open("sqlite", dsn)
if err != nil {
return nil, fmt.Errorf("store: open %q: %w", dsn, err)
}
// A contended writer should WAIT for the lock, not fail immediately — set a
// busy_timeout so concurrent stores don't see spurious SQLITE_BUSY. (The
// doc example advertised this; it's now actually applied for every DSN.)
if _, err := sqldb.Exec("PRAGMA busy_timeout=5000"); err != nil {
sqldb.Close()
return nil, fmt.Errorf("store: set busy_timeout %q: %w", dsn, err)
}
if err := sqldb.Ping(); err != nil {
sqldb.Close()
return nil, fmt.Errorf("store: ping %q: %w", dsn, err)
}
return &DB{sql: sqldb}, nil
}
// Close closes the underlying database.
func (d *DB) Close() error { return d.sql.Close() }
// SQL exposes the underlying *sql.DB for hosts that need direct access.
func (d *DB) SQL() *sql.DB { return d.sql }
+302
View File
@@ -0,0 +1,302 @@
// Package critic is the run-watchdog battery: a two-tier timeout monitor that
// catches a run that has stopped making progress. It plugs into
// run.Ports.Critic.
//
// The split of concerns is deliberate. executus owns the deterministic
// MECHANICS — track activity, fire on a soft timeout, enforce a hard-kill
// backstop, carry steer messages and the extendable deadline back to the
// executor. The POLICY — what to actually do when a run stalls (nudge it,
// extend its deadline, kill it, escalate to a human) — is the Escalator seam.
// Mort plugs its LLM critic-agent in as an Escalator; ExtendOnce is the
// zero-dependency default.
//
// The executor wires run.Ports.Critic (C0b): it feeds the handle activity,
// binds the run context to its extendable Deadline, drains its Steer, and polls
// MaxSteps each step so an Escalator can also raise a long run's step ceiling
// (Decision.RaiseStepsBy).
package critic
import (
"context"
"errors"
"log/slog"
"math"
"sync"
"time"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/executus/run"
)
// Progress is the snapshot the critic hands an Escalator when a run stalls.
type Progress struct {
Iterations int // completed agent-loop iterations so far
LastActivity time.Time // wall-clock of the last step/tool event
Idle time.Duration // now - LastActivity
LastTool string // name of the most recently started tool ("" if none)
}
// Decision is the Escalator's verdict for a stalled run. Zero value = do
// nothing (let the hard backstop eventually kill a truly hung run).
type Decision struct {
Nudge []llm.Message // injected before the agent's next turn (a steer)
ExtendBy time.Duration // push the hard deadline out by this much
RaiseStepsBy int // raise the run's tool-dispatch step ceiling by this
Kill bool // cancel the run now
KillReason string
}
// Escalator decides what to do when a run crosses its soft timeout. It is
// called at most once per idle period (a fresh step/tool event re-arms it).
type Escalator interface {
OnSoftTimeout(ctx context.Context, info run.RunInfo, p Progress) Decision
}
// ExtendOnce is the default Escalator: the first time a given run stalls it
// extends that run's deadline by By (giving a slow-but-healthy run room), then
// takes no further action for it — so a genuinely hung run is later killed by
// the hard backstop. A nil/zero By falls back to one soft-timeout's worth.
//
// The one-shot is keyed PER RUN (by RunInfo.RunID): a single System shares one
// ExtendOnce across every run it monitors, so a global flag would let only the
// first run to stall ever get its extension. The fired set grows with the
// number of distinct runs that stall — fine for a process's run volume; a host
// running unboundedly long can construct a fresh System periodically.
type ExtendOnce struct {
By time.Duration
mu sync.Mutex
fired map[string]bool // run ids that have already had their one extension
}
// OnSoftTimeout implements Escalator.
func (e *ExtendOnce) OnSoftTimeout(_ context.Context, info run.RunInfo, p Progress) Decision {
e.mu.Lock()
defer e.mu.Unlock()
if e.fired[info.RunID] {
return Decision{}
}
if e.fired == nil {
e.fired = map[string]bool{}
}
e.fired[info.RunID] = true
by := e.By
if by <= 0 {
by = p.Idle // ~one soft timeout
}
return Decision{ExtendBy: by}
}
// System implements run.Critic. Construct with New; one System monitors many
// runs concurrently (each Monitor returns an independent handle).
type System struct {
esc Escalator
backstopMul float64 // hard deadline = softTimeout * backstopMul from start
checkInterval time.Duration
now func() time.Time
logger *slog.Logger
}
func (s *System) log() *slog.Logger {
if s.logger != nil {
return s.logger
}
return slog.Default()
}
// New builds a run.Critic. esc is the policy (nil → ExtendOnce). backstopMul is
// the hard-kill backstop as a multiple of each run's soft timeout (<=1 → 3). A
// nil esc + the default backstop gives a safe "extend once, then hard-kill"
// watchdog with no host wiring.
func New(esc Escalator, backstopMul float64) *System {
if esc == nil {
esc = &ExtendOnce{}
}
if backstopMul <= 1 {
backstopMul = 3
}
return &System{esc: esc, backstopMul: backstopMul, now: time.Now}
}
var _ run.Critic = (*System)(nil)
// Monitor starts watching a run and returns its handle. Implements run.Critic.
func (s *System) Monitor(ctx context.Context, info run.RunInfo, softTimeout time.Duration) run.CriticHandle {
if softTimeout <= 0 {
return run.CriticHandle(nil) // no soft timeout → not monitored
}
now := s.now()
check := s.checkInterval
if check <= 0 {
check = softTimeout / 2
if check < time.Second {
check = time.Second
}
}
h := &handle{
sys: s,
info: info,
softTimeout: softTimeout,
now: s.now,
lastActivity: now,
deadline: now.Add(time.Duration(float64(softTimeout) * s.backstopMul)),
maxSteps: info.MaxIterations, // base ceiling; an Escalator may RaiseStepsBy
stopCh: make(chan struct{}),
}
go h.watch(ctx, check)
return h
}
// handle is one run's live critic link. Implements run.CriticHandle.
type handle struct {
sys *System
info run.RunInfo
softTimeout time.Duration
now func() time.Time
mu sync.Mutex
lastActivity time.Time
escalatedAt time.Time // lastActivity value we last escalated for (de-dupes per idle period)
deadline time.Time
steer []llm.Message
iterations int
maxSteps int // current tool-dispatch ceiling (base MaxIterations, raised by RaiseStepsBy)
lastTool string
killed bool // sticky: once an Escalator kills, no later decision un-kills it
killCause error // non-nil once killed; surfaced via KillCause for "killed" status
stopped bool
stopCh chan struct{}
}
func (h *handle) RecordStep(iter int, _ *llm.Response) {
// This battery's Progress tracks iteration count + activity, not per-step
// payload, so the response is unused here; a richer Escalator could record it.
h.mu.Lock()
h.iterations = iter
h.lastActivity = h.now()
h.mu.Unlock()
}
func (h *handle) RecordToolStart(name, _ string) {
h.mu.Lock()
h.lastTool = name
h.lastActivity = h.now()
h.mu.Unlock()
}
func (h *handle) Steer() []llm.Message {
h.mu.Lock()
defer h.mu.Unlock()
if len(h.steer) == 0 {
return nil
}
out := h.steer
h.steer = nil
return out
}
func (h *handle) Deadline() time.Time {
h.mu.Lock()
defer h.mu.Unlock()
return h.deadline
}
func (h *handle) MaxSteps() int {
h.mu.Lock()
defer h.mu.Unlock()
return h.maxSteps
}
func (h *handle) KillCause() error {
h.mu.Lock()
defer h.mu.Unlock()
return h.killCause
}
func (h *handle) Stop() {
h.mu.Lock()
if !h.stopped {
h.stopped = true
close(h.stopCh)
}
h.mu.Unlock()
}
// watch fires the Escalator once per idle period the run crosses its soft
// timeout, and applies the returned Decision.
func (h *handle) watch(ctx context.Context, interval time.Duration) {
// A misbehaving Escalator that panics must not silently kill the watch
// goroutine (which would leave the run unmonitored for its lifetime). Log
// and exit cleanly — the run falls back to the deadline already set.
defer func() {
if r := recover(); r != nil {
h.sys.log().Error("critic watch panicked; run is now unmonitored", "run", h.info.RunID, "panic", r)
}
}()
t := time.NewTicker(interval)
defer t.Stop()
for {
select {
case <-h.stopCh:
return
case <-ctx.Done():
return
case <-t.C:
h.tick(ctx)
}
}
}
func (h *handle) tick(ctx context.Context) {
h.mu.Lock()
// Kill is sticky: once an Escalator has killed this run, no later tick (and
// no later Decision) un-collapses the deadline.
if h.killed {
h.mu.Unlock()
return
}
idle := h.now().Sub(h.lastActivity)
// Only escalate once per idle period: skip if we already escalated for this
// exact lastActivity (a fresh step/tool updates lastActivity and re-arms).
if idle < h.softTimeout || h.escalatedAt.Equal(h.lastActivity) {
h.mu.Unlock()
return
}
h.escalatedAt = h.lastActivity
snap := Progress{Iterations: h.iterations, LastActivity: h.lastActivity, Idle: idle, LastTool: h.lastTool}
h.mu.Unlock()
d := h.sys.esc.OnSoftTimeout(ctx, h.info, snap)
h.mu.Lock()
defer h.mu.Unlock()
if h.killed { // a concurrent tick may have killed while OnSoftTimeout ran
return
}
if d.Kill {
h.killed = true
reason := d.KillReason
if reason == "" {
reason = "critic killed the run"
}
h.killCause = errors.New(reason) // surfaced via KillCause → "killed" status
h.deadline = h.now() // immediate hard deadline → executor cancels
return // ignore any Nudge/ExtendBy paired with a Kill
}
if len(d.Nudge) > 0 {
h.steer = append(h.steer, d.Nudge...)
}
if d.ExtendBy > 0 {
h.deadline = h.deadline.Add(d.ExtendBy)
}
if d.RaiseStepsBy > 0 {
// Overflow-safe: a buggy Escalator returning a huge delta must not wrap
// maxSteps negative (which the executor would read as "defer to base").
if d.RaiseStepsBy > math.MaxInt-h.maxSteps {
h.maxSteps = math.MaxInt
} else {
h.maxSteps += d.RaiseStepsBy
}
}
}
+100
View File
@@ -0,0 +1,100 @@
package critic
import (
"context"
"sync"
"testing"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/run"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
)
// escFunc adapts a func to an Escalator.
type escFunc func(context.Context, run.RunInfo, Progress) Decision
func (f escFunc) OnSoftTimeout(ctx context.Context, i run.RunInfo, p Progress) Decision {
return f(ctx, i, p)
}
func TestMonitorEscalatesOncePerIdlePeriodAndExtends(t *testing.T) {
var mu sync.Mutex
var calls int
esc := escFunc(func(_ context.Context, _ run.RunInfo, p Progress) Decision {
mu.Lock()
calls++
mu.Unlock()
return Decision{ExtendBy: 50 * time.Millisecond, Nudge: []llm.Message{{Role: llm.RoleUser}}}
})
s := New(esc, 3)
s.checkInterval = 5 * time.Millisecond
h := s.Monitor(context.Background(), run.RunInfo{RunID: "r"}, 20*time.Millisecond)
defer h.Stop()
d0 := h.Deadline()
time.Sleep(60 * time.Millisecond) // cross the soft timeout with no activity
mu.Lock()
c := calls
mu.Unlock()
if c < 1 {
t.Fatalf("expected at least one escalation, got %d", c)
}
// Nudge was queued and is drained once.
if msgs := h.Steer(); len(msgs) == 0 {
t.Error("expected a queued steer nudge")
}
if msgs := h.Steer(); len(msgs) != 0 {
t.Error("steer should drain (be empty on second read)")
}
// Deadline was extended.
if !h.Deadline().After(d0) {
t.Error("deadline should have been extended past the original")
}
// A fresh step re-arms; another idle period escalates again.
h.RecordStep(1, nil)
time.Sleep(60 * time.Millisecond)
mu.Lock()
c2 := calls
mu.Unlock()
if c2 <= c {
t.Errorf("a re-armed idle period should escalate again (%d -> %d)", c, c2)
}
}
func TestKillCollapsesDeadline(t *testing.T) {
esc := escFunc(func(context.Context, run.RunInfo, Progress) Decision {
return Decision{Kill: true, KillReason: "hung"}
})
s := New(esc, 10) // big backstop so only Kill collapses it
s.checkInterval = 5 * time.Millisecond
h := s.Monitor(context.Background(), run.RunInfo{RunID: "r"}, 20*time.Millisecond)
defer h.Stop()
time.Sleep(60 * time.Millisecond)
if h.Deadline().After(time.Now().Add(time.Second)) {
t.Error("Kill should collapse the deadline to ~now")
}
}
func TestExtendOnceOnlyFiresOnce(t *testing.T) {
e := &ExtendOnce{By: time.Minute}
// Same run id: only the first call extends.
d1 := e.OnSoftTimeout(context.Background(), run.RunInfo{RunID: "r1"}, Progress{})
d2 := e.OnSoftTimeout(context.Background(), run.RunInfo{RunID: "r1"}, Progress{})
if d1.ExtendBy != time.Minute {
t.Errorf("first decision should extend, got %+v", d1)
}
if d2.ExtendBy != 0 || d2.Kill {
t.Errorf("second call for the same run should be a no-op, got %+v", d2)
}
// A DIFFERENT run still gets its own one extension (per-run, not global).
if d3 := e.OnSoftTimeout(context.Background(), run.RunInfo{RunID: "r2"}, Progress{}); d3.ExtendBy != time.Minute {
t.Errorf("a different run should get its own extension, got %+v", d3)
}
}
func TestZeroSoftTimeoutNotMonitored(t *testing.T) {
s := New(nil, 3)
if h := s.Monitor(context.Background(), run.RunInfo{}, 0); h != nil {
t.Error("zero soft timeout should return a nil handle (not monitored)")
}
}
+35 -13
View File
@@ -1,27 +1,49 @@
// Command minimal demonstrates executus's standalone core primitives available
// today (P0): the config seam + bounded fan-out. The full zero-config "agentic
// in ~12 lines" example arrives once the model, tool, and run packages land
// (P1P3).
// Command minimal is executus's "hello, agentic world": wire a model resolver,
// a tool registry, and the run executor, then run an agent. With no batteries
// (Audit/Budget/Critic/Checkpointer/Palette/Delivery all nil) this is a
// bounded, in-memory run — the light-host shape (gadfly's case).
//
// Run it with a provider key for the configured tier, e.g.
//
// ANTHROPIC_API_KEY=sk-... go run ./examples/minimal
//
// Override a tier from the environment without touching code, e.g.
//
// EXECUTUS_MODEL_TIER_FAST=openai/gpt-4o-mini ANTHROPIC_API_KEY= OPENAI_KEY=sk-... go run ./examples/minimal
package main
import (
"context"
"fmt"
"log"
"gitea.stevedudenhoeffer.com/steve/executus/config"
"gitea.stevedudenhoeffer.com/steve/executus/fanout"
"gitea.stevedudenhoeffer.com/steve/executus/model"
"gitea.stevedudenhoeffer.com/steve/executus/run"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
func main() {
cfg := config.Env("EXECUTUS_") // e.g. EXECUTUS_FANOUT_MAX_CONCURRENT=8
max := cfg.Int("fanout.max_concurrent", 4)
// 1. Configure model tiers: live values come from the environment
// (EXECUTUS_MODEL_TIER_<NAME>), falling back to these defaults.
model.Configure(config.Env("EXECUTUS_"), map[string]string{
"fast": "anthropic/claude-haiku-4-5",
"thinking": "anthropic/claude-opus-4-8",
}, 0)
items := []string{"alpha", "beta", "gamma", "delta"}
results := fanout.Run(context.Background(), items,
fanout.Options[string]{MaxConcurrent: max},
func(_ context.Context, s string) (int, error) { return len(s), nil })
// 2. Build the executor: a tool registry + the model resolver. No batteries.
ex := run.New(run.Config{
Registry: tool.NewRegistry(),
Models: model.ParseModelForContext,
})
for _, r := range results {
fmt.Printf("%-6s -> %d (err=%v)\n", items[r.Index], r.Value, r.Err)
// 3. Run an agent and print its answer.
res := ex.Run(context.Background(),
run.RunnableAgent{Name: "assistant", SystemPrompt: "You are concise.", ModelTier: "fast"},
tool.Invocation{RunID: "demo-1", CallerID: "local"},
"In one sentence, what is an agent harness?")
if res.Err != nil {
log.Fatalf("run failed: %v", res.Err)
}
fmt.Println(res.Output)
}
+38
View File
@@ -0,0 +1,38 @@
# examples/reviewer — the light-tier canary
A **gadfly-shaped adversarial PR reviewer built on the executus core only** — no
batteries, no database, no host adapters. It exists to prove that the core is
sufficient for a static-binary light host (gadfly's shape), and that such a host
keeps a `go.sum` free of `gorm`/`redis`/`discordgo`/`sqlite`.
What it exercises, all from core:
| Concern | executus core piece |
|---|---|
| Env-driven model fleet + tier overrides | `config.Env` + `model.Configure` |
| Tier resolution + failover | `model.ParseModelForContext` |
| N models × M lenses swarm | `fanout.Run` (with `PerKey` per-provider caps) |
| Structured findings per cell | `model.GenerateWith[T]` |
| One report section per model, worst-verdict-led | `Consolidate` (local) |
## Run
```sh
REVIEWER_MODELS=fast,thinking \
ANTHROPIC_API_KEY=sk-... \
go run ./examples/reviewer -diff "$(git diff HEAD~1)"
```
Config (all optional, `REVIEWER_`-prefixed env):
- `REVIEWER_MODELS` — csv of tier names / `provider/model` specs (default `fast`)
- `REVIEWER_MODEL_TIER_<NAME>` — override a tier's resolved spec
- `REVIEWER_MAX_CONCURRENT` — total in-flight swarm cells (default 6)
- `REVIEWER_PROVIDER_CONCURRENCY` — per-provider cap (default 3)
## Test
`reviewer_test.go` runs the whole swarm against majordomo's fake provider
(hermetic, no network) and asserts the consolidated verdicts. A `go list -deps`
check in CI confirms the package pulls in no battery and no DB driver — the
light-tier invariant.
+110
View File
@@ -0,0 +1,110 @@
package main
import (
"context"
"flag"
"fmt"
"io"
"os"
"strings"
"gitea.stevedudenhoeffer.com/steve/executus/config"
"gitea.stevedudenhoeffer.com/steve/executus/fanout"
"gitea.stevedudenhoeffer.com/steve/executus/model"
)
// DefaultLenses is the canary's review suite (mirrors gadfly's default).
var DefaultLenses = []Lens{
{Name: "security", Focus: "auth, injection, secret leakage, unsafe deserialization, SSRF."},
{Name: "correctness", Focus: "logic errors, broken invariants, off-by-one, contract violations."},
{Name: "error-handling", Focus: "swallowed errors, missing timeouts, races, unhandled edge cases."},
}
// Reviewer is configured entirely from the environment (the GADFLY_*-style light
// host): REVIEWER_MODELS (csv of tier/spec), REVIEWER_MODEL_TIER_<NAME> overrides,
// REVIEWER_MAX_CONCURRENT, REVIEWER_PROVIDER_CONCURRENCY. The diff is read from
// -diff or stdin.
//
// REVIEWER_MODELS=fast,thinking ANTHROPIC_API_KEY=... go run ./examples/reviewer < my.diff
func main() {
cfg := config.Env("REVIEWER_")
// Tier table from env, with code defaults.
model.Configure(cfg, map[string]string{
"fast": "anthropic/claude-haiku-4-5",
"thinking": "anthropic/claude-opus-4-8",
}, 0)
fleet := splitCSV(cfg.String("models", "fast"))
maxConc := cfg.Int("max_concurrent", 6)
perProvider := cfg.Int("provider_concurrency", 3)
diffFlag := flag.String("diff", "", "diff text to review; reads stdin when empty")
flag.Parse()
diff := *diffFlag
if strings.TrimSpace(diff) == "" {
// Guard against blocking forever on an interactive TTY (no piped input).
if fi, _ := os.Stdin.Stat(); fi != nil && fi.Mode()&os.ModeCharDevice != 0 {
fmt.Fprintln(os.Stderr, "reviewer: no diff (pass -diff or pipe one on stdin)")
os.Exit(2)
}
b, err := io.ReadAll(os.Stdin)
if err != nil {
fmt.Fprintf(os.Stderr, "reviewer: reading stdin: %v\n", err)
os.Exit(2)
}
diff = string(b)
}
if strings.TrimSpace(diff) == "" {
fmt.Fprintln(os.Stderr, "reviewer: no diff (pass -diff or pipe one on stdin)")
os.Exit(2)
}
ctx := context.Background()
var models []NamedModel
for _, spec := range fleet {
_, m, err := model.ParseModelForContext(ctx, spec)
if err != nil {
fmt.Fprintf(os.Stderr, "reviewer: resolve model %q: %v\n", spec, err)
os.Exit(1)
}
models = append(models, NamedModel{Name: spec, Provider: providerOf(spec), Model: m})
}
results := Review(ctx, models, DefaultLenses, diff, fanout.Options[cell]{
MaxConcurrent: maxConc,
PerKey: perKeyCaps(models, perProvider),
})
fmt.Print(Consolidate(results))
}
func splitCSV(s string) []string {
var out []string
for _, p := range strings.Split(s, ",") {
if p = strings.TrimSpace(p); p != "" {
out = append(out, p)
}
}
return out
}
// providerOf returns a model spec's provider (the first path segment, e.g.
// "anthropic/claude-…" → "anthropic"; a bare tier name → itself).
func providerOf(spec string) string {
if i := strings.IndexByte(spec, '/'); i > 0 {
return spec[:i]
}
return spec // bare tier name → its own bucket (don't collapse distinct tiers)
}
// perKeyCaps builds the PerKey map: each distinct provider capped at perProvider.
func perKeyCaps(models []NamedModel, perProvider int) map[string]int {
if perProvider <= 0 {
return nil
}
caps := map[string]int{}
for _, m := range models {
caps[m.Provider] = perProvider
}
return caps
}
+204
View File
@@ -0,0 +1,204 @@
// Command reviewer is executus's light-tier CANARY: a gadfly-shaped adversarial
// PR reviewer built on the executus CORE ONLY — no batteries, no DB, no host.
// It proves the core is sufficient for a static-binary host like gadfly:
//
// - config.Env → env-driven model fleet + concurrency (GADFLY_*-style)
// - model.Configure/... → tier resolution + failover over majordomo
// - fanout.Run → the N-models × M-lenses swarm, with per-provider caps
// - model.GenerateWith[T] → structured findings per (model, lens)
// - consolidation → one report section per model, worst-verdict-led
//
// The whole thing imports only executus core packages, so a binary built from it
// keeps a go.sum free of gorm/redis/discordgo/sqlite — the light-tier invariant.
//
// See reviewer_test.go for the hermetic swarm test (majordomo's fake provider).
package main
import (
"context"
"fmt"
"sort"
"strings"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/executus/fanout"
"gitea.stevedudenhoeffer.com/steve/executus/model"
)
// Severity orders findings; the rank drives a model's worst-verdict header.
type Severity string
const (
SevTrivial Severity = "trivial"
SevSmall Severity = "small"
SevMedium Severity = "medium"
SevHigh Severity = "high"
SevCritical Severity = "critical"
)
func severityRank(s Severity) int {
switch s {
case SevCritical:
return 4
case SevHigh:
return 3
case SevMedium:
return 2
case SevSmall:
return 1
default:
return 0
}
}
// Finding is one issue a lens reports. It is the structured-output schema the
// model must satisfy (majordomo derives the JSON schema from this struct).
type Finding struct {
Severity Severity `json:"severity" jsonschema:"enum=trivial,enum=small,enum=medium,enum=high,enum=critical"`
Title string `json:"title"`
Detail string `json:"detail"`
}
// lensReport is the per-(model,lens) structured response.
type lensReport struct {
Findings []Finding `json:"findings"`
}
// Lens is one review dimension (security / correctness / …).
type Lens struct {
Name string
Focus string // appended to the base system prompt
}
// NamedModel is a resolved model plus the label + provider used for fan-out
// keying (per-provider concurrency) and reporting.
type NamedModel struct {
Name string // display label (the tier/spec the host configured)
Provider string // fan-out key for PerKey concurrency (e.g. "ollama-cloud")
Model llm.Model
}
// LensResult is one swarm cell's outcome.
type LensResult struct {
Model string
Lens string
Findings []Finding
Err error
}
const baseSystemPrompt = "You are an adversarial code reviewer. Review the diff for real, verifiable problems only — no style nits. Return ONLY JSON matching the schema. Report nothing if you find nothing."
// Review runs every (model × lens) cell of the swarm concurrently, bounded by
// opts (total + per-provider caps), and returns one LensResult per cell. A cell
// whose model call fails carries the error in LensResult.Err — one bad cell
// never aborts the swarm (the closure embeds per-cell errors in LensResult.Err).
func Review(ctx context.Context, models []NamedModel, lenses []Lens, diff string, opts fanout.Options[cell]) []LensResult {
cells := make([]cell, 0, len(models)*len(lenses))
for _, m := range models {
for _, l := range lenses {
cells = append(cells, cell{model: m, lens: l})
}
}
// Key each cell by its provider so PerKey throttles per backend (the
// GADFLY_PROVIDER_CONCURRENCY analogue).
if opts.Key == nil {
opts.Key = func(c cell) string { return c.model.Provider }
}
results := fanout.Run(ctx, cells, opts, func(ctx context.Context, c cell) (LensResult, error) {
sys := baseSystemPrompt
if c.lens.Focus != "" {
sys += "\n\nLens — " + c.lens.Name + ": " + c.lens.Focus
}
msgs := []llm.Message{{Role: llm.RoleUser, Parts: []llm.Part{llm.Text("Diff under review:\n" + diff)}}}
rep, err := model.GenerateWith[lensReport](ctx, c.model.Model, sys, msgs)
lr := LensResult{Model: c.model.Name, Lens: c.lens.Name, Findings: rep.Findings, Err: err}
// Return the value either way (err embedded) so every cell reports.
return lr, nil
})
out := make([]LensResult, 0, len(results))
for _, r := range results {
if r.Err != nil { // a swarm-level error (ctx cancel) with no value
out = append(out, LensResult{Err: r.Err})
continue
}
out = append(out, r.Value)
}
return out
}
// cell is one (model, lens) swarm task.
type cell struct {
model NamedModel
lens Lens
}
// Consolidate renders the swarm's results into one report: a section per model,
// each led by that model's worst finding severity, mirroring gadfly's
// one-comment-per-model output.
func Consolidate(results []LensResult) string {
byModel := map[string][]LensResult{}
var order []string
aborted := 0 // cells dropped before running (swarm cancelled) — no model attribution
for _, r := range results {
if r.Model == "" {
if r.Err != nil {
aborted++
}
continue
}
if _, ok := byModel[r.Model]; !ok {
order = append(order, r.Model)
}
byModel[r.Model] = append(byModel[r.Model], r)
}
sort.Strings(order)
var b strings.Builder
if aborted > 0 {
fmt.Fprintf(&b, "> ⚠ swarm cancelled — %d cell(s) did not run; results below are partial.\n\n", aborted)
}
for _, m := range order {
rs := byModel[m]
var all []Finding
worst := -1
errored := 0
for _, r := range rs {
if r.Err != nil {
errored++
continue
}
all = append(all, r.Findings...)
for _, f := range r.Findings {
if severityRank(f.Severity) > worst {
worst = severityRank(f.Severity)
}
}
}
// A model whose every lens errored produced NO data — saying "no issues
// found" would be misleading, so it gets its own verdict.
successful := len(rs) - errored
verdict := "no issues found"
switch {
case successful == 0 && errored > 0:
verdict = "review incomplete"
case worst >= severityRank(SevHigh):
verdict = "blocking issues found"
case worst >= 0:
verdict = "minor issues"
}
fmt.Fprintf(&b, "## %s — %s", m, verdict)
if errored > 0 {
fmt.Fprintf(&b, " (⚠ %d lens(es) errored)", errored)
}
b.WriteString("\n")
sort.SliceStable(all, func(i, j int) bool {
return severityRank(all[i].Severity) > severityRank(all[j].Severity)
})
for _, f := range all {
fmt.Fprintf(&b, "- [%s] %s — %s\n", f.Severity, f.Title, f.Detail)
}
b.WriteString("\n")
}
return b.String()
}
+128
View File
@@ -0,0 +1,128 @@
package main
import (
"context"
"strings"
"testing"
"gitea.stevedudenhoeffer.com/steve/majordomo/provider/fake"
"gitea.stevedudenhoeffer.com/steve/executus/fanout"
)
// TestReviewSwarm proves the light-tier path end-to-end against the fake
// provider: a 2-model × 3-lens swarm runs, structured findings parse, and
// consolidation produces one verdict-led section per model — no batteries, no
// network.
func TestReviewSwarm(t *testing.T) {
fp := fake.New("fakeprov")
// Model "hot" reports a high-severity finding on every lens; "cold" reports
// nothing. Each model is called once per lens (3×), so enqueue 3 each.
hot := `{"findings":[{"severity":"high","title":"SQL injection","detail":"unsanitized id in query"}]}`
cold := `{"findings":[]}`
for i := 0; i < 3; i++ {
fp.Enqueue("hot", fake.Reply(hot))
fp.Enqueue("cold", fake.Reply(cold))
}
hotM, err := fp.Model("hot")
if err != nil {
t.Fatal(err)
}
coldM, err := fp.Model("cold")
if err != nil {
t.Fatal(err)
}
models := []NamedModel{
{Name: "hot", Provider: "fakeprov", Model: hotM},
{Name: "cold", Provider: "fakeprov", Model: coldM},
}
lenses := []Lens{{Name: "security"}, {Name: "correctness"}, {Name: "error-handling"}}
results := Review(context.Background(), models, lenses, "some diff",
fanout.Options[cell]{MaxConcurrent: 6, PerKey: map[string]int{"fakeprov": 3}})
// 2 models × 3 lenses = 6 cells, all successful.
if len(results) != 6 {
t.Fatalf("got %d cells, want 6", len(results))
}
var hotFindings, coldFindings, errs int
for _, r := range results {
if r.Err != nil {
errs++
continue
}
switch r.Model {
case "hot":
hotFindings += len(r.Findings)
case "cold":
coldFindings += len(r.Findings)
}
}
if errs != 0 {
t.Errorf("expected no cell errors, got %d", errs)
}
if hotFindings != 3 { // one per lens
t.Errorf("hot model findings = %d, want 3", hotFindings)
}
if coldFindings != 0 {
t.Errorf("cold model findings = %d, want 0", coldFindings)
}
report := Consolidate(results)
if !strings.Contains(report, "hot — blocking issues found") {
t.Errorf("hot section should lead with a blocking verdict:\n%s", report)
}
if !strings.Contains(report, "cold — no issues found") {
t.Errorf("cold section should report no issues:\n%s", report)
}
if !strings.Contains(report, "SQL injection") {
t.Errorf("report should surface the finding:\n%s", report)
}
}
// TestConsolidateVerdicts checks the worst-severity-led header logic.
func TestConsolidateVerdicts(t *testing.T) {
got := Consolidate([]LensResult{
{Model: "m", Lens: "a", Findings: []Finding{{Severity: SevSmall, Title: "x"}}},
{Model: "m", Lens: "b", Findings: []Finding{{Severity: SevMedium, Title: "y"}}},
})
if !strings.Contains(got, "m — minor issues") {
t.Errorf("medium-max should be 'minor issues', got:\n%s", got)
}
// An errored lens is surfaced in the header.
got = Consolidate([]LensResult{
{Model: "m", Lens: "a", Findings: []Finding{{Severity: SevCritical, Title: "boom"}}},
{Model: "m", Lens: "b", Err: context.DeadlineExceeded},
})
if !strings.Contains(got, "blocking issues found") || !strings.Contains(got, "errored") {
t.Errorf("critical + errored lens header wrong:\n%s", got)
}
}
// TestConsolidateAllErrored: a model whose every lens errored must NOT be
// labelled "no issues found" (the gadfly P5 finding).
func TestConsolidateAllErrored(t *testing.T) {
got := Consolidate([]LensResult{
{Model: "m", Lens: "a", Err: context.DeadlineExceeded},
{Model: "m", Lens: "b", Err: context.DeadlineExceeded},
})
if !strings.Contains(got, "m — review incomplete") {
t.Errorf("all-errored model should be 'review incomplete', got:\n%s", got)
}
if strings.Contains(got, "no issues found") {
t.Errorf("all-errored model must not say 'no issues found':\n%s", got)
}
}
// TestConsolidateSwarmCancelled: dropped (unattributed) cells surface a banner.
func TestConsolidateSwarmCancelled(t *testing.T) {
got := Consolidate([]LensResult{
{Err: context.Canceled}, // dropped cell, no model
{Model: "m", Lens: "a", Findings: []Finding{{Severity: SevSmall, Title: "x"}}},
})
if !strings.Contains(got, "swarm cancelled") {
t.Errorf("dropped cells should surface a cancellation banner:\n%s", got)
}
}
+22
View File
@@ -4,5 +4,27 @@ go 1.26.2
require (
gitea.stevedudenhoeffer.com/steve/majordomo v0.0.0-20260626223738-1fd7109a42f3
github.com/google/uuid v1.6.0
github.com/robfig/cron/v3 v3.0.1
golang.org/x/crypto v0.53.0
gopkg.in/yaml.v3 v3.0.1
)
require (
cloud.google.com/go v0.116.0 // indirect
cloud.google.com/go/auth v0.9.3 // indirect
cloud.google.com/go/compute/metadata v0.5.0 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/google/go-cmp v0.6.0 // indirect
github.com/google/s2a-go v0.1.8 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.4 // indirect
github.com/gorilla/websocket v1.5.3 // indirect
go.opencensus.io v0.24.0 // indirect
golang.org/x/net v0.55.0 // indirect
golang.org/x/sys v0.46.0 // indirect
golang.org/x/text v0.38.0 // indirect
google.golang.org/genai v1.59.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20240903143218-8af14fe29dc1 // indirect
google.golang.org/grpc v1.66.2 // indirect
google.golang.org/protobuf v1.34.2 // indirect
)
+130
View File
@@ -1,4 +1,134 @@
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.116.0 h1:B3fRrSDkLRt5qSHWe40ERJvhvnQwdZiHu0bJOpldweE=
cloud.google.com/go v0.116.0/go.mod h1:cEPSRWPzZEswwdr9BxE6ChEn01dWlTaF05LiC2Xs70U=
cloud.google.com/go/auth v0.9.3 h1:VOEUIAADkkLtyfr3BLa3R8Ed/j6w1jTBmARx+wb5w5U=
cloud.google.com/go/auth v0.9.3/go.mod h1:7z6VY+7h3KUdRov5F1i8NDP5ZzWKYmEPO842BgCsmTk=
cloud.google.com/go/compute/metadata v0.5.0 h1:Zr0eK8JbFv6+Wi4ilXAR8FJ3wyNdpxHKJNPos6LTZOY=
cloud.google.com/go/compute/metadata v0.5.0/go.mod h1:aHnloV2TPI38yx4s9+wAZhHykWvVCfu7hQbF+9CWoiY=
gitea.stevedudenhoeffer.com/steve/majordomo v0.0.0-20260626223738-1fd7109a42f3 h1:KYKIFFRsXzbbBJVDa99+Fhy0zxl9G0xV/MCrLipsLL4=
gitea.stevedudenhoeffer.com/steve/majordomo v0.0.0-20260626223738-1fd7109a42f3/go.mod h1:UZLveG17SmENt4sne2RSLIbioix30RZbRIQUzBAnOyY=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da h1:oI5xCqsCo564l8iNU+DwB5epxmsaqB+rhGL0m5jtYqE=
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=
github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=
github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs=
github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w=
github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=
github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8=
github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/s2a-go v0.1.8 h1:zZDs9gcbt9ZPLV0ndSyQk6Kacx2g/X+SKYovpnz3SMM=
github.com/google/s2a-go v0.1.8/go.mod h1:6iNWHTpQ+nfNRN5E00MSdfDwVesa8hhS32PhPO8deJA=
github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/googleapis/enterprise-certificate-proxy v0.3.4 h1:XYIDZApgAnrN1c855gTgghdIA6Stxb52D5RnLI1SLyw=
github.com/googleapis/enterprise-certificate-proxy v0.3.4/go.mod h1:YKe7cfqYXjKGpGvmSg28/fFvhNzinZQm8DGnaburhGA=
github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg=
github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/robfig/cron/v3 v3.0.1 h1:WdRxkvbJztn8LMz/QEvLN5sBU+xKpSqwwUO1Pjr4qDs=
github.com/robfig/cron/v3 v3.0.1/go.mod h1:eQICP3HwyT7UooqI/z+Ov+PtYAWygg1TEWWzGIFLtro=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
go.opencensus.io v0.24.0 h1:y73uSU6J157QMP2kn2r30vwW1A2W2WFwSCGnAVxeaD0=
go.opencensus.io v0.24.0/go.mod h1:vNK8G9p7aAivkbmorf4v+7Hgx+Zs0yY+0fOtgBfjQKo=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.53.0 h1:QZ4Muo8THX6CizN2vPPd5fBGHyogrdK9fG4wLPFUsto=
golang.org/x/crypto v0.53.0/go.mod h1:DNLU434OwVakk9PzuwV8w62mAJpRJL3vsgcfp4Qnsio=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.55.0 h1:bcvxaJn3e1U6InsFWt1JUq1aSjnRxLzT2rtD2KfkDF8=
golang.org/x/net v0.55.0/go.mod h1:L5U2KuzuOe1lY7Z+aWVIKK6qEeJXnXV9yzGA+WCHJww=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.21.0 h1:HLII4xRRTtCRkxYp4HNFF0Js/Og6q2i++KXbg0gHCwM=
golang.org/x/sync v0.21.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.46.0 h1:noSf2Fq6F8DBgS+LysIkx7rIExoNHJsxOAtPp4rthXw=
golang.org/x/sys v0.46.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.38.0 h1:sXmwo9DwP3OK9EZ7PqAdaooSGozfl/3a6/xJcbzPRhE=
golang.org/x/text v0.38.0/go.mod h1:YXZt3QhHUKYT53r2lLKFIVi6Ao1jdzrTR/KQ09qyxF4=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/genai v1.59.0 h1:xp+ydkJFW8hO0hTUaAkr8TrLM9HFP3NYAwFhPd0nDqA=
google.golang.org/genai v1.59.0/go.mod h1:mDdPDFXo1Ats7f1WXVyZgWb/CkMzFWTWJruIMy7hGIU=
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo=
google.golang.org/genproto/googleapis/rpc v0.0.0-20240903143218-8af14fe29dc1 h1:pPJltXNxVzT4pK9yD8vR9X75DaWYYmLGMsEvBfFQZzQ=
google.golang.org/genproto/googleapis/rpc v0.0.0-20240903143218-8af14fe29dc1/go.mod h1:UqMtugtsSgubUsoxbuAoiCXvqvErP7Gf0so0mK9tHxU=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=
google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/grpc v1.33.2/go.mod h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc=
google.golang.org/grpc v1.66.2 h1:3QdXkuq3Bkh7w+ywLdLvM56cmGvQHUMZpiCzt6Rqaoo=
google.golang.org/grpc v1.66.2/go.mod h1:s3/l6xSSCURdVfAnL+TqCNMyTDAGN6+lZeVxnZR128Y=
google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE=
google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo=
google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=
google.golang.org/protobuf v1.34.2 h1:6xV6lTsCfpGD21XK49h7MhtcApnLqkfYgPcdHftf6hg=
google.golang.org/protobuf v1.34.2/go.mod h1:qYOHts0dSfpeUzUFpOMr/WGzszTmLH+DiWniOlNbLDw=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
+618
View File
@@ -0,0 +1,618 @@
// Package llmmeta is the shared meta-LLM helper used by the v12
// authoring tools (summarize, translate, extract_entities, classify).
//
// Why a dedicated package: each of those four tools makes "one fast-tier
// LLM call → typed result", with shared concerns (tier allowlist,
// ledger row, JSON-retry on malformed output). Centralising the pattern
// stops every tool from re-implementing the surrounding bookkeeping and
// keeps the audit trail uniform.
//
// The helper itself does NOT know about the four tools — it just exposes
// a Call(ctx, CallSpec) → CallResult shape. Each tool builds its own
// prompt + parses the typed result. The helper records the meta-call
// ledger row on every call, success or failure.
//
// Concurrency / lanes: the helper resolves the tier to an llm.Model via
// model.ParseModelForContext and uses model.Generate. Lane routing is
// already baked in at the LLM transport layer (see
// pkg/logic/llms/lane_transport.go) so each Generate call automatically
// goes through the right lane without further plumbing. Usage recording
// is automatic too: parsed models are instrumented by pkg/logic/llms,
// so the helper does NOT call model.RecordUsage itself.
//
// Tier allowlist: convar `skills.llm_meta.allowed_tiers` (default
// `["fast"]`) controls which tiers a meta-tool may use. A request for
// a disallowed tier returns error_kind="tier_not_allowed" WITHOUT
// making the call AND WITHOUT recording a ledger row (the call did
// not happen).
//
// Test: helper_test.go covers tier allowed, tier rejected, JSON
// retry path, malformed-twice path, and ledger-row emission semantics.
package llmmeta
import (
"context"
"encoding/json"
"fmt"
"log/slog"
"strings"
"time"
llm "gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"github.com/google/uuid"
"gitea.stevedudenhoeffer.com/steve/executus/model"
)
// MetaCall is the domain row written to skill_llm_meta_calls on every
// helper call.
//
// Why a dedicated table (not skill_run_logs): per-skill token
// aggregation is cleaner with typed columns. Folding meta-calls into
// the generic event log would force a SUM-from-JSON path on every
// dashboard query.
//
// Why the field set is tight (no payload columns): the request bodies
// can be 32KB+. The agent's main run already captures system_prompt
// + user_message in the trace; storing them again here would double
// the audit footprint with no diagnostic value (the meta-call's
// inputs are derivable from the parent run's tool-call args).
type MetaCall struct {
ID string
RunID string
SkillID string
ToolName string
TierUsed string // "fast" / "standard"
ModelUsed string // resolved provider/model
InputTokens int
OutputTokens int
DurationMs int
Success bool
ErrorKind string // empty on success; one of the sentinel kinds otherwise
CreatedAt time.Time
}
// Storage is the narrow surface the helper uses to persist meta-call
// ledger rows. Production wires a thin adapter around the skills GORM
// storage; tests substitute a fake.
//
// Why an interface (vs depending on pkg/logic/skills.Storage): the
// skills package imports skilltools (tool registry); having
// skilltools/llmmeta depend back on skills would form an import
// cycle. A narrow interface mirrored across the boundary is the
// project's standard cycle-break pattern (see KVStorage / FileStorage
// in pkg/skilltools/tools/).
type Storage interface {
RecordMetaCall(ctx context.Context, call MetaCall) error
}
// ConvarReader is the narrow surface the helper uses to read
// `skills.llm_meta.allowed_tiers`. The convar package is database-
// backed; tests pass a static fake.
//
// Why an interface (vs reading convars directly): unit tests want to
// fake the allowlist without spinning up a convar manager.
type ConvarReader interface {
// AllowedTiers returns the list of tier names a meta-tool may use.
// Default ["fast"].
AllowedTiers(ctx context.Context) []string
}
// ConvarReaderFunc adapts a closure into a ConvarReader. Useful in
// production wiring (mort.go) where the underlying access is a
// single line of logic.
type ConvarReaderFunc func(ctx context.Context) []string
// AllowedTiers satisfies ConvarReader.
func (f ConvarReaderFunc) AllowedTiers(ctx context.Context) []string {
if f == nil {
return []string{"fast"}
}
return f(ctx)
}
// Helper makes one fast-tier LLM call with surrounding bookkeeping
// (tier allowlist, JSON retry, ledger row).
//
// Construct once at boot; all four meta-tools share the same Helper.
type Helper struct {
storage Storage
convars ConvarReader
}
// New constructs a Helper. storage MUST be non-nil; passing nil makes
// every Call write a no-op ledger row (callers that need a fully no-op
// helper should instead avoid registering the tool).
//
// convars may be nil — the helper falls back to the default allowlist
// `["fast"]`.
//
// Why a constructor with explicit deps (vs Helper{...} struct
// initialiser): forces the deployment-time decision about which
// dependencies are wired vs nil-safe at the construction call site,
// not at the call site of each tool.
func New(storage Storage, convars ConvarReader) *Helper {
return &Helper{
storage: storage,
convars: convars,
}
}
// CallSpec is the per-call input.
//
// Why every field is explicit (vs builder pattern): the four meta-tools
// each populate the spec in one place; a struct literal at the call
// site is more readable than chained setters.
type CallSpec struct {
// Tier is the tier alias to use ("fast" / "standard"). Empty falls
// back to "fast". Disallowed tiers (per the convar allowlist) cause
// Call to return CallResult{Success: false, ErrorKind:
// "tier_not_allowed"} WITHOUT making the LLM call AND without
// writing a ledger row (the call did not happen).
Tier string
// SystemPrompt is the system message. May be empty.
SystemPrompt string
// UserPrompt is the user message. Required.
UserPrompt string
// MaxOutputTokens caps the response. 0 disables the cap (provider
// default). The helper uses this both to bound the cost estimate
// AND to set llm.WithMaxTokens on the request.
MaxOutputTokens int
// ResponseFormat is "text" or "json". When "json", the helper
// attempts to parse the response into JSON. Other values fall
// through as "text".
ResponseFormat string
// RetryOnMalformedJSON, when true and ResponseFormat=="json",
// retries the call ONCE with a stricter JSON-only prompt prefix
// when the first response fails to parse. Second-failure returns
// CallResult{Success: true, Parsed: nil, ErrorKind:
// "malformed_json"} so callers can fall back to result.Text.
RetryOnMalformedJSON bool
// ToolName is the meta-tool name recorded in the ledger row
// ("summarize", "translate", "extract_entities", "classify"). The
// helper does not branch on this value.
ToolName string
// RunID is the calling skill run ID. Recorded in the ledger row;
// also used by the cost-cap callback to find the running 7-day
// total.
RunID string
// SkillID is the calling skill ID. Recorded in the ledger row;
// passed to the cost-cap callback.
SkillID string
// CallerID is the Discord member ID that triggered the parent
// skill run. Passed to the cost-cap callback so the per-user
// 7-day cap can be evaluated.
CallerID string
}
// CallResult is the per-call output.
//
// Why text + parsed (vs only one): JSON-format calls expose both the
// raw response (in .Text) and the parsed map (in .Parsed). Text-format
// calls leave .Parsed nil. Callers requesting JSON that fails to parse
// twice get .Text populated and ErrorKind="malformed_json" so they
// can fall back to text-mode without an error path.
type CallResult struct {
// Text is the raw response text from the LLM. Populated on every
// successful call (success=true) AND when JSON parsing failed
// twice (success=true, parsed=nil, error_kind="malformed_json").
// Empty on tier_not_allowed rejections (no LLM call happened).
Text string
// Parsed is the JSON-decoded response. nil for text-format calls,
// nil for failed JSON parses, populated for successful JSON
// responses. The interior shape is whatever the LLM returned; the
// caller is responsible for asserting a typed view.
Parsed any
// InputTokens is the tokens billed against the input. 0 when the
// provider didn't surface usage.
InputTokens int
// OutputTokens is the tokens billed against the output. 0 when the
// provider didn't surface usage.
OutputTokens int
// DurationMs is wall-clock duration of the LLM call (or call+retry
// in the JSON-retry case).
DurationMs int
// ModelUsed is the resolved provider/model string ("anthropic/
// claude-haiku-4-5-20251001"). Populated on every actual LLM call;
// empty on tier_not_allowed rejections.
ModelUsed string
// Success reports whether the LLM call returned a usable response.
// True on happy-path AND on malformed-json second-failure (the
// caller can fall back to .Text). False on transport errors,
// tier_not_allowed, llm_unavailable.
Success bool
// ErrorKind, when non-empty, is one of:
// - "tier_not_allowed" → no call, no ledger row
// - "llm_unavailable" → call attempted, ledger row written
// - "malformed_json" → call succeeded but JSON parse failed
ErrorKind string
}
// Sentinel error_kind values for CallResult.ErrorKind.
const (
ErrorKindTierNotAllowed = "tier_not_allowed"
ErrorKindLLMUnavailable = "llm_unavailable"
ErrorKindMalformedJSON = "malformed_json"
)
// Call performs the meta-LLM call and returns a typed CallResult.
//
// Why no error return (vs an error second value): every meaningful
// failure is captured as a CallResult.ErrorKind so the caller's branch
// logic stays single-pathed. Internal transport errors are surfaced
// as ErrorKind=llm_unavailable. The function only returns a non-nil
// error for argument-validation failures (empty UserPrompt) — a
// programmer error the caller would have to fix anyway.
//
// Test: helper_test.go covers all outcomes (tier_not_allowed, happy
// text, happy json, malformed_json retry-pass, malformed_json
// retry-fail, llm_unavailable).
func (h *Helper) Call(ctx context.Context, spec CallSpec) (CallResult, error) {
if strings.TrimSpace(spec.UserPrompt) == "" {
return CallResult{}, fmt.Errorf("llmmeta: user_prompt required")
}
tier := strings.TrimSpace(spec.Tier)
if tier == "" {
tier = "fast"
}
// Tier allowlist: rejected tiers do NOT make the call AND do NOT
// record a ledger row.
if !h.tierAllowed(ctx, tier) {
return CallResult{
Success: false,
ErrorKind: ErrorKindTierNotAllowed,
}, nil
}
resolvedModel := model.ResolveModelName(tier)
// Resolve model. ParseModelForContext attaches the resolved model
// name to ctx (for usage attribution) AND returns the llm.Model
// whose Generate already routes through the lane wrapper.
ctx, model, err := model.ParseModelForContext(ctx, tier)
if err != nil {
// Tier convar mis-set: surface as tier_not_allowed to the
// caller (the agent's recovery path is the same as for an
// admin-disabled tier) but DO record the failure for the
// admin who needs to fix the convar.
h.recordLedger(ctx, MetaCall{
ID: uuid.NewString(),
RunID: spec.RunID,
SkillID: spec.SkillID,
ToolName: spec.ToolName,
TierUsed: tier,
ModelUsed: resolvedModel,
Success: false,
ErrorKind: ErrorKindTierNotAllowed,
CreatedAt: time.Now(),
})
return CallResult{
Success: false,
ErrorKind: ErrorKindTierNotAllowed,
}, nil
}
// First call.
start := time.Now()
systemPrompt := spec.SystemPrompt
userMessage := spec.UserPrompt
opts := []llm.Option{}
if spec.MaxOutputTokens > 0 {
opts = append(opts, llm.WithMaxTokens(spec.MaxOutputTokens))
}
text, usage, llmErr := h.complete(ctx, model, systemPrompt, userMessage, opts)
if llmErr != nil {
duration := int(time.Since(start) / time.Millisecond)
h.recordLedger(ctx, MetaCall{
ID: uuid.NewString(),
RunID: spec.RunID,
SkillID: spec.SkillID,
ToolName: spec.ToolName,
TierUsed: tier,
ModelUsed: resolvedModel,
InputTokens: usage.InputTokens,
OutputTokens: usage.OutputTokens,
DurationMs: duration,
Success: false,
ErrorKind: ErrorKindLLMUnavailable,
CreatedAt: time.Now(),
})
return CallResult{
Success: false,
ErrorKind: ErrorKindLLMUnavailable,
ModelUsed: resolvedModel,
DurationMs: duration,
InputTokens: usage.InputTokens,
OutputTokens: usage.OutputTokens,
}, nil
}
// Determine outcome based on response format.
parsed, parsedOK := tryParseJSON(text, spec.ResponseFormat)
wantJSON := strings.EqualFold(spec.ResponseFormat, "json")
if !wantJSON || parsedOK {
// Happy path (text mode OR JSON mode that parsed first try).
duration := int(time.Since(start) / time.Millisecond)
h.recordLedger(ctx, MetaCall{
ID: uuid.NewString(),
RunID: spec.RunID,
SkillID: spec.SkillID,
ToolName: spec.ToolName,
TierUsed: tier,
ModelUsed: resolvedModel,
InputTokens: usage.InputTokens,
OutputTokens: usage.OutputTokens,
DurationMs: duration,
Success: true,
CreatedAt: time.Now(),
})
return CallResult{
Text: text,
Parsed: parsed,
Success: true,
ModelUsed: resolvedModel,
InputTokens: usage.InputTokens,
OutputTokens: usage.OutputTokens,
DurationMs: duration,
}, nil
}
// JSON requested but first response failed to parse.
if !spec.RetryOnMalformedJSON {
duration := int(time.Since(start) / time.Millisecond)
h.recordLedger(ctx, MetaCall{
ID: uuid.NewString(),
RunID: spec.RunID,
SkillID: spec.SkillID,
ToolName: spec.ToolName,
TierUsed: tier,
ModelUsed: resolvedModel,
InputTokens: usage.InputTokens,
OutputTokens: usage.OutputTokens,
DurationMs: duration,
Success: true,
ErrorKind: ErrorKindMalformedJSON,
CreatedAt: time.Now(),
})
return CallResult{
Text: text,
Success: true,
ErrorKind: ErrorKindMalformedJSON,
ModelUsed: resolvedModel,
InputTokens: usage.InputTokens,
OutputTokens: usage.OutputTokens,
DurationMs: duration,
}, nil
}
// Retry once with stricter JSON-only prompt prefix.
stricterPrompt := "Return ONLY valid JSON. No prose, no markdown fencing.\n\n" + userMessage
text2, usage2, llmErr2 := h.complete(ctx, model, systemPrompt, stricterPrompt, opts)
combinedUsage := Tokens{
InputTokens: usage.InputTokens + usage2.InputTokens,
OutputTokens: usage.OutputTokens + usage2.OutputTokens,
}
duration := int(time.Since(start) / time.Millisecond)
if llmErr2 != nil {
// Retry call itself failed transport-wise. Record the round-
// trip tokens and surface llm_unavailable.
h.recordLedger(ctx, MetaCall{
ID: uuid.NewString(),
RunID: spec.RunID,
SkillID: spec.SkillID,
ToolName: spec.ToolName,
TierUsed: tier,
ModelUsed: resolvedModel,
InputTokens: combinedUsage.InputTokens,
OutputTokens: combinedUsage.OutputTokens,
DurationMs: duration,
Success: false,
ErrorKind: ErrorKindLLMUnavailable,
CreatedAt: time.Now(),
})
return CallResult{
Text: text,
Success: false,
ErrorKind: ErrorKindLLMUnavailable,
ModelUsed: resolvedModel,
InputTokens: combinedUsage.InputTokens,
OutputTokens: combinedUsage.OutputTokens,
DurationMs: duration,
}, nil
}
parsed2, parsedOK2 := tryParseJSON(text2, "json")
if parsedOK2 {
h.recordLedger(ctx, MetaCall{
ID: uuid.NewString(),
RunID: spec.RunID,
SkillID: spec.SkillID,
ToolName: spec.ToolName,
TierUsed: tier,
ModelUsed: resolvedModel,
InputTokens: combinedUsage.InputTokens,
OutputTokens: combinedUsage.OutputTokens,
DurationMs: duration,
Success: true,
CreatedAt: time.Now(),
})
return CallResult{
Text: text2,
Parsed: parsed2,
Success: true,
ModelUsed: resolvedModel,
InputTokens: combinedUsage.InputTokens,
OutputTokens: combinedUsage.OutputTokens,
DurationMs: duration,
}, nil
}
// Second-failure path. Caller can fall back to result.Text.
h.recordLedger(ctx, MetaCall{
ID: uuid.NewString(),
RunID: spec.RunID,
SkillID: spec.SkillID,
ToolName: spec.ToolName,
TierUsed: tier,
ModelUsed: resolvedModel,
InputTokens: combinedUsage.InputTokens,
OutputTokens: combinedUsage.OutputTokens,
DurationMs: duration,
Success: true,
ErrorKind: ErrorKindMalformedJSON,
CreatedAt: time.Now(),
})
return CallResult{
Text: text2,
Success: true,
ErrorKind: ErrorKindMalformedJSON,
ModelUsed: resolvedModel,
InputTokens: combinedUsage.InputTokens,
OutputTokens: combinedUsage.OutputTokens,
DurationMs: duration,
}, nil
}
// Tokens is the input/output token count returned by the LLM round-
// trip. Mirrors llm.Usage's two cost-bearing fields. Exported so
// downstream test code (the four meta-tools' tests, integration
// tests) can use SetCompleteForTest.
type Tokens struct {
InputTokens int
OutputTokens int
}
// CompleteFn is the seam used by tests to fake the LLM round-trip
// without spinning up a real provider. Exported for tests in other
// packages (the four meta-tools live in pkg/skilltools/tools/).
type CompleteFn func(ctx context.Context, model llm.Model, systemPrompt, userMessage string, opts []llm.Option) (string, Tokens, error)
// completeOverride is set in tests via SetCompleteForTest. nil falls
// back to the real model.Generate path.
var completeOverride CompleteFn
// complete is the actual LLM round-trip. Calls model.Generate (which
// already routes through the lane transport wrapper) and returns the
// text + usage + error.
//
// Why not call model.SimpleCall: SimpleCall doesn't surface Usage; we
// need the input/output token counts for the ledger row.
//
// Usage attribution to the per-user / per-skill dashboards is handled
// by the instrumented model that model.ParseModelForContext returns —
// a manual model.RecordUsage here would double-count.
func (h *Helper) complete(ctx context.Context, model llm.Model, systemPrompt, userMessage string, opts []llm.Option) (string, Tokens, error) {
if completeOverride != nil {
return completeOverride(ctx, model, systemPrompt, userMessage, opts)
}
req := llm.Request{
System: systemPrompt,
Messages: []llm.Message{llm.UserText(userMessage)},
}
resp, err := model.Generate(ctx, req, opts...)
if err != nil {
return "", Tokens{}, err
}
usage := Tokens{
InputTokens: resp.Usage.InputTokens,
OutputTokens: resp.Usage.OutputTokens,
}
return resp.Text(), usage, nil
}
// SetCompleteForTest installs a fake completer used by Call. Returns a
// restore function that the test deferes to revert the override.
//
// Why exported (vs in a _test.go file): the four meta-tools' tests live
// in pkg/skilltools/tools/, in a different package than the helper.
// They need a way to fake the LLM without depending on a real model.
func SetCompleteForTest(fn CompleteFn) func() {
prev := completeOverride
completeOverride = fn
return func() { completeOverride = prev }
}
// tierAllowed reports whether the given tier appears in the configured
// allowlist. Empty allowlist defaults to ["fast"].
func (h *Helper) tierAllowed(ctx context.Context, tier string) bool {
var allowed []string
if h.convars != nil {
allowed = h.convars.AllowedTiers(ctx)
}
if len(allowed) == 0 {
allowed = []string{"fast"}
}
for _, t := range allowed {
if strings.EqualFold(strings.TrimSpace(t), tier) {
return true
}
}
return false
}
// recordLedger writes one meta-call row. Storage failures are logged
// at the storage layer; the helper does not propagate them — meta-call
// accounting MUST NOT break user-visible execution.
func (h *Helper) recordLedger(ctx context.Context, call MetaCall) {
if h.storage == nil {
return
}
if err := h.storage.RecordMetaCall(ctx, call); err != nil {
slog.Warn("llmmeta: failed to record ledger row", "err", err)
}
}
// tryParseJSON attempts to decode text as JSON. Returns the parsed
// value (any) and ok=true on success. ok=false on failure or when
// format is not "json".
//
// Why we accept arbitrary JSON shapes (vs requiring an object): the
// extract_entities tool returns objects, but classify returns objects
// with arrays inside. Accepting `any` keeps the helper agnostic to the
// caller's downstream typing.
//
// Tolerance: strips a leading "```json" code fence + matching closing
// fence so the agent can include surrounding markdown without
// breaking parse. The stricter retry prompt explicitly asks for no
// fence; this tolerance is for the first-attempt path.
func tryParseJSON(text, format string) (any, bool) {
if !strings.EqualFold(format, "json") {
return nil, false
}
trimmed := strings.TrimSpace(text)
// Strip optional ```json ... ``` fence.
if strings.HasPrefix(trimmed, "```") {
// Drop opening fence (with or without language tag).
if idx := strings.Index(trimmed, "\n"); idx >= 0 {
trimmed = trimmed[idx+1:]
}
// Drop trailing fence.
if idx := strings.LastIndex(trimmed, "```"); idx >= 0 {
trimmed = trimmed[:idx]
}
trimmed = strings.TrimSpace(trimmed)
}
var parsed any
if err := json.Unmarshal([]byte(trimmed), &parsed); err != nil {
return nil, false
}
return parsed, true
}
+282
View File
@@ -0,0 +1,282 @@
package llmmeta
import (
"context"
"errors"
"strings"
"sync"
"testing"
llm "gitea.stevedudenhoeffer.com/steve/majordomo/llm"
)
// fakeStorage records every MetaCall handed to RecordMetaCall and
// makes them available to tests via the captured slice.
type fakeStorage struct {
mu sync.Mutex
calls []MetaCall
err error
}
func (f *fakeStorage) RecordMetaCall(_ context.Context, call MetaCall) error {
f.mu.Lock()
defer f.mu.Unlock()
f.calls = append(f.calls, call)
return f.err
}
func (f *fakeStorage) snapshot() []MetaCall {
f.mu.Lock()
defer f.mu.Unlock()
out := make([]MetaCall, len(f.calls))
copy(out, f.calls)
return out
}
// TestCall_TierNotAllowed: a tier not in the allowlist returns the
// rejection without recording a ledger row — the call did not happen.
func TestCall_TierNotAllowed(t *testing.T) {
store := &fakeStorage{}
convars := ConvarReaderFunc(func(_ context.Context) []string {
return []string{"fast"}
})
h := New(store, convars)
res, err := h.Call(context.Background(), CallSpec{
Tier: "thinking",
UserPrompt: "hello",
ToolName: "summarize",
})
if err != nil {
t.Fatalf("unexpected err: %v", err)
}
if res.Success {
t.Errorf("expected Success=false")
}
if res.ErrorKind != ErrorKindTierNotAllowed {
t.Errorf("ErrorKind = %q, want %q", res.ErrorKind, ErrorKindTierNotAllowed)
}
if len(store.snapshot()) != 0 {
t.Errorf("expected NO ledger row for tier_not_allowed, got %d", len(store.snapshot()))
}
}
// TestCall_TierAllowedHappyText: a permitted tier yields a successful
// text call AND records a ledger row.
func TestCall_TierAllowedHappyText(t *testing.T) {
store := &fakeStorage{}
convars := ConvarReaderFunc(func(_ context.Context) []string {
return []string{"fast"}
})
h := New(store, convars)
restore := SetCompleteForTest(func(_ context.Context, _ llm.Model, _, _ string, _ []llm.Option) (string, Tokens, error) {
return "summary text here", Tokens{InputTokens: 50, OutputTokens: 12}, nil
})
defer restore()
res, err := h.Call(context.Background(), CallSpec{
Tier: "fast",
UserPrompt: "summarise the following ...",
ToolName: "summarize",
ResponseFormat: "text",
RunID: "run-1",
SkillID: "sk-1",
})
if err != nil {
t.Fatalf("unexpected err: %v", err)
}
if !res.Success {
t.Errorf("expected Success=true; got ErrorKind=%q", res.ErrorKind)
}
if res.Text != "summary text here" {
t.Errorf("Text = %q, want %q", res.Text, "summary text here")
}
if res.InputTokens != 50 || res.OutputTokens != 12 {
t.Errorf("token counts wrong: in=%d out=%d", res.InputTokens, res.OutputTokens)
}
if got := len(store.snapshot()); got != 1 {
t.Fatalf("expected 1 ledger row, got %d", got)
}
row := store.snapshot()[0]
if !row.Success {
t.Errorf("ledger Success = false, want true")
}
if row.ToolName != "summarize" {
t.Errorf("ledger ToolName = %q", row.ToolName)
}
if row.RunID != "run-1" {
t.Errorf("ledger RunID = %q", row.RunID)
}
if row.InputTokens != 50 || row.OutputTokens != 12 {
t.Errorf("ledger token counts wrong: in=%d out=%d",
row.InputTokens, row.OutputTokens)
}
}
// TestCall_JSONFirstAttemptParses: JSON-format request, response is
// valid JSON on first try; result.Parsed populated.
func TestCall_JSONFirstAttemptParses(t *testing.T) {
store := &fakeStorage{}
h := New(store, nil)
restore := SetCompleteForTest(func(_ context.Context, _ llm.Model, _, _ string, _ []llm.Option) (string, Tokens, error) {
return `{"foo":"bar","n":42}`, Tokens{InputTokens: 10, OutputTokens: 5}, nil
})
defer restore()
res, _ := h.Call(context.Background(), CallSpec{
UserPrompt: "extract entities",
ToolName: "extract_entities",
ResponseFormat: "json",
RetryOnMalformedJSON: true,
SkillID: "sk-2",
})
if !res.Success || res.ErrorKind != "" {
t.Fatalf("expected success, got %+v", res)
}
m, ok := res.Parsed.(map[string]any)
if !ok {
t.Fatalf("Parsed not a map: %T %v", res.Parsed, res.Parsed)
}
if m["foo"] != "bar" {
t.Errorf("Parsed[foo] = %v", m["foo"])
}
}
// TestCall_JSONRetryPath: first response is malformed JSON; second
// response (after stricter prompt) parses cleanly.
func TestCall_JSONRetryPath(t *testing.T) {
store := &fakeStorage{}
h := New(store, nil)
calls := 0
restore := SetCompleteForTest(func(_ context.Context, _ llm.Model, _, prompt string, _ []llm.Option) (string, Tokens, error) {
calls++
if calls == 1 {
return "Here is your JSON: {oh no I forgot to format it", Tokens{InputTokens: 8, OutputTokens: 12}, nil
}
// Verify stricter prompt prefix appeared on retry.
if !strings.Contains(prompt, "Return ONLY valid JSON") {
t.Errorf("retry prompt missing stricter prefix: %q", prompt)
}
return `{"key":"value"}`, Tokens{InputTokens: 14, OutputTokens: 6}, nil
})
defer restore()
res, _ := h.Call(context.Background(), CallSpec{
UserPrompt: "extract",
ToolName: "extract_entities",
ResponseFormat: "json",
RetryOnMalformedJSON: true,
})
if !res.Success || res.ErrorKind != "" {
t.Fatalf("expected success, got %+v", res)
}
if calls != 2 {
t.Errorf("expected 2 LLM calls, got %d", calls)
}
m, _ := res.Parsed.(map[string]any)
if m["key"] != "value" {
t.Errorf("Parsed = %v", res.Parsed)
}
// Token counts should reflect both attempts.
if res.InputTokens != 22 || res.OutputTokens != 18 {
t.Errorf("combined tokens wrong: in=%d out=%d", res.InputTokens, res.OutputTokens)
}
}
// TestCall_JSONRetryFailsTwice: second attempt also fails to parse.
// Surfaces ErrorKind=malformed_json AND keeps Success=true so the
// caller can fall back to result.Text.
func TestCall_JSONRetryFailsTwice(t *testing.T) {
store := &fakeStorage{}
h := New(store, nil)
restore := SetCompleteForTest(func(_ context.Context, _ llm.Model, _, _ string, _ []llm.Option) (string, Tokens, error) {
return "still not JSON", Tokens{InputTokens: 10, OutputTokens: 4}, nil
})
defer restore()
res, _ := h.Call(context.Background(), CallSpec{
UserPrompt: "extract",
ToolName: "extract_entities",
ResponseFormat: "json",
RetryOnMalformedJSON: true,
})
if !res.Success {
t.Errorf("expected Success=true (fall-back-to-text), got Success=false")
}
if res.ErrorKind != ErrorKindMalformedJSON {
t.Errorf("ErrorKind = %q, want %q", res.ErrorKind, ErrorKindMalformedJSON)
}
if res.Parsed != nil {
t.Errorf("Parsed = %v, want nil after failed retry", res.Parsed)
}
rows := store.snapshot()
if len(rows) != 1 {
t.Fatalf("expected 1 ledger row, got %d", len(rows))
}
if !rows[0].Success || rows[0].ErrorKind != ErrorKindMalformedJSON {
t.Errorf("ledger row mismatch: %+v", rows[0])
}
}
// TestCall_LLMUnavailable: transport error from the model.Generate
// call is surfaced as ErrorKind=llm_unavailable AND records a ledger
// row.
func TestCall_LLMUnavailable(t *testing.T) {
store := &fakeStorage{}
h := New(store, nil)
restore := SetCompleteForTest(func(_ context.Context, _ llm.Model, _, _ string, _ []llm.Option) (string, Tokens, error) {
return "", Tokens{}, errors.New("network error")
})
defer restore()
res, _ := h.Call(context.Background(), CallSpec{
UserPrompt: "hi",
ToolName: "summarize",
})
if res.Success {
t.Errorf("expected Success=false")
}
if res.ErrorKind != ErrorKindLLMUnavailable {
t.Errorf("ErrorKind = %q, want %q", res.ErrorKind, ErrorKindLLMUnavailable)
}
rows := store.snapshot()
if len(rows) != 1 {
t.Fatalf("expected 1 ledger row, got %d", len(rows))
}
}
// TestCall_EmptyUserPromptErrors: programmer-error guard.
func TestCall_EmptyUserPromptErrors(t *testing.T) {
h := New(&fakeStorage{}, nil)
_, err := h.Call(context.Background(), CallSpec{ToolName: "summarize"})
if err == nil {
t.Fatal("expected error for empty user_prompt")
}
}
// TestCall_JSONWithCodeFenceParses: tolerance for the first-attempt
// response wrapped in a ```json ... ``` fence. The retry path uses a
// stricter prompt; this test pins the first-attempt tolerance so
// callers don't waste a round-trip on a benign formatting wrapper.
func TestCall_JSONWithCodeFenceParses(t *testing.T) {
store := &fakeStorage{}
h := New(store, nil)
restore := SetCompleteForTest(func(_ context.Context, _ llm.Model, _, _ string, _ []llm.Option) (string, Tokens, error) {
return "```json\n{\"x\":1}\n```", Tokens{InputTokens: 5, OutputTokens: 4}, nil
})
defer restore()
res, _ := h.Call(context.Background(), CallSpec{
UserPrompt: "extract",
ToolName: "extract_entities",
ResponseFormat: "json",
RetryOnMalformedJSON: true,
})
if res.ErrorKind != "" {
t.Errorf("unexpected ErrorKind %q (fenced JSON should parse on first attempt)", res.ErrorKind)
}
m, _ := res.Parsed.(map[string]any)
if m["x"] != float64(1) {
t.Errorf("Parsed[x] = %v, want 1", m["x"])
}
}
+21
View File
@@ -0,0 +1,21 @@
package llmmeta
import (
"os"
"testing"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/model"
)
// TestMain configures a minimal model tier table so the helper's
// model.ParseModelForContext("fast"/"standard") resolves. The actual LLM call
// is stubbed per-test via SetCompleteForTest, so these specs are only parsed
// (anthropic registers with an empty key and errors at call time, not parse).
func TestMain(m *testing.M) {
model.Configure(nil, map[string]string{
"fast": "anthropic/claude-haiku-4-5",
"standard": "anthropic/claude-sonnet-4-6",
}, time.Minute)
os.Exit(m.Run())
}
+97
View File
@@ -0,0 +1,97 @@
// Package llms — bench.go: the mort-flavored facade over majordomo's
// health tracker for the `.failover` Discord commands and the failover
// web UI.
//
// Why a facade (vs exposing health.Tracker directly): the admin surfaces
// want the historical shape — a benched-only list with a manual/auto
// flag. majordomo's tracker treats manual benches (Bench) and automatic
// backoffs identically, so the manual marker is kept mort-side.
package model
import (
"sync"
"time"
)
// BenchedModel is one currently-benched model for admin display.
type BenchedModel struct {
// Model is the "provider/model" target key.
Model string
// Until is the end of the bench window.
Until time.Time
// ConsecutiveFails is the failure count since the last success.
ConsecutiveFails int
// Manual reports the bench was placed by an operator (BenchModel)
// rather than the automatic failure threshold.
Manual bool
}
var (
manualMu sync.Mutex
manualBenches = map[string]time.Time{}
)
// ListBenched returns the currently-benched models, manual and automatic,
// from the live health tracker.
func ListBenched() []BenchedModel {
now := time.Now()
pruneManual(now)
var out []BenchedModel
for _, st := range Health().Snapshot() {
if !st.Until.After(now) {
continue
}
out = append(out, BenchedModel{
Model: st.Key,
Until: st.Until,
ConsecutiveFails: st.ConsecutiveFailures,
Manual: isManual(st.Key, st.Until),
})
}
return out
}
// BenchModel manually benches a model spec until the given time. The
// chain executor skips benched targets until the window expires (or
// UnbenchModel clears it).
func BenchModel(model string, until time.Time) {
Health().Bench(model, until)
manualMu.Lock()
manualBenches[model] = until
manualMu.Unlock()
}
// UnbenchModel clears the bench on a model. Returns true when the model
// was actually benched.
func UnbenchModel(model string) bool {
now := time.Now()
wasBenched := Health().BackedOffUntil(model).After(now)
Health().Unbench(model)
manualMu.Lock()
delete(manualBenches, model)
manualMu.Unlock()
return wasBenched
}
// isManual reports whether the bench window for key matches a manual
// bench placed via BenchModel. An automatic backoff that outlives the
// manual window supersedes the marker.
func isManual(key string, until time.Time) bool {
manualMu.Lock()
defer manualMu.Unlock()
manualUntil, ok := manualBenches[key]
return ok && !until.After(manualUntil)
}
// pruneManual drops expired manual markers so the map can't grow
// unbounded across a long uptime.
func pruneManual(now time.Time) {
manualMu.Lock()
defer manualMu.Unlock()
for k, until := range manualBenches {
if !until.After(now) {
delete(manualBenches, k)
}
}
}
+415
View File
@@ -0,0 +1,415 @@
package model
import (
"context"
"encoding/json"
"errors"
"fmt"
"log/slog"
"runtime/debug"
"strings"
"time"
majordomo "gitea.stevedudenhoeffer.com/steve/majordomo"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"github.com/google/uuid"
)
// CallResult captures the result of a single tool call execution.
type CallResult struct {
Name string
Arguments string
Result string
Error error
}
// instrumentedModel decorates a parsed model so every successful Generate
// records token usage to the usage sink automatically. This is the
// single usage chokepoint: ANY call through a model from
// ParseModelRequest / ParseModelForContext is accounted, whether it goes
// through the helpers in this file, the agent loop, or a direct
// model.Generate at a call site.
//
// IMPORTANT: do not call RecordUsage on responses from a parsed model —
// that would double-count. RecordUsage exists for models obtained outside
// this package.
type instrumentedModel struct {
inner llm.Model
}
func (m *instrumentedModel) Generate(ctx context.Context, req llm.Request, opts ...llm.Option) (*llm.Response, error) {
resp, err := m.inner.Generate(ctx, req, opts...)
if err == nil && resp != nil {
recordUsage(ctx, resp)
}
return resp, err
}
func (m *instrumentedModel) Stream(ctx context.Context, req llm.Request, opts ...llm.Option) (llm.Stream, error) {
return m.inner.Stream(ctx, req, opts...)
}
func (m *instrumentedModel) Capabilities() llm.Capabilities { return m.inner.Capabilities() }
// CallAndExecute sends messages to the model with a toolbox, executes any
// tool calls, and returns the results. It performs a single round of
// generation + tool execution (no looping) — multi-step loops belong to
// the agent package.
func CallAndExecute(ctx context.Context, model llm.Model, systemPrompt string, toolbox *llm.Toolbox, messages []llm.Message, opts ...llm.Option) ([]CallResult, string, error) {
req := llm.Request{System: systemPrompt, Messages: messages}
allOpts := make([]llm.Option, 0, len(opts)+1)
if toolbox != nil {
allOpts = append(allOpts, llm.WithToolbox(toolbox))
}
allOpts = append(allOpts, opts...)
startTime := time.Now()
resp, err := model.Generate(ctx, req, allOpts...)
if err != nil {
recordSpanFromWrapper(ctx, systemPrompt, messages, toolbox, nil, nil, startTime, err)
return nil, "", fmt.Errorf("completion failed: %w", err)
}
if len(resp.ToolCalls) == 0 || toolbox == nil {
recordSpanFromWrapper(ctx, systemPrompt, messages, toolbox, resp, nil, startTime, nil)
return nil, resp.Text(), nil
}
var results []CallResult
for _, call := range resp.ToolCalls {
tr := toolbox.Execute(ctx, call)
cr := CallResult{
Name: call.Name,
Arguments: string(call.Arguments),
Result: tr.Content,
}
if tr.IsError {
cr.Error = errors.New(tr.Content)
}
results = append(results, cr)
}
recordSpanFromWrapper(ctx, systemPrompt, messages, toolbox, resp, results, startTime, nil)
return results, resp.Text(), nil
}
// GenerateWith sends messages to the model with an optional system prompt and
// returns structured output parsed into T. T must be a struct. Uses
// majordomo's native structured output (response schema derived from T).
func GenerateWith[T any](ctx context.Context, model llm.Model, systemPrompt string, messages []llm.Message, opts ...llm.Option) (T, error) {
req := llm.Request{System: systemPrompt, Messages: messages}
startTime := time.Now()
// Capture the raw response so the trace span carries usage and the
// concrete serving model even though majordomo.Generate only returns T.
capture := &captureModel{inner: model}
result, err := majordomo.Generate[T](ctx, capture, req, opts...)
resolvedModel := resolvedModelName(ctx, capture.resp)
if tracingEnabled(ctx) {
span := Span{
SpanID: uuid.New().String(),
TraceID: traceIDFromContext(ctx),
Model: resolvedModel,
SystemPrompt: systemPrompt,
Messages: marshalMessages(messages),
DurationMs: time.Since(startTime).Milliseconds(),
StartedAt: startTime,
CompletedAt: time.Now(),
CreatedAt: time.Now(),
}
if capture.resp != nil {
span.InputTokens = capture.resp.Usage.InputTokens
span.OutputTokens = capture.resp.Usage.OutputTokens
}
if err != nil {
span.Error = err.Error()
// Structured-output failure: log loudly so operators can chase
// down a regression (e.g. a model returning prose or fenced
// JSON the decoder rejects) from the trace span alone. The
// error string includes the failing field path on decode
// errors.
if isStructuredOutputParseError(err) {
slog.Warn("llms.GenerateWith: structured-output parse failure",
"model", resolvedModel,
"span_id", span.SpanID,
"trace_id", span.TraceID,
"err", err.Error(),
)
}
} else {
b, _ := json.Marshal(result)
span.ResponseText = string(b)
}
traceSink.WriteSpan(span)
} else if err != nil && isStructuredOutputParseError(err) {
// Tracing disabled: slog.Warn is the only breadcrumb operators get.
slog.Warn("llms.GenerateWith: structured-output parse failure (no trace span)",
"model", resolvedModel,
"err", err.Error(),
)
}
return result, err
}
// captureModel records the last successful response so wrappers that
// only see the decoded result (majordomo.Generate) can still attribute
// usage and tracing.
type captureModel struct {
inner llm.Model
resp *llm.Response
}
func (m *captureModel) Generate(ctx context.Context, req llm.Request, opts ...llm.Option) (*llm.Response, error) {
resp, err := m.inner.Generate(ctx, req, opts...)
if err == nil {
m.resp = resp
}
return resp, err
}
func (m *captureModel) Stream(ctx context.Context, req llm.Request, opts ...llm.Option) (llm.Stream, error) {
return m.inner.Stream(ctx, req, opts...)
}
func (m *captureModel) Capabilities() llm.Capabilities { return m.inner.Capabilities() }
// isStructuredOutputParseError reports whether err looks like a
// structured-output failure from majordomo.Generate — either the decode
// path ("decode structured response") or the empty-response path
// ("structured response from ... is empty"). Used to gate the loud
// slog.Warn so transport errors don't get tagged as parse failures.
func isStructuredOutputParseError(err error) bool {
if err == nil {
return false
}
s := err.Error()
return strings.Contains(s, "decode structured response") ||
strings.Contains(s, "structured response from")
}
// SimpleCall sends a single user message to the model with an optional system
// prompt and returns the text response. No tools involved.
func SimpleCall(ctx context.Context, model llm.Model, systemPrompt string, userMessage string, opts ...llm.Option) (string, error) {
msgs := []llm.Message{llm.UserText(userMessage)}
startTime := time.Now()
resp, err := model.Generate(ctx, llm.Request{System: systemPrompt, Messages: msgs}, opts...)
if err != nil {
recordSpanFromWrapper(ctx, systemPrompt, msgs, nil, nil, nil, startTime, err)
return "", fmt.Errorf("completion failed: %w", err)
}
recordSpanFromWrapper(ctx, systemPrompt, msgs, nil, resp, nil, startTime, nil)
return resp.Text(), nil
}
// RecordUsage records LLM token usage from a successful Generate response.
//
// ONLY call this for models obtained outside this package: models returned
// by ParseModelRequest / ParseModelForContext record usage automatically on
// every Generate, and calling RecordUsage on their responses double-counts.
func RecordUsage(ctx context.Context, resp llm.Response) {
recordUsage(ctx, &resp)
}
// RecordSpan records a trace span for a direct model.Generate() call.
// Call this from modules that invoke model.Generate() directly when they
// want the call traced (usage is already recorded automatically for
// parsed models).
func RecordSpan(ctx context.Context, systemPrompt string, messages []llm.Message, toolbox *llm.Toolbox, resp *llm.Response, callResults []CallResult, startTime time.Time, callErr error) {
recordSpanFromWrapper(ctx, systemPrompt, messages, toolbox, resp, callResults, startTime, callErr)
}
// recordUsage records token usage for one response. The model is
// attributed from the response itself when possible (resp.Model names
// the chain element that actually served the request — more precise than
// the requested spec), falling back to the context attribution set by
// ParseModelForContext.
func recordUsage(ctx context.Context, resp *llm.Response) {
if usageSink == nil || resp == nil {
return
}
u := resp.Usage
if u.InputTokens == 0 && u.OutputTokens == 0 && u.CacheReadTokens == 0 && u.CacheWriteTokens == 0 {
return
}
model := resolvedModelName(ctx, resp)
if model == "unknown" || model == "" {
tool := toolFromContext(ctx)
if tool == "unknown" {
slog.Warn("model usage: recording with both unknown model and tool",
"user", userFromContext(ctx), "stack", string(debug.Stack()))
} else {
slog.Warn("model usage: recording with unknown model — caller should set model.WithModel or use model.ParseModelForContext",
"tool", tool, "user", userFromContext(ctx))
}
}
usageSink.Record(ctx, model, u.InputTokens, u.OutputTokens, u.CacheReadTokens, u.CacheWriteTokens)
}
// resolvedModelName picks the usage/trace attribution name: the serving
// model from the response when present ("provider/model" → "model"),
// else the context's requested model resolved through the tier table.
func resolvedModelName(ctx context.Context, resp *llm.Response) string {
if resp != nil && resp.Model != "" {
name := resp.Model
if idx := strings.Index(name, "/"); idx >= 0 {
name = name[idx+1:]
}
return name
}
return ResolveModelName(modelFromContext(ctx))
}
// tracingEnabled returns true if there's an active trace and tracing is enabled.
func tracingEnabled(ctx context.Context) bool {
if traceSink == nil {
return false
}
return traceIDFromContext(ctx) != ""
}
// recordSpanFromWrapper records a trace span if tracing is active.
func recordSpanFromWrapper(ctx context.Context, systemPrompt string, messages []llm.Message, toolbox *llm.Toolbox, resp *llm.Response, callResults []CallResult, startTime time.Time, callErr error) {
if !tracingEnabled(ctx) {
return
}
now := time.Now()
span := Span{
SpanID: uuid.New().String(),
TraceID: traceIDFromContext(ctx),
Model: resolvedModelName(ctx, resp),
SystemPrompt: systemPrompt,
Messages: marshalMessages(messages),
ToolDefinitions: marshalToolDefs(toolbox),
DurationMs: now.Sub(startTime).Milliseconds(),
StartedAt: startTime,
CompletedAt: now,
CreatedAt: now,
}
if callErr != nil {
span.Error = callErr.Error()
}
if resp != nil {
span.ResponseText = resp.Text()
span.InputTokens = resp.Usage.InputTokens
span.OutputTokens = resp.Usage.OutputTokens
if len(resp.ToolCalls) > 0 {
span.ResponseToolCalls = marshalToolCalls(resp.ToolCalls)
}
}
if len(callResults) > 0 {
span.ToolResults = marshalCallResults(callResults)
}
traceSink.WriteSpan(span)
}
// --- Serialization helpers ---
type jsonMessage struct {
Role string `json:"role"`
Text string `json:"text,omitempty"`
ToolCallID string `json:"tool_call_id,omitempty"`
ImageCount int `json:"image_count,omitempty"`
}
func marshalMessages(msgs []llm.Message) string {
out := make([]jsonMessage, 0, len(msgs))
for _, m := range msgs {
jm := jsonMessage{
Role: string(m.Role),
Text: m.Text(),
}
for _, p := range m.Parts {
if _, ok := p.(llm.ImagePart); ok {
jm.ImageCount++
}
}
if len(m.ToolResults) > 0 {
jm.ToolCallID = m.ToolResults[0].ID
}
out = append(out, jm)
}
b, _ := json.Marshal(out)
return string(b)
}
type jsonToolCall struct {
ID string `json:"id"`
Name string `json:"name"`
Arguments string `json:"arguments"`
}
func marshalToolCalls(calls []llm.ToolCall) string {
out := make([]jsonToolCall, 0, len(calls))
for _, c := range calls {
out = append(out, jsonToolCall{
ID: c.ID,
Name: c.Name,
Arguments: string(c.Arguments),
})
}
b, _ := json.Marshal(out)
return string(b)
}
type jsonCallResult struct {
Name string `json:"name"`
Arguments string `json:"arguments"`
Result string `json:"result"`
Error string `json:"error,omitempty"`
}
func marshalCallResults(results []CallResult) string {
out := make([]jsonCallResult, 0, len(results))
for _, r := range results {
jr := jsonCallResult{
Name: r.Name,
Arguments: r.Arguments,
Result: r.Result,
}
if r.Error != nil {
jr.Error = r.Error.Error()
}
out = append(out, jr)
}
b, _ := json.Marshal(out)
return string(b)
}
type jsonToolDef struct {
Name string `json:"name"`
Description string `json:"description"`
}
func marshalToolDefs(tb *llm.Toolbox) string {
if tb == nil {
return ""
}
tools := tb.Tools()
if len(tools) == 0 {
return ""
}
out := make([]jsonToolDef, 0, len(tools))
for _, t := range tools {
out = append(out, jsonToolDef{
Name: t.Name,
Description: t.Description,
})
}
b, _ := json.Marshal(out)
return string(b)
}
+474
View File
@@ -0,0 +1,474 @@
// V15.4 — Ollama Cloud dynamic context-length sync.
//
// Why: the static map in context_limits.go has to be hand-maintained
// for every new Ollama Cloud model. Cloud ships new models monthly,
// and a missing entry silently disables compaction for runs on that
// model (compactionThresholdForModel returns 0 on MaxContextTokens
// miss). Dynamic sync removes the maintenance burden and means new
// cloud models work out-of-the-box.
//
// How: at boot, mort kicks off a CloudOllamaLimitCache.RefreshAll in a
// background goroutine. RefreshAll calls /api/tags to list every
// available cloud model, then concurrently calls /api/show for each
// to extract `<family>.context_length` from the response's model_info
// map. The cache is consulted by the executor's
// compactionThresholdForModel via the cache-aware
// MaxContextTokensWithCache helper.
//
// Periodic refresh: a daily ticker re-runs RefreshAll so newly
// released models surface without a mort restart. The interval is
// intentionally not configurable — cloud model context lengths don't
// change for a given tag (only the tag pointer can move, e.g. :cloud
// → larger model), so daily is conservative.
package model
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"log/slog"
"net/http"
"strings"
"sync"
"time"
)
// defaultCloudEndpoint is the public Ollama Cloud base URL. Override
// in tests via NewCloudOllamaLimitCache's endpoint arg.
const defaultCloudEndpoint = "https://ollama.com"
// CloudOllamaLimitCache holds context-length values for Ollama Cloud
// models, populated dynamically via /api/tags + /api/show. Construct
// with NewCloudOllamaLimitCache. Safe for concurrent use.
//
// Empty when OLLAMA_API_KEY is unset — Refresh returns a clear error
// and the cache stays empty. Lookups return (0, false) and callers
// fall back to the static map / disabled compaction.
type CloudOllamaLimitCache struct {
mu sync.RWMutex
limit map[string]int
negative map[string]time.Time // model → fetch-failure time (for TTL)
endpoint string
apiKey string
httpClient *http.Client
// refreshConcurrency caps the number of concurrent /api/show calls
// during RefreshAll. Default 8 — enough to finish a ~50-model
// catalog in well under a minute without hammering Cloud.
refreshConcurrency int
// negativeTTL is how long a /api/show miss is cached before we
// retry. Prevents hammering Cloud on a typo or recently-removed
// model. Default 10 minutes.
negativeTTL time.Duration
}
// NewCloudOllamaLimitCache constructs a fresh cache. apiKey can be
// empty — RefreshAll then returns an error and the cache stays empty.
// endpoint defaults to https://ollama.com when empty. httpClient
// defaults to a 15s-timeout client.
func NewCloudOllamaLimitCache(endpoint, apiKey string, httpClient *http.Client) *CloudOllamaLimitCache {
if strings.TrimSpace(endpoint) == "" {
endpoint = defaultCloudEndpoint
}
endpoint = strings.TrimRight(endpoint, "/")
if httpClient == nil {
httpClient = &http.Client{Timeout: 15 * time.Second}
}
return &CloudOllamaLimitCache{
limit: make(map[string]int),
negative: make(map[string]time.Time),
endpoint: endpoint,
apiKey: apiKey,
httpClient: httpClient,
refreshConcurrency: 8,
negativeTTL: 10 * time.Minute,
}
}
// SetNegativeTTL overrides the negative-cache lifetime. Tests use this
// to control retry behaviour without sleeping.
func (c *CloudOllamaLimitCache) SetNegativeTTL(d time.Duration) {
if c == nil || d < 0 {
return
}
c.mu.Lock()
c.negativeTTL = d
c.mu.Unlock()
}
// Lookup returns the cached context length for an Ollama Cloud model
// name (e.g. "qwen3.5:cloud", "qwen3-coder:480b"). Returns (0, false)
// on miss. Lookup never makes HTTP calls — it's the hot path consulted
// by the executor before every run.
//
// modelName accepts either the bare model:tag form or the prefixed
// "ollama-cloud/model:tag" form; the prefix is stripped.
func (c *CloudOllamaLimitCache) Lookup(modelName string) (int, bool) {
if c == nil {
return 0, false
}
key := stripCloudPrefix(modelName)
if key == "" {
return 0, false
}
c.mu.RLock()
defer c.mu.RUnlock()
v, ok := c.limit[key]
return v, ok
}
// Size returns the number of cached entries. Useful for logging /
// health checks.
func (c *CloudOllamaLimitCache) Size() int {
if c == nil {
return 0
}
c.mu.RLock()
defer c.mu.RUnlock()
return len(c.limit)
}
// LookupOrFetch returns the cached context length OR, on miss, makes a
// single /api/show call to populate the cache. Negative results
// (model not found, /api/show returns no context_length) are cached
// for negativeTTL to prevent hammering Cloud on a typo. Returns
// (0, false) when the model is genuinely unknown and (size, true) on
// any successful resolve.
//
// Why this exists: Ollama Cloud's /api/tags lists canonical model
// names only (e.g. "qwen3.5:397b") but accepts aliases on /api/show
// (e.g. "qwen3.5:cloud" → same 397b model). The boot-time RefreshAll
// only sees the canonical names, so common aliases miss the cache.
// LookupOrFetch fills the gap.
//
// The cache is therefore self-healing: any unknown model gets one
// live /api/show call, the result lands in the cache, and subsequent
// runs hit immediately. Periodic RefreshAll overwrites everything
// with the canonical-name results but additionally-fetched aliases
// linger as positive entries.
func (c *CloudOllamaLimitCache) LookupOrFetch(ctx context.Context, modelName string) (int, bool) {
if c == nil {
return 0, false
}
key := stripCloudPrefix(modelName)
if key == "" {
return 0, false
}
// Fast path: positive hit.
c.mu.RLock()
if v, ok := c.limit[key]; ok {
c.mu.RUnlock()
return v, true
}
// Negative cache check.
if t, ok := c.negative[key]; ok && time.Since(t) < c.negativeTTL {
c.mu.RUnlock()
return 0, false
}
c.mu.RUnlock()
// No API key configured → can't fetch. Don't write a negative
// entry (when the key gets configured later we want the next call
// to re-try immediately).
if strings.TrimSpace(c.apiKey) == "" {
return 0, false
}
// Slow path: live /api/show.
n, err := c.fetchContextLength(ctx, key)
if err != nil || n <= 0 {
slog.Debug("cloud limit cache: lazy fetch miss",
"model", key, "err", err)
c.mu.Lock()
c.negative[key] = time.Now()
c.mu.Unlock()
return 0, false
}
c.set(key, n)
slog.Info("cloud limit cache: lazy fetch hit", "model", key, "context_length", n)
return n, true
}
// set stores a context length. n <= 0 is a no-op.
func (c *CloudOllamaLimitCache) set(modelName string, n int) {
if c == nil || n <= 0 {
return
}
key := stripCloudPrefix(modelName)
if key == "" {
return
}
c.mu.Lock()
c.limit[key] = n
c.mu.Unlock()
}
// RefreshAll queries /api/tags then concurrently calls /api/show for
// every listed model, populating the cache. Returns the number of
// models successfully cached and the first error encountered (a
// /api/tags failure aborts; individual /api/show failures are logged
// but don't abort the whole refresh).
//
// Safe to call repeatedly. Cache entries are overwritten with the
// fresh values; entries for models that have been removed from Cloud
// are NOT pruned (cheap to keep; pruning risks dropping an entry just
// before a run that needs it).
func (c *CloudOllamaLimitCache) RefreshAll(ctx context.Context) (int, error) {
if c == nil {
return 0, fmt.Errorf("cloud limit cache: nil receiver")
}
if strings.TrimSpace(c.apiKey) == "" {
return 0, fmt.Errorf("cloud limit cache: OLLAMA_API_KEY unset")
}
tags, err := c.fetchTags(ctx)
if err != nil {
return 0, fmt.Errorf("cloud limit cache: /api/tags: %w", err)
}
concurrency := c.refreshConcurrency
if concurrency <= 0 {
concurrency = 8
}
sem := make(chan struct{}, concurrency)
var wg sync.WaitGroup
var (
mu sync.Mutex
success int
)
for _, name := range tags {
name := name
wg.Add(1)
sem <- struct{}{}
go func() {
defer wg.Done()
defer func() { <-sem }()
ctxLen, ferr := c.fetchContextLength(ctx, name)
if ferr != nil {
slog.Debug("cloud limit cache: /api/show miss",
"model", name, "err", ferr)
return
}
c.set(name, ctxLen)
mu.Lock()
success++
mu.Unlock()
}()
}
wg.Wait()
slog.Info("cloud limit cache: refresh complete",
"models_total", len(tags), "cached", success)
return success, nil
}
// StartPeriodicRefresh runs RefreshAll once immediately, then on every
// interval tick. Cancellation via ctx stops the loop. Logs each
// outcome; never returns an error to the caller (this is a background
// task — failures are warnings, not show-stoppers).
//
// Typical usage: a goroutine spawned at mort boot.
//
// go cache.StartPeriodicRefresh(ctx, 24*time.Hour)
func (c *CloudOllamaLimitCache) StartPeriodicRefresh(ctx context.Context, interval time.Duration) {
if c == nil {
return
}
if interval <= 0 {
interval = 24 * time.Hour
}
doOne := func() {
n, err := c.RefreshAll(ctx)
if err != nil {
slog.Warn("cloud limit cache: refresh failed",
"err", err, "cached_size", c.Size())
return
}
slog.Info("cloud limit cache: refreshed",
"newly_cached_or_updated", n, "cached_size", c.Size())
}
doOne()
t := time.NewTicker(interval)
defer t.Stop()
for {
select {
case <-ctx.Done():
return
case <-t.C:
doOne()
}
}
}
// fetchTags calls GET /api/tags and returns the model names.
func (c *CloudOllamaLimitCache) fetchTags(ctx context.Context) ([]string, error) {
url := c.endpoint + "/api/tags"
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return nil, err
}
c.applyAuth(req)
resp, err := c.httpClient.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, err := readCapped(resp.Body)
if err != nil {
return nil, err
}
if resp.StatusCode/100 != 2 {
return nil, fmt.Errorf("status %d: %s", resp.StatusCode, truncate(body, 400))
}
var parsed struct {
Models []struct {
Name string `json:"name"`
Model string `json:"model"`
} `json:"models"`
}
if err := json.Unmarshal(body, &parsed); err != nil {
return nil, fmt.Errorf("parse /api/tags: %w", err)
}
out := make([]string, 0, len(parsed.Models))
for _, m := range parsed.Models {
name := m.Name
if name == "" {
name = m.Model
}
if name == "" {
continue
}
out = append(out, name)
}
return out, nil
}
// fetchContextLength calls POST /api/show for a model and extracts
// the largest *.context_length value from model_info. Returns the
// length and nil on success; (0, err) on any failure.
//
// Why "largest" rather than family-keyed: the family field in the
// /api/show response is sometimes empty or doesn't match the
// model_info key prefix exactly (Ollama Cloud returns the
// architecture as the prefix, which usually but not always matches
// `family`). Scanning for any `*.context_length` is robust.
func (c *CloudOllamaLimitCache) fetchContextLength(ctx context.Context, modelName string) (int, error) {
url := c.endpoint + "/api/show"
body, _ := json.Marshal(map[string]string{"name": modelName})
req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, bytes.NewReader(body))
if err != nil {
return 0, err
}
req.Header.Set("Content-Type", "application/json")
c.applyAuth(req)
resp, err := c.httpClient.Do(req)
if err != nil {
return 0, err
}
defer resp.Body.Close()
respBody, err := readCapped(resp.Body)
if err != nil {
return 0, err
}
if resp.StatusCode/100 != 2 {
return 0, fmt.Errorf("status %d: %s", resp.StatusCode, truncate(respBody, 400))
}
n, err := parseContextLengthJSON(respBody)
if err != nil {
return 0, err
}
return n, nil
}
// parseContextLengthJSON extracts the largest `*.context_length` int
// from an /api/show response body. Exported-ish (lowercase but tested
// in the same package) so the unit test can exercise it without
// spinning up an httptest server.
func parseContextLengthJSON(body []byte) (int, error) {
var parsed struct {
ModelInfo map[string]any `json:"model_info"`
}
if err := json.Unmarshal(body, &parsed); err != nil {
return 0, fmt.Errorf("parse: %w", err)
}
best := 0
for k, v := range parsed.ModelInfo {
if !strings.HasSuffix(k, ".context_length") {
continue
}
n := toInt(v)
if n > best {
best = n
}
}
if best <= 0 {
return 0, fmt.Errorf("no context_length in model_info")
}
return best, nil
}
// toInt coerces a JSON-decoded value to int. Handles float64 (the
// json default) and json.Number; returns 0 for anything else.
func toInt(v any) int {
switch x := v.(type) {
case float64:
return int(x)
case int:
return x
case int64:
return int(x)
case json.Number:
if n, err := x.Int64(); err == nil {
return int(n)
}
}
return 0
}
// applyAuth sets the Bearer token when an API key is configured.
func (c *CloudOllamaLimitCache) applyAuth(req *http.Request) {
if strings.TrimSpace(c.apiKey) == "" {
return
}
req.Header.Set("Authorization", "Bearer "+strings.TrimSpace(c.apiKey))
}
// stripCloudPrefix strips an "ollama-cloud/" prefix (and surrounding
// whitespace). Returns the bare model:tag form.
func stripCloudPrefix(s string) string {
s = strings.TrimSpace(s)
if strings.HasPrefix(s, "ollama-cloud/") {
s = s[len("ollama-cloud/"):]
}
return s
}
// truncate caps a byte slice for error messages.
func truncate(b []byte, n int) string {
if len(b) <= n {
return string(b)
}
return string(b[:n]) + "...(truncated)"
}
// maxLimitCacheResponseBytes bounds the ollama.com limit-cache HTTP responses
// (/api/tags, /api/show) so a misbehaving endpoint can't stream an unbounded
// body before the 15s timeout fires. 1 MiB is far above any real response.
const maxLimitCacheResponseBytes = 1 << 20
// readCapped reads up to maxLimitCacheResponseBytes from r and returns a clear
// error if the response EXCEEDS the cap — rather than silently truncating (as a
// bare io.LimitReader does) and letting downstream json.Unmarshal fail with an
// opaque "unexpected end of JSON input". It reads one extra byte to detect the
// overflow.
func readCapped(r io.Reader) ([]byte, error) {
body, err := io.ReadAll(io.LimitReader(r, maxLimitCacheResponseBytes+1))
if err != nil {
return nil, err
}
if len(body) > maxLimitCacheResponseBytes {
return nil, fmt.Errorf("cloud_sync: response exceeded %d bytes", maxLimitCacheResponseBytes)
}
return body, nil
}
+224
View File
@@ -0,0 +1,224 @@
// V15.2 — per-model context-window limits.
//
// Why: agents need to know when they're about to blow the model's
// max-input cap so they can compact stale tool results out of the
// message history. Pre-15.2 the agent loop had no awareness; a long
// research run that accumulated dozens of HTTP tool results would
// hit Ollama's HTTP 400 "prompt is too long" or Anthropic's similar
// error mid-run with no graceful degradation.
//
// Coverage:
// - Anthropic Claude 4.x (200K default; 1M when the model ID
// includes the "[1m]" suffix per llms.tier reload conventions)
// - OpenAI GPT-4.x / o-series (128K)
// - Gemini 2.x (1M-2M, model-specific)
// - Ollama Cloud (model-specific; hardcoded per known model)
// - Local Ollama: queries `/api/show` once at first use, caches
//
// Returns (0, false) for unknown models — callers should treat
// "unknown" as "don't budget" (the agent's existing iteration cap +
// timeout are the fallback safety nets).
package model
import (
"context"
"strings"
"sync"
)
// MaxContextTokens returns the model's max INPUT context-window size
// in tokens. Output / response tokens are NOT included — most models
// share input + output budget but cap them separately, and the practical
// concern is "how big can my prompt get before the model rejects".
//
// modelID accepts both the bare model name (`claude-sonnet-4-6`) and
// the prefixed form (`anthropic/claude-sonnet-4-6` or
// `ollama-cloud/qwen3-coder:480b`). The prefix is stripped before lookup.
//
// Returns (limit, true) on a known model; (0, false) otherwise.
//
// This function is pure (no I/O). For Ollama Cloud models that aren't
// in the static map, use MaxContextTokensWithCache which consults a
// CloudOllamaLimitCache populated at boot from /api/tags + /api/show.
func MaxContextTokens(modelID string) (int, bool) {
id := normalizeModelID(modelID)
if v, ok := staticContextLimits[id]; ok {
return v, true
}
// Anthropic 1M-context variant marker. Mort's llms tier system
// uses a `[1m]` suffix on the model ID (e.g.
// `claude-opus-4-7[1m]`) to opt into Anthropic's 1M beta context.
if strings.HasSuffix(id, "[1m]") {
return 1_000_000, true
}
// Local-ollama dynamic lookup is wired separately so it can
// query the daemon's /api/show endpoint. The static map covers
// known cloud models.
return 0, false
}
// MaxContextTokensWithCache is the cache-aware variant of
// MaxContextTokens. It tries the static map first; on miss, if the
// model is an Ollama Cloud spec (the `ollama-cloud/` prefix), it
// consults the supplied CloudOllamaLimitCache. Pass nil cache for
// static-only behaviour (equivalent to MaxContextTokens).
//
// This function never makes HTTP calls — the cache must be
// pre-populated (typically via cache.RefreshAll at boot). Callers in
// the hot path can rely on a single map lookup per call. Prefer
// MaxContextTokensResolving when a context is available — it makes a
// single /api/show call to fill the cache on miss, which is essential
// for Cloud aliases that /api/tags doesn't enumerate (e.g. :cloud).
func MaxContextTokensWithCache(modelID string, cloud *CloudOllamaLimitCache) (int, bool) {
if v, ok := MaxContextTokens(modelID); ok {
return v, true
}
if cloud == nil {
return 0, false
}
// Only ollama-cloud/* models are eligible for the cache.
id := strings.TrimSpace(modelID)
if !strings.HasPrefix(id, "ollama-cloud/") {
// Also allow bare model:tag form when the caller has already
// stripped the prefix (some test paths).
if strings.Contains(id, "/") {
return 0, false
}
}
return cloud.Lookup(id)
}
// MaxContextTokensResolving is the cache-aware variant that ALSO
// performs a live /api/show fetch on cache miss (with negative caching
// to prevent thrash). Use this in run-setup paths where one HTTP call
// per unseen model is acceptable — typically the skill executor's
// compaction threshold computation. The fetched result is cached for
// future calls, so subsequent runs hit the in-memory map.
//
// Falls back to the static-only path when the model isn't an
// ollama-cloud/* spec or cache is nil. ctx cancellation aborts the
// fetch and returns (0, false) without writing a negative entry.
func MaxContextTokensResolving(ctx context.Context, modelID string, cloud *CloudOllamaLimitCache) (int, bool) {
if v, ok := MaxContextTokens(modelID); ok {
return v, true
}
if cloud == nil {
return 0, false
}
id := strings.TrimSpace(modelID)
if !strings.HasPrefix(id, "ollama-cloud/") {
if strings.Contains(id, "/") {
return 0, false
}
}
return cloud.LookupOrFetch(ctx, id)
}
// normalizeModelID strips provider prefix and reasoning suffix so a
// lookup keyed on the base name works regardless of caller form.
//
// Examples:
// - "anthropic/claude-sonnet-4-6" → "claude-sonnet-4-6"
// - "ollama-cloud/qwen3-coder:480b" → "qwen3-coder:480b"
// - "claude-opus-4-7:high" → "claude-opus-4-7"
func normalizeModelID(id string) string {
id = strings.TrimSpace(id)
if idx := strings.Index(id, "/"); idx >= 0 {
id = id[idx+1:]
}
// Strip :low/:medium/:high reasoning effort suffix used by some
// OpenAI / Anthropic clients.
for _, suffix := range []string{":low", ":medium", ":high"} {
if strings.HasSuffix(id, suffix) {
id = id[:len(id)-len(suffix)]
break
}
}
return id
}
// staticContextLimits is the source of truth for known cloud models.
// Add new entries when adding a model to the llms tier system.
//
// CRITICAL: keep these in sync with the actual provider docs. A wrong
// number here causes EITHER premature compaction (too low, degrades
// agent quality unnecessarily) OR HTTP 400 mid-run (too high). The
// 410K-token failure on `qwen3-coder:480b` is the kind of bug a
// mistyped value would reintroduce.
var staticContextLimits = map[string]int{
// Anthropic Claude 4.x — default 200K input. 1M variant via
// `[1m]` suffix handled in MaxContextTokens above.
"claude-opus-4-7": 200_000,
"claude-opus-4-6": 200_000,
"claude-opus-4-5": 200_000,
"claude-sonnet-4-6": 200_000,
"claude-sonnet-4-5": 200_000,
"claude-haiku-4-5": 200_000,
"claude-haiku-4-5-20251001": 200_000,
// OpenAI GPT-4.x / o-series — 128K input.
"gpt-4o": 128_000,
"gpt-4o-mini": 128_000,
"gpt-4-turbo": 128_000,
"o1": 200_000,
"o1-mini": 128_000,
"o3-mini": 200_000,
"gpt-5": 400_000,
"gpt-5-mini": 400_000,
// Gemini — varies dramatically by model.
"gemini-2.5-pro": 2_000_000,
"gemini-2.5-flash": 1_000_000,
"gemini-2.5-flash-lite": 1_000_000,
"gemini-1.5-pro": 2_000_000,
"gemini-1.5-flash": 1_000_000,
// Ollama Cloud (turbo). Limits per https://ollama.com/cloud/models
// — verified against the Ollama API show output for each model.
// Update when Ollama publishes new models or extends contexts.
"qwen3-coder:480b": 262_144, // 262K — matches the v15.2 trace
"qwen3:235b": 262_144,
"qwen3:32b": 131_072,
"qwen2.5:72b": 131_072,
"gpt-oss:120b": 131_072,
"gpt-oss:20b": 131_072,
"deepseek-v3.1:671b": 131_072,
"glm-4.6:355b": 131_072,
"kimi-k2:1t": 262_144,
"llama4:scout": 10_000_000, // Llama 4 Scout claims 10M
"llama4:maverick": 1_000_000,
}
// LocalOllamaLimitCache holds the resolved /api/show context_length per
// local-ollama model. Populated on first lookup; never invalidated
// (changing num_ctx requires an ollama restart anyway). Process-wide,
// no per-tenant scoping needed.
type LocalOllamaLimitCache struct {
mu sync.RWMutex
limit map[string]int
}
// NewLocalOllamaLimitCache constructs a fresh cache.
func NewLocalOllamaLimitCache() *LocalOllamaLimitCache {
return &LocalOllamaLimitCache{limit: make(map[string]int)}
}
// Get returns the cached limit or (0, false) when unseen. The caller
// is expected to follow up with a lookup against the live daemon.
func (c *LocalOllamaLimitCache) Get(model string) (int, bool) {
c.mu.RLock()
defer c.mu.RUnlock()
v, ok := c.limit[model]
return v, ok
}
// Set records a resolved limit. Idempotent; no-op when value is <= 0.
func (c *LocalOllamaLimitCache) Set(model string, n int) {
if n <= 0 {
return
}
c.mu.Lock()
defer c.mu.Unlock()
c.limit[model] = n
}
+97
View File
@@ -0,0 +1,97 @@
package model
import (
"testing"
"time"
)
// mapSource is a tiny config.Source for tests: a key->value map, defaults
// returned for misses.
type mapSource map[string]string
func (m mapSource) String(k, d string) string {
if v, ok := m[k]; ok {
return v
}
return d
}
func (m mapSource) Int(string, int) int { panic("unused") }
func (m mapSource) Float(string, float64) float64 { panic("unused") }
func (m mapSource) Bool(string, bool) bool { panic("unused") }
// TestConfigureTierResolution covers the convar->config.Source inversion: the
// host supplies a tier table (names + fallbacks) and a live config source; the
// config value overrides the fallback, and an absent key falls back.
func TestConfigureTierResolution(t *testing.T) {
Configure(
mapSource{"model.tier.fast": "anthropic/claude-haiku-4-5"},
map[string]string{"fast": "openai/gpt-4o-mini", "thinking": "anthropic/claude-opus-4-8"},
time.Minute,
)
defer Configure(nil, nil, 0) // reset package global
if !IsTierName("fast") || !IsTierName("thinking") {
t.Fatal("configured tiers should be registered")
}
if IsTierName("nope") {
t.Fatal("unknown tier must not report as a tier")
}
if names := TierNames(); len(names) != 2 || names[0] != "fast" || names[1] != "thinking" {
t.Fatalf("TierNames = %v, want sorted [fast thinking]", names)
}
// config value overrides the host fallback
if spec, _, ok := defaultResolver.Resolve("fast"); !ok || spec != "anthropic/claude-haiku-4-5" {
t.Fatalf("fast resolve = %q ok=%v; config should override fallback", spec, ok)
}
// fallback used when config has no override for the key
if spec, _, ok := defaultResolver.Resolve("thinking"); !ok || spec != "anthropic/claude-opus-4-8" {
t.Fatalf("thinking resolve = %q ok=%v; should use fallback", spec, ok)
}
// unknown tier
if _, _, ok := defaultResolver.Resolve("nope"); ok {
t.Fatal("Resolve of unknown tier should be ok=false")
}
}
// TestReasoningSuffixOnTier verifies the reasoning-suffix dialect survives the
// move: a tier whose spec carries ":high" yields the bare spec + level "high".
func TestReasoningSuffixOnTier(t *testing.T) {
Configure(nil, map[string]string{"thinking": "anthropic/claude-opus-4-8:high"}, time.Minute)
defer Configure(nil, nil, 0)
spec, level, ok := defaultResolver.Resolve("thinking")
if !ok {
t.Fatal("thinking should resolve")
}
if spec != "anthropic/claude-opus-4-8" {
t.Errorf("spec = %q, want suffix stripped", spec)
}
if level != "high" {
t.Errorf("reasoning level = %q, want high", level)
}
}
func TestValidateTierValueRejectsNestedTier(t *testing.T) {
Configure(nil, map[string]string{"fast": "x/y"}, time.Minute)
defer Configure(nil, nil, 0)
if err := ValidateTierValue("fast,a/b"); err == nil {
t.Error("a chain containing a tier alias must be rejected")
}
if err := ValidateTierValue("a/b,c/d"); err != nil {
t.Errorf("a chain of concrete specs must validate, got %v", err)
}
}
// TestSinksDefaultNil verifies usage/trace recording is inert with no sinks
// installed (the light-host default).
func TestSinksDefaultNil(t *testing.T) {
SetUsageSink(nil)
SetTraceSink(nil)
if TraceSinkActive() {
t.Error("no trace sink should mean inactive")
}
// recordUsage must be a no-op (no panic) with a nil sink.
recordUsage(WithModel(t.Context(), "x"), nil)
}
+91
View File
@@ -0,0 +1,91 @@
// Package llms — lane_mapping.go: maps a model spec to a stable lane
// name. Pure data + a single function; no dependency on the registry,
// no provider wrapping. Kept separate from lane_transport.go so the
// mapping table can be committed and reviewed in isolation, and so
// admin / webui code that just wants to *display* lane assignments
// doesn't drag in the transport machinery.
//
// Why a fixed table: provider concurrency caps differ — Ollama Pro is
// 3 connections, Anthropic Claude has higher per-tier limits, etc.
// Each provider gets its own lane name so they can be configured
// independently via convars (lanes.<name>.max_concurrent). Lane names
// are user-facing (admin dashboard + convar key suffixes) and need to
// stay stable across deploys; an env-overridable map adds complexity
// for no current benefit.
//
// Test: lane_transport_test.go covers TestLaneFor_Mapping.
package model
import "strings"
// Lane name constants. Defined as exported strings so admin code (.skill
// admin set-lane <skill> <lane>), webui dropdowns, and convar consumers
// share a single canonical spelling.
const (
// LaneOllama covers ollama-cloud/* (and any future ollama/* local).
// The local ollama instance is on the same physical resource as
// the cloud account from mort's perspective — the connection cap
// should apply jointly.
LaneOllama = "ollama"
// LaneAnthropicThinking is the lane for Anthropic models in
// extended-thinking mode. Separated from default because thinking
// requests hold connections longer and can starve faster lanes
// when multiplexed.
LaneAnthropicThinking = "anthropic-thinking"
// LaneAnthropicDefault is the lane for non-thinking Anthropic
// requests (haiku, sonnet, opus without -thinking-).
LaneAnthropicDefault = "anthropic-default"
// LaneM1 is the lane for m1/* models (foreman-style router
// pointing at a dedicated local instance). Separated from the
// ollama lane because m1 targets a distinct host with its own
// connection budget.
LaneM1 = "m1"
// LaneLLMDefault is the catch-all lane for any provider/model
// combination not explicitly mapped above.
LaneLLMDefault = "llm-default"
)
// LaneFor returns the lane name for the given model spec. Mapping:
//
// ollama-cloud/* → "ollama" (Pro account: 3 connections)
// anthropic/*-thinking-* → "anthropic-thinking"
// anthropic/* → "anthropic-default"
// (anything else) → "llm-default"
//
// Tier aliases (fast/standard/thinking) flow through this function as
// the resolver's expanded provider/model spec, so callers don't need
// to think about tier indirection. Empty input falls through to
// LaneLLMDefault rather than panicking — defensive against unset
// model specs in edge-case test wiring.
//
// Substring match for "-thinking-" keeps future Anthropic naming
// variations classified correctly without churning this table on
// every model release.
func LaneFor(modelSpec string) string {
s := strings.TrimSpace(modelSpec)
if strings.HasPrefix(s, "ollama-cloud/") {
return LaneOllama
}
if strings.HasPrefix(s, "anthropic/") {
if strings.Contains(s, "-thinking-") {
return LaneAnthropicThinking
}
return LaneAnthropicDefault
}
// Foreman instances are backed by Ollama and share its connection
// cap, so they route to the same lane.
if strings.HasPrefix(s, "foreman/") {
return LaneOllama
}
// m1/ is a foreman-style router pointing at a dedicated local
// instance with its own connection budget. Separate lane so its
// concurrency cap is independent of the shared ollama lane.
if strings.HasPrefix(s, "m1/") {
return LaneM1
}
return LaneLLMDefault
}
+373
View File
@@ -0,0 +1,373 @@
// Package llms — lane_transport.go: the lane-aware decorator. Wraps an
// llm.Provider so every model it mints submits its Generate/Stream calls
// through the matching named lane's bounded worker pool (lane selection
// per lane_mapping.go), and stamps every returned error with per-call
// attribution (caller id, run id, prompt snapshot) for the failover log.
//
// Why intercept at the llm.Provider layer: majordomo's Provider and Model
// are small public interfaces, so the decorator slots between the chain
// executor and the real provider with no fork. Every chain attempt calls
// laneModel.Generate, which queues on the lane, runs the real call, and
// wraps failures with CallInfo — the ChainConfig.Observer (which receives
// no context) recovers the attribution from the error itself.
//
// Test: lane_transport_test.go covers mapping correctness, the
// concurrency-limiting behavior, and error attribution.
// lane_chatbot_test.go is the regression guard proving chatbot-path LLM
// calls actually go through the lane.
package model
import (
"context"
"errors"
"time"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"github.com/google/uuid"
"gitea.stevedudenhoeffer.com/steve/executus/lane"
)
// defaultLaneExecTimeout is the execution backstop applied inside a lane
// job once it leaves the queue: the caller's deadline is detached (queue
// wait must not consume the LLM execution budget) and replaced with this
// hard cap so a hung provider can't leak workers.
const defaultLaneExecTimeout = 5 * time.Minute
// foremanModelTimeout is the hard per-call timeout for foreman targets —
// slow local LLMs that may block on model loads and upstream queues.
const foremanModelTimeout = 30 * time.Minute
// foremanLaneExecTimeout is the lane execution backstop for foreman
// targets. Slightly above foremanModelTimeout so the model-level timeout
// (the documented contract) is the one that fires.
const foremanLaneExecTimeout = foremanModelTimeout + time.Minute
// laneCallerKey is the context key for the per-call caller identity used
// for fair-share queueing.
type laneCallerKey struct{}
// runIDKey is the context key for the per-call run id used for failover
// event attribution.
type runIDKey struct{}
// ContextWithLaneCaller attaches a caller identity to ctx. The lane
// decorator reads this when constructing a Job so fair-share queueing
// can isolate heavy users, and snapshots it into error attribution for
// the failover log.
//
// Empty string is a no-op and lumps every empty-caller invocation into a
// single fair-share bucket; production callers should always populate it.
func ContextWithLaneCaller(ctx context.Context, callerID string) context.Context {
if callerID == "" {
return ctx
}
return context.WithValue(ctx, laneCallerKey{}, callerID)
}
// LaneCallerFromContext returns the caller identity attached via
// ContextWithLaneCaller, or "" if none is set.
func LaneCallerFromContext(ctx context.Context) string {
s, _ := ctx.Value(laneCallerKey{}).(string)
return s
}
// ContextWithRunID attaches a skill/agent run id to ctx. Snapshotted into
// error attribution so failover events can be correlated to runs.
func ContextWithRunID(ctx context.Context, runID string) context.Context {
if runID == "" {
return ctx
}
return context.WithValue(ctx, runIDKey{}, runID)
}
// RunIDFromContext returns the run id attached via ContextWithRunID, or
// "" if none is set.
func RunIDFromContext(ctx context.Context) string {
s, _ := ctx.Value(runIDKey{}).(string)
return s
}
// ---------------------------------------------------------------------------
// Error attribution
// ---------------------------------------------------------------------------
// CallInfo is the per-call attribution snapshot the lane decorator stamps
// onto every error it returns. majordomo's ChainConfig.Observer receives
// a bare FailoverEvent (no context); the failover log recovers caller,
// run id, and the prompt chain from the event's error via
// CallInfoFromError.
type CallInfo struct {
// CallerID is the fair-share caller identity (ContextWithLaneCaller).
CallerID string
// RunID is the skill/agent run id (ContextWithRunID); "" if not threaded.
RunID string
// Messages is the request's message chain at call time, for the
// failover log's persist_prompts feature.
Messages []llm.Message
}
// callInfoError carries CallInfo along an error chain without changing
// the error's message or classification (Unwrap preserves errors.Is/As).
type callInfoError struct {
inner error
info CallInfo
}
func (e *callInfoError) Error() string { return e.inner.Error() }
func (e *callInfoError) Unwrap() error { return e.inner }
// WithCallInfo stamps attribution onto err. nil err returns nil.
func WithCallInfo(err error, info CallInfo) error {
if err == nil {
return nil
}
return &callInfoError{inner: err, info: info}
}
// CallInfoFromError extracts the attribution stamped by the lane
// decorator (or WithCallInfo), if any.
func CallInfoFromError(err error) (CallInfo, bool) {
var cie *callInfoError
if errors.As(err, &cie) {
return cie.info, true
}
return CallInfo{}, false
}
// ---------------------------------------------------------------------------
// Lane decoration
// ---------------------------------------------------------------------------
// LaneRegistry is the narrow surface the lane decorator needs from
// pkg/lane.Registry. Defined as an interface so tests can substitute a
// fake registry without spinning up a real one.
type LaneRegistry interface {
GetOrCreate(ctx context.Context, name string) lane.Lane
}
// laneProvider decorates an llm.Provider so every model it mints routes
// calls through the lane named by LaneFor(provider/model). With a nil
// registry the queueing is skipped but error attribution still applies.
type laneProvider struct {
inner llm.Provider
registry LaneRegistry
execTimeout time.Duration
}
// WrapProviderForLane returns a provider whose models submit each
// Generate/Stream call through the lane named by LaneFor(name/model) in
// the registry, and stamp CallInfo attribution onto every error.
//
// A nil registry disables queueing (calls pass straight through) but the
// decoration — and with it error attribution — remains, so failover
// logging works in lane-less deployments and tests.
func WrapProviderForLane(inner llm.Provider, registry LaneRegistry) llm.Provider {
return wrapProviderForLane(inner, registry, defaultLaneExecTimeout)
}
func wrapProviderForLane(inner llm.Provider, registry LaneRegistry, execTimeout time.Duration) llm.Provider {
if inner == nil {
return nil
}
if execTimeout <= 0 {
execTimeout = defaultLaneExecTimeout
}
return &laneProvider{inner: inner, registry: registry, execTimeout: execTimeout}
}
func (p *laneProvider) Name() string { return p.inner.Name() }
func (p *laneProvider) Model(id string, opts ...llm.ModelOption) (llm.Model, error) {
m, err := p.inner.Model(id, opts...)
if err != nil {
return nil, err
}
return &laneModel{
inner: m,
registry: p.registry,
laneName: LaneFor(p.inner.Name() + "/" + id),
execTimeout: p.execTimeout,
}, nil
}
// laneModel routes one model's calls through its lane and stamps error
// attribution. The lane name is resolved once at Model() time — the
// provider name and model id are both known there, unlike legacy gollm where
// the request had to be inspected per call.
type laneModel struct {
inner llm.Model
registry LaneRegistry
laneName string
execTimeout time.Duration
}
func (m *laneModel) Capabilities() llm.Capabilities { return m.inner.Capabilities() }
// laneJob adapts an in-flight call to the lane.Job interface. The result
// is captured into the struct and read after SubmitWait returns.
type laneJob struct {
id string
callerID string
run func(ctx context.Context) error
}
func (j *laneJob) ID() string { return j.id }
func (j *laneJob) CallerID() string { return j.callerID }
func (j *laneJob) Priority() int { return 0 }
func (j *laneJob) Run(ctx context.Context) error { return j.run(ctx) }
func (m *laneModel) Generate(ctx context.Context, req llm.Request, opts ...llm.Option) (*llm.Response, error) {
// Fold options now so the job closure and the attribution snapshot
// both see the final request.
req = req.Apply(opts...)
info := CallInfo{
CallerID: LaneCallerFromContext(ctx),
RunID: RunIDFromContext(ctx),
Messages: req.Messages,
}
resp, err := m.submit(ctx, func(execCtx context.Context) (*llm.Response, error) {
return m.inner.Generate(execCtx, req)
})
if err != nil {
return resp, WithCallInfo(err, info)
}
return resp, nil
}
func (m *laneModel) Stream(ctx context.Context, req llm.Request, opts ...llm.Option) (llm.Stream, error) {
req = req.Apply(opts...)
info := CallInfo{
CallerID: LaneCallerFromContext(ctx),
RunID: RunIDFromContext(ctx),
Messages: req.Messages,
}
l := m.lane(ctx)
if l == nil {
s, err := m.inner.Stream(ctx, req)
if err != nil {
return nil, WithCallInfo(err, info)
}
return s, nil
}
// Streams hold their lane slot only while ESTABLISHING the stream —
// holding it for the full consumption would deadlock a slow consumer
// against the pool. The caller's ctx is used as-is (no deadline
// detach): severing cancellation from a long-lived stream would leak
// connections.
var (
stream llm.Stream
serr error
)
job := &laneJob{
id: uuid.New().String(),
callerID: info.CallerID,
run: func(context.Context) error {
stream, serr = m.inner.Stream(ctx, req)
return serr
},
}
if err := l.SubmitWait(ctx, job); err != nil {
return nil, WithCallInfo(err, info)
}
if serr != nil {
return nil, WithCallInfo(serr, info)
}
return stream, nil
}
// lane resolves the lane for this model, or nil when queueing is
// disabled (nil registry, or a registry that declines the name).
func (m *laneModel) lane(ctx context.Context) lane.Lane {
if m.registry == nil {
return nil
}
return m.registry.GetOrCreate(ctx, m.laneName)
}
// submit runs fn through the lane (or directly when queueing is off).
//
// Inside a lane job the caller's deadline is detached so queue wait does
// not consume the execution budget — ctx VALUES (usage attribution,
// trace ids) are preserved, only cancellation/deadline are severed — and
// an execTimeout backstop prevents runaway calls. Queue-phase
// cancellation still works: SubmitWait waits on the original ctx, so a
// caller that gives up while queued exits immediately.
func (m *laneModel) submit(ctx context.Context, fn func(context.Context) (*llm.Response, error)) (*llm.Response, error) {
l := m.lane(ctx)
if l == nil {
return fn(ctx)
}
var (
resp *llm.Response
err error
)
job := &laneJob{
id: uuid.New().String(),
callerID: LaneCallerFromContext(ctx),
run: func(context.Context) error {
execCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), m.execTimeout)
defer cancel()
resp, err = fn(execCtx)
// Returning err lets the lane's pool propagate it to
// SubmitWait; the captured err is what we surface.
return err
},
}
if serr := l.SubmitWait(ctx, job); serr != nil && err == nil {
return nil, serr
}
return resp, err
}
// ---------------------------------------------------------------------------
// Model timeout decoration (foreman)
// ---------------------------------------------------------------------------
// timeoutProvider wraps a provider so every minted model enforces a hard
// per-call deadline on Generate. Used for foreman targets (slow local
// LLMs). Stream is passed through: a wall-clock deadline on a long-lived
// stream would sever it mid-consumption.
type timeoutProvider struct {
inner llm.Provider
timeout time.Duration
}
// withModelTimeout decorates p so its models' Generate calls carry a
// hard timeout.
func withModelTimeout(p llm.Provider, d time.Duration) llm.Provider {
if p == nil || d <= 0 {
return p
}
return &timeoutProvider{inner: p, timeout: d}
}
func (p *timeoutProvider) Name() string { return p.inner.Name() }
func (p *timeoutProvider) Model(id string, opts ...llm.ModelOption) (llm.Model, error) {
m, err := p.inner.Model(id, opts...)
if err != nil {
return nil, err
}
return &timeoutModel{inner: m, timeout: p.timeout}, nil
}
type timeoutModel struct {
inner llm.Model
timeout time.Duration
}
func (m *timeoutModel) Capabilities() llm.Capabilities { return m.inner.Capabilities() }
func (m *timeoutModel) Generate(ctx context.Context, req llm.Request, opts ...llm.Option) (*llm.Response, error) {
ctx, cancel := context.WithTimeout(ctx, m.timeout)
defer cancel()
return m.inner.Generate(ctx, req, opts...)
}
func (m *timeoutModel) Stream(ctx context.Context, req llm.Request, opts ...llm.Option) (llm.Stream, error) {
return m.inner.Stream(ctx, req, opts...)
}
+477
View File
@@ -0,0 +1,477 @@
// Package model is executus's config-driven model-access layer over majordomo: it owns the
// package-level *majordomo.Registry (providers with mort's env keys,
// OpenAI-compat presets, lane-aware decoration, the DB-backed tier
// resolver, legacy shortcut aliases, the foreman timeout decorator, and
// failover/health wiring), plus the mort-facing call helpers
// (ParseModelRequest / ParseModelForContext / GenerateWith /
// CallAndExecute / SimpleCall) and usage/trace recording.
//
// The ":low/:medium/:high" reasoning-suffix dialect is an executus convenience:
// majordomo treats model ids as verbatim, so this package strips the
// suffix from specs and tier values and re-applies it per request via
// llm.WithReasoningEffort on a wrapping Model.
package model
import (
"context"
"fmt"
"os"
"strings"
"sync"
"time"
majordomo "gitea.stevedudenhoeffer.com/steve/majordomo"
"gitea.stevedudenhoeffer.com/steve/majordomo/health"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/majordomo/provider/anthropic"
"gitea.stevedudenhoeffer.com/steve/majordomo/provider/google"
"gitea.stevedudenhoeffer.com/steve/majordomo/provider/ollama"
"gitea.stevedudenhoeffer.com/steve/majordomo/provider/openai"
)
// Usage and trace recording live in sink.go: SetUsageSink / SetTraceSink
// install the host seams, and ParseModelForContext stamps the model name on
// the context (via WithModel) for attribution.
// ---------------------------------------------------------------------------
// Package registry
// ---------------------------------------------------------------------------
// buildConfig carries the knobs Wire feeds into buildRegistry. The zero
// value yields a lane-less registry with majordomo's default failover
// behavior — the bootstrap state tests and pre-Wire code paths run on.
type buildConfig struct {
lanes LaneRegistry
// maxRetries maps the llms.failover.max_retries convar onto
// ChainConfig.TransientRetries. <= 0 keeps majordomo's default (1).
maxRetries int
// cooldown maps the llms.failover.cooldown_seconds convar onto
// health.Config.BaseCooldown. <= 0 keeps the mort default (300s).
// Note majordomo grows the cooldown exponentially from this base;
// MaxCooldown is set to max(cooldown, 5m) so the operator dial
// dominates (a 10m base never gets capped below itself).
cooldown time.Duration
// observer receives one event per failover decision (failed attempt,
// bench, benched-skip). Typically failoverlog.NewObserver(...).
observer func(majordomo.FailoverEvent)
}
// defaultFailoverCooldown matches the historical llms.failover.cooldown_seconds
// convar default (300s).
const defaultFailoverCooldown = 300 * time.Second
var (
registryMu sync.RWMutex
registry = buildRegistry(buildConfig{})
)
// Registry returns the current package-level majordomo registry. Most
// callers should use ParseModelRequest / ParseModelForContext instead;
// the registry itself is exposed for admin surfaces (health/bench) and
// for tests that need to substitute providers.
func Registry() *majordomo.Registry {
registryMu.RLock()
defer registryMu.RUnlock()
return registry
}
// Health returns the health tracker of the current registry — the live
// source of truth for benched models. Used by the `.failover` commands
// and the failover web UI (see ListBenched/BenchModel/UnbenchModel for
// the mort-flavored facade).
func Health() *health.Tracker {
return Registry().Health()
}
// setRegistry swaps the package registry. Bench/backoff state of the old
// registry is discarded — Wire is a boot-time operation.
func setRegistry(r *majordomo.Registry) {
registryMu.Lock()
defer registryMu.Unlock()
registry = r
}
// buildRegistry constructs a fully-wired majordomo registry:
//
// - health/chain config from the failover convars (via cfg),
// - mort's providers under their nonstandard env keys (OPENAI_KEY,
// GOOGLE_GEMINI_API_KEY, ...), every one lane-decorated,
// - OpenAI-compat presets (deepseek, moonshot+kimi, xai+grok, groq),
// - scheme factories for LLM_* env DSNs re-registered so DSN-defined
// providers (m1, arbitrary foreman targets) are lane-decorated too,
// with foreman additionally getting the 30-minute model timeout,
// - the legacy shortcut aliases, and
// - the delegating tier resolver (reads defaultResolver at Resolve
// time, so Init() can swap in the DB-backed resolver later).
func buildRegistry(cfg buildConfig) *majordomo.Registry {
cooldown := cfg.cooldown
if cooldown <= 0 {
cooldown = defaultFailoverCooldown
}
maxCooldown := cooldown
if maxCooldown < 5*time.Minute {
maxCooldown = 5 * time.Minute
}
r := majordomo.New(
// Env DSNs are loaded manually below, AFTER the scheme factories
// are overridden — New()'s eager scan would otherwise build
// LLM_*-defined providers with the stock (un-decorated) factories.
majordomo.WithoutEnvProviders(),
majordomo.WithHealthConfig(health.Config{
BaseCooldown: cooldown,
MaxCooldown: maxCooldown,
}),
majordomo.WithChainConfig(majordomo.ChainConfig{
TransientRetries: cfg.maxRetries,
// legacy gollm failed over on request-specific errors (400/413/422)
// without benching; majordomo fails fast on permanent errors by
// default. AdvanceOnPermanent preserves the availability-first
// behavior mort's executors rely on.
AdvanceOnPermanent: true,
Observer: cfg.observer,
}),
)
wrap := func(p llm.Provider) llm.Provider {
return wrapProviderForLane(p, cfg.lanes, defaultLaneExecTimeout)
}
// Core providers with mort's env keys.
r.RegisterProvider(wrap(openai.New(
openai.WithAPIKey(os.Getenv("OPENAI_KEY")),
)))
r.RegisterProvider(wrap(anthropic.New(
anthropic.WithAPIKey(os.Getenv("ANTHROPIC_API_KEY")),
)))
r.RegisterProvider(wrap(google.New(
google.WithAPIKey(os.Getenv("GOOGLE_GEMINI_API_KEY")),
)))
r.RegisterProvider(wrap(localOllamaProvider()))
// ollama.Cloud reads OLLAMA_API_KEY itself; with the key unset the
// provider still registers and errors clearly at call time (parity
// with the previous behavior).
r.RegisterProvider(wrap(ollama.Cloud()))
// OpenAI-compatible presets. Base URLs mirror legacy gollm's defaults.
for _, preset := range []struct {
name, baseURL, envKey string
}{
{"deepseek", "https://api.deepseek.com/v1", "DEEPSEEK_API_KEY"},
{"moonshot", "https://api.moonshot.ai/v1", "MOONSHOT_API_KEY"},
{"kimi", "https://api.moonshot.ai/v1", "MOONSHOT_API_KEY"}, // alias provider for moonshot
{"xai", "https://api.x.ai/v1", "XAI_API_KEY"},
{"grok", "https://api.x.ai/v1", "XAI_API_KEY"}, // alias provider for xai
{"groq", "https://api.groq.com/openai/v1", "GROQ_API_KEY"},
} {
r.RegisterProvider(wrap(openai.New(
openai.WithName(preset.name),
openai.WithBaseURL(preset.baseURL),
openai.WithAPIKey(os.Getenv(preset.envKey)),
)))
}
// Scheme factories for LLM_* env DSNs. Re-registered so DSN-defined
// providers go through the lane decorator like the built-ins.
//
// foreman targets are slow local LLMs (large model loads, queued
// behind other requests), so their models additionally get a hard
// 30-minute timeout and a matching lane execution backstop — the
// default 5-minute lane backstop would strangle them.
r.RegisterScheme("foreman", func(name string, dsn majordomo.DSN) (llm.Provider, error) {
p := ollama.Foreman(dsn.BaseURL(), dsn.Token, ollama.WithName(name))
return wrapProviderForLane(
withModelTimeout(p, foremanModelTimeout),
cfg.lanes,
foremanLaneExecTimeout,
), nil
})
laneScheme := func(factory majordomo.SchemeFactory) majordomo.SchemeFactory {
return func(name string, dsn majordomo.DSN) (llm.Provider, error) {
p, err := factory(name, dsn)
if err != nil {
return nil, err
}
return wrap(p), nil
}
}
ollamaScheme := laneScheme(func(name string, dsn majordomo.DSN) (llm.Provider, error) {
return ollama.New(
ollama.WithName(name),
ollama.WithBaseURL(dsn.BaseURL()),
ollama.WithToken(dsn.Token),
), nil
})
r.RegisterScheme("ollama", ollamaScheme)
r.RegisterScheme("ollama-cloud", ollamaScheme)
r.RegisterScheme("openai", laneScheme(func(name string, dsn majordomo.DSN) (llm.Provider, error) {
return openai.New(
openai.WithName(name),
openai.WithBaseURL(dsn.BaseURL()),
openai.WithAPIKey(dsn.Token),
), nil
}))
r.RegisterScheme("anthropic", laneScheme(func(name string, dsn majordomo.DSN) (llm.Provider, error) {
return anthropic.New(
anthropic.WithName(name),
anthropic.WithBaseURL(dsn.BaseURL()),
anthropic.WithAPIKey(dsn.Token),
), nil
}))
googleScheme := laneScheme(func(name string, dsn majordomo.DSN) (llm.Provider, error) {
return google.New(
google.WithName(name),
google.WithBaseURL(dsn.BaseURL()),
google.WithAPIKey(dsn.Token),
), nil
})
r.RegisterScheme("google", googleScheme)
r.RegisterScheme("gemini", googleScheme)
// Eager LLM_* env scan, now with the decorated scheme factories in
// place. Malformed entries are recorded per-name and surface on use.
env := make(map[string]string)
for _, kv := range os.Environ() {
if k, v, ok := strings.Cut(kv, "="); ok {
env[k] = v
}
}
_ = r.LoadEnv(env)
// Legacy shortcut aliases (sonnet, haiku, ...). Same strings as the
// historical table; kept in sync with legacyAliasSpecs below.
for name, spec := range legacyAliasSpecs {
r.RegisterAlias(name, spec)
}
// Tier resolver: a delegating closure so Init() and test helpers can
// swap defaultResolver without rebuilding the registry. The resolver
// returns specs with the legacy reasoning suffixes already stripped
// (per chain element); the tier's default reasoning level is applied
// by ParseModelRequest, not here.
r.RegisterResolver(majordomo.ResolverFunc(func(name string) (string, bool) {
res := defaultResolver
if res == nil {
return "", false
}
spec, _, ok := res.Resolve(name)
return spec, ok
}))
return r
}
// localOllamaProvider builds the local Ollama provider, honoring
// OLLAMA_BASE_URL when set (mort's historical env var; ollama.Local
// itself honors OLLAMA_HOST).
func localOllamaProvider() llm.Provider {
if url := os.Getenv("OLLAMA_BASE_URL"); url != "" {
return ollama.Local(ollama.WithBaseURL(url))
}
return ollama.Local()
}
// ---------------------------------------------------------------------------
// Spec parsing
// ---------------------------------------------------------------------------
// ParseModelRequest resolves a model request string to a ready-to-use Model.
// It handles, in order:
//
// - empty spec → tier "fast"
// - the legacy ":low/:medium/:high" reasoning suffix, stripped per chain
// element (ollama tags like ":30b" or ":cloud" are preserved); the
// level is applied to every call via llm.WithReasoningEffort
// - tier aliases (DB-backed convars; a tier value's own suffix becomes
// the default level when the caller didn't supply one)
// - legacy shortcut aliases (sonnet, haiku, opus, ...)
// - provider/model lookup and LLM_* env-DSN fallback (majordomo)
// - comma-separated failover chains with health-tracked bench/backoff
//
// The returned Model is instrumented: token usage from every successful
// Generate is recorded to the package usage recorder automatically. Do
// NOT additionally call RecordUsage on responses from a parsed model.
func ParseModelRequest(spec string) (majordomo.Model, error) {
spec = strings.TrimSpace(spec)
if spec == "" {
spec = "fast"
}
clean, level := splitReasoningSpec(spec)
// Tier default reasoning: when the (suffix-free) spec is exactly a
// tier name and the caller didn't ask for a level, the tier value's
// own suffix (e.g. "anthropic/claude-opus-4-6:high") applies.
if level == "" && defaultResolver != nil {
if _, tierLevel, ok := defaultResolver.Resolve(clean); ok {
level = tierLevel
}
}
m, err := Registry().Parse(clean)
if err != nil {
return nil, fmt.Errorf("model %q: %w", spec, err)
}
if level != "" {
m = &reasoningModel{inner: m, level: level}
}
return &instrumentedModel{inner: m}, nil
}
// ParseModelForContext combines ParseModelRequest with llmusage.WithModel so
// that the resolved model name is recorded in the context for usage tracking.
// Prefer this over bare ParseModelRequest in all new code.
func ParseModelForContext(ctx context.Context, req string) (context.Context, majordomo.Model, error) {
model, err := ParseModelRequest(req)
if err != nil {
return ctx, nil, err
}
ctx = WithModel(ctx, ResolveModelName(req))
return ctx, model, nil
}
// reasoningModel applies a default reasoning effort to every request that
// doesn't carry one already. Mort's legacy ":low/:medium/:high" suffix
// dialect resolves to this wrapper because majordomo treats model ids as
// verbatim (no suffix stripping).
type reasoningModel struct {
inner llm.Model
level string
}
func (m *reasoningModel) Generate(ctx context.Context, req llm.Request, opts ...llm.Option) (*llm.Response, error) {
req = req.Apply(opts...)
if req.ReasoningEffort == "" {
req.ReasoningEffort = m.level
}
return m.inner.Generate(ctx, req)
}
func (m *reasoningModel) Stream(ctx context.Context, req llm.Request, opts ...llm.Option) (llm.Stream, error) {
req = req.Apply(opts...)
if req.ReasoningEffort == "" {
req.ReasoningEffort = m.level
}
return m.inner.Stream(ctx, req)
}
func (m *reasoningModel) Capabilities() llm.Capabilities { return m.inner.Capabilities() }
// ---------------------------------------------------------------------------
// Reasoning-suffix dialect
// ---------------------------------------------------------------------------
// reasoningLevels is the set of recognized legacy suffix values.
var reasoningLevels = map[string]bool{"low": true, "medium": true, "high": true}
// splitReasoning peels an optional ":low" / ":medium" / ":high" suffix off
// a single model request string. Returns the input unchanged and "" when no
// recognized level is present, so non-reasoning suffixes (ollama tags like
// ":30b" or ":q4_K_M", date stamps) flow through untouched.
func splitReasoning(s string) (string, string) {
idx := strings.LastIndex(s, ":")
if idx < 0 {
return s, ""
}
if lvl := s[idx+1:]; reasoningLevels[lvl] {
return s[:idx], lvl
}
return s, ""
}
// splitReasoningSpec strips the legacy reasoning suffix from every element
// of a (possibly comma-separated) spec. The returned level is the first
// non-empty per-element level — majordomo chains carry one request-level
// reasoning effort, not one per target, so the head element's preference
// wins. Elements without a suffix are unchanged.
func splitReasoningSpec(spec string) (string, string) {
if !strings.Contains(spec, ",") {
return splitReasoning(strings.TrimSpace(spec))
}
parts := strings.Split(spec, ",")
level := ""
for i, p := range parts {
s, l := splitReasoning(strings.TrimSpace(p))
parts[i] = s
if level == "" {
level = l
}
}
return strings.Join(parts, ","), level
}
// ---------------------------------------------------------------------------
// Usage-attribution name resolution
// ---------------------------------------------------------------------------
// ResolveModelName returns the model portion of a request string, stripping
// any reasoning suffix and resolving tier aliases. The result is used for
// usage attribution (keyed on model name, not provider or reasoning level).
func ResolveModelName(req string) string {
// Strip any reasoning-level suffix before resolving — the level is a
// per-request setting, not part of the model identity.
req, _ = splitReasoning(req)
// Tier expansion: when the request is a tier alias, fold it through the
// resolver and return the model portion of its current convar value. The
// empty string is treated as "fast" for compatibility with callers that
// pre-resolution defaulted to fast.
if defaultResolver != nil {
key := req
if key == "" {
key = "fast"
}
if spec, _, ok := defaultResolver.Resolve(key); ok && spec != "" {
// A tier may resolve to a comma-separated failover chain. Attribute
// usage to the first (preferred) entry's model name rather than the
// whole chain string.
if i := strings.IndexByte(spec, ','); i >= 0 {
spec = strings.TrimSpace(spec[:i])
}
if idx := strings.Index(spec, "/"); idx >= 0 {
return spec[idx+1:]
}
return spec
}
}
// For non-tier requests, return the model portion after the slash.
// Static aliases are NOT expanded here beyond the legacy table below:
// callers that went through ParseModelRequest already carry the
// concrete spec.
if idx := strings.Index(req, "/"); idx >= 0 {
return req[idx+1:]
}
// Legacy shortcut fallback: callers that pass bare names like "sonnet"
// to ResolveModelName (without going through ParseModelRequest) still
// need the concrete model name for usage keys.
if spec, ok := legacyAliasSpecs[req]; ok {
if idx := strings.Index(spec, "/"); idx >= 0 {
return spec[idx+1:]
}
return spec
}
return req
}
// legacyAliasSpecs maps legacy shortcut names to their full provider/model
// spec. Registered with the registry as static aliases AND consulted by
// ResolveModelName for bare-name usage attribution.
var legacyAliasSpecs = map[string]string{
"openai": "openai/gpt-4o-mini",
"gpt-4": "openai/gpt-4",
"gpt-4o": "openai/gpt-4o",
"gpt-4o-mini": "openai/gpt-4o-mini",
"sonnet": "anthropic/claude-sonnet-4-6",
"sonnet-4.5": "anthropic/claude-sonnet-4-5-20250929",
"haiku": "anthropic/claude-haiku-4-5-20251001",
"opus": "anthropic/claude-opus-4-6",
"gemini": "google/gemini-2.0-flash",
"gemini-flash": "google/gemini-2.0-flash",
"gemini-pro": "google/gemini-2.0-pro",
}
+136
View File
@@ -0,0 +1,136 @@
package model
import (
"context"
"time"
)
// This file is executus's inversion of mort's llmusage / llmtrace coupling.
// The model package owns the MECHANISM (instrument every parsed model's
// Generate, attribute by serving model, emit a span when a trace is active);
// WHERE usage/traces land is a host seam. A host registers a UsageSink and/or
// a TraceSink; both are optional (nil = off), so a light host records nothing.
// --- Usage ---
// UsageSink receives one record per successful Generate through a model parsed
// by this package (ParseModelRequest / ParseModelForContext). Implement it to
// meter or bill; the token detail mirrors majordomo's Response.Usage.
//
// IMPORTANT: cacheReadTokens and cacheWriteTokens are PORTIONS of inputTokens,
// not independent additive values (they let a sink price cached vs fresh input
// differently). A sink must NOT compute total = input+output+cacheRead+
// cacheWrite — that double-counts the cached input.
type UsageSink interface {
Record(ctx context.Context, model string, inputTokens, outputTokens, cacheReadTokens, cacheWriteTokens int)
}
var usageSink UsageSink
// SetUsageSink installs the usage sink (nil disables usage recording). Call at
// startup before model calls.
func SetUsageSink(s UsageSink) { usageSink = s }
// --- Trace ---
// Span is one traced model call. The host's TraceSink persists it however it
// likes (a DB row, a log line, an OTel span). String fields carrying structured
// data (Messages, ToolDefinitions, ...) are pre-marshalled JSON.
type Span struct {
SpanID string
TraceID string
Model string
SystemPrompt string
Messages string
ToolDefinitions string
ResponseText string
ResponseToolCalls string
ToolResults string
Error string
InputTokens int
OutputTokens int
DurationMs int64
StartedAt time.Time
CompletedAt time.Time
CreatedAt time.Time
}
// TraceSink receives a Span for each traced call (one is emitted only when a
// trace id is present on the context — see WithTraceID).
type TraceSink interface {
WriteSpan(span Span)
}
var traceSink TraceSink
// SetTraceSink installs the trace sink (nil disables tracing).
func SetTraceSink(s TraceSink) { traceSink = s }
// TraceSinkActive reports whether a trace sink is installed.
func TraceSinkActive() bool { return traceSink != nil }
// --- Context attribution ---
//
// ParseModelForContext stamps the requested model onto the context so usage
// from a response that doesn't name its serving model can still be attributed.
// A host's tracing/usage middleware stamps a trace id and optional caller/tool
// for diagnostics. All reads are nil/empty-safe.
type (
ctxKeyModel struct{}
ctxKeyTrace struct{}
ctxKeyTool struct{}
ctxKeyUser struct{}
)
// WithModel attributes subsequent usage on ctx to the given model name.
func WithModel(ctx context.Context, model string) context.Context {
return context.WithValue(ctx, ctxKeyModel{}, model)
}
func modelFromContext(ctx context.Context) string {
if v, ok := ctx.Value(ctxKeyModel{}).(string); ok {
return v
}
return ""
}
// WithTraceID marks ctx as belonging to a trace; a TraceSink (if installed)
// then receives a Span per call. An empty id (or no id) disables tracing.
func WithTraceID(ctx context.Context, id string) context.Context {
return context.WithValue(ctx, ctxKeyTrace{}, id)
}
func traceIDFromContext(ctx context.Context) string {
if v, ok := ctx.Value(ctxKeyTrace{}).(string); ok {
return v
}
return ""
}
// WithUsageTool / WithUsageUser attach optional attribution used only in the
// "unknown model" diagnostic warning. Default "unknown".
func WithUsageTool(ctx context.Context, tool string) context.Context {
return context.WithValue(ctx, ctxKeyTool{}, tool)
}
func toolFromContext(ctx context.Context) string {
if v, ok := ctx.Value(ctxKeyTool{}).(string); ok && v != "" {
return v
}
return "unknown"
}
func WithUsageUser(ctx context.Context, user string) context.Context {
return context.WithValue(ctx, ctxKeyUser{}, user)
}
func userFromContext(ctx context.Context) string {
if v, ok := ctx.Value(ctxKeyUser{}).(string); ok && v != "" {
return v
}
return "unknown"
}
+162
View File
@@ -0,0 +1,162 @@
package model
import (
"fmt"
"sort"
"strings"
"sync"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/config"
)
// tierResolver expands tier aliases (e.g. "fast", "thinking", "agent-working")
// into a concrete model spec or a comma-separated failover chain. The set of
// tier names and their FALLBACK specs are host-supplied (a map passed at
// Configure time); the live value of each tier is read from a config.Source
// under the key "model.tier.<name>", so a host whose config backend mutates at
// runtime (mort's convar) re-targets tiers without a restart, while a static
// host (gadfly's env) just gets the fallback. A small in-process cache (TTL
// from "model.tier.cache_ttl_seconds", default 30s) saves config round-trips on
// the hot path; ReloadTiers clears it.
//
// This is executus's inversion of mort's convar-bound resolver: the MECHANISM
// (tier lookup, reasoning-suffix dialect, chain validation, cache) is generic;
// the tier MAP content (which tiers exist + their default specs) is host config.
type tierResolver struct {
cfg config.Source
defaults map[string]string // tier name -> fallback spec
ttl time.Duration
mu sync.RWMutex
cache map[string]tierEntry
now func() time.Time // overridable for tests
}
type tierEntry struct {
spec string
reasoning string
expires time.Time
}
const tierConfigPrefix = "model.tier."
// NewTierResolver builds a resolver over cfg with the given tier defaults
// (name -> fallback spec). cfg may be nil (the fallbacks are then always used).
// ttl<=0 reads "model.tier.cache_ttl_seconds" (default 30s).
func NewTierResolver(cfg config.Source, defaults map[string]string, ttl time.Duration) *tierResolver {
if ttl <= 0 {
ttl = time.Duration(config.Int(cfg, tierConfigPrefix+"cache_ttl_seconds", 30)) * time.Second
}
if ttl <= 0 {
ttl = 30 * time.Second
}
cp := make(map[string]string, len(defaults))
for k, v := range defaults {
cp[k] = v
}
return &tierResolver{
cfg: cfg,
defaults: cp,
ttl: ttl,
cache: make(map[string]tierEntry),
now: time.Now,
}
}
func (r *tierResolver) has(name string) bool {
_, ok := r.defaults[name]
return ok
}
func (r *tierResolver) names() []string {
out := make([]string, 0, len(r.defaults))
for k := range r.defaults {
out = append(out, k)
}
sort.Strings(out)
return out
}
// Resolve returns the current model spec and default reasoning level for a tier
// name. ok=false if name is not a registered tier. Legacy reasoning suffixes
// (":low/:medium/:high") are stripped per chain element; the first non-empty
// level becomes the tier's default reasoning level (ollama tags like ":cloud"
// pass through). The live value is read from config with the host-supplied
// fallback; an empty resolved value yields ok=true with an empty spec
// (ParseModelRequest surfaces a clear error in that path).
func (r *tierResolver) Resolve(name string) (string, string, bool) {
if !r.has(name) {
return "", "", false
}
now := r.now()
r.mu.RLock()
if e, hit := r.cache[name]; hit && now.Before(e.expires) {
r.mu.RUnlock()
return e.spec, e.reasoning, true
}
r.mu.RUnlock()
r.mu.Lock()
defer r.mu.Unlock()
if e, hit := r.cache[name]; hit && now.Before(e.expires) {
return e.spec, e.reasoning, true
}
raw := strings.TrimSpace(config.String(r.cfg, tierConfigPrefix+name, r.defaults[name]))
spec, level := splitReasoningSpec(raw)
r.cache[name] = tierEntry{spec: spec, reasoning: level, expires: now.Add(r.ttl)}
return spec, level, true
}
// Reload clears the cache so the next Resolve fetches fresh from config.
func (r *tierResolver) Reload() {
r.mu.Lock()
defer r.mu.Unlock()
r.cache = make(map[string]tierEntry)
}
// --- package-level resolver + facade ---
// defaultResolver is initialized as a package-level var (not in init()) so it
// is ready before any other file's init runs — buildRegistry's delegating
// resolver closure reads it at Resolve time. It starts with no tiers; a host
// installs its tier table via Configure.
var defaultResolver = NewTierResolver(nil, nil, 0)
// Configure installs the host's tier table. cfg is the live config source
// (nil = fallbacks only); defaults maps each tier name to its fallback spec;
// ttl<=0 uses the config'd / 30s default. The package registry's delegating
// resolver reads defaultResolver at Resolve time, so swapping it here is
// sufficient — no registry rebuild needed.
func Configure(cfg config.Source, defaults map[string]string, ttl time.Duration) {
defaultResolver = NewTierResolver(cfg, defaults, ttl)
}
// TierNames returns the registered tier alias names (sorted). Exported so UI
// layers can populate tier dropdowns without hardcoding.
func TierNames() []string { return defaultResolver.names() }
// IsTierName reports whether s is a registered tier alias.
func IsTierName(s string) bool { return defaultResolver.has(s) }
// ReloadTiers clears the package resolver's cache so the next request resolves
// freshly from config.
func ReloadTiers() { defaultResolver.Reload() }
// ValidateTierValue returns an error if value cannot be used as a tier spec —
// specifically, when a chain entry is itself a tier name (which would form a
// resolution loop). Chain entries must be concrete provider/model specs.
func ValidateTierValue(value string) error {
for _, part := range strings.Split(value, ",") {
part = strings.TrimSpace(part)
if part == "" {
continue
}
spec, _ := splitReasoning(part)
if IsTierName(spec) {
return fmt.Errorf("tier value %q contains tier alias %q (chains must use concrete provider/model specs, not nested tiers)", value, spec)
}
}
return nil
}
+110
View File
@@ -0,0 +1,110 @@
// Package llms — wiring.go: the production boot hook that rebuilds the
// package registry with the lane registry, the failover convars, and the
// failover-event observer.
//
// Why a dedicated helper (vs spreading registry construction through
// mort.go): the chatbot regression test in lane_chatbot_test.go and the
// production boot path must call the SAME wiring code. Historically
// mort.go skipped the lane wiring entirely (lanes were defined but never
// installed — 30+ skill_runs in production with 0 skill_queue_jobs rows);
// concentrating the install here means a regression in one wires fails
// the test for the other.
package model
import (
"context"
"log/slog"
"time"
majordomo "gitea.stevedudenhoeffer.com/steve/majordomo"
)
// WireOptions configures Wire. The zero value rebuilds the registry with
// no lanes and default failover behavior.
type WireOptions struct {
// Lanes is the lane registry every provider is decorated with. nil
// disables lane queueing (calls pass straight through) but keeps
// error attribution for the failover log.
Lanes LaneRegistry
// FailoverMaxRetries maps the llms.failover.max_retries convar onto
// majordomo's ChainConfig.TransientRetries (same-target retries after
// a transient error). <= 0 keeps majordomo's default (1).
FailoverMaxRetries int
// FailoverCooldown maps the llms.failover.cooldown_seconds convar
// onto health.Config.BaseCooldown. majordomo grows the cooldown
// exponentially from this base per consecutive bench; the cap is
// max(FailoverCooldown, 5m) so the operator's dial dominates.
// <= 0 keeps the mort default (300s).
FailoverCooldown time.Duration
// FailoverObserver receives one event per failover decision (failed
// attempt, bench, benched-skip). Wire it to failoverlog.NewObserver.
// Attribution (caller/run/prompts) rides on the event's error — see
// CallInfoFromError.
FailoverObserver func(majordomo.FailoverEvent)
}
// Wire rebuilds the package registry from opts and installs it. Call once
// at boot, after the lane registry and the failover convars exist (and
// after Init for DB-backed tiers — though Init and Wire are order-
// independent: the tier resolver is consulted through a delegating
// indirection).
//
// Rebuilding discards in-memory health/bench state — Wire is a boot-time
// operation, not a runtime toggle.
//
// When Lanes is non-nil, the well-known lanes (KnownLanes) are eagerly
// registered so admin dashboards have baseline state from the moment mort
// starts instead of "no lanes registered" until the first LLM call.
//
// Returns the installed registry for inspection (tests, health surfaces).
func Wire(ctx context.Context, opts WireOptions) *majordomo.Registry {
r := buildRegistry(buildConfig{
lanes: opts.Lanes,
maxRetries: opts.FailoverMaxRetries,
cooldown: opts.FailoverCooldown,
observer: opts.FailoverObserver,
})
setRegistry(r)
if opts.Lanes != nil {
names := KnownLanes()
for _, name := range names {
opts.Lanes.GetOrCreate(ctx, name)
}
slog.Info("llms: wired lane-aware registry", "lanes", len(names))
} else {
slog.Warn("llms: Wire called without a lane registry — lane queueing is inert")
}
return r
}
// KnownLanes returns the well-known lane names the LLM transport resolves
// to. Eager-registering these at boot gives admin dashboards
// (`/skills/admin/queues`, `.skill admin queue`) a baseline view from the
// moment mort starts — without this, the dashboard reads "no lanes
// registered" until the first chatbot/skill call materialises the lane
// via lazy GetOrCreate.
//
// Why this list (and not "every lane name ever"): these are the ones
// LaneFor in lane_mapping.go can produce for a real model spec. Future
// non-LLM lanes (e.g. a future image-generation lane) should be eagerly
// registered by their owning subsystem, not here.
//
// LaneSkillDefault is included even though it isn't an LLM-routing
// lane: skills run through it via skillexec.WithLaneRegistry, and the
// skills admin dashboard needs to see it from boot.
//
// Test: wiring_test.go::TestKnownLanes_NonEmpty.
func KnownLanes() []string {
return []string{
LaneOllama,
LaneAnthropicThinking,
LaneAnthropicDefault,
LaneM1,
LaneLLMDefault,
"skill-default",
}
}
+194
View File
@@ -0,0 +1,194 @@
// Package agents implements the Agent noun: a persisted persona +
// execution spec + palette of skills/sub-agents/low-level tools, with
// optional trigger metadata (schedule, webhook, chatbot channel
// listener) and personalization sources.
//
// Phase 1 scope: this package introduces Agent as a persisted noun
// with CRUD only — no execution path, no palette resolution, no
// trigger handling. See /Users/steve/.claude/plans/serene-churning-micali.md
// for the staged rollout. Later phases add agentexec, agent_invoke,
// trigger dispatch (schedule/webhook/chatbot), and CommandBinding.
//
// The three-layer storage pattern from pkg/logic/storage/CLAUDE.md
// applies — when adding a field to Agent, you MUST update
// pkg/logic/storage/agents.go (gormAgent, agentFromStorage,
// toStorage) or persistence will silently break.
package persona
import "time"
// Agent is the domain definition of an Agent persona + execution spec.
//
// Why: an Agent is the "configured invoker" — model tier + base
// system prompt + a palette of capabilities (skills, sub-agents,
// low-level tools) it may exercise during a run. Where a Skill is a
// reusable parameterised callable (a library function), an Agent is
// the actor with a persistent persona that calls those skills. The
// struct is flat — every field lives on its own column on the
// agents table; JSON columns are used only for variable-length
// collections (palette lists, tags, etc.).
//
// What: identity + persona + execution caps + palette + triggers +
// personalization + UX, all on one struct. Several field families
// (Palette, Triggers, Personalization) are persisted now but NOT
// exercised until later phases — they exist so the schema is stable
// and future phases can light up behaviour without DB migrations.
//
// Test: see pkg/logic/agents/storage_round_trip_test.go for
// Save/Get/GetByName/List/Delete coverage.
type Agent struct {
// Identity
ID string // UUID
Name string // unique per OwnerID; human-friendly identifier
Description string
OwnerID string // Discord member ID
AuthoredBy string // Discord member ID; usually == OwnerID
Version int // monotonic, for future versioning
CreatedAt time.Time
UpdatedAt time.Time
// Extends names the parent agent this agent inherits from. Only used
// during builtin loading — the loader resolves extends references and
// merges fields before persisting. The resolved agent is a standalone
// entity; Extends is NOT persisted in the database. Only single-level
// extends is supported (no chains).
Extends string
// SystemPromptPrepend, when non-empty, is prepended to the system
// prompt (with a trailing newline separator). Used by the extends
// mechanism so a child agent can prepend persona instructions to the
// parent's full system prompt without duplicating it. Like Extends,
// this is resolved at load time and NOT persisted — the final
// SystemPrompt on the persisted Agent already has the prepend applied.
SystemPromptPrepend string
// Persona / execution spec
ModelTier string // "fast" | "standard" | "thinking" | provider/model
SystemPrompt string // base persona prompt (Phase 5 layers personalization on top)
MaxIterations int // 0 → use convar default at execution time
MaxToolCalls int // 0 → use convar default at execution time
MaxRuntime time.Duration // stored as MaxRuntimeNs int64 in GORM (avoid duration-driver flakiness)
ExecutionLane string // lane name; empty = default at execution time
EncryptionEnabled bool // Phase 1 stores the flag; envelope-encryption bridge wires in a later phase
// Run-critic (two-tier timeout). When CriticEnabled is false (the
// default) MaxRuntime is the hard kill, exactly as before. When true,
// MaxRuntime becomes a SOFT trigger: at MaxRuntime the run-critic
// activates and periodically reviews the run; the hard backstop (the
// absolute kill) is MaxRuntime × the multiplier. CriticBackstopMultiplier
// of 0 means "use the convar default" (agents.critic.backstop_multiplier_default,
// default 6×). See pkg/logic/agentcritic.
CriticEnabled bool
CriticBackstopMultiplier float64
// Palette — what this Agent may invoke (Phase 2 reads these).
// Stored as JSON arrays; not exercised by Phase 1 CRUD.
SkillPalette []string // skill IDs/names
SubAgentPalette []string // agent IDs/names
LowLevelTools []string // skilltools registry names
// SkillPacks names SKILL.md skill-pack subscriptions activated for a run via
// run.Ports.SkillPacks (catalog folded into the prompt + a skill_use loader).
SkillPacks []string
// Personalization (Phase 5 reads these). Each layer name maps to
// a registered PersonalizationProvider that returns text appended
// to SystemPrompt at run time. Empty list = base prompt only.
PersonalizationSources []string
// Triggers — persisted now, NOT dispatched by Phase 1.
//
// Schedule: cron expression or "daily"/"weekly" shorthand. Empty
// = on-demand only. NextRunAt + LastScheduledRunAt are scheduler
// bookkeeping for Phase 3's per-Agent scheduler.
Schedule string
NextRunAt *time.Time
LastScheduledRunAt *time.Time
// Webhook trigger metadata. WebhookSecret empty = inbound
// webhooks disabled. WebhookSignatureRequired defaults true at
// save time (see Skill's lesson: don't store a GORM default on a
// bool where false is a legitimate explicit value — application
// layer is the source of truth).
WebhookSecret string
WebhookSignatureRequired bool
WebhookIPAllowlist []string // CIDR strings; stored as JSON array
// Chatbot trigger metadata. ChatbotChannelFilter names a filter
// registered in pkg/logic/skills' ChannelFilterRegistry — when
// the migrated chatbot dispatches via this Agent, the filter
// gates which channels it listens in.
ChatbotChannelFilter string
// UX
//
// DefaultEmoji is an optional identity emoji for this agent.
// Used as the __start__ fallback and forwarded to the invoking
// Discord message when a parent calls this agent via agent_invoke.
DefaultEmoji string
// StateReactEmoji maps tool names (and reserved keys "__start__",
// "__end__", "__error__") to Discord emoji that the executor
// reacts with as the run progresses. Empty map = no reactions.
StateReactEmoji map[string]string
// Tags is a free-form set of short labels for organisation +
// discovery on the agents list page (Phase 1 admin commands +
// future web UI).
Tags []string
// Phases defines a multi-phase pipeline for this agent. When
// non-empty, the executor runs agentexec's sequential phase runner
// instead of the single agent loop. Empty = single-loop agent.
//
// Phases IS persisted (JSON struct-slice column `phases` on
// gormAgent). It used to be transient — "TOML is the only source of
// truth" — but every production dispatch path resolves the agent from
// the DB, where the dropped Phases meant research / deepresearch
// silently degraded to a single-loop run (the executor's
// `len(a.Phases) > 0` pipeline branch was dead). The builtin loader
// still seeds phases from YAML; persisting them is what makes the
// pipeline branch fire for DB-loaded agents.
Phases []AgentPhase
}
// AgentPhase describes one stage of a multi-phase pipeline in an
// agent definition. Executed directly by agentexec's phase runner
// (pipeline.go) — there is no intermediate execution-spec struct.
//
// What: name + prompt template + model/iteration overrides + tool
// list + optional/fallback flags + IsRunFunc indicator.
//
// Test: see builtin_loader_test.go for YAML round-trip coverage.
type AgentPhase struct {
// Name identifies the phase (e.g., "scout", "plan", "investigate").
Name string
// SystemPrompt for this phase. Supports template variables:
// {{.Query}} for the original query, and {{.<PhaseName>}} for
// prior phase outputs (e.g., {{.scout}}, {{.plan}}).
SystemPrompt string
// ModelTier overrides the agent's ModelTier for this phase.
// Empty = use agent default.
ModelTier string
// MaxIter overrides the agent's MaxIterations for this phase.
// 0 = use agent default.
MaxIter int
// Tools are tool names for this phase only. These are resolved
// from the agent's low-level tools + palette at execution time.
Tools []string
// Optional means errors in this phase don't abort the pipeline.
Optional bool
// FallbackMessage is used when an optional phase fails.
// Default: "(Phase <Name> encountered an error)"
FallbackMessage string
// IsRunFunc indicates this phase is a bare LLM call (no tool
// loop). When true, the executor makes a single model.Complete
// call instead of running the full agent loop.
IsRunFunc bool
}
+604
View File
@@ -0,0 +1,604 @@
package persona
// Phase 6 — Builtin Agent loader.
//
// Why: Phase 1-5 introduced the Agent noun, runtime, triggers,
// CommandBinding, and ChatBot bridge — but every Agent in production
// was either (a) a wrapper auto-migrated from a triggered Skill, or
// (b) admin-created via `.agent new`. There were no SHIPPED Agents
// authored as builtins. Phase 6 adds an idempotent boot-time loader so
// the repo can ship canonical Agent definitions (alongside the
// existing skills/<name>/skill.yml builtins) without manual admin
// creation per deploy.
//
// What: scans `<builtinsDir>/agents/*/agent.yml`, decodes each YAML
// into an Agent, and upserts via Storage.SaveAgent under the deterministic
// system owner ID "builtin". Skill-palette entries are validated AT LOAD
// TIME against the live skills storage; missing skills warn but do not
// fail the load (the skill might arrive later via a different code
// path, and runtime resolution happens at invocation time anyway).
//
// Bypass note (v3 lesson, mirrored): like skills.LoadBuiltins, this
// loader writes via Storage.SaveAgent directly. There is no agents
// equivalent of SaveUserSkill's save-time gates today (Phase 1-5 don't
// have authoring requirements on agents), but if such gates appear in
// future phases, this loader MUST keep bypassing them — builtins are
// trusted infrastructure.
//
// Test: pkg/logic/agents/builtin_loader_test.go covers happy path,
// idempotent re-load, missing-skill warn capture, and malformed YAML
// surfaced as a per-bundle warning (not a fatal error).
import (
"context"
"errors"
"fmt"
"io/fs"
"log/slog"
"net"
"path"
"strings"
"time"
"github.com/google/uuid"
"gopkg.in/yaml.v3"
)
// BuiltinAgentOwnerID is the deterministic system owner ID used for
// every Agent created by LoadBuiltinAgents. Chosen as a non-empty
// string so the (owner_id, name) unique index distinguishes builtins
// from any user-authored Agent (Discord member IDs are numeric, so
// "builtin" cannot collide). The skills builtin loader uses owner_id=""
// instead; the two systems are independent storage scopes — there's
// no need for consistency here.
const BuiltinAgentOwnerID = "builtin"
// SkillExistenceChecker is the narrow surface LoadBuiltinAgents needs
// to validate skill_palette entries at load time. Production wires
// skills.Storage which already exposes ListByName for non-owner-scoped
// lookups. nil means "skip palette validation" (tests that don't care).
//
// Why a separate narrow interface (vs importing skills.Storage):
// agents already transitively depends on skills via migrate_from_skills,
// but the loader only needs "does a skill with this name exist
// somewhere?" — a single Boolean. Keeping the interface narrow makes
// the loader testable without a full skills storage stub.
type SkillExistenceChecker interface {
// SkillExistsByName reports whether at least one skill row has the
// given name across any owner (builtins live under owner_id="";
// users own their own rows; the loader's validation just wants
// "does ANY row exist with this name?").
SkillExistsByName(ctx context.Context, name string) (bool, error)
}
// LoadBuiltinAgents discovers and seeds builtin Agents from `builtinsDir`.
// `builtinsDir` is the root that contains an `agents/` subdirectory;
// per-agent YAML lives at `agents/<name>/agent.yml`. Returns the count
// of agents seeded or updated (skipped rows do not contribute to the
// count). Returns nil error when the agents/ directory is absent — a
// deployment without any builtin agents is valid; the loader is then a
// no-op.
//
// Idempotency contract: existing Agent rows (matched by (owner_id="builtin",
// name)) are UPDATED to the freshly-parsed YAML on each boot. ID +
// CreatedAt are preserved; UpdatedAt is refreshed. User clones of a
// builtin Agent (different owner_id, same name) are NEVER touched —
// the loader only writes to (owner_id="builtin", name) rows.
//
// `skillChecker` may be nil; when non-nil, each SkillPalette entry is
// looked up and a WARN log emitted (with the agent + missing skill
// name) for absent references. The Agent row is still seeded with the
// palette intact — runtime resolution at invocation time is the
// authoritative gate.
func LoadBuiltinAgents(ctx context.Context, store Storage, builtinsDir fs.FS, skillChecker SkillExistenceChecker) (int, error) {
if store == nil {
return 0, errors.New("agents.LoadBuiltinAgents: nil store")
}
if builtinsDir == nil {
return 0, errors.New("agents.LoadBuiltinAgents: nil builtinsDir FS")
}
entries, err := fs.ReadDir(builtinsDir, "agents")
if err != nil {
// Missing agents/ directory is benign — a deployment may not
// ship any builtins. Other errors propagate so a permission /
// IO problem surfaces loudly.
if errors.Is(err, fs.ErrNotExist) {
slog.Info("agents: no builtin agents directory", "path", "agents")
return 0, nil
}
return 0, fmt.Errorf("agents: read agents dir: %w", err)
}
// Phase 1: parse all agent manifests into a map keyed by name.
// The map is needed so extends references can be resolved before
// any agent is upserted.
type parsedEntry struct {
agent *Agent
dir string
}
parsed := make(map[string]*parsedEntry)
var parseOrder []string // preserve FS iteration order for deterministic upsert
var scanned, failed int
for _, entry := range entries {
if !entry.IsDir() {
continue
}
manifestPath := path.Join("agents", entry.Name(), "agent.yml")
data, readErr := fs.ReadFile(builtinsDir, manifestPath)
if readErr != nil {
slog.Debug("agents: skipping (no agent.yml)", "dir", entry.Name(), "error", readErr)
continue
}
scanned++
ag, parseErr := decodeAgentManifest(data)
if parseErr != nil {
slog.Warn("agents: invalid agent.yml", "dir", entry.Name(), "error", parseErr)
failed++
continue
}
parsed[ag.Name] = &parsedEntry{agent: ag, dir: entry.Name()}
parseOrder = append(parseOrder, ag.Name)
}
// Phase 2: resolve extends references. Only single-level is
// supported — chains (A extends B extends C) are rejected.
for _, name := range parseOrder {
pe := parsed[name]
ag := pe.agent
if ag.Extends == "" {
continue
}
parent, ok := parsed[ag.Extends]
if !ok {
slog.Warn("agents: extends references unknown agent",
"agent", ag.Name, "extends", ag.Extends)
failed++
delete(parsed, name)
continue
}
if parent.agent.Extends != "" {
slog.Warn("agents: extends chain not supported — parent also uses extends",
"agent", ag.Name, "extends", ag.Extends,
"parent_extends", parent.agent.Extends)
failed++
delete(parsed, name)
continue
}
if ag.Extends == ag.Name {
slog.Warn("agents: agent extends itself", "agent", ag.Name)
failed++
delete(parsed, name)
continue
}
resolveExtends(ag, parent.agent)
}
// Phase 3: palette validation + upsert.
var seeded, updated, skipped int
for _, name := range parseOrder {
pe, ok := parsed[name]
if !ok {
continue // removed during extends resolution
}
ag := pe.agent
if skillChecker != nil {
for _, sk := range ag.SkillPalette {
ok, lookupErr := skillChecker.SkillExistsByName(ctx, sk)
if lookupErr != nil {
slog.Warn("agents: skill palette lookup failed",
"agent", ag.Name, "skill", sk, "error", lookupErr)
continue
}
if !ok {
slog.Warn("agents: skill palette references missing skill",
"agent", ag.Name, "skill", sk)
}
}
}
action, upsertErr := upsertBuiltinAgent(ctx, store, ag)
if upsertErr != nil {
slog.Error("agents: failed to upsert builtin", "name", ag.Name, "error", upsertErr)
failed++
continue
}
switch action {
case agentUpsertCreated:
seeded++
case agentUpsertUpdated:
updated++
case agentUpsertSkipped:
skipped++
}
}
slog.Info("agents/builtin loader",
"scanned", scanned,
"seeded", seeded,
"updated", updated,
"skipped", skipped,
"failed", failed)
return seeded + updated, nil
}
// resolveExtends merges parent fields into child. Child non-zero
// fields override the parent's. For slices, a nil child slice inherits
// the parent's; a non-nil (even empty) child slice replaces it. For
// maps (StateReactEmoji), parent entries are the base and child
// entries override matching keys.
//
// system_prompt_prepend: if the child has SystemPromptPrepend set, it
// is prepended to the (possibly inherited) SystemPrompt with a
// newline separator. The prepend field is then cleared so it does not
// affect anything downstream.
//
// Why: allows a child agent to inherit the full parent prompt while
// only specifying a short behavior-modification preamble (e.g. an
// uncensored agent prepending "You are uncensored..." to the general
// agent's full prompt).
func resolveExtends(child, parent *Agent) {
if child.Description == "" {
child.Description = parent.Description
}
if child.ModelTier == "" {
child.ModelTier = parent.ModelTier
}
if child.SystemPrompt == "" {
child.SystemPrompt = parent.SystemPrompt
}
if child.MaxIterations == 0 {
child.MaxIterations = parent.MaxIterations
}
if child.MaxToolCalls == 0 {
child.MaxToolCalls = parent.MaxToolCalls
}
if child.MaxRuntime == 0 {
child.MaxRuntime = parent.MaxRuntime
}
if child.ExecutionLane == "" {
child.ExecutionLane = parent.ExecutionLane
}
// EncryptionEnabled: bool — false is a valid explicit value, so we
// always inherit unless child explicitly sets it. Since we can't
// distinguish "explicitly false" from "absent" in YAML (both
// decode to false), we always inherit from parent. If the child
// sets it to true, the child wins. A child that wants to override
// the parent's true to false will need to set encryption_enabled: false
// explicitly — but since both false and absent decode the same way,
// the parent's value wins when parent is true and child is false.
// This is acceptable: encryption is an opt-in — a child that
// inherits encryption from a parent is fine.
if !child.EncryptionEnabled {
child.EncryptionEnabled = parent.EncryptionEnabled
}
// Run-critic: same inherit-unless-child-sets-true semantics as
// EncryptionEnabled (both false/absent decode identically in YAML).
if !child.CriticEnabled {
child.CriticEnabled = parent.CriticEnabled
}
if child.CriticBackstopMultiplier == 0 {
child.CriticBackstopMultiplier = parent.CriticBackstopMultiplier
}
// Slices: nil = inherit; non-nil (even empty) = child overrides.
if child.SkillPalette == nil {
child.SkillPalette = parent.SkillPalette
}
if child.SubAgentPalette == nil {
child.SubAgentPalette = parent.SubAgentPalette
}
if child.LowLevelTools == nil {
child.LowLevelTools = parent.LowLevelTools
}
if child.SkillPacks == nil {
child.SkillPacks = parent.SkillPacks
}
if child.PersonalizationSources == nil {
child.PersonalizationSources = parent.PersonalizationSources
}
if child.Tags == nil {
child.Tags = parent.Tags
}
if child.WebhookIPAllowlist == nil {
child.WebhookIPAllowlist = parent.WebhookIPAllowlist
}
if child.Phases == nil {
child.Phases = parent.Phases
}
// Triggers (Schedule, ChatbotChannelFilter, WebhookSecret, …) are
// deliberately NOT inherited. A trigger is an ACTIVATION decision —
// "this agent fires on a schedule" / "this agent is a chatbot tool in
// these channels" — and silently inheriting it from a parent persona
// is a behavioural surprise: `uncensored extends general` would inherit
// general's `chatbot_channel_filter: "none"` (match-every-channel) and
// surface the unfiltered model as a direct chatbot tool everywhere the
// instant agents.triggers.enabled flips on. A child that wants a trigger
// must declare it explicitly. (Persona, caps, palette, and tools are
// inherited above — those are capability, not activation.)
// DefaultEmoji: child wins if set; otherwise inherit.
if child.DefaultEmoji == "" {
child.DefaultEmoji = parent.DefaultEmoji
}
// Maps: merge — parent is the base, child entries override.
if child.StateReactEmoji == nil && parent.StateReactEmoji != nil {
child.StateReactEmoji = make(map[string]string, len(parent.StateReactEmoji))
for k, v := range parent.StateReactEmoji {
child.StateReactEmoji[k] = v
}
} else if parent.StateReactEmoji != nil {
merged := make(map[string]string, len(parent.StateReactEmoji)+len(child.StateReactEmoji))
for k, v := range parent.StateReactEmoji {
merged[k] = v
}
for k, v := range child.StateReactEmoji {
merged[k] = v
}
child.StateReactEmoji = merged
}
// SystemPromptPrepend: prepend to the (now resolved) SystemPrompt.
if child.SystemPromptPrepend != "" {
child.SystemPrompt = child.SystemPromptPrepend + "\n\n" + child.SystemPrompt
child.SystemPromptPrepend = "" // consumed
}
// Clear Extends — the resolution is complete, the persisted agent
// is standalone.
child.Extends = ""
}
// agentUpsertAction reports what upsertBuiltinAgent did. Exported only
// to the test in this package; the loader's public surface returns a
// count, not a per-row action.
type agentUpsertAction int
const (
agentUpsertCreated agentUpsertAction = iota
agentUpsertUpdated
agentUpsertSkipped // reserved; current loader never returns this — every parse-OK row is upserted
)
// upsertBuiltinAgent looks up an existing (BuiltinAgentOwnerID, name)
// row. If absent, inserts a new row with a freshly-minted UUID.
// Otherwise updates the existing row in place, preserving ID + CreatedAt.
//
// Why not version-skip like skills.upsertBuiltin: the Agent struct has
// a Version int field but it's a monotonic counter, not a semver
// string for change detection. Agent YAML doesn't carry a "version"
// at the wire shape; every boot writes the latest YAML content,
// trusting the YAML file in-repo IS the source of truth. The Agent's
// internal Version int auto-increments on each loader pass so admin
// inspection (`.agent show`) reveals "how many times has the loader
// touched this row".
func upsertBuiltinAgent(ctx context.Context, store Storage, fresh *Agent) (agentUpsertAction, error) {
existing, err := store.GetAgentByName(ctx, BuiltinAgentOwnerID, fresh.Name)
if err != nil && !errors.Is(err, ErrNotFound) {
return agentUpsertCreated, fmt.Errorf("lookup builtin agent %q: %w", fresh.Name, err)
}
if errors.Is(err, ErrNotFound) {
fresh.ID = uuid.New().String()
fresh.OwnerID = BuiltinAgentOwnerID
fresh.AuthoredBy = BuiltinAgentOwnerID
if fresh.Version == 0 {
fresh.Version = 1
}
now := time.Now()
fresh.CreatedAt = now
fresh.UpdatedAt = now
if saveErr := store.SaveAgent(ctx, fresh); saveErr != nil {
return agentUpsertCreated, saveErr
}
slog.Info("agents: created builtin", "name", fresh.Name, "id", fresh.ID)
return agentUpsertCreated, nil
}
// Update in place. Preserve ID, OwnerID, AuthoredBy, CreatedAt.
// Bump Version so admins can see "the loader has touched this N
// times" — useful when investigating a builtin that was
// hand-edited via the future web UI and unexpectedly reverted on
// next boot.
fresh.ID = existing.ID
fresh.OwnerID = BuiltinAgentOwnerID
fresh.AuthoredBy = BuiltinAgentOwnerID
fresh.Version = existing.Version + 1
fresh.CreatedAt = existing.CreatedAt
fresh.UpdatedAt = time.Now()
// Carry forward operator/scheduler-owned fields that the manifest
// never sets (decodeAgentManifest leaves these zero by design — a
// secret in-repo would be a credential leak). Without this, every
// boot CLOBBERS an operator-armed webhook secret + signature flag
// back to empty/false and nukes the scheduler's next-fire cursor, so
// a scheduled or webhook-armed builtin silently breaks on each deploy.
fresh.WebhookSecret = existing.WebhookSecret
fresh.WebhookSignatureRequired = existing.WebhookSignatureRequired
fresh.NextRunAt = existing.NextRunAt
fresh.LastScheduledRunAt = existing.LastScheduledRunAt
if saveErr := store.SaveAgent(ctx, fresh); saveErr != nil {
return agentUpsertUpdated, saveErr
}
slog.Info("agents: updated builtin",
"name", fresh.Name, "id", fresh.ID, "version", fresh.Version)
return agentUpsertUpdated, nil
}
// builtinAgentManifest is the YAML wire format for agents/<name>/agent.yml.
// The schema is intentionally a SUBSET of the Agent struct — future
// fields can be added without breaking existing manifests so long as
// we keep KnownFields(true) decoding (so a typo on a key surfaces as
// an error rather than silently dropping data).
//
// See pkg/logic/agents/CLAUDE.md for the schema reference.
type builtinAgentManifest struct {
Name string `yaml:"name"`
Description string `yaml:"description"`
ModelTier string `yaml:"model_tier"`
SystemPrompt string `yaml:"system_prompt"`
SystemPromptPrepend string `yaml:"system_prompt_prepend"`
MaxIterations int `yaml:"max_iterations"`
MaxToolCalls int `yaml:"max_tool_calls"`
MaxRuntimeSeconds int `yaml:"max_runtime_seconds"`
ExecutionLane string `yaml:"execution_lane"`
EncryptionEnabled bool `yaml:"encryption_enabled"`
// Run-critic two-tier timeout. CriticEnabled flips MaxRuntime from a
// hard kill into a soft trigger; CriticBackstopMultiplier (0 => convar
// default 6×) sets the hard backstop = MaxRuntime × multiplier.
CriticEnabled bool `yaml:"critic_enabled"`
CriticBackstopMultiplier float64 `yaml:"critic_backstop_multiplier"`
// Extends names a parent agent whose fields are inherited. The
// child's non-zero fields override the parent; nil/empty slices
// inherit the parent's. Maps (state_react) are merged — child
// entries override parent entries with the same key. Only single-
// level extends is supported (no chains).
Extends string `yaml:"extends"`
SkillPalette []string `yaml:"skill_palette"`
SubAgentPalette []string `yaml:"sub_agent_palette"`
LowLevelTools []string `yaml:"low_level_tools"`
SkillPacks []string `yaml:"skill_packs"`
PersonalizationSources []string `yaml:"personalization_sources"`
// Triggers — builtin agents typically don't ship with triggers
// (admins flip these on per-deployment), but the keys are accepted
// so a sufficiently sophisticated builtin (e.g. a scheduled "weekly
// digest" agent) can ship triggers in-repo. Default empty.
Schedule string `yaml:"schedule"`
WebhookIPAllowlist []string `yaml:"webhook_ip_allowlist"`
ChatbotChannelFilter string `yaml:"chatbot_channel_filter"`
DefaultEmoji string `yaml:"default_emoji"`
StateReact map[string]string `yaml:"state_react"`
Tags []string `yaml:"tags"`
// Pipeline phases — when non-empty, the executor runs the
// sequential phase runner instead of the single agent loop.
Phases []builtinAgentPhaseManifest `yaml:"phases"`
}
// builtinAgentPhaseManifest is the YAML wire format for a single
// phases list entry in agents/<name>/agent.yml. Maps 1:1 to
// AgentPhase at decode time.
type builtinAgentPhaseManifest struct {
Name string `yaml:"name"`
SystemPrompt string `yaml:"system_prompt"`
ModelTier string `yaml:"model_tier"`
MaxIter int `yaml:"max_iter"`
Tools []string `yaml:"tools"`
Optional bool `yaml:"optional"`
FallbackMessage string `yaml:"fallback_message"`
IsRunFunc bool `yaml:"is_run_func"`
}
// decodeAgentManifest parses an agent.yml bundle into a domain Agent.
// Uses KnownFields(true) so a typo'd key surfaces as a parse error
// rather than silently dropping the value.
//
// What this method does NOT set:
// - ID (loader mints UUID on insert / preserves existing on update)
// - OwnerID + AuthoredBy (loader sets to BuiltinAgentOwnerID)
// - Version (loader increments on update)
// - CreatedAt + UpdatedAt (loader stamps)
// - WebhookSecret (operator generates via admin tooling at deploy
// time — shipping a secret in-repo would be a credential leak)
// - NextRunAt + LastScheduledRunAt (scheduler bookkeeping; nil at
// load time, populated on first scheduled fire)
// - WebhookSignatureRequired (application-layer default applies on
// first save; a `default:true` GORM tag would substitute on every
// write — see the v8 lesson on this exact trap)
func decodeAgentManifest(data []byte) (*Agent, error) {
var m builtinAgentManifest
dec := yaml.NewDecoder(strings.NewReader(string(data)))
dec.KnownFields(true)
if err := dec.Decode(&m); err != nil {
return nil, fmt.Errorf("decode agent.yml: %w", err)
}
if strings.TrimSpace(m.Name) == "" {
return nil, errors.New("agent.yml: missing required field 'name'")
}
// system_prompt is required UNLESS the agent uses extends (the parent
// will supply it) or system_prompt_prepend (the prepend will be
// combined with the parent's system_prompt after extends resolution).
if strings.TrimSpace(m.SystemPrompt) == "" && strings.TrimSpace(m.Extends) == "" && strings.TrimSpace(m.SystemPromptPrepend) == "" {
return nil, errors.New("agent.yml: missing required field 'system_prompt'")
}
// Convert YAML phase manifests to domain AgentPhase structs.
var phases []AgentPhase
for _, pm := range m.Phases {
if strings.TrimSpace(pm.Name) == "" {
return nil, errors.New("agent.yml: phase missing required field 'name'")
}
phases = append(phases, AgentPhase{
Name: strings.TrimSpace(pm.Name),
SystemPrompt: pm.SystemPrompt,
ModelTier: strings.TrimSpace(pm.ModelTier),
MaxIter: pm.MaxIter,
Tools: pm.Tools,
Optional: pm.Optional,
FallbackMessage: pm.FallbackMessage,
IsRunFunc: pm.IsRunFunc,
})
}
// Validate the webhook IP allow-list (CIDR or bare IP); drop + warn on
// malformed entries so a typo can't silently widen or void the allow-list.
allowlist := validateIPAllowlist(m.WebhookIPAllowlist, m.Name)
ag := &Agent{
Name: strings.TrimSpace(m.Name),
Description: m.Description,
Extends: strings.TrimSpace(m.Extends),
SystemPromptPrepend: m.SystemPromptPrepend,
ModelTier: strings.TrimSpace(m.ModelTier),
SystemPrompt: m.SystemPrompt,
MaxIterations: m.MaxIterations,
MaxToolCalls: m.MaxToolCalls,
MaxRuntime: time.Duration(m.MaxRuntimeSeconds) * time.Second,
ExecutionLane: strings.TrimSpace(m.ExecutionLane),
EncryptionEnabled: m.EncryptionEnabled,
CriticEnabled: m.CriticEnabled,
CriticBackstopMultiplier: m.CriticBackstopMultiplier,
SkillPalette: m.SkillPalette,
SubAgentPalette: m.SubAgentPalette,
LowLevelTools: m.LowLevelTools,
SkillPacks: m.SkillPacks,
PersonalizationSources: m.PersonalizationSources,
Schedule: strings.TrimSpace(m.Schedule),
WebhookIPAllowlist: allowlist,
ChatbotChannelFilter: strings.TrimSpace(m.ChatbotChannelFilter),
DefaultEmoji: m.DefaultEmoji,
StateReactEmoji: m.StateReact,
Tags: m.Tags,
Phases: phases,
}
return ag, nil
}
// validateIPAllowlist keeps only entries that parse as a CIDR block or a bare
// IP; malformed entries are dropped with a warning (a typo must not silently
// widen or void the webhook allow-list). The struct field documents "CIDR
// strings", so this enforces it at load time.
func validateIPAllowlist(entries []string, agent string) []string {
var out []string
for _, e := range entries {
e = strings.TrimSpace(e)
if e == "" {
continue
}
if _, _, err := net.ParseCIDR(e); err == nil {
out = append(out, e)
continue
}
if ip := net.ParseIP(e); ip != nil {
out = append(out, e)
continue
}
slog.Warn("agents: dropping malformed webhook_ip_allowlist entry (not a CIDR or IP)", "agent", agent, "entry", e)
}
return out
}
+17
View File
@@ -0,0 +1,17 @@
package persona
import "testing"
func TestValidateIPAllowlist(t *testing.T) {
in := []string{"10.0.0.0/8", " 192.168.1.5 ", "not-an-ip", "", "2001:db8::/32", "garbage/99"}
got := validateIPAllowlist(in, "test")
want := map[string]bool{"10.0.0.0/8": true, "192.168.1.5": true, "2001:db8::/32": true}
if len(got) != len(want) {
t.Fatalf("got %v, want %d valid entries", got, len(want))
}
for _, e := range got {
if !want[e] {
t.Errorf("unexpected entry kept: %q", e)
}
}
}
+120
View File
@@ -0,0 +1,120 @@
package persona
import (
"context"
"sort"
"sync"
"time"
)
// Memory is a zero-dependency in-process Storage for agent personas — a light
// host (or tests) gets persona persistence with no DB. Mort keeps its
// GORM/MySQL Storage; contrib/store adds a durable SQLite one.
type Memory struct {
mu sync.RWMutex
agents map[string]*Agent // by ID
}
// NewMemory returns an empty in-memory persona Storage.
func NewMemory() *Memory { return &Memory{agents: map[string]*Agent{}} }
var _ Storage = (*Memory)(nil)
func (m *Memory) InitializeAgentStorage(context.Context) error { return nil }
func (m *Memory) SaveAgent(_ context.Context, a *Agent) error {
m.mu.Lock()
defer m.mu.Unlock()
cp := *a
m.agents[a.ID] = &cp
return nil
}
func (m *Memory) GetAgent(_ context.Context, id string) (*Agent, error) {
m.mu.RLock()
defer m.mu.RUnlock()
a, ok := m.agents[id]
if !ok {
return nil, ErrNotFound
}
cp := *a
return &cp, nil
}
func (m *Memory) GetAgentByName(_ context.Context, ownerID, name string) (*Agent, error) {
m.mu.RLock()
defer m.mu.RUnlock()
for _, a := range m.agents {
if a.OwnerID == ownerID && a.Name == name {
cp := *a
return &cp, nil
}
}
return nil, ErrNotFound
}
func (m *Memory) listWhere(keep func(*Agent) bool) []*Agent {
m.mu.RLock()
defer m.mu.RUnlock()
out := make([]*Agent, 0, len(m.agents))
for _, a := range m.agents {
if keep == nil || keep(a) {
cp := *a
out = append(out, &cp)
}
}
sort.Slice(out, func(i, j int) bool { return out[i].Name < out[j].Name })
return out
}
func (m *Memory) ListAgents(_ context.Context, ownerID string) ([]*Agent, error) {
return m.listWhere(func(a *Agent) bool { return a.OwnerID == ownerID }), nil
}
func (m *Memory) ListAllAgents(context.Context) ([]*Agent, error) {
return m.listWhere(nil), nil
}
func (m *Memory) DeleteAgent(_ context.Context, id string) error {
m.mu.Lock()
defer m.mu.Unlock()
delete(m.agents, id)
return nil
}
func (m *Memory) GetAgentByWebhookSecret(_ context.Context, secret string) (*Agent, error) {
if secret == "" {
return nil, ErrNotFound
}
m.mu.RLock()
defer m.mu.RUnlock()
for _, a := range m.agents {
if a.WebhookSecret == secret {
cp := *a
return &cp, nil
}
}
return nil, ErrNotFound
}
func (m *Memory) ListAgentsByChatbotChannelFilter(context.Context) ([]*Agent, error) {
return m.listWhere(func(a *Agent) bool { return a.ChatbotChannelFilter != "" }), nil
}
func (m *Memory) ListScheduledAgents(_ context.Context, dueBefore time.Time) ([]*Agent, error) {
return m.listWhere(func(a *Agent) bool {
return a.Schedule != "" && a.NextRunAt != nil && !a.NextRunAt.After(dueBefore)
}), nil
}
func (m *Memory) MarkAgentScheduledRun(_ context.Context, agentID string, ranAt, nextAt time.Time) error {
m.mu.Lock()
defer m.mu.Unlock()
a, ok := m.agents[agentID]
if !ok {
return ErrNotFound
}
a.LastScheduledRunAt = &ranAt
a.NextRunAt = &nextAt
return nil
}
+45
View File
@@ -0,0 +1,45 @@
package persona
import (
"context"
"testing"
"time"
)
func TestToRunnable(t *testing.T) {
a := &Agent{
ID: "id1", Name: "helper", SystemPrompt: "be nice", ModelTier: "fast",
MaxIterations: 5, MaxRuntime: 30 * time.Second,
LowLevelTools: []string{"think"}, SkillPalette: []string{"animate"},
CriticEnabled: true, CriticBackstopMultiplier: 2,
Phases: []AgentPhase{{Name: "p1", ModelTier: "thinking", MaxIter: 3, Tools: []string{"now"}, Optional: true}},
}
r := a.ToRunnable()
if r.ID != "id1" || r.ModelTier != "fast" || r.MaxIterations != 5 || !r.Critic.Enabled {
t.Fatalf("ToRunnable mapping wrong: %+v", r)
}
if len(r.Phases) != 1 || r.Phases[0].MaxIterations != 3 || !r.Phases[0].Optional {
t.Fatalf("phase mapping wrong: %+v", r.Phases)
}
}
func TestMemoryStoreRoundTrip(t *testing.T) {
ctx := context.Background()
m := NewMemory()
a := &Agent{ID: "a1", Name: "n", OwnerID: "o1"}
if err := m.SaveAgent(ctx, a); err != nil {
t.Fatal(err)
}
got, err := m.GetAgent(ctx, "a1")
if err != nil || got.Name != "n" {
t.Fatalf("GetAgent: %v %+v", err, got)
}
byName, err := m.GetAgentByName(ctx, "o1", "n")
if err != nil || byName.ID != "a1" {
t.Fatalf("GetAgentByName: %v %+v", err, byName)
}
list, _ := m.ListAgents(ctx, "o1")
if len(list) != 1 {
t.Fatalf("ListAgents = %d", len(list))
}
}
+38
View File
@@ -0,0 +1,38 @@
package persona
import "gitea.stevedudenhoeffer.com/steve/executus/run"
// ToRunnable lowers an Agent persona into the kernel's run.RunnableAgent DTO —
// the bridge that lets run.Executor run a persona WITHOUT importing this
// battery (the inversion of mort's agentexec.Run(*agents.Agent)). It maps the
// static shape only; per-run personalization, palette resolution, the critic,
// audit, etc. are supplied separately via run.Ports.
func (a *Agent) ToRunnable() run.RunnableAgent {
ra := run.RunnableAgent{
ID: a.ID,
Name: a.Name,
SystemPrompt: a.SystemPrompt,
ModelTier: a.ModelTier,
MaxIterations: a.MaxIterations,
MaxRuntime: a.MaxRuntime,
LowLevelTools: a.LowLevelTools,
SkillPalette: a.SkillPalette,
SubAgentPalette: a.SubAgentPalette,
SkillPacks: a.SkillPacks,
Critic: run.CriticConfig{
Enabled: a.CriticEnabled,
BackstopMultiplier: a.CriticBackstopMultiplier,
},
}
for _, p := range a.Phases {
ra.Phases = append(ra.Phases, run.Phase{
Name: p.Name,
SystemPrompt: p.SystemPrompt,
ModelTier: p.ModelTier,
MaxIterations: p.MaxIter,
Tools: p.Tools,
Optional: p.Optional,
})
}
return ra
}
+115
View File
@@ -0,0 +1,115 @@
package persona
import (
"context"
"errors"
"time"
)
// ErrNotFound is returned when an agent lookup fails. Callers compare
// with errors.Is(err, ErrNotFound).
var ErrNotFound = errors.New("agent not found")
// Storage is the persistence interface for the agents system.
//
// Why: tests substitute fake implementations; production wires
// through pkg/logic/storage's Grand Storage which embeds this
// interface. Mirrors the three-layer pattern in
// pkg/logic/storage/CLAUDE.md (domain → GORM → DB).
//
// What: Phase 1 CRUD plus Phase 3 trigger queries
// (ListDueScheduled, GetAgentByWebhookSecret,
// ListAgentsByChatbotChannelFilter, MarkScheduledRun). Trigger
// queries are read by the agentsched runner, webhook router, and
// chatbot tool provider; all are gated behind the
// agents.triggers.enabled convar so old skill-driven paths keep
// running until the convar flips.
//
// Test: see storage_round_trip_test.go for round-trip coverage.
type Storage interface {
// (Mort's Discord command-binding CRUD — the CommandBindingStorage
// embedding — stays a host concern and is NOT part of the executus
// persona Storage seam.)
// InitializeAgentStorage prepares storage (e.g. AutoMigrate)
// and is idempotent. Called from the grand storage's
// InitializeAll path.
InitializeAgentStorage(ctx context.Context) error
// SaveAgent creates or updates an Agent by ID. ID must be
// non-empty (Phase 1 admin commands mint a UUID).
SaveAgent(ctx context.Context, a *Agent) error
// GetAgent returns the agent with the given ID, or ErrNotFound.
GetAgent(ctx context.Context, id string) (*Agent, error)
// GetAgentByName resolves (owner_id, name) → agent. ownerID
// must match exactly (Phase 1 has no shared/public visibility
// yet; every agent is owned).
GetAgentByName(ctx context.Context, ownerID, name string) (*Agent, error)
// ListAgents returns every agent owned by the given member ID,
// sorted by Name ASC.
ListAgents(ctx context.Context, ownerID string) ([]*Agent, error)
// ListAllAgents returns every agent across all owners, sorted by
// (OwnerID ASC, Name ASC) so builtin rows (OwnerID="builtin")
// group together, then numeric Discord-ID owners in lexical order,
// then chatbot-shadow rows whose OwnerID is the chatbot owner's
// Discord ID but whose Name carries the "chatbot:" prefix.
//
// Why: Phase 1 admin commands ran owner-scoped (a steve-owned
// agent list shows ONLY steve's rows), which hid builtin and
// shadow Agents from the admin view. `.agent list` for admins now
// uses this method to surface every row. Non-admin invocations
// (or `.agent list --mine`) keep using ListAgents.
//
// Storage MAY back this with a single full-table scan — admin
// row counts are small (dozens to low hundreds), so no need for
// pagination at this phase.
ListAllAgents(ctx context.Context) ([]*Agent, error)
// DeleteAgent removes an agent by ID. Idempotent — deleting a
// missing row returns nil.
DeleteAgent(ctx context.Context, id string) error
// GetAgentByWebhookSecret resolves a posted /webhooks/<secret> URL
// to the matching agent. Returns ErrNotFound when no agent has
// the secret. Phase 3 webhook router consults this AFTER the
// existing Skill lookup falls through, but only when
// agents.triggers.enabled is true.
//
// Empty secret is rejected with ErrNotFound (empty WebhookSecret
// rows are NOT webhook-enabled — the application layer guards
// this, the lookup defends against accidental match).
GetAgentByWebhookSecret(ctx context.Context, secret string) (*Agent, error)
// ListAgentsByChatbotChannelFilter returns every agent with a
// non-empty ChatbotChannelFilter. Phase 3 chatbot tool provider
// uses this on every chatbot turn to assemble the per-channel
// tool list (gated by agents.triggers.enabled). The result is
// not channel-filtered here — the provider applies the channel
// filter predicate (registered in skills.ChannelFilterRegistry)
// to each row.
//
// Why no channel filter at the storage layer: the filter is a
// runtime predicate (e.g. dm_only depends on the live Discord
// channel kind cache), not a static column we can index on.
ListAgentsByChatbotChannelFilter(ctx context.Context) ([]*Agent, error)
// ListScheduledAgents returns every agent with a non-empty
// Schedule whose NextRunAt is at or before `dueBefore`. Result
// is ordered by NextRunAt ASC so the scheduler runner can drain
// in oldest-due-first order. Mirrors skills.Storage.ListDueScheduled.
//
// Phase 3 scheduler reads this on every tick when
// agents.triggers.enabled is true. The (Schedule, NextRunAt)
// composite index backs the query — see gorm tags on gormAgent.
ListScheduledAgents(ctx context.Context, dueBefore time.Time) ([]*Agent, error)
// MarkAgentScheduledRun atomically updates LastScheduledRunAt
// and NextRunAt for the given agent. Called by the agentsched
// runner after each scheduled invocation. Mirrors
// skills.Storage.MarkScheduledRun semantics.
MarkAgentScheduledRun(ctx context.Context, agentID string, ranAt, nextAt time.Time) error
}
+93
View File
@@ -0,0 +1,93 @@
package run
import "time"
// RunnableAgent is the kernel's view of "a thing to run": an identity, a model
// tier, a system prompt, execution caps, and a tool palette. It is a plain DTO
// on purpose — the run kernel never imports a noun battery. The persona Agent
// and the saved Skill each LOWER themselves into a RunnableAgent (a ToRunnable
// method on the battery side), and the kernel runs the DTO. This is the
// inversion of mort's agentexec.Executor.Run(*agents.Agent): the executor no
// longer depends on the persona struct, only on this shape.
//
// A light host can build a RunnableAgent inline (model tier + prompt + a few
// tool names) for a one-shot bounded run, with no persona or skill battery at
// all — that is exactly gadfly's swarm task.
type RunnableAgent struct {
// ID is a stable identifier for the run subject (an agent/skill UUID, or
// any host-chosen id). Used for audit attribution and dispatch-guard
// genealogy. Empty is allowed for anonymous one-shot runs.
ID string
// Name is a human label (audit/logs/delivery). Empty is allowed.
Name string
// SystemPrompt is the agent's base system prompt (before per-run
// personalization, which a host layers via Ports).
SystemPrompt string
// ModelTier is a tier alias or concrete spec resolved through
// model.ParseModelForContext. Empty resolves to the host's default tier.
ModelTier string
// MaxIterations caps the agent loop's tool-dispatch steps. 0 = kernel
// default. MaxRuntime caps wall-clock for the whole run (the kernel starts
// this clock AFTER any lane dequeue, not at submission). 0 = kernel
// default.
MaxIterations int
MaxRuntime time.Duration
// LowLevelTools are tool-registry names the run may call directly.
// SkillPalette / SubAgentPalette name saved skills / sub-agents exposed as
// skill__<name> / agent__<name> delegation tools, resolved through
// Ports.Palette (nil Palette => those entries are inert).
LowLevelTools []string
SkillPalette []string
SubAgentPalette []string
// SkillPacks names SKILL.md skill-pack subscriptions activated for the run
// via Ports.SkillPacks: each pack's name+description joins a catalog folded
// into the system prompt, and a skill_use tool loads a pack's body on demand
// (progressive disclosure). nil Ports.SkillPacks => inert.
SkillPacks []string
// Phases optionally model a multi-step pipeline (each phase its own prompt
// + tier + tools). An empty slice is a single-phase run — the common case.
Phases []Phase
// Critic configures the optional two-tier run-critic (Ports.Critic). The
// zero value (disabled) is the light-host default.
Critic CriticConfig
}
// Phase is one step of a multi-step run: its own system prompt, model tier,
// iteration cap, and tool subset. Phase prompts are Go text/template strings
// expanded against {{.Query}} (the original input) and {{.<PhaseName>}} (a
// prior phase's output) before the phase runs, so a phase can consume earlier
// work. The final phase's output is the run's output.
type Phase struct {
Name string
SystemPrompt string
ModelTier string
MaxIterations int
Tools []string
// Optional swallows a phase's error and substitutes FallbackMessage (or a
// generated note) as its output, so a non-critical phase failing does not
// abort the pipeline.
Optional bool
// FallbackMessage is the substitute output when an Optional phase fails.
// Empty → a generated "(phase %q encountered an error…)" note.
FallbackMessage string
// IsRunFunc marks a phase as a single bare LLM call (no tool loop, no tools
// array) — a deterministic transform step (plan/synthesize) rather than an
// agentic loop. Its Tools/MaxIterations are ignored.
IsRunFunc bool
}
// CriticConfig configures the optional run-critic. Enabled gates whether a
// critic monitor is started at all; BackstopMultiplier sets the hard-kill
// deadline as a multiple of the soft trigger (MaxRuntime). A non-positive
// multiplier uses the kernel default.
type CriticConfig struct {
Enabled bool
BackstopMultiplier float64
}
+103
View File
@@ -0,0 +1,103 @@
package run
import (
"context"
"errors"
"log/slog"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
)
// Durable-recovery plumbing for the executor. The Checkpointer port (set via
// Ports.Checkpointer, a CheckpointerFactory) persists a run's resumable progress
// during the loop; on boot a host re-dispatches an interrupted run through the
// executor with a ResumeState (the saved transcript / completed phases) so it
// CONTINUES rather than restarting, reusing the SAME durable record via an
// existing Checkpointer. Both are carried into Run via the context (mirrors
// mort's agentexec.WithResumeState / WithExistingCheckpointer).
// ResumeState carries a recovered run's prior progress into Run so the run
// continues instead of restarting. The host's recovery path sets it via
// WithResumeState; the executor reads it:
// - single-loop: History seeds the saved transcript (the run continues).
// - multi-phase: CompletedPhases are skipped; the interrupted phase re-runs
// from its start (boundary-granular — there is no mid-phase transcript
// resume, so History is unused for multi-phase runs).
type ResumeState struct {
History []llm.Message // single-loop transcript (unused for multi-phase)
CompletedPhases []PhaseOutput // multi-phase: outputs of finished phases, in order
}
type resumeStateKey struct{}
// WithResumeState carries a recovered run's prior progress into Run.
func WithResumeState(ctx context.Context, rs *ResumeState) context.Context {
return context.WithValue(ctx, resumeStateKey{}, rs)
}
func resumeStateFromContext(ctx context.Context) *ResumeState {
rs, _ := ctx.Value(resumeStateKey{}).(*ResumeState)
return rs
}
type existingCheckpointerKey struct{}
// WithExistingCheckpointer carries a pre-existing Checkpointer into Run so a
// recovery re-run reuses the SAME durable record (the executor uses it instead of
// calling Ports.Checkpointer.Begin).
func WithExistingCheckpointer(ctx context.Context, cp Checkpointer) context.Context {
return context.WithValue(ctx, existingCheckpointerKey{}, cp)
}
func existingCheckpointerFromContext(ctx context.Context) Checkpointer {
cp, _ := ctx.Value(existingCheckpointerKey{}).(Checkpointer)
return cp
}
// checkpointOutcome is the finalize decision for a durable run.
type checkpointOutcome int
const (
checkpointComplete checkpointOutcome = iota
checkpointLeaveRunning
checkpointFail
)
// classifyCheckpointOutcome maps (run error, cancellation cause) to the durable
// finalize action: success clears the checkpoint (Complete); a shutdown-caused
// cancellation leaves the record so boot recovery picks it up (neither
// Complete nor Fail); anything else (model error, tool loop, the run's own
// deadline, a critic kill, a caller cancel) is terminal (Fail). Mirrors mort's
// agentexec.classifyCheckpointOutcome.
func classifyCheckpointOutcome(runErr, cause error) checkpointOutcome {
switch {
case runErr == nil:
return checkpointComplete
case errors.Is(cause, ErrShutdown):
return checkpointLeaveRunning
default:
return checkpointFail
}
}
// finalizeCheckpoint applies the outcome to the per-run checkpointer (nil-safe).
// Runs on a detached context so a cancelled run still records its terminal state.
// Complete/Fail errors are best-effort but logged (a stale record would only
// cause a wasteful boot-recovery retry, not data loss).
func finalizeCheckpoint(ctx context.Context, cp Checkpointer, runErr error, cause error) {
if cp == nil {
return
}
switch classifyCheckpointOutcome(runErr, cause) {
case checkpointComplete:
if err := cp.Complete(detach(ctx)); err != nil {
slog.Warn("run: checkpoint Complete failed", "error", err)
}
case checkpointFail:
if err := cp.Fail(detach(ctx), runErr); err != nil {
slog.Warn("run: checkpoint Fail failed", "error", err)
}
case checkpointLeaveRunning:
// Interrupted by shutdown: leave the record for boot recovery.
}
}
+200
View File
@@ -0,0 +1,200 @@
package run
import (
"context"
"errors"
"testing"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/majordomo/provider/fake"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
// fakeCheckpointer records every Save state + whether Complete/Fail fired.
type fakeCheckpointer struct {
saves []RunCheckpointState
completed bool
failed bool
failErr error
}
func (c *fakeCheckpointer) Save(_ context.Context, st RunCheckpointState) error {
c.saves = append(c.saves, st)
return nil
}
func (c *fakeCheckpointer) Complete(context.Context) error { c.completed = true; return nil }
func (c *fakeCheckpointer) Fail(_ context.Context, err error) error {
c.failed = true
c.failErr = err
return nil
}
// fakeCheckpointFactory hands out one fakeCheckpointer and records the RunInfo.
type fakeCheckpointFactory struct {
cp *fakeCheckpointer
info RunInfo
}
func (f *fakeCheckpointFactory) Begin(_ context.Context, info RunInfo) (Checkpointer, error) {
f.info = info
return f.cp, nil
}
// TestClassifyCheckpointOutcome covers the finalize decision matrix.
func TestClassifyCheckpointOutcome(t *testing.T) {
cases := []struct {
name string
err error
cause error
want checkpointOutcome
}{
{"success", nil, nil, checkpointComplete},
{"shutdown", context.Canceled, ErrShutdown, checkpointLeaveRunning},
{"critic-kill", context.Canceled, ErrCriticKill, checkpointFail},
{"deadline", context.DeadlineExceeded, context.DeadlineExceeded, checkpointFail},
{"model-error", errors.New("boom"), nil, checkpointFail},
{"caller-cancel", context.Canceled, context.Canceled, checkpointFail},
}
for _, tc := range cases {
if got := classifyCheckpointOutcome(tc.err, tc.cause); got != tc.want {
t.Errorf("%s: classifyCheckpointOutcome = %v, want %v", tc.name, got, tc.want)
}
}
}
// TestCheckpoint_SingleLoopSaveAndComplete: a durable single-loop run gets a
// per-run checkpointer (Begin), Saves its transcript each step, and Completes on
// success (clearing the checkpoint). The RunInfo carries the resume meta.
func TestCheckpoint_SingleLoopSaveAndComplete(t *testing.T) {
models, _ := phaseProvider(t, fake.Reply("done"))
cp := &fakeCheckpointer{}
f := &fakeCheckpointFactory{cp: cp}
ex := New(Config{Registry: tool.NewRegistry(), Models: models, Ports: Ports{Checkpointer: f}})
res := ex.Run(context.Background(),
RunnableAgent{ID: "a1", Name: "boss", ModelTier: "test-model"},
tool.Invocation{RunID: "run-x", CallerID: "steve", ChannelID: "chan", GuildID: "g", SkillInputs: map[string]any{"prompt": "go"}},
"go")
if res.Err != nil {
t.Fatalf("run error: %v", res.Err)
}
if f.info.RunID != "run-x" || f.info.SubjectID != "a1" || f.info.ModelTier != "test-model" || f.info.GuildID != "g" {
t.Errorf("Begin RunInfo missing resume meta: %+v", f.info)
}
if len(cp.saves) == 0 {
t.Error("expected at least one checkpoint Save during the run")
} else if len(cp.saves[len(cp.saves)-1].Messages) == 0 {
t.Error("checkpoint Save should carry the running transcript")
}
if !cp.completed {
t.Error("a successful run must Complete (clear) its checkpoint")
}
if cp.failed {
t.Error("a successful run must NOT Fail its checkpoint")
}
}
// TestCheckpoint_TerminalErrorFails: a run that errors (not shutdown) Fails its
// checkpoint (clears it — not a recovery candidate).
func TestCheckpoint_TerminalErrorFails(t *testing.T) {
models, _ := phaseProvider(t, fake.Fail(errors.New("model down")))
cp := &fakeCheckpointer{}
ex := New(Config{Registry: tool.NewRegistry(), Models: models, Ports: Ports{Checkpointer: &fakeCheckpointFactory{cp: cp}}})
res := ex.Run(context.Background(),
RunnableAgent{ID: "a1", ModelTier: "test-model"},
tool.Invocation{RunID: "r", CallerID: "c", SkillInputs: map[string]any{"prompt": "go"}}, "go")
if res.Err == nil {
t.Fatal("expected a run error")
}
if !cp.failed {
t.Error("a terminal (non-shutdown) error must Fail the checkpoint")
}
if cp.completed {
t.Error("a failed run must NOT Complete its checkpoint")
}
}
// TestCheckpoint_ResumeSeedsHistory: a run carrying a ResumeState seeds the saved
// transcript as the model's opening messages (continues) instead of the input.
func TestCheckpoint_ResumeSeedsHistory(t *testing.T) {
models, fp := phaseProvider(t, fake.Reply("continued"))
history := []llm.Message{llm.UserText("prior turn 1"), llm.AssistantText("prior answer 1")}
ctx := WithResumeState(context.Background(), &ResumeState{History: history})
ex := New(Config{Registry: tool.NewRegistry(), Models: models})
res := ex.Run(ctx,
RunnableAgent{ID: "a1", ModelTier: "test-model"},
tool.Invocation{RunID: "r", CallerID: "c", SkillInputs: map[string]any{"prompt": "ignored-on-resume"}}, "ignored-on-resume")
if res.Err != nil {
t.Fatalf("run error: %v", res.Err)
}
got := fp.Calls()[0].Request.Messages
if len(got) != len(history) {
t.Fatalf("resume should seed the saved %d-message transcript, got %d messages", len(history), len(got))
}
}
// TestCheckpoint_PhaseBoundarySavesCompleted: a durable multi-phase run records
// the completed phases at each boundary, growing the list, and Completes on
// success.
func TestCheckpoint_PhaseBoundarySavesCompleted(t *testing.T) {
models, _ := phaseProvider(t, fake.Reply("out-a"), fake.Reply("out-b"))
cp := &fakeCheckpointer{}
ex := New(Config{Registry: tool.NewRegistry(), Models: models, Ports: Ports{Checkpointer: &fakeCheckpointFactory{cp: cp}}})
ra := RunnableAgent{
ID: "p", ModelTier: "test-model",
Phases: []Phase{{Name: "a", SystemPrompt: "A"}, {Name: "b", SystemPrompt: "B"}},
}
if res := ex.Run(context.Background(), ra, tool.Invocation{RunID: "r", CallerID: "c"}, "Q"); res.Err != nil {
t.Fatalf("run error: %v", res.Err)
}
// The final phase-boundary Save must list both completed phases.
var lastPhaseSave *RunCheckpointState
for i := range cp.saves {
if len(cp.saves[i].CompletedPhases) > 0 {
lastPhaseSave = &cp.saves[i]
}
}
if lastPhaseSave == nil || len(lastPhaseSave.CompletedPhases) != 2 {
t.Fatalf("expected a phase-boundary Save listing 2 completed phases; saves=%+v", cp.saves)
}
if !cp.completed {
t.Error("a successful phased run must Complete its checkpoint")
}
}
// TestCheckpoint_ResumeSkipsCompletedPhases: a resumed multi-phase run skips
// phases already in ResumeState.CompletedPhases (only the remaining phase calls
// the model) and threads their outputs into the remaining phase's template.
func TestCheckpoint_ResumeSkipsCompletedPhases(t *testing.T) {
models, fp := phaseProvider(t, fake.Reply("out-b")) // ONLY phase b should call the model
ctx := WithResumeState(context.Background(), &ResumeState{
CompletedPhases: []PhaseOutput{{Name: "a", Output: "saved-a"}},
})
ex := New(Config{Registry: tool.NewRegistry(), Models: models})
ra := RunnableAgent{
ID: "p", ModelTier: "test-model",
Phases: []Phase{
{Name: "a", SystemPrompt: "A"},
{Name: "b", SystemPrompt: "B saw {{.a}}"},
},
}
res := ex.Run(ctx, ra, tool.Invocation{RunID: "r", CallerID: "c"}, "Q")
if res.Err != nil {
t.Fatalf("run error: %v", res.Err)
}
if res.Output != "out-b" {
t.Fatalf("output = %q, want out-b", res.Output)
}
calls := fp.Calls()
if len(calls) != 1 {
t.Fatalf("only the un-completed phase b should call the model; got %d calls", len(calls))
}
if calls[0].Request.System != "B saw saved-a" {
t.Errorf("resumed phase b should see the completed phase a's saved output; system = %q", calls[0].Request.System)
}
}
+136
View File
@@ -0,0 +1,136 @@
package run
import (
"context"
"fmt"
"time"
"gitea.stevedudenhoeffer.com/steve/majordomo/agent"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
)
// criticDeadlineCheck is how often the deadline-watch goroutine polls the
// critic's hard deadline. Small relative to any realistic soft timeout.
const criticDeadlineCheck = time.Second
// criticBinding wires a CriticHandle into a run: the executor forwards activity
// (steps + tool starts) to it, binds the run's hard cancellation to the critic's
// extendable deadline, and exposes the critic's Steer messages as an agent
// RunOption. All methods are nil-safe so the executor can call them
// unconditionally when no critic is configured.
type criticBinding struct {
h CriticHandle
}
// criticOwnsDeadline reports whether a critic is configured AND this run enables
// it — the single predicate that decides the two-tier-timeout path. Used by BOTH
// Run (to choose the generous runaway ceiling over the literal MaxRuntime cap) and
// startCritic (the arm/no-op gate), so the two can never drift.
func (e *Executor) criticOwnsDeadline(ra RunnableAgent) bool {
return e.cfg.Ports.Critic != nil && ra.Critic.Enabled
}
// startCritic begins critic monitoring for this run when one is configured and
// the agent enables it. It launches a goroutine that cancels runCtx (via
// cancelCause) the moment the critic's hard deadline passes — the critic may
// extend that deadline, so a healthy-but-slow run is given room while a hung one
// is killed. When the deadline passes because the critic KILLED the run
// (KillCause() != nil), the cancellation cause is ErrCriticKill (→ status
// "killed"); when the backstop simply expired, it is context.DeadlineExceeded (→
// "timeout"). Returns (nil, no-op stop) when there is no critic. The caller MUST
// defer the returned stop.
//
// softTrigger is the run's resolved MaxRuntime: for a critic-owned run MaxRuntime
// is the soft wake (mort's two-tier semantics — the critic first reviews once the
// run exceeds its nominal budget, and its backstop = softTrigger × multiplier).
// The caller (Run) always passes the resolved MaxRuntime, which withFallbacks
// guarantees is > 0, so no fallback is needed here. (A non-positive soft would make
// the host Monitor return no handle, and Run's unsupervised-run failsafe then bounds
// the run at MaxRuntime — so even that impossible case stays bounded.)
func (e *Executor) startCritic(runCtx context.Context, cancelCause context.CancelCauseFunc, ra RunnableAgent, info RunInfo, softTrigger time.Duration) (*criticBinding, func()) {
noop := func() {}
if !e.criticOwnsDeadline(ra) {
return nil, noop
}
h := e.cfg.Ports.Critic.Monitor(runCtx, info, softTrigger)
if h == nil {
return nil, noop
}
done := make(chan struct{})
go func() {
// A host CriticHandle.Deadline() that panics must not crash the process
// (this runs on its own goroutine, so the executor's top-level recover
// can't catch it). Log-free best-effort: just stop watching.
defer func() { _ = recover() }()
t := time.NewTicker(criticDeadlineCheck)
defer t.Stop()
for {
select {
case <-done:
return
case <-runCtx.Done():
return
case <-t.C:
// A zero deadline = no hard cap (not yet set); otherwise cancel
// once we're at or past it, distinguishing an explicit kill from a
// natural backstop expiry so the run gets the right status.
if d := h.Deadline(); !d.IsZero() && !time.Now().Before(d) {
if cause := h.KillCause(); cause != nil {
cancelCause(fmt.Errorf("%w: %s", ErrCriticKill, cause.Error()))
} else {
cancelCause(context.DeadlineExceeded)
}
return
}
}
}
}()
return &criticBinding{h: h}, func() {
close(done)
h.Stop()
}
}
func (b *criticBinding) recordStep(iter int, resp *llm.Response) {
if b != nil {
b.h.RecordStep(iter, resp)
}
}
// recordToolStart forwards a tool call to the critic. NOTE: majordomo's step
// observer only fires AFTER an iteration completes, so this currently lands
// post-tool, not at dispatch — the activity clock is refreshed once per
// iteration, not mid-tool. A single very long tool call (e.g. a 30-min render)
// therefore won't refresh the clock until it returns; a host that runs such
// tools should feed interim progress to its Critic (mort's InstallProgressBridge
// pattern). A true pre-dispatch refresh needs a majordomo hook (follow-up).
func (b *criticBinding) recordToolStart(name, args string) {
if b != nil {
b.h.RecordToolStart(name, args)
}
}
// maxStepsOption returns the agent step-ceiling Option. With no critic it's a
// fixed WithMaxSteps(base); with a critic it's a DYNAMIC WithMaxStepsFunc that
// polls the handle each step (so the critic can raise a long run's budget),
// falling back to base when the handle defers (MaxSteps() <= 0).
func (b *criticBinding) maxStepsOption(base int) agent.Option {
if b == nil {
return agent.WithMaxSteps(base)
}
return agent.WithMaxStepsFunc(func() int {
if n := b.h.MaxSteps(); n > 0 {
return n
}
return base
})
}
// drainSteer returns the critic's queued steer messages (nil-safe), so the
// executor can merge them with the session steer mailbox into one WithSteer.
func (b *criticBinding) drainSteer() []llm.Message {
if b == nil {
return nil
}
return b.h.Steer()
}
+153
View File
@@ -0,0 +1,153 @@
package run_test
import (
"context"
"sync"
"testing"
"time"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/majordomo/provider/fake"
"gitea.stevedudenhoeffer.com/steve/executus/run"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
// slowToolInvocation builds an Invocation whose session factory adds a "slow"
// tool that sleeps for d (respecting ctx). The model script calls it once, then
// answers — so the run's wall-clock is dominated by d, letting a test set a tiny
// MaxRuntime and observe whether MaxRuntime hard-cancels the run.
func slowToolInvocation(runID string, d time.Duration) tool.Invocation {
slow := llm.DefineTool("slow", "sleeps for a while",
func(ctx context.Context, _ struct{}) (any, error) {
select {
case <-time.After(d):
return "ok", nil
case <-ctx.Done():
return nil, ctx.Err()
}
})
return tool.Invocation{
RunID: runID,
SessionToolFactory: func(_ tool.AgentSession) tool.SessionTools {
return tool.SessionTools{Tools: []llm.Tool{slow}}
},
}
}
func slowModel() llm.Model {
fp := fake.New("fake")
fp.Enqueue("m",
fake.ReplyWith(llm.Response{ToolCalls: []llm.ToolCall{{ID: "c1", Name: "slow", Arguments: []byte(`{}`)}}}),
fake.Reply("done"),
)
m, _ := fp.Model("m")
return m
}
// TestNoCritic_MaxRuntimeIsHardCap: the legacy contract is preserved — without a
// critic, MaxRuntime is a literal WithTimeout that kills a run whose work outlasts
// it. The slow tool (200ms) outlasts MaxRuntime (20ms), so runCtx cancels mid-tool
// and the run ends in error (timeout).
func TestNoCritic_MaxRuntimeIsHardCap(t *testing.T) {
m := slowModel()
ex := run.New(run.Config{
Registry: tool.NewRegistry(),
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) { return ctx, m, nil },
})
res := ex.Run(context.Background(),
run.RunnableAgent{Name: "x", ModelTier: "m", MaxIterations: 5, MaxRuntime: 20 * time.Millisecond},
slowToolInvocation("r", 200*time.Millisecond), "go")
if res.Err == nil {
t.Fatalf("non-critic run should hard-timeout at MaxRuntime; got output=%q err=nil", res.Output)
}
}
// TestCriticOwnsDeadline_SurvivesPastMaxRuntime: the fix — when the critic owns the
// deadline (Ports.Critic set + Critic.Enabled), MaxRuntime becomes the SOFT trigger
// and is NOT a hard cap. The fake critic exposes no hard deadline (Deadline()==zero,
// no kill), so the only hard ceiling is CriticAbsoluteMax (10s here). The slow tool
// (200ms) outlasts the tiny MaxRuntime (20ms) but the run completes — proving the
// old agentexec two-tier semantics are restored.
func TestCriticOwnsDeadline_SurvivesPastMaxRuntime(t *testing.T) {
m := slowModel()
h := &fakeCriticHandle{} // Deadline()==zero → no hard deadline, no kill
ex := run.New(run.Config{
Registry: tool.NewRegistry(),
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) { return ctx, m, nil },
Ports: run.Ports{Critic: &fakeCritic{h: h}},
Defaults: run.Defaults{CriticAbsoluteMax: 10 * time.Second},
})
res := ex.Run(context.Background(),
run.RunnableAgent{Name: "watched", ModelTier: "m", MaxIterations: 5, MaxRuntime: 20 * time.Millisecond,
Critic: run.CriticConfig{Enabled: true}},
slowToolInvocation("r", 200*time.Millisecond), "go")
if res.Err != nil {
t.Fatalf("critic-owned run must survive past MaxRuntime (soft trigger); got err=%v", res.Err)
}
if res.Output != "done" {
t.Errorf("output = %q, want %q", res.Output, "done")
}
}
// capturingCritic records the soft trigger the executor passes to Monitor.
type capturingCritic struct {
mu sync.Mutex
soft time.Duration
h run.CriticHandle
}
func (c *capturingCritic) Monitor(_ context.Context, _ run.RunInfo, soft time.Duration) run.CriticHandle {
c.mu.Lock()
c.soft = soft
c.mu.Unlock()
return c.h
}
// TestCriticSoftTriggerIsMaxRuntime: the soft trigger handed to the host critic is
// the run's resolved MaxRuntime (mort's two-tier model — the critic first wakes once
// the run exceeds its nominal budget), not some global/default value.
func TestCriticSoftTriggerIsMaxRuntime(t *testing.T) {
fp := fake.New("fake")
fp.Enqueue("m", fake.Reply("done"))
m, _ := fp.Model("m")
cc := &capturingCritic{h: &fakeCriticHandle{}}
ex := run.New(run.Config{
Registry: tool.NewRegistry(),
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) { return ctx, m, nil },
Ports: run.Ports{Critic: cc},
})
const wantSoft = 7 * time.Minute
ex.Run(context.Background(),
run.RunnableAgent{Name: "x", ModelTier: "m", MaxRuntime: wantSoft, Critic: run.CriticConfig{Enabled: true}},
tool.Invocation{RunID: "r"}, "go")
cc.mu.Lock()
got := cc.soft
cc.mu.Unlock()
if got != wantSoft {
t.Errorf("soft trigger = %v, want the agent's MaxRuntime %v", got, wantSoft)
}
}
// TestCriticOwnsDeadline_NilHandleFallsBackToMaxRuntime: the agent enables the
// critic but the host Monitor returns NO handle (nil) — there is no deadline-watch,
// so the run is unsupervised. It must fall back to the nominal MaxRuntime hard cap
// (the slow 200ms tool outlasts the 20ms MaxRuntime → the run errors), NOT run free
// up to the generous CriticAbsoluteMax runaway ceiling.
func TestCriticOwnsDeadline_NilHandleFallsBackToMaxRuntime(t *testing.T) {
m := slowModel()
cc := &capturingCritic{} // h is the nil interface → Monitor returns a nil handle
ex := run.New(run.Config{
Registry: tool.NewRegistry(),
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) { return ctx, m, nil },
Ports: run.Ports{Critic: cc},
Defaults: run.Defaults{CriticAbsoluteMax: time.Hour}, // generous ceiling; must NOT be what bounds the run
})
res := ex.Run(context.Background(),
run.RunnableAgent{Name: "x", ModelTier: "m", MaxIterations: 5, MaxRuntime: 20 * time.Millisecond,
Critic: run.CriticConfig{Enabled: true}},
slowToolInvocation("r", 200*time.Millisecond), "go")
if res.Err == nil {
t.Fatalf("critic-enabled run with a nil Monitor handle must fall back to the MaxRuntime hard cap; got output=%q err=nil", res.Output)
}
}
+128
View File
@@ -0,0 +1,128 @@
package run_test
import (
"context"
"sync"
"testing"
"time"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/majordomo/provider/fake"
"gitea.stevedudenhoeffer.com/steve/executus/run"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
type fakeCritic struct{ h *fakeCriticHandle }
func (c *fakeCritic) Monitor(_ context.Context, _ run.RunInfo, _ time.Duration) run.CriticHandle {
return c.h
}
type fakeCriticHandle struct {
mu sync.Mutex
steps, tools, stops int
steered int
maxSteps int // 0 => defer to the run's base MaxIterations
killCause error // non-nil simulates a critic kill
}
func (h *fakeCriticHandle) RecordStep(int, *llm.Response) { h.mu.Lock(); h.steps++; h.mu.Unlock() }
func (h *fakeCriticHandle) KillCause() error {
h.mu.Lock()
defer h.mu.Unlock()
return h.killCause
}
func (h *fakeCriticHandle) RecordToolStart(string, string) {
h.mu.Lock()
h.tools++
h.mu.Unlock()
}
func (h *fakeCriticHandle) Steer() []llm.Message { h.mu.Lock(); h.steered++; h.mu.Unlock(); return nil }
func (h *fakeCriticHandle) Deadline() time.Time { return time.Time{} } // no hard deadline
func (h *fakeCriticHandle) MaxSteps() int { h.mu.Lock(); defer h.mu.Unlock(); return h.maxSteps }
func (h *fakeCriticHandle) Stop() { h.mu.Lock(); h.stops++; h.mu.Unlock() }
// TestCriticRaisesStepCeiling: a critic returning a higher MaxSteps lets the agent
// run PAST its base MaxIterations (the dynamic step ceiling). With base=1 and no
// critic the run would hit ErrMaxSteps after the first tool-dispatch step; the
// critic raises it to 5 so the run completes.
func TestCriticRaisesStepCeiling(t *testing.T) {
h := &fakeCriticHandle{maxSteps: 5}
fp := fake.New("fake")
fp.Enqueue("m",
// two tool-call steps (unknown tool → tolerated error results), then answer
fake.ReplyWith(llm.Response{ToolCalls: []llm.ToolCall{{ID: "c1", Name: "noop", Arguments: []byte(`{}`)}}}),
fake.ReplyWith(llm.Response{ToolCalls: []llm.ToolCall{{ID: "c2", Name: "noop", Arguments: []byte(`{}`)}}}),
fake.Reply("done after 2 tool steps"),
)
m, _ := fp.Model("m")
ex := run.New(run.Config{
Registry: tool.NewRegistry(),
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) { return ctx, m, nil },
Ports: run.Ports{Critic: &fakeCritic{h: h}},
// The fake handle's Deadline() is zero (no hard deadline), so the
// deadline-watch never interferes regardless of the soft trigger.
})
res := ex.Run(context.Background(),
run.RunnableAgent{Name: "x", ModelTier: "m", MaxIterations: 1, Critic: run.CriticConfig{Enabled: true}},
tool.Invocation{RunID: "r"}, "go")
if res.Err != nil {
t.Fatalf("critic raised the ceiling to 5, run should complete past base=1: %v", res.Err)
}
if res.Output != "done after 2 tool steps" {
t.Errorf("output = %q", res.Output)
}
}
// TestCriticWired: an agent with Critic.Enabled gets monitored — Monitor returns
// a handle the executor feeds (RecordStep), drains (Steer), and stops.
func TestCriticWired(t *testing.T) {
h := &fakeCriticHandle{}
fp := fake.New("fake")
fp.Enqueue("m", fake.Reply("done"))
m, _ := fp.Model("m")
ex := run.New(run.Config{
Registry: tool.NewRegistry(),
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) { return ctx, m, nil },
Ports: run.Ports{Critic: &fakeCritic{h: h}},
})
res := ex.Run(context.Background(),
run.RunnableAgent{Name: "watched", ModelTier: "m", Critic: run.CriticConfig{Enabled: true}},
tool.Invocation{RunID: "r"}, "go")
if res.Err != nil {
t.Fatalf("run error: %v", res.Err)
}
h.mu.Lock()
defer h.mu.Unlock()
if h.steps < 1 {
t.Errorf("critic should have seen >=1 step, got %d", h.steps)
}
if h.steered < 1 {
t.Errorf("critic Steer should be drained at least once, got %d", h.steered)
}
if h.stops != 1 {
t.Errorf("critic Stop should be called exactly once, got %d", h.stops)
}
}
// TestCriticDisabledNotMonitored: Critic.Enabled=false → Monitor never called.
func TestCriticDisabledNotMonitored(t *testing.T) {
h := &fakeCriticHandle{}
fp := fake.New("fake")
fp.Enqueue("m", fake.Reply("done"))
m, _ := fp.Model("m")
ex := run.New(run.Config{
Registry: tool.NewRegistry(),
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) { return ctx, m, nil },
Ports: run.Ports{Critic: &fakeCritic{h: h}},
})
ex.Run(context.Background(),
run.RunnableAgent{Name: "x", ModelTier: "m"}, // Critic.Enabled=false
tool.Invocation{RunID: "r"}, "go")
h.mu.Lock()
defer h.mu.Unlock()
if h.stops != 0 || h.steps != 0 {
t.Errorf("disabled critic should not be monitored: steps=%d stops=%d", h.steps, h.stops)
}
}
+114
View File
@@ -0,0 +1,114 @@
package run_test
import (
"context"
"errors"
"testing"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/majordomo/provider/fake"
"gitea.stevedudenhoeffer.com/steve/executus/deliver"
"gitea.stevedudenhoeffer.com/steve/executus/run"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
type recordingDelivery struct {
target deliver.Target
output string
errored error
delivers int
}
func (d *recordingDelivery) Deliver(_ context.Context, t deliver.Target, output string, _ []deliver.Artifact) (string, error) {
d.target, d.output, d.delivers = t, output, d.delivers+1
return "msg-1", nil
}
func (d *recordingDelivery) DeliverError(_ context.Context, t deliver.Target, e error) error {
d.target, d.errored = t, e
return nil
}
func TestDeliveryWired(t *testing.T) {
d := &recordingDelivery{}
fp := fake.New("fake")
fp.Enqueue("m", fake.Reply("the output"))
m, _ := fp.Model("m")
ex := run.New(run.Config{
Registry: tool.NewRegistry(),
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) { return ctx, m, nil },
Ports: run.Ports{Delivery: d},
})
// With a delivery target, the executor posts the output.
ex.Run(context.Background(),
run.RunnableAgent{Name: "x", ModelTier: "m"},
tool.Invocation{RunID: "r", DeliveryKind: "channel", DeliveryID: "chan-9"}, "go")
if d.delivers != 1 || d.output != "the output" || d.target.ID != "chan-9" || d.target.Kind != "channel" {
t.Fatalf("delivery wrong: %+v out=%q", d.target, d.output)
}
}
func TestNoDeliveryWithoutTarget(t *testing.T) {
d := &recordingDelivery{}
fp := fake.New("fake")
fp.Enqueue("m", fake.Reply("x"))
m, _ := fp.Model("m")
ex := run.New(run.Config{
Registry: tool.NewRegistry(),
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) { return ctx, m, nil },
Ports: run.Ports{Delivery: d},
})
// No DeliveryID → executor delivers nothing (caller reads Result.Output).
ex.Run(context.Background(),
run.RunnableAgent{Name: "x", ModelTier: "m"},
tool.Invocation{RunID: "r"}, "go")
if d.delivers != 0 {
t.Errorf("no target should mean no delivery, got %d", d.delivers)
}
}
// TestNoDeliveryOnEarlyResolveError: an error BEFORE the run starts (model
// resolve) returns before delivery is reached — neither Deliver nor DeliverError
// fires. (Delivery covers run OUTCOMES, not pre-run setup failures.)
func TestNoDeliveryOnEarlyResolveError(t *testing.T) {
d := &recordingDelivery{}
ex := run.New(run.Config{
Registry: tool.NewRegistry(),
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) {
return ctx, nil, errors.New("resolve boom")
},
Ports: run.Ports{Delivery: d},
})
ex.Run(context.Background(),
run.RunnableAgent{Name: "x", ModelTier: "m"},
tool.Invocation{RunID: "r", DeliveryKind: "channel", DeliveryID: "chan-9"}, "go")
if d.delivers != 0 || d.errored != nil {
t.Errorf("early resolve failure should neither Deliver nor DeliverError: delivers=%d errored=%v", d.delivers, d.errored)
}
}
// TestDeliverErrorOnRunFailure: an in-loop run failure (the model errors) routes
// through DeliverError with the run error.
func TestDeliverErrorOnRunFailure(t *testing.T) {
d := &recordingDelivery{}
fp := fake.New("fake")
fp.Enqueue("m", fake.Step{Err: errors.New("model boom")}) // model errors mid-run
m, _ := fp.Model("m")
ex := run.New(run.Config{
Registry: tool.NewRegistry(),
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) { return ctx, m, nil },
Ports: run.Ports{Delivery: d},
})
res := ex.Run(context.Background(),
run.RunnableAgent{Name: "x", ModelTier: "m"},
tool.Invocation{RunID: "r", DeliveryKind: "channel", DeliveryID: "chan-9"}, "go")
if res.Err == nil {
t.Fatal("expected a run error")
}
if d.delivers != 0 {
t.Errorf("a failed run should not Deliver (success path), got %d", d.delivers)
}
if d.errored == nil || d.target.ID != "chan-9" {
t.Errorf("a failed run with a target should DeliverError to chan-9, got errored=%v target=%+v", d.errored, d.target)
}
}
+651
View File
@@ -0,0 +1,651 @@
package run
import (
"context"
"errors"
"fmt"
"log/slog"
"time"
"gitea.stevedudenhoeffer.com/steve/majordomo/agent"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/executus/compact"
"gitea.stevedudenhoeffer.com/steve/executus/deliver"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
// ModelResolver resolves a tier alias or concrete spec to a usable llm.Model
// and an enriched context (for usage attribution). model.ParseModelForContext
// satisfies it.
type ModelResolver func(ctx context.Context, tier string) (context.Context, llm.Model, error)
// Defaults are the executor's fallback caps and loop guards, applied per run
// when the RunnableAgent leaves a field zero.
type Defaults struct {
MaxIterations int // tool-dispatch steps; default 12
MaxRuntime time.Duration // wall-clock per run; default 60s
FallbackTier string // tier when the agent's is empty; default "fast"
MaxConsecutiveToolErrors int // loop guard; default 3
MaxSameToolCallRepeats int // retry-storm guard; default 3
CompactionThresholdRatio float64 // fraction of model context to compact at; default 0.7
// CriticAbsoluteMax is the RUNAWAY ceiling for a critic-OWNED run (Ports.Critic
// set AND the agent enables it). For such a run MaxRuntime is the SOFT trigger,
// not a hard cap, and the critic's own extendable backstop is the normal
// deadline. This ceiling exists ONLY to stop a critic that never advances its
// deadline (a broken host handle) from running forever, so it is deliberately
// set FAR beyond any realistic backstop (default 24h): the host clamps its own
// backstop to a much smaller absolute max (e.g. a 6h host convar), so the ceiling
// never pre-empts a healthy supervised run. Keep it well above the host's
// absolute max. Never shorter than the run's MaxRuntime. Non-critic runs ignore
// it (they keep the literal MaxRuntime kill).
CriticAbsoluteMax time.Duration
}
func (d Defaults) withFallbacks() Defaults {
if d.MaxIterations <= 0 {
d.MaxIterations = 12
}
if d.MaxRuntime <= 0 {
d.MaxRuntime = 60 * time.Second
}
if d.FallbackTier == "" {
d.FallbackTier = "fast"
}
if d.MaxConsecutiveToolErrors <= 0 {
d.MaxConsecutiveToolErrors = 3
}
if d.MaxSameToolCallRepeats <= 0 {
d.MaxSameToolCallRepeats = 3
}
if d.CompactionThresholdRatio <= 0 {
d.CompactionThresholdRatio = 0.7
}
if d.CriticAbsoluteMax <= 0 {
d.CriticAbsoluteMax = 24 * time.Hour
}
return d
}
// Config wires an Executor. Registry + Models are required; everything else is
// optional and nil-safe — the zero Config beyond those yields a bounded,
// in-memory run with no persistence/audit/budget/critic/delegation/compaction
// (gadfly's case).
type Config struct {
Registry tool.Registry
Models ModelResolver
Defaults Defaults
Ports Ports
// Compactor mints the per-run context-compaction hook. nil disables
// compaction. ContextTokens resolves a tier's model context-window (for
// the compaction threshold); nil — or a zero return — also disables it.
Compactor compact.CompactorFactory
ContextTokens func(tier string) int
// SystemHeader is an optional platform header prepended to every agent's
// system prompt.
SystemHeader string
}
// Executor runs a RunnableAgent through majordomo's agent loop with the wired
// Ports. Construct with New; safe for concurrent use across runs.
type Executor struct {
cfg Config
}
// New builds an Executor. It panics if Registry or Models is nil — those are
// structural, not runtime, errors.
func New(cfg Config) *Executor {
if cfg.Registry == nil || cfg.Models == nil {
panic("run.New: Registry and Models are required")
}
cfg.Defaults = cfg.Defaults.withFallbacks()
return &Executor{cfg: cfg}
}
// Result is one run's outcome. Err carries the run failure (if any); the other
// fields are populated best-effort even on error (partial output/steps/usage).
type Result struct {
RunID string
Output string
Steps []tool.Step
Usage llm.Usage
Err error
// PostRunResult carries artifacts produced by a SessionToolFactory's PostRun
// hook (rendered images, files). nil when no factory was set or PostRun
// returned nil. The host delivers these (e.g. mort's chat API / Discord).
PostRunResult *tool.PostRunResult
}
// Run executes ra with the given invocation + input and returns the Result. It
// never propagates a panic; failures surface in Result.Err (a top-level recover
// converts any panic — including from a host Port — into a run error).
func (e *Executor) Run(ctx context.Context, ra RunnableAgent, inv tool.Invocation, input string) (res Result) {
started := time.Now()
res = Result{RunID: inv.RunID}
// ckpt is the per-run durable checkpointer (resolved below; nil = non-durable).
// checkpointCause yields the run context's cancellation cause once the run
// context exists; nil before then (an early build-error return).
var ckpt Checkpointer
var checkpointCause func() error
// Enforce the no-panic contract: a panic anywhere in the run (incl. a host
// Critic/Audit/Palette callback on the main goroutine) becomes Result.Err
// rather than unwinding into the caller. This defer ALSO finalizes the
// checkpoint on EVERY exit path — panic, an early build-error return (before
// the run loop), or normal completion — so a recovered run's durable record is
// never left dangling (which would loop boot-recovery on a persistent error).
defer func() {
if r := recover(); r != nil {
res.Err = fmt.Errorf("run.Executor: recovered panic: %v", r)
}
var cause error
if checkpointCause != nil {
cause = checkpointCause()
}
finalizeCheckpoint(ctx, ckpt, res.Err, cause)
}()
tier := ra.ModelTier
if tier == "" {
tier = e.cfg.Defaults.FallbackTier
}
maxIter := ra.MaxIterations
if maxIter <= 0 {
maxIter = e.cfg.Defaults.MaxIterations
}
maxRuntime := ra.MaxRuntime
if maxRuntime <= 0 {
maxRuntime = e.cfg.Defaults.MaxRuntime
}
// Budget gate (pre-run): a rejected run makes no model call.
if e.cfg.Ports.Budget != nil {
if err := e.cfg.Ports.Budget.Check(ctx, inv.CallerID); err != nil {
res.Err = err
return res
}
}
// Resolve the model (enriches ctx for usage attribution).
modelCtx, model, err := e.cfg.Models(ctx, tier)
if err != nil {
res.Err = fmt.Errorf("resolve model %q: %w", tier, err)
return res
}
if model == nil {
// A resolver returning (ctx, nil, nil) would otherwise nil-panic inside
// the agent loop; surface it as a clean error (Run never panics out).
res.Err = fmt.Errorf("resolve model %q: resolver returned a nil model", tier)
return res
}
ctx = modelCtx
// Audit start (optional). The recorder satisfies RunTally; stamp it on the
// invocation so a self-status tool can read live progress.
info := RunInfo{
RunID: inv.RunID,
SubjectID: ra.ID,
Name: ra.Name,
CallerID: inv.CallerID,
ChannelID: inv.ChannelID,
GuildID: inv.GuildID,
ParentRunID: inv.ParentRunID,
ModelTier: tier,
Inputs: inv.SkillInputs,
StartedAt: started,
MaxIterations: maxIter,
}
var rec RunRecorder
var stateAcc *RunStateAccessor
if e.cfg.Ports.Audit != nil {
rec = e.cfg.Ports.Audit.StartRun(ctx, info)
}
if rec != nil {
stateAcc = NewRunStateAccessor(rec, maxIter, 0, started)
inv.RunState = stateAcc
}
// Durable recovery (optional): a recovered run carries a ResumeState (prior
// transcript / completed phases) + an existing Checkpointer in ctx so it
// continues on the SAME durable record; a fresh run mints a per-run
// Checkpointer via the factory (which decides durability — nil = non-durable).
// nil-safe throughout.
resume := resumeStateFromContext(ctx)
ckpt = existingCheckpointerFromContext(ctx)
if ckpt == nil && e.cfg.Ports.Checkpointer != nil {
c, cerr := e.cfg.Ports.Checkpointer.Begin(ctx, info)
if cerr != nil {
// Degrade to non-durable (the documented contract) but log it — a
// failing checkpoint store must not fail the run, yet shouldn't be silent.
slog.Warn("run: checkpointer Begin failed; running non-durable",
"run_id", inv.RunID, "error", cerr)
} else {
ckpt = c
}
}
// Steer mailbox: lets session tools (via inv.AttachImages) feed multimodal
// messages into the running conversation before its next step. Created BEFORE
// the toolbox build so any tool's handler captures the live AttachImages seam.
mailbox := &steerMailbox{}
inv.AttachImages = (&runSession{mailbox: mailbox}).AttachImages
// Build the toolbox from the agent's low-level tools.
toolbox, err := e.cfg.Registry.Build(ra.LowLevelTools, inv, tool.Visibility("private"), nil)
if err != nil {
res.Err = fmt.Errorf("build toolbox: %w", err)
e.finishAudit(ctx, rec, "error", res, started, res.Err)
return res
}
// Add skill__/agent__ delegation tools from the agent's palette (nil-safe:
// no PaletteSource or empty palette → no delegation tools).
if err := addDelegationTools(toolbox, ra, inv, e.cfg.Ports.Palette); err != nil {
res.Err = fmt.Errorf("build delegation tools: %w", err)
e.finishAudit(ctx, rec, "error", res, started, res.Err)
return res
}
// Per-invocation ExtraTools + a SessionToolFactory's per-run tools, added on
// top of the agent's palette. The factory closes over the live session (the
// AttachImages mailbox); its PostRun hook (held for after the run) produces
// artifacts attached to res.PostRunResult, and its Cleanup is deferred. All
// nil-safe.
for _, t := range inv.ExtraTools {
if err := toolbox.Add(t); err != nil {
res.Err = fmt.Errorf("add extra tool: %w", err)
e.finishAudit(ctx, rec, "error", res, started, res.Err)
return res
}
}
var postRun func(ctx context.Context, transcript []llm.Message, output string, runErr error) *tool.PostRunResult
if inv.SessionToolFactory != nil {
st := inv.SessionToolFactory(&runSession{mailbox: mailbox})
if st.Cleanup != nil {
defer safeCleanup(st.Cleanup) // panic-isolated, like runPostRun
}
for _, t := range st.Tools {
if err := toolbox.Add(t); err != nil {
res.Err = fmt.Errorf("add session tool: %w", err)
e.finishAudit(ctx, rec, "error", res, started, res.Err)
return res
}
}
postRun = st.PostRun
}
// Skill packs: resolve the agent's subscribed packs into a catalog (folded
// into the system prompt) + a skill_use loader tool added to the toolbox.
// nil-safe; activation failures are non-fatal — the run proceeds without
// packs rather than dying on a fetch/cache miss.
if len(ra.SkillPacks) > 0 && e.cfg.Ports.SkillPacks != nil {
instr, packTools, aerr := e.cfg.Ports.SkillPacks.ActivateSkillPacks(ctx, ra.SkillPacks, inv.RunID, ra.ID)
if aerr != nil {
slog.Warn("run: skill-pack activation failed; continuing without packs", "run_id", inv.RunID, "error", aerr)
} else {
for _, t := range packTools {
if err := toolbox.Add(t); err != nil {
res.Err = fmt.Errorf("add skill-pack tool: %w", err)
e.finishAudit(ctx, rec, "error", res, started, res.Err)
return res
}
}
if instr != "" {
if ra.SystemPrompt != "" {
ra.SystemPrompt += "\n\n" + instr
} else {
ra.SystemPrompt = instr
}
}
}
}
// Run context: detached from the caller's deadline so a lane/queue wait doesn't
// eat the run budget (mort's V10 lesson). Caller cancellation still propagates
// via MergeCancellation. Created BEFORE the step observer so the observer
// forwards the merged run context (not a possibly-cancelled caller ctx) to
// OnStep consumers.
//
// Two-tier timeout: who owns the hard deadline depends on the critic.
// - NO critic (the default): MaxRuntime is a literal WithTimeout. Its
// DeadlineExceeded propagates through the child chain (→ "timeout"),
// preserving the run's-own-timeout vs caller-cancel distinction.
// - critic OWNS the deadline (Ports.Critic set + ra.Critic.Enabled):
// MaxRuntime becomes the SOFT trigger (passed to startCritic), and the
// critic's extendable backstop — watched in startCritic, which cancels via
// cancelCause — is the real deadline. A slow-but-progressing run is given
// room up to that backstop; only a stalled one is killed. The base context
// gets a WithTimeout at CriticAbsoluteMax (default 24h) purely as a RUNAWAY
// guard for a critic that never advances its deadline: it is set FAR beyond
// any realistic backstop (the host clamps its own backstop to a much smaller
// absolute max, e.g. a 6h host convar), so it does NOT pre-empt a healthy
// supervised run. If the host critic fails to ARM (nil handle), the run is
// unsupervised and we tighten the cap back down to MaxRuntime below.
// A NESTED cause-carrying layer (cancelCause) lets a critic kill surface as a
// distinct "killed": only an ErrCriticKill cause is consulted in statusFor; a
// generic run error, a backstop expiry, or a caller cancel is classified by the
// run error itself.
criticOwns := e.criticOwnsDeadline(ra)
hardCap := maxRuntime
if criticOwns {
// Runaway guard only — the critic's own (extendable) deadline-watch is the
// normal cap. max() keeps it from being shorter than the nominal budget if an
// operator sets MaxRuntime above the runaway ceiling (a degenerate config).
hardCap = max(e.cfg.Defaults.CriticAbsoluteMax, maxRuntime)
}
timeoutCtx, cancelTimeout := context.WithTimeout(context.WithoutCancel(ctx), hardCap)
defer cancelTimeout()
runCtx, cancelCause := context.WithCancelCause(timeoutCtx)
defer cancelCause(nil)
runCtx, mergeCancel := MergeCancellation(runCtx, ctx)
defer mergeCancel()
// Critic (optional): monitors the run for a stall, can nudge/extend/kill via
// its host Escalator. When it owns the deadline, MaxRuntime is its soft trigger
// (so a slow-but-progressing run survives past it); its extendable backstop is
// bound to runCtx (cancel on pass). nil-safe: no-op when no critic is configured
// or the agent doesn't enable it.
critic, stopCritic := e.startCritic(runCtx, cancelCause, ra, info, maxRuntime)
defer stopCritic()
// Unsupervised-run failsafe: the agent enabled the critic (so the base context
// got the generous runaway ceiling instead of MaxRuntime), but the host Monitor
// returned no handle — there is no deadline-watch. Without this the run would be
// bounded only by the 24h ceiling. Tighten it back to the nominal MaxRuntime so
// an unsupervised run can't hold its slot far past budget. mort's adapter always
// arms when the flag is set, so this is pure defence in depth.
if criticOwns && critic == nil {
var cancelUnsupervised context.CancelFunc
runCtx, cancelUnsupervised = context.WithTimeout(runCtx, maxRuntime)
defer cancelUnsupervised()
}
// The finalize defer (top of Run) now has a run context to read the
// cancellation cause from (shutdown vs critic-kill vs deadline vs cancel). Set
// AFTER the unsupervised-failsafe re-wrap so it reads the context the loop runs on.
checkpointCause = func() error { return context.Cause(runCtx) }
// Step instrumentation: accumulate Result.Steps + fire inv.OnStep, feed the
// audit recorder, and keep the live iteration counter fresh. majordomo's
// step observer hands us each completed iteration; we zip the model's tool
// calls with their executed results PAIRWISE — a result without a matching
// call (or a call without a result) is skipped rather than recorded as an
// empty-name "ghost" step.
emitter := newStepEmitter(inv.OnStep)
stepObserver := func(s agent.Step) {
if stateAcc != nil {
stateAcc.SetIteration(s.Index)
}
if rec != nil {
rec.OnStep(s.Index, s.Response)
}
critic.recordStep(s.Index, s.Response) // keep the critic's activity clock fresh + carry the step payload
var calls []llm.ToolCall
if s.Response != nil {
calls = s.Response.ToolCalls
}
n := len(s.Results)
if len(calls) < n {
n = len(calls)
}
for i := 0; i < n; i++ {
call, r := calls[i], s.Results[i]
critic.recordToolStart(call.Name, string(call.Arguments))
emitter.toolStart(runCtx, call.Name, call.Arguments)
emitter.toolEnd(runCtx, call, r.Content, r.IsError)
if rec != nil {
rec.OnTool(call, r.Content)
}
}
}
// Shared agent options used by BOTH the single-loop path and every phase: the
// tool-error guards and optional compaction. The toolbox, step ceiling, AND
// step observer are added per path (the observer is wrapped for checkpointing,
// which differs single-loop vs per-phase).
sharedOpts := []agent.Option{
agent.WithToolErrorLimits(e.cfg.Defaults.MaxConsecutiveToolErrors, e.cfg.Defaults.MaxSameToolCallRepeats),
}
if e.cfg.Compactor != nil && e.cfg.ContextTokens != nil {
if threshold := e.compactionThreshold(tier); threshold > 0 {
// Forward compaction events to the audit log (makes the
// CompactionEvent doc's "logged to the run trace" promise true).
var onFire func(compact.CompactionEvent)
if rec != nil {
onFire = func(ev compact.CompactionEvent) {
rec.LogEvent("compaction_fired", map[string]any{
"messages_before": ev.MessagesBefore,
"messages_after": ev.MessagesAfter,
"tokens_before": ev.TokensBefore,
"tokens_after": ev.TokensAfter,
})
}
}
sharedOpts = append(sharedOpts, agent.WithCompactor(e.cfg.Compactor(threshold, onFire)))
}
}
// Stage non-image input attachments (audio/PDF/binary) into the host file
// store and fold an [ATTACHED FILES] descriptor into the prompt so the agent
// can reach them by file_id. No-op when Ports.InputFiles is nil or there are
// no files. Done after the model/toolbox build but before the loop, so the
// descriptor rides the very first user turn.
input = e.stageInputFiles(runCtx, inv.RunID, ra.ID, inv.InputFiles, input)
// One WithSteer drains BOTH the session mailbox (a tool's AttachImages) and
// the critic's nudges before each step.
steer := func() []llm.Message { return append(mailbox.drain(), critic.drainSteer()...) }
resuming := resume != nil && len(resume.History) > 0
var runRes *agent.Result
var runErr error
if len(ra.Phases) == 0 {
// Single-loop run: the agent's base prompt + full toolbox, with the
// critic's DYNAMIC step ceiling (WithMaxStepsFunc, so it can raise a
// healthy-but-long run's budget mid-flight; falls back to maxIter).
//
// Checkpointing: wrap the step observer to accumulate the running transcript
// and Save it each step. Save is called every step; THROTTLING is the
// Checkpointer's responsibility (the battery + mort's durable-job adapter
// both throttle + size-cap), so the kernel doesn't gate the hot path. The
// accumulated transcript is the pre-compaction one (the observer sees raw
// step responses, not the loop's compacted history) — a host that caps size
// bounds it. A recovered run seeds the saved transcript and continues.
obs := stepObserver
if ckpt != nil {
var acc []llm.Message
if resuming {
acc = append([]llm.Message(nil), resume.History...)
} else {
acc = []llm.Message{multimodalUserMessage(input, inv.Images)}
}
obs = func(s agent.Step) {
stepObserver(s)
if s.Response != nil {
acc = append(acc, s.Response.Message())
}
if len(s.Results) > 0 {
acc = append(acc, llm.ToolResultsMessage(s.Results...))
}
_ = ckpt.Save(runCtx, RunCheckpointState{Messages: acc, Iteration: s.Index + 1})
}
}
opts := append([]agent.Option{
agent.WithToolbox(toolbox),
critic.maxStepsOption(maxIter),
agent.WithStepObserver(obs),
}, sharedOpts...)
ag := agent.New(model, e.systemPrompt(ra), opts...)
if resuming {
// Resume: seed the saved transcript and continue (no new input — the
// completed tool calls in the transcript are NOT re-run).
runRes, runErr = ag.Run(runCtx, "", agent.WithSteer(steer), agent.WithHistory(resume.History))
} else {
runRes, runErr = runAgent(runCtx, ag, input, inv.Images, agent.WithSteer(steer))
}
} else {
// Multi-phase pipeline: each phase runs its own prompt/tier/tools/step-cap
// sequentially, threading outputs through {{.<PhaseName>}} templates. The
// shared step observer (audit/steps/critic) is wired per phase by the phase
// runner; checkpointing is phase-boundary granular (completed phases are
// recorded so a resumed run skips them).
runRes, runErr = e.runPhases(runCtx, ra, phaseDeps{
baseModel: model,
baseToolbox: toolbox,
baseMaxIter: maxIter,
sharedOpts: sharedOpts,
stepObserver: stepObserver,
steer: steer,
rec: rec,
checkpointer: ckpt,
resume: resume,
}, input, inv.Images)
}
// Durable-recovery finalize (Complete/Fail/leave-running) happens in the
// top-of-Run defer so it covers panics + early build-error returns too.
status := statusFor(runCtx, runErr)
if runRes != nil {
res.Output = runRes.Output
res.Usage = runRes.Usage
}
res.Steps = emitter.snapshot()
res.Err = runErr
// PostRun: hand the SessionToolFactory's hook the full transcript (populated
// even on partial results) so it can produce artifacts. Best-effort +
// panic-isolated — a PostRun failure never fails an otherwise-successful run.
if postRun != nil {
var transcript []llm.Message
if runRes != nil {
transcript = runRes.Messages
}
// Detach from the caller's ctx: a finished/cancelled caller must not abort
// artifact production (the hook owns its own bounding, per its contract).
res.PostRunResult = runPostRun(detach(ctx), postRun, transcript, res.Output, runErr)
}
e.finishAudit(ctx, rec, status, res, started, runErr)
if e.cfg.Ports.Budget != nil {
e.cfg.Ports.Budget.Commit(detach(ctx), inv.CallerID, time.Since(started).Seconds())
}
e.deliver(ctx, inv, res, runErr)
return res
}
// statusFor maps a run error to a RunStats.Status, distinguishing a critic kill
// (killed), a deadline (timeout), and a cancellation (cancelled — caller cancel
// or shutdown) from a generic error so audit consumers can tell them apart. The
// run context's cancellation cause carries the distinction (ErrCriticKill /
// DeadlineExceeded), since ctx.Err() alone only reports Canceled.
func statusFor(runCtx context.Context, runErr error) string {
switch {
case runErr == nil:
return "ok"
// Only the kill is recovered from the cancellation cause — a critic kill
// surfaces as a plain Canceled run error, so without this it'd read as
// "cancelled". Everything else is classified by the run error itself, so a
// genuine run error is never relabeled just because the context was later
// cancelled, and a caller cancel/deadline stays "cancelled" (not "timeout").
case errors.Is(context.Cause(runCtx), ErrCriticKill):
return "killed"
case errors.Is(runErr, context.DeadlineExceeded):
return "timeout"
case errors.Is(runErr, context.Canceled):
return "cancelled"
default:
return "error"
}
}
// finishAudit writes the terminal roll-up on a detached context so a cancelled
// run still records (mort's CleanupContextTimeout lesson).
func (e *Executor) finishAudit(ctx context.Context, rec RunRecorder, status string, res Result, started time.Time, runErr error) {
if rec == nil {
return
}
stats := RunStats{
Status: status,
Output: res.Output,
ToolCalls: rec.ToolCallsCount(),
RuntimeSeconds: time.Since(started).Seconds(),
}
if runErr != nil {
stats.Error = runErr.Error()
}
stats.InputTokens, stats.OutputTokens, stats.ThinkingTokens = rec.TokenStats()
rec.Close(detach(ctx), stats)
}
func (e *Executor) systemPrompt(ra RunnableAgent) string {
return e.systemPromptWithBody(ra.SystemPrompt)
}
// systemPromptWithBody composes the optional platform header with an arbitrary
// body. The single-loop path passes ra.SystemPrompt; the phase runner passes a
// phase's expanded instructions, so each phase keeps the platform header.
func (e *Executor) systemPromptWithBody(body string) string {
if e.cfg.SystemHeader == "" {
return body
}
if body == "" {
return e.cfg.SystemHeader
}
return e.cfg.SystemHeader + "\n\n" + body
}
// compactionThreshold returns the token threshold for the tier's model context
// window (ratio × limit), or 0 when the limit is unknown.
func (e *Executor) compactionThreshold(tier string) int {
max := e.cfg.ContextTokens(tier)
if max <= 0 {
return 0
}
return int(float64(max) * e.cfg.Defaults.CompactionThresholdRatio)
}
// deliver posts the run's output (or error) via run.Ports.Delivery when both a
// Delivery and a target (inv.DeliveryID) are set. No target = the caller reads
// Result.Output itself (the synchronous default). Best-effort + detached: a
// delivery failure must not change the run's outcome.
func (e *Executor) deliver(ctx context.Context, inv tool.Invocation, res Result, runErr error) {
if e.cfg.Ports.Delivery == nil || inv.DeliveryID == "" {
return
}
target := deliver.Target{Kind: inv.DeliveryKind, ID: inv.DeliveryID}
dctx := detach(ctx)
if runErr != nil {
_ = e.cfg.Ports.Delivery.DeliverError(dctx, target, runErr)
return
}
_, _ = e.cfg.Ports.Delivery.Deliver(dctx, target, res.Output, nil)
}
// detach derives a bounded cleanup context off ctx, detached from its
// cancellation, for post-run writes. The cancel is intentionally not returned;
// CleanupContextTimeout bounds the lifetime.
func detach(ctx context.Context) context.Context {
c, cancel := context.WithTimeout(context.WithoutCancel(ctx), CleanupContextTimeout)
_ = cancel // bounded by the timeout; nothing to cancel early
return c
}
// runAgent dispatches the majordomo agent loop. majordomo's Run takes a text-only
// input arg, so when the invocation carries images they're folded into the first
// user message (text + image parts) via WithHistory and Run is called with an
// empty input — the model then sees a multimodal opening turn. The image-less path
// passes the prompt straight through.
//
// The text part is omitted when input is blank (image-only run), matching
// runSession.AttachImages so no empty TextPart is sent.
func runAgent(ctx context.Context, ag *agent.Agent, input string, images []llm.ImagePart, opts ...agent.RunOption) (*agent.Result, error) {
if len(images) == 0 {
return ag.Run(ctx, input, opts...)
}
// Copy opts before appending so a caller-supplied backing array is never
// mutated/aliased (the variadic slice can have spare capacity). The multimodal
// opening turn (text + image parts) is built by the shared helper.
opts = append(opts[:len(opts):len(opts)], agent.WithHistory([]llm.Message{multimodalUserMessage(input, images)}))
return ag.Run(ctx, "", opts...)
}
+181
View File
@@ -0,0 +1,181 @@
package run
import (
"context"
"errors"
"fmt"
"testing"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/majordomo/provider/fake"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
// fakeModels returns a ModelResolver backed by a fake provider scripted to
// reply with the given text (no tool calls — the loop terminates immediately).
func fakeModels(t *testing.T, reply string) ModelResolver {
t.Helper()
fp := fake.New("fake")
fp.Enqueue("test-model", fake.Reply(reply))
m, err := fp.Model("test-model")
if err != nil {
t.Fatalf("fake model: %v", err)
}
return func(ctx context.Context, _ string) (context.Context, llm.Model, error) {
return ctx, m, nil
}
}
// TestExecutorRunHelloWorld is the milestone: executus runs an agent end-to-end
// against the fake provider and returns its output. Proves the kernel is
// runnable with the zero Ports (no persistence/audit/budget/critic).
func TestExecutorRunHelloWorld(t *testing.T) {
ex := New(Config{
Registry: tool.NewRegistry(),
Models: fakeModels(t, "hello from executus"),
})
res := ex.Run(context.Background(),
RunnableAgent{Name: "greeter", SystemPrompt: "be brief", ModelTier: "test-model"},
tool.Invocation{RunID: "run-1", CallerID: "caller-1"},
"say hi")
if res.Err != nil {
t.Fatalf("run error: %v", res.Err)
}
if res.Output != "hello from executus" {
t.Fatalf("output = %q, want %q", res.Output, "hello from executus")
}
if res.RunID != "run-1" {
t.Errorf("RunID = %q, want run-1", res.RunID)
}
}
// TestExecutorBudgetRejection: a Budget that denies makes no model call.
func TestExecutorBudgetRejection(t *testing.T) {
denied := errors.New("over budget")
var modelCalled bool
models := func(ctx context.Context, _ string) (context.Context, llm.Model, error) {
modelCalled = true
return ctx, nil, nil
}
ex := New(Config{
Registry: tool.NewRegistry(),
Models: models,
Ports: Ports{Budget: budgetFunc{check: func(string) error { return denied }}},
})
res := ex.Run(context.Background(),
RunnableAgent{ModelTier: "test-model"},
tool.Invocation{RunID: "r", CallerID: "broke"}, "hi")
if !errors.Is(res.Err, denied) {
t.Fatalf("err = %v, want budget denial", res.Err)
}
if modelCalled {
t.Error("model must not be resolved/called when budget denies")
}
}
// TestExecutorAuditWiring: the Audit port receives StartRun + Close with the
// terminal status/output.
func TestExecutorAuditWiring(t *testing.T) {
rec := &captureRecorder{}
ex := New(Config{
Registry: tool.NewRegistry(),
Models: fakeModels(t, "done"),
Ports: Ports{Audit: auditFunc{start: func(RunInfo) RunRecorder { return rec }}},
})
res := ex.Run(context.Background(),
RunnableAgent{ModelTier: "test-model"},
tool.Invocation{RunID: "r2", CallerID: "c"}, "go")
if res.Err != nil {
t.Fatalf("run error: %v", res.Err)
}
if !rec.closed {
t.Fatal("recorder.Close was not called")
}
if rec.stats.Status != "ok" {
t.Errorf("close status = %q, want ok", rec.stats.Status)
}
if rec.stats.Output != "done" {
t.Errorf("close output = %q, want done", rec.stats.Output)
}
}
// --- test doubles ---
type budgetFunc struct{ check func(callerID string) error }
func (b budgetFunc) Check(_ context.Context, callerID string) error { return b.check(callerID) }
func (b budgetFunc) Commit(context.Context, string, float64) {}
type auditFunc struct{ start func(RunInfo) RunRecorder }
func (a auditFunc) StartRun(_ context.Context, info RunInfo) RunRecorder { return a.start(info) }
type captureRecorder struct {
closed bool
stats RunStats
steps int
tools int
}
func (r *captureRecorder) TokenStats() (in, out, thinking int64) { return 0, 0, 0 }
func (r *captureRecorder) ToolCallsCount() int { return r.tools }
func (r *captureRecorder) OnStep(int, *llm.Response) { r.steps++ }
func (r *captureRecorder) OnTool(llm.ToolCall, string) { r.tools++ }
func (r *captureRecorder) LogEvent(string, map[string]any) {}
func (r *captureRecorder) LogError(string) {}
func (r *captureRecorder) Close(_ context.Context, s RunStats) { r.closed = true; r.stats = s }
// TestExecutorNilModelNoPanic: a resolver returning (ctx, nil, nil) yields a
// clean error, not a nil-pointer panic (gadfly F1, high severity).
func TestExecutorNilModelNoPanic(t *testing.T) {
ex := New(Config{
Registry: tool.NewRegistry(),
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) {
return ctx, nil, nil // nil model, nil error
},
})
res := ex.Run(context.Background(),
RunnableAgent{ModelTier: "x"}, tool.Invocation{RunID: "r"}, "hi")
if res.Err == nil {
t.Fatal("expected an error for a nil model, got nil (would have panicked in the loop)")
}
}
// TestStatusFor maps run errors + cancellation cause to RunStats.Status (gadfly F3).
func TestStatusFor(t *testing.T) {
bg := context.Background()
// A context cancelled with the critic-kill cause: ctx.Err() is Canceled, but
// context.Cause carries ErrCriticKill → "killed".
killCtx, killCancel := context.WithCancelCause(context.Background())
killCancel(fmt.Errorf("%w: hung", ErrCriticKill))
// A context cancelled with a non-kill cause must NOT relabel a genuine run
// error: a real error stays "error" even though the ctx was later cancelled.
cancelledCtx, cc := context.WithCancelCause(context.Background())
cc(context.DeadlineExceeded)
cases := []struct {
ctx context.Context
err error
want string
}{
{bg, nil, "ok"},
{bg, context.DeadlineExceeded, "timeout"},
{bg, context.Canceled, "cancelled"},
{bg, fmt.Errorf("wrapped: %w", context.DeadlineExceeded), "timeout"},
{bg, errors.New("boom"), "error"},
{killCtx, context.Canceled, "killed"},
{cancelledCtx, errors.New("boom"), "error"}, // generic error not relabeled by cause
{cancelledCtx, context.Canceled, "cancelled"}, // caller cancel stays cancelled, not timeout
}
for _, c := range cases {
if got := statusFor(c.ctx, c.err); got != c.want {
t.Errorf("statusFor(%v) = %q, want %q", c.err, got, c.want)
}
}
}
+121
View File
@@ -0,0 +1,121 @@
package run_test
import (
"context"
"strings"
"testing"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/majordomo/provider/fake"
"gitea.stevedudenhoeffer.com/steve/executus/run"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
// TestExecutorFoldsInitialImages: when the invocation carries Images, they're
// folded into the first user message (alongside the prompt text) instead of being
// dropped — majordomo's Run input arg is text-only, so the executor seeds the
// multimodal opening turn via history.
func TestExecutorFoldsInitialImages(t *testing.T) {
fp := fake.New("fake")
fp.Enqueue("m", fake.Reply("saw the image"))
m, _ := fp.Model("m")
img := llm.ImagePart{MIME: "image/png", Data: []byte("PNGDATA")}
inv := tool.Invocation{RunID: "r1", Images: []llm.ImagePart{img}}
ex := run.New(run.Config{
Registry: tool.NewRegistry(),
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) { return ctx, m, nil },
})
res := ex.Run(context.Background(), run.RunnableAgent{ModelTier: "m"}, inv, "describe this")
if res.Err != nil {
t.Fatalf("run error: %v", res.Err)
}
calls := fp.Calls()
if len(calls) == 0 {
t.Fatal("no model calls recorded")
}
// The text + image must be CO-LOCATED in a single user message (not split
// across two), so the model reads them as one multimodal turn.
coLocated := false
for _, msg := range calls[0].Request.Messages {
sawImage, sawText := false, false
for _, p := range msg.Parts {
switch pp := p.(type) {
case llm.ImagePart:
if string(pp.Data) == "PNGDATA" {
sawImage = true
}
case llm.TextPart:
if strings.Contains(pp.Text, "describe this") {
sawText = true
}
}
}
if sawImage && sawText {
coLocated = true
}
}
if !coLocated {
t.Error("image + prompt text were not folded into the SAME user message")
}
}
// TestExecutorImageOnlyNoBlankText: an image-only run (blank prompt) must NOT emit
// an empty TextPart — the message carries just the image, matching
// runSession.AttachImages's guard.
func TestExecutorImageOnlyNoBlankText(t *testing.T) {
fp := fake.New("fake")
fp.Enqueue("m", fake.Reply("saw it"))
m, _ := fp.Model("m")
inv := tool.Invocation{RunID: "r3", Images: []llm.ImagePart{{MIME: "image/png", Data: []byte("IMG")}}}
ex := run.New(run.Config{
Registry: tool.NewRegistry(),
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) { return ctx, m, nil },
})
res := ex.Run(context.Background(), run.RunnableAgent{ModelTier: "m"}, inv, " ")
if res.Err != nil {
t.Fatalf("run error: %v", res.Err)
}
for _, msg := range fp.Calls()[0].Request.Messages {
for _, p := range msg.Parts {
if tp, ok := p.(llm.TextPart); ok && strings.TrimSpace(tp.Text) == "" {
t.Error("image-only run emitted a blank TextPart")
}
}
}
}
// TestExecutorTextOnlyUnchanged: with no Images, the prompt flows through as the
// text input (regression guard that the fold path didn't break the common case).
func TestExecutorTextOnlyUnchanged(t *testing.T) {
fp := fake.New("fake")
fp.Enqueue("m", fake.Reply("ok"))
m, _ := fp.Model("m")
ex := run.New(run.Config{
Registry: tool.NewRegistry(),
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) { return ctx, m, nil },
})
res := ex.Run(context.Background(), run.RunnableAgent{ModelTier: "m"}, tool.Invocation{RunID: "r2"}, "plain prompt")
if res.Err != nil {
t.Fatalf("run error: %v", res.Err)
}
calls := fp.Calls()
if len(calls) == 0 {
t.Fatal("no model calls recorded")
}
sawText := false
for _, msg := range calls[0].Request.Messages {
for _, p := range msg.Parts {
if tp, ok := p.(llm.TextPart); ok && strings.Contains(tp.Text, "plain prompt") {
sawText = true
}
}
}
if !sawText {
t.Error("text-only prompt did not reach the model")
}
}
+179
View File
@@ -0,0 +1,179 @@
package run
import (
"context"
"fmt"
"log/slog"
"path"
"strings"
"unicode"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
// maxInputFileBytes is a defense-in-depth cap at the staging boundary. A host's
// extraction path may already cap downloads, but stageInputFiles is the trust
// boundary for the InputFiles seam: a call site or bug that populates InputFiles
// directly must not write an unbounded blob to the host file store.
const maxInputFileBytes = 50_000_000
// maxInputFiles bounds how many attachments a single run stages, independent of
// the per-file byte cap — defense-in-depth against a flood of tiny files.
const maxInputFiles = 32
// stageInputFiles persists each non-image input attachment into the host file
// store (Ports.InputFiles) under run scope and appends a descriptor block to the
// prompt so the agent knows the file_ids it can pass to a worker tool. The bytes
// are NOT inlined into the model context — the LLM can't read raw audio/binary —
// so the agent reaches them via a file_id-aware tool (e.g. code_exec files_in,
// which writes the file to /workspace/<name>).
//
// Best-effort: a nil stager, no files, or a per-file save error degrades to
// "skip that file" — the run still proceeds. Returns the (possibly augmented)
// prompt.
func (e *Executor) stageInputFiles(ctx context.Context, runID, agentID string, files []tool.InputFile, prompt string) string {
if e.cfg.Ports.InputFiles == nil || len(files) == 0 {
return prompt
}
// Count cap: bound how many attachments one run can stage, independent of the
// per-file byte cap (defense-in-depth against a flood of tiny files).
if len(files) > maxInputFiles {
slog.Warn("run: too many input files, truncating",
"agent", agentID, "run_id", runID, "count", len(files), "cap", maxInputFiles)
files = files[:maxInputFiles]
}
type stagedFile struct {
name, mime, fileID string
size int
}
var staged []stagedFile
seenNames := make(map[string]int, len(files))
for _, f := range files {
if len(f.Data) == 0 {
slog.Warn("run: skipping empty input file",
"agent", agentID, "run_id", runID, "name", f.Name)
continue
}
if len(f.Data) > maxInputFileBytes {
slog.Warn("run: skipping oversized input file",
"agent", agentID, "run_id", runID, "name", f.Name,
"size", len(f.Data), "cap", maxInputFileBytes)
continue
}
// Reduce the untrusted filename to a safe base name BEFORE staging or
// inlining: strips ../ and absolute-path components (so it can't escape
// the host store or /workspace/<name>) and drops control chars/newlines
// (so a crafted name can't inject text into the descriptor block below).
// Then disambiguate colliding base names so two attachments don't both map
// to /workspace/<name> (the second would clobber the first).
name := uniqueName(sanitizeName(f.Name), seenNames)
// Sanitize the mime ONCE and pass the clean value to both the host store
// and the descriptor (don't hand the raw value to StageInputFile).
mime := sanitizeField(f.MimeType)
fileID, err := e.cfg.Ports.InputFiles.StageInputFile(ctx, runID, agentID, name, mime, f.Data)
if err != nil {
slog.Warn("run: failed to stage input file",
"agent", agentID, "run_id", runID, "name", name, "error", err)
continue
}
if fileID == "" {
slog.Warn("run: stager returned empty file_id, skipping",
"agent", agentID, "run_id", runID, "name", name)
continue
}
// fileID is host-generated, but sanitize it too before inlining — the
// descriptor must never carry control chars no matter the stager impl.
staged = append(staged, stagedFile{name: name, mime: mime, fileID: sanitizeField(fileID), size: len(f.Data)})
}
if len(staged) == 0 {
return prompt
}
var b strings.Builder
b.WriteString("[ATTACHED FILES]\n")
b.WriteString("The user attached the following file(s). Their contents are NOT included in this prompt and you cannot read them directly. ")
b.WriteString("To work with one, call the code_exec tool with a files_in entry — e.g. ")
b.WriteString(`files_in: [{"name": "<name>", "file_id": "<file_id>"}]`)
b.WriteString(" — which writes it to /workspace/<name> inside the Python sandbox. You may also pass a file_id to any other tool that accepts one.\n")
for _, s := range staged {
fmt.Fprintf(&b, "- %s (%s, %s) → file_id: %s\n", s.name, s.mime, humanizeBytes(s.size), s.fileID)
}
if strings.TrimSpace(prompt) == "" {
return b.String()
}
return prompt + "\n\n" + b.String()
}
// sanitizeName reduces an untrusted attachment filename to a safe base name. It
// drops control characters / newlines (which would otherwise let a crafted name
// inject text into the [ATTACHED FILES] descriptor) and strips every directory
// component — defeating ../ traversal, nested dirs, and absolute / drive paths
// both in the host file store and at /workspace/<name>. Returns "attachment"
// when nothing usable remains (empty, ".", "..").
func sanitizeName(name string) string {
name = sanitizeField(name)
// Normalize backslashes so a Windows-style path also reduces to its base.
base := path.Base(strings.ReplaceAll(name, `\`, "/"))
base = strings.TrimSpace(base)
if base == "" || base == "." || base == ".." {
return "attachment"
}
return base
}
// sanitizeField strips characters that could let a value inlined verbatim into
// the prompt descriptor break out of its line or visually mislead: control
// characters (IsControl covers newlines/tabs) AND Unicode format characters
// (category Cf — e.g. the bidi overrides U+202AU+202E, which can reorder how
// the descriptor renders).
func sanitizeField(s string) string {
return strings.Map(func(r rune) rune {
if unicode.IsControl(r) || unicode.Is(unicode.Cf, r) {
return -1
}
return r
}, s)
}
// uniqueName returns name unchanged the first time it's seen, then name-2,
// name-3, … (suffix inserted before the extension) on repeats, recording each
// result in seen so later collisions keep counting up.
func uniqueName(name string, seen map[string]int) string {
if seen[name] == 0 {
seen[name]++
return name
}
ext := path.Ext(name)
base := strings.TrimSuffix(name, ext)
for {
seen[name]++
candidate := fmt.Sprintf("%s-%d%s", base, seen[name], ext)
if seen[candidate] == 0 {
seen[candidate]++
return candidate
}
}
}
// humanizeBytes renders a byte count as a short human-readable string (e.g.
// "2.1 MB") for the attached-files descriptor block.
func humanizeBytes(n int) string {
if n < 0 {
n = 0
}
const unit = 1024
if n < unit {
return fmt.Sprintf("%d B", n)
}
const prefixes = "KMGTPE"
div, exp := int64(unit), 0
// Clamp exp to the last prefix so an absurd size (≥1024^7) can't index past
// "KMGTPE" and panic — a no-panic guarantee independent of the per-file cap.
for v := int64(n) / unit; v >= unit && exp < len(prefixes)-1; v /= unit {
div *= unit
exp++
}
return fmt.Sprintf("%.1f %cB", float64(n)/float64(div), prefixes[exp])
}
+243
View File
@@ -0,0 +1,243 @@
package run
import (
"context"
"errors"
"strings"
"testing"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
// stagerFunc is a test InputFileStager: it records each staged file and returns
// a deterministic file_id ("file_<name>"), or an error if err is set.
type stagerFunc struct {
staged []stagedRec
err error
}
type stagedRec struct {
runID, agentID, name, mime string
size int
}
func (s *stagerFunc) StageInputFile(_ context.Context, runID, agentID, name, mime string, content []byte) (string, error) {
if s.err != nil {
return "", s.err
}
s.staged = append(s.staged, stagedRec{runID, agentID, name, mime, len(content)})
return "file_" + name, nil
}
func newStagerExecutor(s InputFileStager) *Executor {
return New(Config{
Registry: tool.NewRegistry(),
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) { return ctx, nil, nil },
Ports: Ports{InputFiles: s},
})
}
// TestStageInputFiles: files are staged via the port and an [ATTACHED FILES]
// descriptor (with each file_id) is appended to the prompt.
func TestStageInputFiles(t *testing.T) {
st := &stagerFunc{}
ex := newStagerExecutor(st)
out := ex.stageInputFiles(context.Background(), "run-1", "agent-1",
[]tool.InputFile{{Name: "clip.mp3", MimeType: "audio/mpeg", Data: []byte("abcd")}},
"transcribe this")
if len(st.staged) != 1 || st.staged[0].name != "clip.mp3" {
t.Fatalf("staged = %+v, want one clip.mp3", st.staged)
}
if st.staged[0].runID != "run-1" || st.staged[0].agentID != "agent-1" {
t.Errorf("stager got runID/agentID = %q/%q, want run-1/agent-1", st.staged[0].runID, st.staged[0].agentID)
}
for _, want := range []string{"transcribe this", "[ATTACHED FILES]", "clip.mp3", "file_clip.mp3", "audio/mpeg"} {
if !strings.Contains(out, want) {
t.Errorf("output missing %q:\n%s", want, out)
}
}
}
// TestStageInputFilesNoStager: a nil port leaves the prompt untouched and never
// drops the run.
func TestStageInputFilesNoStager(t *testing.T) {
ex := newStagerExecutor(nil) // Ports.InputFiles == nil
out := ex.stageInputFiles(context.Background(), "r", "a",
[]tool.InputFile{{Name: "x.bin", Data: []byte("z")}}, "prompt")
if out != "prompt" {
t.Errorf("nil stager changed the prompt: %q", out)
}
}
// TestStageInputFilesNoFiles: no attachments leaves the prompt untouched.
func TestStageInputFilesNoFiles(t *testing.T) {
ex := newStagerExecutor(&stagerFunc{})
out := ex.stageInputFiles(context.Background(), "r", "a", nil, "prompt")
if out != "prompt" {
t.Errorf("no files changed the prompt: %q", out)
}
}
// TestStageInputFilesDedup: colliding base names are disambiguated so they don't
// clobber each other at /workspace/<name>.
func TestStageInputFilesDedup(t *testing.T) {
st := &stagerFunc{}
ex := newStagerExecutor(st)
out := ex.stageInputFiles(context.Background(), "r", "a", []tool.InputFile{
{Name: "a.wav", MimeType: "audio/wav", Data: []byte("1")},
{Name: "a.wav", MimeType: "audio/wav", Data: []byte("2")},
}, "go")
if len(st.staged) != 2 {
t.Fatalf("staged %d files, want 2", len(st.staged))
}
if st.staged[0].name != "a.wav" || st.staged[1].name != "a-2.wav" {
t.Errorf("dedup names = %q, %q; want a.wav, a-2.wav", st.staged[0].name, st.staged[1].name)
}
if !strings.Contains(out, "a-2.wav") {
t.Errorf("output missing disambiguated name:\n%s", out)
}
}
// TestStageInputFilesSkipsBad: empty + oversized files are skipped; a save error
// drops only that file. With nothing staged, the prompt is unchanged.
func TestStageInputFilesSkipsBad(t *testing.T) {
// Empty data → skipped; with no good files the prompt is returned as-is.
ex := newStagerExecutor(&stagerFunc{})
if out := ex.stageInputFiles(context.Background(), "r", "a",
[]tool.InputFile{{Name: "empty.bin", Data: nil}}, "p"); out != "p" {
t.Errorf("empty file should be skipped, got %q", out)
}
// A stager error → that file is dropped; nothing staged → prompt unchanged.
exErr := newStagerExecutor(&stagerFunc{err: errors.New("disk full")})
if out := exErr.stageInputFiles(context.Background(), "r", "a",
[]tool.InputFile{{Name: "x.bin", Data: []byte("z")}}, "p"); out != "p" {
t.Errorf("save error should drop the file and leave the prompt, got %q", out)
}
}
// TestStageInputFilesOversize: a file past the byte cap is skipped (prompt
// unchanged), exercising the size guard directly.
func TestStageInputFilesOversize(t *testing.T) {
st := &stagerFunc{}
ex := newStagerExecutor(st)
big := make([]byte, maxInputFileBytes+1)
out := ex.stageInputFiles(context.Background(), "r", "a",
[]tool.InputFile{{Name: "huge.bin", Data: big}}, "p")
if out != "p" || len(st.staged) != 0 {
t.Errorf("oversized file should be skipped: out=%q staged=%d", out, len(st.staged))
}
}
// TestStageInputFilesCountCap: more than maxInputFiles attachments are truncated
// to the cap.
func TestStageInputFilesCountCap(t *testing.T) {
st := &stagerFunc{}
ex := newStagerExecutor(st)
files := make([]tool.InputFile, maxInputFiles+5)
for i := range files {
files[i] = tool.InputFile{Name: "f.bin", Data: []byte("x")}
}
ex.stageInputFiles(context.Background(), "r", "a", files, "p")
if len(st.staged) != maxInputFiles {
t.Errorf("count cap: staged %d, want %d", len(st.staged), maxInputFiles)
}
}
// TestSanitizeName: traversal + absolute + control-char filenames are reduced to
// a safe base name (no path separators, no newlines), with a fallback.
func TestSanitizeName(t *testing.T) {
cases := map[string]string{
"../../etc/passwd": "passwd",
"/etc/cron.d/x": "x",
`..\..\windows\sys`: "sys",
"clip.mp3": "clip.mp3",
"": "attachment",
"..": "attachment",
".": "attachment",
"evil\n- injected": "evil- injected",
"a/b/c.wav": "c.wav",
}
for in, want := range cases {
if got := sanitizeName(in); got != want {
t.Errorf("sanitizeName(%q) = %q, want %q", in, got, want)
}
// A sanitized name must never carry a path separator or newline.
got := sanitizeName(in)
if strings.ContainsAny(got, "/\\\n\r") {
t.Errorf("sanitizeName(%q) = %q still contains a separator/newline", in, got)
}
}
}
// TestStageInputFilesSanitizesTraversal: a traversal filename is staged AND
// described under its safe base name only.
func TestStageInputFilesSanitizesTraversal(t *testing.T) {
st := &stagerFunc{}
ex := newStagerExecutor(st)
out := ex.stageInputFiles(context.Background(), "r", "a",
[]tool.InputFile{{Name: "../../../etc/passwd", MimeType: "text/plain", Data: []byte("x")}}, "go")
if len(st.staged) != 1 || st.staged[0].name != "passwd" {
t.Fatalf("staged name = %+v, want passwd", st.staged)
}
if strings.Contains(out, "..") || strings.Contains(out, "/etc/") {
t.Errorf("descriptor leaked the traversal path:\n%s", out)
}
}
// TestSanitizeFieldStripsBidiAndControl: control chars AND Unicode format/bidi
// overrides are removed from inlined values.
func TestSanitizeFieldStripsBidiAndControl(t *testing.T) {
in := "audio/mpg\n; rm -rf" // bidi override + newline
got := sanitizeField(in)
if strings.ContainsAny(got, "\n\r\t") || strings.ContainsRune(got, '') {
t.Errorf("sanitizeField left control/bidi chars: %q", got)
}
}
// TestStageInputFilesSanitizesMime: a mime with a control char is cleaned in BOTH
// the staged value and the descriptor.
func TestStageInputFilesSanitizesMime(t *testing.T) {
st := &stagerFunc{}
ex := newStagerExecutor(st)
out := ex.stageInputFiles(context.Background(), "r", "a",
[]tool.InputFile{{Name: "c.wav", MimeType: "audio/wav\ninjected", Data: []byte("x")}}, "go")
if len(st.staged) != 1 || strings.ContainsAny(st.staged[0].mime, "\n\r") {
t.Errorf("mime not sanitized before staging: %+v", st.staged)
}
if strings.Contains(out, "\ninjected") {
t.Errorf("descriptor carried an unsanitized mime newline:\n%s", out)
}
}
// TestStageInputFilesEmptyFileID: a stager returning an empty file_id drops the
// file (no blank file_id in the descriptor).
func TestStageInputFilesEmptyFileID(t *testing.T) {
ex := newStagerExecutor(emptyIDStager{})
out := ex.stageInputFiles(context.Background(), "r", "a",
[]tool.InputFile{{Name: "x.bin", Data: []byte("z")}}, "p")
if out != "p" {
t.Errorf("empty file_id should drop the file, got %q", out)
}
}
type emptyIDStager struct{}
func (emptyIDStager) StageInputFile(context.Context, string, string, string, string, []byte) (string, error) {
return "", nil
}
// TestHumanizeBytesNoPanic: an absurd size clamps to the last prefix instead of
// indexing past "KMGTPE".
func TestHumanizeBytesNoPanic(t *testing.T) {
defer func() {
if r := recover(); r != nil {
t.Fatalf("humanizeBytes panicked: %v", r)
}
}()
for _, n := range []int{0, 512, 2048, 5_000_000, 1 << 62} {
_ = humanizeBytes(n)
}
}
+102
View File
@@ -0,0 +1,102 @@
package run
import (
"context"
"fmt"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
// addDelegationTools adds a delegation tool to the toolbox for each
// SkillPalette / SubAgentPalette entry, backed by the PaletteSource:
//
// - skill__<name> invokes the named saved skill with structured inputs.
// - agent__<name> invokes the named sub-agent with a prompt.
//
// Each delegated call runs as a CHILD of the current run (parentRunID =
// inv.RunID), inheriting the caller + channel. No-op when palette is nil or both
// palettes are empty — so an agent with no palette (or a host with no
// PaletteSource) simply has no delegation tools, exactly as before.
func addDelegationTools(box *llm.Toolbox, ra RunnableAgent, inv tool.Invocation, palette PaletteSource) error {
if palette == nil {
return nil
}
seen := map[string]bool{} // dedupe across both palettes by final tool name
for _, name := range ra.SkillPalette {
name := name // capture
toolName := "skill__" + name
if name == "" || seen[toolName] { // skip empty / duplicate palette entries
continue
}
seen[toolName] = true
t := llm.DefineTool(
toolName,
fmt.Sprintf("Delegate the task to the %q skill. Provide its declared inputs.", name),
func(ctx context.Context, args skillDelegateArgs) (any, error) {
out, _, status, err := palette.InvokeSkill(ctx, inv.CallerID, inv.ChannelID, name, args.Inputs, inv.RunID)
if err != nil {
return nil, delegationErr("skill", name, out, err)
}
return delegationResult(name, "skill", out, status), nil
},
)
if err := box.Add(t); err != nil {
return fmt.Errorf("add %s: %w", toolName, err)
}
}
for _, name := range ra.SubAgentPalette {
name := name // capture
toolName := "agent__" + name
if name == "" || seen[toolName] {
continue
}
seen[toolName] = true
t := llm.DefineTool(
toolName,
fmt.Sprintf("Delegate the task to the %q sub-agent with a natural-language prompt.", name),
func(ctx context.Context, args agentDelegateArgs) (any, error) {
out, _, status, err := palette.InvokeAgent(ctx, inv.CallerID, inv.ChannelID, name, args.Prompt, inv.RunID, "", "", nil, nil)
if err != nil {
return nil, delegationErr("agent", name, out, err)
}
return delegationResult(name, "agent", out, status), nil
},
)
if err := box.Add(t); err != nil {
return fmt.Errorf("add %s: %w", toolName, err)
}
}
return nil
}
// delegationResult surfaces a non-ok child status to the parent agent (so it can
// react to a timeout/cancel/budget stop) while still passing the partial output.
func delegationResult(name, kind, out, status string) string {
if status != "" && status != "ok" {
header := fmt.Sprintf("[%s %q ended with status %q]", kind, name, status)
if out == "" { // no trailing blank line when there's no body
return header
}
return header + "\n" + out
}
return out
}
// delegationErr wraps a hard delegation failure, folding in any partial output
// the child produced before failing (so it isn't silently lost).
func delegationErr(kind, name, partial string, err error) error {
if partial != "" {
return fmt.Errorf("%s %q failed (partial output: %q): %w", kind, name, partial, err)
}
return fmt.Errorf("%s %q failed: %w", kind, name, err)
}
type skillDelegateArgs struct {
Inputs map[string]any `json:"inputs" description:"Inputs for the skill, matching its declared input schema."`
}
type agentDelegateArgs struct {
Prompt string `json:"prompt" description:"The task/prompt to hand the sub-agent."`
}
+125
View File
@@ -0,0 +1,125 @@
package run_test
import (
"context"
"encoding/json"
"testing"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/majordomo/provider/fake"
"gitea.stevedudenhoeffer.com/steve/executus/run"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
// recordingPalette captures the delegation call it received.
type recordingPalette struct {
gotName, gotCaller, gotParent string
gotInputs map[string]any
}
func (p *recordingPalette) ResolveSkill(context.Context, string, string) (string, error) {
return "", nil
}
func (p *recordingPalette) InvokeSkill(_ context.Context, callerID, _, name string, inputs map[string]any, parentRunID string) (string, string, string, error) {
p.gotName, p.gotCaller, p.gotParent, p.gotInputs = name, callerID, parentRunID, inputs
return "the skill output", "child-run-1", "ok", nil
}
func (p *recordingPalette) ResolveAgent(context.Context, string, string) (string, error) {
return "", nil
}
func (p *recordingPalette) InvokeAgent(context.Context, string, string, string, string, string, string, string, []string, func(context.Context, string, string)) (string, string, string, error) {
return "", "", "ok", nil
}
// TestPaletteDelegation: an agent with a SkillPalette gets a skill__<name> tool;
// the model calls it, the executor routes it through run.Ports.Palette as a
// child of the current run, and the result flows back into the loop.
func TestPaletteDelegation(t *testing.T) {
pal := &recordingPalette{}
fp := fake.New("fake")
fp.Enqueue("m",
fake.ReplyWith(llm.Response{ToolCalls: []llm.ToolCall{{
ID: "c1",
Name: "skill__helper",
Arguments: json.RawMessage(`{"inputs":{"q":"hi"}}`),
}}}),
fake.Reply("delegated and done"),
)
m, err := fp.Model("m")
if err != nil {
t.Fatal(err)
}
ex := run.New(run.Config{
Registry: tool.NewRegistry(),
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) { return ctx, m, nil },
Ports: run.Ports{Palette: pal},
})
res := ex.Run(context.Background(),
run.RunnableAgent{ID: "a1", Name: "boss", ModelTier: "m", SkillPalette: []string{"helper"}},
tool.Invocation{RunID: "parent-run", CallerID: "caller-7", ChannelID: "chan"},
"delegate please")
if res.Err != nil {
t.Fatalf("run error: %v", res.Err)
}
if res.Output != "delegated and done" {
t.Errorf("output = %q", res.Output)
}
if pal.gotName != "helper" {
t.Errorf("InvokeSkill name = %q, want helper", pal.gotName)
}
if pal.gotCaller != "caller-7" {
t.Errorf("InvokeSkill caller = %q, want caller-7", pal.gotCaller)
}
if pal.gotParent != "parent-run" {
t.Errorf("InvokeSkill parentRunID = %q, want parent-run (child of the current run)", pal.gotParent)
}
if pal.gotInputs["q"] != "hi" {
t.Errorf("InvokeSkill inputs = %+v, want q=hi", pal.gotInputs)
}
}
// TestNoPaletteNoDelegationTools: nil PaletteSource → no delegation tools, run
// still works (the agent just has no skill__/agent__ tools).
func TestNoPaletteNoDelegationTools(t *testing.T) {
fp := fake.New("fake")
fp.Enqueue("m", fake.Reply("ok"))
m, _ := fp.Model("m")
ex := run.New(run.Config{
Registry: tool.NewRegistry(),
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) { return ctx, m, nil },
})
res := ex.Run(context.Background(),
run.RunnableAgent{Name: "x", ModelTier: "m", SkillPalette: []string{"helper"}},
tool.Invocation{RunID: "r"}, "hi")
if res.Err != nil || res.Output != "ok" {
t.Fatalf("nil-palette run failed: %v / %q", res.Err, res.Output)
}
}
// TestDelegationDedupeAndEmptySkip: empty + duplicate palette names are skipped,
// not turned into "skill__"/duplicate tools that error at box.Add (gadfly C0).
func TestDelegationDedupeAndEmptySkip(t *testing.T) {
pal := &recordingPalette{}
fp := fake.New("fake")
fp.Enqueue("m", fake.Reply("ok"))
m, _ := fp.Model("m")
ex := run.New(run.Config{
Registry: tool.NewRegistry(),
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) { return ctx, m, nil },
Ports: run.Ports{Palette: pal},
})
// "" (empty) and a duplicate "helper" must not break the build.
res := ex.Run(context.Background(),
run.RunnableAgent{Name: "x", ModelTier: "m", SkillPalette: []string{"helper", "", "helper"}},
tool.Invocation{RunID: "r"}, "hi")
if res.Err != nil {
t.Fatalf("empty/duplicate palette names should be skipped, not error: %v", res.Err)
}
if res.Output != "ok" {
t.Fatalf("output = %q", res.Output)
}
}
+398
View File
@@ -0,0 +1,398 @@
package run
import (
"bytes"
"context"
"errors"
"fmt"
"log/slog"
"strings"
"text/template"
"unicode/utf8"
"gitea.stevedudenhoeffer.com/steve/majordomo/agent"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
)
// The multi-step phase runner. A phased RunnableAgent (ra.Phases non-empty) runs
// its phases in order; each phase is a fresh majordomo agent loop (or a single
// bare LLM call for IsRunFunc phases) with its own template-expanded system
// prompt, model tier, step cap, and tool subset. Phase outputs feed later phases
// through {{.<PhaseName>}} template variables; {{.Query}} is the original input.
// The final phase's output is the run's output.
//
// Ported from mort's agentexec pipeline so the executus kernel — which already
// carries RunnableAgent.Phases as a DTO — actually EXECUTES them (it previously
// ignored the slice and ran a single loop with the base prompt). It reuses the
// shared run machinery built once in Run: the same stepObserver (so audit/steps/
// critic-activity accumulate across every phase, including IsRunFunc bare calls),
// the same critic steer, and the same compaction option.
//
// Semantics preserved from mort's pipeline:
// - phases run sequentially; ctx cancellation/deadline/critic-kill aborts the
// run (even mid-phase and even for an Optional phase).
// - IsRunFunc = one bare LLM call, no tools, no loop.
// - Optional phases swallow NON-context errors and substitute FallbackMessage.
// - a non-optional phase that merely exhausts its step/tool budget is NOT fatal:
// its partial transcript is salvaged and the pipeline continues — EXCEPT a
// final phase that salvaged nothing, which is a genuine empty-result failure.
// - per-phase ModelTier resolve failures fall back to the base model with a WARN.
//
// Deliberately NOT carried over (kernel is leaner than mort's legacy pipeline):
// the legacy `submit` capture tool (the kernel relies on majordomo's
// no-tool-call-is-final-answer termination, like its single-loop path), and the
// critic's dynamic iteration ceiling (per-phase caps are fixed at phase start —
// the run-level critic's steer + hard deadline still apply across phases).
//
// NOTE on phase names: {{.<PhaseName>}} resolves a map key, so a phase whose name
// is not a Go-template identifier (hyphens, spaces, leading digit) cannot be
// referenced as {{.my-phase}} — authors must use {{index . "my-phase"}}. A
// template that fails to parse/execute is logged (WARN) and passed through
// unchanged rather than silently dropped (see expandPhaseTemplate). Avoid naming
// a phase "Query" — it shadows the original-input variable.
// phaseDeps carries the per-run state the phase runner shares with Run: the base
// model, the full decorated toolbox (filtered per phase), the base step cap, the
// shared agent options (tool-error limits + compactor — the step observer is
// added per phase, NOT in sharedOpts, so checkpointing can vary per path), the
// shared step observer (wired into each phase's loop AND invoked for IsRunFunc
// bare calls), the critic/session steer, and the audit recorder (phase events).
type phaseDeps struct {
baseModel llm.Model
baseToolbox *llm.Toolbox
baseMaxIter int
sharedOpts []agent.Option
stepObserver func(agent.Step)
steer func() []llm.Message
rec RunRecorder
// checkpointer records phase-boundary progress (completed phases) for durable
// recovery; nil = non-durable. resume carries a recovered run's completed
// phases so they are skipped on re-run. Phase recovery is boundary-granular:
// the interrupted (active) phase re-runs from its start (its mid-phase
// transcript is NOT resumed — only the single-loop path resumes mid-loop).
checkpointer Checkpointer
resume *ResumeState
}
// runPhases executes ra.Phases sequentially and returns a synthetic agent.Result
// whose Output is the final phase's output, with Usage aggregated across phases
// and Messages set to the last phase's transcript (for the PostRun hook). A hard
// (non-optional, non-budget) phase failure — and any context cancellation/
// deadline/critic-kill — returns the error.
func (e *Executor) runPhases(runCtx context.Context, ra RunnableAgent, deps phaseDeps, query string, images []llm.ImagePart) (*agent.Result, error) {
outputs := make(map[string]string, len(ra.Phases))
var completed []PhaseOutput
var lastResult *agent.Result
var lastOutput string
var totalUsage llm.Usage
// resumeSkip is the set of phases already finished on a RECOVERED run — kept
// SEPARATE from the live `outputs` map (which fills as phases run this time) so
// the skip guard only skips RESUME-completed phases, never a fresh run's own
// phases. (Reusing `outputs` would make a second phase with a duplicate name
// skip itself.) Pre-populate outputs + completed so a resumed run threads the
// saved outputs into later phases. The interrupted (active) phase is NOT
// pre-populated, so it re-runs from its start (boundary-granular recovery).
resumeSkip := map[string]bool{}
if deps.resume != nil {
for _, pc := range deps.resume.CompletedPhases {
outputs[pc.Name] = pc.Output
resumeSkip[pc.Name] = true
completed = append(completed, pc)
lastOutput = pc.Output
}
}
// finish stamps the aggregated usage + final output onto the synthetic result.
finish := func(err error) (*agent.Result, error) {
if lastResult == nil {
lastResult = &agent.Result{}
}
lastResult.Usage = totalUsage
if err == nil {
lastResult.Output = lastOutput
}
return lastResult, err
}
for i, phase := range ra.Phases {
// Skip phases already completed on a resumed run.
if resumeSkip[phase.Name] {
continue
}
// A killed/timed-out/cancelled run must not start its next phase.
if err := runCtx.Err(); err != nil {
return finish(err)
}
instructions := expandPhaseTemplate(phase.SystemPrompt, query, outputs)
if deps.rec != nil {
deps.rec.LogEvent("phase_start", map[string]any{"phase": phase.Name})
}
output, res, err := e.runOnePhase(runCtx, ra, deps, phase, instructions, query, images)
if res != nil {
lastResult = res
totalUsage = addUsage(totalUsage, res.Usage)
}
if err != nil {
// A context cancellation / deadline / critic-kill is NEVER swallowed by
// the Optional or budget-salvage branches — the run genuinely ended and
// must surface as cancelled/timeout/killed (statusFor classifies it).
if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
return finish(err)
}
isLast := i == len(ra.Phases)-1
trimmed := strings.TrimSpace(output)
switch {
case phase.Optional:
output = phase.FallbackMessage
if output == "" {
output = fmt.Sprintf("(Phase %q encountered an error -- proceeding without its results)", phase.Name)
}
slog.Warn("run: optional pipeline phase failed",
"agent", ra.Name, "phase", phase.Name, "error", err)
if deps.rec != nil {
deps.rec.LogEvent("phase_failed_optional", map[string]any{"phase": phase.Name, "error": err.Error()})
}
case isPhaseBudgetExhaustion(err) && (!isLast || trimmed != ""):
// Soft stop: the phase ran out of its step/tool budget before
// composing a final answer. Not fatal — it did real work (runOnePhase
// salvaged its partial transcript into output), and aborting would
// discard every completed phase before it. Degrade and continue.
// (A FINAL phase that salvaged nothing falls through to the hard error
// below: there is no result to return.)
if trimmed == "" {
output = fmt.Sprintf("(Phase %q reached its step budget before producing a consolidated result; continuing with its partial findings.)", phase.Name)
} else {
output += fmt.Sprintf("\n\n(Note: phase %q reached its step budget before fully completing; the above is its partial output.)", phase.Name)
}
slog.Warn("run: pipeline phase exhausted its budget; salvaging partial output and continuing",
"agent", ra.Name, "phase", phase.Name, "last_phase", isLast, "error", err)
if deps.rec != nil {
deps.rec.LogEvent("phase_budget_exhausted", map[string]any{"phase": phase.Name, "error": err.Error(), "last_phase": isLast})
}
default:
return finish(fmt.Errorf("pipeline phase %q: %w", phase.Name, err))
}
}
outputs[phase.Name] = output
lastOutput = output
// Checkpoint the phase boundary: this phase is done, so a resumed run skips
// it and continues from the next. (Copy the slice — the checkpointer may
// hold/serialize it asynchronously.)
completed = append(completed, PhaseOutput{Name: phase.Name, Output: output})
if deps.checkpointer != nil {
_ = deps.checkpointer.Save(runCtx, RunCheckpointState{
CompletedPhases: append([]PhaseOutput(nil), completed...),
})
}
}
return finish(nil)
}
// runOnePhase runs a single phase: a bare LLM call for IsRunFunc phases, a fresh
// agent loop otherwise. Returns the phase output, the loop result (nil for a
// failed bare call), and any error. On a budget-exhaustion error the loop's
// partial transcript is salvaged into the returned output.
func (e *Executor) runOnePhase(runCtx context.Context, ra RunnableAgent, deps phaseDeps, phase Phase, instructions, query string, images []llm.ImagePart) (string, *agent.Result, error) {
phaseCtx, model := e.phaseModel(runCtx, deps, ra, phase)
// The phase's expanded instructions are the system prompt (with the platform
// header so tools keep their run ids); the original query is the user message.
system := e.systemPromptWithBody(instructions)
if phase.IsRunFunc {
// Bare LLM call: no tool loop, no tools array (some models 400 on an empty
// tools list). The response is fed through the SAME step observer as a loop
// step so the audit token tally, Result.Steps, AND the critic's activity
// clock all see it (a long synthesize phase must not look idle to the critic).
msgs := []llm.Message{multimodalUserMessage(query, images)}
resp, err := model.Generate(phaseCtx, llm.Request{System: system, Messages: msgs})
if err != nil {
return "", nil, fmt.Errorf("phase %q model call: %w", phase.Name, err)
}
if deps.stepObserver != nil {
deps.stepObserver(agent.Step{Index: 0, Response: resp})
}
return resp.Text(), &agent.Result{
Output: resp.Text(),
Usage: resp.Usage,
Messages: append(msgs, resp.Message()),
}, nil
}
toolbox := filterToolbox(deps.baseToolbox, phase.Tools)
maxIter := phase.MaxIterations
if maxIter <= 0 {
maxIter = deps.baseMaxIter
}
// Per-phase opts: a fixed step ceiling for this phase (the critic's dynamic
// ceiling is intentionally not propagated to phases) + the phase toolbox + the
// shared step observer (audit/steps/critic), on top of the shared opts
// (tool-error limits, compactor).
opts := append([]agent.Option{
agent.WithToolbox(toolbox),
agent.WithMaxSteps(maxIter),
agent.WithStepObserver(deps.stepObserver),
}, deps.sharedOpts...)
ag := agent.New(model, system, opts...)
res, runErr := runAgent(phaseCtx, ag, query, images, agent.WithSteer(deps.steer))
output := ""
if res != nil {
output = res.Output
}
// Budget/guard exhaustion leaves a usable partial transcript but an empty
// final answer; salvage the narrated work so the pipeline can carry it forward.
if runErr != nil && isPhaseBudgetExhaustion(runErr) {
if salvaged := salvagePhaseTranscript(res); salvaged != "" {
output = salvaged
}
}
return output, res, runErr
}
// phaseModel resolves the phase's model tier, returning the resolver's enriched
// context (usage attribution) alongside the model. An empty tier or a resolution
// failure falls back to the base model + the run context (WARN — visible, not
// fatal). Returning the enriched ctx mirrors the single-loop path, which adopts
// ctx = modelCtx, so a non-base-tier phase's calls are attributed correctly.
func (e *Executor) phaseModel(ctx context.Context, deps phaseDeps, ra RunnableAgent, phase Phase) (context.Context, llm.Model) {
if phase.ModelTier == "" {
return ctx, deps.baseModel
}
modelCtx, m, err := e.cfg.Models(ctx, phase.ModelTier)
if err != nil || m == nil {
reason := "resolver returned a nil model"
if err != nil {
reason = err.Error()
}
slog.Warn("run: pipeline phase model resolve failed; using base model",
"agent", ra.Name, "phase", phase.Name, "tier", phase.ModelTier, "reason", reason)
return ctx, deps.baseModel
}
return modelCtx, m
}
// isPhaseBudgetExhaustion reports whether err is a soft budget/guard stop (the
// loop hit its step cap or tripped a tool-error guard) — which leaves a usable
// partial transcript — as opposed to a hard error (cancellation, model failure).
func isPhaseBudgetExhaustion(err error) bool {
return errors.Is(err, agent.ErrMaxSteps) || errors.Is(err, agent.ErrToolLoop)
}
// maxSalvageBytes bounds a salvaged partial transcript so a long phase's narrated
// reasoning doesn't blow up the next phase's prompt (the tail is the most recent,
// most relevant reasoning). Matches mort's pipeline cap.
const maxSalvageBytes = 8000
// salvagePhaseTranscript reconstructs a best-effort phase output from a loop that
// ended without a final answer: the assistant's narrated text across every step,
// tail-trimmed to maxSalvageBytes on a rune boundary. Returns "" when the model
// wrote no prose.
func salvagePhaseTranscript(res *agent.Result) string {
if res == nil {
return ""
}
var b strings.Builder
for _, step := range res.Steps {
if step.Response == nil {
continue
}
if t := strings.TrimSpace(step.Response.Text()); t != "" {
if b.Len() > 0 {
b.WriteString("\n\n")
}
b.WriteString(t)
}
}
out := strings.TrimSpace(b.String())
if len(out) > maxSalvageBytes {
tail := out[len(out)-maxSalvageBytes:]
// Advance to the next rune boundary so the cut never splits a UTF-8 rune.
for len(tail) > 0 && !utf8.RuneStart(tail[0]) {
tail = tail[1:]
}
out = "...(earlier reasoning trimmed)...\n" + tail
}
return out
}
// multimodalUserMessage builds a user message from text + inline images. Shared
// by the phase runner and runAgent so the image-folding lives in one place.
// Empty text with images yields an image-only message (no empty text part).
func multimodalUserMessage(text string, images []llm.ImagePart) llm.Message {
if len(images) == 0 {
return llm.UserText(text)
}
parts := make([]llm.Part, 0, len(images)+1)
if strings.TrimSpace(text) != "" {
parts = append(parts, llm.Text(text))
}
for _, img := range images {
parts = append(parts, img)
}
return llm.UserParts(parts...)
}
// expandPhaseTemplate applies Go text/template substitution to a phase prompt,
// replacing {{.Query}} with the original query and {{.<PhaseName>}} with a prior
// phase's output. On a parse/execute error it logs a WARN and returns the
// template unchanged (best-effort, non-fatal) so a misconfigured prompt is
// visible rather than silently masked.
func expandPhaseTemplate(tmpl, query string, priorOutputs map[string]string) string {
t, err := template.New("phase").Option("missingkey=zero").Parse(tmpl)
if err != nil {
slog.Warn("run: pipeline phase template parse failed; using it unexpanded", "error", err)
return tmpl
}
data := map[string]string{"Query": query}
for k, v := range priorOutputs {
data[k] = v
}
var buf bytes.Buffer
if err := t.Execute(&buf, data); err != nil {
slog.Warn("run: pipeline phase template execute failed; using it unexpanded", "error", err)
return tmpl
}
return buf.String()
}
// filterToolbox returns a toolbox restricted to the named tools (preserving
// palette order). Empty names = the full palette (the base toolbox is returned
// as-is — it is read-only during a run, like the single-loop path). Unknown names
// are skipped with a WARN — a typo'd phase tool list should not abort a run.
func filterToolbox(box *llm.Toolbox, names []string) *llm.Toolbox {
if len(names) == 0 {
return box
}
out := llm.NewToolbox(box.Name())
for _, name := range names {
t, ok := box.Get(name)
if !ok {
slog.Warn("run: pipeline phase references unknown tool; skipping", "tool", name)
continue
}
if err := out.Add(t); err != nil {
slog.Warn("run: pipeline phase tool duplicated; skipping", "tool", name, "error", err)
}
}
return out
}
// addUsage sums two llm.Usage tallies field-by-field so a phased run reports the
// total tokens across all phases. NOTE: if llm.Usage gains a field, add it here
// too — the audit recorder (rec) is the authoritative per-run token source, this
// is the secondary Result.Usage roll-up.
func addUsage(a, b llm.Usage) llm.Usage {
a.InputTokens += b.InputTokens
a.OutputTokens += b.OutputTokens
a.CacheReadTokens += b.CacheReadTokens
a.CacheWriteTokens += b.CacheWriteTokens
a.ReasoningTokens += b.ReasoningTokens
return a
}
+278
View File
@@ -0,0 +1,278 @@
package run
import (
"context"
"encoding/json"
"errors"
"strings"
"testing"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/majordomo/provider/fake"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
// phaseProvider builds a fake provider scripted with the given per-call steps
// (consumed in order across every phase's model call) and a resolver over it,
// returning both so a test can read back each call's request.
func phaseProvider(t *testing.T, steps ...fake.Step) (ModelResolver, *fake.Provider) {
t.Helper()
fp := fake.New("fake")
fp.Enqueue("test-model", steps...)
m, err := fp.Model("test-model")
if err != nil {
t.Fatalf("fake model: %v", err)
}
return func(ctx context.Context, _ string) (context.Context, llm.Model, error) {
return ctx, m, nil
}, fp
}
// TestPhases_SequentialThreadsOutputs: phases run in order, each phase's output
// is threaded into the next via {{.<PhaseName>}}, {{.Query}} reaches a phase, and
// the final phase's output is the run output.
func TestPhases_SequentialThreadsOutputs(t *testing.T) {
models, fp := phaseProvider(t,
fake.Reply("out-a"),
fake.Reply("out-b"),
fake.Reply("out-c"),
)
ex := New(Config{Registry: tool.NewRegistry(), Models: models})
ra := RunnableAgent{
Name: "pipeline",
ModelTier: "test-model",
Phases: []Phase{
{Name: "a", SystemPrompt: "Phase A instructions"},
{Name: "b", SystemPrompt: "B saw: {{.a}}"},
{Name: "c", SystemPrompt: "C saw: {{.b}} and query {{.Query}}"},
},
}
res := ex.Run(context.Background(), ra, tool.Invocation{RunID: "r", CallerID: "c"}, "QUERY-TEXT")
if res.Err != nil {
t.Fatalf("run error: %v", res.Err)
}
if res.Output != "out-c" {
t.Fatalf("final output = %q, want the LAST phase's output out-c", res.Output)
}
calls := fp.Calls()
if len(calls) != 3 {
t.Fatalf("want 3 model calls (one per phase), got %d", len(calls))
}
if got := calls[0].Request.System; got != "Phase A instructions" {
t.Errorf("phase a system = %q", got)
}
if got := calls[1].Request.System; got != "B saw: out-a" {
t.Errorf("phase b should see phase a's output threaded; system = %q", got)
}
if got := calls[2].Request.System; got != "C saw: out-b and query QUERY-TEXT" {
t.Errorf("phase c should see phase b's output + {{.Query}}; system = %q", got)
}
}
// TestPhases_OptionalFailureSubstitutesFallback: an Optional phase that errors
// does not abort the pipeline — its FallbackMessage becomes its output and is
// threaded into later phases, which still run.
func TestPhases_OptionalFailureSubstitutesFallback(t *testing.T) {
models, fp := phaseProvider(t,
fake.Fail(errors.New("provider exploded")), // phase a fails
fake.Reply("out-b"), // phase b runs
)
ex := New(Config{Registry: tool.NewRegistry(), Models: models})
ra := RunnableAgent{
Name: "pipeline",
ModelTier: "test-model",
Phases: []Phase{
{Name: "a", SystemPrompt: "Phase A", Optional: true, FallbackMessage: "FALLBACK-A"},
{Name: "b", SystemPrompt: "B saw: {{.a}}"},
},
}
res := ex.Run(context.Background(), ra, tool.Invocation{RunID: "r", CallerID: "c"}, "Q")
if res.Err != nil {
t.Fatalf("optional-phase failure must not fail the run: %v", res.Err)
}
if res.Output != "out-b" {
t.Fatalf("final output = %q, want out-b", res.Output)
}
calls := fp.Calls()
if len(calls) != 2 {
t.Fatalf("want 2 calls (failed phase a + phase b), got %d", len(calls))
}
if got := calls[1].Request.System; got != "B saw: FALLBACK-A" {
t.Errorf("phase b should see the fallback threaded; system = %q", got)
}
}
// TestPhases_OptionalDoesNotSwallowCancellation: an Optional phase that fails
// with a context cancellation must NOT be swallowed into its FallbackMessage —
// the run genuinely ended (cancel/deadline/critic-kill) and must surface the
// error so the run is classified cancelled/timeout/killed, not "ok".
func TestPhases_OptionalDoesNotSwallowCancellation(t *testing.T) {
models, _ := phaseProvider(t, fake.Fail(context.Canceled))
ex := New(Config{Registry: tool.NewRegistry(), Models: models})
ra := RunnableAgent{
Name: "pipeline",
ModelTier: "test-model",
Phases: []Phase{
// IsRunFunc so the cancellation surfaces directly wrapped (%w).
{Name: "a", SystemPrompt: "Phase A", IsRunFunc: true, Optional: true, FallbackMessage: "FB"},
},
}
res := ex.Run(context.Background(), ra, tool.Invocation{RunID: "r", CallerID: "c"}, "Q")
if !errors.Is(res.Err, context.Canceled) {
t.Fatalf("Optional phase must NOT swallow a cancellation; res.Err = %v", res.Err)
}
if res.Output == "FB" {
t.Error("a cancelled run must not report the fallback message as output")
}
}
// TestPhases_DuplicateNamesBothRun: a fresh (non-resume) run with two phases
// sharing a name must run BOTH — the resume-skip guard keys off a separate
// resume set, not the live outputs map (which fills as phases run), so a phase
// never skips a same-named sibling on a fresh run.
func TestPhases_DuplicateNamesBothRun(t *testing.T) {
models, fp := phaseProvider(t, fake.Reply("first"), fake.Reply("second"))
ex := New(Config{Registry: tool.NewRegistry(), Models: models})
ra := RunnableAgent{
Name: "p", ModelTier: "test-model",
Phases: []Phase{{Name: "x", SystemPrompt: "P1"}, {Name: "x", SystemPrompt: "P2"}},
}
res := ex.Run(context.Background(), ra, tool.Invocation{RunID: "r"}, "Q")
if res.Err != nil {
t.Fatalf("run error: %v", res.Err)
}
if n := len(fp.Calls()); n != 2 {
t.Fatalf("both same-named phases must run on a fresh run; got %d model calls", n)
}
}
// TestPhases_HardErrorAborts: a NON-optional phase that hits a hard error (not a
// budget/step exhaustion) aborts the pipeline; later phases do not run.
func TestPhases_HardErrorAborts(t *testing.T) {
boom := errors.New("model down")
models, fp := phaseProvider(t,
fake.Fail(boom), // phase a (non-optional) fails hard
fake.Reply("out-b"), // must NOT be consumed
)
ex := New(Config{Registry: tool.NewRegistry(), Models: models})
ra := RunnableAgent{
Name: "pipeline",
ModelTier: "test-model",
Phases: []Phase{
{Name: "a", SystemPrompt: "Phase A"},
{Name: "b", SystemPrompt: "Phase B"},
},
}
res := ex.Run(context.Background(), ra, tool.Invocation{RunID: "r", CallerID: "c"}, "Q")
if res.Err == nil {
t.Fatal("a hard non-optional phase error must fail the run")
}
if !errors.Is(res.Err, boom) {
t.Errorf("run error %v should wrap the phase's model error", res.Err)
}
if n := len(fp.Calls()); n != 1 {
t.Errorf("pipeline must abort after phase a; got %d calls (phase b should not run)", n)
}
}
// TestPhases_IsRunFuncBareCall: an IsRunFunc phase produces output via a bare LLM
// call and that output threads into a following loop phase.
func TestPhases_IsRunFuncBareCall(t *testing.T) {
models, fp := phaseProvider(t,
fake.Reply("plan-output"), // IsRunFunc phase a
fake.Reply("final"), // loop phase b
)
ex := New(Config{Registry: tool.NewRegistry(), Models: models})
ra := RunnableAgent{
Name: "pipeline",
ModelTier: "test-model",
Phases: []Phase{
{Name: "plan", SystemPrompt: "Make a plan for {{.Query}}", IsRunFunc: true},
{Name: "exec", SystemPrompt: "Execute: {{.plan}}"},
},
}
res := ex.Run(context.Background(), ra, tool.Invocation{RunID: "r", CallerID: "c"}, "do-thing")
if res.Err != nil {
t.Fatalf("run error: %v", res.Err)
}
if res.Output != "final" {
t.Fatalf("output = %q, want final", res.Output)
}
calls := fp.Calls()
if len(calls) != 2 {
t.Fatalf("want 2 calls, got %d", len(calls))
}
if got := calls[0].Request.System; got != "Make a plan for do-thing" {
t.Errorf("IsRunFunc phase system = %q", got)
}
if got := calls[1].Request.System; got != "Execute: plan-output" {
t.Errorf("exec phase should see the plan output threaded; system = %q", got)
}
}
// TestPhases_SystemHeaderAppliedPerPhase: the platform SystemHeader is prepended
// to every phase's prompt (each phase keeps it).
func TestPhases_SystemHeaderAppliedPerPhase(t *testing.T) {
models, fp := phaseProvider(t, fake.Reply("a"), fake.Reply("b"))
ex := New(Config{Registry: tool.NewRegistry(), Models: models, SystemHeader: "PLATFORM"})
ra := RunnableAgent{
Name: "p",
ModelTier: "test-model",
Phases: []Phase{{Name: "one", SystemPrompt: "P1"}, {Name: "two", SystemPrompt: "P2"}},
}
if res := ex.Run(context.Background(), ra, tool.Invocation{RunID: "r"}, "Q"); res.Err != nil {
t.Fatalf("run error: %v", res.Err)
}
for i, want := range []string{"PLATFORM\n\nP1", "PLATFORM\n\nP2"} {
if got := fp.Calls()[i].Request.System; got != want {
t.Errorf("phase %d system = %q, want %q", i, got, want)
}
}
}
// TestFilterToolbox: a named subset restricts the toolbox (preserving order);
// empty names = the full palette; unknown names are skipped.
func TestFilterToolbox(t *testing.T) {
box := llm.NewToolbox("base")
noop := func(context.Context, json.RawMessage) (any, error) { return "", nil }
for _, name := range []string{"alpha", "beta", "gamma"} {
if err := box.Add(llm.Tool{Name: name, Description: "d", Handler: noop}); err != nil {
t.Fatalf("add %s: %v", name, err)
}
}
full := filterToolbox(box, nil)
if len(full.Tools()) != 3 {
t.Errorf("nil names = full palette; got %d tools", len(full.Tools()))
}
sub := filterToolbox(box, []string{"gamma", "alpha", "nonexistent"})
names := make([]string, 0)
for _, tl := range sub.Tools() {
names = append(names, tl.Name)
}
if strings.Join(names, ",") != "gamma,alpha" {
t.Errorf("subset (order-preserving, unknown skipped) = %v, want [gamma alpha]", names)
}
}
// TestExpandPhaseTemplate: {{.Query}} + prior outputs substitute; a parse error
// returns the template unchanged (best-effort).
func TestExpandPhaseTemplate(t *testing.T) {
got := expandPhaseTemplate("q={{.Query}} a={{.a}}", "QQ", map[string]string{"a": "AA"})
if got != "q=QQ a=AA" {
t.Errorf("expand = %q", got)
}
// Malformed template → returned unchanged.
bad := "{{.Unclosed"
if expandPhaseTemplate(bad, "QQ", nil) != bad {
t.Errorf("malformed template should pass through unchanged")
}
}
+252
View File
@@ -0,0 +1,252 @@
package run
import (
"context"
"errors"
"time"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/executus/deliver"
)
// ErrCriticKill is the cancellation cause the executor stamps on a run the
// critic kills, so a critic kill surfaces as a distinct "killed" status (vs a
// backstop "timeout" or a caller "cancelled"). A host CriticHandle signals a
// kill via KillCause(); the executor wraps that reason with this sentinel.
var ErrCriticKill = errors.New("run: critic killed the run")
// Ports are the host seams the run executor consumes. Every field is nil-safe:
// a light host passes the zero Ports and gets a bounded, in-memory run with no
// persistence, audit, budget, critic, delegation, or delivery — which is
// exactly a gadfly swarm task. A heavy host (mort) wires each one to a battery.
//
// This struct IS the inversion: in mort, agentexec imports agents /
// agentcritic / skillaudit and skillexec imports skills / paste directly; here
// the kernel depends only on these interfaces, and the batteries implement
// them. The mort_*_adapters.go wall becomes the set of impls.
type Ports struct {
// Audit records the run trace (start, per-step/per-tool events, final
// stats). nil = no audit.
Audit Audit
// Budget gates and meters per-caller resource use. nil = unbounded.
Budget Budget
// Critic optionally monitors a long run for hangs/runaways. nil = none.
Critic Critic
// Checkpointer mints a per-run Checkpointer for durable recovery (it decides
// per run whether the run is durable). nil = no checkpointing (a run
// interrupted by shutdown is simply lost).
Checkpointer CheckpointerFactory
// Palette resolves SkillPalette / SubAgentPalette entries into delegation
// tools (skill__<name> / agent__<name>). nil = those entries are inert.
Palette PaletteSource
// Delivery is where the run's output + artifacts go. nil = the caller
// reads the Result in-process (the light-host default).
Delivery deliver.Delivery
// InputFiles persists non-image input attachments (audio, PDF, binary)
// carried on Invocation.InputFiles into a host file store under run scope,
// returning file_ids the agent can hand to a worker tool. nil = input files
// are silently ignored (the run still proceeds, text-only). The bytes are
// never inlined into the model context — the LLM can't read raw audio/binary.
InputFiles InputFileStager
// SkillPacks activates a RunnableAgent.SkillPacks (SKILL.md subscriptions)
// for the run: it folds a catalog into the system prompt and adds a skill_use
// loader tool. nil = SkillPacks are inert. The executus/skillpack battery
// ships a default impl (skillpack.Activator).
SkillPacks SkillPackActivator
}
// SkillPackActivator resolves an agent's subscribed skill-pack names for a run
// into system-prompt instructions (a catalog of what's available on demand) and
// the tools that back them (a single skill_use loader). It receives the run +
// subject ids so the impl can scope any per-run file staging. It returns "" +
// nil when nothing resolves; activation errors are non-fatal to the run. Defined
// here (the consumer) so the battery satisfies it structurally without importing
// run — the same inversion as the other ports.
type SkillPackActivator interface {
ActivateSkillPacks(ctx context.Context, names []string, runID, subjectID string) (instructions string, tools []llm.Tool, err error)
}
// InputFileStager persists a single non-image input attachment into a host file
// store under run scope and returns a file_id the run can reference. It is the
// seam mort's skill FileStorage (and any host blob store) implements so the
// kernel can stage Invocation.InputFiles without importing a storage layer.
type InputFileStager interface {
StageInputFile(ctx context.Context, runID, agentID, name, mime string, content []byte) (fileID string, err error)
}
// RunInfo describes a run at start time — the attribution a recorder/critic
// needs. Host-neutral rename of mort's SkillRun start fields.
type RunInfo struct {
RunID string
SubjectID string // the agent/skill id being run (audit "skill_id")
Name string
CallerID string
ChannelID string
GuildID string // the originating guild/server id (empty for DMs/triggers)
ParentRunID string
ModelTier string // the run's resolved base tier (for checkpoint re-dispatch)
Inputs map[string]any
StartedAt time.Time
// MaxIterations is the run's base tool-dispatch step ceiling, so a critic can
// raise it relative to the baseline (see CriticHandle.MaxSteps).
MaxIterations int
}
// RunStats is the terminal roll-up a recorder's Close writes. Mirrors mort's
// skillaudit/skillexec RunStats.
type RunStats struct {
Status string // ok | error | timeout | budget_exceeded | cancelled | dry_run
Output string
Error string
ToolCalls int
RuntimeSeconds float64
InputTokens int64
OutputTokens int64
ThinkingTokens int64
}
// --- Audit ---
// Audit begins recording a run. StartRun returns a per-run RunRecorder (or nil
// to skip recording this run). The audit battery wires its Storage behind this.
type Audit interface {
StartRun(ctx context.Context, info RunInfo) RunRecorder
}
// RunRecorder records the events of one in-flight run and its final stats. It
// satisfies RunTally so the kernel can surface live token/tool counts to the
// self-status tool. Mirrors mort's skillaudit.Writer.
type RunRecorder interface {
RunTally
// OnStep records one completed agent-loop iteration's model response.
OnStep(iter int, resp *llm.Response)
// OnTool records one executed tool call + its result.
OnTool(call llm.ToolCall, result string)
// LogEvent / LogError append structured events to the run log.
LogEvent(eventType string, payload map[string]any)
LogError(msg string)
// Close writes the terminal roll-up. Detaches from the caller's context
// internally so a cancelled run still records.
Close(ctx context.Context, stats RunStats)
}
// --- Budget ---
// Budget gates and meters per-caller resource use. Mirrors mort's
// skillexec.BudgetTracker.
type Budget interface {
// Check reports whether the caller has remaining budget (nil = allowed).
Check(ctx context.Context, callerID string) error
// Commit records that the caller spent runtimeSeconds on this run.
Commit(ctx context.Context, callerID string, runtimeSeconds float64)
}
// --- Critic ---
// Critic optionally monitors a long-running run (the two-tier soft/hard
// timeout). Monitor returns a handle the executor feeds progress into and
// queries for steer/deadline decisions; a nil handle means "not monitored".
//
// The exact wiring (how the handle's Steer/Deadline bind into majordomo's
// agent.WithSteer / agent.WithMaxStepsFunc / run-context cancellation) is
// finalized in the executor; this is the seam the agentcritic battery adapts.
type Critic interface {
Monitor(ctx context.Context, info RunInfo, softTimeout time.Duration) CriticHandle
}
// CriticHandle is the executor's live link to a run's critic.
//
// Concurrency: the executor calls RecordStep/RecordToolStart/Steer from the run
// goroutine while a separate watch goroutine polls Deadline() and the run's end
// calls Stop() — so implementations MUST be safe for concurrent use across these
// methods (the critic battery's handle guards its state with a mutex).
type CriticHandle interface {
// RecordStep / RecordToolStart keep the critic's activity clock fresh so a
// healthy-but-slow run is not mistaken for a hang. RecordStep also carries the
// completed step's model response (nil-safe) so the critic's Trace can show
// what the agent actually produced, not just an iteration count.
RecordStep(iter int, resp *llm.Response)
RecordToolStart(name, args string)
// Steer returns any messages the critic wants injected into the loop (a
// nudge), drained before each step — matches majordomo agent.WithSteer.
Steer() []llm.Message
// Deadline returns the current hard-kill deadline (the critic may extend
// it); the executor binds the run context to it. Zero = no hard deadline.
Deadline() time.Time
// MaxSteps returns the current tool-dispatch step ceiling, polled by the
// executor each step (via majordomo WithMaxStepsFunc) so a critic can raise a
// healthy-but-long run's iteration budget mid-flight. Return <= 0 to defer to
// the run's base MaxIterations.
MaxSteps() int
// KillCause returns a non-nil reason iff the critic has decided to KILL this
// run (as opposed to letting the hard-deadline backstop expire). The executor
// reads it when the deadline passes: non-nil → cancel the run with
// ErrCriticKill (status "killed"); nil → the backstop expired naturally
// (status "timeout"). Hosts that never distinguish the two may return nil.
KillCause() error
// Stop ends monitoring when the run finishes.
Stop()
}
// --- Checkpointer ---
// CheckpointerFactory decides, per run, whether the run is durable and (if so)
// mints the per-run Checkpointer that records its progress. It returns (nil, nil)
// for a non-durable run (the common short-run case — no checkpointing overhead).
// A storage error should be logged and degraded to (nil, nil) so a failing
// checkpoint store never fails the run. Mirrors mort's
// agentexec.CheckpointerFactory.
type CheckpointerFactory interface {
Begin(ctx context.Context, info RunInfo) (Checkpointer, error)
}
// Checkpointer persists a run's resumable progress for durable recovery.
// Mirrors mort's agentexec.RunCheckpointer.
type Checkpointer interface {
// Save persists the run's current resumable progress (throttled).
Save(ctx context.Context, st RunCheckpointState) error
// Complete clears the checkpoint on success.
Complete(ctx context.Context) error
// Fail clears the checkpoint on terminal failure. A run interrupted by
// shutdown is left untouched so boot recovery picks it up.
Fail(ctx context.Context, err error) error
}
// RunCheckpointState is the resumable snapshot a Checkpointer persists.
type RunCheckpointState struct {
// Messages is the running transcript of a SINGLE-LOOP run (grows each step;
// resumed via WithHistory). nil for multi-phase runs — phase recovery is
// boundary-granular (see CompletedPhases), not mid-phase transcript.
Messages []llm.Message
Iteration int
// CompletedPhases is set only for multi-phase runs: the outputs of phases
// already finished, in phase order, so a resumed run skips them and re-runs
// the interrupted phase from its start. nil for single-loop runs.
CompletedPhases []PhaseOutput
}
// PhaseOutput is one completed pipeline phase's name and output text, recorded in
// a checkpoint so a resumed multi-phase run can skip already-finished phases.
type PhaseOutput struct {
Name string
Output string
}
// --- PaletteSource ---
// PaletteSource resolves a RunnableAgent's SkillPalette / SubAgentPalette names
// into delegation tools and invokes them. Mirrors mort's
// SkillInvokerForPalette + AgentInvokerForPalette. nil Palette => palette
// entries are inert ("not configured" at first call).
type PaletteSource interface {
ResolveSkill(ctx context.Context, callerID, name string) (skillID string, err error)
InvokeSkill(ctx context.Context, callerID, channelID, name string,
inputs map[string]any, parentRunID string) (output, runID, status string, err error)
ResolveAgent(ctx context.Context, callerID, name string) (agentID string, err error)
InvokeAgent(ctx context.Context, callerID, channelID, name string,
prompt, parentRunID, modelTierOverride, promptPrepend string,
toolsSubset []string,
onEvent func(ctx context.Context, event, emoji string)) (output, runID, status string, err error)
}
+157
View File
@@ -0,0 +1,157 @@
// Package run is executus's run kernel: the shared run-loop mechanics around
// majordomo's agent loop, plus the host seams (run.Ports / RunnableAgent) that
// let one executor serve every surface — a light host's bounded one-shot run,
// a heavy host's persona agent or saved skill — without the kernel importing a
// battery.
//
// This file holds the genuinely-identical scaffolding both run shapes need:
// context cancellation merging, the detached-cleanup timeout, the per-run
// progress accessor the self-status tool reads, the legacy `submit`
// compatibility tool (submit.go), the ancestor progress bridge (progress.go),
// and the run-finalizer machinery — one source of truth.
//
// The kernel depends only on majordomo + executus/tool + the run.Ports
// interfaces; persistence, audit, the persona/skill nouns, and the critic are
// host-supplied via Ports (see ports.go) so importing the kernel never drags in
// a store or a battery.
package run
import (
"context"
"errors"
"log/slog"
"sync/atomic"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
// ErrShutdown is the cancellation cause set on mort's base lifecycle context
// when the process is shutting down (SIGTERM after the drain window). The
// agent executor uses it to distinguish a run interrupted by shutdown (which
// should be left durable-recoverable) from a run that errored or hit its own
// deadline (terminal).
var ErrShutdown = errors.New("mort: shutting down")
// CleanupContextTimeout caps how long a run's post-completion cleanup ops
// (budget commit, audit Close, attachment bookkeeping) may wait on
// storage after detaching from the caller's — possibly already
// cancelled — context. 10s is generous for a single-row UPDATE against
// MySQL; longer suggests a hung connection the run goroutine shouldn't
// keep waiting on. Both executors derive their cleanup contexts as
// context.WithTimeout(context.WithoutCancel(ctx), CleanupContextTimeout).
const CleanupContextTimeout = 10 * time.Second
// Reserved state-react lifecycle event keys, shared so both nouns surface
// the same UX shape. Namespaced with double-underscores to make accidental
// collision with a tool name near-impossible.
const (
StateReactStart = "__start__"
StateReactEnd = "__end__"
StateReactError = "__error__"
StateReactBudgetExceeded = "__budget_exceeded__"
)
// MergeCancellation returns a context cancelled when EITHER input is
// cancelled, propagating the cancellation Cause from whichever fired. Used
// by the lane preemption path (the lane's per-job ctx.Cause flows into the
// run context) and by the runtime-detach path (process shutdown still
// reaches a run whose deadline was reset after a lane wait). Always call
// the returned cancel to release the watcher goroutine; it is also invoked
// once when either input fires.
func MergeCancellation(parent, secondary context.Context) (context.Context, context.CancelFunc) {
merged, cancel := context.WithCancelCause(parent)
go func() {
select {
case <-merged.Done():
return
case <-secondary.Done():
cancel(context.Cause(secondary))
}
}()
return merged, func() { cancel(nil) }
}
// RunFinalizer is invoked at run finish so per-run tool state (open HTTP
// streams, per-run code_exec counters, per-run search budgets) is released
// and the process-lifetime maps keyed by run id don't grow unbounded.
// Both executors fire their registered finalizers via FireFinalizers.
type RunFinalizer interface {
FinalizeRun(runID string)
}
// FireFinalizers runs every finalizer for runID, isolating each behind a
// panic-recover so one buggy finalizer can't take down the run goroutine
// or skip the others. Safe to call with a nil/empty slice.
func FireFinalizers(fs []RunFinalizer, runID string) {
for _, f := range fs {
if f == nil {
continue
}
func() {
defer func() {
if r := recover(); r != nil {
slog.Error("runengine: run finalizer panicked",
"run_id", runID, "panic", r)
}
}()
f.FinalizeRun(runID)
}()
}
}
// RunTally is the narrow live-progress source the RunStateAccessor reads —
// the running token and tool-call counts for the in-flight run. The audit
// battery's writer satisfies it; this interface is how the run kernel reads
// live tallies without importing the audit package (the inversion of mort's
// direct *skillaudit.Writer dependency).
type RunTally interface {
// TokenStats returns the running input, output, and thinking token totals.
TokenStats() (in, out, thinking int64)
// ToolCallsCount returns the number of tool calls executed so far.
ToolCallsCount() int
}
// RunStateAccessor is the per-run live-progress accessor the executor
// stamps on Invocation.RunState before building the toolbox, so the
// self-status tool can report iteration / tool-calls / tokens / elapsed for
// the in-flight run. Construct with NewRunStateAccessor; the executor's step
// observer calls SetIteration each loop.
type RunStateAccessor struct {
tally RunTally
iter atomic.Int32
maxIter int
maxCalls int
startedAt time.Time
}
// NewRunStateAccessor builds the accessor. writer supplies the live token
// + tool-call tallies; maxIter / maxCalls are the reported caps (0 =
// uncapped); startedAt anchors the elapsed clock.
func NewRunStateAccessor(tally RunTally, maxIter, maxCalls int, startedAt time.Time) *RunStateAccessor {
return &RunStateAccessor{
tally: tally,
maxIter: maxIter,
maxCalls: maxCalls,
startedAt: startedAt,
}
}
// SetIteration records the current agent-loop iteration (called from the
// executor's step observer).
func (a *RunStateAccessor) SetIteration(iter int) { a.iter.Store(int32(iter)) }
// RunState satisfies tool.RunStateAccessor.
func (a *RunStateAccessor) RunState() tool.RunState {
in, out, think := a.tally.TokenStats()
return tool.RunState{
Iteration: int(a.iter.Load()),
MaxIterations: a.maxIter,
ToolCalls: a.tally.ToolCallsCount(),
MaxToolCalls: a.maxCalls,
InputTokens: in,
OutputTokens: out,
ThinkingTokens: think,
ElapsedSeconds: int(time.Since(a.startedAt).Seconds()),
}
}
+88
View File
@@ -0,0 +1,88 @@
package run
import (
"context"
"log/slog"
"strings"
"sync"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
// runPostRun invokes a SessionToolFactory's PostRun hook with panic isolation:
// a PostRun panic (or a slow artifact build that the hook mishandles) must not
// fail an otherwise-successful run — artifacts are best-effort, the agent's text
// output is the source of truth.
func runPostRun(ctx context.Context,
hook func(context.Context, []llm.Message, string, error) *tool.PostRunResult,
transcript []llm.Message, output string, runErr error) (prr *tool.PostRunResult) {
defer func() {
if r := recover(); r != nil {
slog.Error("run: PostRun hook panicked; no artifacts produced", "panic", r)
prr = nil
}
}()
return hook(ctx, transcript, output, runErr)
}
// steerMailbox is a thread-safe queue of messages a session tool (via
// tool.Invocation.AttachImages) wants injected into the agent loop before its
// next step — the same WithSteer mechanism the critic uses for nudges, exposed
// to ordinary tools so they can show the model content (e.g. a rendered
// preview) it must SEE, not just be told about.
type steerMailbox struct {
mu sync.Mutex
msgs []llm.Message
}
func (m *steerMailbox) push(msg llm.Message) {
m.mu.Lock()
m.msgs = append(m.msgs, msg)
m.mu.Unlock()
}
// drain returns and clears the queued messages (nil when empty).
func (m *steerMailbox) drain() []llm.Message {
m.mu.Lock()
defer m.mu.Unlock()
if len(m.msgs) == 0 {
return nil
}
out := m.msgs
m.msgs = nil
return out
}
// runSession implements tool.AgentSession over a steer mailbox: AttachImages
// queues a user-role multimodal message the agent loop injects before its next
// step. Replaces legacy agentkit's Agent.AttachImages — majordomo's *agent.Agent
// is immutable mid-run, so mutation flows through the run-scoped steer mailbox.
type runSession struct{ mailbox *steerMailbox }
func (s *runSession) AttachImages(text string, images ...llm.ImagePart) {
parts := make([]llm.Part, 0, len(images)+1)
if strings.TrimSpace(text) != "" {
parts = append(parts, llm.Text(text))
}
for _, img := range images {
parts = append(parts, img)
}
if len(parts) == 0 {
return
}
s.mailbox.push(llm.UserParts(parts...))
}
// safeCleanup runs a SessionTools.Cleanup with panic isolation, so a misbehaving
// teardown (temp-dir removal, handle close) can't clobber an otherwise-successful
// run via the executor's top-level recover.
func safeCleanup(fn func()) {
defer func() {
if r := recover(); r != nil {
slog.Error("run: session Cleanup panicked", "panic", r)
}
}()
fn()
}
+94
View File
@@ -0,0 +1,94 @@
package run_test
import (
"context"
"testing"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/majordomo/provider/fake"
"gitea.stevedudenhoeffer.com/steve/executus/run"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
// TestSessionToolFactoryPostRun: a SessionToolFactory's PostRun hook produces an
// artifact (from the run output + transcript) that lands on Result.PostRunResult,
// and its Cleanup is deferred.
func TestSessionToolFactoryPostRun(t *testing.T) {
fp := fake.New("fake")
fp.Enqueue("m", fake.Reply("hello artifacts"))
m, _ := fp.Model("m")
cleanupCalled := false
inv := tool.Invocation{
RunID: "r1",
SessionToolFactory: func(_ tool.AgentSession) tool.SessionTools {
return tool.SessionTools{
PostRun: func(_ context.Context, transcript []llm.Message, output string, _ error) *tool.PostRunResult {
return &tool.PostRunResult{
Artifacts: []tool.Artifact{{Name: "out.txt", MimeType: "text/plain", Data: []byte(output)}},
Metadata: map[string]any{"transcript_len": len(transcript)},
}
},
Cleanup: func() { cleanupCalled = true },
}
},
}
ex := run.New(run.Config{
Registry: tool.NewRegistry(),
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) { return ctx, m, nil },
})
res := ex.Run(context.Background(), run.RunnableAgent{ModelTier: "m"}, inv, "go")
if res.Err != nil {
t.Fatalf("run error: %v", res.Err)
}
if res.PostRunResult == nil {
t.Fatal("Result.PostRunResult is nil — PostRun hook not invoked / not attached")
}
if n := len(res.PostRunResult.Artifacts); n != 1 {
t.Fatalf("artifacts = %d, want 1", n)
}
a := res.PostRunResult.Artifacts[0]
if a.Name != "out.txt" || string(a.Data) != "hello artifacts" {
t.Errorf("artifact = {%q, %q}", a.Name, string(a.Data))
}
if tl, _ := res.PostRunResult.Metadata["transcript_len"].(int); tl < 1 {
t.Errorf("transcript not passed to PostRun (len=%d)", tl)
}
if !cleanupCalled {
t.Error("Cleanup was not deferred/called")
}
}
// TestSessionToolFactoryAddsTool: tools the factory returns join the run's
// toolbox and are callable by the model.
func TestSessionToolFactoryAddsTool(t *testing.T) {
fp := fake.New("fake")
fp.Enqueue("m",
fake.ReplyWith(llm.Response{ToolCalls: []llm.ToolCall{{ID: "c1", Name: "render", Arguments: []byte(`{}`)}}}),
fake.Reply("rendered"),
)
m, _ := fp.Model("m")
toolCalled := false
renderTool := llm.DefineTool("render", "render a preview",
func(_ context.Context, _ struct{}) (any, error) { toolCalled = true; return "ok", nil })
inv := tool.Invocation{
RunID: "r2",
SessionToolFactory: func(_ tool.AgentSession) tool.SessionTools {
return tool.SessionTools{Tools: []llm.Tool{renderTool}}
},
}
ex := run.New(run.Config{
Registry: tool.NewRegistry(),
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) { return ctx, m, nil },
})
res := ex.Run(context.Background(),
run.RunnableAgent{ModelTier: "m", MaxIterations: 5}, inv, "go")
if res.Err != nil {
t.Fatalf("run error: %v", res.Err)
}
if !toolCalled {
t.Error("session-factory tool was not added to the toolbox / not called")
}
}
+419
View File
@@ -0,0 +1,419 @@
package run
// steps.go — the per-run step emitter and the tool→step presentation
// mapping. This is the single place that turns the executor's post-step
// observer into ordered tool.Step records: one per tool call, each with a
// stable id, an open-vocabulary kind, and a human present-tense summary
// that flips running→complete/error.
//
// One source feeds two consumers (mirroring the OnEvent/OnToolEvent/
// PostRunResult pattern): the live tool.Invocation.OnStep callback
// (nil-safe) AND snapshot(), which the executor copies onto Result.Steps.
// Because the Result accumulation does not depend on OnStep being set,
// every surface — chat (JSON + SSE), Discord, cron, sub-agents — carries
// steps; OnStep is needed only for live streaming.
//
// LIMITATION (current): majordomo exposes only a POST-step observer, so
// the executor calls toolStart+toolEnd back-to-back after each tool has
// already run. Steps are therefore recorded faithfully, but step.StartedAt
// ≈ EndedAt and the intermediate "running" phase is never observable to a
// live OnStep consumer. A pre-dispatch hook (wrapping each tool's handler
// to emit toolStart before execution, like mort's state-react decorator)
// is a follow-up that would restore real start timing + the running phase.
// The emitter already supports that two-call shape — toolStart and toolEnd
// are separate methods — so wiring it later is additive.
import (
"context"
"encoding/json"
"fmt"
"net/url"
"strings"
"time"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
// stepSummaryMaxLen caps the human summary length (section G size cap).
// Detail is unused in v1 (no live detail source while replies are
// generated blocking) so there is no Detail cap yet.
const stepSummaryMaxLen = 200
// stepEmitter accumulates ordered steps for one run and fires the live
// OnStep callback.
//
// Concurrency: touched ONLY from the agent-loop goroutine — the executor's
// stepObserver (and, once a pre-dispatch hook is wired, that hook) run
// there; majordomo executes a step's tool calls sequentially, and
// sub-agents build their own Invocation so they never reach this
// emitter. Same single-goroutine contract as the audit Writer and the
// critic ProgressRecorder — no internal lock.
type stepEmitter struct {
onStep func(ctx context.Context, ev tool.StepEvent)
now func() time.Time
seq int
steps []tool.Step // ordered; the snapshot for Result.Steps
byID map[string]int // step id -> index into steps
pending map[string][]string // correlation key -> queued running ids (FIFO)
}
// newStepEmitter returns an emitter that forwards to onStep (nil-safe).
func newStepEmitter(onStep func(ctx context.Context, ev tool.StepEvent)) *stepEmitter {
return &stepEmitter{
onStep: onStep,
now: time.Now,
byID: map[string]int{},
pending: map[string][]string{},
}
}
// corrKey correlates a "start" (name + raw args, no call id available at
// the pre-dispatch hook) with its later "end" (the stepObserver has the
// full call incl. id + the same raw args).
func corrKey(name string, args json.RawMessage) string {
return name + "\x00" + string(args)
}
// toolStart records + emits the "start" of a tool call. Called from the
// pre-dispatch hookToolbox closure, before the tool runs.
func (e *stepEmitter) toolStart(ctx context.Context, name string, args json.RawMessage) {
if e == nil {
return
}
step := e.newStep(name, args)
key := corrKey(name, args)
e.pending[key] = append(e.pending[key], step.ID)
e.fire(ctx, "start", step)
}
// toolEnd records + emits the terminal "end" of a tool call. Called from
// the stepObserver for each completed tool call. If no matching start was
// seen (e.g. a tool with a nil handler the pre-dispatch hook skipped), a
// start is synthesized so the step still appears.
func (e *stepEmitter) toolEnd(ctx context.Context, call llm.ToolCall, result string, isError bool) {
if e == nil {
return
}
id := e.matchPending(call.Name, call.Arguments)
if id == "" {
id = e.newStep(call.Name, call.Arguments).ID
}
idx, ok := e.byID[id]
if !ok {
return
}
step := &e.steps[idx]
end := e.now()
step.EndedAt = &end
if isError {
step.Status = "error"
} else {
step.Status = "complete"
}
if s := summaryForEnd(call.Name, call.Arguments, result, isError); s != "" {
step.Summary = s
}
e.fire(ctx, "end", *step)
}
// newStep mints + appends a running step and returns it (by value).
func (e *stepEmitter) newStep(name string, args json.RawMessage) tool.Step {
e.seq++
step := tool.Step{
ID: fmt.Sprintf("s%d", e.seq),
Kind: kindForTool(name),
Title: name,
Summary: summaryForStart(name, args),
Status: "running",
StartedAt: e.now(),
}
e.byID[step.ID] = len(e.steps)
e.steps = append(e.steps, step)
return step
}
// matchPending pops the oldest running step id for (name, args). Falls back to
// the OLDEST still-running step of the same tool name when the args don't
// byte-match between start and end (e.g. JSON key reordering). FIFO on the
// fallback too, consistent with the per-key queue pop above — closing the
// oldest avoids mis-correlating concurrent same-named calls. Returns "" on no
// match.
func (e *stepEmitter) matchPending(name string, args json.RawMessage) string {
key := corrKey(name, args)
if q := e.pending[key]; len(q) > 0 {
id := q[0]
if len(q) == 1 {
delete(e.pending, key)
} else {
e.pending[key] = q[1:]
}
return id
}
for i := 0; i < len(e.steps); i++ {
if e.steps[i].Title == name && e.steps[i].Status == "running" {
return e.steps[i].ID
}
}
return ""
}
func (e *stepEmitter) fire(ctx context.Context, phase string, step tool.Step) {
if e.onStep == nil {
return
}
e.onStep(ctx, tool.StepEvent{Phase: phase, Step: step})
}
// snapshot returns a copy of the ordered, deduplicated step set for the
// run Result. A step still "running" at run end (e.g. the run was
// cancelled mid-tool-call) is reported as-is.
func (e *stepEmitter) snapshot() []tool.Step {
if e == nil || len(e.steps) == 0 {
return nil
}
out := make([]tool.Step, len(e.steps))
copy(out, e.steps)
return out
}
// kindForTool maps a tool name to an open-vocabulary step kind. Unknown
// tools fall back to "tool" — never an error, just a generic step (the
// client maps unknown kinds to a default icon). Loosely tracks the
// catalog in pkg/skilltools/CLAUDE.md.
func kindForTool(name string) string {
switch name {
case "web_search", "search_reddit", "wikipedia_summary":
return "search"
case "read_page", "read_pdf", "read_reddit", "read_video", "verify_url",
"summary_summarise", "summarize", "file_get_text", "file_get_metadata",
"http_get", "http_post", "http_get_stream", "http_stream_read":
return "read"
case "code_exec", "calculate":
return "code"
case "file_save", "file_get", "file_list", "file_delete", "file_search":
return "file"
case "kv_get", "kv_set", "kv_list", "kv_delete",
"remember", "recall", "chatbot_get_memories":
return "memory"
case "query", "query_research", "deepresearch", "animate",
"agent_invoke", "agent_invoke_parallel", "agent_spawn",
"agent_spawn_parallel", "skill_invoke", "skill_invoke_parallel":
return "delegate"
case "think":
return "thinking"
default:
switch {
case strings.HasPrefix(name, "image") || strings.Contains(name, "draw"):
return "image"
default:
return "tool"
}
}
}
// summaryForStart builds the human present-tense running summary. It
// derives specifics from safe arg fields only; secret-bearing tools
// (mcp_call, email_send, http_*) are summarized without echoing args.
func summaryForStart(name string, args json.RawMessage) string {
var s string
switch name {
case "web_search":
if q := argString(args, "query", "q"); q != "" {
s = fmt.Sprintf("Searching the web for %q", q)
} else {
s = "Searching the web"
}
case "search_reddit":
if q := argString(args, "query", "q"); q != "" {
s = fmt.Sprintf("Searching Reddit for %q", q)
} else {
s = "Searching Reddit"
}
case "wikipedia_summary":
if q := argString(args, "query", "title"); q != "" {
s = fmt.Sprintf("Looking up %q on Wikipedia", q)
} else {
s = "Looking up Wikipedia"
}
case "read_page", "read_pdf", "read_reddit", "read_video", "verify_url":
if u := argString(args, "url", "post", "page"); u != "" {
s = "Reading " + hostOf(u)
} else {
s = "Reading a page"
}
case "http_get", "http_post", "http_get_stream":
// Show host only — a full URL can embed credentials/tokens.
if u := argString(args, "url"); u != "" {
s = "Fetching " + hostOf(u)
} else {
s = "Making an HTTP request"
}
case "summary_summarise", "summarize":
s = "Summarizing text"
case "translate":
if lang := argString(args, "target_lang", "target_language", "lang"); lang != "" {
s = "Translating to " + lang
} else {
s = "Translating text"
}
case "code_exec":
s = "Running code"
case "calculate":
if q := argString(args, "query", "expression", "expr"); q != "" {
s = "Calculating " + truncateStep(q, 60)
} else {
s = "Calculating"
}
case "remember":
// Never echo the stored value.
s = "Saving a memory"
case "recall", "chatbot_get_memories":
s = "Recalling memories"
case "kv_get", "kv_list":
s = "Reading saved data"
case "kv_set":
s = "Saving data"
case "kv_delete":
s = "Deleting saved data"
case "file_save":
if n := argString(args, "name", "filename"); n != "" {
s = "Saving file " + truncateStep(n, 60)
} else {
s = "Saving a file"
}
case "file_get", "file_get_text", "file_get_metadata":
s = "Reading a file"
case "file_list", "file_search":
s = "Listing files"
case "query", "query_research":
if q := argString(args, "query", "question", "prompt", "task"); q != "" {
s = "Researching " + truncateStep(q, 80)
} else {
s = "Researching"
}
case "deepresearch":
s = "Running deep research"
case "animate":
s = "Generating an animation"
case "agent_invoke", "agent_spawn":
if a := argString(args, "agent", "agent_name", "name"); a != "" {
s = "Delegating to " + a
} else {
s = "Delegating to a sub-agent"
}
case "agent_invoke_parallel", "agent_spawn_parallel":
s = "Delegating to sub-agents"
case "skill_invoke":
if sk := argString(args, "skill_name", "skill", "name"); sk != "" {
s = "Running skill " + sk
} else {
s = "Running a skill"
}
case "skill_invoke_parallel":
s = "Running skills in parallel"
case "think":
s = "Thinking"
case "mcp_call":
// Redact: MCP args frequently carry secrets. Name server/tool only.
srv, tl := argString(args, "server"), argString(args, "tool")
switch {
case srv != "" && tl != "":
s = fmt.Sprintf("Calling %s/%s", srv, tl)
case srv != "":
s = "Calling " + srv
default:
s = "Calling an MCP tool"
}
case "email_send":
// Redact recipients + body.
s = "Sending an email"
default:
s = "Using " + name
}
return truncateStep(s, stepSummaryMaxLen)
}
// summaryForEnd optionally upgrades the summary to a cheap result phrase.
// Returns "" to keep the running summary (the caller then just flips the
// status). Never returns a phrase derived from raw result bytes.
func summaryForEnd(name string, _ json.RawMessage, result string, isError bool) string {
if isError {
return ""
}
switch name {
case "web_search", "search_reddit":
if n := countResults(result); n >= 0 {
return fmt.Sprintf("Found %d result%s", n, plural(n))
}
}
return ""
}
// argString pulls the first present non-empty string field from a tool's
// raw JSON args, trying keys in order. Returns "" when none parse.
func argString(args json.RawMessage, keys ...string) string {
if len(args) == 0 {
return ""
}
var m map[string]any
if err := json.Unmarshal(args, &m); err != nil {
return ""
}
for _, k := range keys {
if v, ok := m[k]; ok {
if s, ok := v.(string); ok && strings.TrimSpace(s) != "" {
return strings.TrimSpace(s)
}
}
}
return ""
}
// countResults parses a v11-style {"results":[...]} envelope and returns
// the count, or -1 when the shape doesn't match.
func countResults(result string) int {
if strings.TrimSpace(result) == "" {
return -1
}
var env struct {
Results []json.RawMessage `json:"results"`
}
if err := json.Unmarshal([]byte(result), &env); err != nil || env.Results == nil {
return -1
}
return len(env.Results)
}
// hostOf returns the bare host (no leading www.) of a URL, or a short
// form of the raw string when it doesn't parse as a URL.
func hostOf(raw string) string {
if u, err := url.Parse(raw); err == nil && u.Host != "" {
return strings.TrimPrefix(u.Host, "www.")
}
return truncateStep(raw, 60)
}
// truncateStep rune-safely caps s to max, appending an ellipsis when cut.
func truncateStep(s string, max int) string {
if max <= 0 {
return ""
}
r := []rune(s)
if len(r) <= max {
return s
}
if max == 1 {
return string(r[:1])
}
return string(r[:max-1]) + "…"
}
func plural(n int) string {
if n == 1 {
return ""
}
return "s"
}
+84
View File
@@ -0,0 +1,84 @@
package run
import (
"context"
"strings"
"sync"
llm "gitea.stevedudenhoeffer.com/steve/majordomo/llm"
)
// SubmitCapture records the output a run's `submit` tool received.
//
// Why this exists: legacy agentkit injected a synthetic `submit` tool and
// ended the loop when it fired; years of mort system prompts (agent
// YAMLs, skill manifests, the executors' platform headers) teach the
// model to "call submit with your final answer". majordomo's agent loop
// has no submit concept — it ends when the model replies WITHOUT tool
// calls. Dropping submit cold would make every prompt-trained model
// burn turns on "unknown tool \"submit\"" errors.
//
// The compatibility shape: the executors add NewSubmitTool's tool to
// every run's toolset (unless the palette already defines a `submit`).
// The handler records the FIRST submitted answer and tells the model
// the answer was accepted so its next turn is a bare reply (which ends
// the loop naturally). After the run, the executor consults
// Output(loopOutput, runErr): a captured submission wins over an empty
// or budget-exhausted ending, so a model that submits on its final
// allowed step still produces its answer instead of ErrMaxSteps.
type SubmitCapture struct {
mu sync.Mutex
output string
called bool
}
// Record stores the first submitted answer; later calls are ignored
// (matching legacy agentkit's "multiple calls keep the first" contract).
func (c *SubmitCapture) Record(output string) {
c.mu.Lock()
defer c.mu.Unlock()
if c.called {
return
}
c.called = true
c.output = output
}
// Submitted returns the captured answer and whether submit fired.
func (c *SubmitCapture) Submitted() (string, bool) {
c.mu.Lock()
defer c.mu.Unlock()
return c.output, c.called
}
// Output resolves the run's final output: the submitted answer when the
// model called submit (parity with legacy agentkit, where submit's argument
// WAS the run output), otherwise the loop's own final text. resolvedErr
// is nil when a submission exists — a run that submitted its answer and
// then ran out of steps (or timed out composing the courtesy
// confirmation turn) is a SUCCESS, not an error.
func (c *SubmitCapture) Output(loopOutput string, runErr error) (output string, resolvedErr error) {
if out, ok := c.Submitted(); ok {
return out, nil
}
return loopOutput, runErr
}
// submitArgs mirrors legacy agentkit's synthetic submit tool schema so
// models prompted under the old contract emit compatible calls.
type submitArgs struct {
Output string `json:"output" description:"The final answer, summary, or output for this task."`
}
// NewSubmitTool builds the compatibility `submit` tool bound to the
// given capture. Both executors (skill + agent) install one per run.
func NewSubmitTool(capture *SubmitCapture) llm.Tool {
return llm.DefineTool[submitArgs](
"submit",
"Submit your final answer or output to end this task. Call exactly once when you are done.",
func(_ context.Context, args submitArgs) (any, error) {
capture.Record(strings.TrimSpace(args.Output))
return "Final answer recorded. Do not call any more tools; reply now with a brief closing message.", nil
},
)
}
+132
View File
@@ -0,0 +1,132 @@
// Package schedule is the cron-runner battery: a generic ticker that, each
// interval, asks a store for the jobs whose next-run time has passed, runs each
// one, and stamps its next fire time. It is host-agnostic orchestration — the
// host wires the store (skill.SkillStore.ListDueScheduled /
// persona.Storage.ListScheduledAgents), the run (run.Executor), and the cron
// "next fire" function (a cron library, or skill's schedule parser). The
// battery owns no cron grammar of its own, so it never duplicates the parser.
package schedule
import (
"context"
"errors"
"log/slog"
"time"
)
// Due is one schedulable job: its id and its cron expression.
type Due struct {
ID string
Cron string
}
// Runner periodically fires due jobs. Every func field is required except Now
// (defaults to time.Now) and Logger (defaults to slog.Default). Construct the
// struct directly and call Loop (or Tick for a single pass / tests).
type Runner struct {
// Interval is how often Loop checks for due jobs. <= 0 defaults to 1m.
Interval time.Duration
// Due lists the jobs due at now.
Due func(ctx context.Context, now time.Time) ([]Due, error)
// Run executes one job by id.
Run func(ctx context.Context, id string) error
// Mark records that a job ran at ranAt and is next due at nextAt.
Mark func(ctx context.Context, id string, ranAt, nextAt time.Time) error
// Next computes a cron expression's next fire after a given time.
Next func(cron string, after time.Time) (time.Time, error)
Now func() time.Time
Logger *slog.Logger
}
func (r *Runner) now() time.Time {
if r.Now != nil {
return r.Now()
}
return time.Now()
}
func (r *Runner) log() *slog.Logger {
if r.Logger != nil {
return r.Logger
}
return slog.Default()
}
// Tick runs one pass: every currently-due job is run, then stamped with its
// next fire time. A job whose Run or Next errors is logged and skipped (its
// next-run time is left unchanged so it stays due and retries next tick) — one
// bad job never stalls the others. Returns the error from Due (the only
// pass-fatal step).
func (r *Runner) Tick(ctx context.Context) error {
if err := r.validate(); err != nil {
return err
}
now := r.now()
due, err := r.Due(ctx, now)
if err != nil {
return err
}
for _, j := range due {
// Compute the next fire BEFORE running. A permanently-unparseable cron
// then skips the job entirely (logged) rather than running it — an
// unstamped job stays due, so checking Next first avoids a hot-loop of
// real Run executions every tick.
next, err := r.Next(j.Cron, now)
if err != nil {
r.log().Warn("scheduled job has an unparseable cron; skipping (not run, not rescheduled)", "job", j.ID, "cron", j.Cron, "error", err)
continue
}
if err := r.Run(ctx, j.ID); err != nil {
r.log().Warn("scheduled job failed; stays due, will retry next tick", "job", j.ID, "error", err)
continue
}
// A Mark failure leaves the job due, so it re-runs next tick — Run must
// be idempotent (there is no atomic run+stamp across two host callbacks).
if err := r.Mark(ctx, j.ID, now, next); err != nil {
r.log().Warn("failed to stamp next run; job may re-execute next tick (Run must be idempotent)", "job", j.ID, "error", err)
}
}
return nil
}
// validate reports a misconfigured Runner (a required callback left nil) as a
// clear error rather than a nil-deref panic on first tick.
func (r *Runner) validate() error {
if r.Due == nil || r.Run == nil || r.Mark == nil || r.Next == nil {
return errors.New("schedule: Runner requires non-nil Due, Run, Mark, and Next")
}
return nil
}
// Loop ticks every Interval until ctx is cancelled. A Tick error (the Due
// lister failing) is logged and the loop continues — a transient store hiccup
// shouldn't kill the scheduler — and a panic from any host callback is
// recovered so one bad tick can't silently kill the scheduler goroutine.
func (r *Runner) Loop(ctx context.Context) {
interval := r.Interval
if interval <= 0 {
interval = time.Minute
}
t := time.NewTicker(interval)
defer t.Stop()
for {
select {
case <-ctx.Done():
return
case <-t.C:
r.safeTick(ctx)
}
}
}
func (r *Runner) safeTick(ctx context.Context) {
defer func() {
if rec := recover(); rec != nil {
r.log().Error("schedule tick panicked; scheduler continues", "panic", rec)
}
}()
if err := r.Tick(ctx); err != nil {
r.log().Warn("schedule tick failed", "error", err)
}
}
+111
View File
@@ -0,0 +1,111 @@
package schedule
import (
"context"
"errors"
"testing"
"time"
)
func TestTickRunsDueAndStampsNext(t *testing.T) {
ctx := context.Background()
now := time.Date(2026, 1, 1, 12, 0, 0, 0, time.UTC)
var ran []string
marked := map[string]time.Time{}
r := &Runner{
Now: func() time.Time { return now },
Due: func(_ context.Context, _ time.Time) ([]Due, error) {
return []Due{{ID: "a", Cron: "hourly"}, {ID: "b", Cron: "bad"}}, nil
},
Run: func(_ context.Context, id string) error { ran = append(ran, id); return nil },
Mark: func(_ context.Context, id string, _, next time.Time) error { marked[id] = next; return nil },
Next: func(cron string, after time.Time) (time.Time, error) {
if cron == "bad" {
return time.Time{}, errors.New("unparseable")
}
return after.Add(time.Hour), nil
},
}
if err := r.Tick(ctx); err != nil {
t.Fatal(err)
}
// Next is checked first, so the bad-cron job is skipped BEFORE Run — only
// the parseable job runs and gets stamped (no hot-loop of a bad-cron Run).
if len(ran) != 1 || ran[0] != "a" {
t.Errorf("ran = %v, want only [a] (bad-cron b skipped before Run)", ran)
}
if marked["a"] != now.Add(time.Hour) {
t.Errorf("a next = %v, want +1h", marked["a"])
}
if _, ok := marked["b"]; ok {
t.Errorf("b should not be stamped (bad cron), got %v", marked["b"])
}
}
func TestTickRunFailureDoesNotStampOrStall(t *testing.T) {
ctx := context.Background()
var ran []string
marked := map[string]bool{}
r := &Runner{
Due: func(_ context.Context, _ time.Time) ([]Due, error) {
return []Due{{ID: "x", Cron: "h"}, {ID: "y", Cron: "h"}}, nil
},
Run: func(_ context.Context, id string) error {
ran = append(ran, id)
if id == "x" {
return errors.New("boom")
}
return nil
},
Mark: func(_ context.Context, id string, _, _ time.Time) error { marked[id] = true; return nil },
Next: func(string, time.Time) (time.Time, error) { return time.Now(), nil },
}
if err := r.Tick(ctx); err != nil {
t.Fatal(err)
}
if len(ran) != 2 { // y still runs despite x failing
t.Errorf("ran = %v, want both attempted", ran)
}
if marked["x"] { // failed job NOT stamped -> stays due, retries
t.Error("failed job x should not be stamped")
}
if !marked["y"] {
t.Error("y should be stamped")
}
}
func TestTickDueErrorIsFatalToPass(t *testing.T) {
r := &Runner{
Due: func(context.Context, time.Time) ([]Due, error) { return nil, errors.New("store down") },
Run: func(context.Context, string) error { return nil },
Mark: func(context.Context, string, time.Time, time.Time) error { return nil },
Next: func(string, time.Time) (time.Time, error) { return time.Now(), nil },
}
if err := r.Tick(context.Background()); err == nil {
t.Error("Tick should surface the Due lister error")
}
}
func TestUnparseableCronSkipsRunEntirely(t *testing.T) {
var ran []string
r := &Runner{
Due: func(context.Context, time.Time) ([]Due, error) { return []Due{{ID: "z", Cron: "bad"}}, nil },
Run: func(_ context.Context, id string) error { ran = append(ran, id); return nil },
Mark: func(context.Context, string, time.Time, time.Time) error { return nil },
Next: func(string, time.Time) (time.Time, error) { return time.Time{}, errors.New("bad cron") },
}
if err := r.Tick(context.Background()); err != nil {
t.Fatal(err)
}
if len(ran) != 0 {
t.Errorf("a job with an unparseable cron must NOT be run (avoids hot-loop), ran=%v", ran)
}
}
func TestValidateRejectsNilCallbacks(t *testing.T) {
r := &Runner{Due: func(context.Context, time.Time) ([]Due, error) { return nil, nil }} // missing Run/Mark/Next
if err := r.Tick(context.Background()); err == nil {
t.Error("Tick should return a validation error for a partially-wired Runner, not panic")
}
}
+6
View File
@@ -0,0 +1,6 @@
package skill
// DefaultChatbotInputName is the input-param name a chatbot-exposed skill
// receives the user's message under when its schema doesn't name one. Moved
// from mort's chatbot_provider.go (a host concern) as a host-agnostic default.
const DefaultChatbotInputName = "request"
+422
View File
@@ -0,0 +1,422 @@
package skill
import (
"fmt"
"strings"
)
// This file holds the shared input-parsing primitives used by both the
// chatbot exposure adapter (chatbot_provider.go) and the .skill Discord
// command handler (commands.go) to construct a SkillInputs map from
// caller-supplied raw values. Centralising here avoids the two paths
// drifting in their type-coercion or required-check semantics.
//
// Two layers:
//
// - CoerceInputValue: per-param-type coercion (int/float/bool/string).
// Accepts loosely-typed values (LLM-stringified numbers, JSON
// float64s for ints) and returns a value in the target Go shape.
//
// - CoerceInputs: per-skill validation. Walks the InputSchema, coerces
// each declared param via CoerceInputValue, drops extras silently,
// errors on missing required.
//
// Why exported (capital): both consumers live in the same package, but
// the names are also referenced in test files and the symbols are
// genuinely useful API for any future consumer (webui form handler,
// scheduler in v2). Keep the surface small.
// CoerceInputValue coerces a single raw value to the target InputParam
// type. JSON numbers arrive from json.Unmarshal as float64; bools as
// bool; strings as string. Type-mismatched strings are accepted ("3" →
// int 3, "true" → bool true) because both LLM tool calls and Discord
// command args frequently surface scalars as strings.
//
// Why: LLM tool-call args come through json.Unmarshal of a plain
// map[string]any, which forces every JSON number into float64 and every
// JSON string into string. Without this coerce step, an int parameter
// would arrive in SkillInputs as a float64, a bool sent as "true" would
// arrive as a string, etc. — confusing the skill agent's prompt
// renderer and any tool-side logic that switches on Go type. The
// .skill command handler benefits identically: arg tokens arrive as
// strings, but downstream tools may expect typed values.
//
// Test: TestCoerceInputValue in inputs_test.go covers each branch.
func CoerceInputValue(paramType string, v any) (any, error) {
switch paramType {
case "int":
switch x := v.(type) {
case float64:
return int(x), nil
case int:
return x, nil
case string:
var i int
if _, err := fmt.Sscanf(x, "%d", &i); err != nil {
return nil, fmt.Errorf("not an int: %q", x)
}
return i, nil
default:
return nil, fmt.Errorf("not an int: %T", v)
}
case "float":
switch x := v.(type) {
case float64:
return x, nil
case int:
return float64(x), nil
case string:
var f float64
if _, err := fmt.Sscanf(x, "%f", &f); err != nil {
return nil, fmt.Errorf("not a float: %q", x)
}
return f, nil
default:
return nil, fmt.Errorf("not a float: %T", v)
}
case "bool":
switch x := v.(type) {
case bool:
return x, nil
case string:
switch x {
case "true", "True", "TRUE", "1":
return true, nil
case "false", "False", "FALSE", "0":
return false, nil
default:
return nil, fmt.Errorf("not a bool: %q", x)
}
default:
return nil, fmt.Errorf("not a bool: %T", v)
}
default:
// "string", "user", "channel", "url", and unknown — coerce to
// string. JSON numbers/bools are stringified so the executor's
// validateInputs (which strips e.g. <@!123> wrappers) gets a
// uniform string input.
switch x := v.(type) {
case string:
return x, nil
case float64:
return fmt.Sprintf("%v", x), nil
case bool:
return fmt.Sprintf("%v", x), nil
default:
return fmt.Sprintf("%v", v), nil
}
}
}
// CoerceInputs validates and coerces a map of raw caller-supplied values
// against the declared parameter set:
//
// - Extra keys (not in params) are dropped silently.
// - Missing required keys return an error so the caller can surface
// usage information.
// - Per-param type coercion handles int/float/bool sent as strings.
//
// Returns a fresh map containing only declared params; never mutates the
// input map.
//
// Why: see CoerceInputValue. Both callers (chatbot exposure adapter,
// .skill command handler) need the same required-check + extra-drop
// semantics; previously only the chatbot path implemented them, which
// is exactly why .skill <name> <args> dropped its arguments entirely.
//
// Test: TestCoerceInputs in inputs_test.go.
func CoerceInputs(params []InputParam, raw map[string]any) (map[string]any, error) {
out := make(map[string]any, len(params))
for _, p := range params {
v, present := raw[p.Name]
if !present {
if p.Required {
return nil, fmt.Errorf("missing required parameter %q", p.Name)
}
continue
}
typed, err := CoerceInputValue(p.Type, v)
if err != nil {
return nil, fmt.Errorf("parameter %q: %w", p.Name, err)
}
out[p.Name] = typed
}
return out, nil
}
// ParseCommandInputs parses a free-form command argument string into a
// raw map[string]any keyed by InputSchema parameter names. Three modes
// are supported, picked by the shape of `schema`:
//
// CASE A — empty schema:
// The whole string becomes {"request": "<rest>"}. Mirrors the
// chatbot exposure default (DefaultChatbotInputName) so a skill with
// no declared inputs can still receive its trigger text uniformly
// across both surfaces.
//
// CASE B — exactly one required param (with optional non-required
// tail):
// If the user passed any --key=value or --key value flags they're
// parsed as flags (Case C). Otherwise the WHOLE rest-of-message
// becomes that single required param's value. This is the
// "single-arg convenience" pattern that lets `.skill weather Boston
// today` work without the user typing --city=.
//
// CASE C — multiple params, OR any --flag style input:
// Tokens are parsed as `--name=value` or `--name value`. Bare
// positional tokens after a flag are collected as that flag's value.
// Trailing positional tokens with no preceding flag are dropped
// (the caller's usage string should mention the flag form).
//
// The returned map values are RAW strings (or bool true for
// presence-only flags); type coercion is the caller's job via
// CoerceInputs.
//
// Why this signature instead of returning the typed map directly: the
// caller wants to distinguish "missing required" (→ usage reply) from
// "type coercion failed" (→ explicit error). Splitting parse from
// coerce keeps the message specific.
func ParseCommandInputs(schema []InputParam, raw string) map[string]any {
out := map[string]any{}
raw = strings.TrimSpace(raw)
if raw == "" {
return out
}
// Detect flag-style input regardless of schema shape — even a single
// required-param schema may be invoked via `.skill x --name value`
// for forward compat.
hasFlag := strings.Contains(raw, "--")
switch {
case len(schema) == 0:
// Empty schema: mirror the chatbot exposure adapter's default
// "request" pseudo-param so executor.composePrompt can render
// it uniformly.
out[DefaultChatbotInputName] = raw
case !hasFlag && countRequired(schema) == 1:
// Single-required-param convenience: whole rest-of-message is the
// value, regardless of any other (non-required) params declared.
// They can be supplied via --flag form if needed.
req := firstRequired(schema)
out[req.Name] = raw
default:
// Flag-style parse. Walk tokens looking for --name[=value] or
// --name <value>.
parseFlagStyle(out, schema, raw)
}
return out
}
// countRequired returns the number of params marked Required.
func countRequired(schema []InputParam) int {
n := 0
for _, p := range schema {
if p.Required {
n++
}
}
return n
}
// firstRequired returns the first required param. Caller must have
// already verified at least one exists.
func firstRequired(schema []InputParam) *InputParam {
for i := range schema {
if schema[i].Required {
return &schema[i]
}
}
return nil
}
// parseFlagStyle walks tokens for --name=value and --name value forms.
// Unknown flags (not in schema) are still accepted into the output map
// so the caller can detect and warn about them; CoerceInputs will drop
// extras when constructing the final SkillInputs.
//
// Tokens not preceded by a --flag are dropped. v1 is intentionally
// strict-ish here: we don't try to guess which positional token belongs
// to which param when there are several. The single-required-param
// convenience handles the common ambiguity-free case in the caller.
func parseFlagStyle(out map[string]any, schema []InputParam, raw string) {
tokens := tokeniseCommandLine(raw)
declared := map[string]bool{}
for _, p := range schema {
declared[p.Name] = true
}
i := 0
for i < len(tokens) {
t := tokens[i]
if !strings.HasPrefix(t, "--") {
// Bare positional token outside a flag context — drop. The
// caller's usage string should steer users to flag form.
i++
continue
}
key := t[2:]
// --name=value form
if eq := strings.IndexByte(key, '='); eq >= 0 {
out[key[:eq]] = key[eq+1:]
i++
continue
}
// --name <value> form: take the next token IF it doesn't itself
// start with --. Otherwise treat as a presence-only boolean flag.
if i+1 < len(tokens) && !strings.HasPrefix(tokens[i+1], "--") {
out[key] = tokens[i+1]
i += 2
continue
}
out[key] = "true"
i++
}
_ = declared // reserved for v2 unknown-flag warnings
}
// tokeniseCommandLine splits a free-form Discord command argument
// string into tokens. Quoted spans (single or double quotes) are kept
// as one token so users can pass values with spaces:
//
// .skill weather --city="New York"
// .skill summarise --text 'a long sentence here'
//
// Mirrors the user's intuition without introducing a full shell
// parser. Newlines split as whitespace.
func tokeniseCommandLine(s string) []string {
var out []string
var cur strings.Builder
var quote rune
flush := func() {
if cur.Len() > 0 {
out = append(out, cur.String())
cur.Reset()
}
}
for _, r := range s {
switch {
case quote != 0:
if r == quote {
quote = 0
continue
}
cur.WriteRune(r)
case r == '"' || r == '\'':
quote = r
case r == ' ' || r == '\t' || r == '\n':
flush()
default:
cur.WriteRune(r)
}
}
flush()
return out
}
// ResolveCommandInputs is the one-call helper a Discord .skill handler
// uses to turn a free-form rest-of-message into a coerced
// SkillInputs map ready to hand to the executor. It is the single
// production entry point for command-side input resolution: every
// caller must use it (do NOT chain ParseCommandInputs + CoerceInputs
// directly).
//
// Why this exists as a single function: chaining
// ParseCommandInputs + CoerceInputs at the call site is what broke
// `.skill echo hello world` in production. ParseCommandInputs Case A
// (empty schema) writes the user's text into out["request"], but
// CoerceInputs(emptySchema, …) iterates the DECLARED params and
// silently drops every key not in the schema — so "request" is
// dropped before reaching the executor, and the agent's user-prompt
// renders "(no input provided)". The fix is to mirror the chatbot
// exposure adapter: derive the EFFECTIVE param set (which inflates
// an empty schema to a single required "request" param) and coerce
// against that, not the original empty schema.
//
// What:
// - Empty input_schema → effective params = [{request, required, string}],
// so ParseCommandInputs Case A's "request" key survives Coerce.
// - Non-empty input_schema → effective params = the schema as-is, so
// Case B / Case C parse-and-coerce semantics are unchanged.
//
// Returns the coerced SkillInputs map, or an error suitable for
// surfacing to the user (e.g. via FormatUsage). Never mutates
// `schema`.
//
// Test: TestResolveCommandInputs_* in inputs_test.go cover the three
// cases plus the empty-schema regression.
func ResolveCommandInputs(schema []InputParam, raw string) (map[string]any, error) {
rawInputs := ParseCommandInputs(schema, raw)
effective := effectiveCommandParams(schema)
return CoerceInputs(effective, rawInputs)
}
// effectiveCommandParams returns the parameter set the .skill command
// path should use for coercion. Mirrors chatbotToolParams in
// chatbot_provider.go: an empty input_schema is inflated to a single
// required "request" string param so the user's free-text trigger
// survives CoerceInputs's drop-extras semantics.
//
// Why a separate helper (vs reusing chatbotToolParams): keeping the
// helper local to inputs.go avoids dragging chatbot_provider.go into
// the .skill command path's import surface and makes the intent
// (Discord-side parameter inflation) explicit at the call site.
func effectiveCommandParams(schema []InputParam) []InputParam {
if len(schema) > 0 {
return schema
}
return []InputParam{{
Name: DefaultChatbotInputName,
Description: "The user's free-text trigger.",
Type: "string",
Required: true,
}}
}
// FormatUsage renders a human-readable usage string for the .skill
// invocation form. Used by command handlers when required params are
// missing or coercion fails.
//
// Why: keep the usage message in one place so both the missing-required
// and coercion-failed paths produce identical output.
func FormatUsage(name string, schema []InputParam) string {
var sb strings.Builder
fmt.Fprintf(&sb, "usage: `.skill %s", name)
if len(schema) == 0 {
sb.WriteString(" <text>`")
return sb.String()
}
if countRequired(schema) == 1 {
req := firstRequired(schema)
fmt.Fprintf(&sb, " <%s>`", req.Name)
// Show optional flags (if any).
var optional []InputParam
for _, p := range schema {
if !p.Required {
optional = append(optional, p)
}
}
if len(optional) > 0 {
sb.WriteString("\n optional:")
for _, p := range optional {
fmt.Fprintf(&sb, " --%s=<%s>", p.Name, p.Type)
}
}
return sb.String()
}
// Multi-param: full --flag form.
for _, p := range schema {
if p.Required {
fmt.Fprintf(&sb, " --%s=<%s>", p.Name, p.Type)
}
}
for _, p := range schema {
if !p.Required {
fmt.Fprintf(&sb, " [--%s=<%s>]", p.Name, p.Type)
}
}
sb.WriteString("`")
return sb.String()
}
+169
View File
@@ -0,0 +1,169 @@
package skill
import (
"context"
"sort"
"sync"
"time"
)
// Memory is a zero-dependency in-process SkillStore — a light host or test gets
// saved-skill persistence with no DB. Mort backs SkillStore with GORM/MySQL;
// contrib/store adds durable SQLite.
type Memory struct {
mu sync.RWMutex
skills map[string]*Skill // by ID
versions map[string][]SkillVersion // by skill ID, append order
byVerID map[string]SkillVersion // by version ID
}
// NewMemory returns an empty in-memory SkillStore.
func NewMemory() *Memory {
return &Memory{
skills: map[string]*Skill{},
versions: map[string][]SkillVersion{},
byVerID: map[string]SkillVersion{},
}
}
var _ SkillStore = (*Memory)(nil)
func (m *Memory) Initialize(context.Context) error { return nil }
func (m *Memory) Save(_ context.Context, s *Skill) error {
m.mu.Lock()
defer m.mu.Unlock()
cp := *s
m.skills[s.ID] = &cp
return nil
}
func (m *Memory) Get(_ context.Context, id string) (*Skill, error) {
m.mu.RLock()
defer m.mu.RUnlock()
s, ok := m.skills[id]
if !ok {
return nil, ErrNotFound
}
cp := *s
return &cp, nil
}
func (m *Memory) GetByName(_ context.Context, ownerID, name string) (*Skill, error) {
m.mu.RLock()
defer m.mu.RUnlock()
for _, s := range m.skills {
if s.OwnerID == ownerID && s.Name == name {
cp := *s
return &cp, nil
}
}
return nil, ErrNotFound
}
func (m *Memory) Delete(_ context.Context, id string) error {
m.mu.Lock()
defer m.mu.Unlock()
delete(m.skills, id)
return nil
}
func (m *Memory) listWhere(keep func(*Skill) bool) []Skill {
m.mu.RLock()
defer m.mu.RUnlock()
out := make([]Skill, 0, len(m.skills))
for _, s := range m.skills {
if keep == nil || keep(s) {
out = append(out, *s)
}
}
sort.Slice(out, func(i, j int) bool { return out[i].Name < out[j].Name })
return out
}
func (m *Memory) ListByOwner(_ context.Context, ownerID string) ([]Skill, error) {
return m.listWhere(func(s *Skill) bool { return s.OwnerID == ownerID }), nil
}
func (m *Memory) ListPublic(context.Context) ([]Skill, error) {
return m.listWhere(func(s *Skill) bool { return s.Visibility == VisibilityPublic }), nil
}
func (m *Memory) ListSharedWith(_ context.Context, memberID string) ([]Skill, error) {
return m.listWhere(func(s *Skill) bool {
if s.Visibility != VisibilityShared {
return false
}
for _, id := range s.SharedWith {
if id == memberID {
return true
}
}
return false
}), nil
}
func (m *Memory) ListBuiltinByName(_ context.Context, name string) (*Skill, error) {
m.mu.RLock()
defer m.mu.RUnlock()
for _, s := range m.skills {
if s.Source == SourceBuiltin && s.Name == name {
cp := *s
return &cp, nil
}
}
return nil, ErrNotFound
}
func (m *Memory) ListChatbotExposed(context.Context) ([]Skill, error) {
return m.listWhere(func(s *Skill) bool { return s.ExposeAsChatbotTool }), nil
}
func (m *Memory) ListDueScheduled(_ context.Context, now time.Time) ([]Skill, error) {
return m.listWhere(func(s *Skill) bool { return s.DueAt(now) }), nil
}
func (m *Memory) MarkScheduledRun(_ context.Context, skillID string, ranAt, nextAt time.Time) error {
m.mu.Lock()
defer m.mu.Unlock()
s, ok := m.skills[skillID]
if !ok {
return ErrNotFound
}
s.LastScheduledRunAt = ranAt
s.NextRunAt = nextAt
return nil
}
func (m *Memory) AppendVersion(_ context.Context, sv SkillVersion) error {
m.mu.Lock()
defer m.mu.Unlock()
m.versions[sv.SkillID] = append(m.versions[sv.SkillID], sv)
m.byVerID[sv.ID] = sv
return nil
}
func (m *Memory) ListVersionsBySkill(_ context.Context, skillID string, limit int) ([]SkillVersion, error) {
m.mu.RLock()
defer m.mu.RUnlock()
all := m.versions[skillID]
// newest first
out := make([]SkillVersion, 0, len(all))
for i := len(all) - 1; i >= 0; i-- {
out = append(out, all[i])
if limit > 0 && len(out) >= limit {
break
}
}
return out, nil
}
func (m *Memory) GetVersionByID(_ context.Context, versionID string) (*SkillVersion, error) {
m.mu.RLock()
defer m.mu.RUnlock()
sv, ok := m.byVerID[versionID]
if !ok {
return nil, ErrNotFound
}
return &sv, nil
}
+35
View File
@@ -0,0 +1,35 @@
package skill
import (
"time"
"gitea.stevedudenhoeffer.com/steve/executus/run"
)
// ToRunnable lowers a saved Skill into the kernel's run.RunnableAgent DTO, so
// run.Executor can run a skill WITHOUT importing this battery (the inversion of
// mort's skillexec running a skills.Skill). Maps the static shape only; the
// skill's input schema → prompt rendering, palette resolution, audit, etc. are
// supplied separately (the host renders inputs into the input string and wires
// run.Ports). A skill exposes a flat tool list (no SkillPalette/SubAgentPalette
// — composition is a host concern), so those stay empty.
func (s *Skill) ToRunnable() run.RunnableAgent {
return run.RunnableAgent{
ID: s.ID,
Name: s.Name,
SystemPrompt: s.SystemPrompt,
ModelTier: s.ModelTier,
MaxIterations: s.MaxIterations,
MaxRuntime: s.MaxRuntime,
LowLevelTools: s.Tools,
}
}
// DueAt reports whether a scheduled skill is due at now (cron empty => never).
// Convenience for a host scheduler that doesn't want to re-parse the cron.
func (s *Skill) DueAt(now time.Time) bool {
if s.Schedule == "" || s.NextRunAt.IsZero() {
return false
}
return !s.NextRunAt.After(now)
}
+107
View File
@@ -0,0 +1,107 @@
package skill
import (
"fmt"
"strings"
"time"
"github.com/robfig/cron/v3"
)
// scheduleParser is the cron parser shared across the skills package. It
// accepts the standard 5-field syntax (minute hour dom month dow) plus
// descriptors such as @daily, @hourly, etc. We do not enable the seconds
// field — schedule cadence is governed in minutes, and a seconds field
// would invite specs that fire below the min-interval floor without
// surfacing as such in the spec text.
//
// Why standalone vs. cron.ParseStandard: ParseStandard rejects descriptors
// (@daily, @hourly). Skills callers may want to write @daily as a
// shorthand alongside the explicit "daily" / "weekly" forms we translate
// below.
var scheduleParser = cron.NewParser(
cron.Minute | cron.Hour | cron.Dom | cron.Month | cron.Dow | cron.Descriptor,
)
// ParseSchedule turns a user-supplied schedule expression into a
// cron.Schedule. The empty string returns (nil, nil) — callers should
// treat that as "on-demand only".
//
// Why: Skill.Schedule is a string field stored verbatim; the validator,
// the scheduler runner, and any future tooling all need to round-trip
// through the same parser. Centralising it here avoids drift.
//
// Accepted shorthands:
// - "daily" → "0 0 * * *" (midnight UTC every day)
// - "weekly" → "0 0 * * 0" (midnight UTC every Sunday)
//
// Anything else is fed through robfig/cron/v3's standard parser
// (descriptors enabled).
//
// Test: schedule_test.go covers shorthand expansion and invalid-spec
// rejection.
func ParseSchedule(expr string) (cron.Schedule, error) {
expr = strings.TrimSpace(expr)
if expr == "" {
return nil, nil
}
switch strings.ToLower(expr) {
case "daily":
expr = "0 0 * * *"
case "weekly":
expr = "0 0 * * 0"
}
sched, err := scheduleParser.Parse(expr)
if err != nil {
return nil, fmt.Errorf("invalid schedule %q: %w", expr, err)
}
return sched, nil
}
// ScheduleMinInterval returns an estimate of the smallest gap between
// consecutive fire times for a parsed schedule. It samples the next two
// fire times from a couple of starting points and returns the smallest
// observed gap.
//
// Why: cron.Schedule does not expose a "smallest interval" API. The
// validator needs this to enforce a per-skill min-interval floor (so an
// admin can't accidentally register "* * * * *" and burn GPU minutes).
// Two probe points are enough to catch irregular schedules whose tightest
// gap appears at a particular point in the week (e.g. "0 9 * * 1,5",
// where Mon→Fri is 4d but Fri→Mon is 3d — both sampled).
//
// Returns 0 if sched is nil.
//
// Test: schedule_test.go covers a "* * * * *" minute-interval probe and
// the irregular Mon/Fri case.
func ScheduleMinInterval(sched cron.Schedule) time.Duration {
if sched == nil {
return 0
}
// Probe from a fixed reference and from a midweek offset. Six fire
// times across two starts catches weekly irregularities (the worst
// case is a schedule that fires once a week — we still get one gap
// per probe). Using a wall-clock-independent reference keeps the
// test deterministic.
starts := []time.Time{
time.Date(2024, 1, 1, 0, 0, 0, 0, time.UTC), // Monday 00:00
time.Date(2024, 1, 4, 12, 30, 0, 0, time.UTC), // Thursday 12:30
time.Date(2024, 6, 15, 23, 59, 59, 0, time.UTC), // mid-year, late
}
var min time.Duration
for _, t := range starts {
// Sample three consecutive fires per start to capture two gaps.
f1 := sched.Next(t)
f2 := sched.Next(f1)
f3 := sched.Next(f2)
for _, gap := range []time.Duration{f2.Sub(f1), f3.Sub(f2)} {
if gap <= 0 {
continue
}
if min == 0 || gap < min {
min = gap
}
}
}
return min
}
+424
View File
@@ -0,0 +1,424 @@
// Package skills implements the agentic skills platform: user-creatable
// agent definitions (system prompt + tool whitelist + I/O spec) that run
// in-process via majordomo's agent loop.
//
// A Skill is a saved agent definition. It can be invoked from Discord
// (.skill <name>), exposed to the chatbot as a tool (via the
// SkillsToolProvider), and (in v2) scheduled. Skills compose tools from
// the skilltools registry, gated by a three-stage permission model:
// save-time AuthoringRequirement, share-time SafeForShare, execute-time
// SkillNameGate.
//
// This file declares the domain types only. Storage lives in storage.go;
// validation lives in validate.go. The grand storage pattern documented in
// pkg/logic/storage/CLAUDE.md applies — when adding a field to Skill, you
// MUST also update pkg/logic/skills/gorm_model.go (gormSkill, fromStorage,
// toStorage) or persistence will silently break.
package skill
import "time"
// Skill is the domain definition of an agentic skill.
//
// Why: a skill is a saved agent definition reusable across invocations
// (Discord, chatbot tool, scheduled run in v2). The struct is intentionally
// flat — every field lives on its own column on the skills table; there is
// no JSON-blob spec column. This keeps queries (e.g. "list all skills with
// chatbot exposure") indexable and avoids opaque migration headaches.
//
// What: identity + authoring + agent spec + visibility + chatbot exposure
// fields, all on one struct.
//
// Test: see validate_test.go and integration_test.go for round-trip and
// validation coverage.
type Skill struct {
// Identity
ID string // UUID
OwnerID string // Discord member ID; empty for builtin
Name string // unique per (owner, builtin namespace)
Description string
Source Source // SourceBuiltin | SourceManual
CreatedAt time.Time
UpdatedAt time.Time
// Authoring (copied at save time from the user)
AuthoredBy string // member ID at time of last edit (audit; may differ from owner over time)
// Versioning (for builtins; user skills typically stay at 1.0.0)
Version string // semver; used by builtin loader to decide re-seed
// Spec — agent definition
SystemPrompt string
Tools []string // registry tool names
ModelTier string // "fast" | "standard" | "thinking" | explicit "provider/model"
InputSchema []InputParam
OutputTarget OutputTarget
Schedule string // cron; empty = on-demand only; rejected in v1 (ships in v2)
Visibility Visibility // VisibilityPrivate | VisibilityShared | VisibilityPublic
SharedWith []string // member IDs for visibility=shared
MaxIterations int // 0 → use convar default
MaxToolCalls int // 0 → use convar default
MaxRuntime time.Duration // 0 → use convar default
InitialMessage string
// Chatbot exposure (v1 — proves out the platform via mortventure)
ExposeAsChatbotTool bool
ChatbotToolName string
ChatbotToolDescription string
ChatbotChannelFilter string // named filter from the channel-filter registry
// Admin gating (v2 — public scheduled channel skills require approval).
// DEPRECATED in v3: PinnedVersionID subsumes this flag for non-owner
// invocation gating. CanInvoke no longer references this column.
// Drop in v4.
PendingApproval bool
// Pinned version (v3 — admin-curated invocation gate).
//
// Why: in v3, non-owner invocation requires that an admin explicitly
// pin a known snapshot. This replaces v2's PendingApproval flag —
// pinning is the explicit "approved for general use" signal, and the
// pinned snapshot is what executes for non-owner callers (so an owner
// editing a public skill never accidentally exposes work-in-progress
// to other users).
//
// PinnedVersionID is the SkillVersion.ID (UUID) of the snapshot that
// non-owner invocations resolve to. Empty means "no pin yet" — only
// the owner and admins can invoke.
//
// Schema column is `pinned_version` per the design spec but the field
// name in the domain struct is explicit about the kind of value it
// holds (a snapshot row's UUID, NOT a semver string), which avoids
// the spec ambiguity around "pin to v1.0.5" potentially mapping to
// multiple snapshot rows over time.
PinnedVersionID string
// PinnedAt is the wall-clock time the pin was set. Zero means
// PinnedVersionID is empty (never pinned).
PinnedAt time.Time
// PinnedBy is the admin member ID who set the current pin. Empty
// when PinnedVersionID is empty.
PinnedBy string
// Scheduler bookkeeping (v2). Updated by the scheduler runner after
// a successful (or failed-but-counted) scheduled execution.
//
// LastScheduledRunAt records the wall-clock time of the most recent
// scheduled invocation; zero means "never run on schedule".
//
// NextRunAt is the precomputed wake-up time the scheduler polls for
// (`WHERE next_run_at <= NOW()`). It is recomputed by feeding
// LastScheduledRunAt (or NOW() on first scheduling) through
// ParseSchedule(Schedule).Next(...). Manual / on-demand invocations
// MUST NOT touch these fields.
LastScheduledRunAt time.Time
NextRunAt time.Time
// ExtendedBounds, when true, lets a non-admin author save the skill
// with bounds (MaxIterations / MaxToolCalls / MaxRuntime) above the
// default tier (12/30/60s) up to the extended tier (50/150/600s).
// Set by an admin via `.skill admin grant-extended <name>`. Cleared
// by `.skill admin revoke-extended <name>`. Builtins and admin-
// authored skills bypass the cap entirely (the tier resolution in
// Validate treats AuthorIsAdmin and ExtendedBounds equivalently).
//
// Why a per-skill flag vs a per-user grant: governance is per-skill
// — an admin reviews a specific skill's bounds and decides those
// resource limits are justified for THAT skill. A user grant would
// blanket-allow expensive bounds on every skill they author.
ExtendedBounds bool
// ParallelCompositionAllowed gates whether this skill may use the
// skill_invoke_parallel tool. Default false.
//
// Why a per-skill admin gate: parallel fan-out multiplies blast
// radius (one bad skill spawns N concurrent runs). Admins approve
// each skill that's allowed to use parallel composition; granting
// is per-skill via `.skill admin grant-parallel <name>`. Builtins
// may set this directly in skill.yml (the loader bypasses
// save-time gates by design).
//
// Checked AT INVOCATION TIME (every skill_invoke_parallel call), so
// admins can grant or revoke without redeploying. The check lives
// in the tool handler (pkg/skilltools/tools/skill_invoke_parallel.go)
// via the SkillInvokerProvider.IsParallelAllowed extension.
ParallelCompositionAllowed bool
// ExecutionLane is the named lane the skill's runs are submitted to
// when the executor routes through pkg/lane (v6). Default
// "skill-default"; admin overrides per-skill via
// `.skill admin set-lane <name> <lane>`.
//
// Why per-skill (vs a single global skill lane): different skills
// have different concurrency profiles. A long-running web-research
// skill might warrant a dedicated 1-slot lane to avoid starving
// quick chatbot-exposed skills; an admin should be able to isolate
// it without a code change.
//
// Empty string falls through to "skill-default" at executor time
// — keeping the field nullable lets a future schema change
// distinguish "explicit skill-default" from "never set".
ExecutionLane string
// WebhookSecret enables inbound webhooks (v7). Empty = disabled
// (the default). Non-empty = the random secret URL path segment
// for POST /webhooks/<secret>. Generated by EnableWebhook;
// rotated by RegenerateWebhookSecret. Storage is varchar(64) and
// the secret is 32 random bytes (64 hex chars), so the column
// holds a fully unique secret per skill.
//
// Why store the secret directly (not a hash): the webhook handler
// must look up the skill by the secret on every POST, which would
// require comparing every stored hash against the supplied secret
// — a per-call O(n_skills) operation. The secret is treated as a
// long random URL key (like a paste UUID); compromise is mitigated
// via RegenerateWebhookSecret rotation, not via storage hashing.
WebhookSecret string
// WebhookSignatureRequired controls whether the inbound webhook
// handler verifies HMAC against the X-Mort-Signature header. Default
// true (the storage column default). Toggling to false skips HMAC
// verification — useful for low-stakes integrations behind an IP
// allowlist where the caller can't easily compute HMAC. Owners
// flip this on the management page; admins can also force it
// back on if a leaked allowlist becomes a concern.
WebhookSignatureRequired bool
// WebhookIPAllowlist is a newline-separated list of CIDR blocks
// (or bare IPs). Empty string = no allowlist (accept any source
// IP). The handler parses the list at request time so updates take
// effect immediately without a redeploy. Invalid CIDR entries
// are silently dropped at parse time (the management page form
// shows a parse-error preview before save).
WebhookIPAllowlist string
// EncryptionEnabled (v8) opts the skill into per-skill envelope
// encryption for KV values and file blob content. Default false
// (plaintext storage; matches the legacy default). When true, new
// writes go through the AES-256-GCM helpers in pkg/skilltools and
// the corresponding skill_kv / skill_file_blobs row stamps
// encryption_key_version=1; reads transparently decrypt rows whose
// version > 0 and pass through rows whose version == 0 (mixed
// storage is supported indefinitely).
//
// !!!!! OPERATIONAL WARNING !!!!! This flag is a write-side switch
// only. Disabling encryption for an already-encrypted skill does
// NOT decrypt existing rows — they remain reachable as long as
// the master key is intact. Losing SKILLS_ENCRYPTION_MASTER_KEY
// renders every encrypted row unreadable; back the master key up
// separately from database backups. See pkg/skilltools/encryption.go
// for the full operational rules.
EncryptionEnabled bool
// Preemptible (v9) opts the skill into preemption: when a higher-
// priority job arrives at a full lane, this skill's running job may
// be cancelled mid-flight to free a slot. Default false.
//
// !!!!! OPERATIONAL WARNING !!!!! Preemption means the skill's
// scaddy.Agent context is cancelled mid-step; any partial side
// effects (file writes, KV updates, sent emails, etc.) remain
// committed. Only mark a skill preemptible when it is idempotent
// or read-only — otherwise the user-visible state may be
// inconsistent with the run's "preempted" terminal status.
//
// The lane scheduler will not preempt jobs younger than
// `skills.lane.preemption_min_runtime_seconds` (default 30s) to
// prevent thrashing. The preempted run is recorded with
// status="preempted".
Preemptible bool
// DefaultPriority (v9) is the per-skill default priority used by
// the lane scheduler's fair-share queue ordering. Higher numbers
// run first within a single user's sub-queue. Default 0.
//
// Per-invocation overrides (skill_invoke priority arg, webhook
// X-Mort-Priority header) win over this default. Owners may set
// values in the range [-`skills.priority_max_per_user`,
// +`skills.priority_max_per_user`] (default cap 5); admins may
// exceed the cap.
DefaultPriority int
// Tags is a free-form set of short labels owners attach to a skill
// for organisation + discovery. The list page renders each tag as a
// chip and offers a dropdown filter populated from all visible
// skills' tags.
//
// Why a separate field (vs reusing Description / Tools): tags are a
// curatorial signal, not part of the agent spec — they only matter
// to humans browsing the list. Storing them on the skill row (vs a
// side table) keeps lookups index-only and matches how the rest of
// the skill's flat fields are persisted.
//
// Validate enforces: each tag is trimmed + lowercased; max 32 chars
// per tag; max 16 tags per skill; duplicates within a single skill
// are deduped.
Tags []string
// DeprecatedByAgentID is the Phase 7 soft-retire pointer: when
// non-empty, the Skill is "soft retired" — hidden from default
// listings (`.skill list`, the webui index, chatbot tool exposure)
// but STILL invokable via `.skill <name>` and via `skill_invoke`
// tool calls. The string is the agents.Agent.ID of the replacement
// Agent that supersedes this Skill.
//
// Why a pointer (not a bool): a future audit / migration tool needs
// to follow the soft-retire link back to the replacement. An admin
// browsing the deprecated-skills page wants to see "what should I
// use instead?" without a separate lookup table.
//
// Why keep the Skill row (not drop it): existing skill_invoke calls
// in user-authored skills, scheduled jobs, and webhook integrations
// would break if the row vanished. Soft-retire preserves the
// callable surface while signalling "this is the old name; the
// replacement Agent is the curated version."
//
// Set by the Phase 7 boot migration (pkg/logic/agents/migrate_phase7.go);
// admins may also flip it manually via storage tooling. Listing
// methods filter on this field by default but explicit GetByName /
// GetForInvocation lookups bypass the filter so direct invocation
// continues to work.
DeprecatedByAgentID string
// DefaultEmoji is an optional identity emoji for the skill, shown
// as the __start__ fallback when StateReactEmoji has no __start__
// entry. Also forwarded to the invoking Discord message when a
// parent agent calls this skill via skill_invoke, so the user sees
// the child skill's identity emoji during execution.
DefaultEmoji string
// StateReactEmoji maps tool names (and reserved keys "__start__",
// "__end__", "__error__") to Discord emoji that the bot reacts to
// the invoking message with as the skill progresses. Empty map
// (the default) disables state-react reactions for this skill.
//
// Why: the legacy `.query` agent surfaced live progress via emoji
// reactions on the invoking message (magnifying glass on search,
// page on read, …). Skills inherit the same UX without each
// author having to wire `update_status` for trivial signalling —
// the emoji map is declarative and the executor calls inv.OnEvent
// at the relevant boundaries. update_status remains for richer
// interim text; emoji reactions are an additive lightweight signal.
//
// Reserved keys:
// - __start__: reacted right before agent.Run starts
// - __end__: reacted on successful completion
// - __error__: reacted on terminal error
//
// Tool keys: react fires on each tool dispatch. Repeated reactions
// of the same emoji are no-ops at Discord (idempotent), so a skill
// that calls web_search 5x just leaves one 🔍.
//
// Map values are arbitrary Discord emoji strings (unicode emoji,
// custom emoji `<:name:id>`, animated `<a:name:id>`). Validate does
// not enforce a format — Discord rejects invalid emoji at react
// time and the executor swallows that with a log line.
StateReactEmoji map[string]string
}
// ThreadIDInputKey is the magic key under skilltools.Invocation.SkillInputs
// that the v2 .skill new / .skill edit wizard handlers use to thread a
// pre-created thread channel ID through to delivery. When
// OutputTarget.Kind == "thread" and this key is present in
// inv.SkillInputs, delivery posts directly to that thread channel;
// otherwise it falls back to OutputTarget.Target / inv.ChannelID.
//
// Why a magic input key vs an OutputTarget override field: keeps the
// wire shape (Skill struct) unchanged and keeps the override scoped
// to a single invocation. Wizard commands set this immediately after
// MessageThreadStartComplex; nothing else writes it.
//
// Why defined here vs in skillexec: wizard command handlers in this
// package need to write the key, and skillexec imports skills (so
// the reverse import would cycle). Skillexec aliases this constant.
const ThreadIDInputKey = "__thread_id__"
// Source distinguishes builtins (loaded from skills/<name>/skill.yml on
// boot) from user-authored manual skills.
//
// Why: builtin skills bypass save-time authoring and share-time safety
// checks because the loader is trusted infrastructure.
type Source string
const (
SourceBuiltin Source = "builtin"
SourceManual Source = "manual"
)
// InputParam declares a typed input slot on a skill, populated at
// invocation time from positional/flag args (Discord) or form fields
// (webui).
//
// Why: skills are invoked from heterogeneous surfaces and need a uniform
// schema for input collection and validation. The Type drives string→typed
// coercion in skillexec.validateInputs; Choices restricts to an enum set.
type InputParam struct {
Name string
Description string
Type string // "string"|"int"|"float"|"bool"|"user"|"channel"|"url"
Required bool
Default string // string-encoded; parsed per Type at invocation
Choices []string
}
// OutputTarget controls where the executor delivers a skill's output.
//
// Why: skills run in many contexts and the user shouldn't have to think
// about delivery — the spec encodes it once. The Discord delivery
// implementation in pkg/logic/skillexec/delivery.go reads this struct.
type OutputTarget struct {
Kind string // "channel"|"dm"|"thread"|"webui_only"|"channel_with_summary"
Target string // channel/member/thread ID, or empty for caller-context
}
// Visibility controls who may invoke a skill.
//
// Why: separates *invocation* gating (this struct) from *tool authoring*
// gating (skilltools.Permission) — they are orthogonal. A non-admin can
// invoke an admin-authored public skill that uses db_select; the permission
// model for the underlying tool only fires at save time, not invocation.
type Visibility string
const (
VisibilityPrivate Visibility = "private"
VisibilityShared Visibility = "shared"
VisibilityPublic Visibility = "public"
)
// IsKnownVisibility reports whether v is a recognised visibility value.
// Used by Validate.
func IsKnownVisibility(v Visibility) bool {
switch v {
case VisibilityPrivate, VisibilityShared, VisibilityPublic:
return true
}
return false
}
// IsKnownOutputKind reports whether kind is a recognised OutputTarget.Kind.
// Used by Validate and by the Discord delivery switch.
//
// "channel_with_summary" is the v-research delivery kind: full output
// posts to a configured spam channel (skills.research.spam_channel_id)
// while a generated summary posts in the original channel as a reply
// linking back. Falls through to plain "channel" behaviour when the
// spam channel convar is unset or matches the invocation channel.
// Validate accepts this kind here; the Discord delivery switch in
// pkg/logic/skillexec/delivery_discord.go is the consumer side.
func IsKnownOutputKind(kind string) bool {
switch kind {
case "channel", "dm", "thread", "webui_only", "channel_with_summary":
return true
}
return false
}
// IsKnownInputType reports whether t is a recognised InputParam.Type.
// Used by Validate and by skillexec.validateInputs for coercion dispatch.
func IsKnownInputType(t string) bool {
switch t {
case "string", "int", "float", "bool", "user", "channel", "url":
return true
}
return false
}
+57
View File
@@ -0,0 +1,57 @@
package skill
import (
"context"
"testing"
"time"
)
func TestSkillToRunnable(t *testing.T) {
s := &Skill{
ID: "s1", Name: "summarizer", SystemPrompt: "summarize well", ModelTier: "fast",
MaxIterations: 4, MaxRuntime: 20 * time.Second, Tools: []string{"summarize", "now"},
}
r := s.ToRunnable()
if r.ID != "s1" || r.ModelTier != "fast" || r.MaxIterations != 4 || len(r.LowLevelTools) != 2 {
t.Fatalf("ToRunnable mapping wrong: %+v", r)
}
// A skill exposes a flat tool list, not a palette.
if len(r.SkillPalette) != 0 || len(r.SubAgentPalette) != 0 {
t.Errorf("skill should have empty palettes, got %+v", r)
}
}
func TestMemoryStoreVisibilityAndVersions(t *testing.T) {
ctx := context.Background()
m := NewMemory()
pub := &Skill{ID: "a", Name: "pub", OwnerID: "o1", Visibility: VisibilityPublic}
shared := &Skill{ID: "b", Name: "shr", OwnerID: "o1", Visibility: VisibilityShared, SharedWith: []string{"bob"}}
priv := &Skill{ID: "c", Name: "prv", OwnerID: "o1", Visibility: VisibilityPrivate}
for _, s := range []*Skill{pub, shared, priv} {
if err := m.Save(ctx, s); err != nil {
t.Fatal(err)
}
}
if ps, _ := m.ListPublic(ctx); len(ps) != 1 || ps[0].ID != "a" {
t.Errorf("ListPublic = %+v", ps)
}
if ss, _ := m.ListSharedWith(ctx, "bob"); len(ss) != 1 || ss[0].ID != "b" {
t.Errorf("ListSharedWith(bob) = %+v", ss)
}
if ss, _ := m.ListSharedWith(ctx, "carol"); len(ss) != 0 {
t.Errorf("ListSharedWith(carol) should be empty, got %+v", ss)
}
if all, _ := m.ListByOwner(ctx, "o1"); len(all) != 3 {
t.Errorf("ListByOwner = %d, want 3", len(all))
}
// Versions: newest-first, fetchable by id.
m.AppendVersion(ctx, SkillVersion{ID: "v1", SkillID: "a", Version: "1.0.0"})
m.AppendVersion(ctx, SkillVersion{ID: "v2", SkillID: "a", Version: "1.1.0"})
vs, _ := m.ListVersionsBySkill(ctx, "a", 10)
if len(vs) != 2 || vs[0].ID != "v2" {
t.Errorf("versions newest-first wrong: %+v", vs)
}
if got, err := m.GetVersionByID(ctx, "v1"); err != nil || got.Version != "1.0.0" {
t.Errorf("GetVersionByID: %v %+v", err, got)
}
}
+28
View File
@@ -0,0 +1,28 @@
package skill
import "time"
// SkillVersion is one immutable snapshot of a Skill at the moment it
// was saved. The skill_versions table is append-only; pruning is by
// retention policy in PruneOldVersions.
//
// Why: edit history with rollback (v3) and the admin pin gate (v3 Phase 4)
// both need a stable snapshot of the skill at a known version. The Snapshot
// field carries the FULL Skill struct so a later restore or pin produces
// the exact agent definition that was saved — system_prompt, tools,
// schedule, every field — not a synthesized partial snapshot.
//
// What: identity (UUID per snapshot) + skill ref + version-string copy +
// the full Skill payload + audit fields (saved_by, saved_at, edit_summary).
//
// Test: see skill_version_test.go for round-trip, list ordering, prune
// retention, and version-by-number disambiguation coverage.
type SkillVersion struct {
ID string // UUID per snapshot (NOT the skill's ID)
SkillID string // FK to skills.id (conceptually; not enforced by GORM)
Version string // Skill.Version at save time (semver)
Snapshot Skill // full Skill struct embedded; serialised as JSON
SavedBy string // caller member ID (or "" for builtin loader / pre-v3)
SavedAt time.Time // wall-clock save time
EditSummary string // optional human-readable note ("changed model tier", "...")
}
+44
View File
@@ -0,0 +1,44 @@
package skill
import (
"context"
"errors"
"time"
)
// ErrNotFound is returned when a skill (or version) lookup misses.
var ErrNotFound = errors.New("skill not found")
// SkillStore is the persistence seam for saved skills. This is the DELIBERATELY
// LEAN redesign of mort's 60-method skills.Storage: it carries only skill
// lifecycle (CRUD + visibility), versioning, and scheduling. The KV/file/quota
// sub-stores that were fused into mort's interface are NOT here — they are the
// tools/ store seams (KVStorage / FileStorage / QuotaProvider); email recipients
// and channel grants stay host concerns. A host backs this with its DB; Memory()
// is the zero-dependency default; contrib/store adds durable SQLite.
type SkillStore interface {
// Initialize prepares storage (idempotent).
Initialize(ctx context.Context) error
// --- lifecycle ---
Save(ctx context.Context, s *Skill) error
Get(ctx context.Context, id string) (*Skill, error)
GetByName(ctx context.Context, ownerID, name string) (*Skill, error)
Delete(ctx context.Context, id string) error
// --- listing / visibility ---
ListByOwner(ctx context.Context, ownerID string) ([]Skill, error)
ListPublic(ctx context.Context) ([]Skill, error)
ListSharedWith(ctx context.Context, memberID string) ([]Skill, error)
ListBuiltinByName(ctx context.Context, name string) (*Skill, error)
ListChatbotExposed(ctx context.Context) ([]Skill, error)
// --- scheduling ---
ListDueScheduled(ctx context.Context, now time.Time) ([]Skill, error)
MarkScheduledRun(ctx context.Context, skillID string, ranAt, nextAt time.Time) error
// --- versioning ---
AppendVersion(ctx context.Context, sv SkillVersion) error
ListVersionsBySkill(ctx context.Context, skillID string, limit int) ([]SkillVersion, error)
GetVersionByID(ctx context.Context, versionID string) (*SkillVersion, error)
}
+374
View File
@@ -0,0 +1,374 @@
package skill
import (
"fmt"
"strings"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/model"
)
// ChannelFilterChecker is the subset of ChannelFilterRegistry used by
// Validate to check that a skill references a registered channel filter.
//
// Why: kept narrow so tests can pass a tiny stub; full registry is
// declared in channel_filters.go.
type ChannelFilterChecker interface {
Has(name string) bool
}
// ModelTierChecker reports whether the given model tier or
// "provider/model" spec is recognised. Validate uses this to reject
// typos at save time.
//
// Why: tiers come from llms.tier.* convars (fast/standard/thinking by
// default) but admins may add custom tiers; explicit "provider/model"
// is also valid. Validate accepts anything non-empty matching either
// pattern — finer correctness is the LLM call's job.
type ModelTierChecker interface {
IsValid(spec string) bool
}
// defaultModelTierChecker accepts all registered tier names (via
// model.IsTierName) plus any "provider/model" form (string contains "/").
// Tests can substitute a strict checker via ValidateOpts.ModelTierChecker.
type defaultModelTierChecker struct{}
func (defaultModelTierChecker) IsValid(spec string) bool {
if spec == "" {
return false
}
if model.IsTierName(spec) {
return true
}
// Accept tier-with-reasoning (e.g. "thinking:high")
if i := strings.IndexByte(spec, ':'); i > 0 {
if model.IsTierName(spec[:i]) {
return true
}
}
// Accept explicit "provider/model" or "provider/model:reasoning"
return strings.ContainsRune(spec, '/')
}
// ValidateOpts customises what Validate accepts. All fields are optional;
// nil checkers fall back to permissive defaults.
//
// Why: Validate is called from save paths (which know the registries) and
// from tests (which want to control acceptance). Bundling the deps here
// keeps the Skill API stable.
type ValidateOpts struct {
// Filters is consulted when the skill declares a chatbot channel
// filter. nil → channel-filter validity is not checked (use only in
// tests).
Filters ChannelFilterChecker
// ModelTier checks the ModelTier spec. nil → defaultModelTierChecker.
ModelTier ModelTierChecker
// MinIntervalMinutes is the floor on the smallest gap between
// consecutive fires of a skill's cron schedule. Zero → use the
// package default (defaultMinScheduleIntervalMinutes). Tests pass an
// explicit value to exercise the boundary.
MinIntervalMinutes int
// AuthorIsAdmin tells Validate the author has admin privileges and
// may save with extended-tier bounds without ExtendedBounds=true.
// SaveUserSkill passes this from s.admin.IsAdmin(sk.AuthoredBy).
// Builtin loader sets this true to bypass the per-skill flag check
// (builtins are trusted infrastructure).
AuthorIsAdmin bool
// DefaultMaxIterations / DefaultMaxToolCalls / DefaultMaxRuntimeSecs
// override the package-default tier-1 caps. Zero → fall back to the
// constants below. Production wiring populates these from convars
// (skills.default_max_iterations etc.) so admins can adjust the
// default tier without a redeploy.
DefaultMaxIterations int
DefaultMaxToolCalls int
DefaultMaxRuntimeSecs int
// ExtendedMaxIterations / ExtendedMaxToolCalls / ExtendedMaxRuntimeSecs
// override the package-default tier-2 caps (the ceilings allowed when
// ExtendedBounds=true OR AuthorIsAdmin=true). Zero → fall back to the
// constants below.
ExtendedMaxIterations int
ExtendedMaxToolCalls int
ExtendedMaxRuntimeSecs int
}
// Tiered cap defaults. The DEFAULT tier is what a non-admin author sees
// without an explicit grant; the EXTENDED tier is what admin authors and
// admin-granted skills may use. Values are tuned in the v3 spec
// "Governance: tiered resource caps" section.
//
// The package's existing absolute ceilings (maxIterationsLimit=50 and
// maxRuntime=10m) act as outer floors / sanity bounds; the tier caps
// are the active gate at save time. Extended caps respect the absolute
// ceilings naturally (50 iter, 600s = 10min runtime).
const (
// Default tier — non-admin authors of skills without ExtendedBounds.
DefaultMaxIterations = 12
DefaultMaxToolCalls = 30
DefaultMaxRuntimeSecs = 60
// Extended tier — admin authors OR ExtendedBounds=true.
ExtendedMaxIterations = 50
ExtendedMaxToolCalls = 150
ExtendedMaxRuntimeSecs = 600 // 10m
maxIterationsLimit = 50
minRuntime = time.Second
maxRuntime = 10 * time.Minute
defaultMinScheduleIntervalMinutes = 30
// MaxTagsPerSkill caps the number of organisation tags any single
// skill may carry. Generous compared to typical taxonomies (GitHub
// allows ~10 topics/repo). The cap exists to prevent the list
// page's chip rendering from becoming unmanageable.
MaxTagsPerSkill = 16
// MaxTagLength is the per-tag character ceiling. Long enough for
// hyphenated phrases ("retro-gaming") but short enough that the
// list-page tag dropdown stays readable.
MaxTagLength = 32
)
// Validate enforces the skill spec invariants documented in the design
// spec ("Skill domain model" section). It is called at save time; the
// builtin loader skips authoring/share-safety checks but still runs
// Validate, so all callers can rely on a saved skill being well-formed.
//
// Why: spec rules are easy to violate by hand and silently break
// downstream (e.g. an unknown channel filter never exposes the skill to
// the chatbot). Every rule fails loudly here.
//
// What: returns the first error found; callers may surface it directly to
// users. opts may be the zero value, in which case channel-filter
// validation is skipped (tests).
//
// Test: each rejection branch has a dedicated unit test in
// validate_test.go.
func (s *Skill) Validate(opts ValidateOpts) error {
if s == nil {
return fmt.Errorf("skill is nil")
}
if strings.TrimSpace(s.Name) == "" {
return fmt.Errorf("skill name is required")
}
if strings.TrimSpace(s.SystemPrompt) == "" {
return fmt.Errorf("skill system prompt is required")
}
// ModelTier
tierCheck := opts.ModelTier
if tierCheck == nil {
tierCheck = defaultModelTierChecker{}
}
if !tierCheck.IsValid(s.ModelTier) {
return fmt.Errorf("unknown model tier %q (expected a tier alias or provider/model)", s.ModelTier)
}
// Schedule — empty means on-demand only. A non-empty value must be
// a valid cron expression (or one of the "daily" / "weekly"
// shorthands) AND have a smallest fire-gap >= the configured
// min-interval floor. Both checks share the package-level
// ParseSchedule helper so the scheduler runner uses the same parser.
if expr := strings.TrimSpace(s.Schedule); expr != "" {
sched, err := ParseSchedule(expr)
if err != nil {
return fmt.Errorf("schedule: %w", err)
}
minMinutes := opts.MinIntervalMinutes
if minMinutes == 0 {
minMinutes = defaultMinScheduleIntervalMinutes
}
floor := time.Duration(minMinutes) * time.Minute
if interval := ScheduleMinInterval(sched); interval < floor {
return fmt.Errorf(
"schedule %q runs more often than the minimum (every %s, floor is %s)",
expr, interval.Round(time.Second), floor)
}
}
// Iteration / call / runtime budgets. Zero is allowed — the executor
// substitutes a convar-backed default. Negative is always wrong.
// The absolute ceilings (maxIterationsLimit=50, maxRuntime=10m) are
// outer sanity bounds; the tier caps below are the active gate.
//
// Why admin bypass on the outer ceilings: builtins are trusted
// infrastructure (per the v2 "Builtin loader must bypass save-time
// gates" lesson). The builtin loader passes AuthorIsAdmin=true so
// trusted skills like `deepresearch` (max_iterations=100,
// max_runtime=45m) and `research` (max_runtime=15m) can validate
// without re-tuning the package-wide outer floor for everyone.
// Non-admin authors still hit the original ceilings AND the
// tier-based cap (default 12 iter / 60s runtime, extended 50 iter /
// 600s runtime) — both layers stay intact for the untrusted path.
if s.MaxIterations < 0 {
return fmt.Errorf("max_iterations must be >= 0, got %d", s.MaxIterations)
}
if !opts.AuthorIsAdmin && s.MaxIterations > maxIterationsLimit {
return fmt.Errorf("max_iterations must be 0..%d, got %d", maxIterationsLimit, s.MaxIterations)
}
if s.MaxToolCalls < 0 {
return fmt.Errorf("max_tool_calls must be >= 0, got %d", s.MaxToolCalls)
}
if s.MaxRuntime < 0 {
return fmt.Errorf("max_runtime must be 0 or positive, got %s", s.MaxRuntime)
}
if s.MaxRuntime > 0 && s.MaxRuntime < minRuntime {
return fmt.Errorf("max_runtime must be 0 or >= %s, got %s", minRuntime, s.MaxRuntime)
}
if !opts.AuthorIsAdmin && s.MaxRuntime > maxRuntime {
return fmt.Errorf("max_runtime must be 0 or in [%s..%s], got %s", minRuntime, maxRuntime, s.MaxRuntime)
}
// Tiered cap resolution: a skill saved by an admin OR a skill with
// ExtendedBounds=true (admin-granted) may use the extended tier;
// everything else saturates at the default tier. Builtins go through
// the loader's bypass path (AuthorIsAdmin=true).
defIter := opts.DefaultMaxIterations
if defIter == 0 {
defIter = DefaultMaxIterations
}
defCalls := opts.DefaultMaxToolCalls
if defCalls == 0 {
defCalls = DefaultMaxToolCalls
}
defRuntime := opts.DefaultMaxRuntimeSecs
if defRuntime == 0 {
defRuntime = DefaultMaxRuntimeSecs
}
extIter := opts.ExtendedMaxIterations
if extIter == 0 {
extIter = ExtendedMaxIterations
}
extCalls := opts.ExtendedMaxToolCalls
if extCalls == 0 {
extCalls = ExtendedMaxToolCalls
}
extRuntime := opts.ExtendedMaxRuntimeSecs
if extRuntime == 0 {
extRuntime = ExtendedMaxRuntimeSecs
}
maxIter := defIter
maxCalls := defCalls
maxRuntimeSecs := defRuntime
tier := "default"
hint := "; ask an admin to grant extended_bounds for higher"
if s.ExtendedBounds || opts.AuthorIsAdmin {
maxIter = extIter
maxCalls = extCalls
maxRuntimeSecs = extRuntime
tier = "extended"
hint = "" // already at the highest tier — no upgrade path
}
// Admin bypass on the tier cap: trusted infrastructure (builtins,
// admin-authored skills) may exceed the extended tier. The
// non-admin author still hits the tier cap above. See the
// "trusted infrastructure" rationale on the outer-ceiling block.
if !opts.AuthorIsAdmin {
if s.MaxIterations > maxIter {
return fmt.Errorf("max_iterations %d exceeds %s cap (%d)%s",
s.MaxIterations, tier, maxIter, hint)
}
if s.MaxToolCalls > maxCalls {
return fmt.Errorf("max_tool_calls %d exceeds %s cap (%d)%s",
s.MaxToolCalls, tier, maxCalls, hint)
}
if s.MaxRuntime > 0 && s.MaxRuntime > time.Duration(maxRuntimeSecs)*time.Second {
return fmt.Errorf("max_runtime %s exceeds %s cap (%ds)%s",
s.MaxRuntime, tier, maxRuntimeSecs, hint)
}
}
// Output target
if !IsKnownOutputKind(s.OutputTarget.Kind) {
return fmt.Errorf("unknown output_target.kind %q", s.OutputTarget.Kind)
}
// Input schema
seenInput := map[string]struct{}{}
for i, p := range s.InputSchema {
if strings.TrimSpace(p.Name) == "" {
return fmt.Errorf("input_schema[%d]: Name is required", i)
}
if !IsKnownInputType(p.Type) {
return fmt.Errorf("input_schema[%d] (%q): unknown type %q", i, p.Name, p.Type)
}
if _, dup := seenInput[p.Name]; dup {
return fmt.Errorf("input_schema: duplicate parameter name %q", p.Name)
}
seenInput[p.Name] = struct{}{}
}
// Tools
seenTool := map[string]struct{}{}
for _, t := range s.Tools {
if strings.TrimSpace(t) == "" {
return fmt.Errorf("tools: empty tool name")
}
if _, dup := seenTool[t]; dup {
return fmt.Errorf("tools: duplicate tool name %q", t)
}
seenTool[t] = struct{}{}
}
// Tags — normalise + bounds-check. The caller may pass user input
// directly; we trim, lowercase, dedup, and bound count + per-tag
// length. Mutating the slice in place is intentional so callers
// don't need a separate normalise pass.
//
// Why caps (16 tags / 32 chars): both are generous for human-
// curated organisation labels (compare to GitHub's 10 topics/repo
// + ~50 chars). The aim is rejecting accidental data dumps and
// keeping the list-page chip rendering manageable, not strict
// taxonomy enforcement.
if len(s.Tags) > MaxTagsPerSkill {
return fmt.Errorf("tags: too many (max %d, got %d)", MaxTagsPerSkill, len(s.Tags))
}
if len(s.Tags) > 0 {
seenTag := map[string]struct{}{}
out := make([]string, 0, len(s.Tags))
for _, raw := range s.Tags {
t := strings.ToLower(strings.TrimSpace(raw))
if t == "" {
continue
}
if len(t) > MaxTagLength {
return fmt.Errorf("tags: %q exceeds %d chars", t, MaxTagLength)
}
if _, dup := seenTag[t]; dup {
continue
}
seenTag[t] = struct{}{}
out = append(out, t)
}
s.Tags = out
}
// Visibility
if !IsKnownVisibility(s.Visibility) {
return fmt.Errorf("unknown visibility %q", s.Visibility)
}
if s.Visibility == VisibilityShared && len(s.SharedWith) == 0 {
return fmt.Errorf("visibility=shared requires non-empty shared_with")
}
// Chatbot exposure
if s.ExposeAsChatbotTool {
if strings.TrimSpace(s.ChatbotToolName) == "" {
return fmt.Errorf("expose_as_chatbot_tool=true requires chatbot_tool_name")
}
if strings.TrimSpace(s.ChatbotToolDescription) == "" {
return fmt.Errorf("expose_as_chatbot_tool=true requires chatbot_tool_description")
}
if strings.TrimSpace(s.ChatbotChannelFilter) == "" {
return fmt.Errorf("expose_as_chatbot_tool=true requires chatbot_channel_filter")
}
if opts.Filters != nil && !opts.Filters.Has(s.ChatbotChannelFilter) {
return fmt.Errorf("unknown chatbot_channel_filter %q (not registered)", s.ChatbotChannelFilter)
}
}
return nil
}
+156
View File
@@ -0,0 +1,156 @@
package skillpack
import (
"context"
"errors"
"fmt"
"sort"
"strings"
mdagent "gitea.stevedudenhoeffer.com/steve/majordomo/agent"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
mdskill "gitea.stevedudenhoeffer.com/steve/majordomo/skill"
)
// Resolve loads the pinned Pack for each enabled subscription from the cache. It
// is how a host turns "this agent subscribes to these packs" into activatable
// packs at run time without touching the network. A pinned digest missing from
// the cache is an error (the host should have cached it at pin/apply time).
// Disabled subscriptions are skipped.
func Resolve(ctx context.Context, cache PackCache, subs []Subscription) ([]*Pack, error) {
out := make([]*Pack, 0, len(subs))
for i := range subs {
s := &subs[i]
if !s.Enabled {
continue
}
tree, err := cache.Get(ctx, s.PinnedDigest)
if err != nil {
return nil, fmt.Errorf("skillpack: resolving %q: %w", s.Name, err)
}
pack, err := LoadPack(tree)
if err != nil {
return nil, fmt.Errorf("skillpack: loading %q: %w", s.Name, err)
}
out = append(out, pack)
}
return out, nil
}
// Catalog renders the always-in-prompt block for a set of packs: one line per
// pack (name + description) plus how to load one. This is the whole prompt cost
// of a subscription — the bodies stay out until skill_use is called.
func Catalog(packs []*Pack) string {
sorted := make([]*Pack, 0, len(packs))
for _, p := range packs {
if p != nil && p.Manifest != nil {
sorted = append(sorted, p)
}
}
if len(sorted) == 0 {
return ""
}
sort.Slice(sorted, func(i, j int) bool { return sorted[i].Manifest.Name < sorted[j].Manifest.Name })
var b strings.Builder
b.WriteString("You have access to skills — packaged instructions for specific tasks. ")
b.WriteString("When a task matches one, call skill_use with its name to load its full instructions before proceeding.\n\n")
b.WriteString("Available skills:\n")
for _, p := range sorted {
fmt.Fprintf(&b, "- %s: %s\n", p.Manifest.Name, p.Manifest.Description)
}
return strings.TrimRight(b.String(), "\n")
}
type skillUseArgs struct {
Name string `json:"name" description:"the exact name of the skill to load, from the Available skills list"`
}
// BundleStager makes a pack's bundled files available to the current run and
// returns a short note the model can act on (e.g. where the files are and how to
// reference them). It is called LAZILY, inside the skill_use tool, so a pack's
// files are staged only when the model actually loads that pack — not for every
// subscribed pack on every run. A host implements it over its own file plumbing
// (mort saves the files to run-scoped storage and returns their file_ids). nil =
// no staging: skill_use just lists the bundled file names.
type BundleStager func(ctx context.Context, p *Pack) (string, error)
// Activate turns a set of resolved packs into a majordomo agent.Skill: its
// Instructions are the Catalog, and it contributes a single skill_use tool that
// returns a named pack's full body (progressive disclosure). Attach the result
// to an agent with agent.WithSkill. Returns nil when there are no packs, which
// agent.WithSkill tolerates (a nil skill contributes nothing).
//
// stager, if non-nil, is invoked when skill_use loads a pack with bundled files;
// its returned note is appended to the body so the model knows how to reach the
// staged scripts/references. A stager error degrades gracefully (the
// instructions still return, with a note that the files are unavailable).
func Activate(packs []*Pack, stager BundleStager) mdagent.Skill {
byName := make(map[string]*Pack, len(packs))
for _, p := range packs {
if p != nil && p.Manifest != nil {
byName[p.Manifest.Name] = p
}
}
if len(byName) == 0 {
return nil
}
tool := llm.DefineTool("skill_use",
"Load the full instructions for a skill by name before doing a task it covers. Returns the skill's instructions and, if it has bundled files, how to access them.",
func(ctx context.Context, args skillUseArgs) (any, error) {
p, ok := byName[strings.TrimSpace(args.Name)]
if !ok {
return fmt.Sprintf("No skill named %q. Use one of the names from the Available skills list.", args.Name), nil
}
body := renderPackBody(p)
if stager != nil && len(p.Bundled) > 0 {
note, err := stager(ctx, p)
switch {
case err != nil:
body += "\n\n(bundled files could not be staged: " + err.Error() + ")"
case note != "":
body += "\n\n" + note
}
}
return body, nil
})
tb := llm.NewToolbox("skillpack", tool)
return mdskill.New("skillpacks",
mdskill.WithInstructions(Catalog(packs)),
mdskill.WithToolbox(tb),
)
}
// renderPackBody is the base skill_use payload: the pack's instructions plus, if
// it has any, a list of its bundled file names. A stager (see Activate) appends
// the concrete access note.
func renderPackBody(p *Pack) string {
if p == nil || p.Manifest == nil {
return "Error: invalid skill pack."
}
var b strings.Builder
fmt.Fprintf(&b, "# Skill: %s\n\n%s\n", p.Manifest.Name, p.Manifest.Body)
if len(p.Bundled) > 0 {
b.WriteString("\nBundled files:\n")
for _, f := range p.Bundled {
fmt.Fprintf(&b, "- %s\n", f)
}
}
return strings.TrimRight(b.String(), "\n")
}
// Stage materializes a pack's files under baseDir/<pack name>/ so a host can
// mount them (read-only is the host's concern) into a sandbox the agent's file
// tools can read. Returns the pack's staged directory.
func Stage(p *Pack, baseDir string) (string, error) {
if p == nil || p.Manifest == nil {
return "", errors.New("skillpack: Stage requires a non-nil pack")
}
dir := baseDir + "/" + p.Manifest.Name
if err := p.Tree.WriteTo(dir); err != nil {
return "", err
}
return dir, nil
}
+154
View File
@@ -0,0 +1,154 @@
package skillpack
import (
"context"
"encoding/json"
"strings"
"testing"
)
func mustPack(t *testing.T, name, body string, extra map[string]string) *Pack {
t.Helper()
tr := packTree(name, body)
for k, v := range extra {
tr[k] = []byte(v)
}
p, err := LoadPack(tr)
if err != nil {
t.Fatal(err)
}
return p
}
func TestCatalog(t *testing.T) {
packs := []*Pack{
mustPack(t, "zebra", "z", nil),
mustPack(t, "alpha", "a", nil),
}
cat := Catalog(packs)
if !strings.Contains(cat, "skill_use") {
t.Error("catalog should tell the model how to load a skill")
}
ai := strings.Index(cat, "alpha")
zi := strings.Index(cat, "zebra")
if ai < 0 || zi < 0 || ai > zi {
t.Errorf("catalog should list packs sorted by name:\n%s", cat)
}
if Catalog(nil) != "" {
t.Error("empty catalog should be empty string")
}
}
func TestActivate_SkillUseTool(t *testing.T) {
ctx := context.Background()
packs := []*Pack{
mustPack(t, "pdf", "Use pdfplumber.", map[string]string{"scripts/x.py": "print()"}),
}
staged := 0
stager := func(_ context.Context, p *Pack) (string, error) {
staged++
return "staged " + p.Manifest.Name + " (file_id=abc)", nil
}
sk := Activate(packs, stager)
if sk == nil {
t.Fatal("expected a non-nil skill")
}
if sk.Instructions() != Catalog(packs) {
t.Error("skill instructions should be the catalog")
}
tb := sk.Tools()
tool, ok := tb.Get("skill_use")
if !ok {
t.Fatal("skill_use tool missing from toolbox")
}
if staged != 0 {
t.Error("stager must be lazy — not called until skill_use runs")
}
// load an existing pack
out, err := tool.Handler(ctx, json.RawMessage(`{"name":"pdf"}`))
if err != nil {
t.Fatal(err)
}
body, _ := out.(string)
if !strings.Contains(body, "Use pdfplumber.") {
t.Errorf("skill_use body missing instructions: %q", body)
}
if !strings.Contains(body, "scripts/x.py") {
t.Errorf("skill_use should list bundled files: %q", body)
}
if staged != 1 || !strings.Contains(body, "file_id=abc") {
t.Errorf("stager should run on load and its note append to the body: staged=%d body=%q", staged, body)
}
// unknown pack returns guidance, not an error
out, err = tool.Handler(ctx, json.RawMessage(`{"name":"nope"}`))
if err != nil {
t.Fatal(err)
}
if s, _ := out.(string); !strings.Contains(s, "No skill named") {
t.Errorf("unknown skill should return guidance: %q", s)
}
}
func TestActivate_Empty(t *testing.T) {
if Activate(nil, nil) != nil {
t.Error("no packs should activate to a nil skill")
}
}
func TestNilPackElementsAreSafe(t *testing.T) {
packs := []*Pack{nil, mustPack(t, "real", "b", nil), {Manifest: nil}}
// Neither Catalog nor Activate may panic on nil / malformed elements.
if got := Catalog(packs); !strings.Contains(got, "real") {
t.Errorf("catalog should include the valid pack and skip nils: %q", got)
}
sk := Activate(packs, nil)
if sk == nil {
t.Fatal("a valid pack among nils should still activate")
}
if _, ok := sk.Tools().Get("skill_use"); !ok {
t.Error("skill_use missing")
}
// All-nil activates to nothing rather than panicking.
if Activate([]*Pack{nil, {Manifest: nil}}, nil) != nil {
t.Error("only-nil packs should activate to nil")
}
}
func TestResolveFromCache(t *testing.T) {
ctx := context.Background()
cache := NewMemoryPackCache()
p := mustPack(t, "alpha", "a", nil)
cache.Put(ctx, p.Digest, p.Tree)
subs := []Subscription{
{Name: "alpha", PinnedDigest: p.Digest, Enabled: true},
{Name: "disabled", PinnedDigest: p.Digest, Enabled: false},
}
packs, err := Resolve(ctx, cache, subs)
if err != nil {
t.Fatal(err)
}
if len(packs) != 1 || packs[0].Manifest.Name != "alpha" {
t.Fatalf("resolve should skip disabled subs; got %d packs", len(packs))
}
// missing from cache is an error
subs = []Subscription{{Name: "ghost", PinnedDigest: "deadbeef", Enabled: true}}
if _, err := Resolve(ctx, cache, subs); err == nil {
t.Fatal("expected error resolving an uncached pin")
}
}
func TestStage(t *testing.T) {
dir := t.TempDir()
p := mustPack(t, "pdf", "b", map[string]string{"scripts/x.py": "print()"})
staged, err := Stage(p, dir)
if err != nil {
t.Fatal(err)
}
if !strings.HasSuffix(staged, "/pdf") {
t.Errorf("staged dir = %q", staged)
}
}

Some files were not shown because too many files have changed in this diff Show More