37 Commits

Author SHA1 Message Date
steve be4bbbcad5 run: fix statusFor — don't relabel a generic error / caller-cancel as timeout (gadfly #11)
executus CI / test (pull_request) Successful in 47s
executus CI / test (push) Successful in 45s
The WithCancelCause+timer rewrite made MaxRuntime surface as Canceled (not
DeadlineExceeded), so statusFor's context.Cause(DeadlineExceeded) check could
relabel (a) a genuine run error as 'timeout' and (b) a caller cancel/deadline as
'timeout' (was 'cancelled'). Convergent gadfly finding (glm-5.2 + cluster).

Fix: keep MaxRuntime as WithTimeout (its DeadlineExceeded propagates → 'timeout',
preserving own-timeout vs caller-cancel), add a NESTED WithCancelCause layer only
for the kill. statusFor consults context.Cause ONLY for ErrCriticKill; everything
else is classified by the run error itself. Tests: generic-error-not-relabeled +
caller-cancel-stays-cancelled.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 17:00:26 -04:00
steve 390e6cf905 run: critic parity — fuller RecordStep + cause-carrying Kill (distinct status)
executus CI / test (pull_request) Successful in 46s
Adversarial Review (Gadfly) / review (pull_request) Successful in 22m30s
Completes the run-critic seam so a host adapter (mort's agentcritic) has full
fidelity, closing the two limitations gadfly surfaced on mort #1334.

- RecordStep(iter int, resp *llm.Response): the completed step's model response
  is now passed to the critic (was index-only), so a host that records a trace
  (mort's ProgressRecorder) can show what the agent actually produced, not just
  an iteration count. The executor forwards s.Response; the battery ignores it
  (its Progress is count-based).
- CriticHandle.KillCause() error + ErrCriticKill: the executor now distinguishes
  an explicit critic KILL from a natural backstop expiry. runCtx uses a
  cause-carrying cancel (WithCancelCause + a MaxRuntime timer cancelling with
  DeadlineExceeded); the deadline-watch cancels with ErrCriticKill when
  KillCause()!=nil, else DeadlineExceeded. statusFor reads context.Cause →
  killed / timeout / cancelled are now distinct (were all "cancelled"). The
  battery sets killCause from Decision.KillReason on a Kill.

Tests: statusFor "killed" case (cause=ErrCriticKill, err=Canceled); fake handle
+ battery RecordStep/KillCause signatures. Core stays battery-free.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 16:35:13 -04:00
steve 1a1d5e417b chore: go mod tidy (add missing go.sum entry; CI tidiness gate)
executus CI / test (pull_request) Successful in 2m8s
executus CI / test (push) Successful in 1m45s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 14:53:58 -04:00
steve f3bd43b726 ci(gadfly): drop the m1 reviewer (dead weight; keep m5)
executus CI / test (pull_request) Failing after 1m1s
m1/qwen3:14b proved consistently low-value + slowest in the pool over multiple
PRs. Removed from GADFLY_MODELS + GADFLY_PROVIDER_CONCURRENCY + its endpoint so it
never fires again. m5 retained.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 14:41:14 -04:00
steve 306d575c31 critic: overflow-guard maxSteps += RaiseStepsBy (gadfly 5-model convergence)
executus CI / test (pull_request) Has been cancelled
A buggy/hostile Escalator returning a huge RaiseStepsBy could wrap handle.maxSteps
negative (which the executor reads as defer-to-base). Clamp at math.MaxInt.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 14:38:48 -04:00
steve 4ba83ab905 run: critic can raise a run's step ceiling mid-flight (CriticHandle.MaxSteps)
executus CI / test (pull_request) Failing after 1m1s
Adversarial Review (Gadfly) / review (pull_request) Successful in 21m8s
Prerequisite for a full-fidelity mort agentcritic adapter (which adjusts a
healthy-but-long run's iteration budget, not just its deadline). executus's
CriticHandle was deadline+steer only; this adds the dynamic step ceiling above
an unchanged majordomo (which already exposes WithMaxStepsFunc).

- run.RunInfo += MaxIterations (the run's base ceiling, so a critic can raise it
  relative to the baseline).
- run.CriticHandle += MaxSteps() int — polled by the executor each step via
  agent.WithMaxStepsFunc; <=0 defers to the base. The executor uses
  WithMaxStepsFunc(critic.MaxSteps) when a critic is active, else WithMaxSteps.
- critic battery: handle.maxSteps (initialised from RunInfo.MaxIterations) +
  MaxSteps(); Decision gains RaiseStepsBy so an Escalator can raise the ceiling
  alongside ExtendBy. ExtendOnce default is unchanged (time-only).

Test: a critic returning MaxSteps=5 lets a base-MaxIterations=1 run complete two
tool-dispatch steps past the base ceiling. Core stays battery-free (run doesn't
import critic).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 14:16:03 -04:00
steve a103cc5e9f ci(gadfly): 9-cloud panel @ 3 models x 3 lenses (9 concurrent)
executus CI / test (push) Failing after 1m57s
Match mort: minimax-m3, glm-5.2, glm-5.1 (SWE-Bench Pro SOTA), kimi-k2.7-code,
deepseek-v4-pro, nemotron-3-super, gpt-oss:120b, qwen3-coder:480b, gemma4 (8
families) + m1/m5 locals. ollama-cloud=3 x lens=3 = 9 concurrent (10 budget).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 12:17:24 -04:00
steve 4d28cd6e2c ci(gadfly): 4-cloud pool — add kimi-k2.7-code + deepseek-v4-pro, drop v4-flash
executus CI / test (push) Failing after 1m2s
Match mort's new cloud panel: minimax-m3, glm-5.2, kimi-k2.7-code (Moonshot),
deepseek-v4-pro (frontier, replaces v4-flash). Keeps m1/m5 locals + the existing
ollama-cloud=1 + lens-concurrency=3 serial-model style.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 11:59:13 -04:00
steve dcaefff756 ci(gadfly): add M1/M5 Macs back to the reviewer pool (full fleet)
executus CI / test (push) Failing after 1m23s
Re-adds the local Macs (m1/qwen3:14b, m5/qwen3.6:35b-mlx) via their foreman endpoints alongside the 3 cloud models. Cloud keeps lens fan-out (ollama-cloud=1 model + lens=3); each Mac runs one model with lenses serial (foreman serializes anyway); all provider lanes parallel. Bumps the job timeout 30->90m for the slow local lanes. With findings telemetry now on, gadfly-reports can quantify whether the Macs earn their keep.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 10:44:22 -04:00
steve 97154395e6 C0b: document recordToolStart post-iteration timing (gadfly glm finding)
executus CI / test (pull_request) Failing after 59s
executus CI / test (push) Failing after 1m1s
majordomo's step observer fires post-iteration, so the critic's activity clock
refreshes per-iteration, not mid-tool — a single long tool call won't refresh it
until it returns. Documented + the host-progress-bridge mitigation (mort's
pattern). A true pre-dispatch hook needs majordomo support (follow-up).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 10:10:56 -04:00
steve 4aa06f652e C0b: address verified gadfly findings (panic-safety + test honesty)
executus CI / test (pull_request) Failing after 58s
From PR #9 (minimax + deepseek):
- Run now has a top-level recover() — the "never propagates a panic" promise was
  unenforced; a panicking host Port (Critic/Audit/Palette) on the run goroutine
  now becomes Result.Err instead of unwinding into the caller.
- The critic deadline-watch goroutine recovers panics from a host Deadline()
  (it's a separate goroutine, so Run's recover can't catch it) — a buggy
  CriticHandle can't crash the process.
- CriticHandle interface documents its concurrency contract (Record*/Steer on the
  run goroutine vs Deadline()/Stop() from the watch goroutine — impls must be
  concurrent-safe; the critic battery already is).
- startCritic's dead `soft <= 0 -> noop` guard (withFallbacks already coerces to
  90s) replaced with a defensive inline 90s default, so a bypass of withFallbacks
  still gets a working critic instead of silently none.
- Delivery tests made honest: the old "error path" test only checked the
  early-return (no delivery); added TestDeliverErrorOnRunFailure (in-loop model
  error -> DeliverError to the target) + renamed the early-return test.

Graded all #9 findings in the gadfly MCP.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 10:09:22 -04:00
steve 43b2471737 C0b: wire Critic + Delivery into run.Executor
executus CI / test (pull_request) Failing after 1m0s
Adversarial Review (Gadfly) / review (pull_request) Successful in 5m9s
Continues finishing the executor's run.Ports wiring (after C0's Palette).

Critic (run/critic.go): when Ports.Critic is set and the agent enables it, the
executor calls Monitor at run start, feeds RecordStep/RecordToolStart from the
step observer, drains the critic's Steer messages into the loop via
agent.WithSteer, and binds the run's hard cancellation to the critic's
(extendable) Deadline through a watch goroutine — a healthy-but-slow run gets
room while a hung one is killed. Stop() on run end. Soft timeout from
Defaults.CriticSoftTimeout (default 90s). nil-safe: no critic / not-enabled =
no-op.

Delivery (run/executor.go deliver): after the run, when Ports.Delivery is set
and inv.DeliveryID is non-empty, the executor posts Result.Output (or
DeliverError on failure) to a host-interpreted deliver.Target
{inv.DeliveryKind, inv.DeliveryID}. Empty target = caller reads Result.Output
itself (the synchronous default; the `.agent run` canary). Best-effort +
detached.

tool.Invocation gains DeliveryKind/DeliveryID (host-set egress target).

Tests: critic monitored/fed/steered/stopped when enabled, untouched when not;
delivery posts on a target, skips without one. Deferred: Checkpointer (needs a
majordomo hook to snapshot the running message history).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 10:00:05 -04:00
steve 0c80679719 C0: address verified gadfly findings (trivial fixes)
executus CI / test (pull_request) Failing after 1m31s
executus CI / test (push) Failing after 1m31s
From the PR #8 review (all graded in the gadfly MCP):
- skip empty palette names + dedupe by final tool name, instead of producing a
  "skill__" tool or an opaque box.Add duplicate error.
- delegationResult: no trailing blank line when a non-ok child produced no output.
- delegationErr: fold a child's partial output into the hard-failure error so it
  isn't silently dropped.

Deferred to C0b (design-level, not trivial): route delegation through the
tool.Registry gate/audit wrappers; expose the skill's real input schema to the
LLM instead of a generic inputs map. typed-nil PaletteSource is left as a caller
contract (the == nil guard catches the untyped-nil interface).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 09:53:11 -04:00
steve 9d41987b0e C0: wire Palette delegation into run.Executor (skill__/agent__ tools)
executus CI / test (pull_request) Failing after 1m2s
Adversarial Review (Gadfly) / review (pull_request) Successful in 3m47s
The first cutover prerequisite: the executor now turns an agent's SkillPalette /
SubAgentPalette into delegation tools so a mort agent that delegates works
through run.Executor (the piece the `.agent run` canary needs beyond the
already-wired audit/budget).

- run/palette.go: addDelegationTools builds a skill__<name> tool (structured
  inputs) per SkillPalette entry and an agent__<name> tool (prompt) per
  SubAgentPalette entry, each invoking run.Ports.Palette as a CHILD of the
  current run (parentRunID = inv.RunID, inheriting caller + channel). A non-ok
  child status is surfaced to the parent with the partial output. nil-safe: no
  PaletteSource or empty palette → no delegation tools (unchanged behavior).
- executor.go: call it right after building the low-level toolbox.

Tests: the model calls skill__helper → routed through Palette with the right
name/caller/inputs/parent; nil palette → run still works.

Deferred to C0b (the remaining run.Ports executor wiring): Critic (soft-timeout
monitor + deadline binding + steer), Delivery (output egress for surfaces that
need executor-side delivery), Checkpointer (needs a majordomo message-history
hook to snapshot resumable state). The `.agent run` canary delivers its returned
Result.Output itself, so these aren't on its critical path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 09:28:01 -04:00
steve e37cf415de ci(gadfly): emit findings to gadfly-reports + bump image to sha-d7f364d
executus CI / test (push) Failing after 2m40s
Adds GADFLY_FINDINGS_URL / GADFLY_FINDINGS_TOKEN (user-scope secrets) so each review POSTs its run + findings to the gadfly-reports store, and bumps the pinned gadfly image to sha-d7f364d (the build carrying the findings-emit). Advisory only — emit failures never affect the review.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 09:12:46 -04:00
steve a87e7d2c72 fix: address verified gadfly P5 findings (canary robustness)
executus CI / test (pull_request) Failing after 3s
executus CI / test (push) Failing after 1m9s
All 3 cloud models converged (all "minor" — example code, no blocking):
- Consolidate: a model whose every lens errored now reads "review incomplete",
  not a misleading "no issues found" (all 3 models). + test.
- Consolidate: swarm-cancelled (unattributed) cells now surface a "swarm
  cancelled — N cell(s) did not run" banner instead of vanishing (all 3). + test.
- main: io.ReadAll(os.Stdin) error is surfaced (all 3); a TTY stdin no longer
  hangs forever (TTY guard, minimax).
- providerOf: a bare tier name now keys its own PerKey bucket instead of all
  bare tiers collapsing onto "tier" (minimax, glm-5.2) — distinct tiers throttle
  independently.
- Review doc reworded (the closure, not fanout, carries per-cell errors).

Left as documented example-scope behavior: no per-cell timeout (caller supplies
ctx), unknown-severity → lowest rank (no crash).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 00:34:01 -04:00
steve ea9475da54 P5: light-tier canary — gadfly-shaped reviewer on executus core
executus CI / test (pull_request) Failing after 1m5s
Adversarial Review (Gadfly) / review (pull_request) Successful in 8m18s
examples/reviewer proves the core is sufficient for a static-binary light host
(gadfly's shape) with NO batteries:
- config.Env + model.Configure  -> env-driven model fleet + tier overrides
- model.ParseModelForContext    -> tier resolution + failover
- fanout.Run (PerKey caps)      -> N models x M lenses swarm, per-provider bound
- model.GenerateWith[T]         -> structured findings per (model, lens) cell
- Consolidate                   -> one verdict-led report section per model

Hermetic test runs the full 2x3 swarm against majordomo's fake provider and
asserts the consolidated verdicts. A go list -deps CI check asserts the canary
imports ZERO batteries (the light-tier invariant) — gadfly's go.sum stays free
of gorm/redis/discordgo/sqlite. README + docs updated.

This is the canary; migrating the LIVE gadfly repo onto executus core is a
follow-up (kept separate to not destabilize the active reviewer).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 00:22:02 -04:00
steve dc2d4ec425 P4c: remaining batteries — checkpoint + schedule + critic
executus CI / test (push) Failing after 1m6s
Completes the P4 battery set (squashed onto main from phase-4c-batteries).
- checkpoint/: run.Checkpointer durable-resume (CheckpointStore + throttled
  handle + Memory).
- schedule/: generic cron Runner (Tick/Loop; no cron grammar of its own).
- critic/: two-tier timeout watchdog (run.Critic) + Escalator policy seam +
  ExtendOnce default.
Includes the verified gadfly #6 fixes (ExtendOnce per-run, Kill-sticky, watch
panic-recovery; checkpoint throttle-after-success; schedule Next-before-Run +
nil-guard + Loop recovery).

P4 battery set complete: audit, budget, persona, skill, checkpoint, schedule,
critic — each nil-safe, each with a default, each core-import-clean. Executor
wiring for Critic/Checkpointer remains a P2 follow-up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 00:15:32 -04:00
steve c8559676ed P4b: skill noun + contrib/store (SQLite for budget/persona/skill/audit)
executus CI / test (push) Has been cancelled
Merges the skill half of the persona/skill pair plus the second nested module.
(Squashed onto main from phase-4b-skill; the audit/budget/persona batteries it
was stacked on already landed via the P4 merge.)

- skill/: clean-redesign Skill noun + LEAN SkillStore (lifecycle/versions/
  schedule only) + ToRunnable + Memory default.
- contrib/store/: separate go.mod carrying modernc.org/sqlite, so the driver
  never enters the core go.sum. db.Budget()/Personas()/Skills()/Audit() back
  all four store seams (JSON-blob + indexed columns; round-trip tested).
  Includes the verified gadfly #5 fixes (AppendVersion tx+UNIQUE+error,
  Mark*ScheduledRun atomic json_set, busy_timeout, NaN guard).
- CI: builds + tests the nested module and asserts it owns the sqlite driver.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 00:15:00 -04:00
steve d82cef46b4 fix: address verified gadfly P4/#4 findings (audit/budget/persona)
executus CI / test (push) Failing after 1m4s
Security (all 3 models — HIGH): audit OnTool persisted raw tool args + results
verbatim for the very tools the OnStep narration-redaction flags as secret
(mcp_call/email_send/http_*) — the args/results are what CARRY the secret, so
they landed in skill_run_logs unredacted. Factored the predicate into
isSecretTool() (single source of truth) and OnTool now emits
args_redacted/result_redacted (+ lengths) for secret tools. Test asserts no
secret reaches the log. (persona) webhook_ip_allowlist entries are now
CIDR/IP-validated at load (malformed dropped + warned) instead of accepted raw.

Contract correctness (glm-5.2 + deepseek) — audit Memory now honors its
documented Storage contract: ListChildrenByParent/ListFinishedRunsBefore return
oldest-first; WalkParentChain returns root-first and honors MaxParentChainDepth;
ListRunsFiltered clamps limit (<=0 or >500 -> 50); ListFinishedRunsBefore with
limit<=0 returns none; an explicit RunFilter.Status (incl. "dry_run") matches
regardless of IncludeDryRun; LastRunBySkills counts only status=="ok" unless
includeFailed. (PurgeOlderThan's FinishedAt key is the SAFE behavior — in-flight
runs retained — so the doc was aligned to it, not the impl.)

Error-handling: appendLog now uses a bounded context (auditAppendTimeout=3s) so
a hung backend can't block the run goroutine on the hot path; Sink.StartRun
logs its (still best-effort) failure instead of swallowing it; budget Memory.Get
uses RLock (RWMutex); budget package doc fixed (was skillexec's); Check uses the
budgetWindow constant, not a duplicated literal.

Triaged false-positive: NewNoOpBudget returning BudgetTracker is assignable to
run.Budget (identical method sets) — no change needed.

Core go.sum still free of host/DB deps.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 00:12:19 -04:00
steve 2260480c81 P4: persona noun — Agent + ToRunnable bridge + Memory store
The headline P4 piece (clean redesign): the Agent persona noun, decoupled from
its Discord shell.
- agent.go/storage.go/builtin_loader.go moved from mort's pkg/logic/agents; the
  Storage seam drops the Discord CommandBindingStorage embedding (a host
  concern). The host-entangled files (commands, chatbot_provider, command-
  binding dispatcher, personalization, system) stay in mort.
- runnable.go: Agent.ToRunnable() lowers a persona into run.RunnableAgent — the
  bridge that lets run.Executor run a persona without importing this battery
  (the inversion of agentexec.Run(*agents.Agent)).
- memory.go: NewMemory() — zero-dep in-process persona Storage (all 11 CRUD +
  trigger-query methods).

Tests: ToRunnable field/phase mapping; Memory round-trip. CI invariant: core
imports ZERO from persona.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 00:12:19 -04:00
steve 9116abcae2 P4: budget battery — DBBudget (rolling 7-day) over run.Budget
Second Tier-2 battery, plugging into run.Ports.Budget:
- budget.go: skillexec's BudgetTracker / NoOpBudget / DBBudget moved clean
  (stdlib only). Check/Commit match run.Budget exactly (compile-time proof in
  run.go: NoOpBudget and *DBBudget are run.Budget).
- storage.go: the BudgetStorage seam + SkillBudget domain, split out of mort's
  GORM file (the GORM impl stays in mort).
- memory.go: NewMemory() — zero-dependency in-process BudgetStorage with the
  7-day rolling-window rollover in Add.

Tests: per-user cap enforced, window rolls over after 7 days, NoOp always
allows. CI invariant: core imports ZERO from the budget battery.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 00:12:19 -04:00
steve 4d2f85d139 P4: audit battery — run.Audit Sink + Writer + queryable Memory store
First Tier-2 battery, plugging into run.Ports.Audit:
- storage.go/writer.go: skillaudit's Storage interface + per-run Writer moved
  clean (only utils->fmt); the Writer already matches run.RunRecorder's shape.
- sink.go: Sink adapts a Storage to run.Audit (StartRun -> a run row + a Writer
  wrapped as run.RunRecorder, converting run.RunStats on Close). NewSink(nil) is
  equivalent to no audit. Compile-time proofs: Sink is run.Audit, recorder is
  run.RunRecorder.
- memory.go: NewMemory() — a zero-dependency, queryable in-process Storage
  (retains runs + logs; all 17 read/filter/purge/walk methods) so a light host
  gets run history with no setup. Mort keeps its GORM Storage; contrib/store
  adds durable SQLite at P4.

End-to-end test: wire audit.NewSink(audit.NewMemory()) into the executor, run an
agent, and the run is recorded with terminal status/output and queryable by
caller. CI invariant verified: core imports ZERO from the audit battery (proper
battery direction; battery imports core, never the reverse).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 00:12:19 -04:00
steve d0bd3ec3d9 fix: address verified gadfly P3 review (3-cloud fleet)
executus CI / test (push) Has been cancelled
All 3 cloud models converged on a real access-control bug; fixed it + the
other genuine findings (the false-positives were dropped):

Security (HIGH — all 3 models):
- create_file_url skipped ValidateScope: a same-skill caller could mint a
  PUBLIC url for a file scoped to another user/run. Now runs ValidateScope
  (admin-aware), skipped only for the descendant-grant case — mirroring the
  read tools.

Other real fixes:
- ValidateScope hard-coded `false` at every call site (admin branch dead) ->
  pass inv.CallerIsAdmin (the executor sets it via the host AdminPolicy; still
  false/fail-closed when no admin). Stale "no admin flag" comment corrected.
- create_file_url: ExpiresInSeconds clamped BEFORE the *time.Second multiply
  (huge values overflowed to a negative duration that slipped under the cap,
  minting already-expired tokens); swallowed json.Marshal error now returned.
- RegisterMeta: build the default budget WITH the configured MaxPerRun (was
  NewInMemorySearchBudget(nil) -> hardcoded 10, ignoring MetaDeps.MaxPerRun).
- classify: all-zero scores no longer return a false-positive top-1 winner;
  coerceClassifyScore uses strconv.ParseFloat (rejects trailing garbage like
  "50extra" that fmt.Sscanf silently accepted).
- file_delete: honor the descendant grant (parent can clean up a worker's
  artifacts) — was the lone cross-skill-reject-outright file tool.
- meta tools: input caps truncate at a UTF-8 rune boundary (truncateUTF8), not
  mid-rune.
- think: removed the dead `var _ = fmt.Errorf` import-keeper; file_save default
  aligned to 16 MiB (matched RegisterStore).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 00:11:54 -04:00
steve 78e6858751 P3: store group — kv_* + file_* tools (agent memory)
RegisterStore(reg, StoreDeps) registers the persistent-memory tools over the
host's KV and/or File backends:
- kv_get/set/list/delete (KVStorage seam)
- file_save/get/get_text/get_metadata/list/delete (FileStorage seam), plus
  file_search (FileSearcher) and create_file_url (FileTokenMinter) when wired.

Near-zero-config: Quota defaults to a generous static cap (staticQuota), the
per-value/per-file caps default, and the kv vs file groups register
independently (a host can take just one). Seams moved clean (interface-only):
kv_storage.go, quota_provider.go, file_descendant_grant.go. The default
in-memory KV/File backends come with contrib/store at P4.

Core go.sum still free of gorm/redis/discordgo/sqlite.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 00:11:54 -04:00
steve 1e201550b3 P3: meta + primitive tool group (think/now/cite + classify/extract/summarize)
Grow executus/tools into a real generic tool library:

- Register(reg): the always-available, zero-config tools — think, now (UTC
  unless a CurrentTimeProvider is wired), cite (inert unless a CitationStorage
  is wired). All nil-safe; a light host calls Register and is useful.
- RegisterMeta(reg, MetaDeps): the LLM-backed meta tools — classify,
  extract_entities, summarize — over the llmmeta helper. Budget defaults to the
  shipped in-memory per-run cap; Files optional; caps default.
- Seams moved (interface/type-only, no host coupling): research_providers.go
  (CurrentTimeProvider/CitationStorage/SearchBudget/PageExtractor/PDFFetcher/…)
  and file_storage.go (FileStorage + FileDomainMeta). Plus the in-memory budget
  default (research_defaults.go) and scope_validate.go.

calculate deferred (drags github.com/Krognol/go-wolfram + a module-path replace
— not worth it in the lean core for one tool). Core go.sum still free of
gorm/redis/discordgo/sqlite/wolfram.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 00:11:54 -04:00
steve df95425bb5 P3 (kickoff): generic tools/ library + end-to-end tool-using-agent test
Stand up executus/tools — the generic, host-agnostic tool library — and prove
the full pattern end to end:

- tools/tools.go: Register(reg) adds the always-available zero-dependency tools
  (currently `think`). A light host calls it and is immediately useful; backed
  tools (web/store/meta groups) will register via grouped registrars with
  nil-safe Deps as they land.
- tools/think.go: the `think` tool moved from mort (imports only executus/tool).
- tools/integration_test.go: end-to-end proof that the executor runs an agent
  which CALLS a registered tool — the fake model emits a `think` tool call, the
  executor dispatches it through the registry, the model finalises, and the step
  instrumentation captures the `think` step. Exercises the full tool-dispatch
  loop through run.Executor.

Stacked on phase-2-run-kernel (P3 needs run.Executor). Remaining P3: the
meta/web/net/store/compose groups + their Deps + default backends (splitting
mort's default.go grab-bag).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 00:11:54 -04:00
steve 16ddd90914 ci(gadfly): new build sha-d0de034 + per-lens concurrency
executus CI / test (push) Successful in 59s
Bump the gadfly image to sha-d0de034 (adds GADFLY_PROVIDER_LENS_CONCURRENCY)
and move ollama-cloud's concurrency from the MODEL axis to the LENS axis:
- GADFLY_PROVIDER_CONCURRENCY: ollama-cloud=1 (one model at a time)
- GADFLY_PROVIDER_LENS_CONCURRENCY: ollama-cloud=3 (its 3 lenses concurrent)

Net: still 3 models, but reviewed serially — the first model's consolidated
comment lands sooner and each model finishes faster, while the other two
models' comments arrive in series after it (instead of all 3 in parallel).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 22:57:14 -04:00
steve b35514dfaa ci(gadfly): cloud-only fleet (3 models, drop local Macs)
executus CI / test (push) Successful in 57s
Measured on the P2 review: the local Macs (m1/m5) took 26–29 min with lens
timeouts and found ZERO real bugs, while the two cloud models found every
genuine finding in 6–12 min. Drop the Macs; add glm-5.2:cloud as a third
cloud reviewer. Net: faster (~29→~12 min) and higher signal.

Models: minimax-m3:cloud, deepseek-v4-flash:cloud, glm-5.2:cloud
(ollama-cloud=3 concurrency). timeout-minutes 90→30.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 02:02:21 +00:00
steve 7b3da87c08 fix: address verified gadfly P2 findings (9 real of 18)
Independently verified all 18 gadfly findings against the code (18-agent
fan-out). Fixed the 9 real ones; the other 9 were false-positive /
hallucinated / valid-tradeoff (no change).

High:
- F1 nil model: a Models resolver returning (ctx,nil,nil) flowed into the
  agent loop and nil-panicked. Now a clean error (Run never panics). +test.
- F9 compactor data-leak: renderTranscript sent tool-call args verbatim to
  the summarizer (a possibly-different provider/tier); secret-bearing tool
  args (mcp_call/email_send/http_*/webhook_*) are now redacted, with a doc
  note that result bodies still flow (summary needs them).

Medium/minor:
- F2 compactor error path returned the folded slice, not the original msgs
  (contradicting the documented non-fatal contract) -> return msgs.
- F3 RunStats.Status only ok/error; now timeout (DeadlineExceeded) /
  cancelled (Canceled) via statusFor. +test.
- F4 step-zip emitted empty-name "ghost" steps when results>calls; now pairs
  min(calls,results) only.
- F5 SetIteration was never called -> RunState.Iteration always 0; the step
  observer now updates it each loop.
- F6 matchPending fallback was LIFO; now FIFO (matches the per-key queue).
- F7 estimateTokens had no default arm (future Part kinds counted as 0);
  unknown parts now counted conservatively.
- F8 cloud_sync silently truncated >1MiB responses -> opaque JSON error; now
  a clear "response exceeded N bytes" via readCapped.
- F12 step observer captured the caller ctx; now the merged runCtx.
- F13 compaction onFire was nil (doc claimed it logged); now wired to
  audit LogEvent("compaction_fired").
- F11 (no pre-dispatch hook in majordomo) documented honestly as a known
  limitation; F18 UsageSink doc clarified cache tokens are subsets of input.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 02:02:21 +00:00
steve dfbc5a42b9 P2: run.Executor — executus is runnable
The capstone of the run kernel: run.Executor.Run(ctx, RunnableAgent, inv)
ties model resolution + the tool registry + majordomo's agent loop +
context compaction + run-bounding + step/audit instrumentation into one
path, with every host concern behind the nil-safe run.Ports.

- run/executor.go: New(Config{Registry, Models, Defaults, Ports, Compactor,
  ContextTokens, SystemHeader}) + Run -> Result{RunID, Output, Steps, Usage,
  Err}. Budget gate (pre-run), model resolve, Audit StartRun/recorder
  (satisfies RunTally, stamped on inv.RunState), toolbox build, step observer
  (zips tool calls/results -> emitter + recorder.OnStep/OnTool), V10
  detached-MaxRuntime context with caller-cancel merged back, compaction wired
  from ContextTokens×ratio, audit Close + Budget Commit on a detached cleanup
  ctx. Zero Ports = a bounded in-memory run (gadfly's case).
- run/executor_test.go: hermetic end-to-end run against majordomo's fake
  provider (hello-world), Budget-rejection (no model call), Audit-port wiring
  (StartRun + Close with terminal status/output). All green under -race.
- examples/minimal upgraded to the real "hello, agentic world" (~15 lines:
  Configure tiers -> run.New -> Run -> print). README/CLAUDE.md updated.

Remaining P2 follow-ups (incremental): wire Critic/Checkpointer/PaletteSource/
Delivery into the loop, multi-phase Pipelines, and the no-tools direct path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 02:02:21 +00:00
steve 130c2bdfab P2: move compactor -> compact/ + step instrumentation -> run/steps.go
- compact/compactor.go: the per-run stateful context compactor (token-threshold
  gate, fast-tier middle summarisation, fold memory) lifted from mort's
  skillexec/compactor.go. Self-contained; its only dependency is a ModelResolver
  func (model.ParseModelForContext satisfies it) + a token threshold.
- run/steps.go: the step-emission/instrumentation (stepEmitter, tool->kind/
  summary mapping with redaction, Result.Steps accumulation) from agentexec,
  repointed onto executus/tool.

Both build green. With the run-loop mechanics, RunnableAgent DTO, run.Ports,
compactor, and step instrumentation now all in place, the remaining P2 work is
the run.Executor itself (wiring these + majordomo's agent loop), which makes
executus runnable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 02:02:21 +00:00
steve d9b44387f5 fix: address gadfly P1 review (3 low-risk findings)
Triaged gadfly's P1 review (advisory). Fixed the three clearly-correct,
low-risk items; the rest were pre-existing mort behavior or theoretical:

- model/call.go: recordUsage dropped fully-cached responses (input==0 &&
  output==0 early-return missed CacheRead/CacheWrite-only usage, which
  Anthropic/OpenAI prompt-caching bills). Guard now also checks cache tokens.
- llmmeta/helper.go: recordLedger swallowed Storage.RecordMetaCall errors;
  now logs them (slog.Warn) so a non-logging Storage impl can't silently drop
  audit rows.
- model/cloud_sync.go: the ollama.com limit-cache used unbounded io.ReadAll;
  wrapped both reads in io.LimitReader(1 MiB) so a misbehaving endpoint can't
  exhaust memory before the 15s timeout.

Noted-not-fixed (follow-ups / pre-existing mort semantics): tier_not_allowed
ledger label on resolution failure, unknown-model usage attribution, the
cloud_sync https scheme allowlist, and several theoretical/cosmetic items.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 02:02:21 +00:00
steve e0e2c0451a ci: sync gadfly review config to mort's foreman-provider setup
Mirror mort's updated adversarial-review.yml: m1/m5 pulled in via the
GADFLY_ENDPOINT_M1/_M5 secrets using gadfly's "foreman" provider type
(providers m1/m5; models m1/qwen3:14b, m5/qwen3.6:35b-mlx), 2 cloud models,
3-lens suite, pinned to the gadfly :sha-6e3a83c image. Header adjusted for
executus; functional config identical to mort's tested version.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 02:02:21 +00:00
steve c732a677df P2: define nil-safe run.Ports (the inversion spine)
Add run/ports.go: the host seams the executor will consume, every one
nil-safe so a light host runs with the zero Ports (no persistence/audit/
budget/critic/delegation/delivery) and a heavy host wires each to a battery.

Ports mirror mort's existing interfaces so the batteries implement them
directly:
- Audit + RunRecorder (mort skillaudit.Storage/Writer): StartRun -> per-run
  recorder (OnStep/OnTool/LogEvent/Close), recorder satisfies RunTally.
- Budget (mort skillexec.BudgetTracker): Check / Commit.
- Critic + CriticHandle (mort agentcritic): Monitor -> handle with
  RecordStep/RecordToolStart/Steer/Deadline/Stop (the loop wiring finalizes
  with the executor merge).
- Checkpointer (mort agentexec.RunCheckpointer): Save/Complete/Fail.
- PaletteSource (mort SkillInvokerForPalette + AgentInvokerForPalette):
  Resolve/Invoke skill + agent delegation.
Plus host-neutral RunInfo / RunStats.

This completes the P2 inversion DESIGN; the agentexec+skillexec ->
run.Executor merge that consumes these Ports is the remaining P2 work.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 02:02:21 +00:00
steve fa644f1826 P2 (foundation): run-loop mechanics + RunnableAgent DTO
Stand up the executus/run kernel foundation, decoupled from mort:

- runengine.go: the shared run-loop scaffolding (MergeCancellation,
  CleanupContextTimeout, RunFinalizer/FireFinalizers, RunStateAccessor) moved
  from mort. The accessor's *skillaudit.Writer dependency is inverted to a
  narrow run.RunTally interface (TokenStats + ToolCallsCount) — the kernel
  reads live tallies without importing the audit battery.
- submit.go: the legacy submit-capture compat tool (stdlib + majordomo/llm).
- agent.go: RunnableAgent DTO — the kernel's view of "a thing to run" (tier,
  prompt, caps, palette, phases, critic config). The persona Agent and saved
  Skill will LOWER into this DTO so the kernel never imports a noun battery.
  This is the spine of the agentexec.Run(*agents.Agent) inversion.

run/ builds with only majordomo + executus/tool. The executor merge
(agentexec+skillexec -> run.Executor) and the nil-safe run.Ports
(Audit/Critic/Budget/Checkpointer/PaletteSource) are the next P2 block.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 02:02:21 +00:00
steve b424261aca P1: model layer (convar->config inversion) + llmmeta
executus CI / test (pull_request) Successful in 58s
Adversarial Review (Gadfly) / review (pull_request) Successful in 26m27s
executus CI / test (push) Successful in 1m2s
Lifts mort's pkg/logic/llms into executus/model, decoupled from mort:

- tiers.go: the tier resolver now reads a host-supplied config.Source under
  "model.tier.<name>" with host-supplied fallbacks (Configure(cfg, defaults,
  ttl)), instead of convar.Manager. Tier NAMES + specs are host config; the
  resolution mechanism (cache, reasoning-suffix dialect, chain validation) is
  generic. No tier names hard-coded in the harness.
- sink.go: usage/trace recording inverted off mort's llmusage/llmtrace into
  UsageSink / TraceSink seams + a model-owned Span, with nil-safe context
  attribution helpers (WithModel/WithTraceID/WithUsageTool/WithUsageUser).
  Both sinks optional (nil = off) so a light host records nothing.
- lane decoration repointed to executus/lane; utils.Errorf -> fmt.Errorf.
- call.go keeps GenerateWith[T] (instrumented structured output) — this is the
  structured-output primitive; no separate structured/ package.
- llmmeta moved over model/ (the meta-LLM helper: tier allowlist + JSON retry
  + ledger). Its tests configure a minimal tier table via TestMain.

New tests cover the inversion: config overrides fallback, tier registration,
reasoning-suffix survival, nested-tier rejection, nil-sink no-ops.

Full module: go build/vet/test -race green; core go.sum still free of
gorm/redis/discordgo/sqlite.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 19:47:13 -04:00
115 changed files with 16952 additions and 51 deletions
+32 -16
View File
@@ -1,10 +1,11 @@
# Gadfly — agentic adversarial PR reviewer (https://gitea.stevedudenhoeffer.com/steve/gadfly).
#
# Runs the published Gadfly image (pinned to :v1) as a specialist swarm and posts
# Runs the published Gadfly image (pinned to an immutable :sha- tag — act_runner
# caches :latest, and this build is what carries foreman provider-type support)
# as a specialist swarm and posts
# ONE consolidated review comment as gitea-actions. Advisory only — never blocks a
# merge. This reviews executus PRs (same setup as mort: m1/m5 locals + 2 cloud,
# 3-lens suite). Gadfly is a simple system — treat its findings as advisory and
# double-check before acting.
# merge. This reviews executus PRs with 3 ollama-cloud models (3-lens suite). Gadfly
# is a simple system — findings are advisory; always double-check before acting.
name: Adversarial Review (Gadfly)
@@ -40,24 +41,33 @@ jobs:
|| github.actor == 'fizi'
|| github.actor == 'dazed'))
runs-on: ubuntu-latest
# Full fleet (2 cloud + 2 local Macs, all running concurrently) reviewing
# every PR with the 3-lens suite — the slow local lanes dominate wall time.
# Full fleet: 3 cloud (lens fan-out) + M1/M5 Macs via foreman. The slow local
# lanes dominate wall time, so allow plenty of headroom.
timeout-minutes: 90
steps:
- uses: docker://gitea.stevedudenhoeffer.com/steve/gadfly:v1
- uses: docker://gitea.stevedudenhoeffer.com/steve/gadfly:sha-d7f364d
env:
GITEA_API: ${{ github.server_url }}/api/v1/repos/${{ github.repository }}
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
OLLAMA_CLOUD_API_KEY: ${{ secrets.OLLAMA_CLOUD_API_KEY }}
# Local Ollama boxes (each its own lane, cap 1). NOTE: both Macs must be
# awake/reachable for their reviews to run; if a box is offline, that
# model's comment shows an error and the others still post.
GADFLY_ENDPOINT_M1PRO: "ollama|http://192.168.0.175:11434"
GADFLY_ENDPOINT_M5MAX: "ollama|http://192.168.0.173:11434"
# 2 cloud (parallel) + M1 Pro + M5 Max — one consolidated comment each.
GADFLY_MODELS: "minimax-m3:cloud,deepseek-v4-flash:cloud,m1pro/qwen3:14b,m5max/qwen3.6:35b-mlx"
# cloud runs 2 at once; each Mac one at a time; all three lanes parallel.
GADFLY_PROVIDER_CONCURRENCY: "ollama-cloud=2,m1pro=1,m5max=1"
# Local Macs, reached through their foreman queues (native Ollama on the
# wire). GADFLY_ENDPOINT_M5 registers provider "m5",
# each a foreman-preset Ollama client at the secret's URL, of the form:
# foreman|https://<foreman-host>|<token>
# Needs an image with foreman provider-type support (this one). If a Mac
# is offline that model's comment shows an error and the others still post.
# (Gitea secrets aren't auto-exposed — map each explicitly.)
GADFLY_ENDPOINT_M5: ${{ secrets.GADFLY_ENDPOINT_M5 }}
# Full fleet: 3 cloud + M1 Pro + M5 Max. The Macs are back so the
# gadfly-reports scoreboard can quantify whether they earn their keep
# (they previously took 2629 min for ZERO real findings — now measured).
# Cloud concurrency lives in the LENSES: one cloud model at a time
# (ollama-cloud=1) with its 3 lenses concurrent (LENS ollama-cloud=3) so
# its comment lands sooner; each Mac runs one model, lenses serial (its
# foreman queue serializes anyway). All three provider lanes run parallel.
GADFLY_MODELS: "minimax-m3:cloud,glm-5.2:cloud,glm-5.1:cloud,kimi-k2.7-code:cloud,deepseek-v4-pro:cloud,nemotron-3-super:cloud,gpt-oss:120b-cloud,qwen3-coder:480b-cloud,gemma4:cloud,m5/qwen3.6:35b-mlx"
GADFLY_PROVIDER_CONCURRENCY: "ollama-cloud=3,m5=1"
GADFLY_PROVIDER_LENS_CONCURRENCY: "ollama-cloud=3"
# Default => the 3-lens suite (security, correctness, error-handling).
# Set the repo var GADFLY_SPECIALISTS to override (csv / "all" / "auto").
GADFLY_SPECIALISTS: ${{ vars.GADFLY_SPECIALISTS || 'security,correctness,error-handling' }}
@@ -66,6 +76,12 @@ jobs:
GADFLY_MAX_STEPS: "14"
# Allow-list for the comment trigger (mirrors the job-level if: guard).
GADFLY_ALLOWED_USERS: "steve,fizi,dazed"
# --- findings telemetry: POST runs + findings to the gadfly-reports store ---
# Advisory & off unless GADFLY_FINDINGS_URL is set; failures only log to
# stderr and never affect the review. GADFLY_REPO / GADFLY_PR are derived
# in-container; the URL + token are user-scope secrets.
GADFLY_FINDINGS_URL: ${{ secrets.GADFLY_FINDINGS_URL }}
GADFLY_FINDINGS_TOKEN: ${{ secrets.GADFLY_FINDINGS_TOKEN }}
# --- event context (leave as-is) ---
EVENT_NAME: ${{ github.event_name }}
PR: ${{ github.event.pull_request.number || github.event.issue.number || github.event.inputs.pr_number }}
+23
View File
@@ -103,3 +103,26 @@ jobs:
exit 1
fi
echo "OK: core go.sum is free of host/DB dependencies."
- name: Light-tier canary imports no battery
run: |
# examples/reviewer is gadfly's shape on the CORE only. If it ever
# pulls in a battery (audit/budget/persona/skill/critic/schedule/
# checkpoint/contrib), the light path has regressed.
LEAK=$(go list -deps ./examples/reviewer/... | grep -E 'executus/(audit|budget|persona|skill|critic|schedule|checkpoint|contrib)' || true)
if [ -n "$LEAK" ]; then
echo "ERROR: light-tier canary pulled in a battery:"; echo "$LEAK"; exit 1
fi
echo "OK: examples/reviewer is core-only."
- name: contrib/store (nested SQLite module — isolated from core)
run: |
# contrib/store is a SEPARATE module carrying modernc.org/sqlite; the
# core's `go test ./...` doesn't reach it. Build + test it on its own,
# and confirm it DOES carry the driver the core forbids (proof the
# split works: persistence lives here, not in the core go.sum).
cd contrib/store
go build ./...
go test -race -count=1 -timeout 5m ./...
grep -qE 'modernc.org/sqlite' go.sum || { echo "ERROR: contrib/store should carry the sqlite driver"; exit 1; }
echo "OK: contrib/store builds, tests pass, and owns the SQLite dep."
+47 -17
View File
@@ -43,28 +43,58 @@ CORE (majordomo + stdlib):
fanout/ programmatic N×M swarm [P0 ✓]
deliver/ output egress seam (+ Discard/Stdout) [P0 ✓]
identity/ caller identity seams [P0 ✓]
run/ progress bridge now; the executor kernel + [P0 partial]
nil-safe Ports + RunnableAgent later [P2]
run/ run.Executor is RUNNABLE: model-resolve + [P2 core ✓]
toolbox + majordomo loop + compaction +
run-bounding (V10 detached timeout) + step/
audit observers + Budget gate; RunnableAgent
DTO + nil-safe run.Ports. Palette delegation +
Critic (monitor/deadline/steer) + Delivery
WIRED. Follow-ups: Checkpointer (needs a
majordomo msg-history hook), Phases [C0c]
dispatchguard/ loop/depth/fan-out caps [P0 ✓]
pendingattach/ attachment dedupe [P0 ✓]
tool/ registry + 3-stage permissions + ssrf [P1 ✓]
model/ config-driven tier resolution over majordomo [P1]
llmmeta/ shared meta-LLM helper (moves with model/) [P1]
compact/ context compactor (WithCompactor hook) [P2]
tools/{web,net,store,compose,meta,comms} generic tools [P3]
structured/ Generate[T] convenience over majordomo [P1]
model/ config-driven tier resolution over majordomo [P1]
(convar->config.Source; UsageSink/TraceSink seams; GenerateWith[T]
structured output — no separate structured/ pkg)
llmmeta/ shared meta-LLM helper over model/ [P1 ✓]
compact/ context compactor (WithCompactor hook) [P2 ✓]
tools/ generic tool library: Register (think/now/ [P3 ✓]
cite, zero-config) + RegisterMeta (classify/
extract_entities/summarize) + RegisterStore
(kv_*/file_*, default static quota); seams in
research_providers.go/file_storage.go/
kv_storage.go/quota_provider.go. End-to-end
"agent calls a tool" test green. Remaining
(deferred): web/net/compose groups + backends
BATTERIES (opt-in siblings, each nil-safe + a default):
persona/ Agent noun + AgentStore seam + yml loader [P4]
skill/ rich Skill + SkillStore seam + toml loader [P4]
audit/ run-trace Sink (+ Noop/Slog) [P4]
critic/ two-tier timeout state machine + Escalator [P4]
schedule/ cron runner cores [P4]
checkpoint/ durable resume seam [P4]
budget/ rolling-window tracker (+ NoOp) [P4]
persona/ Agent noun + Storage seam + builtin loader [P4]
+ ToRunnable() bridge to run.RunnableAgent +
Memory default (host: chatbot/commands/personalization)
skill/ Skill noun + LEAN SkillStore (lifecycle/ [P4]
versions/schedule, NOT mort's 60-method
monster) + ToRunnable + Memory default
audit/ run.Audit Sink + Writer + queryable Memory [P4]
default (skillaudit Storage iface; GORM stays in mort)
critic/ two-tier timeout watchdog (run.Critic) + [P4 ✓]
Escalator policy seam + ExtendOnce default
schedule/ generic cron Runner (Tick/Loop over a wired [P4 ✓]
Due/Run/Mark/Next; no cron grammar of its own)
checkpoint/ CheckpointStore + run.Checkpointer handle [P4 ✓]
(throttled Save/Complete/Fail) + Memory
budget/ DBBudget rolling-7d + NoOp (run.Budget); [P4 ✓]
BudgetStorage iface + Memory default
contrib/store/ SECOND module (+ modernc.org/sqlite): [P4]
in-memory + pure-Go SQLite impls of every *Store seam
contrib/store/ SECOND module (+ modernc.org/sqlite): [P4]
pure-Go SQLite impls of ALL store seams: budget +
persona + skill + audit (JSON-blob+indexed cols,
round-trip tested). CI proves the driver lands HERE,
not in the core go.sum.
NOTE: critic/checkpoint executor wiring (run.Ports.Critic /
.Checkpointer call sites) is a P2 follow-up — the batteries +
defaults exist ahead of that wiring.
```
### The one architectural move
@@ -86,7 +116,7 @@ repackaging.
P0 module + zero-coupling moves + core seams (this) → P1 tool registry + model →
P2 run kernel + Ports inversion → P3 generic tools + defaults → P4 persona/skill
redesign + batteries + SQLite store → P5 gadfly on core (light-tier canary) → P6
redesign + batteries + SQLite store → P5 gadfly-on-core canary (examples/reviewer ✓) → P6
rewire mort + tag v0.1.0. The mort-side rewrite reuses mort's existing
`mort_*_adapters.go` wall as the host adapter layer.
+16 -5
View File
@@ -31,15 +31,26 @@ bot) — mort and gadfly are the first two consumers (heavy and light). See
[mort]: https://gitea.stevedudenhoeffer.com/steve/mort
**Available today (P0):**
**Available today:**
- `run/`**executus is runnable.** `run.Executor` ties model resolution, the
tool registry, majordomo's agent loop, context compaction, run-bounding, and
step/audit instrumentation into one `Run(ctx, RunnableAgent, inv) Result`, with
every host concern behind a nil-safe `run.Ports` (Audit/Budget/Critic/
Checkpointer/PaletteSource/Delivery). See `examples/minimal`.
- `model/` — config-driven tier resolution + failover over majordomo, with
pluggable `UsageSink`/`TraceSink` and `GenerateWith[T]` structured output.
- `tool/` — the tool registry + 3-stage permission model + SSRF guard.
- `compact/` — the per-run context compactor.
- `lane/` — bounded worker pool with fair-share queueing (run- and
provider-concurrency).
- `fanout/` — programmatic N×M swarm with bounded global + per-key concurrency.
- `config/` — the host config seam (`Source`) with an env-var default.
- `deliver/` — the output-egress seam with `Discard`/`Stdout` defaults.
- `identity/` — caller-identity seams (`AdminPolicy`, `MemberResolver`).
- `dispatchguard/`, `pendingattach/`, `run/progress.go` — run-safety primitives.
- `config/`, `deliver/`, `identity/` — host seams (config / output / identity),
each with a shipped default.
- `dispatchguard/`, `pendingattach/` — run-safety primitives.
- `examples/reviewer` — a **gadfly-shaped PR reviewer on the core only** (env-config
model fleet → `fanout` N×M swarm → `model.GenerateWith[T]` structured findings →
consolidation), the light-tier canary; CI asserts it pulls in no battery.
## Design
+78
View File
@@ -0,0 +1,78 @@
package audit_test
import (
"context"
"testing"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/majordomo/provider/fake"
"gitea.stevedudenhoeffer.com/steve/executus/audit"
"gitea.stevedudenhoeffer.com/steve/executus/run"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
// TestAuditBatteryEndToEnd wires the audit battery (Memory storage) into
// run.Ports.Audit, runs an agent, and verifies the run was recorded and is
// queryable — proving Sink/Writer/Memory satisfy the core seams end to end.
func TestAuditBatteryEndToEnd(t *testing.T) {
mem := audit.NewMemory()
fp := fake.New("fake")
fp.Enqueue("m", fake.Reply("the answer"))
m, err := fp.Model("m")
if err != nil {
t.Fatal(err)
}
ex := run.New(run.Config{
Registry: tool.NewRegistry(),
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) {
return ctx, m, nil
},
Ports: run.Ports{Audit: audit.NewSink(mem)},
})
res := ex.Run(context.Background(),
run.RunnableAgent{ID: "agent-1", Name: "a", ModelTier: "m"},
tool.Invocation{RunID: "run-xyz", CallerID: "caller-1"},
"question")
if res.Err != nil {
t.Fatalf("run error: %v", res.Err)
}
// The run was recorded with a terminal status + output.
got, err := mem.GetRun(context.Background(), "run-xyz")
if err != nil {
t.Fatalf("GetRun: %v", err)
}
if got.Status != "ok" {
t.Errorf("status = %q, want ok", got.Status)
}
if got.Output != "the answer" {
t.Errorf("output = %q, want %q", got.Output, "the answer")
}
if got.FinishedAt == nil {
t.Error("FinishedAt should be set after the run")
}
if got.SkillID != "agent-1" {
t.Errorf("SkillID = %q, want agent-1 (the subject id)", got.SkillID)
}
// And it is queryable by caller.
runs, err := mem.ListRunsByCaller(context.Background(), "caller-1", 10)
if err != nil {
t.Fatalf("ListRunsByCaller: %v", err)
}
if len(runs) != 1 || runs[0].ID != "run-xyz" {
t.Errorf("ListRunsByCaller = %+v, want [run-xyz]", runs)
}
}
// TestNilSinkRecordsNothing: NewSink(nil) is equivalent to no audit.
func TestNilSinkRecordsNothing(t *testing.T) {
s := audit.NewSink(nil)
if rec := s.StartRun(context.Background(), run.RunInfo{RunID: "r"}); rec != nil {
t.Error("NewSink(nil).StartRun should return a nil recorder")
}
}
+280
View File
@@ -0,0 +1,280 @@
package audit
import (
"context"
"sort"
"sync"
"time"
)
// Memory is an in-process Storage: it retains runs + logs in memory so a light
// host (or a test) gets queryable run history with zero setup. It is bounded
// only by process memory — a host that runs forever should PurgeOlderThan
// periodically, or use a persistent Storage. Construct with NewMemory.
//
// Mort uses its GORM/MySQL Storage; contrib/store adds a durable SQLite one.
// Memory is the zero-dependency default behind audit.NewSink(audit.NewMemory()).
type Memory struct {
mu sync.RWMutex
order []string // run ids in insertion order
runs map[string]SkillRun // by run id
logs map[string][]SkillRunLog // by run id
}
// NewMemory returns an empty in-memory Storage.
func NewMemory() *Memory {
return &Memory{runs: map[string]SkillRun{}, logs: map[string][]SkillRunLog{}}
}
var _ Storage = (*Memory)(nil)
func (m *Memory) Initialize(context.Context) error { return nil }
func (m *Memory) StartRun(_ context.Context, run SkillRun) error {
m.mu.Lock()
defer m.mu.Unlock()
if _, ok := m.runs[run.ID]; !ok {
m.order = append(m.order, run.ID)
}
m.runs[run.ID] = run
return nil
}
func (m *Memory) FinishRun(_ context.Context, runID string, s RunStats) error {
m.mu.Lock()
defer m.mu.Unlock()
r, ok := m.runs[runID]
if !ok {
return ErrNotFound
}
now := time.Now()
r.FinishedAt = &now
r.Status = s.Status
r.Output = s.Output
r.Error = s.Error
r.ToolCallsCount = s.ToolCalls
r.RuntimeSeconds = s.RuntimeSeconds
r.TotalInputTokens = s.InputTokens
r.TotalOutputTokens = s.OutputTokens
r.TotalThinkingTokens = s.ThinkingTokens
m.runs[runID] = r
return nil
}
func (m *Memory) AppendLog(_ context.Context, log SkillRunLog) error {
m.mu.Lock()
defer m.mu.Unlock()
m.logs[log.RunID] = append(m.logs[log.RunID], log)
return nil
}
func (m *Memory) GetRun(_ context.Context, runID string) (*SkillRun, error) {
m.mu.RLock()
defer m.mu.RUnlock()
r, ok := m.runs[runID]
if !ok {
return nil, ErrNotFound
}
return &r, nil
}
func (m *Memory) ListLogsByRun(_ context.Context, runID string) ([]SkillRunLog, error) {
m.mu.RLock()
defer m.mu.RUnlock()
ls := append([]SkillRunLog(nil), m.logs[runID]...)
sort.SliceStable(ls, func(i, j int) bool { return ls[i].Sequence < ls[j].Sequence })
return ls, nil
}
// newestFirst returns the retained runs in reverse insertion order, optionally
// filtered. Caller holds at least RLock.
func (m *Memory) newestFirst(keep func(SkillRun) bool) []SkillRun {
out := make([]SkillRun, 0, len(m.order))
for i := len(m.order) - 1; i >= 0; i-- {
r := m.runs[m.order[i]]
if keep == nil || keep(r) {
out = append(out, r)
}
}
return out
}
// oldestFirst returns the retained runs in insertion (oldest-first) order,
// optionally filtered. Caller holds at least RLock.
func (m *Memory) oldestFirst(keep func(SkillRun) bool) []SkillRun {
out := make([]SkillRun, 0, len(m.order))
for _, id := range m.order {
r := m.runs[id]
if keep == nil || keep(r) {
out = append(out, r)
}
}
return out
}
func page(rs []SkillRun, offset, limit int) []SkillRun {
if offset < 0 {
offset = 0
}
if offset >= len(rs) {
return nil
}
rs = rs[offset:]
if limit > 0 && limit < len(rs) {
rs = rs[:limit]
}
return rs
}
func (m *Memory) ListRunsBySkill(ctx context.Context, skillID string, limit int) ([]SkillRun, error) {
return m.ListRunsBySkillPaginated(ctx, skillID, 0, limit, false)
}
func (m *Memory) ListRunsBySkillPaginated(_ context.Context, skillID string, offset, limit int, includeDryRun bool) ([]SkillRun, error) {
m.mu.RLock()
defer m.mu.RUnlock()
return page(m.newestFirst(func(r SkillRun) bool {
return r.SkillID == skillID && (includeDryRun || r.Status != "dry_run")
}), offset, limit), nil
}
func (m *Memory) CountRunsBySkill(_ context.Context, skillID string, includeDryRun bool) (int64, error) {
m.mu.RLock()
defer m.mu.RUnlock()
return int64(len(m.newestFirst(func(r SkillRun) bool {
return r.SkillID == skillID && (includeDryRun || r.Status != "dry_run")
}))), nil
}
func (m *Memory) ListRunsByCaller(_ context.Context, callerID string, limit int) ([]SkillRun, error) {
m.mu.RLock()
defer m.mu.RUnlock()
return page(m.newestFirst(func(r SkillRun) bool {
return r.CallerID == callerID && r.Status != "dry_run"
}), 0, limit), nil
}
func (m *Memory) matchesFilter(r SkillRun, f RunFilter) bool {
if f.Status != "" {
if r.Status != f.Status {
return false
}
// An explicit Status (even "dry_run") matches regardless of IncludeDryRun.
} else if !f.IncludeDryRun && r.Status == "dry_run" {
return false
}
if f.SkillID != "" && r.SkillID != f.SkillID {
return false
}
if f.CallerID != "" && r.CallerID != f.CallerID {
return false
}
if f.ChannelID != "" && r.ChannelID != f.ChannelID {
return false
}
if f.TopLevelOnly && r.ParentRunID != "" {
return false
}
if !f.Since.IsZero() && r.StartedAt.Before(f.Since) {
return false
}
if !f.Until.IsZero() && r.StartedAt.After(f.Until) {
return false
}
return true
}
func (m *Memory) ListRunsFiltered(_ context.Context, f RunFilter, offset, limit int) ([]SkillRun, error) {
if limit <= 0 || limit > 500 {
limit = 50 // bound admin scans, per the Storage contract
}
m.mu.RLock()
defer m.mu.RUnlock()
return page(m.newestFirst(func(r SkillRun) bool { return m.matchesFilter(r, f) }), offset, limit), nil
}
func (m *Memory) CountRunsFiltered(_ context.Context, f RunFilter) (int64, error) {
m.mu.RLock()
defer m.mu.RUnlock()
return int64(len(m.newestFirst(func(r SkillRun) bool { return m.matchesFilter(r, f) }))), nil
}
func (m *Memory) PurgeOlderThan(_ context.Context, t time.Time) (int64, error) {
m.mu.Lock()
defer m.mu.Unlock()
var purged int64
kept := m.order[:0:0]
for _, id := range m.order {
r := m.runs[id]
if r.FinishedAt != nil && r.FinishedAt.Before(t) {
delete(m.runs, id)
delete(m.logs, id)
purged++
continue
}
kept = append(kept, id)
}
m.order = kept
return purged, nil
}
func (m *Memory) ListChildrenByParent(_ context.Context, parentRunID string) ([]SkillRun, error) {
m.mu.RLock()
defer m.mu.RUnlock()
return m.oldestFirst(func(r SkillRun) bool { return r.ParentRunID == parentRunID }), nil
}
func (m *Memory) WalkParentChain(_ context.Context, runID string) ([]SkillRun, error) {
m.mu.RLock()
defer m.mu.RUnlock()
var chain []SkillRun
seen := map[string]bool{}
for id := runID; id != "" && len(chain) < MaxParentChainDepth; {
r, ok := m.runs[id]
if !ok || seen[id] {
break
}
seen[id] = true
chain = append(chain, r)
id = r.ParentRunID
}
// Contract: root first, the queried run last. We walked child→root, so reverse.
for i, j := 0, len(chain)-1; i < j; i, j = i+1, j-1 {
chain[i], chain[j] = chain[j], chain[i]
}
return chain, nil
}
func (m *Memory) ListFinishedRunsBefore(_ context.Context, cutoff time.Time, limit int) ([]SkillRun, error) {
if limit <= 0 {
return nil, nil // contract: a real bound is required
}
m.mu.RLock()
defer m.mu.RUnlock()
return page(m.oldestFirst(func(r SkillRun) bool {
return r.FinishedAt != nil && r.FinishedAt.Before(cutoff)
}), 0, limit), nil
}
func (m *Memory) LastRunBySkills(_ context.Context, skillIDs []string, includeFailed bool) (map[string]time.Time, error) {
m.mu.RLock()
defer m.mu.RUnlock()
want := map[string]bool{}
for _, id := range skillIDs {
want[id] = true
}
out := map[string]time.Time{}
for _, id := range m.order {
r := m.runs[id]
if !want[r.SkillID] {
continue
}
if !includeFailed && r.Status != "ok" {
continue // contract: only status=="ok" counts unless includeFailed
}
if r.StartedAt.After(out[r.SkillID]) {
out[r.SkillID] = r.StartedAt
}
}
return out, nil
}
+58
View File
@@ -0,0 +1,58 @@
package audit
import (
"context"
"strings"
"testing"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
)
// TestOnToolRedactsSecretTools: a secret-bearing tool's args/result must NOT be
// persisted verbatim in the audit log.
func TestOnToolRedactsSecretTools(t *testing.T) {
ctx := context.Background()
mem := NewMemory()
mem.StartRun(ctx, SkillRun{ID: "r1"})
w := NewWriter(mem, "r1")
secret := `{"url":"https://x","headers":{"Authorization":"Bearer SUPERSECRET"}}`
w.OnTool(llm.ToolCall{Name: "http_get", ID: "1", Arguments: []byte(secret)}, "TOPSECRETBODY")
// a non-secret tool is logged verbatim
w.OnTool(llm.ToolCall{Name: "think", ID: "2", Arguments: []byte(`{"thought":"hi"}`)}, "ok")
logs, _ := mem.ListLogsByRun(ctx, "r1")
var dump strings.Builder
for _, l := range logs {
for k, v := range l.Payload {
dump.WriteString(k)
dump.WriteString("=")
if s, ok := v.(string); ok {
dump.WriteString(s)
}
dump.WriteString(" ")
}
}
all := dump.String()
if strings.Contains(all, "SUPERSECRET") || strings.Contains(all, "TOPSECRETBODY") {
t.Fatalf("secret leaked into audit log: %s", all)
}
// the redaction marker is present, and the non-secret tool's args survive
foundRedacted, foundThink := false, false
for _, l := range logs {
if l.EventType == "tool_call" {
if r, _ := l.Payload["args_redacted"].(bool); r {
foundRedacted = true
}
if a, _ := l.Payload["args"].(string); strings.Contains(a, "thought") {
foundThink = true
}
}
}
if !foundRedacted {
t.Error("secret tool_call should carry args_redacted=true")
}
if !foundThink {
t.Error("non-secret tool args should be logged verbatim")
}
}
+81
View File
@@ -0,0 +1,81 @@
package audit
import (
"context"
"log/slog"
"time"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/executus/run"
)
// Sink adapts an audit Storage to the run.Audit port: StartRun opens a run row
// and returns a per-run recorder (a Writer) that the executor feeds with steps,
// tool calls, and the terminal roll-up. This is what plugs the audit battery
// into run.Ports.Audit — mort backs it with its GORM Storage, a light host with
// Memory() (or omits it entirely).
type Sink struct{ storage Storage }
// NewSink wraps a Storage as a run.Audit. A nil Storage yields a Sink whose
// StartRun returns nil (the executor then records nothing) — so NewSink(nil) is
// equivalent to leaving run.Ports.Audit unset.
func NewSink(storage Storage) *Sink { return &Sink{storage: storage} }
// compile-time proof the adapter satisfies the core seams.
var (
_ run.Audit = (*Sink)(nil)
_ run.RunRecorder = (*recorder)(nil)
)
// StartRun records the run start and returns a recorder. Implements run.Audit.
func (s *Sink) StartRun(ctx context.Context, info run.RunInfo) run.RunRecorder {
if s == nil || s.storage == nil {
return nil
}
started := info.StartedAt
if started.IsZero() {
started = time.Now()
}
// Best-effort: a failed StartRun must not break the user-visible run, but we
// surface it (a swallowed failure leaves orphan log events with no run row).
if err := s.storage.StartRun(ctx, SkillRun{
ID: info.RunID,
SkillID: info.SubjectID,
CallerID: info.CallerID,
ChannelID: info.ChannelID,
ParentRunID: info.ParentRunID,
Inputs: info.Inputs,
StartedAt: started,
Status: "running",
}); err != nil {
slog.Warn("audit: StartRun failed; the run row is missing so its log events will orphan",
"run_id", info.RunID, "error", err)
}
return &recorder{w: NewWriter(s.storage, info.RunID)}
}
// recorder adapts a *Writer to run.RunRecorder, converting run.RunStats to the
// audit RunStats on Close (the two have identical fields).
type recorder struct{ w *Writer }
func (r *recorder) TokenStats() (in, out, thinking int64) { return r.w.TokenStats() }
func (r *recorder) ToolCallsCount() int { return r.w.ToolCallsCount() }
func (r *recorder) OnStep(iter int, resp *llm.Response) { r.w.OnStep(iter, resp) }
func (r *recorder) OnTool(call llm.ToolCall, result string) { r.w.OnTool(call, result) }
func (r *recorder) LogEvent(eventType string, payload map[string]any) {
r.w.LogEvent(eventType, payload)
}
func (r *recorder) LogError(msg string) { r.w.LogError(msg) }
func (r *recorder) Close(ctx context.Context, s run.RunStats) {
r.w.Close(ctx, RunStats{
Status: s.Status,
Output: s.Output,
Error: s.Error,
ToolCalls: s.ToolCalls,
RuntimeSeconds: s.RuntimeSeconds,
InputTokens: s.InputTokens,
OutputTokens: s.OutputTokens,
ThinkingTokens: s.ThinkingTokens,
})
}
+245
View File
@@ -0,0 +1,245 @@
// Package skillaudit persists skill execution traces: per-run summary rows
// (skill_runs) and per-step event logs (skill_run_logs). The executor in
// pkg/logic/skillexec emits events through a Writer; the storage layer is
// kept separate so tests can mock it and so retention pruning has a clear
// home.
//
// Why: agentic runs can be long, multi-tool affairs. Without a structured
// audit trail, debugging "why did the LLM do that?" is impossible. The
// log table is keyed by (run_id, sequence) so insert order is preserved.
package audit
import (
"context"
"errors"
"time"
)
// ErrNotFound is returned when a run lookup fails.
var ErrNotFound = errors.New("skill run not found")
// SkillRun is the per-invocation summary row. One per call to
// Executor.Run. Status transitions through running → ok / error /
// timeout / budget_exceeded / dry_run.
type SkillRun struct {
ID string
SkillID string
CallerID string
ChannelID string
Inputs map[string]any
StartedAt time.Time
FinishedAt *time.Time
Status string // running|ok|error|timeout|budget_exceeded|dry_run
Output string
Error string
ToolCallsCount int
RuntimeSeconds float64
// ParentRunID is the run_id of the parent skill that invoked this
// run via skill_invoke. Empty for top-level invocations. Indexed
// in the gorm model so call-tree queries (ListChildrenByParent +
// WalkParentChain) are cheap.
ParentRunID string
// Token roll-ups, summed across all model completions in this run
// (one Usage per OnStep). All default to 0 when the provider did
// not expose token usage.
TotalInputTokens int64
TotalOutputTokens int64
TotalThinkingTokens int64
}
// RunStats captures the terminal state of a run for FinishRun. Bundling
// these into one struct (vs a long positional argument list) keeps
// callers readable; future fields slot in here without touching every
// call site.
//
// Why: FinishRun originally took six positional args; adding token
// columns would push it higher. A struct is the idiomatic Go way to
// avoid the positional-arg explosion.
type RunStats struct {
Status string // ok|error|timeout|budget_exceeded|dry_run
Output string // final agent output (empty on error)
Error string // error message (empty on success)
ToolCalls int // total OnTool count
RuntimeSeconds float64 // wall-clock duration
// Token roll-ups (all default to 0 when token usage was not
// exposed by the provider).
InputTokens int64
OutputTokens int64
ThinkingTokens int64
}
// SkillRunLog is one event recorded during a run. EventType ∈
// step|tool_call|tool_result|error. Payload is opaque JSON the writer
// emits.
type SkillRunLog struct {
RunID string
Sequence int
EventType string
Payload map[string]any
CreatedAt time.Time
}
// RunFilter is the predicate bundle for the cross-surface "recent runs"
// query (ListRunsFiltered / CountRunsFiltered). Every field is optional;
// the zero value matches the most recent runs across ALL audited surfaces
// (agents + skills). This powers the admin agent-trace debug view and the
// Claude debug API's /runs list.
//
// Why a struct (vs positional args): the debug list filters along several
// independent axes and more will be added; bundling avoids a positional
// explosion and keeps call sites readable.
type RunFilter struct {
Status string // exact status match; "" = all (dry_run excluded unless IncludeDryRun)
SkillID string // exact skill_id (holds the agent UUID for agent runs)
CallerID string // exact caller (Discord member id)
ChannelID string // exact channel id
// TopLevelOnly restricts to root runs (parent_run_id = ''), hiding
// nested sub-agent / sub-skill runs from the firehose. The debug list
// defaults this on; an "include nested" toggle clears it.
TopLevelOnly bool
// IncludeDryRun surfaces status="dry_run" sandbox rows, which are
// excluded by default. Ignored when Status is set explicitly (an
// explicit Status=="dry_run" still matches).
IncludeDryRun bool
// Since / Until bound started_at: started_at >= Since (zero = no lower
// bound) and started_at < Until (zero = no upper bound).
Since time.Time
Until time.Time
}
// Storage is the persistence interface for skill runs and per-step logs.
//
// Why: tests substitute fake implementations; production wires
// NewGormStorage. Keep the interface narrow — the system only needs CRUD
// plus the retention prune helper.
type Storage interface {
Initialize(ctx context.Context) error
// StartRun inserts the run with status=running. The caller MUST
// invoke FinishRun later (or the row stays in running indefinitely
// — operationally that signals a crash mid-run, which is useful
// signal).
StartRun(ctx context.Context, run SkillRun) error
// FinishRun updates the running row with terminal status, output
// and stats. Idempotent on second call (last write wins).
//
// V5: takes a RunStats struct so token + cost columns can be
// written alongside the legacy fields without changing the
// signature for every future addition.
FinishRun(ctx context.Context, runID string, stats RunStats) error
// AppendLog adds one event to the run's log. Sequence numbers must
// be unique per run; the writer is responsible for monotonic
// ordering.
AppendLog(ctx context.Context, log SkillRunLog) error
// GetRun returns the run summary, or ErrNotFound.
GetRun(ctx context.Context, runID string) (*SkillRun, error)
// ListLogsByRun returns all logs for a run in sequence order.
ListLogsByRun(ctx context.Context, runID string) ([]SkillRunLog, error)
// ListRunsBySkill returns recent runs for a skill, newest first,
// capped at limit. Excludes dry-run rows by default — use
// ListRunsBySkillPaginated with includeDryRun=true to see them.
ListRunsBySkill(ctx context.Context, skillID string, limit int) ([]SkillRun, error)
// ListRunsBySkillPaginated returns recent runs for a skill, newest
// first, with offset+limit. When includeDryRun is false, rows with
// status="dry_run" are excluded (matches the wizard's sandbox
// status; see skillaudit.Writer / wizardtools docs).
//
// Why a separate paginated method vs. expanding ListRunsBySkill:
// callers that need the legacy "last N" view (Discord .skill runs,
// chatbot tool result) want the simpler signature; the paginated
// view is webui-specific.
ListRunsBySkillPaginated(ctx context.Context, skillID string,
offset, limit int, includeDryRun bool) ([]SkillRun, error)
// CountRunsBySkill returns the total number of runs for a skill.
// When includeDryRun is false, dry-run rows are excluded so the
// count matches the default ListRunsBySkillPaginated result.
CountRunsBySkill(ctx context.Context, skillID string, includeDryRun bool) (int64, error)
// ListRunsByCaller returns recent runs by a caller, newest first,
// capped at limit.
ListRunsByCaller(ctx context.Context, callerID string, limit int) ([]SkillRun, error)
// ListRunsFiltered returns runs matching f, newest first
// (started_at DESC), with offset+limit. With an all-zero filter it
// returns the most recent runs across EVERY audited surface (agents +
// skills) — the cross-surface feed behind the admin agent-trace debug
// view and the Claude debug API. dry_run rows are excluded unless
// f.IncludeDryRun or f.Status=="dry_run". limit is clamped (<=0 or
// >500 → 50) to bound admin scans.
ListRunsFiltered(ctx context.Context, f RunFilter, offset, limit int) ([]SkillRun, error)
// CountRunsFiltered returns the total rows matching f (ignoring
// offset/limit), for pagination math.
CountRunsFiltered(ctx context.Context, f RunFilter) (int64, error)
// PurgeOlderThan deletes runs (and their logs) whose StartedAt is
// strictly before t. Returns the number of runs deleted.
PurgeOlderThan(ctx context.Context, t time.Time) (int64, error)
// ListChildrenByParent returns all SkillRun rows where
// parent_run_id == parentRunID, oldest first. Used for the
// call-tree view (skill_invoke trace section) and as a building
// block for WalkParentChain.
//
// Returns an empty slice when parentRunID has no children. An
// empty parentRunID never matches anything (no row stores ""
// as a parent — that's the top-level sentinel).
ListChildrenByParent(ctx context.Context, parentRunID string) ([]SkillRun, error)
// WalkParentChain walks from runID up via parent_run_id, returning
// the chain of SkillRun summaries (oldest = root first, newest =
// runID last). Used by the loop guard in skill_invoke.
//
// Cap walk depth at 32 to prevent pathological loops in the data
// itself: if the parent_run_id chain has been corrupted (e.g. by
// a bad migration) and forms a cycle, we want a bounded result
// rather than an infinite loop.
WalkParentChain(ctx context.Context, runID string) ([]SkillRun, error)
// ListFinishedRunsBefore returns runs whose FinishedAt is strictly
// before cutoff, oldest first, capped at limit. limit <= 0 yields
// no rows (the caller is expected to specify a real bound).
//
// Why: skills.StorageSweeper drives the run-scope storage purge from
// this query. The sweeper picks up only finished runs so an
// in-flight run's run-scope KV/files cannot be deleted out from
// under it.
//
// Test: storage_test.go covers the include/exclude boundaries
// (running rows excluded; finished-after-cutoff excluded; finished-
// before-cutoff included).
ListFinishedRunsBefore(ctx context.Context, cutoff time.Time, limit int) ([]SkillRun, error)
// LastRunBySkills returns the most recent StartedAt timestamp per
// skill in the input ID list. Skills with no rows simply have no
// entry in the result map (caller distinguishes "never run" from
// "run but no timestamp" by map key presence).
//
// When includeFailed is true, all non-dry-run statuses count
// (ok / error / timeout / budget_exceeded / preempted / lane_busy).
// When false, only status="ok" rows count — useful for "last
// successful run" semantics on dashboards where errored runs
// shouldn't surface as recent activity.
//
// Empty skillIDs short-circuits to an empty map without touching
// the DB.
LastRunBySkills(ctx context.Context, skillIDs []string, includeFailed bool) (map[string]time.Time, error)
}
// MaxParentChainDepth is the safety cap for WalkParentChain. The loop
// guard in skill_invoke enforces a separate (smaller) MaxInvokeDepth
// at the tool layer; this cap exists only to bound the walk in the
// presence of corrupted data.
const MaxParentChainDepth = 32
+359
View File
@@ -0,0 +1,359 @@
package audit
import (
"context"
"fmt"
"log/slog"
"strings"
"sync"
"sync/atomic"
"time"
llm "gitea.stevedudenhoeffer.com/steve/majordomo/llm"
)
// stepTextMax caps the per-step assistant-text preview persisted on a
// "step" event. Large enough to capture the model's reasoning around a
// (mis)fired tool call — the single best clue to WHY a model emitted a
// malformed call — but bounded so the longtext payload can't balloon.
const stepTextMax = 2000
// Writer wraps a Storage with the OnStep / OnTool callbacks suitable for
// wiring into the majordomo agent loop's step observer, tracking sequence
// numbers and tool-call counts internally.
//
// Why: the agent loop's observer hooks are unaware of run identity; the
// writer captures the runID + skill metadata at construction so the
// per-event callbacks stay simple. AppendLog failures are logged but
// never fatal — audit must not break user-visible execution.
//
// What: NewWriter(storage, runID) → use OnStep / OnTool / Close. Close
// records the final FinishRun. The executors translate each agent.Step
// into one OnStep call (1-indexed iteration, the step's *llm.Response)
// plus one OnTool call per executed tool.
//
// Test: see writer_test.go for sequence ordering and finish semantics.
type Writer struct {
storage Storage
runID string
sequence atomic.Int32
calls atomic.Int32
mu sync.Mutex // guards Close idempotency + token tally
closed bool
// V5 token accumulator — summed across each OnStep's resp.Usage.
// Reads come from TokenStats() so the executor can pass them to
// FinishRun. atomics-on-Int64 would also work, but mu already
// guards Close + we need consistent multi-field reads anyway
// (input + output + thinking). The mutex hot-path overhead is
// negligible vs the LLM call latency that dominates step time.
inputTokens int64
outputTokens int64
thinkingTokens int64
// Per-step wall-clock + run-level model attribution (guarded by mu).
// startedAt anchors the first step's duration; lastStepAt is the
// previous step's observation time; resolvedModelLogged ensures the
// one-shot "resolved_model" run-level event fires at most once.
startedAt time.Time
lastStepAt time.Time
resolvedModelLogged bool
}
// NewWriter constructs a Writer. The caller is expected to have already
// called Storage.StartRun.
func NewWriter(storage Storage, runID string) *Writer {
return &Writer{storage: storage, runID: runID, startedAt: time.Now()}
}
// OnStep records one agent-loop step: a "step" event with the iteration
// number and the response's text size.
//
// V5: also tallies per-step token usage. majordomo populates
// resp.Usage when the provider reports it; for providers that don't,
// the fields stay 0 and the tally stays at 0 — the formatter then
// renders "—" rather than a misleading "$0.00".
//
// Why we tally here vs in the agent loop: the loop's Result.Usage is a
// run total; the audit row needs the same numbers, but the writer also
// serves the live RunState accessor mid-run, so a per-step running sum
// is the right shape. Global usage attribution is handled by the llms
// package's instrumented models — the writer tally is strictly the
// per-run audit roll-up.
func (w *Writer) OnStep(iter int, resp *llm.Response) {
if w == nil || w.storage == nil {
return
}
now := time.Now()
payload := map[string]any{"iter": iter}
w.mu.Lock()
// Per-step wall-clock: time since the previous observed step, or since
// run start for the first step. A long gap localises a slow/hung model
// call — the signal that was missing when an animate step-0 call hung
// ~5 min. NOTE: this is step-to-step wall time (model call + the prior
// step's tool execution), not pure model latency.
prev := w.lastStepAt
if prev.IsZero() {
prev = w.startedAt
}
if !prev.IsZero() {
payload["step_ms"] = now.Sub(prev).Milliseconds()
}
w.lastStepAt = now
if resp != nil {
w.inputTokens += int64(resp.Usage.InputTokens)
w.outputTokens += int64(resp.Usage.OutputTokens)
// Thinking/reasoning tokens are a first-class Usage field in
// majordomo (populated by the providers that report them).
w.thinkingTokens += int64(resp.Usage.ReasoningTokens)
}
// One-shot run-level served-model attribution: the FIRST step with a
// resolved model name emits a "resolved_model" event so a run that
// errors before producing a useful step still records which model
// served it. resp.Model is failover-aware ("provider/model-id" of the
// element that actually served), unlike the static configured head.
logResolvedModel := ""
if resp != nil && resp.Model != "" && !w.resolvedModelLogged {
w.resolvedModelLogged = true
logResolvedModel = resp.Model
}
w.mu.Unlock()
if resp != nil {
payload["text_len"] = len(resp.Text())
// Served model + why generation stopped — the two scalars that turn
// a "model misbehaved" guess into a fact. finish_reason on an
// empty-tool-call step disambiguates truncation (length) from a
// deliberate empty emission (tool_calls).
if resp.Model != "" {
payload["model"] = resp.Model
}
if resp.FinishReason != "" {
payload["finish_reason"] = string(resp.FinishReason)
}
// Per-step token breakdown (OnStep already reads these into the run
// total above; persisting the per-step slice costs nothing more).
payload["in_tokens"] = resp.Usage.InputTokens
payload["out_tokens"] = resp.Usage.OutputTokens
if resp.Usage.ReasoningTokens > 0 {
payload["thinking_tokens"] = resp.Usage.ReasoningTokens
}
if resp.Usage.CacheReadTokens > 0 {
payload["cache_read_tokens"] = resp.Usage.CacheReadTokens
}
// The model's own narration accompanying this step — the smoking gun
// for WHY a malformed tool call was emitted. Capped; suppressed when
// the step fired a secret-bearing tool (mcp_call/email_send/http_*)
// whose narration could echo the secret it's about to send.
if t := strings.TrimSpace(resp.Text()); t != "" {
if stepHasSecretTool(resp) {
payload["text_redacted"] = true
} else {
payload["text"] = truncate(t, stepTextMax)
}
}
} else {
payload["text_len"] = 0
}
w.appendLog("step", payload)
if logResolvedModel != "" {
w.appendLog("resolved_model", map[string]any{"model": logResolvedModel})
}
}
// stepHasSecretTool reports whether a step's response fired a tool whose
// surrounding narration could leak a secret (MCP args, email body/
// recipients, raw HTTP request). Mirrors the steps.go redaction list so
// the audit trace never persists secret-adjacent assistant text.
// isSecretTool reports whether a tool's arguments/results may carry secrets
// (MCP args, email bodies/recipients, HTTP auth headers/bodies) and so must be
// redacted from the persisted audit log. Single source of truth for both the
// step-narration redaction and the OnTool arg/result redaction. NOTE: this is
// a name-prefix allowlist — a NEW secret-bearing tool must be added here or its
// args/results will be logged verbatim.
func isSecretTool(name string) bool {
switch name {
case "mcp_call", "email_send":
return true
}
return strings.HasPrefix(name, "http_")
}
func stepHasSecretTool(resp *llm.Response) bool {
if resp == nil {
return false
}
for _, c := range resp.ToolCalls {
if isSecretTool(c.Name) {
return true
}
}
return false
}
// TokenStats returns the running totals tallied from OnStep.
// Safe to call concurrently. Returned values are a snapshot at call
// time. Used by the executors to populate RunStats before Close
// finalises the audit row.
//
// Why: the executor needs the totals AND a model name to compute cost,
// but cost calculation is a different concern from audit persistence.
// Exposing this getter lets the cost calculation live in the executor
// where the model is known.
func (w *Writer) TokenStats() (input, output, thinking int64) {
if w == nil {
return 0, 0, 0
}
w.mu.Lock()
defer w.mu.Unlock()
return w.inputTokens, w.outputTokens, w.thinkingTokens
}
// OnTool records a "tool_call" event with the tool name and a
// "tool_result" event with the result length. Tool count is incremented
// for each call. The executors call this once per executed tool call
// from their step observers (call + matching result content).
func (w *Writer) OnTool(call llm.ToolCall, result string) {
if w == nil || w.storage == nil {
return
}
w.calls.Add(1)
// Redact the args/result of secret-bearing tools — these fields actually
// CARRY the secret (MCP args, email body/recipients, HTTP auth/body), so
// logging them verbatim would defeat the OnStep narration redaction.
if isSecretTool(call.Name) {
w.appendLog("tool_call", map[string]any{
"name": call.Name,
"id": call.ID,
"args_redacted": true,
"args_len": len(call.Arguments),
})
w.appendLog("tool_result", map[string]any{
"name": call.Name,
"id": call.ID,
"result_redacted": true,
"result_len": len(result),
})
return
}
w.appendLog("tool_call", map[string]any{
"name": call.Name,
"args": string(call.Arguments),
"id": call.ID,
})
w.appendLog("tool_result", map[string]any{
"name": call.Name,
"id": call.ID,
"result": truncate(result, 4000),
"truncated": len(result) > 4000,
})
}
// LogEvent records a custom event mid-run. The executor uses this for
// diagnostic events (e.g. "compaction_setup" / "compaction_fired")
// outside the canonical step / tool_call / tool_result / error set.
// Nil-safe: no-op when receiver or storage is nil.
//
// Why: skill_run_logs is the only sink Steve can read from SQL, so
// diagnostics intended for post-hoc debugging belong here. slog goes
// to mort.log which is harder to reach from outside the host.
func (w *Writer) LogEvent(eventType string, payload map[string]any) {
if w == nil || w.storage == nil {
return
}
w.appendLog(eventType, payload)
}
// LogError records an "error" event mid-run. Distinct from the terminal
// status set by Close.
func (w *Writer) LogError(msg string) {
if w == nil || w.storage == nil {
return
}
w.appendLog("error", map[string]any{"message": msg})
}
// Close finishes the run. The caller assembles a RunStats; the writer
// fills in ToolCalls (which is bookkept on the writer itself) and
// hands the full record to FinishRun.
//
// Idempotent: subsequent calls are no-ops.
//
// Why a struct vs the old positional form: v5 adds four token + cost
// fields on top of the legacy six. The struct keeps call sites readable
// and lets future fields slot in without churning every caller.
//
// Why context.WithoutCancel: the run's terminal status MUST land in
// the audit row regardless of the run ctx's state. Pre-fix, child
// skill runs invoked via skill_invoke / skill_invoke_parallel inherited
// the parent agent's runCtx as their outer ctx; when the parent
// timed out at MaxRuntime, every in-flight child's FinishRun fired
// with that already-cancelled ctx and the row was left in
// status=running forever. Detaching here is defence in depth — the
// caller (skillexec.runInner / agentexec.runInner) ALSO detaches at
// the call site, but a cancelled ctx in the writer's hands MUST NOT
// drop the audit write. The short timeout (auditFinishTimeout) bounds
// the write so a hung DB doesn't pin the run goroutine indefinitely.
func (w *Writer) Close(ctx context.Context, stats RunStats) {
if w == nil || w.storage == nil {
return
}
w.mu.Lock()
defer w.mu.Unlock()
if w.closed {
return
}
w.closed = true
stats.ToolCalls = int(w.calls.Load())
// Detach from the caller's deadline + cancellation. Run cleanup
// must complete even when the run ctx is dead. The fresh
// auditFinishTimeout caps how long we'll wait on the storage.
finishCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), auditFinishTimeout)
defer cancel()
if err := w.storage.FinishRun(finishCtx, w.runID, stats); err != nil {
slog.Warn("skillaudit: FinishRun failed", "run_id", w.runID, "error", err)
}
}
// auditFinishTimeout caps how long Close will wait on the storage's
// FinishRun call after detaching from the caller's ctx. 10s is generous
// for a single-row UPDATE against MySQL — anything longer suggests a
// hung connection that the run goroutine shouldn't keep waiting on.
const auditFinishTimeout = 10 * time.Second
// auditAppendTimeout bounds each per-event AppendLog on the hot path so a hung
// storage backend can't block the run goroutine.
const auditAppendTimeout = 3 * time.Second
// ToolCallsCount returns how many tool invocations OnTool has seen so
// far. Useful for budget enforcement.
func (w *Writer) ToolCallsCount() int { return int(w.calls.Load()) }
func (w *Writer) appendLog(eventType string, payload map[string]any) {
seq := int(w.sequence.Add(1))
log := SkillRunLog{
RunID: w.runID,
Sequence: seq,
EventType: eventType,
Payload: payload,
CreatedAt: time.Now(),
}
// Bound the write: a hung storage backend must not block the run goroutine
// on the hot path (every step/tool event flows through here). Detached from
// any caller deadline — the log write is independent of the run's context.
ctx, cancel := context.WithTimeout(context.Background(), auditAppendTimeout)
defer cancel()
if err := w.storage.AppendLog(ctx, log); err != nil {
slog.Warn("skillaudit: AppendLog failed", "run_id", w.runID, "seq", seq, "type", eventType, "error", err)
}
}
func truncate(s string, max int) string {
if len(s) <= max {
return s
}
return s[:max] + fmt.Sprintf("…[+%d bytes]", len(s)-max)
}
+169
View File
@@ -0,0 +1,169 @@
// Package budget gates and meters per-caller resource use over a rolling
// 7-day window (run.Ports.Budget). DBBudget is the durable tracker; NoOpBudget
// disables metering; the BudgetStorage seam backs it (Memory / contrib SQLite).
// loop (gitea.stevedudenhoeffer.com/steve/majordomo/agent).
//
// Why: a Skill is data; the executor turns data into a running agent
// (resolve model, build toolbox, start audit, run the agent loop,
// finish audit, deliver).
package budget
import (
"context"
"errors"
"fmt"
"log/slog"
"time"
)
// BudgetTracker enforces per-user GPU budgets in v2. v1 ships
// NoOpBudget which always allows. The interface exists now so the v2
// migration is a single line in the executor.
//
// Why interface now: the executor's Check/Commit calls would need to
// be added in v2 anyway; doing it now means v2 only swaps NoOp for
// DBBudget without touching call sites.
type BudgetTracker interface {
// Check reports whether the caller has remaining budget. Returns
// nil for "yes" or an error describing the exhaustion.
Check(ctx context.Context, callerID string) error
// Commit records that the caller spent runtimeSeconds of budget on
// this run. Called after the agent completes (success or error).
Commit(ctx context.Context, callerID string, runtimeSeconds float64)
}
// NoOpBudget always allows and never records. v1 default.
type NoOpBudget struct{}
// NewNoOpBudget constructs the no-op tracker.
func NewNoOpBudget() BudgetTracker { return NoOpBudget{} }
// Check always returns nil.
func (NoOpBudget) Check(_ context.Context, _ string) error { return nil }
// Commit is a no-op.
func (NoOpBudget) Commit(_ context.Context, _ string, _ float64) {}
// ErrBudgetExceeded is returned by DBBudget.Check when the caller's
// 7-day rolling window has hit the convar-configured cap.
//
// Why a sentinel: callers (executor, audit writer) need to distinguish
// budget rejection from generic errors so they can record
// status="budget_exceeded" instead of "error" and skip user-visible
// delivery side-effects.
var ErrBudgetExceeded = errors.New("weekly skill budget exceeded")
// BudgetNotifier is the optional callback DBBudget invokes when a
// Check rejects a caller. Production wires a Discord-DM hook so the
// user knows why their skill failed; tests inject a recorder.
//
// nil is allowed and is silently skipped.
type BudgetNotifier func(ctx context.Context, userID string, secondsUsed, cap float64)
// DBBudget enforces per-user weekly GPU budgets via the BudgetStorage
// interface. The "weekly" cap is a rolling 7-day window — see
// SkillBudget for the rollover semantics.
//
// Why a closure for the limit instead of an int field: the cap comes
// from a runtime convar. Reading it on every Check means a `.convar
// set skills.user_budget_seconds_per_week 7200` takes effect on the
// next call without restarting the bot or rewiring the executor.
type DBBudget struct {
storage BudgetStorage
// weeklyLimit returns the current cap in seconds. Reads convar at
// every Check so a runtime convar bump takes effect on the next
// call.
weeklyLimit func() float64
// notify is called when a Check rejects a caller. Optional —
// production wires a Discord-DM hook so the user knows why their
// skill failed. nil-safe.
notify BudgetNotifier
// now is the time source. Test injects a fake clock; production
// uses time.Now.
now func() time.Time
}
// NewDBBudget constructs a DBBudget. now may be nil — defaults to
// time.Now.
//
// Why time injection: budget rollover is time-sensitive; tests need to
// fast-forward past the 7-day boundary deterministically. now=nil
// means production callers (mort.go) don't have to think about it.
//
// Test: pass a closure that returns a fixed instant; assert rollover
// only happens when (now - WindowStart) >= 7 days.
func NewDBBudget(storage BudgetStorage, weeklyLimit func() float64, notify BudgetNotifier, now func() time.Time) *DBBudget {
if now == nil {
now = time.Now
}
return &DBBudget{
storage: storage,
weeklyLimit: weeklyLimit,
notify: notify,
now: now,
}
}
// Check returns ErrBudgetExceeded if the caller has spent at least
// weeklyLimit seconds in the current rolling 7-day window.
//
// Why anonymous callerID="" is unbudgeted: scheduler-driven and
// system-initiated runs don't have a Discord user to bill; charging
// "system" would conflate them with a real user. The scheduler sets
// CallerID to the skill owner where applicable, so cron-loop
// abusiveness still consumes the owner's budget.
//
// Why cap<=0 means "disabled": operator wants a runtime kill-switch.
// Setting the convar to "0" turns enforcement off without restart.
//
// Test: Get returns nil → Check returns nil; Get returns row with
// SecondsUsed >= cap → Check returns ErrBudgetExceeded and notify is
// invoked; window expired (>=7d) → Check returns nil regardless of
// SecondsUsed.
func (b *DBBudget) Check(ctx context.Context, callerID string) error {
if callerID == "" {
return nil
}
bud, err := b.storage.Get(ctx, callerID)
if err != nil {
return fmt.Errorf("budget: %w", err)
}
if bud != nil {
if b.now().Sub(bud.WindowStart) < budgetWindow {
cap := b.weeklyLimit()
if cap > 0 && bud.SecondsUsed >= cap {
if b.notify != nil {
b.notify(ctx, callerID, bud.SecondsUsed, cap)
}
return ErrBudgetExceeded
}
}
}
return nil
}
// Commit records the run's runtime against the caller's budget.
// Failures are logged but never returned — budget accounting must
// not break user-visible execution.
//
// Why callerID="" is a no-op: matches Check's anonymous-caller
// shortcut; system runs don't get billed.
//
// Why runtimeSeconds<=0 is a no-op: a run that errored before
// resolving a model has wallSecs near 0 in floating-point terms but
// can also be exactly 0 (synthetic test fixtures). Skipping avoids
// spurious 0-runs rows from short-lived failures.
//
// Test: Commit(50) → Get reports SecondsUsed=50; storage failure
// surfaces only as a slog.Warn (no panic, no return).
func (b *DBBudget) Commit(ctx context.Context, callerID string, runtimeSeconds float64) {
if callerID == "" || runtimeSeconds <= 0 {
return
}
if err := b.storage.Add(ctx, callerID, runtimeSeconds, b.now()); err != nil {
slog.Warn("skills budget: commit failed", "user", callerID, "error", err)
}
}
+44
View File
@@ -0,0 +1,44 @@
package budget
import (
"context"
"errors"
"testing"
"time"
)
func TestDBBudgetRollingWindow(t *testing.T) {
ctx := context.Background()
mem := NewMemory()
now := time.Now()
clock := func() time.Time { return now }
b := NewDBBudget(mem, func() float64 { return 100 }, nil, clock)
// Under cap: allowed.
if err := b.Check(ctx, "u"); err != nil {
t.Fatalf("fresh caller should pass: %v", err)
}
b.Commit(ctx, "u", 60)
if err := b.Check(ctx, "u"); err != nil {
t.Fatalf("60/100 should pass: %v", err)
}
// Over cap: rejected.
b.Commit(ctx, "u", 50) // 110 total
if err := b.Check(ctx, "u"); !errors.Is(err, ErrBudgetExceeded) {
t.Fatalf("110/100 should be ErrBudgetExceeded, got %v", err)
}
// Window rolls over after 7 days: allowed again.
now = now.Add(8 * 24 * time.Hour)
b.Commit(ctx, "u", 1) // triggers rollover inside Add
if err := b.Check(ctx, "u"); err != nil {
t.Fatalf("after window rollover should pass: %v", err)
}
}
func TestNoOpBudgetAlwaysAllows(t *testing.T) {
b := NewNoOpBudget()
if err := b.Check(context.Background(), "anyone"); err != nil {
t.Fatalf("NoOp must always allow: %v", err)
}
b.Commit(context.Background(), "anyone", 1e9) // no-op
}
+56
View File
@@ -0,0 +1,56 @@
package budget
import (
"context"
"sync"
"time"
)
// Memory is a zero-dependency in-process BudgetStorage: per-user rolling-window
// usage held in memory (lost on restart). The default behind DBBudget for a
// light host or tests; mort uses its GORM Storage, contrib/store adds SQLite.
type Memory struct {
mu sync.RWMutex
rows map[string]*SkillBudget
}
// NewMemory returns an empty in-memory BudgetStorage.
func NewMemory() *Memory { return &Memory{rows: map[string]*SkillBudget{}} }
var _ BudgetStorage = (*Memory)(nil)
func (m *Memory) Initialize(context.Context) error { return nil }
func (m *Memory) Get(_ context.Context, userID string) (*SkillBudget, error) {
m.mu.RLock()
defer m.mu.RUnlock()
r, ok := m.rows[userID]
if !ok {
return nil, nil
}
cp := *r // copy out so callers can't mutate our row
return &cp, nil
}
func (m *Memory) Add(_ context.Context, userID string, secondsUsed float64, now time.Time) error {
m.mu.Lock()
defer m.mu.Unlock()
r, ok := m.rows[userID]
if !ok {
m.rows[userID] = &SkillBudget{
UserID: userID, WindowStart: now,
SecondsUsed: secondsUsed, RunsCount: 1, UpdatedAt: now,
}
return nil
}
// Roll the window over if it's older than the window length.
if now.Sub(r.WindowStart) >= budgetWindow {
r.WindowStart = now
r.SecondsUsed = 0
r.RunsCount = 0
}
r.SecondsUsed += secondsUsed
r.RunsCount++
r.UpdatedAt = now
return nil
}
+9
View File
@@ -0,0 +1,9 @@
package budget
import "gitea.stevedudenhoeffer.com/steve/executus/run"
// The budget trackers plug directly into run.Ports.Budget (Check/Commit match).
var (
_ run.Budget = NoOpBudget{}
_ run.Budget = (*DBBudget)(nil)
)
+33
View File
@@ -0,0 +1,33 @@
package budget
import (
"context"
"time"
)
// BudgetStorage is the persistence seam behind DBBudget: one budget row per
// user, with an atomic Add that rolls the 7-day window over transparently. Mort
// backs this with GORM/MySQL (the skill_budgets table); Memory() is the
// zero-dependency default; contrib/store adds a durable SQLite one.
type BudgetStorage interface {
// Initialize runs any schema setup. Safe to call repeatedly.
Initialize(ctx context.Context) error
// Get returns the user's current budget row, or (nil, nil) if none exists.
Get(ctx context.Context, userID string) (*SkillBudget, error)
// Add increments seconds_used + runs_count atomically, rolling the window
// over when WindowStart is older than 7 days (reset to now, fresh count).
// Creates the row if absent.
Add(ctx context.Context, userID string, secondsUsed float64, now time.Time) error
}
// SkillBudget is one user's rolling-window usage row.
type SkillBudget struct {
UserID string
WindowStart time.Time
SecondsUsed float64
RunsCount int
UpdatedAt time.Time
}
// budgetWindow is the rolling window length the storage rolls over at.
const budgetWindow = 7 * 24 * time.Hour
+50
View File
@@ -0,0 +1,50 @@
// Package checkpoint is the durable-resume battery: it persists a run's
// resumable progress so a run interrupted by a shutdown can be recovered and
// continued on the next boot, rather than silently lost. It plugs into
// run.Ports.Checkpointer.
//
// Mort backs CheckpointStore with its durable-job table; Memory() is the
// zero-dependency default; contrib/store can add a SQLite one. NOTE: the
// executor's call into run.Ports.Checkpointer is a P2 follow-up — this battery
// provides the seam + impls ahead of that wiring.
package checkpoint
import (
"context"
"time"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
)
// RunCheckpointMeta is the run attribution needed to resume a run from scratch
// (mirrors mort's agentexec.RunCheckpointMeta).
type RunCheckpointMeta struct {
RunID string
AgentID string
AgentName string
CallerID string
ChannelID string
GuildID string
Prompt string
ModelTier string
ParentRunID string
}
// RunCheckpoint is one persisted snapshot of a run's resumable progress.
type RunCheckpoint struct {
Meta RunCheckpointMeta
Messages []llm.Message // conversation so far
Iteration int // completed agent-loop iterations
ActivePhase string // current phase name (multi-phase agents); "" otherwise
UpdatedAt time.Time
}
// CheckpointStore persists run checkpoints keyed by run id. A live checkpoint
// means "this run was in flight and not cleanly finished"; Complete/Fail delete
// it. ListInterrupted returns every surviving checkpoint at boot for recovery.
type CheckpointStore interface {
Save(ctx context.Context, cp RunCheckpoint) error
Load(ctx context.Context, runID string) (*RunCheckpoint, error)
Delete(ctx context.Context, runID string) error
ListInterrupted(ctx context.Context) ([]RunCheckpoint, error)
}
+64
View File
@@ -0,0 +1,64 @@
package checkpoint
import (
"context"
"testing"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/run"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
)
func TestHandleSaveCompleteDelete(t *testing.T) {
ctx := context.Background()
mem := NewMemory()
meta := RunCheckpointMeta{RunID: "r1", AgentID: "a1", CallerID: "c1"}
cp := New(mem, meta, 0, nil) // throttle 0 = save every call
if err := cp.Save(ctx, run.RunCheckpointState{Messages: []llm.Message{{Role: "user"}}, Iteration: 2}); err != nil {
t.Fatal(err)
}
got, _ := mem.Load(ctx, "r1")
if got == nil || got.Iteration != 2 || got.Meta.AgentID != "a1" {
t.Fatalf("checkpoint not persisted: %+v", got)
}
if il, _ := mem.ListInterrupted(ctx); len(il) != 1 {
t.Errorf("ListInterrupted = %d, want 1 (in-flight)", len(il))
}
// Complete clears it (no longer a recovery candidate).
if err := cp.Complete(ctx); err != nil {
t.Fatal(err)
}
if il, _ := mem.ListInterrupted(ctx); len(il) != 0 {
t.Errorf("after Complete, ListInterrupted = %d, want 0", len(il))
}
}
func TestHandleThrottle(t *testing.T) {
ctx := context.Background()
mem := NewMemory()
now := time.Now()
cp := New(mem, RunCheckpointMeta{RunID: "r"}, time.Minute, func() time.Time { return now })
cp.Save(ctx, run.RunCheckpointState{Iteration: 1})
now = now.Add(10 * time.Second) // within throttle window
cp.Save(ctx, run.RunCheckpointState{Iteration: 2})
if got, _ := mem.Load(ctx, "r"); got.Iteration != 1 {
t.Errorf("throttled save should keep iteration 1, got %d", got.Iteration)
}
now = now.Add(time.Minute) // past throttle
cp.Save(ctx, run.RunCheckpointState{Iteration: 3})
if got, _ := mem.Load(ctx, "r"); got.Iteration != 3 {
t.Errorf("post-throttle save should land iteration 3, got %d", got.Iteration)
}
}
func TestNilStoreNoop(t *testing.T) {
cp := New(nil, RunCheckpointMeta{RunID: "r"}, 0, nil)
if err := cp.Save(context.Background(), run.RunCheckpointState{}); err != nil {
t.Errorf("nil-store Save should be a no-op, got %v", err)
}
if err := cp.Complete(context.Background()); err != nil {
t.Error(err)
}
}
+83
View File
@@ -0,0 +1,83 @@
package checkpoint
import (
"context"
"sync"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/run"
)
// handle is a per-run run.Checkpointer bound to one run's id + meta. Save writes
// a fresh snapshot (throttled), Complete/Fail delete the checkpoint (a cleanly
// finished or terminally failed run is NOT a recovery candidate). A run
// interrupted by shutdown never calls Complete/Fail, so its checkpoint survives
// for ListInterrupted at boot.
type handle struct {
store CheckpointStore
meta RunCheckpointMeta
throttle time.Duration
now func() time.Time
mu sync.Mutex
lastSave time.Time
}
var _ run.Checkpointer = (*handle)(nil)
// New returns a run.Checkpointer that persists snapshots of the run identified
// by meta.RunID to store, no more often than throttle (Save calls inside the
// window are skipped). A nil store yields a no-op Checkpointer. throttle <= 0
// saves every call; now defaults to time.Now.
func New(store CheckpointStore, meta RunCheckpointMeta, throttle time.Duration, now func() time.Time) run.Checkpointer {
if store == nil {
return noop{}
}
if now == nil {
now = time.Now
}
return &handle{store: store, meta: meta, throttle: throttle, now: now}
}
func (h *handle) Save(ctx context.Context, st run.RunCheckpointState) error {
h.mu.Lock()
now := h.now()
if h.throttle > 0 && !h.lastSave.IsZero() && now.Sub(h.lastSave) < h.throttle {
h.mu.Unlock()
return nil // throttled — a more recent snapshot will land shortly
}
h.mu.Unlock()
// Advance the throttle clock only AFTER a successful persist. If the store
// write fails, lastSave stays put so the next Save isn't throttled away —
// otherwise a transient store error would silently drop the snapshot the
// caller believes was saved. (A run drives one Save goroutine, so the brief
// unguarded window here can't double-write.)
if err := h.store.Save(ctx, RunCheckpoint{
Meta: h.meta,
Messages: st.Messages,
Iteration: st.Iteration,
UpdatedAt: now,
}); err != nil {
return err
}
h.mu.Lock()
if now.After(h.lastSave) {
h.lastSave = now
}
h.mu.Unlock()
return nil
}
func (h *handle) Complete(ctx context.Context) error { return h.store.Delete(ctx, h.meta.RunID) }
func (h *handle) Fail(ctx context.Context, _ error) error { return h.store.Delete(ctx, h.meta.RunID) }
// noop is the nil-store Checkpointer: every method is a successful no-op.
type noop struct{}
var _ run.Checkpointer = noop{}
func (noop) Save(context.Context, run.RunCheckpointState) error { return nil }
func (noop) Complete(context.Context) error { return nil }
func (noop) Fail(context.Context, error) error { return nil }
+55
View File
@@ -0,0 +1,55 @@
package checkpoint
import (
"context"
"sync"
)
// Memory is a zero-dependency in-process CheckpointStore. NOTE: an in-memory
// checkpoint store does NOT survive the process restart it exists to recover
// from — it is the test/light-host default and makes ListInterrupted meaningful
// only within a single process lifetime. A host that wants real
// crash-recovery wires a durable CheckpointStore (mort's durable-job table).
type Memory struct {
mu sync.RWMutex
cps map[string]RunCheckpoint // by run id
}
// NewMemory returns an empty in-memory CheckpointStore.
func NewMemory() *Memory { return &Memory{cps: map[string]RunCheckpoint{}} }
var _ CheckpointStore = (*Memory)(nil)
func (m *Memory) Save(_ context.Context, cp RunCheckpoint) error {
m.mu.Lock()
defer m.mu.Unlock()
m.cps[cp.Meta.RunID] = cp
return nil
}
func (m *Memory) Load(_ context.Context, runID string) (*RunCheckpoint, error) {
m.mu.RLock()
defer m.mu.RUnlock()
cp, ok := m.cps[runID]
if !ok {
return nil, nil // no checkpoint (not an error — the run finished cleanly or never started)
}
return &cp, nil
}
func (m *Memory) Delete(_ context.Context, runID string) error {
m.mu.Lock()
defer m.mu.Unlock()
delete(m.cps, runID)
return nil
}
func (m *Memory) ListInterrupted(_ context.Context) ([]RunCheckpoint, error) {
m.mu.RLock()
defer m.mu.RUnlock()
out := make([]RunCheckpoint, 0, len(m.cps))
for _, cp := range m.cps {
out = append(out, cp)
}
return out, nil
}
+369
View File
@@ -0,0 +1,369 @@
// V15.2 context compactor (re-based on majordomo).
//
// Why: the agent loop accumulates tool results indefinitely. A
// research-heavy run with many web_search / read_page / http_get
// results easily crosses 200K tokens and trips the model's HTTP-400
// "prompt too long" rejection mid-run (observed at 410K tokens on
// qwen3-coder:480b which has a 262K cap). majordomo's agent loop calls
// the compactor with the full message slice before every model call;
// the compactor returns a shorter slice that preserves the system
// prompt + recent messages, with the middle range replaced by a
// synthetic summary.
//
// Strategy (unchanged from the agentkit era):
// - Keep any leading system message verbatim. (Under majordomo the
// system prompt normally travels in Request.System, not in the
// message slice, so this is defensive.)
// - Keep the last KeepRecent messages verbatim. This ensures the
// agent has fresh tool state to act on; compacting too aggressively
// would strip the in-flight context it needs to make the next
// decision.
// - Compress the middle range via a single fast-tier LLM call that
// receives the middle messages as raw text and produces a paragraph
// summary (URLs visited, key findings, file_ids created, what the
// agent is trying to accomplish).
// - Replace the middle range with one synthetic user-role message
// containing the summary. (user-role chosen because tool-result-role
// would be ambiguous without a matching tool_call_id.)
//
// What moved in the majordomo conversion:
// - The token-threshold gate lives HERE now. agentkit estimated
// tokens and only invoked the compactor past a configured
// threshold; majordomo's hook fires before every model call, so
// the threshold check (estimateTokens vs the per-run threshold the
// executor computes from the model's context limit) is the
// compactor's first step.
// - The compactor is per-run STATEFUL: majordomo does not replace
// the loop's internal transcript with the compacted slice (the
// hook shapes only what is SENT), so without memory the middle
// would be re-summarised from scratch on every step past the
// threshold. The state remembers how far the transcript has been
// folded into the running summary and folds the previous summary
// into the next one instead of re-paying for it.
//
// Failure path: any error (LLM unavailable, malformed response, etc.)
// is returned to the agent loop, which treats compactor errors as
// non-fatal and sends the original slice — if the next model call hits
// the provider's limit, the existing HTTP-400 error path takes over.
package compact
import (
"context"
"errors"
"fmt"
"strings"
"sync"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
)
// ModelResolver resolves a tier/spec to a usable llm.Model (and an enriched
// context for usage attribution). model.ParseModelForContext satisfies it.
type ModelResolver func(ctx context.Context, tier string) (context.Context, llm.Model, error)
// Compactor is the per-run compaction hook handed to the agent loop
// (matches the signature agent.WithCompactor expects).
type Compactor = func(ctx context.Context, msgs []llm.Message) ([]llm.Message, error)
// CompactionEvent describes one fired compaction so the executor can
// log it to skill_run_logs ("compaction_fired").
type CompactionEvent struct {
// MessagesBefore/After count the messages that would have been
// sent without/with this compaction.
MessagesBefore int
MessagesAfter int
// TokensBefore/After are the estimateTokens values for the same
// two slices.
TokensBefore int
TokensAfter int
}
// CompactorFactory mints a fresh per-run Compactor bound to a token
// threshold. onFire (nil-safe) observes every compaction that actually
// fires. A non-positive threshold yields a pass-through compactor.
type CompactorFactory func(thresholdTokens int, onFire func(CompactionEvent)) Compactor
// CompactorConfig controls the v15.2 compactor's behaviour. Construct
// in mort.go from convars + the executor's ModelResolver.
type CompactorConfig struct {
// Models resolves a model spec (typically "fast" or a specific tier)
// to an llm.Model. Required; nil disables compaction.
Models ModelResolver
// SummarizerTier is the model tier used for the compression LLM call.
// Production default "fast"; an admin may set "haiku" or a specific
// model spec. Empty falls back to "fast".
SummarizerTier string
// KeepRecent is the number of trailing messages preserved verbatim.
// Default 8. Lower values compact more aggressively (the next loop
// iteration sees less recent context); higher values keep more.
KeepRecent int
// SummaryWordCap bounds the LLM-generated summary length. Default
// 200 words ≈ 800 chars ≈ 200 tokens — small enough that the
// compaction always shrinks the slice meaningfully.
SummaryWordCap int
}
// NewCompactor returns a CompactorFactory implementing the middle-range
// summarisation strategy. nil cfg.Models returns a factory of no-op
// compactors that always return the input unchanged (degrades to
// v15.1 behaviour).
func NewCompactor(cfg CompactorConfig) CompactorFactory {
if cfg.Models == nil {
return func(int, func(CompactionEvent)) Compactor {
return func(_ context.Context, msgs []llm.Message) ([]llm.Message, error) {
return msgs, nil
}
}
}
if cfg.SummarizerTier == "" {
cfg.SummarizerTier = "fast"
}
if cfg.KeepRecent <= 0 {
cfg.KeepRecent = 8
}
if cfg.SummaryWordCap <= 0 {
cfg.SummaryWordCap = 200
}
return func(threshold int, onFire func(CompactionEvent)) Compactor {
st := &compactionState{}
return func(ctx context.Context, msgs []llm.Message) ([]llm.Message, error) {
return compactIfNeeded(ctx, cfg, st, threshold, onFire, msgs)
}
}
}
// compactionState is the per-run fold memory: msgs[:prefixEnd]
// (excluding a leading system message) are represented by summaryText
// in the rendered slice. Guarded by mu for safety although the agent
// loop invokes the hook from a single goroutine.
type compactionState struct {
mu sync.Mutex
prefixEnd int
summaryText string
}
// compactIfNeeded is the workhorse: render the transcript with any
// existing fold applied, check the threshold, and fold more of the
// middle into the summary when the rendered size still exceeds it.
func compactIfNeeded(ctx context.Context, cfg CompactorConfig, st *compactionState,
threshold int, onFire func(CompactionEvent), msgs []llm.Message) ([]llm.Message, error) {
st.mu.Lock()
defer st.mu.Unlock()
rendered := renderCompacted(st, msgs)
if threshold <= 0 {
return rendered, nil
}
tokensBefore := estimateTokens(rendered)
if tokensBefore < threshold {
return rendered, nil
}
// Determine the new middle range to fold: everything between what
// is already summarised (or the optional leading system message)
// and the KeepRecent tail.
startMiddle := st.prefixEnd
if startMiddle == 0 && len(msgs) > 0 && msgs[0].Role == llm.RoleSystem {
startMiddle = 1
}
endMiddle := len(msgs) - cfg.KeepRecent
if endMiddle <= startMiddle {
// Nothing new to fold (the tail alone exceeds the threshold).
// Return the rendered slice; the model call may still succeed.
return rendered, nil
}
middle := msgs[startMiddle:endMiddle]
summary, err := summariseMiddle(ctx, cfg, st.summaryText, middle)
if err != nil {
// Non-fatal upstream: the agent loop sends the ORIGINAL slice. Return
// msgs, not `rendered` — on a second+ compaction `rendered` already
// carries a prior synthetic summary, which is not the documented
// "original slice" the loop expects on a compactor error.
return msgs, fmt.Errorf("compactor: summarise middle: %w", err)
}
st.summaryText = summary
st.prefixEnd = endMiddle
out := renderCompacted(st, msgs)
if onFire != nil {
onFire(CompactionEvent{
MessagesBefore: len(rendered),
MessagesAfter: len(out),
TokensBefore: tokensBefore,
TokensAfter: estimateTokens(out),
})
}
return out, nil
}
// renderCompacted applies the fold state to msgs: [optional system] +
// [synthetic summary] + msgs[prefixEnd:]. With no fold yet, msgs is
// returned unchanged.
func renderCompacted(st *compactionState, msgs []llm.Message) []llm.Message {
if st.prefixEnd <= 0 || st.prefixEnd > len(msgs) {
return msgs
}
tail := msgs[st.prefixEnd:]
out := make([]llm.Message, 0, len(tail)+2)
if msgs[0].Role == llm.RoleSystem {
out = append(out, msgs[0])
}
out = append(out, llm.UserText(
"[CONTEXT COMPACTED] The earlier portion of this conversation was summarised "+
"to fit the model's context window. Summary:\n\n"+st.summaryText+
"\n\nResume from the recent messages below."))
out = append(out, tail...)
return out
}
// summariseMiddle composes a "compress this transcript" prompt and
// fires one fast-tier LLM call. prevSummary (may be empty) is the
// running summary from earlier compactions; it is folded into the new
// summary so prior context is not lost.
func summariseMiddle(ctx context.Context, cfg CompactorConfig, prevSummary string, middle []llm.Message) (string, error) {
if len(middle) == 0 {
return "", errors.New("compactor: empty middle range")
}
modelCtx, model, err := cfg.Models(ctx, cfg.SummarizerTier)
if err != nil {
return "", fmt.Errorf("compactor: resolve summarizer model %q: %w", cfg.SummarizerTier, err)
}
if model == nil {
return "", errors.New("compactor: summarizer model resolved to nil")
}
if modelCtx != nil {
ctx = modelCtx
}
var prior string
if strings.TrimSpace(prevSummary) != "" {
prior = "AN EARLIER PORTION WAS ALREADY SUMMARISED AS:\n" + prevSummary +
"\n\nFold that summary into your new one — its facts must survive.\n\n"
}
transcript := renderTranscript(middle)
prompt := fmt.Sprintf(
"You are compressing an in-flight agent's conversation transcript so the agent "+
"can continue working without blowing its model context. The transcript below is "+
"a sequence of tool calls and their results. Produce a single paragraph (under %d words) "+
"that captures:\n"+
" - WHAT the agent has been trying to accomplish.\n"+
" - WHICH URLs were visited / fetched (list inline, comma-separated).\n"+
" - KEY findings or decisions (factual results the agent will need later).\n"+
" - ANY file_ids or KV keys the agent created — these are persistent state references the agent must keep.\n"+
" - ANY errors or dead-ends that the agent should not re-try.\n"+
"DO NOT include verbose HTTP headers, tool-call metadata, error stack traces, or repetitive content. "+
"DO NOT add commentary or markdown headers. Output prose only.\n\n"+
"%sTRANSCRIPT TO COMPRESS:\n%s",
cfg.SummaryWordCap,
prior,
transcript,
)
resp, err := model.Generate(ctx, llm.Request{Messages: []llm.Message{llm.UserText(prompt)}})
if err != nil {
return "", fmt.Errorf("compactor: summarise LLM call: %w", err)
}
text := strings.TrimSpace(resp.Text())
if text == "" {
return "", errors.New("compactor: summarizer returned empty text")
}
return text, nil
}
// estimateTokens is the chars/4 heuristic over a message slice's text,
// tool calls, and tool results. Images count a flat ~1K tokens each.
// It intentionally matches the coarse estimator the old agentkit loop
// used — the 0.7 threshold ratio provides the safety margin.
func estimateTokens(msgs []llm.Message) int {
chars := 0
for _, m := range msgs {
for _, p := range m.Parts {
switch v := p.(type) {
case llm.TextPart:
chars += len(v.Text)
case llm.ImagePart:
chars += 4096
default:
// llm.Part is a sealed-but-extensible interface (future media
// kinds). Count an unknown part conservatively (like an image)
// rather than 0, so a transcript of unrecognised content can't
// silently slip under the compaction threshold and 400 the
// model. Bump this if a large new part kind lands.
_ = v
chars += 4096
}
}
for _, tc := range m.ToolCalls {
chars += len(tc.Name) + len(tc.Arguments)
}
for _, tr := range m.ToolResults {
chars += len(tr.Content)
}
}
return chars / 4
}
// transcriptMessageCap bounds individual message bodies at ~2KB so a
// single ultra-long tool result can't dominate the prompt sent to the
// summarizer.
const transcriptMessageCap = 2048
// secretBearingTools name tools whose ARGUMENTS routinely carry credentials or
// message bodies (bearer tokens, API keys, recipients, request bodies). Their
// args are dropped before the transcript reaches the summarizer model — which
// may be a different provider/tier than the run model — mirroring the redaction
// run/steps.go applies to user-facing step summaries. http_* and webhook_* are
// matched by prefix below.
var secretBearingTools = map[string]bool{
"mcp_call": true,
"email_send": true,
}
// redactToolArgs returns a summariser-safe rendering of a tool call's args:
// "[redacted]" for known secret-bearing tools, the args verbatim otherwise.
func redactToolArgs(name, args string) string {
if secretBearingTools[name] ||
strings.HasPrefix(name, "http_") ||
strings.HasPrefix(name, "webhook_") {
return "[redacted]"
}
return args
}
// renderTranscript flattens a message slice to a plain-text transcript
// suitable for the summarisation prompt. Tool calls show name + (redacted) args,
// tool results show name + body. Empty fields are skipped.
//
// NOTE: tool-RESULT bodies are forwarded to the summarizer (the summary needs
// the findings). A host whose tool results may contain secrets and whose
// summarizer tier resolves to an untrusted provider should ensure that tier is
// trusted, or pre-sanitise results before they reach the agent loop.
func renderTranscript(msgs []llm.Message) string {
var sb strings.Builder
for i, m := range msgs {
fmt.Fprintf(&sb, "---\n[%d] role=%s\n", i+1, m.Role)
if text := m.Text(); text != "" {
sb.WriteString(truncate(text, transcriptMessageCap))
sb.WriteString("\n")
}
for _, tc := range m.ToolCalls {
fmt.Fprintf(&sb, "tool_call name=%s args=%s\n", tc.Name, truncate(redactToolArgs(tc.Name, string(tc.Arguments)), transcriptMessageCap))
}
for _, tr := range m.ToolResults {
fmt.Fprintf(&sb, "tool_result name=%s body=%s\n", tr.Name, truncate(tr.Content, transcriptMessageCap))
}
}
return sb.String()
}
func truncate(s string, n int) string {
if len(s) <= n {
return s
}
return s[:n] + "...(truncated)"
}
+356
View File
@@ -0,0 +1,356 @@
package store
import (
"context"
"database/sql"
"encoding/json"
"errors"
"fmt"
"strings"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/audit"
)
// auditStore is the SQLite-backed audit.Storage: one row per run (+ a JSON
// `inputs` blob), one row per log event. The run-list/filter/walk queries are
// indexed on the columns they filter; the log payload is a JSON blob.
type auditStore struct{ db *sql.DB }
// Audit returns a durable audit.Storage backed by this database.
func (d *DB) Audit() audit.Storage { return &auditStore{db: d.sql} }
var _ audit.Storage = (*auditStore)(nil)
func (s *auditStore) Initialize(ctx context.Context) error {
_, err := s.db.ExecContext(ctx, `
CREATE TABLE IF NOT EXISTS skill_runs (
id TEXT PRIMARY KEY,
skill_id TEXT NOT NULL DEFAULT '',
caller_id TEXT NOT NULL DEFAULT '',
channel_id TEXT NOT NULL DEFAULT '',
parent_run_id TEXT NOT NULL DEFAULT '',
inputs TEXT NOT NULL DEFAULT '{}',
started_at INTEGER NOT NULL DEFAULT 0,
finished_at INTEGER NOT NULL DEFAULT 0, -- 0 = still running
status TEXT NOT NULL DEFAULT 'running',
output TEXT NOT NULL DEFAULT '',
error TEXT NOT NULL DEFAULT '',
tool_calls INTEGER NOT NULL DEFAULT 0,
runtime_seconds REAL NOT NULL DEFAULT 0,
total_input_tokens INTEGER NOT NULL DEFAULT 0,
total_output_tokens INTEGER NOT NULL DEFAULT 0,
total_thinking_tokens INTEGER NOT NULL DEFAULT 0
);
CREATE INDEX IF NOT EXISTS idx_runs_skill ON skill_runs(skill_id, started_at);
CREATE INDEX IF NOT EXISTS idx_runs_caller ON skill_runs(caller_id, started_at);
CREATE INDEX IF NOT EXISTS idx_runs_parent ON skill_runs(parent_run_id);
CREATE INDEX IF NOT EXISTS idx_runs_started ON skill_runs(started_at);
CREATE TABLE IF NOT EXISTS skill_run_logs (
run_id TEXT NOT NULL,
seq INTEGER NOT NULL,
event_type TEXT NOT NULL,
payload TEXT NOT NULL DEFAULT '{}',
created_at INTEGER NOT NULL,
PRIMARY KEY (run_id, seq)
);`)
if err != nil {
return fmt.Errorf("auditStore.Initialize: %w", err)
}
return nil
}
func unixOrZero(t time.Time) int64 {
if t.IsZero() {
return 0
}
return t.Unix()
}
func (s *auditStore) StartRun(ctx context.Context, r audit.SkillRun) error {
inputs, _ := json.Marshal(r.Inputs)
var fin int64
if r.FinishedAt != nil {
fin = unixOrZero(*r.FinishedAt)
}
status := r.Status
if status == "" {
status = "running"
}
_, err := s.db.ExecContext(ctx, `
INSERT INTO skill_runs (id, skill_id, caller_id, channel_id, parent_run_id, inputs, started_at, finished_at,
status, output, error, tool_calls, runtime_seconds, total_input_tokens, total_output_tokens, total_thinking_tokens)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(id) DO UPDATE SET
skill_id=excluded.skill_id, caller_id=excluded.caller_id, channel_id=excluded.channel_id,
parent_run_id=excluded.parent_run_id, inputs=excluded.inputs, started_at=excluded.started_at`,
r.ID, r.SkillID, r.CallerID, r.ChannelID, r.ParentRunID, string(inputs), unixOrZero(r.StartedAt), fin,
status, r.Output, r.Error, r.ToolCallsCount, r.RuntimeSeconds,
r.TotalInputTokens, r.TotalOutputTokens, r.TotalThinkingTokens)
if err != nil {
return fmt.Errorf("auditStore.StartRun: %w", err)
}
return nil
}
func (s *auditStore) FinishRun(ctx context.Context, runID string, st audit.RunStats) error {
res, err := s.db.ExecContext(ctx, `
UPDATE skill_runs SET finished_at=?, status=?, output=?, error=?, tool_calls=?, runtime_seconds=?,
total_input_tokens=?, total_output_tokens=?, total_thinking_tokens=? WHERE id=?`,
time.Now().Unix(), st.Status, st.Output, st.Error, st.ToolCalls, st.RuntimeSeconds,
st.InputTokens, st.OutputTokens, st.ThinkingTokens, runID)
if err != nil {
return fmt.Errorf("auditStore.FinishRun: %w", err)
}
if n, _ := res.RowsAffected(); n == 0 {
return audit.ErrNotFound
}
return nil
}
func (s *auditStore) AppendLog(ctx context.Context, l audit.SkillRunLog) error {
payload, _ := json.Marshal(l.Payload)
created := unixOrZero(l.CreatedAt)
if created == 0 {
created = time.Now().Unix()
}
_, err := s.db.ExecContext(ctx,
`INSERT OR REPLACE INTO skill_run_logs (run_id, seq, event_type, payload, created_at) VALUES (?, ?, ?, ?, ?)`,
l.RunID, l.Sequence, l.EventType, string(payload), created)
if err != nil {
return fmt.Errorf("auditStore.AppendLog: %w", err)
}
return nil
}
// runCols is the SELECT column list matching scanRun.
const runCols = `id, skill_id, caller_id, channel_id, parent_run_id, inputs, started_at, finished_at,
status, output, error, tool_calls, runtime_seconds, total_input_tokens, total_output_tokens, total_thinking_tokens`
func scanRun(sc interface{ Scan(...any) error }) (*audit.SkillRun, error) {
var r audit.SkillRun
var inputs string
var started, finished int64
if err := sc.Scan(&r.ID, &r.SkillID, &r.CallerID, &r.ChannelID, &r.ParentRunID, &inputs,
&started, &finished, &r.Status, &r.Output, &r.Error, &r.ToolCallsCount, &r.RuntimeSeconds,
&r.TotalInputTokens, &r.TotalOutputTokens, &r.TotalThinkingTokens); err != nil {
return nil, err
}
_ = json.Unmarshal([]byte(inputs), &r.Inputs)
r.StartedAt = time.Unix(started, 0).UTC()
if finished > 0 {
t := time.Unix(finished, 0).UTC()
r.FinishedAt = &t
}
return &r, nil
}
func (s *auditStore) GetRun(ctx context.Context, runID string) (*audit.SkillRun, error) {
row := s.db.QueryRowContext(ctx, `SELECT `+runCols+` FROM skill_runs WHERE id = ?`, runID)
r, err := scanRun(row)
if errors.Is(err, sql.ErrNoRows) {
return nil, audit.ErrNotFound
}
return r, err
}
func (s *auditStore) queryRuns(ctx context.Context, tail string, args ...any) ([]audit.SkillRun, error) {
rows, err := s.db.QueryContext(ctx, `SELECT `+runCols+` FROM skill_runs `+tail, args...)
if err != nil {
return nil, err
}
defer rows.Close()
var out []audit.SkillRun
for rows.Next() {
r, err := scanRun(rows)
if err != nil {
return nil, err
}
out = append(out, *r)
}
return out, rows.Err()
}
func (s *auditStore) ListLogsByRun(ctx context.Context, runID string) ([]audit.SkillRunLog, error) {
rows, err := s.db.QueryContext(ctx,
`SELECT run_id, seq, event_type, payload, created_at FROM skill_run_logs WHERE run_id = ? ORDER BY seq`, runID)
if err != nil {
return nil, fmt.Errorf("auditStore.ListLogsByRun: %w", err)
}
defer rows.Close()
var out []audit.SkillRunLog
for rows.Next() {
var l audit.SkillRunLog
var payload string
var created int64
if err := rows.Scan(&l.RunID, &l.Sequence, &l.EventType, &payload, &created); err != nil {
return nil, err
}
_ = json.Unmarshal([]byte(payload), &l.Payload)
l.CreatedAt = time.Unix(created, 0).UTC()
out = append(out, l)
}
return out, rows.Err()
}
func (s *auditStore) ListRunsBySkill(ctx context.Context, skillID string, limit int) ([]audit.SkillRun, error) {
return s.ListRunsBySkillPaginated(ctx, skillID, 0, limit, false)
}
func (s *auditStore) ListRunsBySkillPaginated(ctx context.Context, skillID string, offset, limit int, includeDryRun bool) ([]audit.SkillRun, error) {
w := `WHERE skill_id = ?`
args := []any{skillID}
if !includeDryRun {
w += ` AND status != 'dry_run'`
}
return s.queryRuns(ctx, w+` ORDER BY started_at DESC `+limitOffset(limit, offset), args...)
}
func (s *auditStore) CountRunsBySkill(ctx context.Context, skillID string, includeDryRun bool) (int64, error) {
q := `SELECT COUNT(*) FROM skill_runs WHERE skill_id = ?`
if !includeDryRun {
q += ` AND status != 'dry_run'`
}
var n int64
err := s.db.QueryRowContext(ctx, q, skillID).Scan(&n)
return n, err
}
func (s *auditStore) ListRunsByCaller(ctx context.Context, callerID string, limit int) ([]audit.SkillRun, error) {
return s.queryRuns(ctx, `WHERE caller_id = ? AND status != 'dry_run' ORDER BY started_at DESC `+limitOffset(limit, 0), callerID)
}
func (s *auditStore) buildFilter(f audit.RunFilter) (string, []any) {
var conds []string
var args []any
if !f.IncludeDryRun {
conds = append(conds, `status != 'dry_run'`)
}
if f.Status != "" {
conds = append(conds, `status = ?`)
args = append(args, f.Status)
}
if f.SkillID != "" {
conds = append(conds, `skill_id = ?`)
args = append(args, f.SkillID)
}
if f.CallerID != "" {
conds = append(conds, `caller_id = ?`)
args = append(args, f.CallerID)
}
if f.ChannelID != "" {
conds = append(conds, `channel_id = ?`)
args = append(args, f.ChannelID)
}
if f.TopLevelOnly {
conds = append(conds, `parent_run_id = ''`)
}
if !f.Since.IsZero() {
conds = append(conds, `started_at >= ?`)
args = append(args, f.Since.Unix())
}
if !f.Until.IsZero() {
conds = append(conds, `started_at <= ?`)
args = append(args, f.Until.Unix())
}
where := ""
if len(conds) > 0 {
where = `WHERE ` + strings.Join(conds, " AND ")
}
return where, args
}
func (s *auditStore) ListRunsFiltered(ctx context.Context, f audit.RunFilter, offset, limit int) ([]audit.SkillRun, error) {
where, args := s.buildFilter(f)
return s.queryRuns(ctx, where+` ORDER BY started_at DESC `+limitOffset(limit, offset), args...)
}
func (s *auditStore) CountRunsFiltered(ctx context.Context, f audit.RunFilter) (int64, error) {
where, args := s.buildFilter(f)
var n int64
err := s.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM skill_runs `+where, args...).Scan(&n)
return n, err
}
func (s *auditStore) PurgeOlderThan(ctx context.Context, t time.Time) (int64, error) {
res, err := s.db.ExecContext(ctx, `DELETE FROM skill_runs WHERE finished_at > 0 AND finished_at < ?`, t.Unix())
if err != nil {
return 0, fmt.Errorf("auditStore.PurgeOlderThan: %w", err)
}
n, _ := res.RowsAffected()
// Best-effort orphan-log cleanup.
_, _ = s.db.ExecContext(ctx, `DELETE FROM skill_run_logs WHERE run_id NOT IN (SELECT id FROM skill_runs)`)
return n, nil
}
func (s *auditStore) ListChildrenByParent(ctx context.Context, parentRunID string) ([]audit.SkillRun, error) {
return s.queryRuns(ctx, `WHERE parent_run_id = ? ORDER BY started_at DESC`, parentRunID)
}
func (s *auditStore) WalkParentChain(ctx context.Context, runID string) ([]audit.SkillRun, error) {
var chain []audit.SkillRun
seen := map[string]bool{}
for id := runID; id != ""; {
if seen[id] {
break
}
seen[id] = true
r, err := s.GetRun(ctx, id)
if errors.Is(err, audit.ErrNotFound) {
break
}
if err != nil {
return nil, err
}
chain = append(chain, *r)
id = r.ParentRunID
}
return chain, nil
}
func (s *auditStore) ListFinishedRunsBefore(ctx context.Context, cutoff time.Time, limit int) ([]audit.SkillRun, error) {
return s.queryRuns(ctx,
`WHERE finished_at > 0 AND finished_at < ? ORDER BY started_at DESC `+limitOffset(limit, 0), cutoff.Unix())
}
func (s *auditStore) LastRunBySkills(ctx context.Context, skillIDs []string, includeFailed bool) (map[string]time.Time, error) {
out := map[string]time.Time{}
if len(skillIDs) == 0 {
return out, nil
}
q := `SELECT skill_id, MAX(started_at) FROM skill_runs WHERE skill_id IN (` +
strings.TrimSuffix(strings.Repeat("?,", len(skillIDs)), ",") + `)`
args := make([]any, 0, len(skillIDs))
for _, id := range skillIDs {
args = append(args, id)
}
if !includeFailed {
q += ` AND status NOT IN ('error','timeout')`
}
q += ` GROUP BY skill_id`
rows, err := s.db.QueryContext(ctx, q, args...)
if err != nil {
return nil, fmt.Errorf("auditStore.LastRunBySkills: %w", err)
}
defer rows.Close()
for rows.Next() {
var id string
var ts int64
if err := rows.Scan(&id, &ts); err != nil {
return nil, err
}
out[id] = time.Unix(ts, 0).UTC()
}
return out, rows.Err()
}
// limitOffset renders an optional LIMIT/OFFSET clause (limit<=0 = no limit).
func limitOffset(limit, offset int) string {
if limit <= 0 {
return ""
}
if offset > 0 {
return fmt.Sprintf("LIMIT %d OFFSET %d", limit, offset)
}
return fmt.Sprintf("LIMIT %d", limit)
}
+67
View File
@@ -0,0 +1,67 @@
package store
import (
"context"
"testing"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/audit"
)
func TestSQLiteAuditStore(t *testing.T) {
ctx := context.Background()
db, err := Open(":memory:")
if err != nil {
t.Fatal(err)
}
defer db.Close()
st := db.Audit()
if err := st.Initialize(ctx); err != nil {
t.Fatal(err)
}
now := time.Now().UTC()
// parent run
if err := st.StartRun(ctx, audit.SkillRun{ID: "r1", SkillID: "agent-x", CallerID: "c1",
Inputs: map[string]any{"q": "hi"}, StartedAt: now}); err != nil {
t.Fatal(err)
}
// child run
st.StartRun(ctx, audit.SkillRun{ID: "r2", SkillID: "skill-y", CallerID: "c1", ParentRunID: "r1", StartedAt: now.Add(time.Second)})
st.AppendLog(ctx, audit.SkillRunLog{RunID: "r1", Sequence: 1, EventType: "step", Payload: map[string]any{"i": 1}, CreatedAt: now})
if err := st.FinishRun(ctx, "r1", audit.RunStats{Status: "ok", Output: "done", ToolCalls: 2, InputTokens: 10, OutputTokens: 5}); err != nil {
t.Fatal(err)
}
got, err := st.GetRun(ctx, "r1")
if err != nil || got.Status != "ok" || got.Output != "done" || got.FinishedAt == nil ||
got.Inputs["q"] != "hi" || got.TotalInputTokens != 10 {
t.Fatalf("GetRun: %v %+v", err, got)
}
if logs, _ := st.ListLogsByRun(ctx, "r1"); len(logs) != 1 || logs[0].EventType != "step" {
t.Errorf("ListLogsByRun = %+v", logs)
}
if kids, _ := st.ListChildrenByParent(ctx, "r1"); len(kids) != 1 || kids[0].ID != "r2" {
t.Errorf("ListChildrenByParent = %+v", kids)
}
if chain, _ := st.WalkParentChain(ctx, "r2"); len(chain) != 2 || chain[1].ID != "r1" {
t.Errorf("WalkParentChain = %+v", chain)
}
if byCaller, _ := st.ListRunsByCaller(ctx, "c1", 10); len(byCaller) != 2 {
t.Errorf("ListRunsByCaller = %d, want 2", len(byCaller))
}
// filter: top-level only
tl, _ := st.ListRunsFiltered(ctx, audit.RunFilter{TopLevelOnly: true}, 0, 10)
if len(tl) != 1 || tl[0].ID != "r1" {
t.Errorf("TopLevelOnly filter = %+v", tl)
}
// last-run map
last, _ := st.LastRunBySkills(ctx, []string{"agent-x", "skill-y"}, true)
if _, ok := last["agent-x"]; !ok {
t.Errorf("LastRunBySkills missing agent-x: %+v", last)
}
if n, _ := st.CountRunsBySkill(ctx, "agent-x", false); n != 1 {
t.Errorf("CountRunsBySkill = %d, want 1", n)
}
}
+105
View File
@@ -0,0 +1,105 @@
package store
import (
"context"
"database/sql"
"errors"
"fmt"
"math"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/budget"
)
// budgetStore is the SQLite-backed budget.BudgetStorage.
type budgetStore struct{ db *sql.DB }
// Budget returns a durable budget.BudgetStorage backed by this database.
func (d *DB) Budget() budget.BudgetStorage { return &budgetStore{db: d.sql} }
var _ budget.BudgetStorage = (*budgetStore)(nil)
func (s *budgetStore) Initialize(ctx context.Context) error {
_, err := s.db.ExecContext(ctx, `
CREATE TABLE IF NOT EXISTS skill_budgets (
user_id TEXT PRIMARY KEY,
window_start INTEGER NOT NULL, -- unix seconds
seconds_used REAL NOT NULL,
runs_count INTEGER NOT NULL,
updated_at INTEGER NOT NULL
)`)
if err != nil {
return fmt.Errorf("budgetStore.Initialize: %w", err)
}
return nil
}
func (s *budgetStore) Get(ctx context.Context, userID string) (*budget.SkillBudget, error) {
row := s.db.QueryRowContext(ctx,
`SELECT window_start, seconds_used, runs_count, updated_at FROM skill_budgets WHERE user_id = ?`, userID)
var ws, ua int64
var used float64
var runs int
switch err := row.Scan(&ws, &used, &runs, &ua); {
case errors.Is(err, sql.ErrNoRows):
return nil, nil // no row yet — documented (nil, nil)
case err != nil:
return nil, fmt.Errorf("budgetStore.Get: %w", err)
}
return &budget.SkillBudget{
UserID: userID,
WindowStart: time.Unix(ws, 0).UTC(),
SecondsUsed: used,
RunsCount: runs,
UpdatedAt: time.Unix(ua, 0).UTC(),
}, nil
}
// Add increments usage atomically, rolling the 7-day window over inside one
// transaction so concurrent Adds can't race the read-modify-write.
func (s *budgetStore) Add(ctx context.Context, userID string, secondsUsed float64, now time.Time) error {
// A NaN/Inf would poison the seconds_used column irrecoverably (NaN
// propagates through every later add), so reject it at the boundary.
if math.IsNaN(secondsUsed) || math.IsInf(secondsUsed, 0) {
return fmt.Errorf("budgetStore.Add: invalid secondsUsed %v", secondsUsed)
}
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return fmt.Errorf("budgetStore.Add: begin: %w", err)
}
defer tx.Rollback() //nolint:errcheck // no-op after Commit
var ws int64
var used float64
var runs int
err = tx.QueryRowContext(ctx,
`SELECT window_start, seconds_used, runs_count FROM skill_budgets WHERE user_id = ?`, userID).
Scan(&ws, &used, &runs)
switch {
case errors.Is(err, sql.ErrNoRows):
ws, used, runs = now.Unix(), 0, 0
case err != nil:
return fmt.Errorf("budgetStore.Add: select: %w", err)
}
// Roll the window over if older than 7 days.
if now.Sub(time.Unix(ws, 0)) >= 7*24*time.Hour {
ws, used, runs = now.Unix(), 0, 0
}
used += secondsUsed
runs++
if _, err := tx.ExecContext(ctx, `
INSERT INTO skill_budgets (user_id, window_start, seconds_used, runs_count, updated_at)
VALUES (?, ?, ?, ?, ?)
ON CONFLICT(user_id) DO UPDATE SET
window_start = excluded.window_start,
seconds_used = excluded.seconds_used,
runs_count = excluded.runs_count,
updated_at = excluded.updated_at`,
userID, ws, used, runs, now.Unix()); err != nil {
return fmt.Errorf("budgetStore.Add: upsert: %w", err)
}
if err := tx.Commit(); err != nil {
return fmt.Errorf("budgetStore.Add: commit: %w", err)
}
return nil
}
+65
View File
@@ -0,0 +1,65 @@
package store
import (
"context"
"errors"
"testing"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/budget"
)
// TestSQLiteBudgetConformance runs the budget battery over the SQLite store and
// asserts the same rolling-window contract the in-memory store must satisfy.
func TestSQLiteBudgetConformance(t *testing.T) {
ctx := context.Background()
db, err := Open(":memory:")
if err != nil {
t.Fatal(err)
}
defer db.Close()
st := db.Budget()
if err := st.Initialize(ctx); err != nil {
t.Fatal(err)
}
now := time.Now().UTC()
b := budget.NewDBBudget(st, func() float64 { return 100 }, nil, func() time.Time { return now })
if err := b.Check(ctx, "u"); err != nil {
t.Fatalf("fresh caller should pass: %v", err)
}
b.Commit(ctx, "u", 60)
if err := b.Check(ctx, "u"); err != nil {
t.Fatalf("60/100 should pass: %v", err)
}
b.Commit(ctx, "u", 50) // 110 total
if err := b.Check(ctx, "u"); !errors.Is(err, budget.ErrBudgetExceeded) {
t.Fatalf("110/100 should be ErrBudgetExceeded, got %v", err)
}
// Direct Get reflects the persisted row.
row, err := st.Get(ctx, "u")
if err != nil || row == nil {
t.Fatalf("Get: %v %+v", err, row)
}
if row.SecondsUsed != 110 || row.RunsCount != 2 {
t.Errorf("row = %+v, want seconds=110 runs=2", row)
}
// Window rolls over after 7 days.
now = now.Add(8 * 24 * time.Hour)
b.Commit(ctx, "u", 1)
if err := b.Check(ctx, "u"); err != nil {
t.Fatalf("after rollover should pass: %v", err)
}
row, _ = st.Get(ctx, "u")
if row.SecondsUsed != 1 || row.RunsCount != 1 {
t.Errorf("post-rollover row = %+v, want seconds=1 runs=1", row)
}
// Unknown user -> (nil, nil).
if r, err := st.Get(ctx, "nobody"); err != nil || r != nil {
t.Errorf("Get(unknown) = %+v %v, want nil,nil", r, err)
}
}
+54
View File
@@ -0,0 +1,54 @@
module gitea.stevedudenhoeffer.com/steve/executus/contrib/store
go 1.26.2
require (
gitea.stevedudenhoeffer.com/steve/executus v0.0.0
modernc.org/sqlite v1.34.4
)
require (
cloud.google.com/go v0.123.0 // indirect
cloud.google.com/go/auth v0.18.1 // indirect
cloud.google.com/go/compute/metadata v0.9.0 // indirect
gitea.stevedudenhoeffer.com/steve/majordomo v0.0.0-20260626223738-1fd7109a42f3 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/google/go-cmp v0.7.0 // indirect
github.com/google/s2a-go v0.1.9 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.11 // indirect
github.com/googleapis/gax-go/v2 v2.17.0 // indirect
github.com/gorilla/websocket v1.5.3 // indirect
github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/ncruces/go-strftime v0.1.9 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
github.com/robfig/cron/v3 v3.0.1 // indirect
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.65.0 // indirect
go.opentelemetry.io/otel v1.40.0 // indirect
go.opentelemetry.io/otel/metric v1.40.0 // indirect
go.opentelemetry.io/otel/trace v1.40.0 // indirect
golang.org/x/crypto v0.53.0 // indirect
golang.org/x/net v0.55.0 // indirect
golang.org/x/sys v0.46.0 // indirect
golang.org/x/text v0.38.0 // indirect
google.golang.org/genai v1.59.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20260203192932-546029d2fa20 // indirect
google.golang.org/grpc v1.78.0 // indirect
google.golang.org/protobuf v1.36.11 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6 // indirect
modernc.org/libc v1.55.3 // indirect
modernc.org/mathutil v1.6.0 // indirect
modernc.org/memory v1.8.0 // indirect
modernc.org/strutil v1.2.0 // indirect
modernc.org/token v1.1.0 // indirect
)
// Co-developed against the local checkout; dropped (pinned) at executus v0.1.0.
replace gitea.stevedudenhoeffer.com/steve/executus => ../../
+105
View File
@@ -0,0 +1,105 @@
cloud.google.com/go v0.123.0 h1:2NAUJwPR47q+E35uaJeYoNhuNEM9kM8SjgRgdeOJUSE=
cloud.google.com/go v0.123.0/go.mod h1:xBoMV08QcqUGuPW65Qfm1o9Y4zKZBpGS+7bImXLTAZU=
cloud.google.com/go/auth v0.18.1 h1:IwTEx92GFUo2pJ6Qea0EU3zYvKnTAeRCODxfA/G5UWs=
cloud.google.com/go/auth v0.18.1/go.mod h1:GfTYoS9G3CWpRA3Va9doKN9mjPGRS+v41jmZAhBzbrA=
cloud.google.com/go/compute/metadata v0.9.0 h1:pDUj4QMoPejqq20dK0Pg2N4yG9zIkYGdBtwLoEkH9Zs=
cloud.google.com/go/compute/metadata v0.9.0/go.mod h1:E0bWwX5wTnLPedCKqk3pJmVgCBSM6qQI1yTBdEb3C10=
gitea.stevedudenhoeffer.com/steve/majordomo v0.0.0-20260626223738-1fd7109a42f3 h1:KYKIFFRsXzbbBJVDa99+Fhy0zxl9G0xV/MCrLipsLL4=
gitea.stevedudenhoeffer.com/steve/majordomo v0.0.0-20260626223738-1fd7109a42f3/go.mod h1:UZLveG17SmENt4sne2RSLIbioix30RZbRIQUzBAnOyY=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=
github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/pprof v0.0.0-20240409012703-83162a5b38cd h1:gbpYu9NMq8jhDVbvlGkMFWCjLFlqqEZjEmObmhUy6Vo=
github.com/google/pprof v0.0.0-20240409012703-83162a5b38cd/go.mod h1:kf6iHlnVGwgKolg33glAes7Yg/8iWP8ukqeldJSO7jw=
github.com/google/s2a-go v0.1.9 h1:LGD7gtMgezd8a/Xak7mEWL0PjoTQFvpRudN895yqKW0=
github.com/google/s2a-go v0.1.9/go.mod h1:YA0Ei2ZQL3acow2O62kdp9UlnvMmU7kA6Eutn0dXayM=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/googleapis/enterprise-certificate-proxy v0.3.11 h1:vAe81Msw+8tKUxi2Dqh/NZMz7475yUvmRIkXr4oN2ao=
github.com/googleapis/enterprise-certificate-proxy v0.3.11/go.mod h1:RFV7MUdlb7AgEq2v7FmMCfeSMCllAzWxFgRdusoGks8=
github.com/googleapis/gax-go/v2 v2.17.0 h1:RksgfBpxqff0EZkDWYuz9q/uWsTVz+kf43LsZ1J6SMc=
github.com/googleapis/gax-go/v2 v2.17.0/go.mod h1:mzaqghpQp4JDh3HvADwrat+6M3MOIDp5YKHhb9PAgDY=
github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg=
github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/ncruces/go-strftime v0.1.9 h1:bY0MQC28UADQmHmaF5dgpLmImcShSi2kHU9XLdhx/f4=
github.com/ncruces/go-strftime v0.1.9/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
github.com/robfig/cron/v3 v3.0.1 h1:WdRxkvbJztn8LMz/QEvLN5sBU+xKpSqwwUO1Pjr4qDs=
github.com/robfig/cron/v3 v3.0.1/go.mod h1:eQICP3HwyT7UooqI/z+Ov+PtYAWygg1TEWWzGIFLtro=
go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=
go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.65.0 h1:7iP2uCb7sGddAr30RRS6xjKy7AZ2JtTOPA3oolgVSw8=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.65.0/go.mod h1:c7hN3ddxs/z6q9xwvfLPk+UHlWRQyaeR1LdgfL/66l0=
go.opentelemetry.io/otel v1.40.0 h1:oA5YeOcpRTXq6NN7frwmwFR0Cn3RhTVZvXsP4duvCms=
go.opentelemetry.io/otel v1.40.0/go.mod h1:IMb+uXZUKkMXdPddhwAHm6UfOwJyh4ct1ybIlV14J0g=
go.opentelemetry.io/otel/metric v1.40.0 h1:rcZe317KPftE2rstWIBitCdVp89A2HqjkxR3c11+p9g=
go.opentelemetry.io/otel/metric v1.40.0/go.mod h1:ib/crwQH7N3r5kfiBZQbwrTge743UDc7DTFVZrrXnqc=
go.opentelemetry.io/otel/trace v1.40.0 h1:WA4etStDttCSYuhwvEa8OP8I5EWu24lkOzp+ZYblVjw=
go.opentelemetry.io/otel/trace v1.40.0/go.mod h1:zeAhriXecNGP/s2SEG3+Y8X9ujcJOTqQ5RgdEJcawiA=
golang.org/x/crypto v0.53.0 h1:QZ4Muo8THX6CizN2vPPd5fBGHyogrdK9fG4wLPFUsto=
golang.org/x/crypto v0.53.0/go.mod h1:DNLU434OwVakk9PzuwV8w62mAJpRJL3vsgcfp4Qnsio=
golang.org/x/mod v0.16.0 h1:QX4fJ0Rr5cPQCF7O9lh9Se4pmwfwskqZfq5moyldzic=
golang.org/x/mod v0.16.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/net v0.55.0 h1:bcvxaJn3e1U6InsFWt1JUq1aSjnRxLzT2rtD2KfkDF8=
golang.org/x/net v0.55.0/go.mod h1:L5U2KuzuOe1lY7Z+aWVIKK6qEeJXnXV9yzGA+WCHJww=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.46.0 h1:noSf2Fq6F8DBgS+LysIkx7rIExoNHJsxOAtPp4rthXw=
golang.org/x/sys v0.46.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/text v0.38.0 h1:sXmwo9DwP3OK9EZ7PqAdaooSGozfl/3a6/xJcbzPRhE=
golang.org/x/text v0.38.0/go.mod h1:YXZt3QhHUKYT53r2lLKFIVi6Ao1jdzrTR/KQ09qyxF4=
golang.org/x/tools v0.19.0 h1:tfGCXNR1OsFG+sVdLAitlpjAvD/I6dHDKnYrpEZUHkw=
golang.org/x/tools v0.19.0/go.mod h1:qoJWxmGSIBmAeriMx19ogtrEPrGtDbPK634QFIcLAhc=
google.golang.org/genai v1.59.0 h1:xp+ydkJFW8hO0hTUaAkr8TrLM9HFP3NYAwFhPd0nDqA=
google.golang.org/genai v1.59.0/go.mod h1:mDdPDFXo1Ats7f1WXVyZgWb/CkMzFWTWJruIMy7hGIU=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260203192932-546029d2fa20 h1:Jr5R2J6F6qWyzINc+4AM8t5pfUz6beZpHp678GNrMbE=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260203192932-546029d2fa20/go.mod h1:j9x/tPzZkyxcgEFkiKEEGxfvyumM01BEtsW8xzOahRQ=
google.golang.org/grpc v1.78.0 h1:K1XZG/yGDJnzMdd/uZHAkVqJE+xIDOcmdSFZkBUicNc=
google.golang.org/grpc v1.78.0/go.mod h1:I47qjTo4OKbMkjA/aOOwxDIiPSBofUtQUI5EfpWvW7U=
google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
modernc.org/cc/v4 v4.21.4 h1:3Be/Rdo1fpr8GrQ7IVw9OHtplU4gWbb+wNgeoBMmGLQ=
modernc.org/cc/v4 v4.21.4/go.mod h1:HM7VJTZbUCR3rV8EYBi9wxnJ0ZBRiGE5OeGXNA0IsLQ=
modernc.org/ccgo/v4 v4.19.2 h1:lwQZgvboKD0jBwdaeVCTouxhxAyN6iawF3STraAal8Y=
modernc.org/ccgo/v4 v4.19.2/go.mod h1:ysS3mxiMV38XGRTTcgo0DQTeTmAO4oCmJl1nX9VFI3s=
modernc.org/fileutil v1.3.0 h1:gQ5SIzK3H9kdfai/5x41oQiKValumqNTDXMvKo62HvE=
modernc.org/fileutil v1.3.0/go.mod h1:XatxS8fZi3pS8/hKG2GH/ArUogfxjpEKs3Ku3aK4JyQ=
modernc.org/gc/v2 v2.4.1 h1:9cNzOqPyMJBvrUipmynX0ZohMhcxPtMccYgGOJdOiBw=
modernc.org/gc/v2 v2.4.1/go.mod h1:wzN5dK1AzVGoH6XOzc3YZ+ey/jPgYHLuVckd62P0GYU=
modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6 h1:5D53IMaUuA5InSeMu9eJtlQXS2NxAhyWQvkKEgXZhHI=
modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6/go.mod h1:Qz0X07sNOR1jWYCrJMEnbW/X55x206Q7Vt4mz6/wHp4=
modernc.org/libc v1.55.3 h1:AzcW1mhlPNrRtjS5sS+eW2ISCgSOLLNyFzRh/V3Qj/U=
modernc.org/libc v1.55.3/go.mod h1:qFXepLhz+JjFThQ4kzwzOjA/y/artDeg+pcYnY+Q83w=
modernc.org/mathutil v1.6.0 h1:fRe9+AmYlaej+64JsEEhoWuAYBkOtQiMEU7n/XgfYi4=
modernc.org/mathutil v1.6.0/go.mod h1:Ui5Q9q1TR2gFm0AQRqQUaBWFLAhQpCwNcuhBOSedWPo=
modernc.org/memory v1.8.0 h1:IqGTL6eFMaDZZhEWwcREgeMXYwmW83LYW8cROZYkg+E=
modernc.org/memory v1.8.0/go.mod h1:XPZ936zp5OMKGWPqbD3JShgd/ZoQ7899TUuQqxY+peU=
modernc.org/opt v0.1.3 h1:3XOZf2yznlhC+ibLltsDGzABUGVx8J6pnFMS3E4dcq4=
modernc.org/opt v0.1.3/go.mod h1:WdSiB5evDcignE70guQKxYUl14mgWtbClRi5wmkkTX0=
modernc.org/sortutil v1.2.0 h1:jQiD3PfS2REGJNzNCMMaLSp/wdMNieTbKX920Cqdgqc=
modernc.org/sortutil v1.2.0/go.mod h1:TKU2s7kJMf1AE84OoiGppNHJwvB753OYfNl2WRb++Ss=
modernc.org/sqlite v1.34.4 h1:sjdARozcL5KJBvYQvLlZEmctRgW9xqIZc2ncN7PU0P8=
modernc.org/sqlite v1.34.4/go.mod h1:3QQFCG2SEMtc2nv+Wq4cQCH7Hjcg+p/RMlS1XK+zwbk=
modernc.org/strutil v1.2.0 h1:agBi9dp1I+eOnxXeiZawM8F4LawKv4NzGWSaLfyeNZA=
modernc.org/strutil v1.2.0/go.mod h1:/mdcBmfOibveCTBxUl5B5l6W+TTH1FXPLHZE6bTosX0=
modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=
+174
View File
@@ -0,0 +1,174 @@
package store
import (
"context"
"database/sql"
"encoding/json"
"errors"
"fmt"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/persona"
)
// personaStore is the SQLite-backed persona.Storage. It stores each Agent as a
// JSON blob in `data` with a handful of extracted, indexed columns for the
// query methods — so the FULL struct round-trips (no domain↔GORM↔DB field-loss
// footgun) while owner/name/webhook/schedule lookups stay indexable.
type personaStore struct{ db *sql.DB }
// Personas returns a durable persona.Storage backed by this database.
func (d *DB) Personas() persona.Storage { return &personaStore{db: d.sql} }
var _ persona.Storage = (*personaStore)(nil)
func (s *personaStore) InitializeAgentStorage(ctx context.Context) error {
_, err := s.db.ExecContext(ctx, `
CREATE TABLE IF NOT EXISTS agents (
id TEXT PRIMARY KEY,
owner_id TEXT NOT NULL DEFAULT '',
name TEXT NOT NULL DEFAULT '',
webhook_secret TEXT NOT NULL DEFAULT '',
chatbot_channel_filter TEXT NOT NULL DEFAULT '',
schedule TEXT NOT NULL DEFAULT '',
next_run_at INTEGER NOT NULL DEFAULT 0, -- unix seconds; 0 = unset
data TEXT NOT NULL -- full Agent as JSON
);
CREATE INDEX IF NOT EXISTS idx_agents_owner ON agents(owner_id);
CREATE UNIQUE INDEX IF NOT EXISTS idx_agents_owner_name ON agents(owner_id, name);
CREATE INDEX IF NOT EXISTS idx_agents_sched ON agents(schedule, next_run_at);`)
if err != nil {
return fmt.Errorf("personaStore.Initialize: %w", err)
}
return nil
}
func (s *personaStore) SaveAgent(ctx context.Context, a *persona.Agent) error {
blob, err := json.Marshal(a)
if err != nil {
return fmt.Errorf("personaStore.SaveAgent: marshal: %w", err)
}
var next int64
if a.NextRunAt != nil && !a.NextRunAt.IsZero() {
next = a.NextRunAt.Unix()
}
_, err = s.db.ExecContext(ctx, `
INSERT INTO agents (id, owner_id, name, webhook_secret, chatbot_channel_filter, schedule, next_run_at, data)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(id) DO UPDATE SET
owner_id=excluded.owner_id, name=excluded.name, webhook_secret=excluded.webhook_secret,
chatbot_channel_filter=excluded.chatbot_channel_filter, schedule=excluded.schedule,
next_run_at=excluded.next_run_at, data=excluded.data`,
a.ID, a.OwnerID, a.Name, a.WebhookSecret, a.ChatbotChannelFilter, a.Schedule, next, string(blob))
if err != nil {
return fmt.Errorf("personaStore.SaveAgent: %w", err)
}
return nil
}
// scanAgents unmarshals the `data` column of every row in rows.
func scanAgents(rows *sql.Rows) ([]*persona.Agent, error) {
defer rows.Close()
var out []*persona.Agent
for rows.Next() {
var blob string
if err := rows.Scan(&blob); err != nil {
return nil, err
}
var a persona.Agent
if err := json.Unmarshal([]byte(blob), &a); err != nil {
return nil, err
}
out = append(out, &a)
}
return out, rows.Err()
}
func (s *personaStore) getOne(ctx context.Context, where string, arg ...any) (*persona.Agent, error) {
var blob string
err := s.db.QueryRowContext(ctx, `SELECT data FROM agents WHERE `+where, arg...).Scan(&blob)
switch {
case errors.Is(err, sql.ErrNoRows):
return nil, persona.ErrNotFound
case err != nil:
return nil, err
}
var a persona.Agent
if err := json.Unmarshal([]byte(blob), &a); err != nil {
return nil, err
}
return &a, nil
}
func (s *personaStore) GetAgent(ctx context.Context, id string) (*persona.Agent, error) {
return s.getOne(ctx, "id = ?", id)
}
func (s *personaStore) GetAgentByName(ctx context.Context, ownerID, name string) (*persona.Agent, error) {
return s.getOne(ctx, "owner_id = ? AND name = ?", ownerID, name)
}
func (s *personaStore) GetAgentByWebhookSecret(ctx context.Context, secret string) (*persona.Agent, error) {
if secret == "" {
return nil, persona.ErrNotFound
}
return s.getOne(ctx, "webhook_secret = ?", secret)
}
func (s *personaStore) ListAgents(ctx context.Context, ownerID string) ([]*persona.Agent, error) {
rows, err := s.db.QueryContext(ctx, `SELECT data FROM agents WHERE owner_id = ? ORDER BY name`, ownerID)
if err != nil {
return nil, fmt.Errorf("personaStore.ListAgents: %w", err)
}
return scanAgents(rows)
}
func (s *personaStore) ListAllAgents(ctx context.Context) ([]*persona.Agent, error) {
rows, err := s.db.QueryContext(ctx, `SELECT data FROM agents ORDER BY name`)
if err != nil {
return nil, fmt.Errorf("personaStore.ListAllAgents: %w", err)
}
return scanAgents(rows)
}
func (s *personaStore) DeleteAgent(ctx context.Context, id string) error {
if _, err := s.db.ExecContext(ctx, `DELETE FROM agents WHERE id = ?`, id); err != nil {
return fmt.Errorf("personaStore.DeleteAgent: %w", err)
}
return nil
}
func (s *personaStore) ListAgentsByChatbotChannelFilter(ctx context.Context) ([]*persona.Agent, error) {
rows, err := s.db.QueryContext(ctx, `SELECT data FROM agents WHERE chatbot_channel_filter != '' ORDER BY name`)
if err != nil {
return nil, fmt.Errorf("personaStore.ListAgentsByChatbotChannelFilter: %w", err)
}
return scanAgents(rows)
}
func (s *personaStore) ListScheduledAgents(ctx context.Context, dueBefore time.Time) ([]*persona.Agent, error) {
rows, err := s.db.QueryContext(ctx,
`SELECT data FROM agents WHERE schedule != '' AND next_run_at > 0 AND next_run_at <= ? ORDER BY next_run_at`,
dueBefore.Unix())
if err != nil {
return nil, fmt.Errorf("personaStore.ListScheduledAgents: %w", err)
}
return scanAgents(rows)
}
func (s *personaStore) MarkAgentScheduledRun(ctx context.Context, agentID string, ranAt, nextAt time.Time) error {
// Single atomic statement, not Get→mutate→Save: closes the lost-update
// window a concurrent Mark/edit would otherwise open. json_set keeps the
// blob's *time.Time fields consistent with the next_run_at column (Go
// encodes time.Time as RFC3339Nano, so it round-trips through GetAgent).
res, err := s.db.ExecContext(ctx,
`UPDATE agents SET next_run_at=?, data=json_set(data,'$.NextRunAt',?,'$.LastScheduledRunAt',?) WHERE id=?`,
nextAt.Unix(), nextAt.Format(time.RFC3339Nano), ranAt.Format(time.RFC3339Nano), agentID)
if err != nil {
return fmt.Errorf("personaStore.MarkAgentScheduledRun: %w", err)
}
if n, _ := res.RowsAffected(); n == 0 {
return persona.ErrNotFound
}
return nil
}
+106
View File
@@ -0,0 +1,106 @@
package store
import (
"context"
"testing"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/persona"
)
func TestSQLitePersonaStore(t *testing.T) {
ctx := context.Background()
db, err := Open(":memory:")
if err != nil {
t.Fatal(err)
}
defer db.Close()
st := db.Personas()
if err := st.InitializeAgentStorage(ctx); err != nil {
t.Fatal(err)
}
// Full struct round-trips through the JSON blob (incl. nested + map fields).
a := &persona.Agent{
ID: "a1", Name: "helper", OwnerID: "o1", SystemPrompt: "be nice",
ModelTier: "fast", SkillPalette: []string{"animate"},
StateReactEmoji: map[string]string{"running": "⏳"},
ChatbotChannelFilter: "general",
}
if err := st.SaveAgent(ctx, a); err != nil {
t.Fatal(err)
}
got, err := st.GetAgent(ctx, "a1")
if err != nil || got.SystemPrompt != "be nice" || len(got.SkillPalette) != 1 ||
got.StateReactEmoji["running"] != "⏳" {
t.Fatalf("round-trip lost fields: %+v (err %v)", got, err)
}
if byName, err := st.GetAgentByName(ctx, "o1", "helper"); err != nil || byName.ID != "a1" {
t.Fatalf("GetAgentByName: %v %+v", err, byName)
}
if cf, _ := st.ListAgentsByChatbotChannelFilter(ctx); len(cf) != 1 {
t.Errorf("ListAgentsByChatbotChannelFilter = %d, want 1", len(cf))
}
// Scheduling: due query + MarkAgentScheduledRun round-trip.
now := time.Now().UTC()
sched := &persona.Agent{ID: "s1", Name: "cron", OwnerID: "o1", Schedule: "0 * * * *"}
due := now.Add(-time.Minute)
sched.NextRunAt = &due
if err := st.SaveAgent(ctx, sched); err != nil {
t.Fatal(err)
}
dueList, _ := st.ListScheduledAgents(ctx, now)
if len(dueList) != 1 || dueList[0].ID != "s1" {
t.Fatalf("ListScheduledAgents = %+v", dueList)
}
next := now.Add(time.Hour)
if err := st.MarkAgentScheduledRun(ctx, "s1", now, next); err != nil {
t.Fatal(err)
}
if again, _ := st.ListScheduledAgents(ctx, now); len(again) != 0 {
t.Errorf("after MarkAgentScheduledRun, nothing should be due before now: %+v", again)
}
if err := st.DeleteAgent(ctx, "a1"); err != nil {
t.Fatal(err)
}
if _, err := st.GetAgent(ctx, "a1"); err != persona.ErrNotFound {
t.Errorf("GetAgent after delete = %v, want ErrNotFound", err)
}
}
// TestMarkAgentScheduledRunBlobRoundTrips guards the json_set atomic update:
// the JSON blob must stay parseable and reflect the new scheduled times.
func TestMarkAgentScheduledRunBlobRoundTrips(t *testing.T) {
ctx := context.Background()
db, _ := Open(":memory:")
defer db.Close()
st := db.Personas()
st.InitializeAgentStorage(ctx)
start := time.Now().UTC()
a := &persona.Agent{ID: "m1", Name: "n", OwnerID: "o", Schedule: "0 * * * *"}
a.NextRunAt = &start
if err := st.SaveAgent(ctx, a); err != nil {
t.Fatal(err)
}
ran := start
next := start.Add(time.Hour)
if err := st.MarkAgentScheduledRun(ctx, "m1", ran, next); err != nil {
t.Fatal(err)
}
got, err := st.GetAgent(ctx, "m1") // blob must still unmarshal
if err != nil {
t.Fatalf("GetAgent after json_set Mark failed (blob corrupt?): %v", err)
}
if got.NextRunAt == nil || !got.NextRunAt.Equal(next) {
t.Errorf("blob NextRunAt = %v, want %v", got.NextRunAt, next)
}
if got.LastScheduledRunAt == nil || !got.LastScheduledRunAt.Equal(ran) {
t.Errorf("blob LastScheduledRunAt = %v, want %v", got.LastScheduledRunAt, ran)
}
// Unknown id -> ErrNotFound.
if err := st.MarkAgentScheduledRun(ctx, "nope", ran, next); err != persona.ErrNotFound {
t.Errorf("Mark(unknown) = %v, want ErrNotFound", err)
}
}
+280
View File
@@ -0,0 +1,280 @@
package store
import (
"context"
"database/sql"
"encoding/json"
"errors"
"fmt"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/skill"
)
// skillStore is the SQLite-backed skill.SkillStore. Same JSON-blob + indexed
// columns approach as personaStore: the full Skill round-trips, lookups stay
// indexed. Versions live in their own table (each SkillVersion embeds a full
// Skill snapshot, stored as a JSON blob).
type skillStore struct{ db *sql.DB }
// Skills returns a durable skill.SkillStore backed by this database.
func (d *DB) Skills() skill.SkillStore { return &skillStore{db: d.sql} }
var _ skill.SkillStore = (*skillStore)(nil)
func (s *skillStore) Initialize(ctx context.Context) error {
_, err := s.db.ExecContext(ctx, `
CREATE TABLE IF NOT EXISTS skills (
id TEXT PRIMARY KEY,
owner_id TEXT NOT NULL DEFAULT '',
name TEXT NOT NULL DEFAULT '',
source TEXT NOT NULL DEFAULT '',
visibility TEXT NOT NULL DEFAULT '',
chatbot INTEGER NOT NULL DEFAULT 0, -- ExposeAsChatbotTool
schedule TEXT NOT NULL DEFAULT '',
next_run_at INTEGER NOT NULL DEFAULT 0,
data TEXT NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_skills_owner ON skills(owner_id);
CREATE INDEX IF NOT EXISTS idx_skills_vis ON skills(visibility);
CREATE INDEX IF NOT EXISTS idx_skills_sched ON skills(schedule, next_run_at);
CREATE TABLE IF NOT EXISTS skill_versions (
id TEXT PRIMARY KEY,
skill_id TEXT NOT NULL,
version TEXT NOT NULL DEFAULT '',
seq INTEGER NOT NULL, -- append order, for newest-first
data TEXT NOT NULL
);
CREATE UNIQUE INDEX IF NOT EXISTS idx_skill_versions_skill ON skill_versions(skill_id, seq);`)
if err != nil {
return fmt.Errorf("skillStore.Initialize: %w", err)
}
return nil
}
func (s *skillStore) Save(ctx context.Context, sk *skill.Skill) error {
blob, err := json.Marshal(sk)
if err != nil {
return fmt.Errorf("skillStore.Save: marshal: %w", err)
}
var next int64
if !sk.NextRunAt.IsZero() {
next = sk.NextRunAt.Unix()
}
chatbot := 0
if sk.ExposeAsChatbotTool {
chatbot = 1
}
_, err = s.db.ExecContext(ctx, `
INSERT INTO skills (id, owner_id, name, source, visibility, chatbot, schedule, next_run_at, data)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(id) DO UPDATE SET
owner_id=excluded.owner_id, name=excluded.name, source=excluded.source,
visibility=excluded.visibility, chatbot=excluded.chatbot, schedule=excluded.schedule,
next_run_at=excluded.next_run_at, data=excluded.data`,
sk.ID, sk.OwnerID, sk.Name, string(sk.Source), string(sk.Visibility), chatbot,
sk.Schedule, next, string(blob))
if err != nil {
return fmt.Errorf("skillStore.Save: %w", err)
}
return nil
}
func scanSkills(rows *sql.Rows) ([]skill.Skill, error) {
defer rows.Close()
var out []skill.Skill
for rows.Next() {
var blob string
if err := rows.Scan(&blob); err != nil {
return nil, err
}
var sk skill.Skill
if err := json.Unmarshal([]byte(blob), &sk); err != nil {
return nil, err
}
out = append(out, sk)
}
return out, rows.Err()
}
func (s *skillStore) getOne(ctx context.Context, where string, arg ...any) (*skill.Skill, error) {
var blob string
err := s.db.QueryRowContext(ctx, `SELECT data FROM skills WHERE `+where, arg...).Scan(&blob)
switch {
case errors.Is(err, sql.ErrNoRows):
return nil, skill.ErrNotFound
case err != nil:
return nil, err
}
var sk skill.Skill
if err := json.Unmarshal([]byte(blob), &sk); err != nil {
return nil, err
}
return &sk, nil
}
func (s *skillStore) Get(ctx context.Context, id string) (*skill.Skill, error) {
return s.getOne(ctx, "id = ?", id)
}
func (s *skillStore) GetByName(ctx context.Context, ownerID, name string) (*skill.Skill, error) {
return s.getOne(ctx, "owner_id = ? AND name = ?", ownerID, name)
}
func (s *skillStore) ListBuiltinByName(ctx context.Context, name string) (*skill.Skill, error) {
return s.getOne(ctx, "source = ? AND name = ?", string(skill.SourceBuiltin), name)
}
func (s *skillStore) Delete(ctx context.Context, id string) error {
if _, err := s.db.ExecContext(ctx, `DELETE FROM skills WHERE id = ?`, id); err != nil {
return fmt.Errorf("skillStore.Delete: %w", err)
}
return nil
}
func (s *skillStore) query(ctx context.Context, where string, arg ...any) ([]skill.Skill, error) {
rows, err := s.db.QueryContext(ctx, `SELECT data FROM skills WHERE `+where+` ORDER BY name`, arg...)
if err != nil {
return nil, err
}
return scanSkills(rows)
}
func (s *skillStore) ListByOwner(ctx context.Context, ownerID string) ([]skill.Skill, error) {
return s.query(ctx, "owner_id = ?", ownerID)
}
func (s *skillStore) ListPublic(ctx context.Context) ([]skill.Skill, error) {
return s.query(ctx, "visibility = ?", string(skill.VisibilityPublic))
}
func (s *skillStore) ListChatbotExposed(ctx context.Context) ([]skill.Skill, error) {
return s.query(ctx, "chatbot = 1")
}
// ListSharedWith loads visibility=shared rows and filters SharedWith in Go (the
// shared set per skill is small; avoids a JSON-array query).
func (s *skillStore) ListSharedWith(ctx context.Context, memberID string) ([]skill.Skill, error) {
shared, err := s.query(ctx, "visibility = ?", string(skill.VisibilityShared))
if err != nil {
return nil, err
}
out := shared[:0]
for _, sk := range shared {
for _, id := range sk.SharedWith {
if id == memberID {
out = append(out, sk)
break
}
}
}
return out, nil
}
func (s *skillStore) ListDueScheduled(ctx context.Context, now time.Time) ([]skill.Skill, error) {
rows, err := s.db.QueryContext(ctx,
`SELECT data FROM skills WHERE schedule != '' AND next_run_at > 0 AND next_run_at <= ? ORDER BY next_run_at`,
now.Unix())
if err != nil {
return nil, fmt.Errorf("skillStore.ListDueScheduled: %w", err)
}
return scanSkills(rows)
}
func (s *skillStore) MarkScheduledRun(ctx context.Context, skillID string, ranAt, nextAt time.Time) error {
// Single atomic statement instead of Get→mutate→Save: a concurrent Mark or
// admin edit can't lose this update (no read-modify-write window). json_set
// keeps the JSON blob's NextRunAt/LastScheduledRunAt consistent with the
// indexed next_run_at column; RFC3339Nano matches Go's time JSON encoding so
// the blob still round-trips through Get.
var next int64
if !nextAt.IsZero() {
next = nextAt.Unix()
}
res, err := s.db.ExecContext(ctx,
`UPDATE skills SET next_run_at=?, data=json_set(data,'$.NextRunAt',?,'$.LastScheduledRunAt',?) WHERE id=?`,
next, nextAt.Format(time.RFC3339Nano), ranAt.Format(time.RFC3339Nano), skillID)
if err != nil {
return fmt.Errorf("skillStore.MarkScheduledRun: %w", err)
}
if n, _ := res.RowsAffected(); n == 0 {
return skill.ErrNotFound
}
return nil
}
func (s *skillStore) AppendVersion(ctx context.Context, sv skill.SkillVersion) error {
if sv.SkillID == "" {
return fmt.Errorf("skillStore.AppendVersion: skill_id is required")
}
blob, err := json.Marshal(sv)
if err != nil {
return fmt.Errorf("skillStore.AppendVersion: marshal: %w", err)
}
// seq = current max+1 for this skill (newest-first ordering key). The
// MAX-then-INSERT runs in ONE transaction and the (skill_id, seq) index is
// UNIQUE, so two concurrent appends can't both land the same seq: the loser
// fails loudly on commit instead of silently corrupting the ordering. The
// Scan error is propagated (was swallowed, leaving seq=0 on failure).
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return fmt.Errorf("skillStore.AppendVersion: begin: %w", err)
}
defer tx.Rollback() //nolint:errcheck // no-op after Commit
var seq int64
if err := tx.QueryRowContext(ctx, `SELECT COALESCE(MAX(seq),0)+1 FROM skill_versions WHERE skill_id = ?`, sv.SkillID).Scan(&seq); err != nil {
return fmt.Errorf("skillStore.AppendVersion: seq: %w", err)
}
if _, err := tx.ExecContext(ctx,
`INSERT INTO skill_versions (id, skill_id, version, seq, data) VALUES (?, ?, ?, ?, ?)`,
sv.ID, sv.SkillID, sv.Version, seq, string(blob)); err != nil {
return fmt.Errorf("skillStore.AppendVersion: insert: %w", err)
}
if err := tx.Commit(); err != nil {
return fmt.Errorf("skillStore.AppendVersion: commit: %w", err)
}
return nil
}
func (s *skillStore) ListVersionsBySkill(ctx context.Context, skillID string, limit int) ([]skill.SkillVersion, error) {
q := `SELECT data FROM skill_versions WHERE skill_id = ? ORDER BY seq DESC`
args := []any{skillID}
if limit > 0 {
q += ` LIMIT ?`
args = append(args, limit)
}
rows, err := s.db.QueryContext(ctx, q, args...)
if err != nil {
return nil, fmt.Errorf("skillStore.ListVersionsBySkill: %w", err)
}
defer rows.Close()
var out []skill.SkillVersion
for rows.Next() {
var blob string
if err := rows.Scan(&blob); err != nil {
return nil, err
}
var sv skill.SkillVersion
if err := json.Unmarshal([]byte(blob), &sv); err != nil {
return nil, err
}
out = append(out, sv)
}
return out, rows.Err()
}
func (s *skillStore) GetVersionByID(ctx context.Context, versionID string) (*skill.SkillVersion, error) {
var blob string
err := s.db.QueryRowContext(ctx, `SELECT data FROM skill_versions WHERE id = ?`, versionID).Scan(&blob)
switch {
case errors.Is(err, sql.ErrNoRows):
return nil, skill.ErrNotFound
case err != nil:
return nil, err
}
var sv skill.SkillVersion
if err := json.Unmarshal([]byte(blob), &sv); err != nil {
return nil, err
}
return &sv, nil
}
+72
View File
@@ -0,0 +1,72 @@
package store
import (
"context"
"testing"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/skill"
)
func TestSQLiteSkillStore(t *testing.T) {
ctx := context.Background()
db, err := Open(":memory:")
if err != nil {
t.Fatal(err)
}
defer db.Close()
st := db.Skills()
if err := st.Initialize(ctx); err != nil {
t.Fatal(err)
}
pub := &skill.Skill{ID: "a", Name: "pub", OwnerID: "o1", Visibility: skill.VisibilityPublic,
Tools: []string{"summarize"}, ExposeAsChatbotTool: true}
shared := &skill.Skill{ID: "b", Name: "shr", OwnerID: "o1", Visibility: skill.VisibilityShared, SharedWith: []string{"bob"}}
if err := st.Save(ctx, pub); err != nil {
t.Fatal(err)
}
if err := st.Save(ctx, shared); err != nil {
t.Fatal(err)
}
got, err := st.Get(ctx, "a")
if err != nil || len(got.Tools) != 1 || !got.ExposeAsChatbotTool {
t.Fatalf("round-trip: %v %+v", err, got)
}
if ps, _ := st.ListPublic(ctx); len(ps) != 1 || ps[0].ID != "a" {
t.Errorf("ListPublic = %+v", ps)
}
if ss, _ := st.ListSharedWith(ctx, "bob"); len(ss) != 1 || ss[0].ID != "b" {
t.Errorf("ListSharedWith(bob) = %+v", ss)
}
if ss, _ := st.ListSharedWith(ctx, "carol"); len(ss) != 0 {
t.Errorf("ListSharedWith(carol) should be empty: %+v", ss)
}
if ce, _ := st.ListChatbotExposed(ctx); len(ce) != 1 {
t.Errorf("ListChatbotExposed = %d, want 1", len(ce))
}
// Versions newest-first + by id.
st.AppendVersion(ctx, skill.SkillVersion{ID: "v1", SkillID: "a", Version: "1.0.0"})
st.AppendVersion(ctx, skill.SkillVersion{ID: "v2", SkillID: "a", Version: "1.1.0"})
vs, _ := st.ListVersionsBySkill(ctx, "a", 10)
if len(vs) != 2 || vs[0].ID != "v2" {
t.Errorf("versions newest-first: %+v", vs)
}
if gv, err := st.GetVersionByID(ctx, "v1"); err != nil || gv.Version != "1.0.0" {
t.Errorf("GetVersionByID: %v %+v", err, gv)
}
// Scheduling.
now := time.Now().UTC()
cron := &skill.Skill{ID: "c", Name: "cron", OwnerID: "o1", Schedule: "0 * * * *", NextRunAt: now.Add(-time.Minute)}
st.Save(ctx, cron)
if due, _ := st.ListDueScheduled(ctx, now); len(due) != 1 || due[0].ID != "c" {
t.Fatalf("ListDueScheduled = %+v", due)
}
st.MarkScheduledRun(ctx, "c", now, now.Add(time.Hour))
if due, _ := st.ListDueScheduled(ctx, now); len(due) != 0 {
t.Errorf("after MarkScheduledRun nothing due: %+v", due)
}
}
+54
View File
@@ -0,0 +1,54 @@
// Package store provides durable, pure-Go SQLite implementations of executus's
// battery store seams (audit, budget, persona, skill). It is a SEPARATE nested
// module so the SQLite driver (modernc.org/sqlite — pure Go, no cgo) never
// enters the executus core go.sum: a static-binary host (gadfly) that imports
// only the core stays static, while a host that wants turnkey persistence
// imports this module and gets every *Store seam backed by one SQLite file.
//
// db, _ := store.Open("file:executus.db?_pragma=busy_timeout(5000)")
// defer db.Close()
// budgetStore := db.Budget() // satisfies budget.BudgetStorage
package store
import (
"database/sql"
"fmt"
_ "modernc.org/sqlite" // pure-Go driver, registered as "sqlite"
)
// DB is a handle to one SQLite database backing the executus store seams. Each
// accessor (Budget(), …) returns a seam implementation sharing this connection.
// Safe for concurrent use (SQLite serializes writes; busy_timeout handles
// contention). Construct with Open; close with Close.
type DB struct {
sql *sql.DB
}
// Open opens (creating if absent) a SQLite database at dsn and returns a DB. A
// dsn of ":memory:" yields an ephemeral in-memory database. The caller owns the
// returned DB and must Close it.
func Open(dsn string) (*DB, error) {
sqldb, err := sql.Open("sqlite", dsn)
if err != nil {
return nil, fmt.Errorf("store: open %q: %w", dsn, err)
}
// A contended writer should WAIT for the lock, not fail immediately — set a
// busy_timeout so concurrent stores don't see spurious SQLITE_BUSY. (The
// doc example advertised this; it's now actually applied for every DSN.)
if _, err := sqldb.Exec("PRAGMA busy_timeout=5000"); err != nil {
sqldb.Close()
return nil, fmt.Errorf("store: set busy_timeout %q: %w", dsn, err)
}
if err := sqldb.Ping(); err != nil {
sqldb.Close()
return nil, fmt.Errorf("store: ping %q: %w", dsn, err)
}
return &DB{sql: sqldb}, nil
}
// Close closes the underlying database.
func (d *DB) Close() error { return d.sql.Close() }
// SQL exposes the underlying *sql.DB for hosts that need direct access.
func (d *DB) SQL() *sql.DB { return d.sql }
+302
View File
@@ -0,0 +1,302 @@
// Package critic is the run-watchdog battery: a two-tier timeout monitor that
// catches a run that has stopped making progress. It plugs into
// run.Ports.Critic.
//
// The split of concerns is deliberate. executus owns the deterministic
// MECHANICS — track activity, fire on a soft timeout, enforce a hard-kill
// backstop, carry steer messages and the extendable deadline back to the
// executor. The POLICY — what to actually do when a run stalls (nudge it,
// extend its deadline, kill it, escalate to a human) — is the Escalator seam.
// Mort plugs its LLM critic-agent in as an Escalator; ExtendOnce is the
// zero-dependency default.
//
// The executor wires run.Ports.Critic (C0b): it feeds the handle activity,
// binds the run context to its extendable Deadline, drains its Steer, and polls
// MaxSteps each step so an Escalator can also raise a long run's step ceiling
// (Decision.RaiseStepsBy).
package critic
import (
"context"
"errors"
"log/slog"
"math"
"sync"
"time"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/executus/run"
)
// Progress is the snapshot the critic hands an Escalator when a run stalls.
type Progress struct {
Iterations int // completed agent-loop iterations so far
LastActivity time.Time // wall-clock of the last step/tool event
Idle time.Duration // now - LastActivity
LastTool string // name of the most recently started tool ("" if none)
}
// Decision is the Escalator's verdict for a stalled run. Zero value = do
// nothing (let the hard backstop eventually kill a truly hung run).
type Decision struct {
Nudge []llm.Message // injected before the agent's next turn (a steer)
ExtendBy time.Duration // push the hard deadline out by this much
RaiseStepsBy int // raise the run's tool-dispatch step ceiling by this
Kill bool // cancel the run now
KillReason string
}
// Escalator decides what to do when a run crosses its soft timeout. It is
// called at most once per idle period (a fresh step/tool event re-arms it).
type Escalator interface {
OnSoftTimeout(ctx context.Context, info run.RunInfo, p Progress) Decision
}
// ExtendOnce is the default Escalator: the first time a given run stalls it
// extends that run's deadline by By (giving a slow-but-healthy run room), then
// takes no further action for it — so a genuinely hung run is later killed by
// the hard backstop. A nil/zero By falls back to one soft-timeout's worth.
//
// The one-shot is keyed PER RUN (by RunInfo.RunID): a single System shares one
// ExtendOnce across every run it monitors, so a global flag would let only the
// first run to stall ever get its extension. The fired set grows with the
// number of distinct runs that stall — fine for a process's run volume; a host
// running unboundedly long can construct a fresh System periodically.
type ExtendOnce struct {
By time.Duration
mu sync.Mutex
fired map[string]bool // run ids that have already had their one extension
}
// OnSoftTimeout implements Escalator.
func (e *ExtendOnce) OnSoftTimeout(_ context.Context, info run.RunInfo, p Progress) Decision {
e.mu.Lock()
defer e.mu.Unlock()
if e.fired[info.RunID] {
return Decision{}
}
if e.fired == nil {
e.fired = map[string]bool{}
}
e.fired[info.RunID] = true
by := e.By
if by <= 0 {
by = p.Idle // ~one soft timeout
}
return Decision{ExtendBy: by}
}
// System implements run.Critic. Construct with New; one System monitors many
// runs concurrently (each Monitor returns an independent handle).
type System struct {
esc Escalator
backstopMul float64 // hard deadline = softTimeout * backstopMul from start
checkInterval time.Duration
now func() time.Time
logger *slog.Logger
}
func (s *System) log() *slog.Logger {
if s.logger != nil {
return s.logger
}
return slog.Default()
}
// New builds a run.Critic. esc is the policy (nil → ExtendOnce). backstopMul is
// the hard-kill backstop as a multiple of each run's soft timeout (<=1 → 3). A
// nil esc + the default backstop gives a safe "extend once, then hard-kill"
// watchdog with no host wiring.
func New(esc Escalator, backstopMul float64) *System {
if esc == nil {
esc = &ExtendOnce{}
}
if backstopMul <= 1 {
backstopMul = 3
}
return &System{esc: esc, backstopMul: backstopMul, now: time.Now}
}
var _ run.Critic = (*System)(nil)
// Monitor starts watching a run and returns its handle. Implements run.Critic.
func (s *System) Monitor(ctx context.Context, info run.RunInfo, softTimeout time.Duration) run.CriticHandle {
if softTimeout <= 0 {
return run.CriticHandle(nil) // no soft timeout → not monitored
}
now := s.now()
check := s.checkInterval
if check <= 0 {
check = softTimeout / 2
if check < time.Second {
check = time.Second
}
}
h := &handle{
sys: s,
info: info,
softTimeout: softTimeout,
now: s.now,
lastActivity: now,
deadline: now.Add(time.Duration(float64(softTimeout) * s.backstopMul)),
maxSteps: info.MaxIterations, // base ceiling; an Escalator may RaiseStepsBy
stopCh: make(chan struct{}),
}
go h.watch(ctx, check)
return h
}
// handle is one run's live critic link. Implements run.CriticHandle.
type handle struct {
sys *System
info run.RunInfo
softTimeout time.Duration
now func() time.Time
mu sync.Mutex
lastActivity time.Time
escalatedAt time.Time // lastActivity value we last escalated for (de-dupes per idle period)
deadline time.Time
steer []llm.Message
iterations int
maxSteps int // current tool-dispatch ceiling (base MaxIterations, raised by RaiseStepsBy)
lastTool string
killed bool // sticky: once an Escalator kills, no later decision un-kills it
killCause error // non-nil once killed; surfaced via KillCause for "killed" status
stopped bool
stopCh chan struct{}
}
func (h *handle) RecordStep(iter int, _ *llm.Response) {
// This battery's Progress tracks iteration count + activity, not per-step
// payload, so the response is unused here; a richer Escalator could record it.
h.mu.Lock()
h.iterations = iter
h.lastActivity = h.now()
h.mu.Unlock()
}
func (h *handle) RecordToolStart(name, _ string) {
h.mu.Lock()
h.lastTool = name
h.lastActivity = h.now()
h.mu.Unlock()
}
func (h *handle) Steer() []llm.Message {
h.mu.Lock()
defer h.mu.Unlock()
if len(h.steer) == 0 {
return nil
}
out := h.steer
h.steer = nil
return out
}
func (h *handle) Deadline() time.Time {
h.mu.Lock()
defer h.mu.Unlock()
return h.deadline
}
func (h *handle) MaxSteps() int {
h.mu.Lock()
defer h.mu.Unlock()
return h.maxSteps
}
func (h *handle) KillCause() error {
h.mu.Lock()
defer h.mu.Unlock()
return h.killCause
}
func (h *handle) Stop() {
h.mu.Lock()
if !h.stopped {
h.stopped = true
close(h.stopCh)
}
h.mu.Unlock()
}
// watch fires the Escalator once per idle period the run crosses its soft
// timeout, and applies the returned Decision.
func (h *handle) watch(ctx context.Context, interval time.Duration) {
// A misbehaving Escalator that panics must not silently kill the watch
// goroutine (which would leave the run unmonitored for its lifetime). Log
// and exit cleanly — the run falls back to the deadline already set.
defer func() {
if r := recover(); r != nil {
h.sys.log().Error("critic watch panicked; run is now unmonitored", "run", h.info.RunID, "panic", r)
}
}()
t := time.NewTicker(interval)
defer t.Stop()
for {
select {
case <-h.stopCh:
return
case <-ctx.Done():
return
case <-t.C:
h.tick(ctx)
}
}
}
func (h *handle) tick(ctx context.Context) {
h.mu.Lock()
// Kill is sticky: once an Escalator has killed this run, no later tick (and
// no later Decision) un-collapses the deadline.
if h.killed {
h.mu.Unlock()
return
}
idle := h.now().Sub(h.lastActivity)
// Only escalate once per idle period: skip if we already escalated for this
// exact lastActivity (a fresh step/tool updates lastActivity and re-arms).
if idle < h.softTimeout || h.escalatedAt.Equal(h.lastActivity) {
h.mu.Unlock()
return
}
h.escalatedAt = h.lastActivity
snap := Progress{Iterations: h.iterations, LastActivity: h.lastActivity, Idle: idle, LastTool: h.lastTool}
h.mu.Unlock()
d := h.sys.esc.OnSoftTimeout(ctx, h.info, snap)
h.mu.Lock()
defer h.mu.Unlock()
if h.killed { // a concurrent tick may have killed while OnSoftTimeout ran
return
}
if d.Kill {
h.killed = true
reason := d.KillReason
if reason == "" {
reason = "critic killed the run"
}
h.killCause = errors.New(reason) // surfaced via KillCause → "killed" status
h.deadline = h.now() // immediate hard deadline → executor cancels
return // ignore any Nudge/ExtendBy paired with a Kill
}
if len(d.Nudge) > 0 {
h.steer = append(h.steer, d.Nudge...)
}
if d.ExtendBy > 0 {
h.deadline = h.deadline.Add(d.ExtendBy)
}
if d.RaiseStepsBy > 0 {
// Overflow-safe: a buggy Escalator returning a huge delta must not wrap
// maxSteps negative (which the executor would read as "defer to base").
if d.RaiseStepsBy > math.MaxInt-h.maxSteps {
h.maxSteps = math.MaxInt
} else {
h.maxSteps += d.RaiseStepsBy
}
}
}
+100
View File
@@ -0,0 +1,100 @@
package critic
import (
"context"
"sync"
"testing"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/run"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
)
// escFunc adapts a func to an Escalator.
type escFunc func(context.Context, run.RunInfo, Progress) Decision
func (f escFunc) OnSoftTimeout(ctx context.Context, i run.RunInfo, p Progress) Decision {
return f(ctx, i, p)
}
func TestMonitorEscalatesOncePerIdlePeriodAndExtends(t *testing.T) {
var mu sync.Mutex
var calls int
esc := escFunc(func(_ context.Context, _ run.RunInfo, p Progress) Decision {
mu.Lock()
calls++
mu.Unlock()
return Decision{ExtendBy: 50 * time.Millisecond, Nudge: []llm.Message{{Role: llm.RoleUser}}}
})
s := New(esc, 3)
s.checkInterval = 5 * time.Millisecond
h := s.Monitor(context.Background(), run.RunInfo{RunID: "r"}, 20*time.Millisecond)
defer h.Stop()
d0 := h.Deadline()
time.Sleep(60 * time.Millisecond) // cross the soft timeout with no activity
mu.Lock()
c := calls
mu.Unlock()
if c < 1 {
t.Fatalf("expected at least one escalation, got %d", c)
}
// Nudge was queued and is drained once.
if msgs := h.Steer(); len(msgs) == 0 {
t.Error("expected a queued steer nudge")
}
if msgs := h.Steer(); len(msgs) != 0 {
t.Error("steer should drain (be empty on second read)")
}
// Deadline was extended.
if !h.Deadline().After(d0) {
t.Error("deadline should have been extended past the original")
}
// A fresh step re-arms; another idle period escalates again.
h.RecordStep(1, nil)
time.Sleep(60 * time.Millisecond)
mu.Lock()
c2 := calls
mu.Unlock()
if c2 <= c {
t.Errorf("a re-armed idle period should escalate again (%d -> %d)", c, c2)
}
}
func TestKillCollapsesDeadline(t *testing.T) {
esc := escFunc(func(context.Context, run.RunInfo, Progress) Decision {
return Decision{Kill: true, KillReason: "hung"}
})
s := New(esc, 10) // big backstop so only Kill collapses it
s.checkInterval = 5 * time.Millisecond
h := s.Monitor(context.Background(), run.RunInfo{RunID: "r"}, 20*time.Millisecond)
defer h.Stop()
time.Sleep(60 * time.Millisecond)
if h.Deadline().After(time.Now().Add(time.Second)) {
t.Error("Kill should collapse the deadline to ~now")
}
}
func TestExtendOnceOnlyFiresOnce(t *testing.T) {
e := &ExtendOnce{By: time.Minute}
// Same run id: only the first call extends.
d1 := e.OnSoftTimeout(context.Background(), run.RunInfo{RunID: "r1"}, Progress{})
d2 := e.OnSoftTimeout(context.Background(), run.RunInfo{RunID: "r1"}, Progress{})
if d1.ExtendBy != time.Minute {
t.Errorf("first decision should extend, got %+v", d1)
}
if d2.ExtendBy != 0 || d2.Kill {
t.Errorf("second call for the same run should be a no-op, got %+v", d2)
}
// A DIFFERENT run still gets its own one extension (per-run, not global).
if d3 := e.OnSoftTimeout(context.Background(), run.RunInfo{RunID: "r2"}, Progress{}); d3.ExtendBy != time.Minute {
t.Errorf("a different run should get its own extension, got %+v", d3)
}
}
func TestZeroSoftTimeoutNotMonitored(t *testing.T) {
s := New(nil, 3)
if h := s.Monitor(context.Background(), run.RunInfo{}, 0); h != nil {
t.Error("zero soft timeout should return a nil handle (not monitored)")
}
}
+35 -13
View File
@@ -1,27 +1,49 @@
// Command minimal demonstrates executus's standalone core primitives available
// today (P0): the config seam + bounded fan-out. The full zero-config "agentic
// in ~12 lines" example arrives once the model, tool, and run packages land
// (P1P3).
// Command minimal is executus's "hello, agentic world": wire a model resolver,
// a tool registry, and the run executor, then run an agent. With no batteries
// (Audit/Budget/Critic/Checkpointer/Palette/Delivery all nil) this is a
// bounded, in-memory run — the light-host shape (gadfly's case).
//
// Run it with a provider key for the configured tier, e.g.
//
// ANTHROPIC_API_KEY=sk-... go run ./examples/minimal
//
// Override a tier from the environment without touching code, e.g.
//
// EXECUTUS_MODEL_TIER_FAST=openai/gpt-4o-mini ANTHROPIC_API_KEY= OPENAI_KEY=sk-... go run ./examples/minimal
package main
import (
"context"
"fmt"
"log"
"gitea.stevedudenhoeffer.com/steve/executus/config"
"gitea.stevedudenhoeffer.com/steve/executus/fanout"
"gitea.stevedudenhoeffer.com/steve/executus/model"
"gitea.stevedudenhoeffer.com/steve/executus/run"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
func main() {
cfg := config.Env("EXECUTUS_") // e.g. EXECUTUS_FANOUT_MAX_CONCURRENT=8
max := cfg.Int("fanout.max_concurrent", 4)
// 1. Configure model tiers: live values come from the environment
// (EXECUTUS_MODEL_TIER_<NAME>), falling back to these defaults.
model.Configure(config.Env("EXECUTUS_"), map[string]string{
"fast": "anthropic/claude-haiku-4-5",
"thinking": "anthropic/claude-opus-4-8",
}, 0)
items := []string{"alpha", "beta", "gamma", "delta"}
results := fanout.Run(context.Background(), items,
fanout.Options[string]{MaxConcurrent: max},
func(_ context.Context, s string) (int, error) { return len(s), nil })
// 2. Build the executor: a tool registry + the model resolver. No batteries.
ex := run.New(run.Config{
Registry: tool.NewRegistry(),
Models: model.ParseModelForContext,
})
for _, r := range results {
fmt.Printf("%-6s -> %d (err=%v)\n", items[r.Index], r.Value, r.Err)
// 3. Run an agent and print its answer.
res := ex.Run(context.Background(),
run.RunnableAgent{Name: "assistant", SystemPrompt: "You are concise.", ModelTier: "fast"},
tool.Invocation{RunID: "demo-1", CallerID: "local"},
"In one sentence, what is an agent harness?")
if res.Err != nil {
log.Fatalf("run failed: %v", res.Err)
}
fmt.Println(res.Output)
}
+38
View File
@@ -0,0 +1,38 @@
# examples/reviewer — the light-tier canary
A **gadfly-shaped adversarial PR reviewer built on the executus core only** — no
batteries, no database, no host adapters. It exists to prove that the core is
sufficient for a static-binary light host (gadfly's shape), and that such a host
keeps a `go.sum` free of `gorm`/`redis`/`discordgo`/`sqlite`.
What it exercises, all from core:
| Concern | executus core piece |
|---|---|
| Env-driven model fleet + tier overrides | `config.Env` + `model.Configure` |
| Tier resolution + failover | `model.ParseModelForContext` |
| N models × M lenses swarm | `fanout.Run` (with `PerKey` per-provider caps) |
| Structured findings per cell | `model.GenerateWith[T]` |
| One report section per model, worst-verdict-led | `Consolidate` (local) |
## Run
```sh
REVIEWER_MODELS=fast,thinking \
ANTHROPIC_API_KEY=sk-... \
go run ./examples/reviewer -diff "$(git diff HEAD~1)"
```
Config (all optional, `REVIEWER_`-prefixed env):
- `REVIEWER_MODELS` — csv of tier names / `provider/model` specs (default `fast`)
- `REVIEWER_MODEL_TIER_<NAME>` — override a tier's resolved spec
- `REVIEWER_MAX_CONCURRENT` — total in-flight swarm cells (default 6)
- `REVIEWER_PROVIDER_CONCURRENCY` — per-provider cap (default 3)
## Test
`reviewer_test.go` runs the whole swarm against majordomo's fake provider
(hermetic, no network) and asserts the consolidated verdicts. A `go list -deps`
check in CI confirms the package pulls in no battery and no DB driver — the
light-tier invariant.
+110
View File
@@ -0,0 +1,110 @@
package main
import (
"context"
"flag"
"fmt"
"io"
"os"
"strings"
"gitea.stevedudenhoeffer.com/steve/executus/config"
"gitea.stevedudenhoeffer.com/steve/executus/fanout"
"gitea.stevedudenhoeffer.com/steve/executus/model"
)
// DefaultLenses is the canary's review suite (mirrors gadfly's default).
var DefaultLenses = []Lens{
{Name: "security", Focus: "auth, injection, secret leakage, unsafe deserialization, SSRF."},
{Name: "correctness", Focus: "logic errors, broken invariants, off-by-one, contract violations."},
{Name: "error-handling", Focus: "swallowed errors, missing timeouts, races, unhandled edge cases."},
}
// Reviewer is configured entirely from the environment (the GADFLY_*-style light
// host): REVIEWER_MODELS (csv of tier/spec), REVIEWER_MODEL_TIER_<NAME> overrides,
// REVIEWER_MAX_CONCURRENT, REVIEWER_PROVIDER_CONCURRENCY. The diff is read from
// -diff or stdin.
//
// REVIEWER_MODELS=fast,thinking ANTHROPIC_API_KEY=... go run ./examples/reviewer < my.diff
func main() {
cfg := config.Env("REVIEWER_")
// Tier table from env, with code defaults.
model.Configure(cfg, map[string]string{
"fast": "anthropic/claude-haiku-4-5",
"thinking": "anthropic/claude-opus-4-8",
}, 0)
fleet := splitCSV(cfg.String("models", "fast"))
maxConc := cfg.Int("max_concurrent", 6)
perProvider := cfg.Int("provider_concurrency", 3)
diffFlag := flag.String("diff", "", "diff text to review; reads stdin when empty")
flag.Parse()
diff := *diffFlag
if strings.TrimSpace(diff) == "" {
// Guard against blocking forever on an interactive TTY (no piped input).
if fi, _ := os.Stdin.Stat(); fi != nil && fi.Mode()&os.ModeCharDevice != 0 {
fmt.Fprintln(os.Stderr, "reviewer: no diff (pass -diff or pipe one on stdin)")
os.Exit(2)
}
b, err := io.ReadAll(os.Stdin)
if err != nil {
fmt.Fprintf(os.Stderr, "reviewer: reading stdin: %v\n", err)
os.Exit(2)
}
diff = string(b)
}
if strings.TrimSpace(diff) == "" {
fmt.Fprintln(os.Stderr, "reviewer: no diff (pass -diff or pipe one on stdin)")
os.Exit(2)
}
ctx := context.Background()
var models []NamedModel
for _, spec := range fleet {
_, m, err := model.ParseModelForContext(ctx, spec)
if err != nil {
fmt.Fprintf(os.Stderr, "reviewer: resolve model %q: %v\n", spec, err)
os.Exit(1)
}
models = append(models, NamedModel{Name: spec, Provider: providerOf(spec), Model: m})
}
results := Review(ctx, models, DefaultLenses, diff, fanout.Options[cell]{
MaxConcurrent: maxConc,
PerKey: perKeyCaps(models, perProvider),
})
fmt.Print(Consolidate(results))
}
func splitCSV(s string) []string {
var out []string
for _, p := range strings.Split(s, ",") {
if p = strings.TrimSpace(p); p != "" {
out = append(out, p)
}
}
return out
}
// providerOf returns a model spec's provider (the first path segment, e.g.
// "anthropic/claude-…" → "anthropic"; a bare tier name → itself).
func providerOf(spec string) string {
if i := strings.IndexByte(spec, '/'); i > 0 {
return spec[:i]
}
return spec // bare tier name → its own bucket (don't collapse distinct tiers)
}
// perKeyCaps builds the PerKey map: each distinct provider capped at perProvider.
func perKeyCaps(models []NamedModel, perProvider int) map[string]int {
if perProvider <= 0 {
return nil
}
caps := map[string]int{}
for _, m := range models {
caps[m.Provider] = perProvider
}
return caps
}
+204
View File
@@ -0,0 +1,204 @@
// Command reviewer is executus's light-tier CANARY: a gadfly-shaped adversarial
// PR reviewer built on the executus CORE ONLY — no batteries, no DB, no host.
// It proves the core is sufficient for a static-binary host like gadfly:
//
// - config.Env → env-driven model fleet + concurrency (GADFLY_*-style)
// - model.Configure/... → tier resolution + failover over majordomo
// - fanout.Run → the N-models × M-lenses swarm, with per-provider caps
// - model.GenerateWith[T] → structured findings per (model, lens)
// - consolidation → one report section per model, worst-verdict-led
//
// The whole thing imports only executus core packages, so a binary built from it
// keeps a go.sum free of gorm/redis/discordgo/sqlite — the light-tier invariant.
//
// See reviewer_test.go for the hermetic swarm test (majordomo's fake provider).
package main
import (
"context"
"fmt"
"sort"
"strings"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/executus/fanout"
"gitea.stevedudenhoeffer.com/steve/executus/model"
)
// Severity orders findings; the rank drives a model's worst-verdict header.
type Severity string
const (
SevTrivial Severity = "trivial"
SevSmall Severity = "small"
SevMedium Severity = "medium"
SevHigh Severity = "high"
SevCritical Severity = "critical"
)
func severityRank(s Severity) int {
switch s {
case SevCritical:
return 4
case SevHigh:
return 3
case SevMedium:
return 2
case SevSmall:
return 1
default:
return 0
}
}
// Finding is one issue a lens reports. It is the structured-output schema the
// model must satisfy (majordomo derives the JSON schema from this struct).
type Finding struct {
Severity Severity `json:"severity" jsonschema:"enum=trivial,enum=small,enum=medium,enum=high,enum=critical"`
Title string `json:"title"`
Detail string `json:"detail"`
}
// lensReport is the per-(model,lens) structured response.
type lensReport struct {
Findings []Finding `json:"findings"`
}
// Lens is one review dimension (security / correctness / …).
type Lens struct {
Name string
Focus string // appended to the base system prompt
}
// NamedModel is a resolved model plus the label + provider used for fan-out
// keying (per-provider concurrency) and reporting.
type NamedModel struct {
Name string // display label (the tier/spec the host configured)
Provider string // fan-out key for PerKey concurrency (e.g. "ollama-cloud")
Model llm.Model
}
// LensResult is one swarm cell's outcome.
type LensResult struct {
Model string
Lens string
Findings []Finding
Err error
}
const baseSystemPrompt = "You are an adversarial code reviewer. Review the diff for real, verifiable problems only — no style nits. Return ONLY JSON matching the schema. Report nothing if you find nothing."
// Review runs every (model × lens) cell of the swarm concurrently, bounded by
// opts (total + per-provider caps), and returns one LensResult per cell. A cell
// whose model call fails carries the error in LensResult.Err — one bad cell
// never aborts the swarm (the closure embeds per-cell errors in LensResult.Err).
func Review(ctx context.Context, models []NamedModel, lenses []Lens, diff string, opts fanout.Options[cell]) []LensResult {
cells := make([]cell, 0, len(models)*len(lenses))
for _, m := range models {
for _, l := range lenses {
cells = append(cells, cell{model: m, lens: l})
}
}
// Key each cell by its provider so PerKey throttles per backend (the
// GADFLY_PROVIDER_CONCURRENCY analogue).
if opts.Key == nil {
opts.Key = func(c cell) string { return c.model.Provider }
}
results := fanout.Run(ctx, cells, opts, func(ctx context.Context, c cell) (LensResult, error) {
sys := baseSystemPrompt
if c.lens.Focus != "" {
sys += "\n\nLens — " + c.lens.Name + ": " + c.lens.Focus
}
msgs := []llm.Message{{Role: llm.RoleUser, Parts: []llm.Part{llm.Text("Diff under review:\n" + diff)}}}
rep, err := model.GenerateWith[lensReport](ctx, c.model.Model, sys, msgs)
lr := LensResult{Model: c.model.Name, Lens: c.lens.Name, Findings: rep.Findings, Err: err}
// Return the value either way (err embedded) so every cell reports.
return lr, nil
})
out := make([]LensResult, 0, len(results))
for _, r := range results {
if r.Err != nil { // a swarm-level error (ctx cancel) with no value
out = append(out, LensResult{Err: r.Err})
continue
}
out = append(out, r.Value)
}
return out
}
// cell is one (model, lens) swarm task.
type cell struct {
model NamedModel
lens Lens
}
// Consolidate renders the swarm's results into one report: a section per model,
// each led by that model's worst finding severity, mirroring gadfly's
// one-comment-per-model output.
func Consolidate(results []LensResult) string {
byModel := map[string][]LensResult{}
var order []string
aborted := 0 // cells dropped before running (swarm cancelled) — no model attribution
for _, r := range results {
if r.Model == "" {
if r.Err != nil {
aborted++
}
continue
}
if _, ok := byModel[r.Model]; !ok {
order = append(order, r.Model)
}
byModel[r.Model] = append(byModel[r.Model], r)
}
sort.Strings(order)
var b strings.Builder
if aborted > 0 {
fmt.Fprintf(&b, "> ⚠ swarm cancelled — %d cell(s) did not run; results below are partial.\n\n", aborted)
}
for _, m := range order {
rs := byModel[m]
var all []Finding
worst := -1
errored := 0
for _, r := range rs {
if r.Err != nil {
errored++
continue
}
all = append(all, r.Findings...)
for _, f := range r.Findings {
if severityRank(f.Severity) > worst {
worst = severityRank(f.Severity)
}
}
}
// A model whose every lens errored produced NO data — saying "no issues
// found" would be misleading, so it gets its own verdict.
successful := len(rs) - errored
verdict := "no issues found"
switch {
case successful == 0 && errored > 0:
verdict = "review incomplete"
case worst >= severityRank(SevHigh):
verdict = "blocking issues found"
case worst >= 0:
verdict = "minor issues"
}
fmt.Fprintf(&b, "## %s — %s", m, verdict)
if errored > 0 {
fmt.Fprintf(&b, " (⚠ %d lens(es) errored)", errored)
}
b.WriteString("\n")
sort.SliceStable(all, func(i, j int) bool {
return severityRank(all[i].Severity) > severityRank(all[j].Severity)
})
for _, f := range all {
fmt.Fprintf(&b, "- [%s] %s — %s\n", f.Severity, f.Title, f.Detail)
}
b.WriteString("\n")
}
return b.String()
}
+128
View File
@@ -0,0 +1,128 @@
package main
import (
"context"
"strings"
"testing"
"gitea.stevedudenhoeffer.com/steve/majordomo/provider/fake"
"gitea.stevedudenhoeffer.com/steve/executus/fanout"
)
// TestReviewSwarm proves the light-tier path end-to-end against the fake
// provider: a 2-model × 3-lens swarm runs, structured findings parse, and
// consolidation produces one verdict-led section per model — no batteries, no
// network.
func TestReviewSwarm(t *testing.T) {
fp := fake.New("fakeprov")
// Model "hot" reports a high-severity finding on every lens; "cold" reports
// nothing. Each model is called once per lens (3×), so enqueue 3 each.
hot := `{"findings":[{"severity":"high","title":"SQL injection","detail":"unsanitized id in query"}]}`
cold := `{"findings":[]}`
for i := 0; i < 3; i++ {
fp.Enqueue("hot", fake.Reply(hot))
fp.Enqueue("cold", fake.Reply(cold))
}
hotM, err := fp.Model("hot")
if err != nil {
t.Fatal(err)
}
coldM, err := fp.Model("cold")
if err != nil {
t.Fatal(err)
}
models := []NamedModel{
{Name: "hot", Provider: "fakeprov", Model: hotM},
{Name: "cold", Provider: "fakeprov", Model: coldM},
}
lenses := []Lens{{Name: "security"}, {Name: "correctness"}, {Name: "error-handling"}}
results := Review(context.Background(), models, lenses, "some diff",
fanout.Options[cell]{MaxConcurrent: 6, PerKey: map[string]int{"fakeprov": 3}})
// 2 models × 3 lenses = 6 cells, all successful.
if len(results) != 6 {
t.Fatalf("got %d cells, want 6", len(results))
}
var hotFindings, coldFindings, errs int
for _, r := range results {
if r.Err != nil {
errs++
continue
}
switch r.Model {
case "hot":
hotFindings += len(r.Findings)
case "cold":
coldFindings += len(r.Findings)
}
}
if errs != 0 {
t.Errorf("expected no cell errors, got %d", errs)
}
if hotFindings != 3 { // one per lens
t.Errorf("hot model findings = %d, want 3", hotFindings)
}
if coldFindings != 0 {
t.Errorf("cold model findings = %d, want 0", coldFindings)
}
report := Consolidate(results)
if !strings.Contains(report, "hot — blocking issues found") {
t.Errorf("hot section should lead with a blocking verdict:\n%s", report)
}
if !strings.Contains(report, "cold — no issues found") {
t.Errorf("cold section should report no issues:\n%s", report)
}
if !strings.Contains(report, "SQL injection") {
t.Errorf("report should surface the finding:\n%s", report)
}
}
// TestConsolidateVerdicts checks the worst-severity-led header logic.
func TestConsolidateVerdicts(t *testing.T) {
got := Consolidate([]LensResult{
{Model: "m", Lens: "a", Findings: []Finding{{Severity: SevSmall, Title: "x"}}},
{Model: "m", Lens: "b", Findings: []Finding{{Severity: SevMedium, Title: "y"}}},
})
if !strings.Contains(got, "m — minor issues") {
t.Errorf("medium-max should be 'minor issues', got:\n%s", got)
}
// An errored lens is surfaced in the header.
got = Consolidate([]LensResult{
{Model: "m", Lens: "a", Findings: []Finding{{Severity: SevCritical, Title: "boom"}}},
{Model: "m", Lens: "b", Err: context.DeadlineExceeded},
})
if !strings.Contains(got, "blocking issues found") || !strings.Contains(got, "errored") {
t.Errorf("critical + errored lens header wrong:\n%s", got)
}
}
// TestConsolidateAllErrored: a model whose every lens errored must NOT be
// labelled "no issues found" (the gadfly P5 finding).
func TestConsolidateAllErrored(t *testing.T) {
got := Consolidate([]LensResult{
{Model: "m", Lens: "a", Err: context.DeadlineExceeded},
{Model: "m", Lens: "b", Err: context.DeadlineExceeded},
})
if !strings.Contains(got, "m — review incomplete") {
t.Errorf("all-errored model should be 'review incomplete', got:\n%s", got)
}
if strings.Contains(got, "no issues found") {
t.Errorf("all-errored model must not say 'no issues found':\n%s", got)
}
}
// TestConsolidateSwarmCancelled: dropped (unattributed) cells surface a banner.
func TestConsolidateSwarmCancelled(t *testing.T) {
got := Consolidate([]LensResult{
{Err: context.Canceled}, // dropped cell, no model
{Model: "m", Lens: "a", Findings: []Finding{{Severity: SevSmall, Title: "x"}}},
})
if !strings.Contains(got, "swarm cancelled") {
t.Errorf("dropped cells should surface a cancellation banner:\n%s", got)
}
}
+22
View File
@@ -4,5 +4,27 @@ go 1.26.2
require (
gitea.stevedudenhoeffer.com/steve/majordomo v0.0.0-20260626223738-1fd7109a42f3
github.com/google/uuid v1.6.0
github.com/robfig/cron/v3 v3.0.1
golang.org/x/crypto v0.53.0
gopkg.in/yaml.v3 v3.0.1
)
require (
cloud.google.com/go v0.116.0 // indirect
cloud.google.com/go/auth v0.9.3 // indirect
cloud.google.com/go/compute/metadata v0.5.0 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/google/go-cmp v0.6.0 // indirect
github.com/google/s2a-go v0.1.8 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.4 // indirect
github.com/gorilla/websocket v1.5.3 // indirect
go.opencensus.io v0.24.0 // indirect
golang.org/x/net v0.55.0 // indirect
golang.org/x/sys v0.46.0 // indirect
golang.org/x/text v0.38.0 // indirect
google.golang.org/genai v1.59.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20240903143218-8af14fe29dc1 // indirect
google.golang.org/grpc v1.66.2 // indirect
google.golang.org/protobuf v1.34.2 // indirect
)
+130
View File
@@ -1,4 +1,134 @@
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.116.0 h1:B3fRrSDkLRt5qSHWe40ERJvhvnQwdZiHu0bJOpldweE=
cloud.google.com/go v0.116.0/go.mod h1:cEPSRWPzZEswwdr9BxE6ChEn01dWlTaF05LiC2Xs70U=
cloud.google.com/go/auth v0.9.3 h1:VOEUIAADkkLtyfr3BLa3R8Ed/j6w1jTBmARx+wb5w5U=
cloud.google.com/go/auth v0.9.3/go.mod h1:7z6VY+7h3KUdRov5F1i8NDP5ZzWKYmEPO842BgCsmTk=
cloud.google.com/go/compute/metadata v0.5.0 h1:Zr0eK8JbFv6+Wi4ilXAR8FJ3wyNdpxHKJNPos6LTZOY=
cloud.google.com/go/compute/metadata v0.5.0/go.mod h1:aHnloV2TPI38yx4s9+wAZhHykWvVCfu7hQbF+9CWoiY=
gitea.stevedudenhoeffer.com/steve/majordomo v0.0.0-20260626223738-1fd7109a42f3 h1:KYKIFFRsXzbbBJVDa99+Fhy0zxl9G0xV/MCrLipsLL4=
gitea.stevedudenhoeffer.com/steve/majordomo v0.0.0-20260626223738-1fd7109a42f3/go.mod h1:UZLveG17SmENt4sne2RSLIbioix30RZbRIQUzBAnOyY=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da h1:oI5xCqsCo564l8iNU+DwB5epxmsaqB+rhGL0m5jtYqE=
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=
github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=
github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs=
github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w=
github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=
github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8=
github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/s2a-go v0.1.8 h1:zZDs9gcbt9ZPLV0ndSyQk6Kacx2g/X+SKYovpnz3SMM=
github.com/google/s2a-go v0.1.8/go.mod h1:6iNWHTpQ+nfNRN5E00MSdfDwVesa8hhS32PhPO8deJA=
github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/googleapis/enterprise-certificate-proxy v0.3.4 h1:XYIDZApgAnrN1c855gTgghdIA6Stxb52D5RnLI1SLyw=
github.com/googleapis/enterprise-certificate-proxy v0.3.4/go.mod h1:YKe7cfqYXjKGpGvmSg28/fFvhNzinZQm8DGnaburhGA=
github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg=
github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/robfig/cron/v3 v3.0.1 h1:WdRxkvbJztn8LMz/QEvLN5sBU+xKpSqwwUO1Pjr4qDs=
github.com/robfig/cron/v3 v3.0.1/go.mod h1:eQICP3HwyT7UooqI/z+Ov+PtYAWygg1TEWWzGIFLtro=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
go.opencensus.io v0.24.0 h1:y73uSU6J157QMP2kn2r30vwW1A2W2WFwSCGnAVxeaD0=
go.opencensus.io v0.24.0/go.mod h1:vNK8G9p7aAivkbmorf4v+7Hgx+Zs0yY+0fOtgBfjQKo=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.53.0 h1:QZ4Muo8THX6CizN2vPPd5fBGHyogrdK9fG4wLPFUsto=
golang.org/x/crypto v0.53.0/go.mod h1:DNLU434OwVakk9PzuwV8w62mAJpRJL3vsgcfp4Qnsio=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.55.0 h1:bcvxaJn3e1U6InsFWt1JUq1aSjnRxLzT2rtD2KfkDF8=
golang.org/x/net v0.55.0/go.mod h1:L5U2KuzuOe1lY7Z+aWVIKK6qEeJXnXV9yzGA+WCHJww=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.21.0 h1:HLII4xRRTtCRkxYp4HNFF0Js/Og6q2i++KXbg0gHCwM=
golang.org/x/sync v0.21.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.46.0 h1:noSf2Fq6F8DBgS+LysIkx7rIExoNHJsxOAtPp4rthXw=
golang.org/x/sys v0.46.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.38.0 h1:sXmwo9DwP3OK9EZ7PqAdaooSGozfl/3a6/xJcbzPRhE=
golang.org/x/text v0.38.0/go.mod h1:YXZt3QhHUKYT53r2lLKFIVi6Ao1jdzrTR/KQ09qyxF4=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/genai v1.59.0 h1:xp+ydkJFW8hO0hTUaAkr8TrLM9HFP3NYAwFhPd0nDqA=
google.golang.org/genai v1.59.0/go.mod h1:mDdPDFXo1Ats7f1WXVyZgWb/CkMzFWTWJruIMy7hGIU=
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo=
google.golang.org/genproto/googleapis/rpc v0.0.0-20240903143218-8af14fe29dc1 h1:pPJltXNxVzT4pK9yD8vR9X75DaWYYmLGMsEvBfFQZzQ=
google.golang.org/genproto/googleapis/rpc v0.0.0-20240903143218-8af14fe29dc1/go.mod h1:UqMtugtsSgubUsoxbuAoiCXvqvErP7Gf0so0mK9tHxU=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=
google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/grpc v1.33.2/go.mod h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc=
google.golang.org/grpc v1.66.2 h1:3QdXkuq3Bkh7w+ywLdLvM56cmGvQHUMZpiCzt6Rqaoo=
google.golang.org/grpc v1.66.2/go.mod h1:s3/l6xSSCURdVfAnL+TqCNMyTDAGN6+lZeVxnZR128Y=
google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE=
google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo=
google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=
google.golang.org/protobuf v1.34.2 h1:6xV6lTsCfpGD21XK49h7MhtcApnLqkfYgPcdHftf6hg=
google.golang.org/protobuf v1.34.2/go.mod h1:qYOHts0dSfpeUzUFpOMr/WGzszTmLH+DiWniOlNbLDw=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
+618
View File
@@ -0,0 +1,618 @@
// Package llmmeta is the shared meta-LLM helper used by the v12
// authoring tools (summarize, translate, extract_entities, classify).
//
// Why a dedicated package: each of those four tools makes "one fast-tier
// LLM call → typed result", with shared concerns (tier allowlist,
// ledger row, JSON-retry on malformed output). Centralising the pattern
// stops every tool from re-implementing the surrounding bookkeeping and
// keeps the audit trail uniform.
//
// The helper itself does NOT know about the four tools — it just exposes
// a Call(ctx, CallSpec) → CallResult shape. Each tool builds its own
// prompt + parses the typed result. The helper records the meta-call
// ledger row on every call, success or failure.
//
// Concurrency / lanes: the helper resolves the tier to an llm.Model via
// model.ParseModelForContext and uses model.Generate. Lane routing is
// already baked in at the LLM transport layer (see
// pkg/logic/llms/lane_transport.go) so each Generate call automatically
// goes through the right lane without further plumbing. Usage recording
// is automatic too: parsed models are instrumented by pkg/logic/llms,
// so the helper does NOT call model.RecordUsage itself.
//
// Tier allowlist: convar `skills.llm_meta.allowed_tiers` (default
// `["fast"]`) controls which tiers a meta-tool may use. A request for
// a disallowed tier returns error_kind="tier_not_allowed" WITHOUT
// making the call AND WITHOUT recording a ledger row (the call did
// not happen).
//
// Test: helper_test.go covers tier allowed, tier rejected, JSON
// retry path, malformed-twice path, and ledger-row emission semantics.
package llmmeta
import (
"context"
"encoding/json"
"fmt"
"log/slog"
"strings"
"time"
llm "gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"github.com/google/uuid"
"gitea.stevedudenhoeffer.com/steve/executus/model"
)
// MetaCall is the domain row written to skill_llm_meta_calls on every
// helper call.
//
// Why a dedicated table (not skill_run_logs): per-skill token
// aggregation is cleaner with typed columns. Folding meta-calls into
// the generic event log would force a SUM-from-JSON path on every
// dashboard query.
//
// Why the field set is tight (no payload columns): the request bodies
// can be 32KB+. The agent's main run already captures system_prompt
// + user_message in the trace; storing them again here would double
// the audit footprint with no diagnostic value (the meta-call's
// inputs are derivable from the parent run's tool-call args).
type MetaCall struct {
ID string
RunID string
SkillID string
ToolName string
TierUsed string // "fast" / "standard"
ModelUsed string // resolved provider/model
InputTokens int
OutputTokens int
DurationMs int
Success bool
ErrorKind string // empty on success; one of the sentinel kinds otherwise
CreatedAt time.Time
}
// Storage is the narrow surface the helper uses to persist meta-call
// ledger rows. Production wires a thin adapter around the skills GORM
// storage; tests substitute a fake.
//
// Why an interface (vs depending on pkg/logic/skills.Storage): the
// skills package imports skilltools (tool registry); having
// skilltools/llmmeta depend back on skills would form an import
// cycle. A narrow interface mirrored across the boundary is the
// project's standard cycle-break pattern (see KVStorage / FileStorage
// in pkg/skilltools/tools/).
type Storage interface {
RecordMetaCall(ctx context.Context, call MetaCall) error
}
// ConvarReader is the narrow surface the helper uses to read
// `skills.llm_meta.allowed_tiers`. The convar package is database-
// backed; tests pass a static fake.
//
// Why an interface (vs reading convars directly): unit tests want to
// fake the allowlist without spinning up a convar manager.
type ConvarReader interface {
// AllowedTiers returns the list of tier names a meta-tool may use.
// Default ["fast"].
AllowedTiers(ctx context.Context) []string
}
// ConvarReaderFunc adapts a closure into a ConvarReader. Useful in
// production wiring (mort.go) where the underlying access is a
// single line of logic.
type ConvarReaderFunc func(ctx context.Context) []string
// AllowedTiers satisfies ConvarReader.
func (f ConvarReaderFunc) AllowedTiers(ctx context.Context) []string {
if f == nil {
return []string{"fast"}
}
return f(ctx)
}
// Helper makes one fast-tier LLM call with surrounding bookkeeping
// (tier allowlist, JSON retry, ledger row).
//
// Construct once at boot; all four meta-tools share the same Helper.
type Helper struct {
storage Storage
convars ConvarReader
}
// New constructs a Helper. storage MUST be non-nil; passing nil makes
// every Call write a no-op ledger row (callers that need a fully no-op
// helper should instead avoid registering the tool).
//
// convars may be nil — the helper falls back to the default allowlist
// `["fast"]`.
//
// Why a constructor with explicit deps (vs Helper{...} struct
// initialiser): forces the deployment-time decision about which
// dependencies are wired vs nil-safe at the construction call site,
// not at the call site of each tool.
func New(storage Storage, convars ConvarReader) *Helper {
return &Helper{
storage: storage,
convars: convars,
}
}
// CallSpec is the per-call input.
//
// Why every field is explicit (vs builder pattern): the four meta-tools
// each populate the spec in one place; a struct literal at the call
// site is more readable than chained setters.
type CallSpec struct {
// Tier is the tier alias to use ("fast" / "standard"). Empty falls
// back to "fast". Disallowed tiers (per the convar allowlist) cause
// Call to return CallResult{Success: false, ErrorKind:
// "tier_not_allowed"} WITHOUT making the LLM call AND without
// writing a ledger row (the call did not happen).
Tier string
// SystemPrompt is the system message. May be empty.
SystemPrompt string
// UserPrompt is the user message. Required.
UserPrompt string
// MaxOutputTokens caps the response. 0 disables the cap (provider
// default). The helper uses this both to bound the cost estimate
// AND to set llm.WithMaxTokens on the request.
MaxOutputTokens int
// ResponseFormat is "text" or "json". When "json", the helper
// attempts to parse the response into JSON. Other values fall
// through as "text".
ResponseFormat string
// RetryOnMalformedJSON, when true and ResponseFormat=="json",
// retries the call ONCE with a stricter JSON-only prompt prefix
// when the first response fails to parse. Second-failure returns
// CallResult{Success: true, Parsed: nil, ErrorKind:
// "malformed_json"} so callers can fall back to result.Text.
RetryOnMalformedJSON bool
// ToolName is the meta-tool name recorded in the ledger row
// ("summarize", "translate", "extract_entities", "classify"). The
// helper does not branch on this value.
ToolName string
// RunID is the calling skill run ID. Recorded in the ledger row;
// also used by the cost-cap callback to find the running 7-day
// total.
RunID string
// SkillID is the calling skill ID. Recorded in the ledger row;
// passed to the cost-cap callback.
SkillID string
// CallerID is the Discord member ID that triggered the parent
// skill run. Passed to the cost-cap callback so the per-user
// 7-day cap can be evaluated.
CallerID string
}
// CallResult is the per-call output.
//
// Why text + parsed (vs only one): JSON-format calls expose both the
// raw response (in .Text) and the parsed map (in .Parsed). Text-format
// calls leave .Parsed nil. Callers requesting JSON that fails to parse
// twice get .Text populated and ErrorKind="malformed_json" so they
// can fall back to text-mode without an error path.
type CallResult struct {
// Text is the raw response text from the LLM. Populated on every
// successful call (success=true) AND when JSON parsing failed
// twice (success=true, parsed=nil, error_kind="malformed_json").
// Empty on tier_not_allowed rejections (no LLM call happened).
Text string
// Parsed is the JSON-decoded response. nil for text-format calls,
// nil for failed JSON parses, populated for successful JSON
// responses. The interior shape is whatever the LLM returned; the
// caller is responsible for asserting a typed view.
Parsed any
// InputTokens is the tokens billed against the input. 0 when the
// provider didn't surface usage.
InputTokens int
// OutputTokens is the tokens billed against the output. 0 when the
// provider didn't surface usage.
OutputTokens int
// DurationMs is wall-clock duration of the LLM call (or call+retry
// in the JSON-retry case).
DurationMs int
// ModelUsed is the resolved provider/model string ("anthropic/
// claude-haiku-4-5-20251001"). Populated on every actual LLM call;
// empty on tier_not_allowed rejections.
ModelUsed string
// Success reports whether the LLM call returned a usable response.
// True on happy-path AND on malformed-json second-failure (the
// caller can fall back to .Text). False on transport errors,
// tier_not_allowed, llm_unavailable.
Success bool
// ErrorKind, when non-empty, is one of:
// - "tier_not_allowed" → no call, no ledger row
// - "llm_unavailable" → call attempted, ledger row written
// - "malformed_json" → call succeeded but JSON parse failed
ErrorKind string
}
// Sentinel error_kind values for CallResult.ErrorKind.
const (
ErrorKindTierNotAllowed = "tier_not_allowed"
ErrorKindLLMUnavailable = "llm_unavailable"
ErrorKindMalformedJSON = "malformed_json"
)
// Call performs the meta-LLM call and returns a typed CallResult.
//
// Why no error return (vs an error second value): every meaningful
// failure is captured as a CallResult.ErrorKind so the caller's branch
// logic stays single-pathed. Internal transport errors are surfaced
// as ErrorKind=llm_unavailable. The function only returns a non-nil
// error for argument-validation failures (empty UserPrompt) — a
// programmer error the caller would have to fix anyway.
//
// Test: helper_test.go covers all outcomes (tier_not_allowed, happy
// text, happy json, malformed_json retry-pass, malformed_json
// retry-fail, llm_unavailable).
func (h *Helper) Call(ctx context.Context, spec CallSpec) (CallResult, error) {
if strings.TrimSpace(spec.UserPrompt) == "" {
return CallResult{}, fmt.Errorf("llmmeta: user_prompt required")
}
tier := strings.TrimSpace(spec.Tier)
if tier == "" {
tier = "fast"
}
// Tier allowlist: rejected tiers do NOT make the call AND do NOT
// record a ledger row.
if !h.tierAllowed(ctx, tier) {
return CallResult{
Success: false,
ErrorKind: ErrorKindTierNotAllowed,
}, nil
}
resolvedModel := model.ResolveModelName(tier)
// Resolve model. ParseModelForContext attaches the resolved model
// name to ctx (for usage attribution) AND returns the llm.Model
// whose Generate already routes through the lane wrapper.
ctx, model, err := model.ParseModelForContext(ctx, tier)
if err != nil {
// Tier convar mis-set: surface as tier_not_allowed to the
// caller (the agent's recovery path is the same as for an
// admin-disabled tier) but DO record the failure for the
// admin who needs to fix the convar.
h.recordLedger(ctx, MetaCall{
ID: uuid.NewString(),
RunID: spec.RunID,
SkillID: spec.SkillID,
ToolName: spec.ToolName,
TierUsed: tier,
ModelUsed: resolvedModel,
Success: false,
ErrorKind: ErrorKindTierNotAllowed,
CreatedAt: time.Now(),
})
return CallResult{
Success: false,
ErrorKind: ErrorKindTierNotAllowed,
}, nil
}
// First call.
start := time.Now()
systemPrompt := spec.SystemPrompt
userMessage := spec.UserPrompt
opts := []llm.Option{}
if spec.MaxOutputTokens > 0 {
opts = append(opts, llm.WithMaxTokens(spec.MaxOutputTokens))
}
text, usage, llmErr := h.complete(ctx, model, systemPrompt, userMessage, opts)
if llmErr != nil {
duration := int(time.Since(start) / time.Millisecond)
h.recordLedger(ctx, MetaCall{
ID: uuid.NewString(),
RunID: spec.RunID,
SkillID: spec.SkillID,
ToolName: spec.ToolName,
TierUsed: tier,
ModelUsed: resolvedModel,
InputTokens: usage.InputTokens,
OutputTokens: usage.OutputTokens,
DurationMs: duration,
Success: false,
ErrorKind: ErrorKindLLMUnavailable,
CreatedAt: time.Now(),
})
return CallResult{
Success: false,
ErrorKind: ErrorKindLLMUnavailable,
ModelUsed: resolvedModel,
DurationMs: duration,
InputTokens: usage.InputTokens,
OutputTokens: usage.OutputTokens,
}, nil
}
// Determine outcome based on response format.
parsed, parsedOK := tryParseJSON(text, spec.ResponseFormat)
wantJSON := strings.EqualFold(spec.ResponseFormat, "json")
if !wantJSON || parsedOK {
// Happy path (text mode OR JSON mode that parsed first try).
duration := int(time.Since(start) / time.Millisecond)
h.recordLedger(ctx, MetaCall{
ID: uuid.NewString(),
RunID: spec.RunID,
SkillID: spec.SkillID,
ToolName: spec.ToolName,
TierUsed: tier,
ModelUsed: resolvedModel,
InputTokens: usage.InputTokens,
OutputTokens: usage.OutputTokens,
DurationMs: duration,
Success: true,
CreatedAt: time.Now(),
})
return CallResult{
Text: text,
Parsed: parsed,
Success: true,
ModelUsed: resolvedModel,
InputTokens: usage.InputTokens,
OutputTokens: usage.OutputTokens,
DurationMs: duration,
}, nil
}
// JSON requested but first response failed to parse.
if !spec.RetryOnMalformedJSON {
duration := int(time.Since(start) / time.Millisecond)
h.recordLedger(ctx, MetaCall{
ID: uuid.NewString(),
RunID: spec.RunID,
SkillID: spec.SkillID,
ToolName: spec.ToolName,
TierUsed: tier,
ModelUsed: resolvedModel,
InputTokens: usage.InputTokens,
OutputTokens: usage.OutputTokens,
DurationMs: duration,
Success: true,
ErrorKind: ErrorKindMalformedJSON,
CreatedAt: time.Now(),
})
return CallResult{
Text: text,
Success: true,
ErrorKind: ErrorKindMalformedJSON,
ModelUsed: resolvedModel,
InputTokens: usage.InputTokens,
OutputTokens: usage.OutputTokens,
DurationMs: duration,
}, nil
}
// Retry once with stricter JSON-only prompt prefix.
stricterPrompt := "Return ONLY valid JSON. No prose, no markdown fencing.\n\n" + userMessage
text2, usage2, llmErr2 := h.complete(ctx, model, systemPrompt, stricterPrompt, opts)
combinedUsage := Tokens{
InputTokens: usage.InputTokens + usage2.InputTokens,
OutputTokens: usage.OutputTokens + usage2.OutputTokens,
}
duration := int(time.Since(start) / time.Millisecond)
if llmErr2 != nil {
// Retry call itself failed transport-wise. Record the round-
// trip tokens and surface llm_unavailable.
h.recordLedger(ctx, MetaCall{
ID: uuid.NewString(),
RunID: spec.RunID,
SkillID: spec.SkillID,
ToolName: spec.ToolName,
TierUsed: tier,
ModelUsed: resolvedModel,
InputTokens: combinedUsage.InputTokens,
OutputTokens: combinedUsage.OutputTokens,
DurationMs: duration,
Success: false,
ErrorKind: ErrorKindLLMUnavailable,
CreatedAt: time.Now(),
})
return CallResult{
Text: text,
Success: false,
ErrorKind: ErrorKindLLMUnavailable,
ModelUsed: resolvedModel,
InputTokens: combinedUsage.InputTokens,
OutputTokens: combinedUsage.OutputTokens,
DurationMs: duration,
}, nil
}
parsed2, parsedOK2 := tryParseJSON(text2, "json")
if parsedOK2 {
h.recordLedger(ctx, MetaCall{
ID: uuid.NewString(),
RunID: spec.RunID,
SkillID: spec.SkillID,
ToolName: spec.ToolName,
TierUsed: tier,
ModelUsed: resolvedModel,
InputTokens: combinedUsage.InputTokens,
OutputTokens: combinedUsage.OutputTokens,
DurationMs: duration,
Success: true,
CreatedAt: time.Now(),
})
return CallResult{
Text: text2,
Parsed: parsed2,
Success: true,
ModelUsed: resolvedModel,
InputTokens: combinedUsage.InputTokens,
OutputTokens: combinedUsage.OutputTokens,
DurationMs: duration,
}, nil
}
// Second-failure path. Caller can fall back to result.Text.
h.recordLedger(ctx, MetaCall{
ID: uuid.NewString(),
RunID: spec.RunID,
SkillID: spec.SkillID,
ToolName: spec.ToolName,
TierUsed: tier,
ModelUsed: resolvedModel,
InputTokens: combinedUsage.InputTokens,
OutputTokens: combinedUsage.OutputTokens,
DurationMs: duration,
Success: true,
ErrorKind: ErrorKindMalformedJSON,
CreatedAt: time.Now(),
})
return CallResult{
Text: text2,
Success: true,
ErrorKind: ErrorKindMalformedJSON,
ModelUsed: resolvedModel,
InputTokens: combinedUsage.InputTokens,
OutputTokens: combinedUsage.OutputTokens,
DurationMs: duration,
}, nil
}
// Tokens is the input/output token count returned by the LLM round-
// trip. Mirrors llm.Usage's two cost-bearing fields. Exported so
// downstream test code (the four meta-tools' tests, integration
// tests) can use SetCompleteForTest.
type Tokens struct {
InputTokens int
OutputTokens int
}
// CompleteFn is the seam used by tests to fake the LLM round-trip
// without spinning up a real provider. Exported for tests in other
// packages (the four meta-tools live in pkg/skilltools/tools/).
type CompleteFn func(ctx context.Context, model llm.Model, systemPrompt, userMessage string, opts []llm.Option) (string, Tokens, error)
// completeOverride is set in tests via SetCompleteForTest. nil falls
// back to the real model.Generate path.
var completeOverride CompleteFn
// complete is the actual LLM round-trip. Calls model.Generate (which
// already routes through the lane transport wrapper) and returns the
// text + usage + error.
//
// Why not call model.SimpleCall: SimpleCall doesn't surface Usage; we
// need the input/output token counts for the ledger row.
//
// Usage attribution to the per-user / per-skill dashboards is handled
// by the instrumented model that model.ParseModelForContext returns —
// a manual model.RecordUsage here would double-count.
func (h *Helper) complete(ctx context.Context, model llm.Model, systemPrompt, userMessage string, opts []llm.Option) (string, Tokens, error) {
if completeOverride != nil {
return completeOverride(ctx, model, systemPrompt, userMessage, opts)
}
req := llm.Request{
System: systemPrompt,
Messages: []llm.Message{llm.UserText(userMessage)},
}
resp, err := model.Generate(ctx, req, opts...)
if err != nil {
return "", Tokens{}, err
}
usage := Tokens{
InputTokens: resp.Usage.InputTokens,
OutputTokens: resp.Usage.OutputTokens,
}
return resp.Text(), usage, nil
}
// SetCompleteForTest installs a fake completer used by Call. Returns a
// restore function that the test deferes to revert the override.
//
// Why exported (vs in a _test.go file): the four meta-tools' tests live
// in pkg/skilltools/tools/, in a different package than the helper.
// They need a way to fake the LLM without depending on a real model.
func SetCompleteForTest(fn CompleteFn) func() {
prev := completeOverride
completeOverride = fn
return func() { completeOverride = prev }
}
// tierAllowed reports whether the given tier appears in the configured
// allowlist. Empty allowlist defaults to ["fast"].
func (h *Helper) tierAllowed(ctx context.Context, tier string) bool {
var allowed []string
if h.convars != nil {
allowed = h.convars.AllowedTiers(ctx)
}
if len(allowed) == 0 {
allowed = []string{"fast"}
}
for _, t := range allowed {
if strings.EqualFold(strings.TrimSpace(t), tier) {
return true
}
}
return false
}
// recordLedger writes one meta-call row. Storage failures are logged
// at the storage layer; the helper does not propagate them — meta-call
// accounting MUST NOT break user-visible execution.
func (h *Helper) recordLedger(ctx context.Context, call MetaCall) {
if h.storage == nil {
return
}
if err := h.storage.RecordMetaCall(ctx, call); err != nil {
slog.Warn("llmmeta: failed to record ledger row", "err", err)
}
}
// tryParseJSON attempts to decode text as JSON. Returns the parsed
// value (any) and ok=true on success. ok=false on failure or when
// format is not "json".
//
// Why we accept arbitrary JSON shapes (vs requiring an object): the
// extract_entities tool returns objects, but classify returns objects
// with arrays inside. Accepting `any` keeps the helper agnostic to the
// caller's downstream typing.
//
// Tolerance: strips a leading "```json" code fence + matching closing
// fence so the agent can include surrounding markdown without
// breaking parse. The stricter retry prompt explicitly asks for no
// fence; this tolerance is for the first-attempt path.
func tryParseJSON(text, format string) (any, bool) {
if !strings.EqualFold(format, "json") {
return nil, false
}
trimmed := strings.TrimSpace(text)
// Strip optional ```json ... ``` fence.
if strings.HasPrefix(trimmed, "```") {
// Drop opening fence (with or without language tag).
if idx := strings.Index(trimmed, "\n"); idx >= 0 {
trimmed = trimmed[idx+1:]
}
// Drop trailing fence.
if idx := strings.LastIndex(trimmed, "```"); idx >= 0 {
trimmed = trimmed[:idx]
}
trimmed = strings.TrimSpace(trimmed)
}
var parsed any
if err := json.Unmarshal([]byte(trimmed), &parsed); err != nil {
return nil, false
}
return parsed, true
}
+282
View File
@@ -0,0 +1,282 @@
package llmmeta
import (
"context"
"errors"
"strings"
"sync"
"testing"
llm "gitea.stevedudenhoeffer.com/steve/majordomo/llm"
)
// fakeStorage records every MetaCall handed to RecordMetaCall and
// makes them available to tests via the captured slice.
type fakeStorage struct {
mu sync.Mutex
calls []MetaCall
err error
}
func (f *fakeStorage) RecordMetaCall(_ context.Context, call MetaCall) error {
f.mu.Lock()
defer f.mu.Unlock()
f.calls = append(f.calls, call)
return f.err
}
func (f *fakeStorage) snapshot() []MetaCall {
f.mu.Lock()
defer f.mu.Unlock()
out := make([]MetaCall, len(f.calls))
copy(out, f.calls)
return out
}
// TestCall_TierNotAllowed: a tier not in the allowlist returns the
// rejection without recording a ledger row — the call did not happen.
func TestCall_TierNotAllowed(t *testing.T) {
store := &fakeStorage{}
convars := ConvarReaderFunc(func(_ context.Context) []string {
return []string{"fast"}
})
h := New(store, convars)
res, err := h.Call(context.Background(), CallSpec{
Tier: "thinking",
UserPrompt: "hello",
ToolName: "summarize",
})
if err != nil {
t.Fatalf("unexpected err: %v", err)
}
if res.Success {
t.Errorf("expected Success=false")
}
if res.ErrorKind != ErrorKindTierNotAllowed {
t.Errorf("ErrorKind = %q, want %q", res.ErrorKind, ErrorKindTierNotAllowed)
}
if len(store.snapshot()) != 0 {
t.Errorf("expected NO ledger row for tier_not_allowed, got %d", len(store.snapshot()))
}
}
// TestCall_TierAllowedHappyText: a permitted tier yields a successful
// text call AND records a ledger row.
func TestCall_TierAllowedHappyText(t *testing.T) {
store := &fakeStorage{}
convars := ConvarReaderFunc(func(_ context.Context) []string {
return []string{"fast"}
})
h := New(store, convars)
restore := SetCompleteForTest(func(_ context.Context, _ llm.Model, _, _ string, _ []llm.Option) (string, Tokens, error) {
return "summary text here", Tokens{InputTokens: 50, OutputTokens: 12}, nil
})
defer restore()
res, err := h.Call(context.Background(), CallSpec{
Tier: "fast",
UserPrompt: "summarise the following ...",
ToolName: "summarize",
ResponseFormat: "text",
RunID: "run-1",
SkillID: "sk-1",
})
if err != nil {
t.Fatalf("unexpected err: %v", err)
}
if !res.Success {
t.Errorf("expected Success=true; got ErrorKind=%q", res.ErrorKind)
}
if res.Text != "summary text here" {
t.Errorf("Text = %q, want %q", res.Text, "summary text here")
}
if res.InputTokens != 50 || res.OutputTokens != 12 {
t.Errorf("token counts wrong: in=%d out=%d", res.InputTokens, res.OutputTokens)
}
if got := len(store.snapshot()); got != 1 {
t.Fatalf("expected 1 ledger row, got %d", got)
}
row := store.snapshot()[0]
if !row.Success {
t.Errorf("ledger Success = false, want true")
}
if row.ToolName != "summarize" {
t.Errorf("ledger ToolName = %q", row.ToolName)
}
if row.RunID != "run-1" {
t.Errorf("ledger RunID = %q", row.RunID)
}
if row.InputTokens != 50 || row.OutputTokens != 12 {
t.Errorf("ledger token counts wrong: in=%d out=%d",
row.InputTokens, row.OutputTokens)
}
}
// TestCall_JSONFirstAttemptParses: JSON-format request, response is
// valid JSON on first try; result.Parsed populated.
func TestCall_JSONFirstAttemptParses(t *testing.T) {
store := &fakeStorage{}
h := New(store, nil)
restore := SetCompleteForTest(func(_ context.Context, _ llm.Model, _, _ string, _ []llm.Option) (string, Tokens, error) {
return `{"foo":"bar","n":42}`, Tokens{InputTokens: 10, OutputTokens: 5}, nil
})
defer restore()
res, _ := h.Call(context.Background(), CallSpec{
UserPrompt: "extract entities",
ToolName: "extract_entities",
ResponseFormat: "json",
RetryOnMalformedJSON: true,
SkillID: "sk-2",
})
if !res.Success || res.ErrorKind != "" {
t.Fatalf("expected success, got %+v", res)
}
m, ok := res.Parsed.(map[string]any)
if !ok {
t.Fatalf("Parsed not a map: %T %v", res.Parsed, res.Parsed)
}
if m["foo"] != "bar" {
t.Errorf("Parsed[foo] = %v", m["foo"])
}
}
// TestCall_JSONRetryPath: first response is malformed JSON; second
// response (after stricter prompt) parses cleanly.
func TestCall_JSONRetryPath(t *testing.T) {
store := &fakeStorage{}
h := New(store, nil)
calls := 0
restore := SetCompleteForTest(func(_ context.Context, _ llm.Model, _, prompt string, _ []llm.Option) (string, Tokens, error) {
calls++
if calls == 1 {
return "Here is your JSON: {oh no I forgot to format it", Tokens{InputTokens: 8, OutputTokens: 12}, nil
}
// Verify stricter prompt prefix appeared on retry.
if !strings.Contains(prompt, "Return ONLY valid JSON") {
t.Errorf("retry prompt missing stricter prefix: %q", prompt)
}
return `{"key":"value"}`, Tokens{InputTokens: 14, OutputTokens: 6}, nil
})
defer restore()
res, _ := h.Call(context.Background(), CallSpec{
UserPrompt: "extract",
ToolName: "extract_entities",
ResponseFormat: "json",
RetryOnMalformedJSON: true,
})
if !res.Success || res.ErrorKind != "" {
t.Fatalf("expected success, got %+v", res)
}
if calls != 2 {
t.Errorf("expected 2 LLM calls, got %d", calls)
}
m, _ := res.Parsed.(map[string]any)
if m["key"] != "value" {
t.Errorf("Parsed = %v", res.Parsed)
}
// Token counts should reflect both attempts.
if res.InputTokens != 22 || res.OutputTokens != 18 {
t.Errorf("combined tokens wrong: in=%d out=%d", res.InputTokens, res.OutputTokens)
}
}
// TestCall_JSONRetryFailsTwice: second attempt also fails to parse.
// Surfaces ErrorKind=malformed_json AND keeps Success=true so the
// caller can fall back to result.Text.
func TestCall_JSONRetryFailsTwice(t *testing.T) {
store := &fakeStorage{}
h := New(store, nil)
restore := SetCompleteForTest(func(_ context.Context, _ llm.Model, _, _ string, _ []llm.Option) (string, Tokens, error) {
return "still not JSON", Tokens{InputTokens: 10, OutputTokens: 4}, nil
})
defer restore()
res, _ := h.Call(context.Background(), CallSpec{
UserPrompt: "extract",
ToolName: "extract_entities",
ResponseFormat: "json",
RetryOnMalformedJSON: true,
})
if !res.Success {
t.Errorf("expected Success=true (fall-back-to-text), got Success=false")
}
if res.ErrorKind != ErrorKindMalformedJSON {
t.Errorf("ErrorKind = %q, want %q", res.ErrorKind, ErrorKindMalformedJSON)
}
if res.Parsed != nil {
t.Errorf("Parsed = %v, want nil after failed retry", res.Parsed)
}
rows := store.snapshot()
if len(rows) != 1 {
t.Fatalf("expected 1 ledger row, got %d", len(rows))
}
if !rows[0].Success || rows[0].ErrorKind != ErrorKindMalformedJSON {
t.Errorf("ledger row mismatch: %+v", rows[0])
}
}
// TestCall_LLMUnavailable: transport error from the model.Generate
// call is surfaced as ErrorKind=llm_unavailable AND records a ledger
// row.
func TestCall_LLMUnavailable(t *testing.T) {
store := &fakeStorage{}
h := New(store, nil)
restore := SetCompleteForTest(func(_ context.Context, _ llm.Model, _, _ string, _ []llm.Option) (string, Tokens, error) {
return "", Tokens{}, errors.New("network error")
})
defer restore()
res, _ := h.Call(context.Background(), CallSpec{
UserPrompt: "hi",
ToolName: "summarize",
})
if res.Success {
t.Errorf("expected Success=false")
}
if res.ErrorKind != ErrorKindLLMUnavailable {
t.Errorf("ErrorKind = %q, want %q", res.ErrorKind, ErrorKindLLMUnavailable)
}
rows := store.snapshot()
if len(rows) != 1 {
t.Fatalf("expected 1 ledger row, got %d", len(rows))
}
}
// TestCall_EmptyUserPromptErrors: programmer-error guard.
func TestCall_EmptyUserPromptErrors(t *testing.T) {
h := New(&fakeStorage{}, nil)
_, err := h.Call(context.Background(), CallSpec{ToolName: "summarize"})
if err == nil {
t.Fatal("expected error for empty user_prompt")
}
}
// TestCall_JSONWithCodeFenceParses: tolerance for the first-attempt
// response wrapped in a ```json ... ``` fence. The retry path uses a
// stricter prompt; this test pins the first-attempt tolerance so
// callers don't waste a round-trip on a benign formatting wrapper.
func TestCall_JSONWithCodeFenceParses(t *testing.T) {
store := &fakeStorage{}
h := New(store, nil)
restore := SetCompleteForTest(func(_ context.Context, _ llm.Model, _, _ string, _ []llm.Option) (string, Tokens, error) {
return "```json\n{\"x\":1}\n```", Tokens{InputTokens: 5, OutputTokens: 4}, nil
})
defer restore()
res, _ := h.Call(context.Background(), CallSpec{
UserPrompt: "extract",
ToolName: "extract_entities",
ResponseFormat: "json",
RetryOnMalformedJSON: true,
})
if res.ErrorKind != "" {
t.Errorf("unexpected ErrorKind %q (fenced JSON should parse on first attempt)", res.ErrorKind)
}
m, _ := res.Parsed.(map[string]any)
if m["x"] != float64(1) {
t.Errorf("Parsed[x] = %v, want 1", m["x"])
}
}
+21
View File
@@ -0,0 +1,21 @@
package llmmeta
import (
"os"
"testing"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/model"
)
// TestMain configures a minimal model tier table so the helper's
// model.ParseModelForContext("fast"/"standard") resolves. The actual LLM call
// is stubbed per-test via SetCompleteForTest, so these specs are only parsed
// (anthropic registers with an empty key and errors at call time, not parse).
func TestMain(m *testing.M) {
model.Configure(nil, map[string]string{
"fast": "anthropic/claude-haiku-4-5",
"standard": "anthropic/claude-sonnet-4-6",
}, time.Minute)
os.Exit(m.Run())
}
+97
View File
@@ -0,0 +1,97 @@
// Package llms — bench.go: the mort-flavored facade over majordomo's
// health tracker for the `.failover` Discord commands and the failover
// web UI.
//
// Why a facade (vs exposing health.Tracker directly): the admin surfaces
// want the historical shape — a benched-only list with a manual/auto
// flag. majordomo's tracker treats manual benches (Bench) and automatic
// backoffs identically, so the manual marker is kept mort-side.
package model
import (
"sync"
"time"
)
// BenchedModel is one currently-benched model for admin display.
type BenchedModel struct {
// Model is the "provider/model" target key.
Model string
// Until is the end of the bench window.
Until time.Time
// ConsecutiveFails is the failure count since the last success.
ConsecutiveFails int
// Manual reports the bench was placed by an operator (BenchModel)
// rather than the automatic failure threshold.
Manual bool
}
var (
manualMu sync.Mutex
manualBenches = map[string]time.Time{}
)
// ListBenched returns the currently-benched models, manual and automatic,
// from the live health tracker.
func ListBenched() []BenchedModel {
now := time.Now()
pruneManual(now)
var out []BenchedModel
for _, st := range Health().Snapshot() {
if !st.Until.After(now) {
continue
}
out = append(out, BenchedModel{
Model: st.Key,
Until: st.Until,
ConsecutiveFails: st.ConsecutiveFailures,
Manual: isManual(st.Key, st.Until),
})
}
return out
}
// BenchModel manually benches a model spec until the given time. The
// chain executor skips benched targets until the window expires (or
// UnbenchModel clears it).
func BenchModel(model string, until time.Time) {
Health().Bench(model, until)
manualMu.Lock()
manualBenches[model] = until
manualMu.Unlock()
}
// UnbenchModel clears the bench on a model. Returns true when the model
// was actually benched.
func UnbenchModel(model string) bool {
now := time.Now()
wasBenched := Health().BackedOffUntil(model).After(now)
Health().Unbench(model)
manualMu.Lock()
delete(manualBenches, model)
manualMu.Unlock()
return wasBenched
}
// isManual reports whether the bench window for key matches a manual
// bench placed via BenchModel. An automatic backoff that outlives the
// manual window supersedes the marker.
func isManual(key string, until time.Time) bool {
manualMu.Lock()
defer manualMu.Unlock()
manualUntil, ok := manualBenches[key]
return ok && !until.After(manualUntil)
}
// pruneManual drops expired manual markers so the map can't grow
// unbounded across a long uptime.
func pruneManual(now time.Time) {
manualMu.Lock()
defer manualMu.Unlock()
for k, until := range manualBenches {
if !until.After(now) {
delete(manualBenches, k)
}
}
}
+415
View File
@@ -0,0 +1,415 @@
package model
import (
"context"
"encoding/json"
"errors"
"fmt"
"log/slog"
"runtime/debug"
"strings"
"time"
majordomo "gitea.stevedudenhoeffer.com/steve/majordomo"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"github.com/google/uuid"
)
// CallResult captures the result of a single tool call execution.
type CallResult struct {
Name string
Arguments string
Result string
Error error
}
// instrumentedModel decorates a parsed model so every successful Generate
// records token usage to the usage sink automatically. This is the
// single usage chokepoint: ANY call through a model from
// ParseModelRequest / ParseModelForContext is accounted, whether it goes
// through the helpers in this file, the agent loop, or a direct
// model.Generate at a call site.
//
// IMPORTANT: do not call RecordUsage on responses from a parsed model —
// that would double-count. RecordUsage exists for models obtained outside
// this package.
type instrumentedModel struct {
inner llm.Model
}
func (m *instrumentedModel) Generate(ctx context.Context, req llm.Request, opts ...llm.Option) (*llm.Response, error) {
resp, err := m.inner.Generate(ctx, req, opts...)
if err == nil && resp != nil {
recordUsage(ctx, resp)
}
return resp, err
}
func (m *instrumentedModel) Stream(ctx context.Context, req llm.Request, opts ...llm.Option) (llm.Stream, error) {
return m.inner.Stream(ctx, req, opts...)
}
func (m *instrumentedModel) Capabilities() llm.Capabilities { return m.inner.Capabilities() }
// CallAndExecute sends messages to the model with a toolbox, executes any
// tool calls, and returns the results. It performs a single round of
// generation + tool execution (no looping) — multi-step loops belong to
// the agent package.
func CallAndExecute(ctx context.Context, model llm.Model, systemPrompt string, toolbox *llm.Toolbox, messages []llm.Message, opts ...llm.Option) ([]CallResult, string, error) {
req := llm.Request{System: systemPrompt, Messages: messages}
allOpts := make([]llm.Option, 0, len(opts)+1)
if toolbox != nil {
allOpts = append(allOpts, llm.WithToolbox(toolbox))
}
allOpts = append(allOpts, opts...)
startTime := time.Now()
resp, err := model.Generate(ctx, req, allOpts...)
if err != nil {
recordSpanFromWrapper(ctx, systemPrompt, messages, toolbox, nil, nil, startTime, err)
return nil, "", fmt.Errorf("completion failed: %w", err)
}
if len(resp.ToolCalls) == 0 || toolbox == nil {
recordSpanFromWrapper(ctx, systemPrompt, messages, toolbox, resp, nil, startTime, nil)
return nil, resp.Text(), nil
}
var results []CallResult
for _, call := range resp.ToolCalls {
tr := toolbox.Execute(ctx, call)
cr := CallResult{
Name: call.Name,
Arguments: string(call.Arguments),
Result: tr.Content,
}
if tr.IsError {
cr.Error = errors.New(tr.Content)
}
results = append(results, cr)
}
recordSpanFromWrapper(ctx, systemPrompt, messages, toolbox, resp, results, startTime, nil)
return results, resp.Text(), nil
}
// GenerateWith sends messages to the model with an optional system prompt and
// returns structured output parsed into T. T must be a struct. Uses
// majordomo's native structured output (response schema derived from T).
func GenerateWith[T any](ctx context.Context, model llm.Model, systemPrompt string, messages []llm.Message, opts ...llm.Option) (T, error) {
req := llm.Request{System: systemPrompt, Messages: messages}
startTime := time.Now()
// Capture the raw response so the trace span carries usage and the
// concrete serving model even though majordomo.Generate only returns T.
capture := &captureModel{inner: model}
result, err := majordomo.Generate[T](ctx, capture, req, opts...)
resolvedModel := resolvedModelName(ctx, capture.resp)
if tracingEnabled(ctx) {
span := Span{
SpanID: uuid.New().String(),
TraceID: traceIDFromContext(ctx),
Model: resolvedModel,
SystemPrompt: systemPrompt,
Messages: marshalMessages(messages),
DurationMs: time.Since(startTime).Milliseconds(),
StartedAt: startTime,
CompletedAt: time.Now(),
CreatedAt: time.Now(),
}
if capture.resp != nil {
span.InputTokens = capture.resp.Usage.InputTokens
span.OutputTokens = capture.resp.Usage.OutputTokens
}
if err != nil {
span.Error = err.Error()
// Structured-output failure: log loudly so operators can chase
// down a regression (e.g. a model returning prose or fenced
// JSON the decoder rejects) from the trace span alone. The
// error string includes the failing field path on decode
// errors.
if isStructuredOutputParseError(err) {
slog.Warn("llms.GenerateWith: structured-output parse failure",
"model", resolvedModel,
"span_id", span.SpanID,
"trace_id", span.TraceID,
"err", err.Error(),
)
}
} else {
b, _ := json.Marshal(result)
span.ResponseText = string(b)
}
traceSink.WriteSpan(span)
} else if err != nil && isStructuredOutputParseError(err) {
// Tracing disabled: slog.Warn is the only breadcrumb operators get.
slog.Warn("llms.GenerateWith: structured-output parse failure (no trace span)",
"model", resolvedModel,
"err", err.Error(),
)
}
return result, err
}
// captureModel records the last successful response so wrappers that
// only see the decoded result (majordomo.Generate) can still attribute
// usage and tracing.
type captureModel struct {
inner llm.Model
resp *llm.Response
}
func (m *captureModel) Generate(ctx context.Context, req llm.Request, opts ...llm.Option) (*llm.Response, error) {
resp, err := m.inner.Generate(ctx, req, opts...)
if err == nil {
m.resp = resp
}
return resp, err
}
func (m *captureModel) Stream(ctx context.Context, req llm.Request, opts ...llm.Option) (llm.Stream, error) {
return m.inner.Stream(ctx, req, opts...)
}
func (m *captureModel) Capabilities() llm.Capabilities { return m.inner.Capabilities() }
// isStructuredOutputParseError reports whether err looks like a
// structured-output failure from majordomo.Generate — either the decode
// path ("decode structured response") or the empty-response path
// ("structured response from ... is empty"). Used to gate the loud
// slog.Warn so transport errors don't get tagged as parse failures.
func isStructuredOutputParseError(err error) bool {
if err == nil {
return false
}
s := err.Error()
return strings.Contains(s, "decode structured response") ||
strings.Contains(s, "structured response from")
}
// SimpleCall sends a single user message to the model with an optional system
// prompt and returns the text response. No tools involved.
func SimpleCall(ctx context.Context, model llm.Model, systemPrompt string, userMessage string, opts ...llm.Option) (string, error) {
msgs := []llm.Message{llm.UserText(userMessage)}
startTime := time.Now()
resp, err := model.Generate(ctx, llm.Request{System: systemPrompt, Messages: msgs}, opts...)
if err != nil {
recordSpanFromWrapper(ctx, systemPrompt, msgs, nil, nil, nil, startTime, err)
return "", fmt.Errorf("completion failed: %w", err)
}
recordSpanFromWrapper(ctx, systemPrompt, msgs, nil, resp, nil, startTime, nil)
return resp.Text(), nil
}
// RecordUsage records LLM token usage from a successful Generate response.
//
// ONLY call this for models obtained outside this package: models returned
// by ParseModelRequest / ParseModelForContext record usage automatically on
// every Generate, and calling RecordUsage on their responses double-counts.
func RecordUsage(ctx context.Context, resp llm.Response) {
recordUsage(ctx, &resp)
}
// RecordSpan records a trace span for a direct model.Generate() call.
// Call this from modules that invoke model.Generate() directly when they
// want the call traced (usage is already recorded automatically for
// parsed models).
func RecordSpan(ctx context.Context, systemPrompt string, messages []llm.Message, toolbox *llm.Toolbox, resp *llm.Response, callResults []CallResult, startTime time.Time, callErr error) {
recordSpanFromWrapper(ctx, systemPrompt, messages, toolbox, resp, callResults, startTime, callErr)
}
// recordUsage records token usage for one response. The model is
// attributed from the response itself when possible (resp.Model names
// the chain element that actually served the request — more precise than
// the requested spec), falling back to the context attribution set by
// ParseModelForContext.
func recordUsage(ctx context.Context, resp *llm.Response) {
if usageSink == nil || resp == nil {
return
}
u := resp.Usage
if u.InputTokens == 0 && u.OutputTokens == 0 && u.CacheReadTokens == 0 && u.CacheWriteTokens == 0 {
return
}
model := resolvedModelName(ctx, resp)
if model == "unknown" || model == "" {
tool := toolFromContext(ctx)
if tool == "unknown" {
slog.Warn("model usage: recording with both unknown model and tool",
"user", userFromContext(ctx), "stack", string(debug.Stack()))
} else {
slog.Warn("model usage: recording with unknown model — caller should set model.WithModel or use model.ParseModelForContext",
"tool", tool, "user", userFromContext(ctx))
}
}
usageSink.Record(ctx, model, u.InputTokens, u.OutputTokens, u.CacheReadTokens, u.CacheWriteTokens)
}
// resolvedModelName picks the usage/trace attribution name: the serving
// model from the response when present ("provider/model" → "model"),
// else the context's requested model resolved through the tier table.
func resolvedModelName(ctx context.Context, resp *llm.Response) string {
if resp != nil && resp.Model != "" {
name := resp.Model
if idx := strings.Index(name, "/"); idx >= 0 {
name = name[idx+1:]
}
return name
}
return ResolveModelName(modelFromContext(ctx))
}
// tracingEnabled returns true if there's an active trace and tracing is enabled.
func tracingEnabled(ctx context.Context) bool {
if traceSink == nil {
return false
}
return traceIDFromContext(ctx) != ""
}
// recordSpanFromWrapper records a trace span if tracing is active.
func recordSpanFromWrapper(ctx context.Context, systemPrompt string, messages []llm.Message, toolbox *llm.Toolbox, resp *llm.Response, callResults []CallResult, startTime time.Time, callErr error) {
if !tracingEnabled(ctx) {
return
}
now := time.Now()
span := Span{
SpanID: uuid.New().String(),
TraceID: traceIDFromContext(ctx),
Model: resolvedModelName(ctx, resp),
SystemPrompt: systemPrompt,
Messages: marshalMessages(messages),
ToolDefinitions: marshalToolDefs(toolbox),
DurationMs: now.Sub(startTime).Milliseconds(),
StartedAt: startTime,
CompletedAt: now,
CreatedAt: now,
}
if callErr != nil {
span.Error = callErr.Error()
}
if resp != nil {
span.ResponseText = resp.Text()
span.InputTokens = resp.Usage.InputTokens
span.OutputTokens = resp.Usage.OutputTokens
if len(resp.ToolCalls) > 0 {
span.ResponseToolCalls = marshalToolCalls(resp.ToolCalls)
}
}
if len(callResults) > 0 {
span.ToolResults = marshalCallResults(callResults)
}
traceSink.WriteSpan(span)
}
// --- Serialization helpers ---
type jsonMessage struct {
Role string `json:"role"`
Text string `json:"text,omitempty"`
ToolCallID string `json:"tool_call_id,omitempty"`
ImageCount int `json:"image_count,omitempty"`
}
func marshalMessages(msgs []llm.Message) string {
out := make([]jsonMessage, 0, len(msgs))
for _, m := range msgs {
jm := jsonMessage{
Role: string(m.Role),
Text: m.Text(),
}
for _, p := range m.Parts {
if _, ok := p.(llm.ImagePart); ok {
jm.ImageCount++
}
}
if len(m.ToolResults) > 0 {
jm.ToolCallID = m.ToolResults[0].ID
}
out = append(out, jm)
}
b, _ := json.Marshal(out)
return string(b)
}
type jsonToolCall struct {
ID string `json:"id"`
Name string `json:"name"`
Arguments string `json:"arguments"`
}
func marshalToolCalls(calls []llm.ToolCall) string {
out := make([]jsonToolCall, 0, len(calls))
for _, c := range calls {
out = append(out, jsonToolCall{
ID: c.ID,
Name: c.Name,
Arguments: string(c.Arguments),
})
}
b, _ := json.Marshal(out)
return string(b)
}
type jsonCallResult struct {
Name string `json:"name"`
Arguments string `json:"arguments"`
Result string `json:"result"`
Error string `json:"error,omitempty"`
}
func marshalCallResults(results []CallResult) string {
out := make([]jsonCallResult, 0, len(results))
for _, r := range results {
jr := jsonCallResult{
Name: r.Name,
Arguments: r.Arguments,
Result: r.Result,
}
if r.Error != nil {
jr.Error = r.Error.Error()
}
out = append(out, jr)
}
b, _ := json.Marshal(out)
return string(b)
}
type jsonToolDef struct {
Name string `json:"name"`
Description string `json:"description"`
}
func marshalToolDefs(tb *llm.Toolbox) string {
if tb == nil {
return ""
}
tools := tb.Tools()
if len(tools) == 0 {
return ""
}
out := make([]jsonToolDef, 0, len(tools))
for _, t := range tools {
out = append(out, jsonToolDef{
Name: t.Name,
Description: t.Description,
})
}
b, _ := json.Marshal(out)
return string(b)
}
+474
View File
@@ -0,0 +1,474 @@
// V15.4 — Ollama Cloud dynamic context-length sync.
//
// Why: the static map in context_limits.go has to be hand-maintained
// for every new Ollama Cloud model. Cloud ships new models monthly,
// and a missing entry silently disables compaction for runs on that
// model (compactionThresholdForModel returns 0 on MaxContextTokens
// miss). Dynamic sync removes the maintenance burden and means new
// cloud models work out-of-the-box.
//
// How: at boot, mort kicks off a CloudOllamaLimitCache.RefreshAll in a
// background goroutine. RefreshAll calls /api/tags to list every
// available cloud model, then concurrently calls /api/show for each
// to extract `<family>.context_length` from the response's model_info
// map. The cache is consulted by the executor's
// compactionThresholdForModel via the cache-aware
// MaxContextTokensWithCache helper.
//
// Periodic refresh: a daily ticker re-runs RefreshAll so newly
// released models surface without a mort restart. The interval is
// intentionally not configurable — cloud model context lengths don't
// change for a given tag (only the tag pointer can move, e.g. :cloud
// → larger model), so daily is conservative.
package model
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"log/slog"
"net/http"
"strings"
"sync"
"time"
)
// defaultCloudEndpoint is the public Ollama Cloud base URL. Override
// in tests via NewCloudOllamaLimitCache's endpoint arg.
const defaultCloudEndpoint = "https://ollama.com"
// CloudOllamaLimitCache holds context-length values for Ollama Cloud
// models, populated dynamically via /api/tags + /api/show. Construct
// with NewCloudOllamaLimitCache. Safe for concurrent use.
//
// Empty when OLLAMA_API_KEY is unset — Refresh returns a clear error
// and the cache stays empty. Lookups return (0, false) and callers
// fall back to the static map / disabled compaction.
type CloudOllamaLimitCache struct {
mu sync.RWMutex
limit map[string]int
negative map[string]time.Time // model → fetch-failure time (for TTL)
endpoint string
apiKey string
httpClient *http.Client
// refreshConcurrency caps the number of concurrent /api/show calls
// during RefreshAll. Default 8 — enough to finish a ~50-model
// catalog in well under a minute without hammering Cloud.
refreshConcurrency int
// negativeTTL is how long a /api/show miss is cached before we
// retry. Prevents hammering Cloud on a typo or recently-removed
// model. Default 10 minutes.
negativeTTL time.Duration
}
// NewCloudOllamaLimitCache constructs a fresh cache. apiKey can be
// empty — RefreshAll then returns an error and the cache stays empty.
// endpoint defaults to https://ollama.com when empty. httpClient
// defaults to a 15s-timeout client.
func NewCloudOllamaLimitCache(endpoint, apiKey string, httpClient *http.Client) *CloudOllamaLimitCache {
if strings.TrimSpace(endpoint) == "" {
endpoint = defaultCloudEndpoint
}
endpoint = strings.TrimRight(endpoint, "/")
if httpClient == nil {
httpClient = &http.Client{Timeout: 15 * time.Second}
}
return &CloudOllamaLimitCache{
limit: make(map[string]int),
negative: make(map[string]time.Time),
endpoint: endpoint,
apiKey: apiKey,
httpClient: httpClient,
refreshConcurrency: 8,
negativeTTL: 10 * time.Minute,
}
}
// SetNegativeTTL overrides the negative-cache lifetime. Tests use this
// to control retry behaviour without sleeping.
func (c *CloudOllamaLimitCache) SetNegativeTTL(d time.Duration) {
if c == nil || d < 0 {
return
}
c.mu.Lock()
c.negativeTTL = d
c.mu.Unlock()
}
// Lookup returns the cached context length for an Ollama Cloud model
// name (e.g. "qwen3.5:cloud", "qwen3-coder:480b"). Returns (0, false)
// on miss. Lookup never makes HTTP calls — it's the hot path consulted
// by the executor before every run.
//
// modelName accepts either the bare model:tag form or the prefixed
// "ollama-cloud/model:tag" form; the prefix is stripped.
func (c *CloudOllamaLimitCache) Lookup(modelName string) (int, bool) {
if c == nil {
return 0, false
}
key := stripCloudPrefix(modelName)
if key == "" {
return 0, false
}
c.mu.RLock()
defer c.mu.RUnlock()
v, ok := c.limit[key]
return v, ok
}
// Size returns the number of cached entries. Useful for logging /
// health checks.
func (c *CloudOllamaLimitCache) Size() int {
if c == nil {
return 0
}
c.mu.RLock()
defer c.mu.RUnlock()
return len(c.limit)
}
// LookupOrFetch returns the cached context length OR, on miss, makes a
// single /api/show call to populate the cache. Negative results
// (model not found, /api/show returns no context_length) are cached
// for negativeTTL to prevent hammering Cloud on a typo. Returns
// (0, false) when the model is genuinely unknown and (size, true) on
// any successful resolve.
//
// Why this exists: Ollama Cloud's /api/tags lists canonical model
// names only (e.g. "qwen3.5:397b") but accepts aliases on /api/show
// (e.g. "qwen3.5:cloud" → same 397b model). The boot-time RefreshAll
// only sees the canonical names, so common aliases miss the cache.
// LookupOrFetch fills the gap.
//
// The cache is therefore self-healing: any unknown model gets one
// live /api/show call, the result lands in the cache, and subsequent
// runs hit immediately. Periodic RefreshAll overwrites everything
// with the canonical-name results but additionally-fetched aliases
// linger as positive entries.
func (c *CloudOllamaLimitCache) LookupOrFetch(ctx context.Context, modelName string) (int, bool) {
if c == nil {
return 0, false
}
key := stripCloudPrefix(modelName)
if key == "" {
return 0, false
}
// Fast path: positive hit.
c.mu.RLock()
if v, ok := c.limit[key]; ok {
c.mu.RUnlock()
return v, true
}
// Negative cache check.
if t, ok := c.negative[key]; ok && time.Since(t) < c.negativeTTL {
c.mu.RUnlock()
return 0, false
}
c.mu.RUnlock()
// No API key configured → can't fetch. Don't write a negative
// entry (when the key gets configured later we want the next call
// to re-try immediately).
if strings.TrimSpace(c.apiKey) == "" {
return 0, false
}
// Slow path: live /api/show.
n, err := c.fetchContextLength(ctx, key)
if err != nil || n <= 0 {
slog.Debug("cloud limit cache: lazy fetch miss",
"model", key, "err", err)
c.mu.Lock()
c.negative[key] = time.Now()
c.mu.Unlock()
return 0, false
}
c.set(key, n)
slog.Info("cloud limit cache: lazy fetch hit", "model", key, "context_length", n)
return n, true
}
// set stores a context length. n <= 0 is a no-op.
func (c *CloudOllamaLimitCache) set(modelName string, n int) {
if c == nil || n <= 0 {
return
}
key := stripCloudPrefix(modelName)
if key == "" {
return
}
c.mu.Lock()
c.limit[key] = n
c.mu.Unlock()
}
// RefreshAll queries /api/tags then concurrently calls /api/show for
// every listed model, populating the cache. Returns the number of
// models successfully cached and the first error encountered (a
// /api/tags failure aborts; individual /api/show failures are logged
// but don't abort the whole refresh).
//
// Safe to call repeatedly. Cache entries are overwritten with the
// fresh values; entries for models that have been removed from Cloud
// are NOT pruned (cheap to keep; pruning risks dropping an entry just
// before a run that needs it).
func (c *CloudOllamaLimitCache) RefreshAll(ctx context.Context) (int, error) {
if c == nil {
return 0, fmt.Errorf("cloud limit cache: nil receiver")
}
if strings.TrimSpace(c.apiKey) == "" {
return 0, fmt.Errorf("cloud limit cache: OLLAMA_API_KEY unset")
}
tags, err := c.fetchTags(ctx)
if err != nil {
return 0, fmt.Errorf("cloud limit cache: /api/tags: %w", err)
}
concurrency := c.refreshConcurrency
if concurrency <= 0 {
concurrency = 8
}
sem := make(chan struct{}, concurrency)
var wg sync.WaitGroup
var (
mu sync.Mutex
success int
)
for _, name := range tags {
name := name
wg.Add(1)
sem <- struct{}{}
go func() {
defer wg.Done()
defer func() { <-sem }()
ctxLen, ferr := c.fetchContextLength(ctx, name)
if ferr != nil {
slog.Debug("cloud limit cache: /api/show miss",
"model", name, "err", ferr)
return
}
c.set(name, ctxLen)
mu.Lock()
success++
mu.Unlock()
}()
}
wg.Wait()
slog.Info("cloud limit cache: refresh complete",
"models_total", len(tags), "cached", success)
return success, nil
}
// StartPeriodicRefresh runs RefreshAll once immediately, then on every
// interval tick. Cancellation via ctx stops the loop. Logs each
// outcome; never returns an error to the caller (this is a background
// task — failures are warnings, not show-stoppers).
//
// Typical usage: a goroutine spawned at mort boot.
//
// go cache.StartPeriodicRefresh(ctx, 24*time.Hour)
func (c *CloudOllamaLimitCache) StartPeriodicRefresh(ctx context.Context, interval time.Duration) {
if c == nil {
return
}
if interval <= 0 {
interval = 24 * time.Hour
}
doOne := func() {
n, err := c.RefreshAll(ctx)
if err != nil {
slog.Warn("cloud limit cache: refresh failed",
"err", err, "cached_size", c.Size())
return
}
slog.Info("cloud limit cache: refreshed",
"newly_cached_or_updated", n, "cached_size", c.Size())
}
doOne()
t := time.NewTicker(interval)
defer t.Stop()
for {
select {
case <-ctx.Done():
return
case <-t.C:
doOne()
}
}
}
// fetchTags calls GET /api/tags and returns the model names.
func (c *CloudOllamaLimitCache) fetchTags(ctx context.Context) ([]string, error) {
url := c.endpoint + "/api/tags"
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return nil, err
}
c.applyAuth(req)
resp, err := c.httpClient.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, err := readCapped(resp.Body)
if err != nil {
return nil, err
}
if resp.StatusCode/100 != 2 {
return nil, fmt.Errorf("status %d: %s", resp.StatusCode, truncate(body, 400))
}
var parsed struct {
Models []struct {
Name string `json:"name"`
Model string `json:"model"`
} `json:"models"`
}
if err := json.Unmarshal(body, &parsed); err != nil {
return nil, fmt.Errorf("parse /api/tags: %w", err)
}
out := make([]string, 0, len(parsed.Models))
for _, m := range parsed.Models {
name := m.Name
if name == "" {
name = m.Model
}
if name == "" {
continue
}
out = append(out, name)
}
return out, nil
}
// fetchContextLength calls POST /api/show for a model and extracts
// the largest *.context_length value from model_info. Returns the
// length and nil on success; (0, err) on any failure.
//
// Why "largest" rather than family-keyed: the family field in the
// /api/show response is sometimes empty or doesn't match the
// model_info key prefix exactly (Ollama Cloud returns the
// architecture as the prefix, which usually but not always matches
// `family`). Scanning for any `*.context_length` is robust.
func (c *CloudOllamaLimitCache) fetchContextLength(ctx context.Context, modelName string) (int, error) {
url := c.endpoint + "/api/show"
body, _ := json.Marshal(map[string]string{"name": modelName})
req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, bytes.NewReader(body))
if err != nil {
return 0, err
}
req.Header.Set("Content-Type", "application/json")
c.applyAuth(req)
resp, err := c.httpClient.Do(req)
if err != nil {
return 0, err
}
defer resp.Body.Close()
respBody, err := readCapped(resp.Body)
if err != nil {
return 0, err
}
if resp.StatusCode/100 != 2 {
return 0, fmt.Errorf("status %d: %s", resp.StatusCode, truncate(respBody, 400))
}
n, err := parseContextLengthJSON(respBody)
if err != nil {
return 0, err
}
return n, nil
}
// parseContextLengthJSON extracts the largest `*.context_length` int
// from an /api/show response body. Exported-ish (lowercase but tested
// in the same package) so the unit test can exercise it without
// spinning up an httptest server.
func parseContextLengthJSON(body []byte) (int, error) {
var parsed struct {
ModelInfo map[string]any `json:"model_info"`
}
if err := json.Unmarshal(body, &parsed); err != nil {
return 0, fmt.Errorf("parse: %w", err)
}
best := 0
for k, v := range parsed.ModelInfo {
if !strings.HasSuffix(k, ".context_length") {
continue
}
n := toInt(v)
if n > best {
best = n
}
}
if best <= 0 {
return 0, fmt.Errorf("no context_length in model_info")
}
return best, nil
}
// toInt coerces a JSON-decoded value to int. Handles float64 (the
// json default) and json.Number; returns 0 for anything else.
func toInt(v any) int {
switch x := v.(type) {
case float64:
return int(x)
case int:
return x
case int64:
return int(x)
case json.Number:
if n, err := x.Int64(); err == nil {
return int(n)
}
}
return 0
}
// applyAuth sets the Bearer token when an API key is configured.
func (c *CloudOllamaLimitCache) applyAuth(req *http.Request) {
if strings.TrimSpace(c.apiKey) == "" {
return
}
req.Header.Set("Authorization", "Bearer "+strings.TrimSpace(c.apiKey))
}
// stripCloudPrefix strips an "ollama-cloud/" prefix (and surrounding
// whitespace). Returns the bare model:tag form.
func stripCloudPrefix(s string) string {
s = strings.TrimSpace(s)
if strings.HasPrefix(s, "ollama-cloud/") {
s = s[len("ollama-cloud/"):]
}
return s
}
// truncate caps a byte slice for error messages.
func truncate(b []byte, n int) string {
if len(b) <= n {
return string(b)
}
return string(b[:n]) + "...(truncated)"
}
// maxLimitCacheResponseBytes bounds the ollama.com limit-cache HTTP responses
// (/api/tags, /api/show) so a misbehaving endpoint can't stream an unbounded
// body before the 15s timeout fires. 1 MiB is far above any real response.
const maxLimitCacheResponseBytes = 1 << 20
// readCapped reads up to maxLimitCacheResponseBytes from r and returns a clear
// error if the response EXCEEDS the cap — rather than silently truncating (as a
// bare io.LimitReader does) and letting downstream json.Unmarshal fail with an
// opaque "unexpected end of JSON input". It reads one extra byte to detect the
// overflow.
func readCapped(r io.Reader) ([]byte, error) {
body, err := io.ReadAll(io.LimitReader(r, maxLimitCacheResponseBytes+1))
if err != nil {
return nil, err
}
if len(body) > maxLimitCacheResponseBytes {
return nil, fmt.Errorf("cloud_sync: response exceeded %d bytes", maxLimitCacheResponseBytes)
}
return body, nil
}
+224
View File
@@ -0,0 +1,224 @@
// V15.2 — per-model context-window limits.
//
// Why: agents need to know when they're about to blow the model's
// max-input cap so they can compact stale tool results out of the
// message history. Pre-15.2 the agent loop had no awareness; a long
// research run that accumulated dozens of HTTP tool results would
// hit Ollama's HTTP 400 "prompt is too long" or Anthropic's similar
// error mid-run with no graceful degradation.
//
// Coverage:
// - Anthropic Claude 4.x (200K default; 1M when the model ID
// includes the "[1m]" suffix per llms.tier reload conventions)
// - OpenAI GPT-4.x / o-series (128K)
// - Gemini 2.x (1M-2M, model-specific)
// - Ollama Cloud (model-specific; hardcoded per known model)
// - Local Ollama: queries `/api/show` once at first use, caches
//
// Returns (0, false) for unknown models — callers should treat
// "unknown" as "don't budget" (the agent's existing iteration cap +
// timeout are the fallback safety nets).
package model
import (
"context"
"strings"
"sync"
)
// MaxContextTokens returns the model's max INPUT context-window size
// in tokens. Output / response tokens are NOT included — most models
// share input + output budget but cap them separately, and the practical
// concern is "how big can my prompt get before the model rejects".
//
// modelID accepts both the bare model name (`claude-sonnet-4-6`) and
// the prefixed form (`anthropic/claude-sonnet-4-6` or
// `ollama-cloud/qwen3-coder:480b`). The prefix is stripped before lookup.
//
// Returns (limit, true) on a known model; (0, false) otherwise.
//
// This function is pure (no I/O). For Ollama Cloud models that aren't
// in the static map, use MaxContextTokensWithCache which consults a
// CloudOllamaLimitCache populated at boot from /api/tags + /api/show.
func MaxContextTokens(modelID string) (int, bool) {
id := normalizeModelID(modelID)
if v, ok := staticContextLimits[id]; ok {
return v, true
}
// Anthropic 1M-context variant marker. Mort's llms tier system
// uses a `[1m]` suffix on the model ID (e.g.
// `claude-opus-4-7[1m]`) to opt into Anthropic's 1M beta context.
if strings.HasSuffix(id, "[1m]") {
return 1_000_000, true
}
// Local-ollama dynamic lookup is wired separately so it can
// query the daemon's /api/show endpoint. The static map covers
// known cloud models.
return 0, false
}
// MaxContextTokensWithCache is the cache-aware variant of
// MaxContextTokens. It tries the static map first; on miss, if the
// model is an Ollama Cloud spec (the `ollama-cloud/` prefix), it
// consults the supplied CloudOllamaLimitCache. Pass nil cache for
// static-only behaviour (equivalent to MaxContextTokens).
//
// This function never makes HTTP calls — the cache must be
// pre-populated (typically via cache.RefreshAll at boot). Callers in
// the hot path can rely on a single map lookup per call. Prefer
// MaxContextTokensResolving when a context is available — it makes a
// single /api/show call to fill the cache on miss, which is essential
// for Cloud aliases that /api/tags doesn't enumerate (e.g. :cloud).
func MaxContextTokensWithCache(modelID string, cloud *CloudOllamaLimitCache) (int, bool) {
if v, ok := MaxContextTokens(modelID); ok {
return v, true
}
if cloud == nil {
return 0, false
}
// Only ollama-cloud/* models are eligible for the cache.
id := strings.TrimSpace(modelID)
if !strings.HasPrefix(id, "ollama-cloud/") {
// Also allow bare model:tag form when the caller has already
// stripped the prefix (some test paths).
if strings.Contains(id, "/") {
return 0, false
}
}
return cloud.Lookup(id)
}
// MaxContextTokensResolving is the cache-aware variant that ALSO
// performs a live /api/show fetch on cache miss (with negative caching
// to prevent thrash). Use this in run-setup paths where one HTTP call
// per unseen model is acceptable — typically the skill executor's
// compaction threshold computation. The fetched result is cached for
// future calls, so subsequent runs hit the in-memory map.
//
// Falls back to the static-only path when the model isn't an
// ollama-cloud/* spec or cache is nil. ctx cancellation aborts the
// fetch and returns (0, false) without writing a negative entry.
func MaxContextTokensResolving(ctx context.Context, modelID string, cloud *CloudOllamaLimitCache) (int, bool) {
if v, ok := MaxContextTokens(modelID); ok {
return v, true
}
if cloud == nil {
return 0, false
}
id := strings.TrimSpace(modelID)
if !strings.HasPrefix(id, "ollama-cloud/") {
if strings.Contains(id, "/") {
return 0, false
}
}
return cloud.LookupOrFetch(ctx, id)
}
// normalizeModelID strips provider prefix and reasoning suffix so a
// lookup keyed on the base name works regardless of caller form.
//
// Examples:
// - "anthropic/claude-sonnet-4-6" → "claude-sonnet-4-6"
// - "ollama-cloud/qwen3-coder:480b" → "qwen3-coder:480b"
// - "claude-opus-4-7:high" → "claude-opus-4-7"
func normalizeModelID(id string) string {
id = strings.TrimSpace(id)
if idx := strings.Index(id, "/"); idx >= 0 {
id = id[idx+1:]
}
// Strip :low/:medium/:high reasoning effort suffix used by some
// OpenAI / Anthropic clients.
for _, suffix := range []string{":low", ":medium", ":high"} {
if strings.HasSuffix(id, suffix) {
id = id[:len(id)-len(suffix)]
break
}
}
return id
}
// staticContextLimits is the source of truth for known cloud models.
// Add new entries when adding a model to the llms tier system.
//
// CRITICAL: keep these in sync with the actual provider docs. A wrong
// number here causes EITHER premature compaction (too low, degrades
// agent quality unnecessarily) OR HTTP 400 mid-run (too high). The
// 410K-token failure on `qwen3-coder:480b` is the kind of bug a
// mistyped value would reintroduce.
var staticContextLimits = map[string]int{
// Anthropic Claude 4.x — default 200K input. 1M variant via
// `[1m]` suffix handled in MaxContextTokens above.
"claude-opus-4-7": 200_000,
"claude-opus-4-6": 200_000,
"claude-opus-4-5": 200_000,
"claude-sonnet-4-6": 200_000,
"claude-sonnet-4-5": 200_000,
"claude-haiku-4-5": 200_000,
"claude-haiku-4-5-20251001": 200_000,
// OpenAI GPT-4.x / o-series — 128K input.
"gpt-4o": 128_000,
"gpt-4o-mini": 128_000,
"gpt-4-turbo": 128_000,
"o1": 200_000,
"o1-mini": 128_000,
"o3-mini": 200_000,
"gpt-5": 400_000,
"gpt-5-mini": 400_000,
// Gemini — varies dramatically by model.
"gemini-2.5-pro": 2_000_000,
"gemini-2.5-flash": 1_000_000,
"gemini-2.5-flash-lite": 1_000_000,
"gemini-1.5-pro": 2_000_000,
"gemini-1.5-flash": 1_000_000,
// Ollama Cloud (turbo). Limits per https://ollama.com/cloud/models
// — verified against the Ollama API show output for each model.
// Update when Ollama publishes new models or extends contexts.
"qwen3-coder:480b": 262_144, // 262K — matches the v15.2 trace
"qwen3:235b": 262_144,
"qwen3:32b": 131_072,
"qwen2.5:72b": 131_072,
"gpt-oss:120b": 131_072,
"gpt-oss:20b": 131_072,
"deepseek-v3.1:671b": 131_072,
"glm-4.6:355b": 131_072,
"kimi-k2:1t": 262_144,
"llama4:scout": 10_000_000, // Llama 4 Scout claims 10M
"llama4:maverick": 1_000_000,
}
// LocalOllamaLimitCache holds the resolved /api/show context_length per
// local-ollama model. Populated on first lookup; never invalidated
// (changing num_ctx requires an ollama restart anyway). Process-wide,
// no per-tenant scoping needed.
type LocalOllamaLimitCache struct {
mu sync.RWMutex
limit map[string]int
}
// NewLocalOllamaLimitCache constructs a fresh cache.
func NewLocalOllamaLimitCache() *LocalOllamaLimitCache {
return &LocalOllamaLimitCache{limit: make(map[string]int)}
}
// Get returns the cached limit or (0, false) when unseen. The caller
// is expected to follow up with a lookup against the live daemon.
func (c *LocalOllamaLimitCache) Get(model string) (int, bool) {
c.mu.RLock()
defer c.mu.RUnlock()
v, ok := c.limit[model]
return v, ok
}
// Set records a resolved limit. Idempotent; no-op when value is <= 0.
func (c *LocalOllamaLimitCache) Set(model string, n int) {
if n <= 0 {
return
}
c.mu.Lock()
defer c.mu.Unlock()
c.limit[model] = n
}
+97
View File
@@ -0,0 +1,97 @@
package model
import (
"testing"
"time"
)
// mapSource is a tiny config.Source for tests: a key->value map, defaults
// returned for misses.
type mapSource map[string]string
func (m mapSource) String(k, d string) string {
if v, ok := m[k]; ok {
return v
}
return d
}
func (m mapSource) Int(string, int) int { panic("unused") }
func (m mapSource) Float(string, float64) float64 { panic("unused") }
func (m mapSource) Bool(string, bool) bool { panic("unused") }
// TestConfigureTierResolution covers the convar->config.Source inversion: the
// host supplies a tier table (names + fallbacks) and a live config source; the
// config value overrides the fallback, and an absent key falls back.
func TestConfigureTierResolution(t *testing.T) {
Configure(
mapSource{"model.tier.fast": "anthropic/claude-haiku-4-5"},
map[string]string{"fast": "openai/gpt-4o-mini", "thinking": "anthropic/claude-opus-4-8"},
time.Minute,
)
defer Configure(nil, nil, 0) // reset package global
if !IsTierName("fast") || !IsTierName("thinking") {
t.Fatal("configured tiers should be registered")
}
if IsTierName("nope") {
t.Fatal("unknown tier must not report as a tier")
}
if names := TierNames(); len(names) != 2 || names[0] != "fast" || names[1] != "thinking" {
t.Fatalf("TierNames = %v, want sorted [fast thinking]", names)
}
// config value overrides the host fallback
if spec, _, ok := defaultResolver.Resolve("fast"); !ok || spec != "anthropic/claude-haiku-4-5" {
t.Fatalf("fast resolve = %q ok=%v; config should override fallback", spec, ok)
}
// fallback used when config has no override for the key
if spec, _, ok := defaultResolver.Resolve("thinking"); !ok || spec != "anthropic/claude-opus-4-8" {
t.Fatalf("thinking resolve = %q ok=%v; should use fallback", spec, ok)
}
// unknown tier
if _, _, ok := defaultResolver.Resolve("nope"); ok {
t.Fatal("Resolve of unknown tier should be ok=false")
}
}
// TestReasoningSuffixOnTier verifies the reasoning-suffix dialect survives the
// move: a tier whose spec carries ":high" yields the bare spec + level "high".
func TestReasoningSuffixOnTier(t *testing.T) {
Configure(nil, map[string]string{"thinking": "anthropic/claude-opus-4-8:high"}, time.Minute)
defer Configure(nil, nil, 0)
spec, level, ok := defaultResolver.Resolve("thinking")
if !ok {
t.Fatal("thinking should resolve")
}
if spec != "anthropic/claude-opus-4-8" {
t.Errorf("spec = %q, want suffix stripped", spec)
}
if level != "high" {
t.Errorf("reasoning level = %q, want high", level)
}
}
func TestValidateTierValueRejectsNestedTier(t *testing.T) {
Configure(nil, map[string]string{"fast": "x/y"}, time.Minute)
defer Configure(nil, nil, 0)
if err := ValidateTierValue("fast,a/b"); err == nil {
t.Error("a chain containing a tier alias must be rejected")
}
if err := ValidateTierValue("a/b,c/d"); err != nil {
t.Errorf("a chain of concrete specs must validate, got %v", err)
}
}
// TestSinksDefaultNil verifies usage/trace recording is inert with no sinks
// installed (the light-host default).
func TestSinksDefaultNil(t *testing.T) {
SetUsageSink(nil)
SetTraceSink(nil)
if TraceSinkActive() {
t.Error("no trace sink should mean inactive")
}
// recordUsage must be a no-op (no panic) with a nil sink.
recordUsage(WithModel(t.Context(), "x"), nil)
}
+91
View File
@@ -0,0 +1,91 @@
// Package llms — lane_mapping.go: maps a model spec to a stable lane
// name. Pure data + a single function; no dependency on the registry,
// no provider wrapping. Kept separate from lane_transport.go so the
// mapping table can be committed and reviewed in isolation, and so
// admin / webui code that just wants to *display* lane assignments
// doesn't drag in the transport machinery.
//
// Why a fixed table: provider concurrency caps differ — Ollama Pro is
// 3 connections, Anthropic Claude has higher per-tier limits, etc.
// Each provider gets its own lane name so they can be configured
// independently via convars (lanes.<name>.max_concurrent). Lane names
// are user-facing (admin dashboard + convar key suffixes) and need to
// stay stable across deploys; an env-overridable map adds complexity
// for no current benefit.
//
// Test: lane_transport_test.go covers TestLaneFor_Mapping.
package model
import "strings"
// Lane name constants. Defined as exported strings so admin code (.skill
// admin set-lane <skill> <lane>), webui dropdowns, and convar consumers
// share a single canonical spelling.
const (
// LaneOllama covers ollama-cloud/* (and any future ollama/* local).
// The local ollama instance is on the same physical resource as
// the cloud account from mort's perspective — the connection cap
// should apply jointly.
LaneOllama = "ollama"
// LaneAnthropicThinking is the lane for Anthropic models in
// extended-thinking mode. Separated from default because thinking
// requests hold connections longer and can starve faster lanes
// when multiplexed.
LaneAnthropicThinking = "anthropic-thinking"
// LaneAnthropicDefault is the lane for non-thinking Anthropic
// requests (haiku, sonnet, opus without -thinking-).
LaneAnthropicDefault = "anthropic-default"
// LaneM1 is the lane for m1/* models (foreman-style router
// pointing at a dedicated local instance). Separated from the
// ollama lane because m1 targets a distinct host with its own
// connection budget.
LaneM1 = "m1"
// LaneLLMDefault is the catch-all lane for any provider/model
// combination not explicitly mapped above.
LaneLLMDefault = "llm-default"
)
// LaneFor returns the lane name for the given model spec. Mapping:
//
// ollama-cloud/* → "ollama" (Pro account: 3 connections)
// anthropic/*-thinking-* → "anthropic-thinking"
// anthropic/* → "anthropic-default"
// (anything else) → "llm-default"
//
// Tier aliases (fast/standard/thinking) flow through this function as
// the resolver's expanded provider/model spec, so callers don't need
// to think about tier indirection. Empty input falls through to
// LaneLLMDefault rather than panicking — defensive against unset
// model specs in edge-case test wiring.
//
// Substring match for "-thinking-" keeps future Anthropic naming
// variations classified correctly without churning this table on
// every model release.
func LaneFor(modelSpec string) string {
s := strings.TrimSpace(modelSpec)
if strings.HasPrefix(s, "ollama-cloud/") {
return LaneOllama
}
if strings.HasPrefix(s, "anthropic/") {
if strings.Contains(s, "-thinking-") {
return LaneAnthropicThinking
}
return LaneAnthropicDefault
}
// Foreman instances are backed by Ollama and share its connection
// cap, so they route to the same lane.
if strings.HasPrefix(s, "foreman/") {
return LaneOllama
}
// m1/ is a foreman-style router pointing at a dedicated local
// instance with its own connection budget. Separate lane so its
// concurrency cap is independent of the shared ollama lane.
if strings.HasPrefix(s, "m1/") {
return LaneM1
}
return LaneLLMDefault
}
+373
View File
@@ -0,0 +1,373 @@
// Package llms — lane_transport.go: the lane-aware decorator. Wraps an
// llm.Provider so every model it mints submits its Generate/Stream calls
// through the matching named lane's bounded worker pool (lane selection
// per lane_mapping.go), and stamps every returned error with per-call
// attribution (caller id, run id, prompt snapshot) for the failover log.
//
// Why intercept at the llm.Provider layer: majordomo's Provider and Model
// are small public interfaces, so the decorator slots between the chain
// executor and the real provider with no fork. Every chain attempt calls
// laneModel.Generate, which queues on the lane, runs the real call, and
// wraps failures with CallInfo — the ChainConfig.Observer (which receives
// no context) recovers the attribution from the error itself.
//
// Test: lane_transport_test.go covers mapping correctness, the
// concurrency-limiting behavior, and error attribution.
// lane_chatbot_test.go is the regression guard proving chatbot-path LLM
// calls actually go through the lane.
package model
import (
"context"
"errors"
"time"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"github.com/google/uuid"
"gitea.stevedudenhoeffer.com/steve/executus/lane"
)
// defaultLaneExecTimeout is the execution backstop applied inside a lane
// job once it leaves the queue: the caller's deadline is detached (queue
// wait must not consume the LLM execution budget) and replaced with this
// hard cap so a hung provider can't leak workers.
const defaultLaneExecTimeout = 5 * time.Minute
// foremanModelTimeout is the hard per-call timeout for foreman targets —
// slow local LLMs that may block on model loads and upstream queues.
const foremanModelTimeout = 30 * time.Minute
// foremanLaneExecTimeout is the lane execution backstop for foreman
// targets. Slightly above foremanModelTimeout so the model-level timeout
// (the documented contract) is the one that fires.
const foremanLaneExecTimeout = foremanModelTimeout + time.Minute
// laneCallerKey is the context key for the per-call caller identity used
// for fair-share queueing.
type laneCallerKey struct{}
// runIDKey is the context key for the per-call run id used for failover
// event attribution.
type runIDKey struct{}
// ContextWithLaneCaller attaches a caller identity to ctx. The lane
// decorator reads this when constructing a Job so fair-share queueing
// can isolate heavy users, and snapshots it into error attribution for
// the failover log.
//
// Empty string is a no-op and lumps every empty-caller invocation into a
// single fair-share bucket; production callers should always populate it.
func ContextWithLaneCaller(ctx context.Context, callerID string) context.Context {
if callerID == "" {
return ctx
}
return context.WithValue(ctx, laneCallerKey{}, callerID)
}
// LaneCallerFromContext returns the caller identity attached via
// ContextWithLaneCaller, or "" if none is set.
func LaneCallerFromContext(ctx context.Context) string {
s, _ := ctx.Value(laneCallerKey{}).(string)
return s
}
// ContextWithRunID attaches a skill/agent run id to ctx. Snapshotted into
// error attribution so failover events can be correlated to runs.
func ContextWithRunID(ctx context.Context, runID string) context.Context {
if runID == "" {
return ctx
}
return context.WithValue(ctx, runIDKey{}, runID)
}
// RunIDFromContext returns the run id attached via ContextWithRunID, or
// "" if none is set.
func RunIDFromContext(ctx context.Context) string {
s, _ := ctx.Value(runIDKey{}).(string)
return s
}
// ---------------------------------------------------------------------------
// Error attribution
// ---------------------------------------------------------------------------
// CallInfo is the per-call attribution snapshot the lane decorator stamps
// onto every error it returns. majordomo's ChainConfig.Observer receives
// a bare FailoverEvent (no context); the failover log recovers caller,
// run id, and the prompt chain from the event's error via
// CallInfoFromError.
type CallInfo struct {
// CallerID is the fair-share caller identity (ContextWithLaneCaller).
CallerID string
// RunID is the skill/agent run id (ContextWithRunID); "" if not threaded.
RunID string
// Messages is the request's message chain at call time, for the
// failover log's persist_prompts feature.
Messages []llm.Message
}
// callInfoError carries CallInfo along an error chain without changing
// the error's message or classification (Unwrap preserves errors.Is/As).
type callInfoError struct {
inner error
info CallInfo
}
func (e *callInfoError) Error() string { return e.inner.Error() }
func (e *callInfoError) Unwrap() error { return e.inner }
// WithCallInfo stamps attribution onto err. nil err returns nil.
func WithCallInfo(err error, info CallInfo) error {
if err == nil {
return nil
}
return &callInfoError{inner: err, info: info}
}
// CallInfoFromError extracts the attribution stamped by the lane
// decorator (or WithCallInfo), if any.
func CallInfoFromError(err error) (CallInfo, bool) {
var cie *callInfoError
if errors.As(err, &cie) {
return cie.info, true
}
return CallInfo{}, false
}
// ---------------------------------------------------------------------------
// Lane decoration
// ---------------------------------------------------------------------------
// LaneRegistry is the narrow surface the lane decorator needs from
// pkg/lane.Registry. Defined as an interface so tests can substitute a
// fake registry without spinning up a real one.
type LaneRegistry interface {
GetOrCreate(ctx context.Context, name string) lane.Lane
}
// laneProvider decorates an llm.Provider so every model it mints routes
// calls through the lane named by LaneFor(provider/model). With a nil
// registry the queueing is skipped but error attribution still applies.
type laneProvider struct {
inner llm.Provider
registry LaneRegistry
execTimeout time.Duration
}
// WrapProviderForLane returns a provider whose models submit each
// Generate/Stream call through the lane named by LaneFor(name/model) in
// the registry, and stamp CallInfo attribution onto every error.
//
// A nil registry disables queueing (calls pass straight through) but the
// decoration — and with it error attribution — remains, so failover
// logging works in lane-less deployments and tests.
func WrapProviderForLane(inner llm.Provider, registry LaneRegistry) llm.Provider {
return wrapProviderForLane(inner, registry, defaultLaneExecTimeout)
}
func wrapProviderForLane(inner llm.Provider, registry LaneRegistry, execTimeout time.Duration) llm.Provider {
if inner == nil {
return nil
}
if execTimeout <= 0 {
execTimeout = defaultLaneExecTimeout
}
return &laneProvider{inner: inner, registry: registry, execTimeout: execTimeout}
}
func (p *laneProvider) Name() string { return p.inner.Name() }
func (p *laneProvider) Model(id string, opts ...llm.ModelOption) (llm.Model, error) {
m, err := p.inner.Model(id, opts...)
if err != nil {
return nil, err
}
return &laneModel{
inner: m,
registry: p.registry,
laneName: LaneFor(p.inner.Name() + "/" + id),
execTimeout: p.execTimeout,
}, nil
}
// laneModel routes one model's calls through its lane and stamps error
// attribution. The lane name is resolved once at Model() time — the
// provider name and model id are both known there, unlike legacy gollm where
// the request had to be inspected per call.
type laneModel struct {
inner llm.Model
registry LaneRegistry
laneName string
execTimeout time.Duration
}
func (m *laneModel) Capabilities() llm.Capabilities { return m.inner.Capabilities() }
// laneJob adapts an in-flight call to the lane.Job interface. The result
// is captured into the struct and read after SubmitWait returns.
type laneJob struct {
id string
callerID string
run func(ctx context.Context) error
}
func (j *laneJob) ID() string { return j.id }
func (j *laneJob) CallerID() string { return j.callerID }
func (j *laneJob) Priority() int { return 0 }
func (j *laneJob) Run(ctx context.Context) error { return j.run(ctx) }
func (m *laneModel) Generate(ctx context.Context, req llm.Request, opts ...llm.Option) (*llm.Response, error) {
// Fold options now so the job closure and the attribution snapshot
// both see the final request.
req = req.Apply(opts...)
info := CallInfo{
CallerID: LaneCallerFromContext(ctx),
RunID: RunIDFromContext(ctx),
Messages: req.Messages,
}
resp, err := m.submit(ctx, func(execCtx context.Context) (*llm.Response, error) {
return m.inner.Generate(execCtx, req)
})
if err != nil {
return resp, WithCallInfo(err, info)
}
return resp, nil
}
func (m *laneModel) Stream(ctx context.Context, req llm.Request, opts ...llm.Option) (llm.Stream, error) {
req = req.Apply(opts...)
info := CallInfo{
CallerID: LaneCallerFromContext(ctx),
RunID: RunIDFromContext(ctx),
Messages: req.Messages,
}
l := m.lane(ctx)
if l == nil {
s, err := m.inner.Stream(ctx, req)
if err != nil {
return nil, WithCallInfo(err, info)
}
return s, nil
}
// Streams hold their lane slot only while ESTABLISHING the stream —
// holding it for the full consumption would deadlock a slow consumer
// against the pool. The caller's ctx is used as-is (no deadline
// detach): severing cancellation from a long-lived stream would leak
// connections.
var (
stream llm.Stream
serr error
)
job := &laneJob{
id: uuid.New().String(),
callerID: info.CallerID,
run: func(context.Context) error {
stream, serr = m.inner.Stream(ctx, req)
return serr
},
}
if err := l.SubmitWait(ctx, job); err != nil {
return nil, WithCallInfo(err, info)
}
if serr != nil {
return nil, WithCallInfo(serr, info)
}
return stream, nil
}
// lane resolves the lane for this model, or nil when queueing is
// disabled (nil registry, or a registry that declines the name).
func (m *laneModel) lane(ctx context.Context) lane.Lane {
if m.registry == nil {
return nil
}
return m.registry.GetOrCreate(ctx, m.laneName)
}
// submit runs fn through the lane (or directly when queueing is off).
//
// Inside a lane job the caller's deadline is detached so queue wait does
// not consume the execution budget — ctx VALUES (usage attribution,
// trace ids) are preserved, only cancellation/deadline are severed — and
// an execTimeout backstop prevents runaway calls. Queue-phase
// cancellation still works: SubmitWait waits on the original ctx, so a
// caller that gives up while queued exits immediately.
func (m *laneModel) submit(ctx context.Context, fn func(context.Context) (*llm.Response, error)) (*llm.Response, error) {
l := m.lane(ctx)
if l == nil {
return fn(ctx)
}
var (
resp *llm.Response
err error
)
job := &laneJob{
id: uuid.New().String(),
callerID: LaneCallerFromContext(ctx),
run: func(context.Context) error {
execCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), m.execTimeout)
defer cancel()
resp, err = fn(execCtx)
// Returning err lets the lane's pool propagate it to
// SubmitWait; the captured err is what we surface.
return err
},
}
if serr := l.SubmitWait(ctx, job); serr != nil && err == nil {
return nil, serr
}
return resp, err
}
// ---------------------------------------------------------------------------
// Model timeout decoration (foreman)
// ---------------------------------------------------------------------------
// timeoutProvider wraps a provider so every minted model enforces a hard
// per-call deadline on Generate. Used for foreman targets (slow local
// LLMs). Stream is passed through: a wall-clock deadline on a long-lived
// stream would sever it mid-consumption.
type timeoutProvider struct {
inner llm.Provider
timeout time.Duration
}
// withModelTimeout decorates p so its models' Generate calls carry a
// hard timeout.
func withModelTimeout(p llm.Provider, d time.Duration) llm.Provider {
if p == nil || d <= 0 {
return p
}
return &timeoutProvider{inner: p, timeout: d}
}
func (p *timeoutProvider) Name() string { return p.inner.Name() }
func (p *timeoutProvider) Model(id string, opts ...llm.ModelOption) (llm.Model, error) {
m, err := p.inner.Model(id, opts...)
if err != nil {
return nil, err
}
return &timeoutModel{inner: m, timeout: p.timeout}, nil
}
type timeoutModel struct {
inner llm.Model
timeout time.Duration
}
func (m *timeoutModel) Capabilities() llm.Capabilities { return m.inner.Capabilities() }
func (m *timeoutModel) Generate(ctx context.Context, req llm.Request, opts ...llm.Option) (*llm.Response, error) {
ctx, cancel := context.WithTimeout(ctx, m.timeout)
defer cancel()
return m.inner.Generate(ctx, req, opts...)
}
func (m *timeoutModel) Stream(ctx context.Context, req llm.Request, opts ...llm.Option) (llm.Stream, error) {
return m.inner.Stream(ctx, req, opts...)
}
+477
View File
@@ -0,0 +1,477 @@
// Package model is executus's config-driven model-access layer over majordomo: it owns the
// package-level *majordomo.Registry (providers with mort's env keys,
// OpenAI-compat presets, lane-aware decoration, the DB-backed tier
// resolver, legacy shortcut aliases, the foreman timeout decorator, and
// failover/health wiring), plus the mort-facing call helpers
// (ParseModelRequest / ParseModelForContext / GenerateWith /
// CallAndExecute / SimpleCall) and usage/trace recording.
//
// The ":low/:medium/:high" reasoning-suffix dialect is an executus convenience:
// majordomo treats model ids as verbatim, so this package strips the
// suffix from specs and tier values and re-applies it per request via
// llm.WithReasoningEffort on a wrapping Model.
package model
import (
"context"
"fmt"
"os"
"strings"
"sync"
"time"
majordomo "gitea.stevedudenhoeffer.com/steve/majordomo"
"gitea.stevedudenhoeffer.com/steve/majordomo/health"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/majordomo/provider/anthropic"
"gitea.stevedudenhoeffer.com/steve/majordomo/provider/google"
"gitea.stevedudenhoeffer.com/steve/majordomo/provider/ollama"
"gitea.stevedudenhoeffer.com/steve/majordomo/provider/openai"
)
// Usage and trace recording live in sink.go: SetUsageSink / SetTraceSink
// install the host seams, and ParseModelForContext stamps the model name on
// the context (via WithModel) for attribution.
// ---------------------------------------------------------------------------
// Package registry
// ---------------------------------------------------------------------------
// buildConfig carries the knobs Wire feeds into buildRegistry. The zero
// value yields a lane-less registry with majordomo's default failover
// behavior — the bootstrap state tests and pre-Wire code paths run on.
type buildConfig struct {
lanes LaneRegistry
// maxRetries maps the llms.failover.max_retries convar onto
// ChainConfig.TransientRetries. <= 0 keeps majordomo's default (1).
maxRetries int
// cooldown maps the llms.failover.cooldown_seconds convar onto
// health.Config.BaseCooldown. <= 0 keeps the mort default (300s).
// Note majordomo grows the cooldown exponentially from this base;
// MaxCooldown is set to max(cooldown, 5m) so the operator dial
// dominates (a 10m base never gets capped below itself).
cooldown time.Duration
// observer receives one event per failover decision (failed attempt,
// bench, benched-skip). Typically failoverlog.NewObserver(...).
observer func(majordomo.FailoverEvent)
}
// defaultFailoverCooldown matches the historical llms.failover.cooldown_seconds
// convar default (300s).
const defaultFailoverCooldown = 300 * time.Second
var (
registryMu sync.RWMutex
registry = buildRegistry(buildConfig{})
)
// Registry returns the current package-level majordomo registry. Most
// callers should use ParseModelRequest / ParseModelForContext instead;
// the registry itself is exposed for admin surfaces (health/bench) and
// for tests that need to substitute providers.
func Registry() *majordomo.Registry {
registryMu.RLock()
defer registryMu.RUnlock()
return registry
}
// Health returns the health tracker of the current registry — the live
// source of truth for benched models. Used by the `.failover` commands
// and the failover web UI (see ListBenched/BenchModel/UnbenchModel for
// the mort-flavored facade).
func Health() *health.Tracker {
return Registry().Health()
}
// setRegistry swaps the package registry. Bench/backoff state of the old
// registry is discarded — Wire is a boot-time operation.
func setRegistry(r *majordomo.Registry) {
registryMu.Lock()
defer registryMu.Unlock()
registry = r
}
// buildRegistry constructs a fully-wired majordomo registry:
//
// - health/chain config from the failover convars (via cfg),
// - mort's providers under their nonstandard env keys (OPENAI_KEY,
// GOOGLE_GEMINI_API_KEY, ...), every one lane-decorated,
// - OpenAI-compat presets (deepseek, moonshot+kimi, xai+grok, groq),
// - scheme factories for LLM_* env DSNs re-registered so DSN-defined
// providers (m1, arbitrary foreman targets) are lane-decorated too,
// with foreman additionally getting the 30-minute model timeout,
// - the legacy shortcut aliases, and
// - the delegating tier resolver (reads defaultResolver at Resolve
// time, so Init() can swap in the DB-backed resolver later).
func buildRegistry(cfg buildConfig) *majordomo.Registry {
cooldown := cfg.cooldown
if cooldown <= 0 {
cooldown = defaultFailoverCooldown
}
maxCooldown := cooldown
if maxCooldown < 5*time.Minute {
maxCooldown = 5 * time.Minute
}
r := majordomo.New(
// Env DSNs are loaded manually below, AFTER the scheme factories
// are overridden — New()'s eager scan would otherwise build
// LLM_*-defined providers with the stock (un-decorated) factories.
majordomo.WithoutEnvProviders(),
majordomo.WithHealthConfig(health.Config{
BaseCooldown: cooldown,
MaxCooldown: maxCooldown,
}),
majordomo.WithChainConfig(majordomo.ChainConfig{
TransientRetries: cfg.maxRetries,
// legacy gollm failed over on request-specific errors (400/413/422)
// without benching; majordomo fails fast on permanent errors by
// default. AdvanceOnPermanent preserves the availability-first
// behavior mort's executors rely on.
AdvanceOnPermanent: true,
Observer: cfg.observer,
}),
)
wrap := func(p llm.Provider) llm.Provider {
return wrapProviderForLane(p, cfg.lanes, defaultLaneExecTimeout)
}
// Core providers with mort's env keys.
r.RegisterProvider(wrap(openai.New(
openai.WithAPIKey(os.Getenv("OPENAI_KEY")),
)))
r.RegisterProvider(wrap(anthropic.New(
anthropic.WithAPIKey(os.Getenv("ANTHROPIC_API_KEY")),
)))
r.RegisterProvider(wrap(google.New(
google.WithAPIKey(os.Getenv("GOOGLE_GEMINI_API_KEY")),
)))
r.RegisterProvider(wrap(localOllamaProvider()))
// ollama.Cloud reads OLLAMA_API_KEY itself; with the key unset the
// provider still registers and errors clearly at call time (parity
// with the previous behavior).
r.RegisterProvider(wrap(ollama.Cloud()))
// OpenAI-compatible presets. Base URLs mirror legacy gollm's defaults.
for _, preset := range []struct {
name, baseURL, envKey string
}{
{"deepseek", "https://api.deepseek.com/v1", "DEEPSEEK_API_KEY"},
{"moonshot", "https://api.moonshot.ai/v1", "MOONSHOT_API_KEY"},
{"kimi", "https://api.moonshot.ai/v1", "MOONSHOT_API_KEY"}, // alias provider for moonshot
{"xai", "https://api.x.ai/v1", "XAI_API_KEY"},
{"grok", "https://api.x.ai/v1", "XAI_API_KEY"}, // alias provider for xai
{"groq", "https://api.groq.com/openai/v1", "GROQ_API_KEY"},
} {
r.RegisterProvider(wrap(openai.New(
openai.WithName(preset.name),
openai.WithBaseURL(preset.baseURL),
openai.WithAPIKey(os.Getenv(preset.envKey)),
)))
}
// Scheme factories for LLM_* env DSNs. Re-registered so DSN-defined
// providers go through the lane decorator like the built-ins.
//
// foreman targets are slow local LLMs (large model loads, queued
// behind other requests), so their models additionally get a hard
// 30-minute timeout and a matching lane execution backstop — the
// default 5-minute lane backstop would strangle them.
r.RegisterScheme("foreman", func(name string, dsn majordomo.DSN) (llm.Provider, error) {
p := ollama.Foreman(dsn.BaseURL(), dsn.Token, ollama.WithName(name))
return wrapProviderForLane(
withModelTimeout(p, foremanModelTimeout),
cfg.lanes,
foremanLaneExecTimeout,
), nil
})
laneScheme := func(factory majordomo.SchemeFactory) majordomo.SchemeFactory {
return func(name string, dsn majordomo.DSN) (llm.Provider, error) {
p, err := factory(name, dsn)
if err != nil {
return nil, err
}
return wrap(p), nil
}
}
ollamaScheme := laneScheme(func(name string, dsn majordomo.DSN) (llm.Provider, error) {
return ollama.New(
ollama.WithName(name),
ollama.WithBaseURL(dsn.BaseURL()),
ollama.WithToken(dsn.Token),
), nil
})
r.RegisterScheme("ollama", ollamaScheme)
r.RegisterScheme("ollama-cloud", ollamaScheme)
r.RegisterScheme("openai", laneScheme(func(name string, dsn majordomo.DSN) (llm.Provider, error) {
return openai.New(
openai.WithName(name),
openai.WithBaseURL(dsn.BaseURL()),
openai.WithAPIKey(dsn.Token),
), nil
}))
r.RegisterScheme("anthropic", laneScheme(func(name string, dsn majordomo.DSN) (llm.Provider, error) {
return anthropic.New(
anthropic.WithName(name),
anthropic.WithBaseURL(dsn.BaseURL()),
anthropic.WithAPIKey(dsn.Token),
), nil
}))
googleScheme := laneScheme(func(name string, dsn majordomo.DSN) (llm.Provider, error) {
return google.New(
google.WithName(name),
google.WithBaseURL(dsn.BaseURL()),
google.WithAPIKey(dsn.Token),
), nil
})
r.RegisterScheme("google", googleScheme)
r.RegisterScheme("gemini", googleScheme)
// Eager LLM_* env scan, now with the decorated scheme factories in
// place. Malformed entries are recorded per-name and surface on use.
env := make(map[string]string)
for _, kv := range os.Environ() {
if k, v, ok := strings.Cut(kv, "="); ok {
env[k] = v
}
}
_ = r.LoadEnv(env)
// Legacy shortcut aliases (sonnet, haiku, ...). Same strings as the
// historical table; kept in sync with legacyAliasSpecs below.
for name, spec := range legacyAliasSpecs {
r.RegisterAlias(name, spec)
}
// Tier resolver: a delegating closure so Init() and test helpers can
// swap defaultResolver without rebuilding the registry. The resolver
// returns specs with the legacy reasoning suffixes already stripped
// (per chain element); the tier's default reasoning level is applied
// by ParseModelRequest, not here.
r.RegisterResolver(majordomo.ResolverFunc(func(name string) (string, bool) {
res := defaultResolver
if res == nil {
return "", false
}
spec, _, ok := res.Resolve(name)
return spec, ok
}))
return r
}
// localOllamaProvider builds the local Ollama provider, honoring
// OLLAMA_BASE_URL when set (mort's historical env var; ollama.Local
// itself honors OLLAMA_HOST).
func localOllamaProvider() llm.Provider {
if url := os.Getenv("OLLAMA_BASE_URL"); url != "" {
return ollama.Local(ollama.WithBaseURL(url))
}
return ollama.Local()
}
// ---------------------------------------------------------------------------
// Spec parsing
// ---------------------------------------------------------------------------
// ParseModelRequest resolves a model request string to a ready-to-use Model.
// It handles, in order:
//
// - empty spec → tier "fast"
// - the legacy ":low/:medium/:high" reasoning suffix, stripped per chain
// element (ollama tags like ":30b" or ":cloud" are preserved); the
// level is applied to every call via llm.WithReasoningEffort
// - tier aliases (DB-backed convars; a tier value's own suffix becomes
// the default level when the caller didn't supply one)
// - legacy shortcut aliases (sonnet, haiku, opus, ...)
// - provider/model lookup and LLM_* env-DSN fallback (majordomo)
// - comma-separated failover chains with health-tracked bench/backoff
//
// The returned Model is instrumented: token usage from every successful
// Generate is recorded to the package usage recorder automatically. Do
// NOT additionally call RecordUsage on responses from a parsed model.
func ParseModelRequest(spec string) (majordomo.Model, error) {
spec = strings.TrimSpace(spec)
if spec == "" {
spec = "fast"
}
clean, level := splitReasoningSpec(spec)
// Tier default reasoning: when the (suffix-free) spec is exactly a
// tier name and the caller didn't ask for a level, the tier value's
// own suffix (e.g. "anthropic/claude-opus-4-6:high") applies.
if level == "" && defaultResolver != nil {
if _, tierLevel, ok := defaultResolver.Resolve(clean); ok {
level = tierLevel
}
}
m, err := Registry().Parse(clean)
if err != nil {
return nil, fmt.Errorf("model %q: %w", spec, err)
}
if level != "" {
m = &reasoningModel{inner: m, level: level}
}
return &instrumentedModel{inner: m}, nil
}
// ParseModelForContext combines ParseModelRequest with llmusage.WithModel so
// that the resolved model name is recorded in the context for usage tracking.
// Prefer this over bare ParseModelRequest in all new code.
func ParseModelForContext(ctx context.Context, req string) (context.Context, majordomo.Model, error) {
model, err := ParseModelRequest(req)
if err != nil {
return ctx, nil, err
}
ctx = WithModel(ctx, ResolveModelName(req))
return ctx, model, nil
}
// reasoningModel applies a default reasoning effort to every request that
// doesn't carry one already. Mort's legacy ":low/:medium/:high" suffix
// dialect resolves to this wrapper because majordomo treats model ids as
// verbatim (no suffix stripping).
type reasoningModel struct {
inner llm.Model
level string
}
func (m *reasoningModel) Generate(ctx context.Context, req llm.Request, opts ...llm.Option) (*llm.Response, error) {
req = req.Apply(opts...)
if req.ReasoningEffort == "" {
req.ReasoningEffort = m.level
}
return m.inner.Generate(ctx, req)
}
func (m *reasoningModel) Stream(ctx context.Context, req llm.Request, opts ...llm.Option) (llm.Stream, error) {
req = req.Apply(opts...)
if req.ReasoningEffort == "" {
req.ReasoningEffort = m.level
}
return m.inner.Stream(ctx, req)
}
func (m *reasoningModel) Capabilities() llm.Capabilities { return m.inner.Capabilities() }
// ---------------------------------------------------------------------------
// Reasoning-suffix dialect
// ---------------------------------------------------------------------------
// reasoningLevels is the set of recognized legacy suffix values.
var reasoningLevels = map[string]bool{"low": true, "medium": true, "high": true}
// splitReasoning peels an optional ":low" / ":medium" / ":high" suffix off
// a single model request string. Returns the input unchanged and "" when no
// recognized level is present, so non-reasoning suffixes (ollama tags like
// ":30b" or ":q4_K_M", date stamps) flow through untouched.
func splitReasoning(s string) (string, string) {
idx := strings.LastIndex(s, ":")
if idx < 0 {
return s, ""
}
if lvl := s[idx+1:]; reasoningLevels[lvl] {
return s[:idx], lvl
}
return s, ""
}
// splitReasoningSpec strips the legacy reasoning suffix from every element
// of a (possibly comma-separated) spec. The returned level is the first
// non-empty per-element level — majordomo chains carry one request-level
// reasoning effort, not one per target, so the head element's preference
// wins. Elements without a suffix are unchanged.
func splitReasoningSpec(spec string) (string, string) {
if !strings.Contains(spec, ",") {
return splitReasoning(strings.TrimSpace(spec))
}
parts := strings.Split(spec, ",")
level := ""
for i, p := range parts {
s, l := splitReasoning(strings.TrimSpace(p))
parts[i] = s
if level == "" {
level = l
}
}
return strings.Join(parts, ","), level
}
// ---------------------------------------------------------------------------
// Usage-attribution name resolution
// ---------------------------------------------------------------------------
// ResolveModelName returns the model portion of a request string, stripping
// any reasoning suffix and resolving tier aliases. The result is used for
// usage attribution (keyed on model name, not provider or reasoning level).
func ResolveModelName(req string) string {
// Strip any reasoning-level suffix before resolving — the level is a
// per-request setting, not part of the model identity.
req, _ = splitReasoning(req)
// Tier expansion: when the request is a tier alias, fold it through the
// resolver and return the model portion of its current convar value. The
// empty string is treated as "fast" for compatibility with callers that
// pre-resolution defaulted to fast.
if defaultResolver != nil {
key := req
if key == "" {
key = "fast"
}
if spec, _, ok := defaultResolver.Resolve(key); ok && spec != "" {
// A tier may resolve to a comma-separated failover chain. Attribute
// usage to the first (preferred) entry's model name rather than the
// whole chain string.
if i := strings.IndexByte(spec, ','); i >= 0 {
spec = strings.TrimSpace(spec[:i])
}
if idx := strings.Index(spec, "/"); idx >= 0 {
return spec[idx+1:]
}
return spec
}
}
// For non-tier requests, return the model portion after the slash.
// Static aliases are NOT expanded here beyond the legacy table below:
// callers that went through ParseModelRequest already carry the
// concrete spec.
if idx := strings.Index(req, "/"); idx >= 0 {
return req[idx+1:]
}
// Legacy shortcut fallback: callers that pass bare names like "sonnet"
// to ResolveModelName (without going through ParseModelRequest) still
// need the concrete model name for usage keys.
if spec, ok := legacyAliasSpecs[req]; ok {
if idx := strings.Index(spec, "/"); idx >= 0 {
return spec[idx+1:]
}
return spec
}
return req
}
// legacyAliasSpecs maps legacy shortcut names to their full provider/model
// spec. Registered with the registry as static aliases AND consulted by
// ResolveModelName for bare-name usage attribution.
var legacyAliasSpecs = map[string]string{
"openai": "openai/gpt-4o-mini",
"gpt-4": "openai/gpt-4",
"gpt-4o": "openai/gpt-4o",
"gpt-4o-mini": "openai/gpt-4o-mini",
"sonnet": "anthropic/claude-sonnet-4-6",
"sonnet-4.5": "anthropic/claude-sonnet-4-5-20250929",
"haiku": "anthropic/claude-haiku-4-5-20251001",
"opus": "anthropic/claude-opus-4-6",
"gemini": "google/gemini-2.0-flash",
"gemini-flash": "google/gemini-2.0-flash",
"gemini-pro": "google/gemini-2.0-pro",
}
+136
View File
@@ -0,0 +1,136 @@
package model
import (
"context"
"time"
)
// This file is executus's inversion of mort's llmusage / llmtrace coupling.
// The model package owns the MECHANISM (instrument every parsed model's
// Generate, attribute by serving model, emit a span when a trace is active);
// WHERE usage/traces land is a host seam. A host registers a UsageSink and/or
// a TraceSink; both are optional (nil = off), so a light host records nothing.
// --- Usage ---
// UsageSink receives one record per successful Generate through a model parsed
// by this package (ParseModelRequest / ParseModelForContext). Implement it to
// meter or bill; the token detail mirrors majordomo's Response.Usage.
//
// IMPORTANT: cacheReadTokens and cacheWriteTokens are PORTIONS of inputTokens,
// not independent additive values (they let a sink price cached vs fresh input
// differently). A sink must NOT compute total = input+output+cacheRead+
// cacheWrite — that double-counts the cached input.
type UsageSink interface {
Record(ctx context.Context, model string, inputTokens, outputTokens, cacheReadTokens, cacheWriteTokens int)
}
var usageSink UsageSink
// SetUsageSink installs the usage sink (nil disables usage recording). Call at
// startup before model calls.
func SetUsageSink(s UsageSink) { usageSink = s }
// --- Trace ---
// Span is one traced model call. The host's TraceSink persists it however it
// likes (a DB row, a log line, an OTel span). String fields carrying structured
// data (Messages, ToolDefinitions, ...) are pre-marshalled JSON.
type Span struct {
SpanID string
TraceID string
Model string
SystemPrompt string
Messages string
ToolDefinitions string
ResponseText string
ResponseToolCalls string
ToolResults string
Error string
InputTokens int
OutputTokens int
DurationMs int64
StartedAt time.Time
CompletedAt time.Time
CreatedAt time.Time
}
// TraceSink receives a Span for each traced call (one is emitted only when a
// trace id is present on the context — see WithTraceID).
type TraceSink interface {
WriteSpan(span Span)
}
var traceSink TraceSink
// SetTraceSink installs the trace sink (nil disables tracing).
func SetTraceSink(s TraceSink) { traceSink = s }
// TraceSinkActive reports whether a trace sink is installed.
func TraceSinkActive() bool { return traceSink != nil }
// --- Context attribution ---
//
// ParseModelForContext stamps the requested model onto the context so usage
// from a response that doesn't name its serving model can still be attributed.
// A host's tracing/usage middleware stamps a trace id and optional caller/tool
// for diagnostics. All reads are nil/empty-safe.
type (
ctxKeyModel struct{}
ctxKeyTrace struct{}
ctxKeyTool struct{}
ctxKeyUser struct{}
)
// WithModel attributes subsequent usage on ctx to the given model name.
func WithModel(ctx context.Context, model string) context.Context {
return context.WithValue(ctx, ctxKeyModel{}, model)
}
func modelFromContext(ctx context.Context) string {
if v, ok := ctx.Value(ctxKeyModel{}).(string); ok {
return v
}
return ""
}
// WithTraceID marks ctx as belonging to a trace; a TraceSink (if installed)
// then receives a Span per call. An empty id (or no id) disables tracing.
func WithTraceID(ctx context.Context, id string) context.Context {
return context.WithValue(ctx, ctxKeyTrace{}, id)
}
func traceIDFromContext(ctx context.Context) string {
if v, ok := ctx.Value(ctxKeyTrace{}).(string); ok {
return v
}
return ""
}
// WithUsageTool / WithUsageUser attach optional attribution used only in the
// "unknown model" diagnostic warning. Default "unknown".
func WithUsageTool(ctx context.Context, tool string) context.Context {
return context.WithValue(ctx, ctxKeyTool{}, tool)
}
func toolFromContext(ctx context.Context) string {
if v, ok := ctx.Value(ctxKeyTool{}).(string); ok && v != "" {
return v
}
return "unknown"
}
func WithUsageUser(ctx context.Context, user string) context.Context {
return context.WithValue(ctx, ctxKeyUser{}, user)
}
func userFromContext(ctx context.Context) string {
if v, ok := ctx.Value(ctxKeyUser{}).(string); ok && v != "" {
return v
}
return "unknown"
}
+162
View File
@@ -0,0 +1,162 @@
package model
import (
"fmt"
"sort"
"strings"
"sync"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/config"
)
// tierResolver expands tier aliases (e.g. "fast", "thinking", "agent-working")
// into a concrete model spec or a comma-separated failover chain. The set of
// tier names and their FALLBACK specs are host-supplied (a map passed at
// Configure time); the live value of each tier is read from a config.Source
// under the key "model.tier.<name>", so a host whose config backend mutates at
// runtime (mort's convar) re-targets tiers without a restart, while a static
// host (gadfly's env) just gets the fallback. A small in-process cache (TTL
// from "model.tier.cache_ttl_seconds", default 30s) saves config round-trips on
// the hot path; ReloadTiers clears it.
//
// This is executus's inversion of mort's convar-bound resolver: the MECHANISM
// (tier lookup, reasoning-suffix dialect, chain validation, cache) is generic;
// the tier MAP content (which tiers exist + their default specs) is host config.
type tierResolver struct {
cfg config.Source
defaults map[string]string // tier name -> fallback spec
ttl time.Duration
mu sync.RWMutex
cache map[string]tierEntry
now func() time.Time // overridable for tests
}
type tierEntry struct {
spec string
reasoning string
expires time.Time
}
const tierConfigPrefix = "model.tier."
// NewTierResolver builds a resolver over cfg with the given tier defaults
// (name -> fallback spec). cfg may be nil (the fallbacks are then always used).
// ttl<=0 reads "model.tier.cache_ttl_seconds" (default 30s).
func NewTierResolver(cfg config.Source, defaults map[string]string, ttl time.Duration) *tierResolver {
if ttl <= 0 {
ttl = time.Duration(config.Int(cfg, tierConfigPrefix+"cache_ttl_seconds", 30)) * time.Second
}
if ttl <= 0 {
ttl = 30 * time.Second
}
cp := make(map[string]string, len(defaults))
for k, v := range defaults {
cp[k] = v
}
return &tierResolver{
cfg: cfg,
defaults: cp,
ttl: ttl,
cache: make(map[string]tierEntry),
now: time.Now,
}
}
func (r *tierResolver) has(name string) bool {
_, ok := r.defaults[name]
return ok
}
func (r *tierResolver) names() []string {
out := make([]string, 0, len(r.defaults))
for k := range r.defaults {
out = append(out, k)
}
sort.Strings(out)
return out
}
// Resolve returns the current model spec and default reasoning level for a tier
// name. ok=false if name is not a registered tier. Legacy reasoning suffixes
// (":low/:medium/:high") are stripped per chain element; the first non-empty
// level becomes the tier's default reasoning level (ollama tags like ":cloud"
// pass through). The live value is read from config with the host-supplied
// fallback; an empty resolved value yields ok=true with an empty spec
// (ParseModelRequest surfaces a clear error in that path).
func (r *tierResolver) Resolve(name string) (string, string, bool) {
if !r.has(name) {
return "", "", false
}
now := r.now()
r.mu.RLock()
if e, hit := r.cache[name]; hit && now.Before(e.expires) {
r.mu.RUnlock()
return e.spec, e.reasoning, true
}
r.mu.RUnlock()
r.mu.Lock()
defer r.mu.Unlock()
if e, hit := r.cache[name]; hit && now.Before(e.expires) {
return e.spec, e.reasoning, true
}
raw := strings.TrimSpace(config.String(r.cfg, tierConfigPrefix+name, r.defaults[name]))
spec, level := splitReasoningSpec(raw)
r.cache[name] = tierEntry{spec: spec, reasoning: level, expires: now.Add(r.ttl)}
return spec, level, true
}
// Reload clears the cache so the next Resolve fetches fresh from config.
func (r *tierResolver) Reload() {
r.mu.Lock()
defer r.mu.Unlock()
r.cache = make(map[string]tierEntry)
}
// --- package-level resolver + facade ---
// defaultResolver is initialized as a package-level var (not in init()) so it
// is ready before any other file's init runs — buildRegistry's delegating
// resolver closure reads it at Resolve time. It starts with no tiers; a host
// installs its tier table via Configure.
var defaultResolver = NewTierResolver(nil, nil, 0)
// Configure installs the host's tier table. cfg is the live config source
// (nil = fallbacks only); defaults maps each tier name to its fallback spec;
// ttl<=0 uses the config'd / 30s default. The package registry's delegating
// resolver reads defaultResolver at Resolve time, so swapping it here is
// sufficient — no registry rebuild needed.
func Configure(cfg config.Source, defaults map[string]string, ttl time.Duration) {
defaultResolver = NewTierResolver(cfg, defaults, ttl)
}
// TierNames returns the registered tier alias names (sorted). Exported so UI
// layers can populate tier dropdowns without hardcoding.
func TierNames() []string { return defaultResolver.names() }
// IsTierName reports whether s is a registered tier alias.
func IsTierName(s string) bool { return defaultResolver.has(s) }
// ReloadTiers clears the package resolver's cache so the next request resolves
// freshly from config.
func ReloadTiers() { defaultResolver.Reload() }
// ValidateTierValue returns an error if value cannot be used as a tier spec —
// specifically, when a chain entry is itself a tier name (which would form a
// resolution loop). Chain entries must be concrete provider/model specs.
func ValidateTierValue(value string) error {
for _, part := range strings.Split(value, ",") {
part = strings.TrimSpace(part)
if part == "" {
continue
}
spec, _ := splitReasoning(part)
if IsTierName(spec) {
return fmt.Errorf("tier value %q contains tier alias %q (chains must use concrete provider/model specs, not nested tiers)", value, spec)
}
}
return nil
}
+110
View File
@@ -0,0 +1,110 @@
// Package llms — wiring.go: the production boot hook that rebuilds the
// package registry with the lane registry, the failover convars, and the
// failover-event observer.
//
// Why a dedicated helper (vs spreading registry construction through
// mort.go): the chatbot regression test in lane_chatbot_test.go and the
// production boot path must call the SAME wiring code. Historically
// mort.go skipped the lane wiring entirely (lanes were defined but never
// installed — 30+ skill_runs in production with 0 skill_queue_jobs rows);
// concentrating the install here means a regression in one wires fails
// the test for the other.
package model
import (
"context"
"log/slog"
"time"
majordomo "gitea.stevedudenhoeffer.com/steve/majordomo"
)
// WireOptions configures Wire. The zero value rebuilds the registry with
// no lanes and default failover behavior.
type WireOptions struct {
// Lanes is the lane registry every provider is decorated with. nil
// disables lane queueing (calls pass straight through) but keeps
// error attribution for the failover log.
Lanes LaneRegistry
// FailoverMaxRetries maps the llms.failover.max_retries convar onto
// majordomo's ChainConfig.TransientRetries (same-target retries after
// a transient error). <= 0 keeps majordomo's default (1).
FailoverMaxRetries int
// FailoverCooldown maps the llms.failover.cooldown_seconds convar
// onto health.Config.BaseCooldown. majordomo grows the cooldown
// exponentially from this base per consecutive bench; the cap is
// max(FailoverCooldown, 5m) so the operator's dial dominates.
// <= 0 keeps the mort default (300s).
FailoverCooldown time.Duration
// FailoverObserver receives one event per failover decision (failed
// attempt, bench, benched-skip). Wire it to failoverlog.NewObserver.
// Attribution (caller/run/prompts) rides on the event's error — see
// CallInfoFromError.
FailoverObserver func(majordomo.FailoverEvent)
}
// Wire rebuilds the package registry from opts and installs it. Call once
// at boot, after the lane registry and the failover convars exist (and
// after Init for DB-backed tiers — though Init and Wire are order-
// independent: the tier resolver is consulted through a delegating
// indirection).
//
// Rebuilding discards in-memory health/bench state — Wire is a boot-time
// operation, not a runtime toggle.
//
// When Lanes is non-nil, the well-known lanes (KnownLanes) are eagerly
// registered so admin dashboards have baseline state from the moment mort
// starts instead of "no lanes registered" until the first LLM call.
//
// Returns the installed registry for inspection (tests, health surfaces).
func Wire(ctx context.Context, opts WireOptions) *majordomo.Registry {
r := buildRegistry(buildConfig{
lanes: opts.Lanes,
maxRetries: opts.FailoverMaxRetries,
cooldown: opts.FailoverCooldown,
observer: opts.FailoverObserver,
})
setRegistry(r)
if opts.Lanes != nil {
names := KnownLanes()
for _, name := range names {
opts.Lanes.GetOrCreate(ctx, name)
}
slog.Info("llms: wired lane-aware registry", "lanes", len(names))
} else {
slog.Warn("llms: Wire called without a lane registry — lane queueing is inert")
}
return r
}
// KnownLanes returns the well-known lane names the LLM transport resolves
// to. Eager-registering these at boot gives admin dashboards
// (`/skills/admin/queues`, `.skill admin queue`) a baseline view from the
// moment mort starts — without this, the dashboard reads "no lanes
// registered" until the first chatbot/skill call materialises the lane
// via lazy GetOrCreate.
//
// Why this list (and not "every lane name ever"): these are the ones
// LaneFor in lane_mapping.go can produce for a real model spec. Future
// non-LLM lanes (e.g. a future image-generation lane) should be eagerly
// registered by their owning subsystem, not here.
//
// LaneSkillDefault is included even though it isn't an LLM-routing
// lane: skills run through it via skillexec.WithLaneRegistry, and the
// skills admin dashboard needs to see it from boot.
//
// Test: wiring_test.go::TestKnownLanes_NonEmpty.
func KnownLanes() []string {
return []string{
LaneOllama,
LaneAnthropicThinking,
LaneAnthropicDefault,
LaneM1,
LaneLLMDefault,
"skill-default",
}
}
+191
View File
@@ -0,0 +1,191 @@
// Package agents implements the Agent noun: a persisted persona +
// execution spec + palette of skills/sub-agents/low-level tools, with
// optional trigger metadata (schedule, webhook, chatbot channel
// listener) and personalization sources.
//
// Phase 1 scope: this package introduces Agent as a persisted noun
// with CRUD only — no execution path, no palette resolution, no
// trigger handling. See /Users/steve/.claude/plans/serene-churning-micali.md
// for the staged rollout. Later phases add agentexec, agent_invoke,
// trigger dispatch (schedule/webhook/chatbot), and CommandBinding.
//
// The three-layer storage pattern from pkg/logic/storage/CLAUDE.md
// applies — when adding a field to Agent, you MUST update
// pkg/logic/storage/agents.go (gormAgent, agentFromStorage,
// toStorage) or persistence will silently break.
package persona
import "time"
// Agent is the domain definition of an Agent persona + execution spec.
//
// Why: an Agent is the "configured invoker" — model tier + base
// system prompt + a palette of capabilities (skills, sub-agents,
// low-level tools) it may exercise during a run. Where a Skill is a
// reusable parameterised callable (a library function), an Agent is
// the actor with a persistent persona that calls those skills. The
// struct is flat — every field lives on its own column on the
// agents table; JSON columns are used only for variable-length
// collections (palette lists, tags, etc.).
//
// What: identity + persona + execution caps + palette + triggers +
// personalization + UX, all on one struct. Several field families
// (Palette, Triggers, Personalization) are persisted now but NOT
// exercised until later phases — they exist so the schema is stable
// and future phases can light up behaviour without DB migrations.
//
// Test: see pkg/logic/agents/storage_round_trip_test.go for
// Save/Get/GetByName/List/Delete coverage.
type Agent struct {
// Identity
ID string // UUID
Name string // unique per OwnerID; human-friendly identifier
Description string
OwnerID string // Discord member ID
AuthoredBy string // Discord member ID; usually == OwnerID
Version int // monotonic, for future versioning
CreatedAt time.Time
UpdatedAt time.Time
// Extends names the parent agent this agent inherits from. Only used
// during builtin loading — the loader resolves extends references and
// merges fields before persisting. The resolved agent is a standalone
// entity; Extends is NOT persisted in the database. Only single-level
// extends is supported (no chains).
Extends string
// SystemPromptPrepend, when non-empty, is prepended to the system
// prompt (with a trailing newline separator). Used by the extends
// mechanism so a child agent can prepend persona instructions to the
// parent's full system prompt without duplicating it. Like Extends,
// this is resolved at load time and NOT persisted — the final
// SystemPrompt on the persisted Agent already has the prepend applied.
SystemPromptPrepend string
// Persona / execution spec
ModelTier string // "fast" | "standard" | "thinking" | provider/model
SystemPrompt string // base persona prompt (Phase 5 layers personalization on top)
MaxIterations int // 0 → use convar default at execution time
MaxToolCalls int // 0 → use convar default at execution time
MaxRuntime time.Duration // stored as MaxRuntimeNs int64 in GORM (avoid duration-driver flakiness)
ExecutionLane string // lane name; empty = default at execution time
EncryptionEnabled bool // Phase 1 stores the flag; envelope-encryption bridge wires in a later phase
// Run-critic (two-tier timeout). When CriticEnabled is false (the
// default) MaxRuntime is the hard kill, exactly as before. When true,
// MaxRuntime becomes a SOFT trigger: at MaxRuntime the run-critic
// activates and periodically reviews the run; the hard backstop (the
// absolute kill) is MaxRuntime × the multiplier. CriticBackstopMultiplier
// of 0 means "use the convar default" (agents.critic.backstop_multiplier_default,
// default 6×). See pkg/logic/agentcritic.
CriticEnabled bool
CriticBackstopMultiplier float64
// Palette — what this Agent may invoke (Phase 2 reads these).
// Stored as JSON arrays; not exercised by Phase 1 CRUD.
SkillPalette []string // skill IDs/names
SubAgentPalette []string // agent IDs/names
LowLevelTools []string // skilltools registry names
// Personalization (Phase 5 reads these). Each layer name maps to
// a registered PersonalizationProvider that returns text appended
// to SystemPrompt at run time. Empty list = base prompt only.
PersonalizationSources []string
// Triggers — persisted now, NOT dispatched by Phase 1.
//
// Schedule: cron expression or "daily"/"weekly" shorthand. Empty
// = on-demand only. NextRunAt + LastScheduledRunAt are scheduler
// bookkeeping for Phase 3's per-Agent scheduler.
Schedule string
NextRunAt *time.Time
LastScheduledRunAt *time.Time
// Webhook trigger metadata. WebhookSecret empty = inbound
// webhooks disabled. WebhookSignatureRequired defaults true at
// save time (see Skill's lesson: don't store a GORM default on a
// bool where false is a legitimate explicit value — application
// layer is the source of truth).
WebhookSecret string
WebhookSignatureRequired bool
WebhookIPAllowlist []string // CIDR strings; stored as JSON array
// Chatbot trigger metadata. ChatbotChannelFilter names a filter
// registered in pkg/logic/skills' ChannelFilterRegistry — when
// the migrated chatbot dispatches via this Agent, the filter
// gates which channels it listens in.
ChatbotChannelFilter string
// UX
//
// DefaultEmoji is an optional identity emoji for this agent.
// Used as the __start__ fallback and forwarded to the invoking
// Discord message when a parent calls this agent via agent_invoke.
DefaultEmoji string
// StateReactEmoji maps tool names (and reserved keys "__start__",
// "__end__", "__error__") to Discord emoji that the executor
// reacts with as the run progresses. Empty map = no reactions.
StateReactEmoji map[string]string
// Tags is a free-form set of short labels for organisation +
// discovery on the agents list page (Phase 1 admin commands +
// future web UI).
Tags []string
// Phases defines a multi-phase pipeline for this agent. When
// non-empty, the executor runs agentexec's sequential phase runner
// instead of the single agent loop. Empty = single-loop agent.
//
// Phases IS persisted (JSON struct-slice column `phases` on
// gormAgent). It used to be transient — "TOML is the only source of
// truth" — but every production dispatch path resolves the agent from
// the DB, where the dropped Phases meant research / deepresearch
// silently degraded to a single-loop run (the executor's
// `len(a.Phases) > 0` pipeline branch was dead). The builtin loader
// still seeds phases from YAML; persisting them is what makes the
// pipeline branch fire for DB-loaded agents.
Phases []AgentPhase
}
// AgentPhase describes one stage of a multi-phase pipeline in an
// agent definition. Executed directly by agentexec's phase runner
// (pipeline.go) — there is no intermediate execution-spec struct.
//
// What: name + prompt template + model/iteration overrides + tool
// list + optional/fallback flags + IsRunFunc indicator.
//
// Test: see builtin_loader_test.go for YAML round-trip coverage.
type AgentPhase struct {
// Name identifies the phase (e.g., "scout", "plan", "investigate").
Name string
// SystemPrompt for this phase. Supports template variables:
// {{.Query}} for the original query, and {{.<PhaseName>}} for
// prior phase outputs (e.g., {{.scout}}, {{.plan}}).
SystemPrompt string
// ModelTier overrides the agent's ModelTier for this phase.
// Empty = use agent default.
ModelTier string
// MaxIter overrides the agent's MaxIterations for this phase.
// 0 = use agent default.
MaxIter int
// Tools are tool names for this phase only. These are resolved
// from the agent's low-level tools + palette at execution time.
Tools []string
// Optional means errors in this phase don't abort the pipeline.
Optional bool
// FallbackMessage is used when an optional phase fails.
// Default: "(Phase <Name> encountered an error)"
FallbackMessage string
// IsRunFunc indicates this phase is a bare LLM call (no tool
// loop). When true, the executor makes a single model.Complete
// call instead of running the full agent loop.
IsRunFunc bool
}
+599
View File
@@ -0,0 +1,599 @@
package persona
// Phase 6 — Builtin Agent loader.
//
// Why: Phase 1-5 introduced the Agent noun, runtime, triggers,
// CommandBinding, and ChatBot bridge — but every Agent in production
// was either (a) a wrapper auto-migrated from a triggered Skill, or
// (b) admin-created via `.agent new`. There were no SHIPPED Agents
// authored as builtins. Phase 6 adds an idempotent boot-time loader so
// the repo can ship canonical Agent definitions (alongside the
// existing skills/<name>/skill.yml builtins) without manual admin
// creation per deploy.
//
// What: scans `<builtinsDir>/agents/*/agent.yml`, decodes each YAML
// into an Agent, and upserts via Storage.SaveAgent under the deterministic
// system owner ID "builtin". Skill-palette entries are validated AT LOAD
// TIME against the live skills storage; missing skills warn but do not
// fail the load (the skill might arrive later via a different code
// path, and runtime resolution happens at invocation time anyway).
//
// Bypass note (v3 lesson, mirrored): like skills.LoadBuiltins, this
// loader writes via Storage.SaveAgent directly. There is no agents
// equivalent of SaveUserSkill's save-time gates today (Phase 1-5 don't
// have authoring requirements on agents), but if such gates appear in
// future phases, this loader MUST keep bypassing them — builtins are
// trusted infrastructure.
//
// Test: pkg/logic/agents/builtin_loader_test.go covers happy path,
// idempotent re-load, missing-skill warn capture, and malformed YAML
// surfaced as a per-bundle warning (not a fatal error).
import (
"context"
"errors"
"fmt"
"io/fs"
"log/slog"
"net"
"path"
"strings"
"time"
"github.com/google/uuid"
"gopkg.in/yaml.v3"
)
// BuiltinAgentOwnerID is the deterministic system owner ID used for
// every Agent created by LoadBuiltinAgents. Chosen as a non-empty
// string so the (owner_id, name) unique index distinguishes builtins
// from any user-authored Agent (Discord member IDs are numeric, so
// "builtin" cannot collide). The skills builtin loader uses owner_id=""
// instead; the two systems are independent storage scopes — there's
// no need for consistency here.
const BuiltinAgentOwnerID = "builtin"
// SkillExistenceChecker is the narrow surface LoadBuiltinAgents needs
// to validate skill_palette entries at load time. Production wires
// skills.Storage which already exposes ListByName for non-owner-scoped
// lookups. nil means "skip palette validation" (tests that don't care).
//
// Why a separate narrow interface (vs importing skills.Storage):
// agents already transitively depends on skills via migrate_from_skills,
// but the loader only needs "does a skill with this name exist
// somewhere?" — a single Boolean. Keeping the interface narrow makes
// the loader testable without a full skills storage stub.
type SkillExistenceChecker interface {
// SkillExistsByName reports whether at least one skill row has the
// given name across any owner (builtins live under owner_id="";
// users own their own rows; the loader's validation just wants
// "does ANY row exist with this name?").
SkillExistsByName(ctx context.Context, name string) (bool, error)
}
// LoadBuiltinAgents discovers and seeds builtin Agents from `builtinsDir`.
// `builtinsDir` is the root that contains an `agents/` subdirectory;
// per-agent YAML lives at `agents/<name>/agent.yml`. Returns the count
// of agents seeded or updated (skipped rows do not contribute to the
// count). Returns nil error when the agents/ directory is absent — a
// deployment without any builtin agents is valid; the loader is then a
// no-op.
//
// Idempotency contract: existing Agent rows (matched by (owner_id="builtin",
// name)) are UPDATED to the freshly-parsed YAML on each boot. ID +
// CreatedAt are preserved; UpdatedAt is refreshed. User clones of a
// builtin Agent (different owner_id, same name) are NEVER touched —
// the loader only writes to (owner_id="builtin", name) rows.
//
// `skillChecker` may be nil; when non-nil, each SkillPalette entry is
// looked up and a WARN log emitted (with the agent + missing skill
// name) for absent references. The Agent row is still seeded with the
// palette intact — runtime resolution at invocation time is the
// authoritative gate.
func LoadBuiltinAgents(ctx context.Context, store Storage, builtinsDir fs.FS, skillChecker SkillExistenceChecker) (int, error) {
if store == nil {
return 0, errors.New("agents.LoadBuiltinAgents: nil store")
}
if builtinsDir == nil {
return 0, errors.New("agents.LoadBuiltinAgents: nil builtinsDir FS")
}
entries, err := fs.ReadDir(builtinsDir, "agents")
if err != nil {
// Missing agents/ directory is benign — a deployment may not
// ship any builtins. Other errors propagate so a permission /
// IO problem surfaces loudly.
if errors.Is(err, fs.ErrNotExist) {
slog.Info("agents: no builtin agents directory", "path", "agents")
return 0, nil
}
return 0, fmt.Errorf("agents: read agents dir: %w", err)
}
// Phase 1: parse all agent manifests into a map keyed by name.
// The map is needed so extends references can be resolved before
// any agent is upserted.
type parsedEntry struct {
agent *Agent
dir string
}
parsed := make(map[string]*parsedEntry)
var parseOrder []string // preserve FS iteration order for deterministic upsert
var scanned, failed int
for _, entry := range entries {
if !entry.IsDir() {
continue
}
manifestPath := path.Join("agents", entry.Name(), "agent.yml")
data, readErr := fs.ReadFile(builtinsDir, manifestPath)
if readErr != nil {
slog.Debug("agents: skipping (no agent.yml)", "dir", entry.Name(), "error", readErr)
continue
}
scanned++
ag, parseErr := decodeAgentManifest(data)
if parseErr != nil {
slog.Warn("agents: invalid agent.yml", "dir", entry.Name(), "error", parseErr)
failed++
continue
}
parsed[ag.Name] = &parsedEntry{agent: ag, dir: entry.Name()}
parseOrder = append(parseOrder, ag.Name)
}
// Phase 2: resolve extends references. Only single-level is
// supported — chains (A extends B extends C) are rejected.
for _, name := range parseOrder {
pe := parsed[name]
ag := pe.agent
if ag.Extends == "" {
continue
}
parent, ok := parsed[ag.Extends]
if !ok {
slog.Warn("agents: extends references unknown agent",
"agent", ag.Name, "extends", ag.Extends)
failed++
delete(parsed, name)
continue
}
if parent.agent.Extends != "" {
slog.Warn("agents: extends chain not supported — parent also uses extends",
"agent", ag.Name, "extends", ag.Extends,
"parent_extends", parent.agent.Extends)
failed++
delete(parsed, name)
continue
}
if ag.Extends == ag.Name {
slog.Warn("agents: agent extends itself", "agent", ag.Name)
failed++
delete(parsed, name)
continue
}
resolveExtends(ag, parent.agent)
}
// Phase 3: palette validation + upsert.
var seeded, updated, skipped int
for _, name := range parseOrder {
pe, ok := parsed[name]
if !ok {
continue // removed during extends resolution
}
ag := pe.agent
if skillChecker != nil {
for _, sk := range ag.SkillPalette {
ok, lookupErr := skillChecker.SkillExistsByName(ctx, sk)
if lookupErr != nil {
slog.Warn("agents: skill palette lookup failed",
"agent", ag.Name, "skill", sk, "error", lookupErr)
continue
}
if !ok {
slog.Warn("agents: skill palette references missing skill",
"agent", ag.Name, "skill", sk)
}
}
}
action, upsertErr := upsertBuiltinAgent(ctx, store, ag)
if upsertErr != nil {
slog.Error("agents: failed to upsert builtin", "name", ag.Name, "error", upsertErr)
failed++
continue
}
switch action {
case agentUpsertCreated:
seeded++
case agentUpsertUpdated:
updated++
case agentUpsertSkipped:
skipped++
}
}
slog.Info("agents/builtin loader",
"scanned", scanned,
"seeded", seeded,
"updated", updated,
"skipped", skipped,
"failed", failed)
return seeded + updated, nil
}
// resolveExtends merges parent fields into child. Child non-zero
// fields override the parent's. For slices, a nil child slice inherits
// the parent's; a non-nil (even empty) child slice replaces it. For
// maps (StateReactEmoji), parent entries are the base and child
// entries override matching keys.
//
// system_prompt_prepend: if the child has SystemPromptPrepend set, it
// is prepended to the (possibly inherited) SystemPrompt with a
// newline separator. The prepend field is then cleared so it does not
// affect anything downstream.
//
// Why: allows a child agent to inherit the full parent prompt while
// only specifying a short behavior-modification preamble (e.g. an
// uncensored agent prepending "You are uncensored..." to the general
// agent's full prompt).
func resolveExtends(child, parent *Agent) {
if child.Description == "" {
child.Description = parent.Description
}
if child.ModelTier == "" {
child.ModelTier = parent.ModelTier
}
if child.SystemPrompt == "" {
child.SystemPrompt = parent.SystemPrompt
}
if child.MaxIterations == 0 {
child.MaxIterations = parent.MaxIterations
}
if child.MaxToolCalls == 0 {
child.MaxToolCalls = parent.MaxToolCalls
}
if child.MaxRuntime == 0 {
child.MaxRuntime = parent.MaxRuntime
}
if child.ExecutionLane == "" {
child.ExecutionLane = parent.ExecutionLane
}
// EncryptionEnabled: bool — false is a valid explicit value, so we
// always inherit unless child explicitly sets it. Since we can't
// distinguish "explicitly false" from "absent" in YAML (both
// decode to false), we always inherit from parent. If the child
// sets it to true, the child wins. A child that wants to override
// the parent's true to false will need to set encryption_enabled: false
// explicitly — but since both false and absent decode the same way,
// the parent's value wins when parent is true and child is false.
// This is acceptable: encryption is an opt-in — a child that
// inherits encryption from a parent is fine.
if !child.EncryptionEnabled {
child.EncryptionEnabled = parent.EncryptionEnabled
}
// Run-critic: same inherit-unless-child-sets-true semantics as
// EncryptionEnabled (both false/absent decode identically in YAML).
if !child.CriticEnabled {
child.CriticEnabled = parent.CriticEnabled
}
if child.CriticBackstopMultiplier == 0 {
child.CriticBackstopMultiplier = parent.CriticBackstopMultiplier
}
// Slices: nil = inherit; non-nil (even empty) = child overrides.
if child.SkillPalette == nil {
child.SkillPalette = parent.SkillPalette
}
if child.SubAgentPalette == nil {
child.SubAgentPalette = parent.SubAgentPalette
}
if child.LowLevelTools == nil {
child.LowLevelTools = parent.LowLevelTools
}
if child.PersonalizationSources == nil {
child.PersonalizationSources = parent.PersonalizationSources
}
if child.Tags == nil {
child.Tags = parent.Tags
}
if child.WebhookIPAllowlist == nil {
child.WebhookIPAllowlist = parent.WebhookIPAllowlist
}
if child.Phases == nil {
child.Phases = parent.Phases
}
// Triggers (Schedule, ChatbotChannelFilter, WebhookSecret, …) are
// deliberately NOT inherited. A trigger is an ACTIVATION decision —
// "this agent fires on a schedule" / "this agent is a chatbot tool in
// these channels" — and silently inheriting it from a parent persona
// is a behavioural surprise: `uncensored extends general` would inherit
// general's `chatbot_channel_filter: "none"` (match-every-channel) and
// surface the unfiltered model as a direct chatbot tool everywhere the
// instant agents.triggers.enabled flips on. A child that wants a trigger
// must declare it explicitly. (Persona, caps, palette, and tools are
// inherited above — those are capability, not activation.)
// DefaultEmoji: child wins if set; otherwise inherit.
if child.DefaultEmoji == "" {
child.DefaultEmoji = parent.DefaultEmoji
}
// Maps: merge — parent is the base, child entries override.
if child.StateReactEmoji == nil && parent.StateReactEmoji != nil {
child.StateReactEmoji = make(map[string]string, len(parent.StateReactEmoji))
for k, v := range parent.StateReactEmoji {
child.StateReactEmoji[k] = v
}
} else if parent.StateReactEmoji != nil {
merged := make(map[string]string, len(parent.StateReactEmoji)+len(child.StateReactEmoji))
for k, v := range parent.StateReactEmoji {
merged[k] = v
}
for k, v := range child.StateReactEmoji {
merged[k] = v
}
child.StateReactEmoji = merged
}
// SystemPromptPrepend: prepend to the (now resolved) SystemPrompt.
if child.SystemPromptPrepend != "" {
child.SystemPrompt = child.SystemPromptPrepend + "\n\n" + child.SystemPrompt
child.SystemPromptPrepend = "" // consumed
}
// Clear Extends — the resolution is complete, the persisted agent
// is standalone.
child.Extends = ""
}
// agentUpsertAction reports what upsertBuiltinAgent did. Exported only
// to the test in this package; the loader's public surface returns a
// count, not a per-row action.
type agentUpsertAction int
const (
agentUpsertCreated agentUpsertAction = iota
agentUpsertUpdated
agentUpsertSkipped // reserved; current loader never returns this — every parse-OK row is upserted
)
// upsertBuiltinAgent looks up an existing (BuiltinAgentOwnerID, name)
// row. If absent, inserts a new row with a freshly-minted UUID.
// Otherwise updates the existing row in place, preserving ID + CreatedAt.
//
// Why not version-skip like skills.upsertBuiltin: the Agent struct has
// a Version int field but it's a monotonic counter, not a semver
// string for change detection. Agent YAML doesn't carry a "version"
// at the wire shape; every boot writes the latest YAML content,
// trusting the YAML file in-repo IS the source of truth. The Agent's
// internal Version int auto-increments on each loader pass so admin
// inspection (`.agent show`) reveals "how many times has the loader
// touched this row".
func upsertBuiltinAgent(ctx context.Context, store Storage, fresh *Agent) (agentUpsertAction, error) {
existing, err := store.GetAgentByName(ctx, BuiltinAgentOwnerID, fresh.Name)
if err != nil && !errors.Is(err, ErrNotFound) {
return agentUpsertCreated, fmt.Errorf("lookup builtin agent %q: %w", fresh.Name, err)
}
if errors.Is(err, ErrNotFound) {
fresh.ID = uuid.New().String()
fresh.OwnerID = BuiltinAgentOwnerID
fresh.AuthoredBy = BuiltinAgentOwnerID
if fresh.Version == 0 {
fresh.Version = 1
}
now := time.Now()
fresh.CreatedAt = now
fresh.UpdatedAt = now
if saveErr := store.SaveAgent(ctx, fresh); saveErr != nil {
return agentUpsertCreated, saveErr
}
slog.Info("agents: created builtin", "name", fresh.Name, "id", fresh.ID)
return agentUpsertCreated, nil
}
// Update in place. Preserve ID, OwnerID, AuthoredBy, CreatedAt.
// Bump Version so admins can see "the loader has touched this N
// times" — useful when investigating a builtin that was
// hand-edited via the future web UI and unexpectedly reverted on
// next boot.
fresh.ID = existing.ID
fresh.OwnerID = BuiltinAgentOwnerID
fresh.AuthoredBy = BuiltinAgentOwnerID
fresh.Version = existing.Version + 1
fresh.CreatedAt = existing.CreatedAt
fresh.UpdatedAt = time.Now()
// Carry forward operator/scheduler-owned fields that the manifest
// never sets (decodeAgentManifest leaves these zero by design — a
// secret in-repo would be a credential leak). Without this, every
// boot CLOBBERS an operator-armed webhook secret + signature flag
// back to empty/false and nukes the scheduler's next-fire cursor, so
// a scheduled or webhook-armed builtin silently breaks on each deploy.
fresh.WebhookSecret = existing.WebhookSecret
fresh.WebhookSignatureRequired = existing.WebhookSignatureRequired
fresh.NextRunAt = existing.NextRunAt
fresh.LastScheduledRunAt = existing.LastScheduledRunAt
if saveErr := store.SaveAgent(ctx, fresh); saveErr != nil {
return agentUpsertUpdated, saveErr
}
slog.Info("agents: updated builtin",
"name", fresh.Name, "id", fresh.ID, "version", fresh.Version)
return agentUpsertUpdated, nil
}
// builtinAgentManifest is the YAML wire format for agents/<name>/agent.yml.
// The schema is intentionally a SUBSET of the Agent struct — future
// fields can be added without breaking existing manifests so long as
// we keep KnownFields(true) decoding (so a typo on a key surfaces as
// an error rather than silently dropping data).
//
// See pkg/logic/agents/CLAUDE.md for the schema reference.
type builtinAgentManifest struct {
Name string `yaml:"name"`
Description string `yaml:"description"`
ModelTier string `yaml:"model_tier"`
SystemPrompt string `yaml:"system_prompt"`
SystemPromptPrepend string `yaml:"system_prompt_prepend"`
MaxIterations int `yaml:"max_iterations"`
MaxToolCalls int `yaml:"max_tool_calls"`
MaxRuntimeSeconds int `yaml:"max_runtime_seconds"`
ExecutionLane string `yaml:"execution_lane"`
EncryptionEnabled bool `yaml:"encryption_enabled"`
// Run-critic two-tier timeout. CriticEnabled flips MaxRuntime from a
// hard kill into a soft trigger; CriticBackstopMultiplier (0 => convar
// default 6×) sets the hard backstop = MaxRuntime × multiplier.
CriticEnabled bool `yaml:"critic_enabled"`
CriticBackstopMultiplier float64 `yaml:"critic_backstop_multiplier"`
// Extends names a parent agent whose fields are inherited. The
// child's non-zero fields override the parent; nil/empty slices
// inherit the parent's. Maps (state_react) are merged — child
// entries override parent entries with the same key. Only single-
// level extends is supported (no chains).
Extends string `yaml:"extends"`
SkillPalette []string `yaml:"skill_palette"`
SubAgentPalette []string `yaml:"sub_agent_palette"`
LowLevelTools []string `yaml:"low_level_tools"`
PersonalizationSources []string `yaml:"personalization_sources"`
// Triggers — builtin agents typically don't ship with triggers
// (admins flip these on per-deployment), but the keys are accepted
// so a sufficiently sophisticated builtin (e.g. a scheduled "weekly
// digest" agent) can ship triggers in-repo. Default empty.
Schedule string `yaml:"schedule"`
WebhookIPAllowlist []string `yaml:"webhook_ip_allowlist"`
ChatbotChannelFilter string `yaml:"chatbot_channel_filter"`
DefaultEmoji string `yaml:"default_emoji"`
StateReact map[string]string `yaml:"state_react"`
Tags []string `yaml:"tags"`
// Pipeline phases — when non-empty, the executor runs the
// sequential phase runner instead of the single agent loop.
Phases []builtinAgentPhaseManifest `yaml:"phases"`
}
// builtinAgentPhaseManifest is the YAML wire format for a single
// phases list entry in agents/<name>/agent.yml. Maps 1:1 to
// AgentPhase at decode time.
type builtinAgentPhaseManifest struct {
Name string `yaml:"name"`
SystemPrompt string `yaml:"system_prompt"`
ModelTier string `yaml:"model_tier"`
MaxIter int `yaml:"max_iter"`
Tools []string `yaml:"tools"`
Optional bool `yaml:"optional"`
FallbackMessage string `yaml:"fallback_message"`
IsRunFunc bool `yaml:"is_run_func"`
}
// decodeAgentManifest parses an agent.yml bundle into a domain Agent.
// Uses KnownFields(true) so a typo'd key surfaces as a parse error
// rather than silently dropping the value.
//
// What this method does NOT set:
// - ID (loader mints UUID on insert / preserves existing on update)
// - OwnerID + AuthoredBy (loader sets to BuiltinAgentOwnerID)
// - Version (loader increments on update)
// - CreatedAt + UpdatedAt (loader stamps)
// - WebhookSecret (operator generates via admin tooling at deploy
// time — shipping a secret in-repo would be a credential leak)
// - NextRunAt + LastScheduledRunAt (scheduler bookkeeping; nil at
// load time, populated on first scheduled fire)
// - WebhookSignatureRequired (application-layer default applies on
// first save; a `default:true` GORM tag would substitute on every
// write — see the v8 lesson on this exact trap)
func decodeAgentManifest(data []byte) (*Agent, error) {
var m builtinAgentManifest
dec := yaml.NewDecoder(strings.NewReader(string(data)))
dec.KnownFields(true)
if err := dec.Decode(&m); err != nil {
return nil, fmt.Errorf("decode agent.yml: %w", err)
}
if strings.TrimSpace(m.Name) == "" {
return nil, errors.New("agent.yml: missing required field 'name'")
}
// system_prompt is required UNLESS the agent uses extends (the parent
// will supply it) or system_prompt_prepend (the prepend will be
// combined with the parent's system_prompt after extends resolution).
if strings.TrimSpace(m.SystemPrompt) == "" && strings.TrimSpace(m.Extends) == "" && strings.TrimSpace(m.SystemPromptPrepend) == "" {
return nil, errors.New("agent.yml: missing required field 'system_prompt'")
}
// Convert YAML phase manifests to domain AgentPhase structs.
var phases []AgentPhase
for _, pm := range m.Phases {
if strings.TrimSpace(pm.Name) == "" {
return nil, errors.New("agent.yml: phase missing required field 'name'")
}
phases = append(phases, AgentPhase{
Name: strings.TrimSpace(pm.Name),
SystemPrompt: pm.SystemPrompt,
ModelTier: strings.TrimSpace(pm.ModelTier),
MaxIter: pm.MaxIter,
Tools: pm.Tools,
Optional: pm.Optional,
FallbackMessage: pm.FallbackMessage,
IsRunFunc: pm.IsRunFunc,
})
}
// Validate the webhook IP allow-list (CIDR or bare IP); drop + warn on
// malformed entries so a typo can't silently widen or void the allow-list.
allowlist := validateIPAllowlist(m.WebhookIPAllowlist, m.Name)
ag := &Agent{
Name: strings.TrimSpace(m.Name),
Description: m.Description,
Extends: strings.TrimSpace(m.Extends),
SystemPromptPrepend: m.SystemPromptPrepend,
ModelTier: strings.TrimSpace(m.ModelTier),
SystemPrompt: m.SystemPrompt,
MaxIterations: m.MaxIterations,
MaxToolCalls: m.MaxToolCalls,
MaxRuntime: time.Duration(m.MaxRuntimeSeconds) * time.Second,
ExecutionLane: strings.TrimSpace(m.ExecutionLane),
EncryptionEnabled: m.EncryptionEnabled,
CriticEnabled: m.CriticEnabled,
CriticBackstopMultiplier: m.CriticBackstopMultiplier,
SkillPalette: m.SkillPalette,
SubAgentPalette: m.SubAgentPalette,
LowLevelTools: m.LowLevelTools,
PersonalizationSources: m.PersonalizationSources,
Schedule: strings.TrimSpace(m.Schedule),
WebhookIPAllowlist: allowlist,
ChatbotChannelFilter: strings.TrimSpace(m.ChatbotChannelFilter),
DefaultEmoji: m.DefaultEmoji,
StateReactEmoji: m.StateReact,
Tags: m.Tags,
Phases: phases,
}
return ag, nil
}
// validateIPAllowlist keeps only entries that parse as a CIDR block or a bare
// IP; malformed entries are dropped with a warning (a typo must not silently
// widen or void the webhook allow-list). The struct field documents "CIDR
// strings", so this enforces it at load time.
func validateIPAllowlist(entries []string, agent string) []string {
var out []string
for _, e := range entries {
e = strings.TrimSpace(e)
if e == "" {
continue
}
if _, _, err := net.ParseCIDR(e); err == nil {
out = append(out, e)
continue
}
if ip := net.ParseIP(e); ip != nil {
out = append(out, e)
continue
}
slog.Warn("agents: dropping malformed webhook_ip_allowlist entry (not a CIDR or IP)", "agent", agent, "entry", e)
}
return out
}
+17
View File
@@ -0,0 +1,17 @@
package persona
import "testing"
func TestValidateIPAllowlist(t *testing.T) {
in := []string{"10.0.0.0/8", " 192.168.1.5 ", "not-an-ip", "", "2001:db8::/32", "garbage/99"}
got := validateIPAllowlist(in, "test")
want := map[string]bool{"10.0.0.0/8": true, "192.168.1.5": true, "2001:db8::/32": true}
if len(got) != len(want) {
t.Fatalf("got %v, want %d valid entries", got, len(want))
}
for _, e := range got {
if !want[e] {
t.Errorf("unexpected entry kept: %q", e)
}
}
}
+120
View File
@@ -0,0 +1,120 @@
package persona
import (
"context"
"sort"
"sync"
"time"
)
// Memory is a zero-dependency in-process Storage for agent personas — a light
// host (or tests) gets persona persistence with no DB. Mort keeps its
// GORM/MySQL Storage; contrib/store adds a durable SQLite one.
type Memory struct {
mu sync.RWMutex
agents map[string]*Agent // by ID
}
// NewMemory returns an empty in-memory persona Storage.
func NewMemory() *Memory { return &Memory{agents: map[string]*Agent{}} }
var _ Storage = (*Memory)(nil)
func (m *Memory) InitializeAgentStorage(context.Context) error { return nil }
func (m *Memory) SaveAgent(_ context.Context, a *Agent) error {
m.mu.Lock()
defer m.mu.Unlock()
cp := *a
m.agents[a.ID] = &cp
return nil
}
func (m *Memory) GetAgent(_ context.Context, id string) (*Agent, error) {
m.mu.RLock()
defer m.mu.RUnlock()
a, ok := m.agents[id]
if !ok {
return nil, ErrNotFound
}
cp := *a
return &cp, nil
}
func (m *Memory) GetAgentByName(_ context.Context, ownerID, name string) (*Agent, error) {
m.mu.RLock()
defer m.mu.RUnlock()
for _, a := range m.agents {
if a.OwnerID == ownerID && a.Name == name {
cp := *a
return &cp, nil
}
}
return nil, ErrNotFound
}
func (m *Memory) listWhere(keep func(*Agent) bool) []*Agent {
m.mu.RLock()
defer m.mu.RUnlock()
out := make([]*Agent, 0, len(m.agents))
for _, a := range m.agents {
if keep == nil || keep(a) {
cp := *a
out = append(out, &cp)
}
}
sort.Slice(out, func(i, j int) bool { return out[i].Name < out[j].Name })
return out
}
func (m *Memory) ListAgents(_ context.Context, ownerID string) ([]*Agent, error) {
return m.listWhere(func(a *Agent) bool { return a.OwnerID == ownerID }), nil
}
func (m *Memory) ListAllAgents(context.Context) ([]*Agent, error) {
return m.listWhere(nil), nil
}
func (m *Memory) DeleteAgent(_ context.Context, id string) error {
m.mu.Lock()
defer m.mu.Unlock()
delete(m.agents, id)
return nil
}
func (m *Memory) GetAgentByWebhookSecret(_ context.Context, secret string) (*Agent, error) {
if secret == "" {
return nil, ErrNotFound
}
m.mu.RLock()
defer m.mu.RUnlock()
for _, a := range m.agents {
if a.WebhookSecret == secret {
cp := *a
return &cp, nil
}
}
return nil, ErrNotFound
}
func (m *Memory) ListAgentsByChatbotChannelFilter(context.Context) ([]*Agent, error) {
return m.listWhere(func(a *Agent) bool { return a.ChatbotChannelFilter != "" }), nil
}
func (m *Memory) ListScheduledAgents(_ context.Context, dueBefore time.Time) ([]*Agent, error) {
return m.listWhere(func(a *Agent) bool {
return a.Schedule != "" && a.NextRunAt != nil && !a.NextRunAt.After(dueBefore)
}), nil
}
func (m *Memory) MarkAgentScheduledRun(_ context.Context, agentID string, ranAt, nextAt time.Time) error {
m.mu.Lock()
defer m.mu.Unlock()
a, ok := m.agents[agentID]
if !ok {
return ErrNotFound
}
a.LastScheduledRunAt = &ranAt
a.NextRunAt = &nextAt
return nil
}
+45
View File
@@ -0,0 +1,45 @@
package persona
import (
"context"
"testing"
"time"
)
func TestToRunnable(t *testing.T) {
a := &Agent{
ID: "id1", Name: "helper", SystemPrompt: "be nice", ModelTier: "fast",
MaxIterations: 5, MaxRuntime: 30 * time.Second,
LowLevelTools: []string{"think"}, SkillPalette: []string{"animate"},
CriticEnabled: true, CriticBackstopMultiplier: 2,
Phases: []AgentPhase{{Name: "p1", ModelTier: "thinking", MaxIter: 3, Tools: []string{"now"}, Optional: true}},
}
r := a.ToRunnable()
if r.ID != "id1" || r.ModelTier != "fast" || r.MaxIterations != 5 || !r.Critic.Enabled {
t.Fatalf("ToRunnable mapping wrong: %+v", r)
}
if len(r.Phases) != 1 || r.Phases[0].MaxIterations != 3 || !r.Phases[0].Optional {
t.Fatalf("phase mapping wrong: %+v", r.Phases)
}
}
func TestMemoryStoreRoundTrip(t *testing.T) {
ctx := context.Background()
m := NewMemory()
a := &Agent{ID: "a1", Name: "n", OwnerID: "o1"}
if err := m.SaveAgent(ctx, a); err != nil {
t.Fatal(err)
}
got, err := m.GetAgent(ctx, "a1")
if err != nil || got.Name != "n" {
t.Fatalf("GetAgent: %v %+v", err, got)
}
byName, err := m.GetAgentByName(ctx, "o1", "n")
if err != nil || byName.ID != "a1" {
t.Fatalf("GetAgentByName: %v %+v", err, byName)
}
list, _ := m.ListAgents(ctx, "o1")
if len(list) != 1 {
t.Fatalf("ListAgents = %d", len(list))
}
}
+37
View File
@@ -0,0 +1,37 @@
package persona
import "gitea.stevedudenhoeffer.com/steve/executus/run"
// ToRunnable lowers an Agent persona into the kernel's run.RunnableAgent DTO —
// the bridge that lets run.Executor run a persona WITHOUT importing this
// battery (the inversion of mort's agentexec.Run(*agents.Agent)). It maps the
// static shape only; per-run personalization, palette resolution, the critic,
// audit, etc. are supplied separately via run.Ports.
func (a *Agent) ToRunnable() run.RunnableAgent {
ra := run.RunnableAgent{
ID: a.ID,
Name: a.Name,
SystemPrompt: a.SystemPrompt,
ModelTier: a.ModelTier,
MaxIterations: a.MaxIterations,
MaxRuntime: a.MaxRuntime,
LowLevelTools: a.LowLevelTools,
SkillPalette: a.SkillPalette,
SubAgentPalette: a.SubAgentPalette,
Critic: run.CriticConfig{
Enabled: a.CriticEnabled,
BackstopMultiplier: a.CriticBackstopMultiplier,
},
}
for _, p := range a.Phases {
ra.Phases = append(ra.Phases, run.Phase{
Name: p.Name,
SystemPrompt: p.SystemPrompt,
ModelTier: p.ModelTier,
MaxIterations: p.MaxIter,
Tools: p.Tools,
Optional: p.Optional,
})
}
return ra
}
+115
View File
@@ -0,0 +1,115 @@
package persona
import (
"context"
"errors"
"time"
)
// ErrNotFound is returned when an agent lookup fails. Callers compare
// with errors.Is(err, ErrNotFound).
var ErrNotFound = errors.New("agent not found")
// Storage is the persistence interface for the agents system.
//
// Why: tests substitute fake implementations; production wires
// through pkg/logic/storage's Grand Storage which embeds this
// interface. Mirrors the three-layer pattern in
// pkg/logic/storage/CLAUDE.md (domain → GORM → DB).
//
// What: Phase 1 CRUD plus Phase 3 trigger queries
// (ListDueScheduled, GetAgentByWebhookSecret,
// ListAgentsByChatbotChannelFilter, MarkScheduledRun). Trigger
// queries are read by the agentsched runner, webhook router, and
// chatbot tool provider; all are gated behind the
// agents.triggers.enabled convar so old skill-driven paths keep
// running until the convar flips.
//
// Test: see storage_round_trip_test.go for round-trip coverage.
type Storage interface {
// (Mort's Discord command-binding CRUD — the CommandBindingStorage
// embedding — stays a host concern and is NOT part of the executus
// persona Storage seam.)
// InitializeAgentStorage prepares storage (e.g. AutoMigrate)
// and is idempotent. Called from the grand storage's
// InitializeAll path.
InitializeAgentStorage(ctx context.Context) error
// SaveAgent creates or updates an Agent by ID. ID must be
// non-empty (Phase 1 admin commands mint a UUID).
SaveAgent(ctx context.Context, a *Agent) error
// GetAgent returns the agent with the given ID, or ErrNotFound.
GetAgent(ctx context.Context, id string) (*Agent, error)
// GetAgentByName resolves (owner_id, name) → agent. ownerID
// must match exactly (Phase 1 has no shared/public visibility
// yet; every agent is owned).
GetAgentByName(ctx context.Context, ownerID, name string) (*Agent, error)
// ListAgents returns every agent owned by the given member ID,
// sorted by Name ASC.
ListAgents(ctx context.Context, ownerID string) ([]*Agent, error)
// ListAllAgents returns every agent across all owners, sorted by
// (OwnerID ASC, Name ASC) so builtin rows (OwnerID="builtin")
// group together, then numeric Discord-ID owners in lexical order,
// then chatbot-shadow rows whose OwnerID is the chatbot owner's
// Discord ID but whose Name carries the "chatbot:" prefix.
//
// Why: Phase 1 admin commands ran owner-scoped (a steve-owned
// agent list shows ONLY steve's rows), which hid builtin and
// shadow Agents from the admin view. `.agent list` for admins now
// uses this method to surface every row. Non-admin invocations
// (or `.agent list --mine`) keep using ListAgents.
//
// Storage MAY back this with a single full-table scan — admin
// row counts are small (dozens to low hundreds), so no need for
// pagination at this phase.
ListAllAgents(ctx context.Context) ([]*Agent, error)
// DeleteAgent removes an agent by ID. Idempotent — deleting a
// missing row returns nil.
DeleteAgent(ctx context.Context, id string) error
// GetAgentByWebhookSecret resolves a posted /webhooks/<secret> URL
// to the matching agent. Returns ErrNotFound when no agent has
// the secret. Phase 3 webhook router consults this AFTER the
// existing Skill lookup falls through, but only when
// agents.triggers.enabled is true.
//
// Empty secret is rejected with ErrNotFound (empty WebhookSecret
// rows are NOT webhook-enabled — the application layer guards
// this, the lookup defends against accidental match).
GetAgentByWebhookSecret(ctx context.Context, secret string) (*Agent, error)
// ListAgentsByChatbotChannelFilter returns every agent with a
// non-empty ChatbotChannelFilter. Phase 3 chatbot tool provider
// uses this on every chatbot turn to assemble the per-channel
// tool list (gated by agents.triggers.enabled). The result is
// not channel-filtered here — the provider applies the channel
// filter predicate (registered in skills.ChannelFilterRegistry)
// to each row.
//
// Why no channel filter at the storage layer: the filter is a
// runtime predicate (e.g. dm_only depends on the live Discord
// channel kind cache), not a static column we can index on.
ListAgentsByChatbotChannelFilter(ctx context.Context) ([]*Agent, error)
// ListScheduledAgents returns every agent with a non-empty
// Schedule whose NextRunAt is at or before `dueBefore`. Result
// is ordered by NextRunAt ASC so the scheduler runner can drain
// in oldest-due-first order. Mirrors skills.Storage.ListDueScheduled.
//
// Phase 3 scheduler reads this on every tick when
// agents.triggers.enabled is true. The (Schedule, NextRunAt)
// composite index backs the query — see gorm tags on gormAgent.
ListScheduledAgents(ctx context.Context, dueBefore time.Time) ([]*Agent, error)
// MarkAgentScheduledRun atomically updates LastScheduledRunAt
// and NextRunAt for the given agent. Called by the agentsched
// runner after each scheduled invocation. Mirrors
// skills.Storage.MarkScheduledRun semantics.
MarkAgentScheduledRun(ctx context.Context, agentID string, ranAt, nextAt time.Time) error
}
+76
View File
@@ -0,0 +1,76 @@
package run
import "time"
// RunnableAgent is the kernel's view of "a thing to run": an identity, a model
// tier, a system prompt, execution caps, and a tool palette. It is a plain DTO
// on purpose — the run kernel never imports a noun battery. The persona Agent
// and the saved Skill each LOWER themselves into a RunnableAgent (a ToRunnable
// method on the battery side), and the kernel runs the DTO. This is the
// inversion of mort's agentexec.Executor.Run(*agents.Agent): the executor no
// longer depends on the persona struct, only on this shape.
//
// A light host can build a RunnableAgent inline (model tier + prompt + a few
// tool names) for a one-shot bounded run, with no persona or skill battery at
// all — that is exactly gadfly's swarm task.
type RunnableAgent struct {
// ID is a stable identifier for the run subject (an agent/skill UUID, or
// any host-chosen id). Used for audit attribution and dispatch-guard
// genealogy. Empty is allowed for anonymous one-shot runs.
ID string
// Name is a human label (audit/logs/delivery). Empty is allowed.
Name string
// SystemPrompt is the agent's base system prompt (before per-run
// personalization, which a host layers via Ports).
SystemPrompt string
// ModelTier is a tier alias or concrete spec resolved through
// model.ParseModelForContext. Empty resolves to the host's default tier.
ModelTier string
// MaxIterations caps the agent loop's tool-dispatch steps. 0 = kernel
// default. MaxRuntime caps wall-clock for the whole run (the kernel starts
// this clock AFTER any lane dequeue, not at submission). 0 = kernel
// default.
MaxIterations int
MaxRuntime time.Duration
// LowLevelTools are tool-registry names the run may call directly.
// SkillPalette / SubAgentPalette name saved skills / sub-agents exposed as
// skill__<name> / agent__<name> delegation tools, resolved through
// Ports.Palette (nil Palette => those entries are inert).
LowLevelTools []string
SkillPalette []string
SubAgentPalette []string
// Phases optionally model a multi-step pipeline (each phase its own prompt
// + tier + tools). An empty slice is a single-phase run — the common case.
Phases []Phase
// Critic configures the optional two-tier run-critic (Ports.Critic). The
// zero value (disabled) is the light-host default.
Critic CriticConfig
}
// Phase is one step of a multi-step run: its own system prompt, model tier,
// iteration cap, and tool subset. Optional phases may be skipped by the
// pipeline when their precondition isn't met.
type Phase struct {
Name string
SystemPrompt string
ModelTier string
MaxIterations int
Tools []string
Optional bool
}
// CriticConfig configures the optional run-critic. Enabled gates whether a
// critic monitor is started at all; BackstopMultiplier sets the hard-kill
// deadline as a multiple of the soft trigger (MaxRuntime). A non-positive
// multiplier uses the kernel default.
type CriticConfig struct {
Enabled bool
BackstopMultiplier float64
}
+124
View File
@@ -0,0 +1,124 @@
package run
import (
"context"
"fmt"
"time"
"gitea.stevedudenhoeffer.com/steve/majordomo/agent"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
)
// criticDeadlineCheck is how often the deadline-watch goroutine polls the
// critic's hard deadline. Small relative to any realistic soft timeout.
const criticDeadlineCheck = time.Second
// criticBinding wires a CriticHandle into a run: the executor forwards activity
// (steps + tool starts) to it, binds the run's hard cancellation to the critic's
// extendable deadline, and exposes the critic's Steer messages as an agent
// RunOption. All methods are nil-safe so the executor can call them
// unconditionally when no critic is configured.
type criticBinding struct {
h CriticHandle
}
// startCritic begins critic monitoring for this run when one is configured and
// the agent enables it. It launches a goroutine that cancels runCtx (via
// cancelCause) the moment the critic's hard deadline passes — the critic may
// extend that deadline, so a healthy-but-slow run is given room while a hung one
// is killed. When the deadline passes because the critic KILLED the run
// (KillCause() != nil), the cancellation cause is ErrCriticKill (→ status
// "killed"); when the backstop simply expired, it is context.DeadlineExceeded (→
// "timeout"). Returns (nil, no-op stop) when there is no critic. The caller MUST
// defer the returned stop.
func (e *Executor) startCritic(runCtx context.Context, cancelCause context.CancelCauseFunc, ra RunnableAgent, info RunInfo) (*criticBinding, func()) {
noop := func() {}
if e.cfg.Ports.Critic == nil || !ra.Critic.Enabled {
return nil, noop
}
soft := e.cfg.Defaults.CriticSoftTimeout
if soft <= 0 {
soft = 90 * time.Second // defensive: withFallbacks normally guarantees >0
}
h := e.cfg.Ports.Critic.Monitor(runCtx, info, soft)
if h == nil {
return nil, noop
}
done := make(chan struct{})
go func() {
// A host CriticHandle.Deadline() that panics must not crash the process
// (this runs on its own goroutine, so the executor's top-level recover
// can't catch it). Log-free best-effort: just stop watching.
defer func() { _ = recover() }()
t := time.NewTicker(criticDeadlineCheck)
defer t.Stop()
for {
select {
case <-done:
return
case <-runCtx.Done():
return
case <-t.C:
// A zero deadline = no hard cap (not yet set); otherwise cancel
// once we're at or past it, distinguishing an explicit kill from a
// natural backstop expiry so the run gets the right status.
if d := h.Deadline(); !d.IsZero() && !time.Now().Before(d) {
if cause := h.KillCause(); cause != nil {
cancelCause(fmt.Errorf("%w: %s", ErrCriticKill, cause.Error()))
} else {
cancelCause(context.DeadlineExceeded)
}
return
}
}
}
}()
return &criticBinding{h: h}, func() {
close(done)
h.Stop()
}
}
func (b *criticBinding) recordStep(iter int, resp *llm.Response) {
if b != nil {
b.h.RecordStep(iter, resp)
}
}
// recordToolStart forwards a tool call to the critic. NOTE: majordomo's step
// observer only fires AFTER an iteration completes, so this currently lands
// post-tool, not at dispatch — the activity clock is refreshed once per
// iteration, not mid-tool. A single very long tool call (e.g. a 30-min render)
// therefore won't refresh the clock until it returns; a host that runs such
// tools should feed interim progress to its Critic (mort's InstallProgressBridge
// pattern). A true pre-dispatch refresh needs a majordomo hook (follow-up).
func (b *criticBinding) recordToolStart(name, args string) {
if b != nil {
b.h.RecordToolStart(name, args)
}
}
// maxStepsOption returns the agent step-ceiling Option. With no critic it's a
// fixed WithMaxSteps(base); with a critic it's a DYNAMIC WithMaxStepsFunc that
// polls the handle each step (so the critic can raise a long run's budget),
// falling back to base when the handle defers (MaxSteps() <= 0).
func (b *criticBinding) maxStepsOption(base int) agent.Option {
if b == nil {
return agent.WithMaxSteps(base)
}
return agent.WithMaxStepsFunc(func() int {
if n := b.h.MaxSteps(); n > 0 {
return n
}
return base
})
}
// steerOptions returns the agent RunOptions that drain the critic's steer
// messages into the loop. Empty when there is no critic.
func (b *criticBinding) steerOptions() []agent.RunOption {
if b == nil {
return nil
}
return []agent.RunOption{agent.WithSteer(b.h.Steer)}
}
+128
View File
@@ -0,0 +1,128 @@
package run_test
import (
"context"
"sync"
"testing"
"time"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/majordomo/provider/fake"
"gitea.stevedudenhoeffer.com/steve/executus/run"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
type fakeCritic struct{ h *fakeCriticHandle }
func (c *fakeCritic) Monitor(_ context.Context, _ run.RunInfo, _ time.Duration) run.CriticHandle {
return c.h
}
type fakeCriticHandle struct {
mu sync.Mutex
steps, tools, stops int
steered int
maxSteps int // 0 => defer to the run's base MaxIterations
killCause error // non-nil simulates a critic kill
}
func (h *fakeCriticHandle) RecordStep(int, *llm.Response) { h.mu.Lock(); h.steps++; h.mu.Unlock() }
func (h *fakeCriticHandle) KillCause() error {
h.mu.Lock()
defer h.mu.Unlock()
return h.killCause
}
func (h *fakeCriticHandle) RecordToolStart(string, string) {
h.mu.Lock()
h.tools++
h.mu.Unlock()
}
func (h *fakeCriticHandle) Steer() []llm.Message { h.mu.Lock(); h.steered++; h.mu.Unlock(); return nil }
func (h *fakeCriticHandle) Deadline() time.Time { return time.Time{} } // no hard deadline
func (h *fakeCriticHandle) MaxSteps() int { h.mu.Lock(); defer h.mu.Unlock(); return h.maxSteps }
func (h *fakeCriticHandle) Stop() { h.mu.Lock(); h.stops++; h.mu.Unlock() }
// TestCriticRaisesStepCeiling: a critic returning a higher MaxSteps lets the agent
// run PAST its base MaxIterations (the dynamic step ceiling). With base=1 and no
// critic the run would hit ErrMaxSteps after the first tool-dispatch step; the
// critic raises it to 5 so the run completes.
func TestCriticRaisesStepCeiling(t *testing.T) {
h := &fakeCriticHandle{maxSteps: 5}
fp := fake.New("fake")
fp.Enqueue("m",
// two tool-call steps (unknown tool → tolerated error results), then answer
fake.ReplyWith(llm.Response{ToolCalls: []llm.ToolCall{{ID: "c1", Name: "noop", Arguments: []byte(`{}`)}}}),
fake.ReplyWith(llm.Response{ToolCalls: []llm.ToolCall{{ID: "c2", Name: "noop", Arguments: []byte(`{}`)}}}),
fake.Reply("done after 2 tool steps"),
)
m, _ := fp.Model("m")
ex := run.New(run.Config{
Registry: tool.NewRegistry(),
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) { return ctx, m, nil },
Ports: run.Ports{Critic: &fakeCritic{h: h}},
// large soft timeout so the deadline-watch never interferes in the test
Defaults: run.Defaults{CriticSoftTimeout: time.Hour},
})
res := ex.Run(context.Background(),
run.RunnableAgent{Name: "x", ModelTier: "m", MaxIterations: 1, Critic: run.CriticConfig{Enabled: true}},
tool.Invocation{RunID: "r"}, "go")
if res.Err != nil {
t.Fatalf("critic raised the ceiling to 5, run should complete past base=1: %v", res.Err)
}
if res.Output != "done after 2 tool steps" {
t.Errorf("output = %q", res.Output)
}
}
// TestCriticWired: an agent with Critic.Enabled gets monitored — Monitor returns
// a handle the executor feeds (RecordStep), drains (Steer), and stops.
func TestCriticWired(t *testing.T) {
h := &fakeCriticHandle{}
fp := fake.New("fake")
fp.Enqueue("m", fake.Reply("done"))
m, _ := fp.Model("m")
ex := run.New(run.Config{
Registry: tool.NewRegistry(),
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) { return ctx, m, nil },
Ports: run.Ports{Critic: &fakeCritic{h: h}},
})
res := ex.Run(context.Background(),
run.RunnableAgent{Name: "watched", ModelTier: "m", Critic: run.CriticConfig{Enabled: true}},
tool.Invocation{RunID: "r"}, "go")
if res.Err != nil {
t.Fatalf("run error: %v", res.Err)
}
h.mu.Lock()
defer h.mu.Unlock()
if h.steps < 1 {
t.Errorf("critic should have seen >=1 step, got %d", h.steps)
}
if h.steered < 1 {
t.Errorf("critic Steer should be drained at least once, got %d", h.steered)
}
if h.stops != 1 {
t.Errorf("critic Stop should be called exactly once, got %d", h.stops)
}
}
// TestCriticDisabledNotMonitored: Critic.Enabled=false → Monitor never called.
func TestCriticDisabledNotMonitored(t *testing.T) {
h := &fakeCriticHandle{}
fp := fake.New("fake")
fp.Enqueue("m", fake.Reply("done"))
m, _ := fp.Model("m")
ex := run.New(run.Config{
Registry: tool.NewRegistry(),
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) { return ctx, m, nil },
Ports: run.Ports{Critic: &fakeCritic{h: h}},
})
ex.Run(context.Background(),
run.RunnableAgent{Name: "x", ModelTier: "m"}, // Critic.Enabled=false
tool.Invocation{RunID: "r"}, "go")
h.mu.Lock()
defer h.mu.Unlock()
if h.stops != 0 || h.steps != 0 {
t.Errorf("disabled critic should not be monitored: steps=%d stops=%d", h.steps, h.stops)
}
}
+114
View File
@@ -0,0 +1,114 @@
package run_test
import (
"context"
"errors"
"testing"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/majordomo/provider/fake"
"gitea.stevedudenhoeffer.com/steve/executus/deliver"
"gitea.stevedudenhoeffer.com/steve/executus/run"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
type recordingDelivery struct {
target deliver.Target
output string
errored error
delivers int
}
func (d *recordingDelivery) Deliver(_ context.Context, t deliver.Target, output string, _ []deliver.Artifact) (string, error) {
d.target, d.output, d.delivers = t, output, d.delivers+1
return "msg-1", nil
}
func (d *recordingDelivery) DeliverError(_ context.Context, t deliver.Target, e error) error {
d.target, d.errored = t, e
return nil
}
func TestDeliveryWired(t *testing.T) {
d := &recordingDelivery{}
fp := fake.New("fake")
fp.Enqueue("m", fake.Reply("the output"))
m, _ := fp.Model("m")
ex := run.New(run.Config{
Registry: tool.NewRegistry(),
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) { return ctx, m, nil },
Ports: run.Ports{Delivery: d},
})
// With a delivery target, the executor posts the output.
ex.Run(context.Background(),
run.RunnableAgent{Name: "x", ModelTier: "m"},
tool.Invocation{RunID: "r", DeliveryKind: "channel", DeliveryID: "chan-9"}, "go")
if d.delivers != 1 || d.output != "the output" || d.target.ID != "chan-9" || d.target.Kind != "channel" {
t.Fatalf("delivery wrong: %+v out=%q", d.target, d.output)
}
}
func TestNoDeliveryWithoutTarget(t *testing.T) {
d := &recordingDelivery{}
fp := fake.New("fake")
fp.Enqueue("m", fake.Reply("x"))
m, _ := fp.Model("m")
ex := run.New(run.Config{
Registry: tool.NewRegistry(),
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) { return ctx, m, nil },
Ports: run.Ports{Delivery: d},
})
// No DeliveryID → executor delivers nothing (caller reads Result.Output).
ex.Run(context.Background(),
run.RunnableAgent{Name: "x", ModelTier: "m"},
tool.Invocation{RunID: "r"}, "go")
if d.delivers != 0 {
t.Errorf("no target should mean no delivery, got %d", d.delivers)
}
}
// TestNoDeliveryOnEarlyResolveError: an error BEFORE the run starts (model
// resolve) returns before delivery is reached — neither Deliver nor DeliverError
// fires. (Delivery covers run OUTCOMES, not pre-run setup failures.)
func TestNoDeliveryOnEarlyResolveError(t *testing.T) {
d := &recordingDelivery{}
ex := run.New(run.Config{
Registry: tool.NewRegistry(),
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) {
return ctx, nil, errors.New("resolve boom")
},
Ports: run.Ports{Delivery: d},
})
ex.Run(context.Background(),
run.RunnableAgent{Name: "x", ModelTier: "m"},
tool.Invocation{RunID: "r", DeliveryKind: "channel", DeliveryID: "chan-9"}, "go")
if d.delivers != 0 || d.errored != nil {
t.Errorf("early resolve failure should neither Deliver nor DeliverError: delivers=%d errored=%v", d.delivers, d.errored)
}
}
// TestDeliverErrorOnRunFailure: an in-loop run failure (the model errors) routes
// through DeliverError with the run error.
func TestDeliverErrorOnRunFailure(t *testing.T) {
d := &recordingDelivery{}
fp := fake.New("fake")
fp.Enqueue("m", fake.Step{Err: errors.New("model boom")}) // model errors mid-run
m, _ := fp.Model("m")
ex := run.New(run.Config{
Registry: tool.NewRegistry(),
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) { return ctx, m, nil },
Ports: run.Ports{Delivery: d},
})
res := ex.Run(context.Background(),
run.RunnableAgent{Name: "x", ModelTier: "m"},
tool.Invocation{RunID: "r", DeliveryKind: "channel", DeliveryID: "chan-9"}, "go")
if res.Err == nil {
t.Fatal("expected a run error")
}
if d.delivers != 0 {
t.Errorf("a failed run should not Deliver (success path), got %d", d.delivers)
}
if d.errored == nil || d.target.ID != "chan-9" {
t.Errorf("a failed run with a target should DeliverError to chan-9, got errored=%v target=%+v", d.errored, d.target)
}
}
+388
View File
@@ -0,0 +1,388 @@
package run
import (
"context"
"errors"
"fmt"
"time"
"gitea.stevedudenhoeffer.com/steve/majordomo/agent"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/executus/compact"
"gitea.stevedudenhoeffer.com/steve/executus/deliver"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
// ModelResolver resolves a tier alias or concrete spec to a usable llm.Model
// and an enriched context (for usage attribution). model.ParseModelForContext
// satisfies it.
type ModelResolver func(ctx context.Context, tier string) (context.Context, llm.Model, error)
// Defaults are the executor's fallback caps and loop guards, applied per run
// when the RunnableAgent leaves a field zero.
type Defaults struct {
MaxIterations int // tool-dispatch steps; default 12
MaxRuntime time.Duration // wall-clock per run; default 60s
FallbackTier string // tier when the agent's is empty; default "fast"
MaxConsecutiveToolErrors int // loop guard; default 3
MaxSameToolCallRepeats int // retry-storm guard; default 3
CompactionThresholdRatio float64 // fraction of model context to compact at; default 0.7
CriticSoftTimeout time.Duration // idle window before the critic wakes; default 90s
}
func (d Defaults) withFallbacks() Defaults {
if d.MaxIterations <= 0 {
d.MaxIterations = 12
}
if d.MaxRuntime <= 0 {
d.MaxRuntime = 60 * time.Second
}
if d.FallbackTier == "" {
d.FallbackTier = "fast"
}
if d.MaxConsecutiveToolErrors <= 0 {
d.MaxConsecutiveToolErrors = 3
}
if d.MaxSameToolCallRepeats <= 0 {
d.MaxSameToolCallRepeats = 3
}
if d.CompactionThresholdRatio <= 0 {
d.CompactionThresholdRatio = 0.7
}
if d.CriticSoftTimeout <= 0 {
d.CriticSoftTimeout = 90 * time.Second
}
return d
}
// Config wires an Executor. Registry + Models are required; everything else is
// optional and nil-safe — the zero Config beyond those yields a bounded,
// in-memory run with no persistence/audit/budget/critic/delegation/compaction
// (gadfly's case).
type Config struct {
Registry tool.Registry
Models ModelResolver
Defaults Defaults
Ports Ports
// Compactor mints the per-run context-compaction hook. nil disables
// compaction. ContextTokens resolves a tier's model context-window (for
// the compaction threshold); nil — or a zero return — also disables it.
Compactor compact.CompactorFactory
ContextTokens func(tier string) int
// SystemHeader is an optional platform header prepended to every agent's
// system prompt.
SystemHeader string
}
// Executor runs a RunnableAgent through majordomo's agent loop with the wired
// Ports. Construct with New; safe for concurrent use across runs.
type Executor struct {
cfg Config
}
// New builds an Executor. It panics if Registry or Models is nil — those are
// structural, not runtime, errors.
func New(cfg Config) *Executor {
if cfg.Registry == nil || cfg.Models == nil {
panic("run.New: Registry and Models are required")
}
cfg.Defaults = cfg.Defaults.withFallbacks()
return &Executor{cfg: cfg}
}
// Result is one run's outcome. Err carries the run failure (if any); the other
// fields are populated best-effort even on error (partial output/steps/usage).
type Result struct {
RunID string
Output string
Steps []tool.Step
Usage llm.Usage
Err error
}
// Run executes ra with the given invocation + input and returns the Result. It
// never propagates a panic; failures surface in Result.Err (a top-level recover
// converts any panic — including from a host Port — into a run error).
func (e *Executor) Run(ctx context.Context, ra RunnableAgent, inv tool.Invocation, input string) (res Result) {
started := time.Now()
res = Result{RunID: inv.RunID}
// Enforce the no-panic contract: a panic anywhere in the run (incl. a host
// Critic/Audit/Palette callback on the main goroutine) becomes Result.Err
// rather than unwinding into the caller.
defer func() {
if r := recover(); r != nil {
res.Err = fmt.Errorf("run.Executor: recovered panic: %v", r)
}
}()
tier := ra.ModelTier
if tier == "" {
tier = e.cfg.Defaults.FallbackTier
}
maxIter := ra.MaxIterations
if maxIter <= 0 {
maxIter = e.cfg.Defaults.MaxIterations
}
maxRuntime := ra.MaxRuntime
if maxRuntime <= 0 {
maxRuntime = e.cfg.Defaults.MaxRuntime
}
// Budget gate (pre-run): a rejected run makes no model call.
if e.cfg.Ports.Budget != nil {
if err := e.cfg.Ports.Budget.Check(ctx, inv.CallerID); err != nil {
res.Err = err
return res
}
}
// Resolve the model (enriches ctx for usage attribution).
modelCtx, model, err := e.cfg.Models(ctx, tier)
if err != nil {
res.Err = fmt.Errorf("resolve model %q: %w", tier, err)
return res
}
if model == nil {
// A resolver returning (ctx, nil, nil) would otherwise nil-panic inside
// the agent loop; surface it as a clean error (Run never panics out).
res.Err = fmt.Errorf("resolve model %q: resolver returned a nil model", tier)
return res
}
ctx = modelCtx
// Audit start (optional). The recorder satisfies RunTally; stamp it on the
// invocation so a self-status tool can read live progress.
info := RunInfo{
RunID: inv.RunID,
SubjectID: ra.ID,
Name: ra.Name,
CallerID: inv.CallerID,
ChannelID: inv.ChannelID,
ParentRunID: inv.ParentRunID,
Inputs: inv.SkillInputs,
StartedAt: started,
MaxIterations: maxIter,
}
var rec RunRecorder
var stateAcc *RunStateAccessor
if e.cfg.Ports.Audit != nil {
rec = e.cfg.Ports.Audit.StartRun(ctx, info)
}
if rec != nil {
stateAcc = NewRunStateAccessor(rec, maxIter, 0, started)
inv.RunState = stateAcc
}
// Build the toolbox from the agent's low-level tools.
toolbox, err := e.cfg.Registry.Build(ra.LowLevelTools, inv, tool.Visibility("private"), nil)
if err != nil {
res.Err = fmt.Errorf("build toolbox: %w", err)
e.finishAudit(ctx, rec, "error", res, started, res.Err)
return res
}
// Add skill__/agent__ delegation tools from the agent's palette (nil-safe:
// no PaletteSource or empty palette → no delegation tools).
if err := addDelegationTools(toolbox, ra, inv, e.cfg.Ports.Palette); err != nil {
res.Err = fmt.Errorf("build delegation tools: %w", err)
e.finishAudit(ctx, rec, "error", res, started, res.Err)
return res
}
// Run context: bound by MaxRuntime, detached from the caller's deadline so a
// lane/queue wait doesn't eat the run budget (mort's V10 lesson). Caller
// cancellation still propagates via MergeCancellation. Created BEFORE the
// step observer so the observer forwards the merged run context (not a
// possibly-cancelled caller ctx) to OnStep consumers.
// MaxRuntime stays a WithTimeout so its DeadlineExceeded propagates through the
// child chain (→ "timeout"), preserving the run's-own-timeout vs caller-cancel
// distinction. A NESTED cause-carrying layer lets a critic kill surface as a
// distinct "killed" without disturbing that: only an ErrCriticKill cause is
// consulted in statusFor; a generic run error or a caller cancel is classified
// by the run error itself.
timeoutCtx, cancelTimeout := context.WithTimeout(context.WithoutCancel(ctx), maxRuntime)
defer cancelTimeout()
runCtx, cancelCause := context.WithCancelCause(timeoutCtx)
defer cancelCause(nil)
runCtx, mergeCancel := MergeCancellation(runCtx, ctx)
defer mergeCancel()
// Critic (optional): monitors the run for a stall, can nudge/extend/kill via
// its host Escalator. Its hard deadline is bound to runCtx (cancel on pass).
// nil-safe: no-op when no critic is configured or the agent doesn't enable it.
critic, stopCritic := e.startCritic(runCtx, cancelCause, ra, info)
defer stopCritic()
// Step instrumentation: accumulate Result.Steps + fire inv.OnStep, feed the
// audit recorder, and keep the live iteration counter fresh. majordomo's
// step observer hands us each completed iteration; we zip the model's tool
// calls with their executed results PAIRWISE — a result without a matching
// call (or a call without a result) is skipped rather than recorded as an
// empty-name "ghost" step.
emitter := newStepEmitter(inv.OnStep)
stepObserver := func(s agent.Step) {
if stateAcc != nil {
stateAcc.SetIteration(s.Index)
}
if rec != nil {
rec.OnStep(s.Index, s.Response)
}
critic.recordStep(s.Index, s.Response) // keep the critic's activity clock fresh + carry the step payload
var calls []llm.ToolCall
if s.Response != nil {
calls = s.Response.ToolCalls
}
n := len(s.Results)
if len(calls) < n {
n = len(calls)
}
for i := 0; i < n; i++ {
call, r := calls[i], s.Results[i]
critic.recordToolStart(call.Name, string(call.Arguments))
emitter.toolStart(runCtx, call.Name, call.Arguments)
emitter.toolEnd(runCtx, call, r.Content, r.IsError)
if rec != nil {
rec.OnTool(call, r.Content)
}
}
}
opts := []agent.Option{
agent.WithToolbox(toolbox),
// Step ceiling: a fixed WithMaxSteps(maxIter) normally, but when a critic is
// active it owns a DYNAMIC ceiling (WithMaxStepsFunc) so it can raise a
// healthy-but-long run's budget mid-flight. Falls back to maxIter.
critic.maxStepsOption(maxIter),
agent.WithToolErrorLimits(e.cfg.Defaults.MaxConsecutiveToolErrors, e.cfg.Defaults.MaxSameToolCallRepeats),
agent.WithStepObserver(stepObserver),
}
if e.cfg.Compactor != nil && e.cfg.ContextTokens != nil {
if threshold := e.compactionThreshold(tier); threshold > 0 {
// Forward compaction events to the audit log (makes the
// CompactionEvent doc's "logged to the run trace" promise true).
var onFire func(compact.CompactionEvent)
if rec != nil {
onFire = func(ev compact.CompactionEvent) {
rec.LogEvent("compaction_fired", map[string]any{
"messages_before": ev.MessagesBefore,
"messages_after": ev.MessagesAfter,
"tokens_before": ev.TokensBefore,
"tokens_after": ev.TokensAfter,
})
}
}
opts = append(opts, agent.WithCompactor(e.cfg.Compactor(threshold, onFire)))
}
}
ag := agent.New(model, e.systemPrompt(ra), opts...)
runRes, runErr := ag.Run(runCtx, input, critic.steerOptions()...)
status := statusFor(runCtx, runErr)
if runRes != nil {
res.Output = runRes.Output
res.Usage = runRes.Usage
}
res.Steps = emitter.snapshot()
res.Err = runErr
e.finishAudit(ctx, rec, status, res, started, runErr)
if e.cfg.Ports.Budget != nil {
e.cfg.Ports.Budget.Commit(detach(ctx), inv.CallerID, time.Since(started).Seconds())
}
e.deliver(ctx, inv, res, runErr)
return res
}
// statusFor maps a run error to a RunStats.Status, distinguishing a critic kill
// (killed), a deadline (timeout), and a cancellation (cancelled — caller cancel
// or shutdown) from a generic error so audit consumers can tell them apart. The
// run context's cancellation cause carries the distinction (ErrCriticKill /
// DeadlineExceeded), since ctx.Err() alone only reports Canceled.
func statusFor(runCtx context.Context, runErr error) string {
switch {
case runErr == nil:
return "ok"
// Only the kill is recovered from the cancellation cause — a critic kill
// surfaces as a plain Canceled run error, so without this it'd read as
// "cancelled". Everything else is classified by the run error itself, so a
// genuine run error is never relabeled just because the context was later
// cancelled, and a caller cancel/deadline stays "cancelled" (not "timeout").
case errors.Is(context.Cause(runCtx), ErrCriticKill):
return "killed"
case errors.Is(runErr, context.DeadlineExceeded):
return "timeout"
case errors.Is(runErr, context.Canceled):
return "cancelled"
default:
return "error"
}
}
// finishAudit writes the terminal roll-up on a detached context so a cancelled
// run still records (mort's CleanupContextTimeout lesson).
func (e *Executor) finishAudit(ctx context.Context, rec RunRecorder, status string, res Result, started time.Time, runErr error) {
if rec == nil {
return
}
stats := RunStats{
Status: status,
Output: res.Output,
ToolCalls: rec.ToolCallsCount(),
RuntimeSeconds: time.Since(started).Seconds(),
}
if runErr != nil {
stats.Error = runErr.Error()
}
stats.InputTokens, stats.OutputTokens, stats.ThinkingTokens = rec.TokenStats()
rec.Close(detach(ctx), stats)
}
func (e *Executor) systemPrompt(ra RunnableAgent) string {
if e.cfg.SystemHeader == "" {
return ra.SystemPrompt
}
if ra.SystemPrompt == "" {
return e.cfg.SystemHeader
}
return e.cfg.SystemHeader + "\n\n" + ra.SystemPrompt
}
// compactionThreshold returns the token threshold for the tier's model context
// window (ratio × limit), or 0 when the limit is unknown.
func (e *Executor) compactionThreshold(tier string) int {
max := e.cfg.ContextTokens(tier)
if max <= 0 {
return 0
}
return int(float64(max) * e.cfg.Defaults.CompactionThresholdRatio)
}
// deliver posts the run's output (or error) via run.Ports.Delivery when both a
// Delivery and a target (inv.DeliveryID) are set. No target = the caller reads
// Result.Output itself (the synchronous default). Best-effort + detached: a
// delivery failure must not change the run's outcome.
func (e *Executor) deliver(ctx context.Context, inv tool.Invocation, res Result, runErr error) {
if e.cfg.Ports.Delivery == nil || inv.DeliveryID == "" {
return
}
target := deliver.Target{Kind: inv.DeliveryKind, ID: inv.DeliveryID}
dctx := detach(ctx)
if runErr != nil {
_ = e.cfg.Ports.Delivery.DeliverError(dctx, target, runErr)
return
}
_, _ = e.cfg.Ports.Delivery.Deliver(dctx, target, res.Output, nil)
}
// detach derives a bounded cleanup context off ctx, detached from its
// cancellation, for post-run writes. The cancel is intentionally not returned;
// CleanupContextTimeout bounds the lifetime.
func detach(ctx context.Context) context.Context {
c, cancel := context.WithTimeout(context.WithoutCancel(ctx), CleanupContextTimeout)
_ = cancel // bounded by the timeout; nothing to cancel early
return c
}
+181
View File
@@ -0,0 +1,181 @@
package run
import (
"context"
"errors"
"fmt"
"testing"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/majordomo/provider/fake"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
// fakeModels returns a ModelResolver backed by a fake provider scripted to
// reply with the given text (no tool calls — the loop terminates immediately).
func fakeModels(t *testing.T, reply string) ModelResolver {
t.Helper()
fp := fake.New("fake")
fp.Enqueue("test-model", fake.Reply(reply))
m, err := fp.Model("test-model")
if err != nil {
t.Fatalf("fake model: %v", err)
}
return func(ctx context.Context, _ string) (context.Context, llm.Model, error) {
return ctx, m, nil
}
}
// TestExecutorRunHelloWorld is the milestone: executus runs an agent end-to-end
// against the fake provider and returns its output. Proves the kernel is
// runnable with the zero Ports (no persistence/audit/budget/critic).
func TestExecutorRunHelloWorld(t *testing.T) {
ex := New(Config{
Registry: tool.NewRegistry(),
Models: fakeModels(t, "hello from executus"),
})
res := ex.Run(context.Background(),
RunnableAgent{Name: "greeter", SystemPrompt: "be brief", ModelTier: "test-model"},
tool.Invocation{RunID: "run-1", CallerID: "caller-1"},
"say hi")
if res.Err != nil {
t.Fatalf("run error: %v", res.Err)
}
if res.Output != "hello from executus" {
t.Fatalf("output = %q, want %q", res.Output, "hello from executus")
}
if res.RunID != "run-1" {
t.Errorf("RunID = %q, want run-1", res.RunID)
}
}
// TestExecutorBudgetRejection: a Budget that denies makes no model call.
func TestExecutorBudgetRejection(t *testing.T) {
denied := errors.New("over budget")
var modelCalled bool
models := func(ctx context.Context, _ string) (context.Context, llm.Model, error) {
modelCalled = true
return ctx, nil, nil
}
ex := New(Config{
Registry: tool.NewRegistry(),
Models: models,
Ports: Ports{Budget: budgetFunc{check: func(string) error { return denied }}},
})
res := ex.Run(context.Background(),
RunnableAgent{ModelTier: "test-model"},
tool.Invocation{RunID: "r", CallerID: "broke"}, "hi")
if !errors.Is(res.Err, denied) {
t.Fatalf("err = %v, want budget denial", res.Err)
}
if modelCalled {
t.Error("model must not be resolved/called when budget denies")
}
}
// TestExecutorAuditWiring: the Audit port receives StartRun + Close with the
// terminal status/output.
func TestExecutorAuditWiring(t *testing.T) {
rec := &captureRecorder{}
ex := New(Config{
Registry: tool.NewRegistry(),
Models: fakeModels(t, "done"),
Ports: Ports{Audit: auditFunc{start: func(RunInfo) RunRecorder { return rec }}},
})
res := ex.Run(context.Background(),
RunnableAgent{ModelTier: "test-model"},
tool.Invocation{RunID: "r2", CallerID: "c"}, "go")
if res.Err != nil {
t.Fatalf("run error: %v", res.Err)
}
if !rec.closed {
t.Fatal("recorder.Close was not called")
}
if rec.stats.Status != "ok" {
t.Errorf("close status = %q, want ok", rec.stats.Status)
}
if rec.stats.Output != "done" {
t.Errorf("close output = %q, want done", rec.stats.Output)
}
}
// --- test doubles ---
type budgetFunc struct{ check func(callerID string) error }
func (b budgetFunc) Check(_ context.Context, callerID string) error { return b.check(callerID) }
func (b budgetFunc) Commit(context.Context, string, float64) {}
type auditFunc struct{ start func(RunInfo) RunRecorder }
func (a auditFunc) StartRun(_ context.Context, info RunInfo) RunRecorder { return a.start(info) }
type captureRecorder struct {
closed bool
stats RunStats
steps int
tools int
}
func (r *captureRecorder) TokenStats() (in, out, thinking int64) { return 0, 0, 0 }
func (r *captureRecorder) ToolCallsCount() int { return r.tools }
func (r *captureRecorder) OnStep(int, *llm.Response) { r.steps++ }
func (r *captureRecorder) OnTool(llm.ToolCall, string) { r.tools++ }
func (r *captureRecorder) LogEvent(string, map[string]any) {}
func (r *captureRecorder) LogError(string) {}
func (r *captureRecorder) Close(_ context.Context, s RunStats) { r.closed = true; r.stats = s }
// TestExecutorNilModelNoPanic: a resolver returning (ctx, nil, nil) yields a
// clean error, not a nil-pointer panic (gadfly F1, high severity).
func TestExecutorNilModelNoPanic(t *testing.T) {
ex := New(Config{
Registry: tool.NewRegistry(),
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) {
return ctx, nil, nil // nil model, nil error
},
})
res := ex.Run(context.Background(),
RunnableAgent{ModelTier: "x"}, tool.Invocation{RunID: "r"}, "hi")
if res.Err == nil {
t.Fatal("expected an error for a nil model, got nil (would have panicked in the loop)")
}
}
// TestStatusFor maps run errors + cancellation cause to RunStats.Status (gadfly F3).
func TestStatusFor(t *testing.T) {
bg := context.Background()
// A context cancelled with the critic-kill cause: ctx.Err() is Canceled, but
// context.Cause carries ErrCriticKill → "killed".
killCtx, killCancel := context.WithCancelCause(context.Background())
killCancel(fmt.Errorf("%w: hung", ErrCriticKill))
// A context cancelled with a non-kill cause must NOT relabel a genuine run
// error: a real error stays "error" even though the ctx was later cancelled.
cancelledCtx, cc := context.WithCancelCause(context.Background())
cc(context.DeadlineExceeded)
cases := []struct {
ctx context.Context
err error
want string
}{
{bg, nil, "ok"},
{bg, context.DeadlineExceeded, "timeout"},
{bg, context.Canceled, "cancelled"},
{bg, fmt.Errorf("wrapped: %w", context.DeadlineExceeded), "timeout"},
{bg, errors.New("boom"), "error"},
{killCtx, context.Canceled, "killed"},
{cancelledCtx, errors.New("boom"), "error"}, // generic error not relabeled by cause
{cancelledCtx, context.Canceled, "cancelled"}, // caller cancel stays cancelled, not timeout
}
for _, c := range cases {
if got := statusFor(c.ctx, c.err); got != c.want {
t.Errorf("statusFor(%v) = %q, want %q", c.err, got, c.want)
}
}
}
+102
View File
@@ -0,0 +1,102 @@
package run
import (
"context"
"fmt"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
// addDelegationTools adds a delegation tool to the toolbox for each
// SkillPalette / SubAgentPalette entry, backed by the PaletteSource:
//
// - skill__<name> invokes the named saved skill with structured inputs.
// - agent__<name> invokes the named sub-agent with a prompt.
//
// Each delegated call runs as a CHILD of the current run (parentRunID =
// inv.RunID), inheriting the caller + channel. No-op when palette is nil or both
// palettes are empty — so an agent with no palette (or a host with no
// PaletteSource) simply has no delegation tools, exactly as before.
func addDelegationTools(box *llm.Toolbox, ra RunnableAgent, inv tool.Invocation, palette PaletteSource) error {
if palette == nil {
return nil
}
seen := map[string]bool{} // dedupe across both palettes by final tool name
for _, name := range ra.SkillPalette {
name := name // capture
toolName := "skill__" + name
if name == "" || seen[toolName] { // skip empty / duplicate palette entries
continue
}
seen[toolName] = true
t := llm.DefineTool(
toolName,
fmt.Sprintf("Delegate the task to the %q skill. Provide its declared inputs.", name),
func(ctx context.Context, args skillDelegateArgs) (any, error) {
out, _, status, err := palette.InvokeSkill(ctx, inv.CallerID, inv.ChannelID, name, args.Inputs, inv.RunID)
if err != nil {
return nil, delegationErr("skill", name, out, err)
}
return delegationResult(name, "skill", out, status), nil
},
)
if err := box.Add(t); err != nil {
return fmt.Errorf("add %s: %w", toolName, err)
}
}
for _, name := range ra.SubAgentPalette {
name := name // capture
toolName := "agent__" + name
if name == "" || seen[toolName] {
continue
}
seen[toolName] = true
t := llm.DefineTool(
toolName,
fmt.Sprintf("Delegate the task to the %q sub-agent with a natural-language prompt.", name),
func(ctx context.Context, args agentDelegateArgs) (any, error) {
out, _, status, err := palette.InvokeAgent(ctx, inv.CallerID, inv.ChannelID, name, args.Prompt, inv.RunID, "", "", nil, nil)
if err != nil {
return nil, delegationErr("agent", name, out, err)
}
return delegationResult(name, "agent", out, status), nil
},
)
if err := box.Add(t); err != nil {
return fmt.Errorf("add %s: %w", toolName, err)
}
}
return nil
}
// delegationResult surfaces a non-ok child status to the parent agent (so it can
// react to a timeout/cancel/budget stop) while still passing the partial output.
func delegationResult(name, kind, out, status string) string {
if status != "" && status != "ok" {
header := fmt.Sprintf("[%s %q ended with status %q]", kind, name, status)
if out == "" { // no trailing blank line when there's no body
return header
}
return header + "\n" + out
}
return out
}
// delegationErr wraps a hard delegation failure, folding in any partial output
// the child produced before failing (so it isn't silently lost).
func delegationErr(kind, name, partial string, err error) error {
if partial != "" {
return fmt.Errorf("%s %q failed (partial output: %q): %w", kind, name, partial, err)
}
return fmt.Errorf("%s %q failed: %w", kind, name, err)
}
type skillDelegateArgs struct {
Inputs map[string]any `json:"inputs" description:"Inputs for the skill, matching its declared input schema."`
}
type agentDelegateArgs struct {
Prompt string `json:"prompt" description:"The task/prompt to hand the sub-agent."`
}
+125
View File
@@ -0,0 +1,125 @@
package run_test
import (
"context"
"encoding/json"
"testing"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/majordomo/provider/fake"
"gitea.stevedudenhoeffer.com/steve/executus/run"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
// recordingPalette captures the delegation call it received.
type recordingPalette struct {
gotName, gotCaller, gotParent string
gotInputs map[string]any
}
func (p *recordingPalette) ResolveSkill(context.Context, string, string) (string, error) {
return "", nil
}
func (p *recordingPalette) InvokeSkill(_ context.Context, callerID, _, name string, inputs map[string]any, parentRunID string) (string, string, string, error) {
p.gotName, p.gotCaller, p.gotParent, p.gotInputs = name, callerID, parentRunID, inputs
return "the skill output", "child-run-1", "ok", nil
}
func (p *recordingPalette) ResolveAgent(context.Context, string, string) (string, error) {
return "", nil
}
func (p *recordingPalette) InvokeAgent(context.Context, string, string, string, string, string, string, string, []string, func(context.Context, string, string)) (string, string, string, error) {
return "", "", "ok", nil
}
// TestPaletteDelegation: an agent with a SkillPalette gets a skill__<name> tool;
// the model calls it, the executor routes it through run.Ports.Palette as a
// child of the current run, and the result flows back into the loop.
func TestPaletteDelegation(t *testing.T) {
pal := &recordingPalette{}
fp := fake.New("fake")
fp.Enqueue("m",
fake.ReplyWith(llm.Response{ToolCalls: []llm.ToolCall{{
ID: "c1",
Name: "skill__helper",
Arguments: json.RawMessage(`{"inputs":{"q":"hi"}}`),
}}}),
fake.Reply("delegated and done"),
)
m, err := fp.Model("m")
if err != nil {
t.Fatal(err)
}
ex := run.New(run.Config{
Registry: tool.NewRegistry(),
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) { return ctx, m, nil },
Ports: run.Ports{Palette: pal},
})
res := ex.Run(context.Background(),
run.RunnableAgent{ID: "a1", Name: "boss", ModelTier: "m", SkillPalette: []string{"helper"}},
tool.Invocation{RunID: "parent-run", CallerID: "caller-7", ChannelID: "chan"},
"delegate please")
if res.Err != nil {
t.Fatalf("run error: %v", res.Err)
}
if res.Output != "delegated and done" {
t.Errorf("output = %q", res.Output)
}
if pal.gotName != "helper" {
t.Errorf("InvokeSkill name = %q, want helper", pal.gotName)
}
if pal.gotCaller != "caller-7" {
t.Errorf("InvokeSkill caller = %q, want caller-7", pal.gotCaller)
}
if pal.gotParent != "parent-run" {
t.Errorf("InvokeSkill parentRunID = %q, want parent-run (child of the current run)", pal.gotParent)
}
if pal.gotInputs["q"] != "hi" {
t.Errorf("InvokeSkill inputs = %+v, want q=hi", pal.gotInputs)
}
}
// TestNoPaletteNoDelegationTools: nil PaletteSource → no delegation tools, run
// still works (the agent just has no skill__/agent__ tools).
func TestNoPaletteNoDelegationTools(t *testing.T) {
fp := fake.New("fake")
fp.Enqueue("m", fake.Reply("ok"))
m, _ := fp.Model("m")
ex := run.New(run.Config{
Registry: tool.NewRegistry(),
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) { return ctx, m, nil },
})
res := ex.Run(context.Background(),
run.RunnableAgent{Name: "x", ModelTier: "m", SkillPalette: []string{"helper"}},
tool.Invocation{RunID: "r"}, "hi")
if res.Err != nil || res.Output != "ok" {
t.Fatalf("nil-palette run failed: %v / %q", res.Err, res.Output)
}
}
// TestDelegationDedupeAndEmptySkip: empty + duplicate palette names are skipped,
// not turned into "skill__"/duplicate tools that error at box.Add (gadfly C0).
func TestDelegationDedupeAndEmptySkip(t *testing.T) {
pal := &recordingPalette{}
fp := fake.New("fake")
fp.Enqueue("m", fake.Reply("ok"))
m, _ := fp.Model("m")
ex := run.New(run.Config{
Registry: tool.NewRegistry(),
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) { return ctx, m, nil },
Ports: run.Ports{Palette: pal},
})
// "" (empty) and a duplicate "helper" must not break the build.
res := ex.Run(context.Background(),
run.RunnableAgent{Name: "x", ModelTier: "m", SkillPalette: []string{"helper", "", "helper"}},
tool.Invocation{RunID: "r"}, "hi")
if res.Err != nil {
t.Fatalf("empty/duplicate palette names should be skipped, not error: %v", res.Err)
}
if res.Output != "ok" {
t.Fatalf("output = %q", res.Output)
}
}
+196
View File
@@ -0,0 +1,196 @@
package run
import (
"context"
"errors"
"time"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/executus/deliver"
)
// ErrCriticKill is the cancellation cause the executor stamps on a run the
// critic kills, so a critic kill surfaces as a distinct "killed" status (vs a
// backstop "timeout" or a caller "cancelled"). A host CriticHandle signals a
// kill via KillCause(); the executor wraps that reason with this sentinel.
var ErrCriticKill = errors.New("run: critic killed the run")
// Ports are the host seams the run executor consumes. Every field is nil-safe:
// a light host passes the zero Ports and gets a bounded, in-memory run with no
// persistence, audit, budget, critic, delegation, or delivery — which is
// exactly a gadfly swarm task. A heavy host (mort) wires each one to a battery.
//
// This struct IS the inversion: in mort, agentexec imports agents /
// agentcritic / skillaudit and skillexec imports skills / paste directly; here
// the kernel depends only on these interfaces, and the batteries implement
// them. The mort_*_adapters.go wall becomes the set of impls.
type Ports struct {
// Audit records the run trace (start, per-step/per-tool events, final
// stats). nil = no audit.
Audit Audit
// Budget gates and meters per-caller resource use. nil = unbounded.
Budget Budget
// Critic optionally monitors a long run for hangs/runaways. nil = none.
Critic Critic
// Checkpointer persists resumable progress for durable recovery. nil = no
// checkpointing (a run interrupted by shutdown is simply lost).
Checkpointer Checkpointer
// Palette resolves SkillPalette / SubAgentPalette entries into delegation
// tools (skill__<name> / agent__<name>). nil = those entries are inert.
Palette PaletteSource
// Delivery is where the run's output + artifacts go. nil = the caller
// reads the Result in-process (the light-host default).
Delivery deliver.Delivery
}
// RunInfo describes a run at start time — the attribution a recorder/critic
// needs. Host-neutral rename of mort's SkillRun start fields.
type RunInfo struct {
RunID string
SubjectID string // the agent/skill id being run (audit "skill_id")
Name string
CallerID string
ChannelID string
ParentRunID string
Inputs map[string]any
StartedAt time.Time
// MaxIterations is the run's base tool-dispatch step ceiling, so a critic can
// raise it relative to the baseline (see CriticHandle.MaxSteps).
MaxIterations int
}
// RunStats is the terminal roll-up a recorder's Close writes. Mirrors mort's
// skillaudit/skillexec RunStats.
type RunStats struct {
Status string // ok | error | timeout | budget_exceeded | cancelled | dry_run
Output string
Error string
ToolCalls int
RuntimeSeconds float64
InputTokens int64
OutputTokens int64
ThinkingTokens int64
}
// --- Audit ---
// Audit begins recording a run. StartRun returns a per-run RunRecorder (or nil
// to skip recording this run). The audit battery wires its Storage behind this.
type Audit interface {
StartRun(ctx context.Context, info RunInfo) RunRecorder
}
// RunRecorder records the events of one in-flight run and its final stats. It
// satisfies RunTally so the kernel can surface live token/tool counts to the
// self-status tool. Mirrors mort's skillaudit.Writer.
type RunRecorder interface {
RunTally
// OnStep records one completed agent-loop iteration's model response.
OnStep(iter int, resp *llm.Response)
// OnTool records one executed tool call + its result.
OnTool(call llm.ToolCall, result string)
// LogEvent / LogError append structured events to the run log.
LogEvent(eventType string, payload map[string]any)
LogError(msg string)
// Close writes the terminal roll-up. Detaches from the caller's context
// internally so a cancelled run still records.
Close(ctx context.Context, stats RunStats)
}
// --- Budget ---
// Budget gates and meters per-caller resource use. Mirrors mort's
// skillexec.BudgetTracker.
type Budget interface {
// Check reports whether the caller has remaining budget (nil = allowed).
Check(ctx context.Context, callerID string) error
// Commit records that the caller spent runtimeSeconds on this run.
Commit(ctx context.Context, callerID string, runtimeSeconds float64)
}
// --- Critic ---
// Critic optionally monitors a long-running run (the two-tier soft/hard
// timeout). Monitor returns a handle the executor feeds progress into and
// queries for steer/deadline decisions; a nil handle means "not monitored".
//
// The exact wiring (how the handle's Steer/Deadline bind into majordomo's
// agent.WithSteer / agent.WithMaxStepsFunc / run-context cancellation) is
// finalized in the executor; this is the seam the agentcritic battery adapts.
type Critic interface {
Monitor(ctx context.Context, info RunInfo, softTimeout time.Duration) CriticHandle
}
// CriticHandle is the executor's live link to a run's critic.
//
// Concurrency: the executor calls RecordStep/RecordToolStart/Steer from the run
// goroutine while a separate watch goroutine polls Deadline() and the run's end
// calls Stop() — so implementations MUST be safe for concurrent use across these
// methods (the critic battery's handle guards its state with a mutex).
type CriticHandle interface {
// RecordStep / RecordToolStart keep the critic's activity clock fresh so a
// healthy-but-slow run is not mistaken for a hang. RecordStep also carries the
// completed step's model response (nil-safe) so the critic's Trace can show
// what the agent actually produced, not just an iteration count.
RecordStep(iter int, resp *llm.Response)
RecordToolStart(name, args string)
// Steer returns any messages the critic wants injected into the loop (a
// nudge), drained before each step — matches majordomo agent.WithSteer.
Steer() []llm.Message
// Deadline returns the current hard-kill deadline (the critic may extend
// it); the executor binds the run context to it. Zero = no hard deadline.
Deadline() time.Time
// MaxSteps returns the current tool-dispatch step ceiling, polled by the
// executor each step (via majordomo WithMaxStepsFunc) so a critic can raise a
// healthy-but-long run's iteration budget mid-flight. Return <= 0 to defer to
// the run's base MaxIterations.
MaxSteps() int
// KillCause returns a non-nil reason iff the critic has decided to KILL this
// run (as opposed to letting the hard-deadline backstop expire). The executor
// reads it when the deadline passes: non-nil → cancel the run with
// ErrCriticKill (status "killed"); nil → the backstop expired naturally
// (status "timeout"). Hosts that never distinguish the two may return nil.
KillCause() error
// Stop ends monitoring when the run finishes.
Stop()
}
// --- Checkpointer ---
// Checkpointer persists a run's resumable progress for durable recovery.
// Mirrors mort's agentexec.RunCheckpointer.
type Checkpointer interface {
// Save persists the run's current resumable progress (throttled).
Save(ctx context.Context, st RunCheckpointState) error
// Complete clears the checkpoint on success.
Complete(ctx context.Context) error
// Fail clears the checkpoint on terminal failure. A run interrupted by
// shutdown is left untouched so boot recovery picks it up.
Fail(ctx context.Context, err error) error
}
// RunCheckpointState is the resumable snapshot a Checkpointer persists. Kept
// minimal here; the executor extends what it records during the merge.
type RunCheckpointState struct {
Messages []llm.Message
Iteration int
}
// --- PaletteSource ---
// PaletteSource resolves a RunnableAgent's SkillPalette / SubAgentPalette names
// into delegation tools and invokes them. Mirrors mort's
// SkillInvokerForPalette + AgentInvokerForPalette. nil Palette => palette
// entries are inert ("not configured" at first call).
type PaletteSource interface {
ResolveSkill(ctx context.Context, callerID, name string) (skillID string, err error)
InvokeSkill(ctx context.Context, callerID, channelID, name string,
inputs map[string]any, parentRunID string) (output, runID, status string, err error)
ResolveAgent(ctx context.Context, callerID, name string) (agentID string, err error)
InvokeAgent(ctx context.Context, callerID, channelID, name string,
prompt, parentRunID, modelTierOverride, promptPrepend string,
toolsSubset []string,
onEvent func(ctx context.Context, event, emoji string)) (output, runID, status string, err error)
}
+157
View File
@@ -0,0 +1,157 @@
// Package run is executus's run kernel: the shared run-loop mechanics around
// majordomo's agent loop, plus the host seams (run.Ports / RunnableAgent) that
// let one executor serve every surface — a light host's bounded one-shot run,
// a heavy host's persona agent or saved skill — without the kernel importing a
// battery.
//
// This file holds the genuinely-identical scaffolding both run shapes need:
// context cancellation merging, the detached-cleanup timeout, the per-run
// progress accessor the self-status tool reads, the legacy `submit`
// compatibility tool (submit.go), the ancestor progress bridge (progress.go),
// and the run-finalizer machinery — one source of truth.
//
// The kernel depends only on majordomo + executus/tool + the run.Ports
// interfaces; persistence, audit, the persona/skill nouns, and the critic are
// host-supplied via Ports (see ports.go) so importing the kernel never drags in
// a store or a battery.
package run
import (
"context"
"errors"
"log/slog"
"sync/atomic"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
// ErrShutdown is the cancellation cause set on mort's base lifecycle context
// when the process is shutting down (SIGTERM after the drain window). The
// agent executor uses it to distinguish a run interrupted by shutdown (which
// should be left durable-recoverable) from a run that errored or hit its own
// deadline (terminal).
var ErrShutdown = errors.New("mort: shutting down")
// CleanupContextTimeout caps how long a run's post-completion cleanup ops
// (budget commit, audit Close, attachment bookkeeping) may wait on
// storage after detaching from the caller's — possibly already
// cancelled — context. 10s is generous for a single-row UPDATE against
// MySQL; longer suggests a hung connection the run goroutine shouldn't
// keep waiting on. Both executors derive their cleanup contexts as
// context.WithTimeout(context.WithoutCancel(ctx), CleanupContextTimeout).
const CleanupContextTimeout = 10 * time.Second
// Reserved state-react lifecycle event keys, shared so both nouns surface
// the same UX shape. Namespaced with double-underscores to make accidental
// collision with a tool name near-impossible.
const (
StateReactStart = "__start__"
StateReactEnd = "__end__"
StateReactError = "__error__"
StateReactBudgetExceeded = "__budget_exceeded__"
)
// MergeCancellation returns a context cancelled when EITHER input is
// cancelled, propagating the cancellation Cause from whichever fired. Used
// by the lane preemption path (the lane's per-job ctx.Cause flows into the
// run context) and by the runtime-detach path (process shutdown still
// reaches a run whose deadline was reset after a lane wait). Always call
// the returned cancel to release the watcher goroutine; it is also invoked
// once when either input fires.
func MergeCancellation(parent, secondary context.Context) (context.Context, context.CancelFunc) {
merged, cancel := context.WithCancelCause(parent)
go func() {
select {
case <-merged.Done():
return
case <-secondary.Done():
cancel(context.Cause(secondary))
}
}()
return merged, func() { cancel(nil) }
}
// RunFinalizer is invoked at run finish so per-run tool state (open HTTP
// streams, per-run code_exec counters, per-run search budgets) is released
// and the process-lifetime maps keyed by run id don't grow unbounded.
// Both executors fire their registered finalizers via FireFinalizers.
type RunFinalizer interface {
FinalizeRun(runID string)
}
// FireFinalizers runs every finalizer for runID, isolating each behind a
// panic-recover so one buggy finalizer can't take down the run goroutine
// or skip the others. Safe to call with a nil/empty slice.
func FireFinalizers(fs []RunFinalizer, runID string) {
for _, f := range fs {
if f == nil {
continue
}
func() {
defer func() {
if r := recover(); r != nil {
slog.Error("runengine: run finalizer panicked",
"run_id", runID, "panic", r)
}
}()
f.FinalizeRun(runID)
}()
}
}
// RunTally is the narrow live-progress source the RunStateAccessor reads —
// the running token and tool-call counts for the in-flight run. The audit
// battery's writer satisfies it; this interface is how the run kernel reads
// live tallies without importing the audit package (the inversion of mort's
// direct *skillaudit.Writer dependency).
type RunTally interface {
// TokenStats returns the running input, output, and thinking token totals.
TokenStats() (in, out, thinking int64)
// ToolCallsCount returns the number of tool calls executed so far.
ToolCallsCount() int
}
// RunStateAccessor is the per-run live-progress accessor the executor
// stamps on Invocation.RunState before building the toolbox, so the
// self-status tool can report iteration / tool-calls / tokens / elapsed for
// the in-flight run. Construct with NewRunStateAccessor; the executor's step
// observer calls SetIteration each loop.
type RunStateAccessor struct {
tally RunTally
iter atomic.Int32
maxIter int
maxCalls int
startedAt time.Time
}
// NewRunStateAccessor builds the accessor. writer supplies the live token
// + tool-call tallies; maxIter / maxCalls are the reported caps (0 =
// uncapped); startedAt anchors the elapsed clock.
func NewRunStateAccessor(tally RunTally, maxIter, maxCalls int, startedAt time.Time) *RunStateAccessor {
return &RunStateAccessor{
tally: tally,
maxIter: maxIter,
maxCalls: maxCalls,
startedAt: startedAt,
}
}
// SetIteration records the current agent-loop iteration (called from the
// executor's step observer).
func (a *RunStateAccessor) SetIteration(iter int) { a.iter.Store(int32(iter)) }
// RunState satisfies tool.RunStateAccessor.
func (a *RunStateAccessor) RunState() tool.RunState {
in, out, think := a.tally.TokenStats()
return tool.RunState{
Iteration: int(a.iter.Load()),
MaxIterations: a.maxIter,
ToolCalls: a.tally.ToolCallsCount(),
MaxToolCalls: a.maxCalls,
InputTokens: in,
OutputTokens: out,
ThinkingTokens: think,
ElapsedSeconds: int(time.Since(a.startedAt).Seconds()),
}
}
+419
View File
@@ -0,0 +1,419 @@
package run
// steps.go — the per-run step emitter and the tool→step presentation
// mapping. This is the single place that turns the executor's post-step
// observer into ordered tool.Step records: one per tool call, each with a
// stable id, an open-vocabulary kind, and a human present-tense summary
// that flips running→complete/error.
//
// One source feeds two consumers (mirroring the OnEvent/OnToolEvent/
// PostRunResult pattern): the live tool.Invocation.OnStep callback
// (nil-safe) AND snapshot(), which the executor copies onto Result.Steps.
// Because the Result accumulation does not depend on OnStep being set,
// every surface — chat (JSON + SSE), Discord, cron, sub-agents — carries
// steps; OnStep is needed only for live streaming.
//
// LIMITATION (current): majordomo exposes only a POST-step observer, so
// the executor calls toolStart+toolEnd back-to-back after each tool has
// already run. Steps are therefore recorded faithfully, but step.StartedAt
// ≈ EndedAt and the intermediate "running" phase is never observable to a
// live OnStep consumer. A pre-dispatch hook (wrapping each tool's handler
// to emit toolStart before execution, like mort's state-react decorator)
// is a follow-up that would restore real start timing + the running phase.
// The emitter already supports that two-call shape — toolStart and toolEnd
// are separate methods — so wiring it later is additive.
import (
"context"
"encoding/json"
"fmt"
"net/url"
"strings"
"time"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
// stepSummaryMaxLen caps the human summary length (section G size cap).
// Detail is unused in v1 (no live detail source while replies are
// generated blocking) so there is no Detail cap yet.
const stepSummaryMaxLen = 200
// stepEmitter accumulates ordered steps for one run and fires the live
// OnStep callback.
//
// Concurrency: touched ONLY from the agent-loop goroutine — the executor's
// stepObserver (and, once a pre-dispatch hook is wired, that hook) run
// there; majordomo executes a step's tool calls sequentially, and
// sub-agents build their own Invocation so they never reach this
// emitter. Same single-goroutine contract as the audit Writer and the
// critic ProgressRecorder — no internal lock.
type stepEmitter struct {
onStep func(ctx context.Context, ev tool.StepEvent)
now func() time.Time
seq int
steps []tool.Step // ordered; the snapshot for Result.Steps
byID map[string]int // step id -> index into steps
pending map[string][]string // correlation key -> queued running ids (FIFO)
}
// newStepEmitter returns an emitter that forwards to onStep (nil-safe).
func newStepEmitter(onStep func(ctx context.Context, ev tool.StepEvent)) *stepEmitter {
return &stepEmitter{
onStep: onStep,
now: time.Now,
byID: map[string]int{},
pending: map[string][]string{},
}
}
// corrKey correlates a "start" (name + raw args, no call id available at
// the pre-dispatch hook) with its later "end" (the stepObserver has the
// full call incl. id + the same raw args).
func corrKey(name string, args json.RawMessage) string {
return name + "\x00" + string(args)
}
// toolStart records + emits the "start" of a tool call. Called from the
// pre-dispatch hookToolbox closure, before the tool runs.
func (e *stepEmitter) toolStart(ctx context.Context, name string, args json.RawMessage) {
if e == nil {
return
}
step := e.newStep(name, args)
key := corrKey(name, args)
e.pending[key] = append(e.pending[key], step.ID)
e.fire(ctx, "start", step)
}
// toolEnd records + emits the terminal "end" of a tool call. Called from
// the stepObserver for each completed tool call. If no matching start was
// seen (e.g. a tool with a nil handler the pre-dispatch hook skipped), a
// start is synthesized so the step still appears.
func (e *stepEmitter) toolEnd(ctx context.Context, call llm.ToolCall, result string, isError bool) {
if e == nil {
return
}
id := e.matchPending(call.Name, call.Arguments)
if id == "" {
id = e.newStep(call.Name, call.Arguments).ID
}
idx, ok := e.byID[id]
if !ok {
return
}
step := &e.steps[idx]
end := e.now()
step.EndedAt = &end
if isError {
step.Status = "error"
} else {
step.Status = "complete"
}
if s := summaryForEnd(call.Name, call.Arguments, result, isError); s != "" {
step.Summary = s
}
e.fire(ctx, "end", *step)
}
// newStep mints + appends a running step and returns it (by value).
func (e *stepEmitter) newStep(name string, args json.RawMessage) tool.Step {
e.seq++
step := tool.Step{
ID: fmt.Sprintf("s%d", e.seq),
Kind: kindForTool(name),
Title: name,
Summary: summaryForStart(name, args),
Status: "running",
StartedAt: e.now(),
}
e.byID[step.ID] = len(e.steps)
e.steps = append(e.steps, step)
return step
}
// matchPending pops the oldest running step id for (name, args). Falls back to
// the OLDEST still-running step of the same tool name when the args don't
// byte-match between start and end (e.g. JSON key reordering). FIFO on the
// fallback too, consistent with the per-key queue pop above — closing the
// oldest avoids mis-correlating concurrent same-named calls. Returns "" on no
// match.
func (e *stepEmitter) matchPending(name string, args json.RawMessage) string {
key := corrKey(name, args)
if q := e.pending[key]; len(q) > 0 {
id := q[0]
if len(q) == 1 {
delete(e.pending, key)
} else {
e.pending[key] = q[1:]
}
return id
}
for i := 0; i < len(e.steps); i++ {
if e.steps[i].Title == name && e.steps[i].Status == "running" {
return e.steps[i].ID
}
}
return ""
}
func (e *stepEmitter) fire(ctx context.Context, phase string, step tool.Step) {
if e.onStep == nil {
return
}
e.onStep(ctx, tool.StepEvent{Phase: phase, Step: step})
}
// snapshot returns a copy of the ordered, deduplicated step set for the
// run Result. A step still "running" at run end (e.g. the run was
// cancelled mid-tool-call) is reported as-is.
func (e *stepEmitter) snapshot() []tool.Step {
if e == nil || len(e.steps) == 0 {
return nil
}
out := make([]tool.Step, len(e.steps))
copy(out, e.steps)
return out
}
// kindForTool maps a tool name to an open-vocabulary step kind. Unknown
// tools fall back to "tool" — never an error, just a generic step (the
// client maps unknown kinds to a default icon). Loosely tracks the
// catalog in pkg/skilltools/CLAUDE.md.
func kindForTool(name string) string {
switch name {
case "web_search", "search_reddit", "wikipedia_summary":
return "search"
case "read_page", "read_pdf", "read_reddit", "read_video", "verify_url",
"summary_summarise", "summarize", "file_get_text", "file_get_metadata",
"http_get", "http_post", "http_get_stream", "http_stream_read":
return "read"
case "code_exec", "calculate":
return "code"
case "file_save", "file_get", "file_list", "file_delete", "file_search":
return "file"
case "kv_get", "kv_set", "kv_list", "kv_delete",
"remember", "recall", "chatbot_get_memories":
return "memory"
case "query", "query_research", "deepresearch", "animate",
"agent_invoke", "agent_invoke_parallel", "agent_spawn",
"agent_spawn_parallel", "skill_invoke", "skill_invoke_parallel":
return "delegate"
case "think":
return "thinking"
default:
switch {
case strings.HasPrefix(name, "image") || strings.Contains(name, "draw"):
return "image"
default:
return "tool"
}
}
}
// summaryForStart builds the human present-tense running summary. It
// derives specifics from safe arg fields only; secret-bearing tools
// (mcp_call, email_send, http_*) are summarized without echoing args.
func summaryForStart(name string, args json.RawMessage) string {
var s string
switch name {
case "web_search":
if q := argString(args, "query", "q"); q != "" {
s = fmt.Sprintf("Searching the web for %q", q)
} else {
s = "Searching the web"
}
case "search_reddit":
if q := argString(args, "query", "q"); q != "" {
s = fmt.Sprintf("Searching Reddit for %q", q)
} else {
s = "Searching Reddit"
}
case "wikipedia_summary":
if q := argString(args, "query", "title"); q != "" {
s = fmt.Sprintf("Looking up %q on Wikipedia", q)
} else {
s = "Looking up Wikipedia"
}
case "read_page", "read_pdf", "read_reddit", "read_video", "verify_url":
if u := argString(args, "url", "post", "page"); u != "" {
s = "Reading " + hostOf(u)
} else {
s = "Reading a page"
}
case "http_get", "http_post", "http_get_stream":
// Show host only — a full URL can embed credentials/tokens.
if u := argString(args, "url"); u != "" {
s = "Fetching " + hostOf(u)
} else {
s = "Making an HTTP request"
}
case "summary_summarise", "summarize":
s = "Summarizing text"
case "translate":
if lang := argString(args, "target_lang", "target_language", "lang"); lang != "" {
s = "Translating to " + lang
} else {
s = "Translating text"
}
case "code_exec":
s = "Running code"
case "calculate":
if q := argString(args, "query", "expression", "expr"); q != "" {
s = "Calculating " + truncateStep(q, 60)
} else {
s = "Calculating"
}
case "remember":
// Never echo the stored value.
s = "Saving a memory"
case "recall", "chatbot_get_memories":
s = "Recalling memories"
case "kv_get", "kv_list":
s = "Reading saved data"
case "kv_set":
s = "Saving data"
case "kv_delete":
s = "Deleting saved data"
case "file_save":
if n := argString(args, "name", "filename"); n != "" {
s = "Saving file " + truncateStep(n, 60)
} else {
s = "Saving a file"
}
case "file_get", "file_get_text", "file_get_metadata":
s = "Reading a file"
case "file_list", "file_search":
s = "Listing files"
case "query", "query_research":
if q := argString(args, "query", "question", "prompt", "task"); q != "" {
s = "Researching " + truncateStep(q, 80)
} else {
s = "Researching"
}
case "deepresearch":
s = "Running deep research"
case "animate":
s = "Generating an animation"
case "agent_invoke", "agent_spawn":
if a := argString(args, "agent", "agent_name", "name"); a != "" {
s = "Delegating to " + a
} else {
s = "Delegating to a sub-agent"
}
case "agent_invoke_parallel", "agent_spawn_parallel":
s = "Delegating to sub-agents"
case "skill_invoke":
if sk := argString(args, "skill_name", "skill", "name"); sk != "" {
s = "Running skill " + sk
} else {
s = "Running a skill"
}
case "skill_invoke_parallel":
s = "Running skills in parallel"
case "think":
s = "Thinking"
case "mcp_call":
// Redact: MCP args frequently carry secrets. Name server/tool only.
srv, tl := argString(args, "server"), argString(args, "tool")
switch {
case srv != "" && tl != "":
s = fmt.Sprintf("Calling %s/%s", srv, tl)
case srv != "":
s = "Calling " + srv
default:
s = "Calling an MCP tool"
}
case "email_send":
// Redact recipients + body.
s = "Sending an email"
default:
s = "Using " + name
}
return truncateStep(s, stepSummaryMaxLen)
}
// summaryForEnd optionally upgrades the summary to a cheap result phrase.
// Returns "" to keep the running summary (the caller then just flips the
// status). Never returns a phrase derived from raw result bytes.
func summaryForEnd(name string, _ json.RawMessage, result string, isError bool) string {
if isError {
return ""
}
switch name {
case "web_search", "search_reddit":
if n := countResults(result); n >= 0 {
return fmt.Sprintf("Found %d result%s", n, plural(n))
}
}
return ""
}
// argString pulls the first present non-empty string field from a tool's
// raw JSON args, trying keys in order. Returns "" when none parse.
func argString(args json.RawMessage, keys ...string) string {
if len(args) == 0 {
return ""
}
var m map[string]any
if err := json.Unmarshal(args, &m); err != nil {
return ""
}
for _, k := range keys {
if v, ok := m[k]; ok {
if s, ok := v.(string); ok && strings.TrimSpace(s) != "" {
return strings.TrimSpace(s)
}
}
}
return ""
}
// countResults parses a v11-style {"results":[...]} envelope and returns
// the count, or -1 when the shape doesn't match.
func countResults(result string) int {
if strings.TrimSpace(result) == "" {
return -1
}
var env struct {
Results []json.RawMessage `json:"results"`
}
if err := json.Unmarshal([]byte(result), &env); err != nil || env.Results == nil {
return -1
}
return len(env.Results)
}
// hostOf returns the bare host (no leading www.) of a URL, or a short
// form of the raw string when it doesn't parse as a URL.
func hostOf(raw string) string {
if u, err := url.Parse(raw); err == nil && u.Host != "" {
return strings.TrimPrefix(u.Host, "www.")
}
return truncateStep(raw, 60)
}
// truncateStep rune-safely caps s to max, appending an ellipsis when cut.
func truncateStep(s string, max int) string {
if max <= 0 {
return ""
}
r := []rune(s)
if len(r) <= max {
return s
}
if max == 1 {
return string(r[:1])
}
return string(r[:max-1]) + "…"
}
func plural(n int) string {
if n == 1 {
return ""
}
return "s"
}
+84
View File
@@ -0,0 +1,84 @@
package run
import (
"context"
"strings"
"sync"
llm "gitea.stevedudenhoeffer.com/steve/majordomo/llm"
)
// SubmitCapture records the output a run's `submit` tool received.
//
// Why this exists: legacy agentkit injected a synthetic `submit` tool and
// ended the loop when it fired; years of mort system prompts (agent
// YAMLs, skill manifests, the executors' platform headers) teach the
// model to "call submit with your final answer". majordomo's agent loop
// has no submit concept — it ends when the model replies WITHOUT tool
// calls. Dropping submit cold would make every prompt-trained model
// burn turns on "unknown tool \"submit\"" errors.
//
// The compatibility shape: the executors add NewSubmitTool's tool to
// every run's toolset (unless the palette already defines a `submit`).
// The handler records the FIRST submitted answer and tells the model
// the answer was accepted so its next turn is a bare reply (which ends
// the loop naturally). After the run, the executor consults
// Output(loopOutput, runErr): a captured submission wins over an empty
// or budget-exhausted ending, so a model that submits on its final
// allowed step still produces its answer instead of ErrMaxSteps.
type SubmitCapture struct {
mu sync.Mutex
output string
called bool
}
// Record stores the first submitted answer; later calls are ignored
// (matching legacy agentkit's "multiple calls keep the first" contract).
func (c *SubmitCapture) Record(output string) {
c.mu.Lock()
defer c.mu.Unlock()
if c.called {
return
}
c.called = true
c.output = output
}
// Submitted returns the captured answer and whether submit fired.
func (c *SubmitCapture) Submitted() (string, bool) {
c.mu.Lock()
defer c.mu.Unlock()
return c.output, c.called
}
// Output resolves the run's final output: the submitted answer when the
// model called submit (parity with legacy agentkit, where submit's argument
// WAS the run output), otherwise the loop's own final text. resolvedErr
// is nil when a submission exists — a run that submitted its answer and
// then ran out of steps (or timed out composing the courtesy
// confirmation turn) is a SUCCESS, not an error.
func (c *SubmitCapture) Output(loopOutput string, runErr error) (output string, resolvedErr error) {
if out, ok := c.Submitted(); ok {
return out, nil
}
return loopOutput, runErr
}
// submitArgs mirrors legacy agentkit's synthetic submit tool schema so
// models prompted under the old contract emit compatible calls.
type submitArgs struct {
Output string `json:"output" description:"The final answer, summary, or output for this task."`
}
// NewSubmitTool builds the compatibility `submit` tool bound to the
// given capture. Both executors (skill + agent) install one per run.
func NewSubmitTool(capture *SubmitCapture) llm.Tool {
return llm.DefineTool[submitArgs](
"submit",
"Submit your final answer or output to end this task. Call exactly once when you are done.",
func(_ context.Context, args submitArgs) (any, error) {
capture.Record(strings.TrimSpace(args.Output))
return "Final answer recorded. Do not call any more tools; reply now with a brief closing message.", nil
},
)
}
+132
View File
@@ -0,0 +1,132 @@
// Package schedule is the cron-runner battery: a generic ticker that, each
// interval, asks a store for the jobs whose next-run time has passed, runs each
// one, and stamps its next fire time. It is host-agnostic orchestration — the
// host wires the store (skill.SkillStore.ListDueScheduled /
// persona.Storage.ListScheduledAgents), the run (run.Executor), and the cron
// "next fire" function (a cron library, or skill's schedule parser). The
// battery owns no cron grammar of its own, so it never duplicates the parser.
package schedule
import (
"context"
"errors"
"log/slog"
"time"
)
// Due is one schedulable job: its id and its cron expression.
type Due struct {
ID string
Cron string
}
// Runner periodically fires due jobs. Every func field is required except Now
// (defaults to time.Now) and Logger (defaults to slog.Default). Construct the
// struct directly and call Loop (or Tick for a single pass / tests).
type Runner struct {
// Interval is how often Loop checks for due jobs. <= 0 defaults to 1m.
Interval time.Duration
// Due lists the jobs due at now.
Due func(ctx context.Context, now time.Time) ([]Due, error)
// Run executes one job by id.
Run func(ctx context.Context, id string) error
// Mark records that a job ran at ranAt and is next due at nextAt.
Mark func(ctx context.Context, id string, ranAt, nextAt time.Time) error
// Next computes a cron expression's next fire after a given time.
Next func(cron string, after time.Time) (time.Time, error)
Now func() time.Time
Logger *slog.Logger
}
func (r *Runner) now() time.Time {
if r.Now != nil {
return r.Now()
}
return time.Now()
}
func (r *Runner) log() *slog.Logger {
if r.Logger != nil {
return r.Logger
}
return slog.Default()
}
// Tick runs one pass: every currently-due job is run, then stamped with its
// next fire time. A job whose Run or Next errors is logged and skipped (its
// next-run time is left unchanged so it stays due and retries next tick) — one
// bad job never stalls the others. Returns the error from Due (the only
// pass-fatal step).
func (r *Runner) Tick(ctx context.Context) error {
if err := r.validate(); err != nil {
return err
}
now := r.now()
due, err := r.Due(ctx, now)
if err != nil {
return err
}
for _, j := range due {
// Compute the next fire BEFORE running. A permanently-unparseable cron
// then skips the job entirely (logged) rather than running it — an
// unstamped job stays due, so checking Next first avoids a hot-loop of
// real Run executions every tick.
next, err := r.Next(j.Cron, now)
if err != nil {
r.log().Warn("scheduled job has an unparseable cron; skipping (not run, not rescheduled)", "job", j.ID, "cron", j.Cron, "error", err)
continue
}
if err := r.Run(ctx, j.ID); err != nil {
r.log().Warn("scheduled job failed; stays due, will retry next tick", "job", j.ID, "error", err)
continue
}
// A Mark failure leaves the job due, so it re-runs next tick — Run must
// be idempotent (there is no atomic run+stamp across two host callbacks).
if err := r.Mark(ctx, j.ID, now, next); err != nil {
r.log().Warn("failed to stamp next run; job may re-execute next tick (Run must be idempotent)", "job", j.ID, "error", err)
}
}
return nil
}
// validate reports a misconfigured Runner (a required callback left nil) as a
// clear error rather than a nil-deref panic on first tick.
func (r *Runner) validate() error {
if r.Due == nil || r.Run == nil || r.Mark == nil || r.Next == nil {
return errors.New("schedule: Runner requires non-nil Due, Run, Mark, and Next")
}
return nil
}
// Loop ticks every Interval until ctx is cancelled. A Tick error (the Due
// lister failing) is logged and the loop continues — a transient store hiccup
// shouldn't kill the scheduler — and a panic from any host callback is
// recovered so one bad tick can't silently kill the scheduler goroutine.
func (r *Runner) Loop(ctx context.Context) {
interval := r.Interval
if interval <= 0 {
interval = time.Minute
}
t := time.NewTicker(interval)
defer t.Stop()
for {
select {
case <-ctx.Done():
return
case <-t.C:
r.safeTick(ctx)
}
}
}
func (r *Runner) safeTick(ctx context.Context) {
defer func() {
if rec := recover(); rec != nil {
r.log().Error("schedule tick panicked; scheduler continues", "panic", rec)
}
}()
if err := r.Tick(ctx); err != nil {
r.log().Warn("schedule tick failed", "error", err)
}
}
+111
View File
@@ -0,0 +1,111 @@
package schedule
import (
"context"
"errors"
"testing"
"time"
)
func TestTickRunsDueAndStampsNext(t *testing.T) {
ctx := context.Background()
now := time.Date(2026, 1, 1, 12, 0, 0, 0, time.UTC)
var ran []string
marked := map[string]time.Time{}
r := &Runner{
Now: func() time.Time { return now },
Due: func(_ context.Context, _ time.Time) ([]Due, error) {
return []Due{{ID: "a", Cron: "hourly"}, {ID: "b", Cron: "bad"}}, nil
},
Run: func(_ context.Context, id string) error { ran = append(ran, id); return nil },
Mark: func(_ context.Context, id string, _, next time.Time) error { marked[id] = next; return nil },
Next: func(cron string, after time.Time) (time.Time, error) {
if cron == "bad" {
return time.Time{}, errors.New("unparseable")
}
return after.Add(time.Hour), nil
},
}
if err := r.Tick(ctx); err != nil {
t.Fatal(err)
}
// Next is checked first, so the bad-cron job is skipped BEFORE Run — only
// the parseable job runs and gets stamped (no hot-loop of a bad-cron Run).
if len(ran) != 1 || ran[0] != "a" {
t.Errorf("ran = %v, want only [a] (bad-cron b skipped before Run)", ran)
}
if marked["a"] != now.Add(time.Hour) {
t.Errorf("a next = %v, want +1h", marked["a"])
}
if _, ok := marked["b"]; ok {
t.Errorf("b should not be stamped (bad cron), got %v", marked["b"])
}
}
func TestTickRunFailureDoesNotStampOrStall(t *testing.T) {
ctx := context.Background()
var ran []string
marked := map[string]bool{}
r := &Runner{
Due: func(_ context.Context, _ time.Time) ([]Due, error) {
return []Due{{ID: "x", Cron: "h"}, {ID: "y", Cron: "h"}}, nil
},
Run: func(_ context.Context, id string) error {
ran = append(ran, id)
if id == "x" {
return errors.New("boom")
}
return nil
},
Mark: func(_ context.Context, id string, _, _ time.Time) error { marked[id] = true; return nil },
Next: func(string, time.Time) (time.Time, error) { return time.Now(), nil },
}
if err := r.Tick(ctx); err != nil {
t.Fatal(err)
}
if len(ran) != 2 { // y still runs despite x failing
t.Errorf("ran = %v, want both attempted", ran)
}
if marked["x"] { // failed job NOT stamped -> stays due, retries
t.Error("failed job x should not be stamped")
}
if !marked["y"] {
t.Error("y should be stamped")
}
}
func TestTickDueErrorIsFatalToPass(t *testing.T) {
r := &Runner{
Due: func(context.Context, time.Time) ([]Due, error) { return nil, errors.New("store down") },
Run: func(context.Context, string) error { return nil },
Mark: func(context.Context, string, time.Time, time.Time) error { return nil },
Next: func(string, time.Time) (time.Time, error) { return time.Now(), nil },
}
if err := r.Tick(context.Background()); err == nil {
t.Error("Tick should surface the Due lister error")
}
}
func TestUnparseableCronSkipsRunEntirely(t *testing.T) {
var ran []string
r := &Runner{
Due: func(context.Context, time.Time) ([]Due, error) { return []Due{{ID: "z", Cron: "bad"}}, nil },
Run: func(_ context.Context, id string) error { ran = append(ran, id); return nil },
Mark: func(context.Context, string, time.Time, time.Time) error { return nil },
Next: func(string, time.Time) (time.Time, error) { return time.Time{}, errors.New("bad cron") },
}
if err := r.Tick(context.Background()); err != nil {
t.Fatal(err)
}
if len(ran) != 0 {
t.Errorf("a job with an unparseable cron must NOT be run (avoids hot-loop), ran=%v", ran)
}
}
func TestValidateRejectsNilCallbacks(t *testing.T) {
r := &Runner{Due: func(context.Context, time.Time) ([]Due, error) { return nil, nil }} // missing Run/Mark/Next
if err := r.Tick(context.Background()); err == nil {
t.Error("Tick should return a validation error for a partially-wired Runner, not panic")
}
}
+6
View File
@@ -0,0 +1,6 @@
package skill
// DefaultChatbotInputName is the input-param name a chatbot-exposed skill
// receives the user's message under when its schema doesn't name one. Moved
// from mort's chatbot_provider.go (a host concern) as a host-agnostic default.
const DefaultChatbotInputName = "request"
+422
View File
@@ -0,0 +1,422 @@
package skill
import (
"fmt"
"strings"
)
// This file holds the shared input-parsing primitives used by both the
// chatbot exposure adapter (chatbot_provider.go) and the .skill Discord
// command handler (commands.go) to construct a SkillInputs map from
// caller-supplied raw values. Centralising here avoids the two paths
// drifting in their type-coercion or required-check semantics.
//
// Two layers:
//
// - CoerceInputValue: per-param-type coercion (int/float/bool/string).
// Accepts loosely-typed values (LLM-stringified numbers, JSON
// float64s for ints) and returns a value in the target Go shape.
//
// - CoerceInputs: per-skill validation. Walks the InputSchema, coerces
// each declared param via CoerceInputValue, drops extras silently,
// errors on missing required.
//
// Why exported (capital): both consumers live in the same package, but
// the names are also referenced in test files and the symbols are
// genuinely useful API for any future consumer (webui form handler,
// scheduler in v2). Keep the surface small.
// CoerceInputValue coerces a single raw value to the target InputParam
// type. JSON numbers arrive from json.Unmarshal as float64; bools as
// bool; strings as string. Type-mismatched strings are accepted ("3" →
// int 3, "true" → bool true) because both LLM tool calls and Discord
// command args frequently surface scalars as strings.
//
// Why: LLM tool-call args come through json.Unmarshal of a plain
// map[string]any, which forces every JSON number into float64 and every
// JSON string into string. Without this coerce step, an int parameter
// would arrive in SkillInputs as a float64, a bool sent as "true" would
// arrive as a string, etc. — confusing the skill agent's prompt
// renderer and any tool-side logic that switches on Go type. The
// .skill command handler benefits identically: arg tokens arrive as
// strings, but downstream tools may expect typed values.
//
// Test: TestCoerceInputValue in inputs_test.go covers each branch.
func CoerceInputValue(paramType string, v any) (any, error) {
switch paramType {
case "int":
switch x := v.(type) {
case float64:
return int(x), nil
case int:
return x, nil
case string:
var i int
if _, err := fmt.Sscanf(x, "%d", &i); err != nil {
return nil, fmt.Errorf("not an int: %q", x)
}
return i, nil
default:
return nil, fmt.Errorf("not an int: %T", v)
}
case "float":
switch x := v.(type) {
case float64:
return x, nil
case int:
return float64(x), nil
case string:
var f float64
if _, err := fmt.Sscanf(x, "%f", &f); err != nil {
return nil, fmt.Errorf("not a float: %q", x)
}
return f, nil
default:
return nil, fmt.Errorf("not a float: %T", v)
}
case "bool":
switch x := v.(type) {
case bool:
return x, nil
case string:
switch x {
case "true", "True", "TRUE", "1":
return true, nil
case "false", "False", "FALSE", "0":
return false, nil
default:
return nil, fmt.Errorf("not a bool: %q", x)
}
default:
return nil, fmt.Errorf("not a bool: %T", v)
}
default:
// "string", "user", "channel", "url", and unknown — coerce to
// string. JSON numbers/bools are stringified so the executor's
// validateInputs (which strips e.g. <@!123> wrappers) gets a
// uniform string input.
switch x := v.(type) {
case string:
return x, nil
case float64:
return fmt.Sprintf("%v", x), nil
case bool:
return fmt.Sprintf("%v", x), nil
default:
return fmt.Sprintf("%v", v), nil
}
}
}
// CoerceInputs validates and coerces a map of raw caller-supplied values
// against the declared parameter set:
//
// - Extra keys (not in params) are dropped silently.
// - Missing required keys return an error so the caller can surface
// usage information.
// - Per-param type coercion handles int/float/bool sent as strings.
//
// Returns a fresh map containing only declared params; never mutates the
// input map.
//
// Why: see CoerceInputValue. Both callers (chatbot exposure adapter,
// .skill command handler) need the same required-check + extra-drop
// semantics; previously only the chatbot path implemented them, which
// is exactly why .skill <name> <args> dropped its arguments entirely.
//
// Test: TestCoerceInputs in inputs_test.go.
func CoerceInputs(params []InputParam, raw map[string]any) (map[string]any, error) {
out := make(map[string]any, len(params))
for _, p := range params {
v, present := raw[p.Name]
if !present {
if p.Required {
return nil, fmt.Errorf("missing required parameter %q", p.Name)
}
continue
}
typed, err := CoerceInputValue(p.Type, v)
if err != nil {
return nil, fmt.Errorf("parameter %q: %w", p.Name, err)
}
out[p.Name] = typed
}
return out, nil
}
// ParseCommandInputs parses a free-form command argument string into a
// raw map[string]any keyed by InputSchema parameter names. Three modes
// are supported, picked by the shape of `schema`:
//
// CASE A — empty schema:
// The whole string becomes {"request": "<rest>"}. Mirrors the
// chatbot exposure default (DefaultChatbotInputName) so a skill with
// no declared inputs can still receive its trigger text uniformly
// across both surfaces.
//
// CASE B — exactly one required param (with optional non-required
// tail):
// If the user passed any --key=value or --key value flags they're
// parsed as flags (Case C). Otherwise the WHOLE rest-of-message
// becomes that single required param's value. This is the
// "single-arg convenience" pattern that lets `.skill weather Boston
// today` work without the user typing --city=.
//
// CASE C — multiple params, OR any --flag style input:
// Tokens are parsed as `--name=value` or `--name value`. Bare
// positional tokens after a flag are collected as that flag's value.
// Trailing positional tokens with no preceding flag are dropped
// (the caller's usage string should mention the flag form).
//
// The returned map values are RAW strings (or bool true for
// presence-only flags); type coercion is the caller's job via
// CoerceInputs.
//
// Why this signature instead of returning the typed map directly: the
// caller wants to distinguish "missing required" (→ usage reply) from
// "type coercion failed" (→ explicit error). Splitting parse from
// coerce keeps the message specific.
func ParseCommandInputs(schema []InputParam, raw string) map[string]any {
out := map[string]any{}
raw = strings.TrimSpace(raw)
if raw == "" {
return out
}
// Detect flag-style input regardless of schema shape — even a single
// required-param schema may be invoked via `.skill x --name value`
// for forward compat.
hasFlag := strings.Contains(raw, "--")
switch {
case len(schema) == 0:
// Empty schema: mirror the chatbot exposure adapter's default
// "request" pseudo-param so executor.composePrompt can render
// it uniformly.
out[DefaultChatbotInputName] = raw
case !hasFlag && countRequired(schema) == 1:
// Single-required-param convenience: whole rest-of-message is the
// value, regardless of any other (non-required) params declared.
// They can be supplied via --flag form if needed.
req := firstRequired(schema)
out[req.Name] = raw
default:
// Flag-style parse. Walk tokens looking for --name[=value] or
// --name <value>.
parseFlagStyle(out, schema, raw)
}
return out
}
// countRequired returns the number of params marked Required.
func countRequired(schema []InputParam) int {
n := 0
for _, p := range schema {
if p.Required {
n++
}
}
return n
}
// firstRequired returns the first required param. Caller must have
// already verified at least one exists.
func firstRequired(schema []InputParam) *InputParam {
for i := range schema {
if schema[i].Required {
return &schema[i]
}
}
return nil
}
// parseFlagStyle walks tokens for --name=value and --name value forms.
// Unknown flags (not in schema) are still accepted into the output map
// so the caller can detect and warn about them; CoerceInputs will drop
// extras when constructing the final SkillInputs.
//
// Tokens not preceded by a --flag are dropped. v1 is intentionally
// strict-ish here: we don't try to guess which positional token belongs
// to which param when there are several. The single-required-param
// convenience handles the common ambiguity-free case in the caller.
func parseFlagStyle(out map[string]any, schema []InputParam, raw string) {
tokens := tokeniseCommandLine(raw)
declared := map[string]bool{}
for _, p := range schema {
declared[p.Name] = true
}
i := 0
for i < len(tokens) {
t := tokens[i]
if !strings.HasPrefix(t, "--") {
// Bare positional token outside a flag context — drop. The
// caller's usage string should steer users to flag form.
i++
continue
}
key := t[2:]
// --name=value form
if eq := strings.IndexByte(key, '='); eq >= 0 {
out[key[:eq]] = key[eq+1:]
i++
continue
}
// --name <value> form: take the next token IF it doesn't itself
// start with --. Otherwise treat as a presence-only boolean flag.
if i+1 < len(tokens) && !strings.HasPrefix(tokens[i+1], "--") {
out[key] = tokens[i+1]
i += 2
continue
}
out[key] = "true"
i++
}
_ = declared // reserved for v2 unknown-flag warnings
}
// tokeniseCommandLine splits a free-form Discord command argument
// string into tokens. Quoted spans (single or double quotes) are kept
// as one token so users can pass values with spaces:
//
// .skill weather --city="New York"
// .skill summarise --text 'a long sentence here'
//
// Mirrors the user's intuition without introducing a full shell
// parser. Newlines split as whitespace.
func tokeniseCommandLine(s string) []string {
var out []string
var cur strings.Builder
var quote rune
flush := func() {
if cur.Len() > 0 {
out = append(out, cur.String())
cur.Reset()
}
}
for _, r := range s {
switch {
case quote != 0:
if r == quote {
quote = 0
continue
}
cur.WriteRune(r)
case r == '"' || r == '\'':
quote = r
case r == ' ' || r == '\t' || r == '\n':
flush()
default:
cur.WriteRune(r)
}
}
flush()
return out
}
// ResolveCommandInputs is the one-call helper a Discord .skill handler
// uses to turn a free-form rest-of-message into a coerced
// SkillInputs map ready to hand to the executor. It is the single
// production entry point for command-side input resolution: every
// caller must use it (do NOT chain ParseCommandInputs + CoerceInputs
// directly).
//
// Why this exists as a single function: chaining
// ParseCommandInputs + CoerceInputs at the call site is what broke
// `.skill echo hello world` in production. ParseCommandInputs Case A
// (empty schema) writes the user's text into out["request"], but
// CoerceInputs(emptySchema, …) iterates the DECLARED params and
// silently drops every key not in the schema — so "request" is
// dropped before reaching the executor, and the agent's user-prompt
// renders "(no input provided)". The fix is to mirror the chatbot
// exposure adapter: derive the EFFECTIVE param set (which inflates
// an empty schema to a single required "request" param) and coerce
// against that, not the original empty schema.
//
// What:
// - Empty input_schema → effective params = [{request, required, string}],
// so ParseCommandInputs Case A's "request" key survives Coerce.
// - Non-empty input_schema → effective params = the schema as-is, so
// Case B / Case C parse-and-coerce semantics are unchanged.
//
// Returns the coerced SkillInputs map, or an error suitable for
// surfacing to the user (e.g. via FormatUsage). Never mutates
// `schema`.
//
// Test: TestResolveCommandInputs_* in inputs_test.go cover the three
// cases plus the empty-schema regression.
func ResolveCommandInputs(schema []InputParam, raw string) (map[string]any, error) {
rawInputs := ParseCommandInputs(schema, raw)
effective := effectiveCommandParams(schema)
return CoerceInputs(effective, rawInputs)
}
// effectiveCommandParams returns the parameter set the .skill command
// path should use for coercion. Mirrors chatbotToolParams in
// chatbot_provider.go: an empty input_schema is inflated to a single
// required "request" string param so the user's free-text trigger
// survives CoerceInputs's drop-extras semantics.
//
// Why a separate helper (vs reusing chatbotToolParams): keeping the
// helper local to inputs.go avoids dragging chatbot_provider.go into
// the .skill command path's import surface and makes the intent
// (Discord-side parameter inflation) explicit at the call site.
func effectiveCommandParams(schema []InputParam) []InputParam {
if len(schema) > 0 {
return schema
}
return []InputParam{{
Name: DefaultChatbotInputName,
Description: "The user's free-text trigger.",
Type: "string",
Required: true,
}}
}
// FormatUsage renders a human-readable usage string for the .skill
// invocation form. Used by command handlers when required params are
// missing or coercion fails.
//
// Why: keep the usage message in one place so both the missing-required
// and coercion-failed paths produce identical output.
func FormatUsage(name string, schema []InputParam) string {
var sb strings.Builder
fmt.Fprintf(&sb, "usage: `.skill %s", name)
if len(schema) == 0 {
sb.WriteString(" <text>`")
return sb.String()
}
if countRequired(schema) == 1 {
req := firstRequired(schema)
fmt.Fprintf(&sb, " <%s>`", req.Name)
// Show optional flags (if any).
var optional []InputParam
for _, p := range schema {
if !p.Required {
optional = append(optional, p)
}
}
if len(optional) > 0 {
sb.WriteString("\n optional:")
for _, p := range optional {
fmt.Fprintf(&sb, " --%s=<%s>", p.Name, p.Type)
}
}
return sb.String()
}
// Multi-param: full --flag form.
for _, p := range schema {
if p.Required {
fmt.Fprintf(&sb, " --%s=<%s>", p.Name, p.Type)
}
}
for _, p := range schema {
if !p.Required {
fmt.Fprintf(&sb, " [--%s=<%s>]", p.Name, p.Type)
}
}
sb.WriteString("`")
return sb.String()
}
+169
View File
@@ -0,0 +1,169 @@
package skill
import (
"context"
"sort"
"sync"
"time"
)
// Memory is a zero-dependency in-process SkillStore — a light host or test gets
// saved-skill persistence with no DB. Mort backs SkillStore with GORM/MySQL;
// contrib/store adds durable SQLite.
type Memory struct {
mu sync.RWMutex
skills map[string]*Skill // by ID
versions map[string][]SkillVersion // by skill ID, append order
byVerID map[string]SkillVersion // by version ID
}
// NewMemory returns an empty in-memory SkillStore.
func NewMemory() *Memory {
return &Memory{
skills: map[string]*Skill{},
versions: map[string][]SkillVersion{},
byVerID: map[string]SkillVersion{},
}
}
var _ SkillStore = (*Memory)(nil)
func (m *Memory) Initialize(context.Context) error { return nil }
func (m *Memory) Save(_ context.Context, s *Skill) error {
m.mu.Lock()
defer m.mu.Unlock()
cp := *s
m.skills[s.ID] = &cp
return nil
}
func (m *Memory) Get(_ context.Context, id string) (*Skill, error) {
m.mu.RLock()
defer m.mu.RUnlock()
s, ok := m.skills[id]
if !ok {
return nil, ErrNotFound
}
cp := *s
return &cp, nil
}
func (m *Memory) GetByName(_ context.Context, ownerID, name string) (*Skill, error) {
m.mu.RLock()
defer m.mu.RUnlock()
for _, s := range m.skills {
if s.OwnerID == ownerID && s.Name == name {
cp := *s
return &cp, nil
}
}
return nil, ErrNotFound
}
func (m *Memory) Delete(_ context.Context, id string) error {
m.mu.Lock()
defer m.mu.Unlock()
delete(m.skills, id)
return nil
}
func (m *Memory) listWhere(keep func(*Skill) bool) []Skill {
m.mu.RLock()
defer m.mu.RUnlock()
out := make([]Skill, 0, len(m.skills))
for _, s := range m.skills {
if keep == nil || keep(s) {
out = append(out, *s)
}
}
sort.Slice(out, func(i, j int) bool { return out[i].Name < out[j].Name })
return out
}
func (m *Memory) ListByOwner(_ context.Context, ownerID string) ([]Skill, error) {
return m.listWhere(func(s *Skill) bool { return s.OwnerID == ownerID }), nil
}
func (m *Memory) ListPublic(context.Context) ([]Skill, error) {
return m.listWhere(func(s *Skill) bool { return s.Visibility == VisibilityPublic }), nil
}
func (m *Memory) ListSharedWith(_ context.Context, memberID string) ([]Skill, error) {
return m.listWhere(func(s *Skill) bool {
if s.Visibility != VisibilityShared {
return false
}
for _, id := range s.SharedWith {
if id == memberID {
return true
}
}
return false
}), nil
}
func (m *Memory) ListBuiltinByName(_ context.Context, name string) (*Skill, error) {
m.mu.RLock()
defer m.mu.RUnlock()
for _, s := range m.skills {
if s.Source == SourceBuiltin && s.Name == name {
cp := *s
return &cp, nil
}
}
return nil, ErrNotFound
}
func (m *Memory) ListChatbotExposed(context.Context) ([]Skill, error) {
return m.listWhere(func(s *Skill) bool { return s.ExposeAsChatbotTool }), nil
}
func (m *Memory) ListDueScheduled(_ context.Context, now time.Time) ([]Skill, error) {
return m.listWhere(func(s *Skill) bool { return s.DueAt(now) }), nil
}
func (m *Memory) MarkScheduledRun(_ context.Context, skillID string, ranAt, nextAt time.Time) error {
m.mu.Lock()
defer m.mu.Unlock()
s, ok := m.skills[skillID]
if !ok {
return ErrNotFound
}
s.LastScheduledRunAt = ranAt
s.NextRunAt = nextAt
return nil
}
func (m *Memory) AppendVersion(_ context.Context, sv SkillVersion) error {
m.mu.Lock()
defer m.mu.Unlock()
m.versions[sv.SkillID] = append(m.versions[sv.SkillID], sv)
m.byVerID[sv.ID] = sv
return nil
}
func (m *Memory) ListVersionsBySkill(_ context.Context, skillID string, limit int) ([]SkillVersion, error) {
m.mu.RLock()
defer m.mu.RUnlock()
all := m.versions[skillID]
// newest first
out := make([]SkillVersion, 0, len(all))
for i := len(all) - 1; i >= 0; i-- {
out = append(out, all[i])
if limit > 0 && len(out) >= limit {
break
}
}
return out, nil
}
func (m *Memory) GetVersionByID(_ context.Context, versionID string) (*SkillVersion, error) {
m.mu.RLock()
defer m.mu.RUnlock()
sv, ok := m.byVerID[versionID]
if !ok {
return nil, ErrNotFound
}
return &sv, nil
}
+35
View File
@@ -0,0 +1,35 @@
package skill
import (
"time"
"gitea.stevedudenhoeffer.com/steve/executus/run"
)
// ToRunnable lowers a saved Skill into the kernel's run.RunnableAgent DTO, so
// run.Executor can run a skill WITHOUT importing this battery (the inversion of
// mort's skillexec running a skills.Skill). Maps the static shape only; the
// skill's input schema → prompt rendering, palette resolution, audit, etc. are
// supplied separately (the host renders inputs into the input string and wires
// run.Ports). A skill exposes a flat tool list (no SkillPalette/SubAgentPalette
// — composition is a host concern), so those stay empty.
func (s *Skill) ToRunnable() run.RunnableAgent {
return run.RunnableAgent{
ID: s.ID,
Name: s.Name,
SystemPrompt: s.SystemPrompt,
ModelTier: s.ModelTier,
MaxIterations: s.MaxIterations,
MaxRuntime: s.MaxRuntime,
LowLevelTools: s.Tools,
}
}
// DueAt reports whether a scheduled skill is due at now (cron empty => never).
// Convenience for a host scheduler that doesn't want to re-parse the cron.
func (s *Skill) DueAt(now time.Time) bool {
if s.Schedule == "" || s.NextRunAt.IsZero() {
return false
}
return !s.NextRunAt.After(now)
}
+107
View File
@@ -0,0 +1,107 @@
package skill
import (
"fmt"
"strings"
"time"
"github.com/robfig/cron/v3"
)
// scheduleParser is the cron parser shared across the skills package. It
// accepts the standard 5-field syntax (minute hour dom month dow) plus
// descriptors such as @daily, @hourly, etc. We do not enable the seconds
// field — schedule cadence is governed in minutes, and a seconds field
// would invite specs that fire below the min-interval floor without
// surfacing as such in the spec text.
//
// Why standalone vs. cron.ParseStandard: ParseStandard rejects descriptors
// (@daily, @hourly). Skills callers may want to write @daily as a
// shorthand alongside the explicit "daily" / "weekly" forms we translate
// below.
var scheduleParser = cron.NewParser(
cron.Minute | cron.Hour | cron.Dom | cron.Month | cron.Dow | cron.Descriptor,
)
// ParseSchedule turns a user-supplied schedule expression into a
// cron.Schedule. The empty string returns (nil, nil) — callers should
// treat that as "on-demand only".
//
// Why: Skill.Schedule is a string field stored verbatim; the validator,
// the scheduler runner, and any future tooling all need to round-trip
// through the same parser. Centralising it here avoids drift.
//
// Accepted shorthands:
// - "daily" → "0 0 * * *" (midnight UTC every day)
// - "weekly" → "0 0 * * 0" (midnight UTC every Sunday)
//
// Anything else is fed through robfig/cron/v3's standard parser
// (descriptors enabled).
//
// Test: schedule_test.go covers shorthand expansion and invalid-spec
// rejection.
func ParseSchedule(expr string) (cron.Schedule, error) {
expr = strings.TrimSpace(expr)
if expr == "" {
return nil, nil
}
switch strings.ToLower(expr) {
case "daily":
expr = "0 0 * * *"
case "weekly":
expr = "0 0 * * 0"
}
sched, err := scheduleParser.Parse(expr)
if err != nil {
return nil, fmt.Errorf("invalid schedule %q: %w", expr, err)
}
return sched, nil
}
// ScheduleMinInterval returns an estimate of the smallest gap between
// consecutive fire times for a parsed schedule. It samples the next two
// fire times from a couple of starting points and returns the smallest
// observed gap.
//
// Why: cron.Schedule does not expose a "smallest interval" API. The
// validator needs this to enforce a per-skill min-interval floor (so an
// admin can't accidentally register "* * * * *" and burn GPU minutes).
// Two probe points are enough to catch irregular schedules whose tightest
// gap appears at a particular point in the week (e.g. "0 9 * * 1,5",
// where Mon→Fri is 4d but Fri→Mon is 3d — both sampled).
//
// Returns 0 if sched is nil.
//
// Test: schedule_test.go covers a "* * * * *" minute-interval probe and
// the irregular Mon/Fri case.
func ScheduleMinInterval(sched cron.Schedule) time.Duration {
if sched == nil {
return 0
}
// Probe from a fixed reference and from a midweek offset. Six fire
// times across two starts catches weekly irregularities (the worst
// case is a schedule that fires once a week — we still get one gap
// per probe). Using a wall-clock-independent reference keeps the
// test deterministic.
starts := []time.Time{
time.Date(2024, 1, 1, 0, 0, 0, 0, time.UTC), // Monday 00:00
time.Date(2024, 1, 4, 12, 30, 0, 0, time.UTC), // Thursday 12:30
time.Date(2024, 6, 15, 23, 59, 59, 0, time.UTC), // mid-year, late
}
var min time.Duration
for _, t := range starts {
// Sample three consecutive fires per start to capture two gaps.
f1 := sched.Next(t)
f2 := sched.Next(f1)
f3 := sched.Next(f2)
for _, gap := range []time.Duration{f2.Sub(f1), f3.Sub(f2)} {
if gap <= 0 {
continue
}
if min == 0 || gap < min {
min = gap
}
}
}
return min
}
+424
View File
@@ -0,0 +1,424 @@
// Package skills implements the agentic skills platform: user-creatable
// agent definitions (system prompt + tool whitelist + I/O spec) that run
// in-process via majordomo's agent loop.
//
// A Skill is a saved agent definition. It can be invoked from Discord
// (.skill <name>), exposed to the chatbot as a tool (via the
// SkillsToolProvider), and (in v2) scheduled. Skills compose tools from
// the skilltools registry, gated by a three-stage permission model:
// save-time AuthoringRequirement, share-time SafeForShare, execute-time
// SkillNameGate.
//
// This file declares the domain types only. Storage lives in storage.go;
// validation lives in validate.go. The grand storage pattern documented in
// pkg/logic/storage/CLAUDE.md applies — when adding a field to Skill, you
// MUST also update pkg/logic/skills/gorm_model.go (gormSkill, fromStorage,
// toStorage) or persistence will silently break.
package skill
import "time"
// Skill is the domain definition of an agentic skill.
//
// Why: a skill is a saved agent definition reusable across invocations
// (Discord, chatbot tool, scheduled run in v2). The struct is intentionally
// flat — every field lives on its own column on the skills table; there is
// no JSON-blob spec column. This keeps queries (e.g. "list all skills with
// chatbot exposure") indexable and avoids opaque migration headaches.
//
// What: identity + authoring + agent spec + visibility + chatbot exposure
// fields, all on one struct.
//
// Test: see validate_test.go and integration_test.go for round-trip and
// validation coverage.
type Skill struct {
// Identity
ID string // UUID
OwnerID string // Discord member ID; empty for builtin
Name string // unique per (owner, builtin namespace)
Description string
Source Source // SourceBuiltin | SourceManual
CreatedAt time.Time
UpdatedAt time.Time
// Authoring (copied at save time from the user)
AuthoredBy string // member ID at time of last edit (audit; may differ from owner over time)
// Versioning (for builtins; user skills typically stay at 1.0.0)
Version string // semver; used by builtin loader to decide re-seed
// Spec — agent definition
SystemPrompt string
Tools []string // registry tool names
ModelTier string // "fast" | "standard" | "thinking" | explicit "provider/model"
InputSchema []InputParam
OutputTarget OutputTarget
Schedule string // cron; empty = on-demand only; rejected in v1 (ships in v2)
Visibility Visibility // VisibilityPrivate | VisibilityShared | VisibilityPublic
SharedWith []string // member IDs for visibility=shared
MaxIterations int // 0 → use convar default
MaxToolCalls int // 0 → use convar default
MaxRuntime time.Duration // 0 → use convar default
InitialMessage string
// Chatbot exposure (v1 — proves out the platform via mortventure)
ExposeAsChatbotTool bool
ChatbotToolName string
ChatbotToolDescription string
ChatbotChannelFilter string // named filter from the channel-filter registry
// Admin gating (v2 — public scheduled channel skills require approval).
// DEPRECATED in v3: PinnedVersionID subsumes this flag for non-owner
// invocation gating. CanInvoke no longer references this column.
// Drop in v4.
PendingApproval bool
// Pinned version (v3 — admin-curated invocation gate).
//
// Why: in v3, non-owner invocation requires that an admin explicitly
// pin a known snapshot. This replaces v2's PendingApproval flag —
// pinning is the explicit "approved for general use" signal, and the
// pinned snapshot is what executes for non-owner callers (so an owner
// editing a public skill never accidentally exposes work-in-progress
// to other users).
//
// PinnedVersionID is the SkillVersion.ID (UUID) of the snapshot that
// non-owner invocations resolve to. Empty means "no pin yet" — only
// the owner and admins can invoke.
//
// Schema column is `pinned_version` per the design spec but the field
// name in the domain struct is explicit about the kind of value it
// holds (a snapshot row's UUID, NOT a semver string), which avoids
// the spec ambiguity around "pin to v1.0.5" potentially mapping to
// multiple snapshot rows over time.
PinnedVersionID string
// PinnedAt is the wall-clock time the pin was set. Zero means
// PinnedVersionID is empty (never pinned).
PinnedAt time.Time
// PinnedBy is the admin member ID who set the current pin. Empty
// when PinnedVersionID is empty.
PinnedBy string
// Scheduler bookkeeping (v2). Updated by the scheduler runner after
// a successful (or failed-but-counted) scheduled execution.
//
// LastScheduledRunAt records the wall-clock time of the most recent
// scheduled invocation; zero means "never run on schedule".
//
// NextRunAt is the precomputed wake-up time the scheduler polls for
// (`WHERE next_run_at <= NOW()`). It is recomputed by feeding
// LastScheduledRunAt (or NOW() on first scheduling) through
// ParseSchedule(Schedule).Next(...). Manual / on-demand invocations
// MUST NOT touch these fields.
LastScheduledRunAt time.Time
NextRunAt time.Time
// ExtendedBounds, when true, lets a non-admin author save the skill
// with bounds (MaxIterations / MaxToolCalls / MaxRuntime) above the
// default tier (12/30/60s) up to the extended tier (50/150/600s).
// Set by an admin via `.skill admin grant-extended <name>`. Cleared
// by `.skill admin revoke-extended <name>`. Builtins and admin-
// authored skills bypass the cap entirely (the tier resolution in
// Validate treats AuthorIsAdmin and ExtendedBounds equivalently).
//
// Why a per-skill flag vs a per-user grant: governance is per-skill
// — an admin reviews a specific skill's bounds and decides those
// resource limits are justified for THAT skill. A user grant would
// blanket-allow expensive bounds on every skill they author.
ExtendedBounds bool
// ParallelCompositionAllowed gates whether this skill may use the
// skill_invoke_parallel tool. Default false.
//
// Why a per-skill admin gate: parallel fan-out multiplies blast
// radius (one bad skill spawns N concurrent runs). Admins approve
// each skill that's allowed to use parallel composition; granting
// is per-skill via `.skill admin grant-parallel <name>`. Builtins
// may set this directly in skill.yml (the loader bypasses
// save-time gates by design).
//
// Checked AT INVOCATION TIME (every skill_invoke_parallel call), so
// admins can grant or revoke without redeploying. The check lives
// in the tool handler (pkg/skilltools/tools/skill_invoke_parallel.go)
// via the SkillInvokerProvider.IsParallelAllowed extension.
ParallelCompositionAllowed bool
// ExecutionLane is the named lane the skill's runs are submitted to
// when the executor routes through pkg/lane (v6). Default
// "skill-default"; admin overrides per-skill via
// `.skill admin set-lane <name> <lane>`.
//
// Why per-skill (vs a single global skill lane): different skills
// have different concurrency profiles. A long-running web-research
// skill might warrant a dedicated 1-slot lane to avoid starving
// quick chatbot-exposed skills; an admin should be able to isolate
// it without a code change.
//
// Empty string falls through to "skill-default" at executor time
// — keeping the field nullable lets a future schema change
// distinguish "explicit skill-default" from "never set".
ExecutionLane string
// WebhookSecret enables inbound webhooks (v7). Empty = disabled
// (the default). Non-empty = the random secret URL path segment
// for POST /webhooks/<secret>. Generated by EnableWebhook;
// rotated by RegenerateWebhookSecret. Storage is varchar(64) and
// the secret is 32 random bytes (64 hex chars), so the column
// holds a fully unique secret per skill.
//
// Why store the secret directly (not a hash): the webhook handler
// must look up the skill by the secret on every POST, which would
// require comparing every stored hash against the supplied secret
// — a per-call O(n_skills) operation. The secret is treated as a
// long random URL key (like a paste UUID); compromise is mitigated
// via RegenerateWebhookSecret rotation, not via storage hashing.
WebhookSecret string
// WebhookSignatureRequired controls whether the inbound webhook
// handler verifies HMAC against the X-Mort-Signature header. Default
// true (the storage column default). Toggling to false skips HMAC
// verification — useful for low-stakes integrations behind an IP
// allowlist where the caller can't easily compute HMAC. Owners
// flip this on the management page; admins can also force it
// back on if a leaked allowlist becomes a concern.
WebhookSignatureRequired bool
// WebhookIPAllowlist is a newline-separated list of CIDR blocks
// (or bare IPs). Empty string = no allowlist (accept any source
// IP). The handler parses the list at request time so updates take
// effect immediately without a redeploy. Invalid CIDR entries
// are silently dropped at parse time (the management page form
// shows a parse-error preview before save).
WebhookIPAllowlist string
// EncryptionEnabled (v8) opts the skill into per-skill envelope
// encryption for KV values and file blob content. Default false
// (plaintext storage; matches the legacy default). When true, new
// writes go through the AES-256-GCM helpers in pkg/skilltools and
// the corresponding skill_kv / skill_file_blobs row stamps
// encryption_key_version=1; reads transparently decrypt rows whose
// version > 0 and pass through rows whose version == 0 (mixed
// storage is supported indefinitely).
//
// !!!!! OPERATIONAL WARNING !!!!! This flag is a write-side switch
// only. Disabling encryption for an already-encrypted skill does
// NOT decrypt existing rows — they remain reachable as long as
// the master key is intact. Losing SKILLS_ENCRYPTION_MASTER_KEY
// renders every encrypted row unreadable; back the master key up
// separately from database backups. See pkg/skilltools/encryption.go
// for the full operational rules.
EncryptionEnabled bool
// Preemptible (v9) opts the skill into preemption: when a higher-
// priority job arrives at a full lane, this skill's running job may
// be cancelled mid-flight to free a slot. Default false.
//
// !!!!! OPERATIONAL WARNING !!!!! Preemption means the skill's
// scaddy.Agent context is cancelled mid-step; any partial side
// effects (file writes, KV updates, sent emails, etc.) remain
// committed. Only mark a skill preemptible when it is idempotent
// or read-only — otherwise the user-visible state may be
// inconsistent with the run's "preempted" terminal status.
//
// The lane scheduler will not preempt jobs younger than
// `skills.lane.preemption_min_runtime_seconds` (default 30s) to
// prevent thrashing. The preempted run is recorded with
// status="preempted".
Preemptible bool
// DefaultPriority (v9) is the per-skill default priority used by
// the lane scheduler's fair-share queue ordering. Higher numbers
// run first within a single user's sub-queue. Default 0.
//
// Per-invocation overrides (skill_invoke priority arg, webhook
// X-Mort-Priority header) win over this default. Owners may set
// values in the range [-`skills.priority_max_per_user`,
// +`skills.priority_max_per_user`] (default cap 5); admins may
// exceed the cap.
DefaultPriority int
// Tags is a free-form set of short labels owners attach to a skill
// for organisation + discovery. The list page renders each tag as a
// chip and offers a dropdown filter populated from all visible
// skills' tags.
//
// Why a separate field (vs reusing Description / Tools): tags are a
// curatorial signal, not part of the agent spec — they only matter
// to humans browsing the list. Storing them on the skill row (vs a
// side table) keeps lookups index-only and matches how the rest of
// the skill's flat fields are persisted.
//
// Validate enforces: each tag is trimmed + lowercased; max 32 chars
// per tag; max 16 tags per skill; duplicates within a single skill
// are deduped.
Tags []string
// DeprecatedByAgentID is the Phase 7 soft-retire pointer: when
// non-empty, the Skill is "soft retired" — hidden from default
// listings (`.skill list`, the webui index, chatbot tool exposure)
// but STILL invokable via `.skill <name>` and via `skill_invoke`
// tool calls. The string is the agents.Agent.ID of the replacement
// Agent that supersedes this Skill.
//
// Why a pointer (not a bool): a future audit / migration tool needs
// to follow the soft-retire link back to the replacement. An admin
// browsing the deprecated-skills page wants to see "what should I
// use instead?" without a separate lookup table.
//
// Why keep the Skill row (not drop it): existing skill_invoke calls
// in user-authored skills, scheduled jobs, and webhook integrations
// would break if the row vanished. Soft-retire preserves the
// callable surface while signalling "this is the old name; the
// replacement Agent is the curated version."
//
// Set by the Phase 7 boot migration (pkg/logic/agents/migrate_phase7.go);
// admins may also flip it manually via storage tooling. Listing
// methods filter on this field by default but explicit GetByName /
// GetForInvocation lookups bypass the filter so direct invocation
// continues to work.
DeprecatedByAgentID string
// DefaultEmoji is an optional identity emoji for the skill, shown
// as the __start__ fallback when StateReactEmoji has no __start__
// entry. Also forwarded to the invoking Discord message when a
// parent agent calls this skill via skill_invoke, so the user sees
// the child skill's identity emoji during execution.
DefaultEmoji string
// StateReactEmoji maps tool names (and reserved keys "__start__",
// "__end__", "__error__") to Discord emoji that the bot reacts to
// the invoking message with as the skill progresses. Empty map
// (the default) disables state-react reactions for this skill.
//
// Why: the legacy `.query` agent surfaced live progress via emoji
// reactions on the invoking message (magnifying glass on search,
// page on read, …). Skills inherit the same UX without each
// author having to wire `update_status` for trivial signalling —
// the emoji map is declarative and the executor calls inv.OnEvent
// at the relevant boundaries. update_status remains for richer
// interim text; emoji reactions are an additive lightweight signal.
//
// Reserved keys:
// - __start__: reacted right before agent.Run starts
// - __end__: reacted on successful completion
// - __error__: reacted on terminal error
//
// Tool keys: react fires on each tool dispatch. Repeated reactions
// of the same emoji are no-ops at Discord (idempotent), so a skill
// that calls web_search 5x just leaves one 🔍.
//
// Map values are arbitrary Discord emoji strings (unicode emoji,
// custom emoji `<:name:id>`, animated `<a:name:id>`). Validate does
// not enforce a format — Discord rejects invalid emoji at react
// time and the executor swallows that with a log line.
StateReactEmoji map[string]string
}
// ThreadIDInputKey is the magic key under skilltools.Invocation.SkillInputs
// that the v2 .skill new / .skill edit wizard handlers use to thread a
// pre-created thread channel ID through to delivery. When
// OutputTarget.Kind == "thread" and this key is present in
// inv.SkillInputs, delivery posts directly to that thread channel;
// otherwise it falls back to OutputTarget.Target / inv.ChannelID.
//
// Why a magic input key vs an OutputTarget override field: keeps the
// wire shape (Skill struct) unchanged and keeps the override scoped
// to a single invocation. Wizard commands set this immediately after
// MessageThreadStartComplex; nothing else writes it.
//
// Why defined here vs in skillexec: wizard command handlers in this
// package need to write the key, and skillexec imports skills (so
// the reverse import would cycle). Skillexec aliases this constant.
const ThreadIDInputKey = "__thread_id__"
// Source distinguishes builtins (loaded from skills/<name>/skill.yml on
// boot) from user-authored manual skills.
//
// Why: builtin skills bypass save-time authoring and share-time safety
// checks because the loader is trusted infrastructure.
type Source string
const (
SourceBuiltin Source = "builtin"
SourceManual Source = "manual"
)
// InputParam declares a typed input slot on a skill, populated at
// invocation time from positional/flag args (Discord) or form fields
// (webui).
//
// Why: skills are invoked from heterogeneous surfaces and need a uniform
// schema for input collection and validation. The Type drives string→typed
// coercion in skillexec.validateInputs; Choices restricts to an enum set.
type InputParam struct {
Name string
Description string
Type string // "string"|"int"|"float"|"bool"|"user"|"channel"|"url"
Required bool
Default string // string-encoded; parsed per Type at invocation
Choices []string
}
// OutputTarget controls where the executor delivers a skill's output.
//
// Why: skills run in many contexts and the user shouldn't have to think
// about delivery — the spec encodes it once. The Discord delivery
// implementation in pkg/logic/skillexec/delivery.go reads this struct.
type OutputTarget struct {
Kind string // "channel"|"dm"|"thread"|"webui_only"|"channel_with_summary"
Target string // channel/member/thread ID, or empty for caller-context
}
// Visibility controls who may invoke a skill.
//
// Why: separates *invocation* gating (this struct) from *tool authoring*
// gating (skilltools.Permission) — they are orthogonal. A non-admin can
// invoke an admin-authored public skill that uses db_select; the permission
// model for the underlying tool only fires at save time, not invocation.
type Visibility string
const (
VisibilityPrivate Visibility = "private"
VisibilityShared Visibility = "shared"
VisibilityPublic Visibility = "public"
)
// IsKnownVisibility reports whether v is a recognised visibility value.
// Used by Validate.
func IsKnownVisibility(v Visibility) bool {
switch v {
case VisibilityPrivate, VisibilityShared, VisibilityPublic:
return true
}
return false
}
// IsKnownOutputKind reports whether kind is a recognised OutputTarget.Kind.
// Used by Validate and by the Discord delivery switch.
//
// "channel_with_summary" is the v-research delivery kind: full output
// posts to a configured spam channel (skills.research.spam_channel_id)
// while a generated summary posts in the original channel as a reply
// linking back. Falls through to plain "channel" behaviour when the
// spam channel convar is unset or matches the invocation channel.
// Validate accepts this kind here; the Discord delivery switch in
// pkg/logic/skillexec/delivery_discord.go is the consumer side.
func IsKnownOutputKind(kind string) bool {
switch kind {
case "channel", "dm", "thread", "webui_only", "channel_with_summary":
return true
}
return false
}
// IsKnownInputType reports whether t is a recognised InputParam.Type.
// Used by Validate and by skillexec.validateInputs for coercion dispatch.
func IsKnownInputType(t string) bool {
switch t {
case "string", "int", "float", "bool", "user", "channel", "url":
return true
}
return false
}
+57
View File
@@ -0,0 +1,57 @@
package skill
import (
"context"
"testing"
"time"
)
func TestSkillToRunnable(t *testing.T) {
s := &Skill{
ID: "s1", Name: "summarizer", SystemPrompt: "summarize well", ModelTier: "fast",
MaxIterations: 4, MaxRuntime: 20 * time.Second, Tools: []string{"summarize", "now"},
}
r := s.ToRunnable()
if r.ID != "s1" || r.ModelTier != "fast" || r.MaxIterations != 4 || len(r.LowLevelTools) != 2 {
t.Fatalf("ToRunnable mapping wrong: %+v", r)
}
// A skill exposes a flat tool list, not a palette.
if len(r.SkillPalette) != 0 || len(r.SubAgentPalette) != 0 {
t.Errorf("skill should have empty palettes, got %+v", r)
}
}
func TestMemoryStoreVisibilityAndVersions(t *testing.T) {
ctx := context.Background()
m := NewMemory()
pub := &Skill{ID: "a", Name: "pub", OwnerID: "o1", Visibility: VisibilityPublic}
shared := &Skill{ID: "b", Name: "shr", OwnerID: "o1", Visibility: VisibilityShared, SharedWith: []string{"bob"}}
priv := &Skill{ID: "c", Name: "prv", OwnerID: "o1", Visibility: VisibilityPrivate}
for _, s := range []*Skill{pub, shared, priv} {
if err := m.Save(ctx, s); err != nil {
t.Fatal(err)
}
}
if ps, _ := m.ListPublic(ctx); len(ps) != 1 || ps[0].ID != "a" {
t.Errorf("ListPublic = %+v", ps)
}
if ss, _ := m.ListSharedWith(ctx, "bob"); len(ss) != 1 || ss[0].ID != "b" {
t.Errorf("ListSharedWith(bob) = %+v", ss)
}
if ss, _ := m.ListSharedWith(ctx, "carol"); len(ss) != 0 {
t.Errorf("ListSharedWith(carol) should be empty, got %+v", ss)
}
if all, _ := m.ListByOwner(ctx, "o1"); len(all) != 3 {
t.Errorf("ListByOwner = %d, want 3", len(all))
}
// Versions: newest-first, fetchable by id.
m.AppendVersion(ctx, SkillVersion{ID: "v1", SkillID: "a", Version: "1.0.0"})
m.AppendVersion(ctx, SkillVersion{ID: "v2", SkillID: "a", Version: "1.1.0"})
vs, _ := m.ListVersionsBySkill(ctx, "a", 10)
if len(vs) != 2 || vs[0].ID != "v2" {
t.Errorf("versions newest-first wrong: %+v", vs)
}
if got, err := m.GetVersionByID(ctx, "v1"); err != nil || got.Version != "1.0.0" {
t.Errorf("GetVersionByID: %v %+v", err, got)
}
}
+28
View File
@@ -0,0 +1,28 @@
package skill
import "time"
// SkillVersion is one immutable snapshot of a Skill at the moment it
// was saved. The skill_versions table is append-only; pruning is by
// retention policy in PruneOldVersions.
//
// Why: edit history with rollback (v3) and the admin pin gate (v3 Phase 4)
// both need a stable snapshot of the skill at a known version. The Snapshot
// field carries the FULL Skill struct so a later restore or pin produces
// the exact agent definition that was saved — system_prompt, tools,
// schedule, every field — not a synthesized partial snapshot.
//
// What: identity (UUID per snapshot) + skill ref + version-string copy +
// the full Skill payload + audit fields (saved_by, saved_at, edit_summary).
//
// Test: see skill_version_test.go for round-trip, list ordering, prune
// retention, and version-by-number disambiguation coverage.
type SkillVersion struct {
ID string // UUID per snapshot (NOT the skill's ID)
SkillID string // FK to skills.id (conceptually; not enforced by GORM)
Version string // Skill.Version at save time (semver)
Snapshot Skill // full Skill struct embedded; serialised as JSON
SavedBy string // caller member ID (or "" for builtin loader / pre-v3)
SavedAt time.Time // wall-clock save time
EditSummary string // optional human-readable note ("changed model tier", "...")
}
+44
View File
@@ -0,0 +1,44 @@
package skill
import (
"context"
"errors"
"time"
)
// ErrNotFound is returned when a skill (or version) lookup misses.
var ErrNotFound = errors.New("skill not found")
// SkillStore is the persistence seam for saved skills. This is the DELIBERATELY
// LEAN redesign of mort's 60-method skills.Storage: it carries only skill
// lifecycle (CRUD + visibility), versioning, and scheduling. The KV/file/quota
// sub-stores that were fused into mort's interface are NOT here — they are the
// tools/ store seams (KVStorage / FileStorage / QuotaProvider); email recipients
// and channel grants stay host concerns. A host backs this with its DB; Memory()
// is the zero-dependency default; contrib/store adds durable SQLite.
type SkillStore interface {
// Initialize prepares storage (idempotent).
Initialize(ctx context.Context) error
// --- lifecycle ---
Save(ctx context.Context, s *Skill) error
Get(ctx context.Context, id string) (*Skill, error)
GetByName(ctx context.Context, ownerID, name string) (*Skill, error)
Delete(ctx context.Context, id string) error
// --- listing / visibility ---
ListByOwner(ctx context.Context, ownerID string) ([]Skill, error)
ListPublic(ctx context.Context) ([]Skill, error)
ListSharedWith(ctx context.Context, memberID string) ([]Skill, error)
ListBuiltinByName(ctx context.Context, name string) (*Skill, error)
ListChatbotExposed(ctx context.Context) ([]Skill, error)
// --- scheduling ---
ListDueScheduled(ctx context.Context, now time.Time) ([]Skill, error)
MarkScheduledRun(ctx context.Context, skillID string, ranAt, nextAt time.Time) error
// --- versioning ---
AppendVersion(ctx context.Context, sv SkillVersion) error
ListVersionsBySkill(ctx context.Context, skillID string, limit int) ([]SkillVersion, error)
GetVersionByID(ctx context.Context, versionID string) (*SkillVersion, error)
}
+374
View File
@@ -0,0 +1,374 @@
package skill
import (
"fmt"
"strings"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/model"
)
// ChannelFilterChecker is the subset of ChannelFilterRegistry used by
// Validate to check that a skill references a registered channel filter.
//
// Why: kept narrow so tests can pass a tiny stub; full registry is
// declared in channel_filters.go.
type ChannelFilterChecker interface {
Has(name string) bool
}
// ModelTierChecker reports whether the given model tier or
// "provider/model" spec is recognised. Validate uses this to reject
// typos at save time.
//
// Why: tiers come from llms.tier.* convars (fast/standard/thinking by
// default) but admins may add custom tiers; explicit "provider/model"
// is also valid. Validate accepts anything non-empty matching either
// pattern — finer correctness is the LLM call's job.
type ModelTierChecker interface {
IsValid(spec string) bool
}
// defaultModelTierChecker accepts all registered tier names (via
// model.IsTierName) plus any "provider/model" form (string contains "/").
// Tests can substitute a strict checker via ValidateOpts.ModelTierChecker.
type defaultModelTierChecker struct{}
func (defaultModelTierChecker) IsValid(spec string) bool {
if spec == "" {
return false
}
if model.IsTierName(spec) {
return true
}
// Accept tier-with-reasoning (e.g. "thinking:high")
if i := strings.IndexByte(spec, ':'); i > 0 {
if model.IsTierName(spec[:i]) {
return true
}
}
// Accept explicit "provider/model" or "provider/model:reasoning"
return strings.ContainsRune(spec, '/')
}
// ValidateOpts customises what Validate accepts. All fields are optional;
// nil checkers fall back to permissive defaults.
//
// Why: Validate is called from save paths (which know the registries) and
// from tests (which want to control acceptance). Bundling the deps here
// keeps the Skill API stable.
type ValidateOpts struct {
// Filters is consulted when the skill declares a chatbot channel
// filter. nil → channel-filter validity is not checked (use only in
// tests).
Filters ChannelFilterChecker
// ModelTier checks the ModelTier spec. nil → defaultModelTierChecker.
ModelTier ModelTierChecker
// MinIntervalMinutes is the floor on the smallest gap between
// consecutive fires of a skill's cron schedule. Zero → use the
// package default (defaultMinScheduleIntervalMinutes). Tests pass an
// explicit value to exercise the boundary.
MinIntervalMinutes int
// AuthorIsAdmin tells Validate the author has admin privileges and
// may save with extended-tier bounds without ExtendedBounds=true.
// SaveUserSkill passes this from s.admin.IsAdmin(sk.AuthoredBy).
// Builtin loader sets this true to bypass the per-skill flag check
// (builtins are trusted infrastructure).
AuthorIsAdmin bool
// DefaultMaxIterations / DefaultMaxToolCalls / DefaultMaxRuntimeSecs
// override the package-default tier-1 caps. Zero → fall back to the
// constants below. Production wiring populates these from convars
// (skills.default_max_iterations etc.) so admins can adjust the
// default tier without a redeploy.
DefaultMaxIterations int
DefaultMaxToolCalls int
DefaultMaxRuntimeSecs int
// ExtendedMaxIterations / ExtendedMaxToolCalls / ExtendedMaxRuntimeSecs
// override the package-default tier-2 caps (the ceilings allowed when
// ExtendedBounds=true OR AuthorIsAdmin=true). Zero → fall back to the
// constants below.
ExtendedMaxIterations int
ExtendedMaxToolCalls int
ExtendedMaxRuntimeSecs int
}
// Tiered cap defaults. The DEFAULT tier is what a non-admin author sees
// without an explicit grant; the EXTENDED tier is what admin authors and
// admin-granted skills may use. Values are tuned in the v3 spec
// "Governance: tiered resource caps" section.
//
// The package's existing absolute ceilings (maxIterationsLimit=50 and
// maxRuntime=10m) act as outer floors / sanity bounds; the tier caps
// are the active gate at save time. Extended caps respect the absolute
// ceilings naturally (50 iter, 600s = 10min runtime).
const (
// Default tier — non-admin authors of skills without ExtendedBounds.
DefaultMaxIterations = 12
DefaultMaxToolCalls = 30
DefaultMaxRuntimeSecs = 60
// Extended tier — admin authors OR ExtendedBounds=true.
ExtendedMaxIterations = 50
ExtendedMaxToolCalls = 150
ExtendedMaxRuntimeSecs = 600 // 10m
maxIterationsLimit = 50
minRuntime = time.Second
maxRuntime = 10 * time.Minute
defaultMinScheduleIntervalMinutes = 30
// MaxTagsPerSkill caps the number of organisation tags any single
// skill may carry. Generous compared to typical taxonomies (GitHub
// allows ~10 topics/repo). The cap exists to prevent the list
// page's chip rendering from becoming unmanageable.
MaxTagsPerSkill = 16
// MaxTagLength is the per-tag character ceiling. Long enough for
// hyphenated phrases ("retro-gaming") but short enough that the
// list-page tag dropdown stays readable.
MaxTagLength = 32
)
// Validate enforces the skill spec invariants documented in the design
// spec ("Skill domain model" section). It is called at save time; the
// builtin loader skips authoring/share-safety checks but still runs
// Validate, so all callers can rely on a saved skill being well-formed.
//
// Why: spec rules are easy to violate by hand and silently break
// downstream (e.g. an unknown channel filter never exposes the skill to
// the chatbot). Every rule fails loudly here.
//
// What: returns the first error found; callers may surface it directly to
// users. opts may be the zero value, in which case channel-filter
// validation is skipped (tests).
//
// Test: each rejection branch has a dedicated unit test in
// validate_test.go.
func (s *Skill) Validate(opts ValidateOpts) error {
if s == nil {
return fmt.Errorf("skill is nil")
}
if strings.TrimSpace(s.Name) == "" {
return fmt.Errorf("skill name is required")
}
if strings.TrimSpace(s.SystemPrompt) == "" {
return fmt.Errorf("skill system prompt is required")
}
// ModelTier
tierCheck := opts.ModelTier
if tierCheck == nil {
tierCheck = defaultModelTierChecker{}
}
if !tierCheck.IsValid(s.ModelTier) {
return fmt.Errorf("unknown model tier %q (expected a tier alias or provider/model)", s.ModelTier)
}
// Schedule — empty means on-demand only. A non-empty value must be
// a valid cron expression (or one of the "daily" / "weekly"
// shorthands) AND have a smallest fire-gap >= the configured
// min-interval floor. Both checks share the package-level
// ParseSchedule helper so the scheduler runner uses the same parser.
if expr := strings.TrimSpace(s.Schedule); expr != "" {
sched, err := ParseSchedule(expr)
if err != nil {
return fmt.Errorf("schedule: %w", err)
}
minMinutes := opts.MinIntervalMinutes
if minMinutes == 0 {
minMinutes = defaultMinScheduleIntervalMinutes
}
floor := time.Duration(minMinutes) * time.Minute
if interval := ScheduleMinInterval(sched); interval < floor {
return fmt.Errorf(
"schedule %q runs more often than the minimum (every %s, floor is %s)",
expr, interval.Round(time.Second), floor)
}
}
// Iteration / call / runtime budgets. Zero is allowed — the executor
// substitutes a convar-backed default. Negative is always wrong.
// The absolute ceilings (maxIterationsLimit=50, maxRuntime=10m) are
// outer sanity bounds; the tier caps below are the active gate.
//
// Why admin bypass on the outer ceilings: builtins are trusted
// infrastructure (per the v2 "Builtin loader must bypass save-time
// gates" lesson). The builtin loader passes AuthorIsAdmin=true so
// trusted skills like `deepresearch` (max_iterations=100,
// max_runtime=45m) and `research` (max_runtime=15m) can validate
// without re-tuning the package-wide outer floor for everyone.
// Non-admin authors still hit the original ceilings AND the
// tier-based cap (default 12 iter / 60s runtime, extended 50 iter /
// 600s runtime) — both layers stay intact for the untrusted path.
if s.MaxIterations < 0 {
return fmt.Errorf("max_iterations must be >= 0, got %d", s.MaxIterations)
}
if !opts.AuthorIsAdmin && s.MaxIterations > maxIterationsLimit {
return fmt.Errorf("max_iterations must be 0..%d, got %d", maxIterationsLimit, s.MaxIterations)
}
if s.MaxToolCalls < 0 {
return fmt.Errorf("max_tool_calls must be >= 0, got %d", s.MaxToolCalls)
}
if s.MaxRuntime < 0 {
return fmt.Errorf("max_runtime must be 0 or positive, got %s", s.MaxRuntime)
}
if s.MaxRuntime > 0 && s.MaxRuntime < minRuntime {
return fmt.Errorf("max_runtime must be 0 or >= %s, got %s", minRuntime, s.MaxRuntime)
}
if !opts.AuthorIsAdmin && s.MaxRuntime > maxRuntime {
return fmt.Errorf("max_runtime must be 0 or in [%s..%s], got %s", minRuntime, maxRuntime, s.MaxRuntime)
}
// Tiered cap resolution: a skill saved by an admin OR a skill with
// ExtendedBounds=true (admin-granted) may use the extended tier;
// everything else saturates at the default tier. Builtins go through
// the loader's bypass path (AuthorIsAdmin=true).
defIter := opts.DefaultMaxIterations
if defIter == 0 {
defIter = DefaultMaxIterations
}
defCalls := opts.DefaultMaxToolCalls
if defCalls == 0 {
defCalls = DefaultMaxToolCalls
}
defRuntime := opts.DefaultMaxRuntimeSecs
if defRuntime == 0 {
defRuntime = DefaultMaxRuntimeSecs
}
extIter := opts.ExtendedMaxIterations
if extIter == 0 {
extIter = ExtendedMaxIterations
}
extCalls := opts.ExtendedMaxToolCalls
if extCalls == 0 {
extCalls = ExtendedMaxToolCalls
}
extRuntime := opts.ExtendedMaxRuntimeSecs
if extRuntime == 0 {
extRuntime = ExtendedMaxRuntimeSecs
}
maxIter := defIter
maxCalls := defCalls
maxRuntimeSecs := defRuntime
tier := "default"
hint := "; ask an admin to grant extended_bounds for higher"
if s.ExtendedBounds || opts.AuthorIsAdmin {
maxIter = extIter
maxCalls = extCalls
maxRuntimeSecs = extRuntime
tier = "extended"
hint = "" // already at the highest tier — no upgrade path
}
// Admin bypass on the tier cap: trusted infrastructure (builtins,
// admin-authored skills) may exceed the extended tier. The
// non-admin author still hits the tier cap above. See the
// "trusted infrastructure" rationale on the outer-ceiling block.
if !opts.AuthorIsAdmin {
if s.MaxIterations > maxIter {
return fmt.Errorf("max_iterations %d exceeds %s cap (%d)%s",
s.MaxIterations, tier, maxIter, hint)
}
if s.MaxToolCalls > maxCalls {
return fmt.Errorf("max_tool_calls %d exceeds %s cap (%d)%s",
s.MaxToolCalls, tier, maxCalls, hint)
}
if s.MaxRuntime > 0 && s.MaxRuntime > time.Duration(maxRuntimeSecs)*time.Second {
return fmt.Errorf("max_runtime %s exceeds %s cap (%ds)%s",
s.MaxRuntime, tier, maxRuntimeSecs, hint)
}
}
// Output target
if !IsKnownOutputKind(s.OutputTarget.Kind) {
return fmt.Errorf("unknown output_target.kind %q", s.OutputTarget.Kind)
}
// Input schema
seenInput := map[string]struct{}{}
for i, p := range s.InputSchema {
if strings.TrimSpace(p.Name) == "" {
return fmt.Errorf("input_schema[%d]: Name is required", i)
}
if !IsKnownInputType(p.Type) {
return fmt.Errorf("input_schema[%d] (%q): unknown type %q", i, p.Name, p.Type)
}
if _, dup := seenInput[p.Name]; dup {
return fmt.Errorf("input_schema: duplicate parameter name %q", p.Name)
}
seenInput[p.Name] = struct{}{}
}
// Tools
seenTool := map[string]struct{}{}
for _, t := range s.Tools {
if strings.TrimSpace(t) == "" {
return fmt.Errorf("tools: empty tool name")
}
if _, dup := seenTool[t]; dup {
return fmt.Errorf("tools: duplicate tool name %q", t)
}
seenTool[t] = struct{}{}
}
// Tags — normalise + bounds-check. The caller may pass user input
// directly; we trim, lowercase, dedup, and bound count + per-tag
// length. Mutating the slice in place is intentional so callers
// don't need a separate normalise pass.
//
// Why caps (16 tags / 32 chars): both are generous for human-
// curated organisation labels (compare to GitHub's 10 topics/repo
// + ~50 chars). The aim is rejecting accidental data dumps and
// keeping the list-page chip rendering manageable, not strict
// taxonomy enforcement.
if len(s.Tags) > MaxTagsPerSkill {
return fmt.Errorf("tags: too many (max %d, got %d)", MaxTagsPerSkill, len(s.Tags))
}
if len(s.Tags) > 0 {
seenTag := map[string]struct{}{}
out := make([]string, 0, len(s.Tags))
for _, raw := range s.Tags {
t := strings.ToLower(strings.TrimSpace(raw))
if t == "" {
continue
}
if len(t) > MaxTagLength {
return fmt.Errorf("tags: %q exceeds %d chars", t, MaxTagLength)
}
if _, dup := seenTag[t]; dup {
continue
}
seenTag[t] = struct{}{}
out = append(out, t)
}
s.Tags = out
}
// Visibility
if !IsKnownVisibility(s.Visibility) {
return fmt.Errorf("unknown visibility %q", s.Visibility)
}
if s.Visibility == VisibilityShared && len(s.SharedWith) == 0 {
return fmt.Errorf("visibility=shared requires non-empty shared_with")
}
// Chatbot exposure
if s.ExposeAsChatbotTool {
if strings.TrimSpace(s.ChatbotToolName) == "" {
return fmt.Errorf("expose_as_chatbot_tool=true requires chatbot_tool_name")
}
if strings.TrimSpace(s.ChatbotToolDescription) == "" {
return fmt.Errorf("expose_as_chatbot_tool=true requires chatbot_tool_description")
}
if strings.TrimSpace(s.ChatbotChannelFilter) == "" {
return fmt.Errorf("expose_as_chatbot_tool=true requires chatbot_channel_filter")
}
if opts.Filters != nil && !opts.Filters.Has(s.ChatbotChannelFilter) {
return fmt.Errorf("unknown chatbot_channel_filter %q (not registered)", s.ChatbotChannelFilter)
}
}
return nil
}
+7
View File
@@ -173,6 +173,13 @@ type Invocation struct {
CallerID string
ChannelID string
GuildID string
// DeliveryKind / DeliveryID name where the executor posts the run's output
// via run.Ports.Delivery — a host-interpreted Target ("channel"/"dm"/
// "thread"/...). An empty DeliveryID means the executor delivers nothing
// and the caller reads Result.Output itself (the synchronous default; the
// `.agent run` canary works this way).
DeliveryKind string
DeliveryID string
// CallerIsAdmin is true when the caller is a mort admin (Member.Admin).
// Populated by the executor at run dispatch via Bot.GetMember; defaults
// to false on any lookup failure (member not found, DB error, empty
+128
View File
@@ -0,0 +1,128 @@
// Package tools — v11 cite.
//
// Anti-hallucination forcing function. The convention: agents call
// cite(claim, url) for every numbered reference in their final
// answer. The tool verifies the URL appears in the run's
// touched-URL set (populated by web_search results +
// read_page/read_pdf/read_video). If yes → write to
// skill_run_sources, return {ok: true}. If no → return
// {ok: false, reason: "url_not_in_run_history"} and DO NOT write.
//
// Skills authored without this discipline don't lose anything;
// skills WITH it produce more reliable citations. The webui
// renders the skill_run_sources rows as a Sources panel on the
// run trace page — invisible to skills that don't use cite().
package tools
import (
"context"
"encoding/json"
"fmt"
"strings"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
// citeParams is the LLM-facing param struct.
type citeParams struct {
Claim string `json:"claim" description:"The claim or fact you are asserting (e.g. 'Mort was published in 1987')."`
URL string `json:"url" description:"The URL that supports the claim. MUST be a URL the agent has previously read via read_page/read_pdf/read_video or seen as a web_search result."`
}
// citeResponse is the JSON envelope returned to the agent.
//
// On success: ok=true, the skill_run_sources row was written.
// On failure: ok=false, reason=<one of the documented sentinels>.
type citeResponse struct {
OK bool `json:"ok"`
Reason string `json:"reason,omitempty"`
Claim string `json:"claim,omitempty"`
URL string `json:"url,omitempty"`
}
// NewCite constructs the v11 cite tool. cs may be nil — handler
// returns "not configured" at first call.
//
// The "anyone author / share-safe" permission shape matches every
// other v11 research-class tool. Skills that adopt cite() get the
// Sources panel automatically; skills that don't are unaffected.
func NewCite(cs CitationStorage) tool.Tool {
return tool.NewGatedTool[citeParams](
"cite",
"Record a citation: a claim + the URL that supports it. The URL MUST be one the agent has actually fetched via read_page/read_pdf/read_video or seen as a web_search result — citing a URL the agent never visited is rejected with reason 'url_not_in_run_history'. Successful citations populate the run's Sources panel.",
tool.Permission{
AuthoringRequirement: tool.RequirementAnyone,
OperatesOn: tool.ScopeGlobal,
SafeForShare: true,
Categories: []string{"citation"},
},
func(ctx context.Context, inv tool.Invocation, p citeParams) (string, error) {
if cs == nil {
return "", fmt.Errorf("cite: citation storage not configured")
}
claim := strings.TrimSpace(p.Claim)
urlStr := strings.TrimSpace(p.URL)
if claim == "" {
return marshalCite(citeResponse{
OK: false,
Reason: "claim_empty",
}), nil
}
if urlStr == "" {
return marshalCite(citeResponse{
OK: false,
Reason: "url_empty",
}), nil
}
if inv.RunID == "" {
// No run id → cite() can't verify history. Bail loud.
return marshalCite(citeResponse{
OK: false,
Reason: "no_run_context",
Claim: claim,
URL: urlStr,
}), nil
}
touched, err := cs.GetTouchedURLs(ctx, inv.RunID)
if err != nil {
return marshalCite(citeResponse{
OK: false,
Reason: fmt.Sprintf("touched_lookup_failed: %v", err),
Claim: claim,
URL: urlStr,
}), nil
}
if _, ok := touched[urlStr]; !ok {
return marshalCite(citeResponse{
OK: false,
Reason: "url_not_in_run_history",
Claim: claim,
URL: urlStr,
}), nil
}
if err := cs.RecordCitation(ctx, inv.RunID, urlStr, claim); err != nil {
return marshalCite(citeResponse{
OK: false,
Reason: fmt.Sprintf("record_failed: %v", err),
Claim: claim,
URL: urlStr,
}), nil
}
return marshalCite(citeResponse{
OK: true,
Claim: claim,
URL: urlStr,
}), nil
},
)
}
func marshalCite(r citeResponse) string {
b, err := json.Marshal(r)
if err != nil {
return fmt.Sprintf(`{"ok":false,"reason":"marshal_failed: %v"}`, err)
}
return string(b)
}
+327
View File
@@ -0,0 +1,327 @@
// Package tools — v12 classify.
//
// Classification primitive: text + categories → labels + per-category
// scores. Single-label mode (default) returns the top-1 category;
// multi-label mode returns every category whose score crosses the
// threshold.
//
// Why a dedicated tool (vs reusing extract_entities for one-of-N
// classification): classification has a typed result (labels[] +
// scores{}) that downstream agents consume programmatically. Folding
// it into extract_entities would force every author to re-spec the
// scoring schema.
//
// Score normalisation: the LLM's reply is normalised so each score
// lands in [0, 1]. The single-label result returns scores for ALL
// categories so the author can read the distribution; multi-label
// returns labels[] of categories above 0.5.
//
// Test: classify_test.go covers single-label, multi-label, score
// normalisation, > 20 categories rejected, unknown category in the
// reply silently dropped.
package tools
import (
"context"
"encoding/json"
"fmt"
"strconv"
"strings"
"gitea.stevedudenhoeffer.com/steve/executus/llmmeta"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
// classifyMaxInputBytes is the input cap.
const classifyMaxInputBytes = 16 * 1024
// classifyMaxCategories is the hard cap on category count.
const classifyMaxCategories = 20
// classifyMultiLabelThreshold is the score threshold above which a
// category appears in the labels[] array in multi-label mode.
const classifyMultiLabelThreshold = 0.5
// classifyFallbackMaxPerRun is the per-run cap when ClassifyConfig is
// nil.
const classifyFallbackMaxPerRun = 20
// ClassifyConfig is the narrow per-deployment config surface.
type ClassifyConfig interface {
MaxPerRun(ctx context.Context) int
}
// classifyArgs is the LLM-facing param struct.
type classifyArgs struct {
Text string `json:"text" description:"The text to classify. Required. Capped at 16KB."`
Categories []string `json:"categories" description:"List of categories to score the text against. Required. Max 20."`
MultiLabel bool `json:"multi_label,omitempty" description:"When true, returns every category scoring above 0.5. Default false → single-label (top-1) result."`
}
type classifyResult struct {
Labels []string `json:"labels,omitempty"`
Scores map[string]float64 `json:"scores,omitempty"`
ModelUsed string `json:"model_used,omitempty"`
RawReply string `json:"raw_reply,omitempty"`
Error string `json:"error,omitempty"`
BudgetMsg string `json:"budget_message,omitempty"`
}
// NewClassify constructs the classify tool.
func NewClassify(helper *llmmeta.Helper, cfg ClassifyConfig, budget SearchBudget) tool.Tool {
return tool.NewGatedTool[classifyArgs](
"classify",
"Classify text into one of N categories (or multiple via multi_label=true). Returns labels[] (top-1 by default) + scores{category: 0..1}. Counts against per-run and 7-day cost budgets.",
tool.Permission{
AuthoringRequirement: tool.RequirementAnyone,
OperatesOn: tool.ScopeCaller,
SafeForShare: true,
Categories: []string{"llm-meta", "cost-bearing"},
},
func(ctx context.Context, inv tool.Invocation, args classifyArgs) (string, error) {
if helper == nil {
return "", fmt.Errorf("classify: not configured")
}
text := args.Text
if strings.TrimSpace(text) == "" {
return marshalClassifyResult(classifyResult{Error: "text is empty"}), nil
}
if len(args.Categories) == 0 {
return marshalClassifyResult(classifyResult{Error: "categories is empty"}), nil
}
if len(args.Categories) > classifyMaxCategories {
return marshalClassifyResult(classifyResult{
Error: fmt.Sprintf("too many categories (%d > %d)", len(args.Categories), classifyMaxCategories),
}), nil
}
// Trim + dedupe categories so the LLM sees a clean
// schema. Order is preserved for the prompt; the result
// map is order-agnostic.
categories := make([]string, 0, len(args.Categories))
seen := make(map[string]bool, len(args.Categories))
for _, c := range args.Categories {
c = strings.TrimSpace(c)
if c == "" || seen[c] {
continue
}
seen[c] = true
categories = append(categories, c)
}
if len(categories) == 0 {
return marshalClassifyResult(classifyResult{Error: "categories has no non-empty entries"}), nil
}
if len(text) > classifyMaxInputBytes {
text = truncateUTF8(text, classifyMaxInputBytes)
}
// Per-run budget gate.
if budget == nil {
maxPerRun := classifyFallbackMaxPerRun
if cfg != nil {
maxPerRun = cfg.MaxPerRun(ctx)
}
budget = NewInMemorySearchBudget(map[string]int{
"classify": maxPerRun,
})
}
count, max, exceeded := budget.CheckAndIncrement(ctx, inv.RunID, "classify")
if exceeded {
return marshalClassifyResult(classifyResult{
Error: "classify_budget_exceeded",
BudgetMsg: fmt.Sprintf("per-run classify budget exceeded (%d/%d). Ask an admin to raise skills.classify.max_per_run.", count, max),
}), nil
}
systemPrompt := "You classify text into a fixed set of categories. Return ONLY JSON. Score each category in [0,1] (1 = perfect fit). Sum of all scores does NOT need to be 1 — high overlap across categories is allowed."
userPrompt := buildClassifyPrompt(text, categories, args.MultiLabel)
res, callErr := helper.Call(ctx, llmmeta.CallSpec{
Tier: "fast",
SystemPrompt: systemPrompt,
UserPrompt: userPrompt,
MaxOutputTokens: 2048,
ResponseFormat: "json",
RetryOnMalformedJSON: true,
ToolName: "classify",
RunID: inv.RunID,
SkillID: inv.SkillID,
CallerID: inv.CallerID,
})
if callErr != nil {
return "", callErr
}
if !res.Success {
kind := res.ErrorKind
if kind == "" {
kind = "llm_unavailable"
}
return marshalClassifyResult(classifyResult{Error: kind}), nil
}
if res.ErrorKind == llmmeta.ErrorKindMalformedJSON || res.Parsed == nil {
return marshalClassifyResult(classifyResult{
Error: "classification_failed",
RawReply: res.Text,
ModelUsed: res.ModelUsed,
}), nil
}
parsedMap, ok := res.Parsed.(map[string]any)
if !ok {
return marshalClassifyResult(classifyResult{
Error: "classification_failed_not_object",
RawReply: res.Text,
ModelUsed: res.ModelUsed,
}), nil
}
scores := normaliseClassifyScores(parsedMap, categories)
labels := selectClassifyLabels(scores, categories, args.MultiLabel)
return marshalClassifyResult(classifyResult{
Labels: labels,
Scores: scores,
ModelUsed: res.ModelUsed,
}), nil
},
)
}
// buildClassifyPrompt composes the user message.
func buildClassifyPrompt(text string, categories []string, multiLabel bool) string {
var sb strings.Builder
sb.WriteString("Classify the text below.\n\nCategories:\n")
for _, c := range categories {
sb.WriteString("- ")
sb.WriteString(c)
sb.WriteString("\n")
}
sb.WriteString("\nText:\n")
sb.WriteString(text)
sb.WriteString("\n\nReturn ONLY a JSON object: {\"scores\": {\"<category>\": <0..1 float>, ...}}.")
if multiLabel {
sb.WriteString(" The same text may score high in MULTIPLE categories — score each independently.")
} else {
sb.WriteString(" Score each category; the highest-scoring one will be the chosen label.")
}
return sb.String()
}
// normaliseClassifyScores extracts the scores map from the LLM's
// reply and clamps each value into [0, 1]. Categories absent from the
// reply default to 0.
//
// Why we accept either {"scores": {...}} or {...}: some models reply
// with the inner object directly, dropping the wrapping key. Both
// shapes are valid as long as the keys match the requested category
// names.
func normaliseClassifyScores(parsed map[string]any, categories []string) map[string]float64 {
scoresIn, ok := parsed["scores"].(map[string]any)
if !ok {
// Accept the bare-map shape too.
scoresIn = parsed
}
out := make(map[string]float64, len(categories))
for _, c := range categories {
v, has := scoresIn[c]
if !has {
out[c] = 0
continue
}
f, ok := coerceClassifyScore(v)
if !ok {
out[c] = 0
continue
}
// Clamp into [0, 1].
if f < 0 {
f = 0
}
if f > 1 {
f = 1
}
out[c] = f
}
return out
}
// coerceClassifyScore reads a JSON value as a float in [0, 1]. Accepts
// floats, ints, and percent-strings ("85%" → 0.85).
func coerceClassifyScore(raw any) (float64, bool) {
switch v := raw.(type) {
case float64:
return v, true
case int:
return float64(v), true
case int64:
return float64(v), true
case string:
trimmed := strings.TrimSpace(v)
hasPct := strings.HasSuffix(trimmed, "%")
s := strings.TrimSuffix(trimmed, "%")
// strconv.ParseFloat (unlike fmt.Sscanf %f) rejects trailing garbage,
// so "50extra" / "0.5x" are refused instead of silently parsed as 50/0.5.
f, err := strconv.ParseFloat(strings.TrimSpace(s), 64)
if err == nil {
if hasPct {
f = f / 100.0
}
return f, true
}
}
return 0, false
}
// selectClassifyLabels picks the labels to surface. Single-label mode
// returns the highest-scoring category. Multi-label returns every
// category above the threshold (sorted by score desc for stable
// rendering).
func selectClassifyLabels(scores map[string]float64, categories []string, multiLabel bool) []string {
if multiLabel {
var labels []string
for _, c := range categories {
if scores[c] >= classifyMultiLabelThreshold {
labels = append(labels, c)
}
}
// Sort labels by score desc, then category-list order for ties.
sortClassifyLabelsByScore(labels, scores)
return labels
}
// Single-label: top-1.
bestCat := ""
bestScore := -1.0
for _, c := range categories {
if scores[c] > bestScore {
bestScore = scores[c]
bestCat = c
}
}
// No category fit: an all-zero score set must not yield a false-positive
// top-1 (the first category trivially beats the -1.0 sentinel). Returning
// no label keeps "nothing matched" distinguishable from "category A won".
if bestCat == "" || bestScore <= 0 {
return nil
}
return []string{bestCat}
}
// sortClassifyLabelsByScore sorts labels desc by score. Stable on
// ties (preserves category-list order).
func sortClassifyLabelsByScore(labels []string, scores map[string]float64) {
for i := 1; i < len(labels); i++ {
j := i
for j > 0 && scores[labels[j]] > scores[labels[j-1]] {
labels[j], labels[j-1] = labels[j-1], labels[j]
j--
}
}
}
func marshalClassifyResult(r classifyResult) string {
b, err := json.Marshal(r)
if err != nil {
return fmt.Sprintf(`{"error":"marshal_failed: %v"}`, err)
}
return string(b)
}
+222
View File
@@ -0,0 +1,222 @@
// create_file_url mints a public-token URL (mort.sh/files/<token>)
// that resolves to a saved file_id. Use it for artifacts that are too
// large for Discord (>25 MiB), need a stable link to share outside
// Discord, or where the recipient is not in mort's auth domain.
//
// Why a separate tool (vs always returning a URL from file_save):
// most files are private working state — only some need a public URL,
// and minting one is a deliberate act. Decoupling save from
// publication keeps the storage layer cheap (no token row per file)
// and the audit clean (you can grep skill_file_tokens for "who
// published what").
//
// Cycle-break: this tool can't import pkg/logic/skills directly
// (pkg/logic/skills imports pkg/skilltools). The narrow interface
// FileTokenMinter is declared here; mort.go bridges to
// *skills.System.Storage() at wiring time.
package tools
import (
"context"
"crypto/rand"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"strings"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
// FileToken is the wire-shape of the storage row that backs the
// public /files/<token> URL. Mirrors pkg/logic/skills.FileToken
// field-for-field; the adapter in mort.go is a struct copy.
//
// Why mirror (vs import skills.FileToken): same cycle constraint as
// FileDomainMeta / KVDomainEntry — the tool layer cannot import
// pkg/logic/skills.
type FileToken struct {
Token string
FileID string
SkillID string
CallerID string
CreatedAt time.Time
ExpiresAt *time.Time
MaxViews *int
Views int
}
// FileTokenMinter is the narrow interface the create_file_url tool
// needs to persist a new token. Production wires to
// *skills.gormStorage via a thin adapter in mort.go.
type FileTokenMinter interface {
SaveFileToken(ctx context.Context, t FileToken) error
}
// Caps for create_file_url. Public so tests can assert against them.
const (
// DefaultFileURLExpiry is the default lifetime applied when the
// caller doesn't supply expires_in_seconds.
DefaultFileURLExpiry = 24 * time.Hour
// MaxFileURLExpiry is the per-tool hard cap. 30 days is generous
// enough for "share this report with someone" without becoming
// effectively-permanent. Operators can lower via the
// SkillFileURLConfigProvider; this is the floor below which the
// admin gate doesn't apply.
MaxFileURLExpiry = 30 * 24 * time.Hour
// MaxFileURLViews is the per-tool hard cap on max_views. 1000 is
// the largest value an LLM might plausibly set; anything beyond
// is "unlimited" semantically and the caller should leave the
// field absent.
MaxFileURLViews = 1000
)
type createFileURLArgs struct {
FileID string `json:"file_id" description:"file_id previously saved by this skill (from file_save, code_exec, etc)."`
ExpiresInSeconds int `json:"expires_in_seconds,omitempty" description:"How long the URL stays valid in seconds. Default 86400 (24h). Max 2592000 (30 days)."`
MaxViews int `json:"max_views,omitempty" description:"Optional cap on the number of times the URL can be fetched. Max 1000. Omit (or 0) for unlimited within the lifetime."`
}
type createFileURLResult struct {
URL string `json:"url"`
Token string `json:"token"`
ExpiresAt string `json:"expires_at,omitempty"` // RFC3339
MaxViews int `json:"max_views,omitempty"`
Note string `json:"note,omitempty"`
}
// NewCreateFileURL constructs the create_file_url tool. nil minter →
// "not configured" at execute time; nil fileStorage same. baseURL is
// the public site (e.g. "https://mort.sh"); the path "/files/<token>"
// is appended.
//
// Permission shape: anyone-authoring + caller-scope + share-safe +
// files/discord/composition. The "publishing" act is a tool call,
// not a save-time / share-time concern — every caller of a shared
// skill mints into their own audit trail.
func NewCreateFileURL(minter FileTokenMinter, fileStorage FileStorage, baseURL string) tool.Tool {
baseURL = strings.TrimRight(baseURL, "/")
return tool.NewGatedTool[createFileURLArgs](
"create_file_url",
"Mint a public URL (mort.sh/files/<token>) for a saved file_id. Use for files too large for Discord (>25 MiB) or when a stable link is preferred over an attachment. Default expiry 24h; max 30 days. Optional view-count cap (max 1000).",
tool.Permission{
AuthoringRequirement: tool.RequirementAnyone,
OperatesOn: tool.ScopeCaller,
SafeForShare: true,
Categories: []string{"files", "discord"},
},
func(ctx context.Context, inv tool.Invocation, args createFileURLArgs) (string, error) {
if minter == nil || fileStorage == nil {
return "", fmt.Errorf("create_file_url: not configured")
}
if strings.TrimSpace(args.FileID) == "" {
return "", fmt.Errorf("create_file_url: file_id required")
}
// Cross-skill rejection: the file MUST belong to the
// calling skill. Without this, a hostile skill could mint
// a URL for ANY file by file_id.
meta, _, err := fileStorage.FileGet(ctx, args.FileID)
if err != nil {
if errors.Is(err, ErrFileNotFound) {
return "", fmt.Errorf("create_file_url: file_id %q not found", args.FileID)
}
return "", fmt.Errorf("create_file_url: %w", err)
}
grantedViaDescendant := false
if meta.SkillID != inv.SkillID {
if !descendantFileGrant(ctx, fileStorage, inv, meta.SkillID) {
return "", fmt.Errorf("create_file_url: file_id %q does not belong to this skill (cross-skill refs rejected)", args.FileID)
}
grantedViaDescendant = true
}
// Scope gate — this is a PUBLICATION primitive (it mints an
// unauthenticated link), so it must enforce the same per-user/per-run
// scope isolation the read tools do: a same-skill caller must not be
// able to publish a file scoped to another user/run. Skipped only for
// the descendant-grant case (the worker's file scope is the worker's
// run, not the caller's).
if !grantedViaDescendant {
if err := ValidateScope(inv, meta.Scope, inv.CallerIsAdmin); err != nil {
return "", fmt.Errorf("create_file_url: %w", err)
}
}
// Resolve expiry. Clamp the caller's seconds BEFORE the multiply so a
// huge value can't overflow int64 nanoseconds into a negative
// duration that slips under the max-expiry cap (minting an
// already-expired token).
expiry := DefaultFileURLExpiry
if args.ExpiresInSeconds > 0 {
maxSecs := int(MaxFileURLExpiry / time.Second)
secs := args.ExpiresInSeconds
if secs > maxSecs {
secs = maxSecs
}
expiry = time.Duration(secs) * time.Second
}
if expiry > MaxFileURLExpiry {
expiry = MaxFileURLExpiry
}
expiresAt := time.Now().Add(expiry)
// Resolve max_views.
var maxViews *int
if args.MaxViews > 0 {
mv := args.MaxViews
if mv > MaxFileURLViews {
mv = MaxFileURLViews
}
maxViews = &mv
}
// Mint a 32-byte random token, base64url-encoded
// (padless). 43 chars long; the storage column is 64 so
// there's room to grow without a migration.
token, err := mintFileURLToken()
if err != nil {
return "", fmt.Errorf("create_file_url: token generation: %w", err)
}
// Persist.
if err := minter.SaveFileToken(ctx, FileToken{
Token: token,
FileID: args.FileID,
SkillID: inv.SkillID,
CallerID: inv.CallerID,
ExpiresAt: &expiresAt,
MaxViews: maxViews,
}); err != nil {
return "", fmt.Errorf("create_file_url: save: %w", err)
}
url := baseURL + "/files/" + token
res := createFileURLResult{
URL: url,
Token: token,
ExpiresAt: expiresAt.UTC().Format(time.RFC3339),
Note: "URL is public — anyone with the link can fetch this file until it expires or the view cap is reached.",
}
if maxViews != nil {
res.MaxViews = *maxViews
}
b, err := json.Marshal(res)
if err != nil {
return "", fmt.Errorf("create_file_url: marshal: %w", err)
}
return string(b), nil
},
)
}
// mintFileURLToken returns a 32-byte random token, base64url-encoded
// without padding. ~190 bits of entropy, well above the
// collision-resistance threshold for the 64-char storage column.
func mintFileURLToken() (string, error) {
var b [32]byte
if _, err := rand.Read(b[:]); err != nil {
return "", err
}
return base64.RawURLEncoding.EncodeToString(b[:]), nil
}
+342
View File
@@ -0,0 +1,342 @@
// Package tools — v12 extract_entities.
//
// Structured-output workhorse: text + field schema → typed JSON
// object. The author specifies which fields they want and what
// types; the tool builds an appropriate prompt, asks for JSON, and
// validates + coerces the response back into the requested types.
//
// Why a structured-output tool (vs forcing the agent to write its
// own prompt): every agentic skill that needs to "pull X, Y, Z out
// of unstructured text" otherwise re-invents the same prompt-
// engineering pattern. extract_entities centralises it so authors
// just describe the schema.
//
// Type coercion: an LLM responding with "42" when an int field was
// requested is normal noise. The tool coerces strings to
// int/float/bool when possible; coercion failures land the field in
// missing_fields rather than the entities map.
//
// Test: extract_entities_test.go covers happy path, missing optional
// field, missing required field surfaces in missing_fields, malformed
// JSON retry, second-attempt failure, type coercion (string→int,
// string→bool), unknown field type rejected at args validation.
package tools
import (
"context"
"encoding/json"
"fmt"
"strconv"
"strings"
"gitea.stevedudenhoeffer.com/steve/executus/llmmeta"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
// extractEntitiesMaxInputBytes is the hard input cap.
const extractEntitiesMaxInputBytes = 32 * 1024
// extractEntitiesFallbackMaxPerRun is the per-run cap when
// ExtractEntitiesConfig is nil.
const extractEntitiesFallbackMaxPerRun = 10
// ExtractEntitiesConfig is the narrow per-deployment config surface
// extract_entities reads at execute time.
type ExtractEntitiesConfig interface {
MaxPerRun(ctx context.Context) int
}
// extractField is one row in the schema the agent supplies. The four
// supported types match the JSON-shape primitives we can validate +
// coerce reliably.
//
// Why an enum-shaped Type field (vs free-form): we need to know how
// to validate the LLM's reply. Free-form ("integer", "Number",
// "boolean") would invite typos that silently miss the validation.
type extractField struct {
Name string `json:"name" description:"Field name to populate (e.g. 'author', 'year_published'). Becomes a key in the returned entities object."`
Description string `json:"description" description:"Short description of what to extract (e.g. 'the book author', 'the year the article was published'). Helps the model find the right value."`
Type string `json:"type" description:"One of: 'string', 'int', 'float', 'bool', 'list_of_strings'. Determines how the LLM's reply is validated and coerced."`
Required bool `json:"required,omitempty" description:"When true, a missing/uncoercible value lands in missing_fields rather than skipping silently."`
}
// extractEntitiesArgs is the LLM-facing param struct.
type extractEntitiesArgs struct {
Text string `json:"text" description:"The text to extract from. Required. Capped at 32KB."`
Fields []extractField `json:"fields" description:"Schema describing what to extract. Each field has name, description, type, and optional required flag."`
}
type extractEntitiesResult struct {
Entities map[string]any `json:"entities,omitempty"`
MissingFields []string `json:"missing_fields,omitempty"`
ModelUsed string `json:"model_used,omitempty"`
RawReply string `json:"raw_reply,omitempty"`
Error string `json:"error,omitempty"`
BudgetMsg string `json:"budget_message,omitempty"`
}
// validExtractTypes is the closed set of Type strings the tool
// accepts. Anything else is rejected at args validation.
var validExtractTypes = map[string]bool{
"string": true,
"int": true,
"float": true,
"bool": true,
"list_of_strings": true,
}
// NewExtractEntities constructs the extract_entities tool.
func NewExtractEntities(helper *llmmeta.Helper, cfg ExtractEntitiesConfig, budget SearchBudget) tool.Tool {
return tool.NewGatedTool[extractEntitiesArgs](
"extract_entities",
"Extract structured fields from unstructured text via a fast LLM. Caller supplies a schema (each field has name + description + type + required); tool returns an entities object with values matching the requested types. Types: string, int, float, bool, list_of_strings. Counts against per-run and 7-day cost budgets.",
tool.Permission{
AuthoringRequirement: tool.RequirementAnyone,
OperatesOn: tool.ScopeCaller,
SafeForShare: true,
Categories: []string{"llm-meta", "cost-bearing"},
},
func(ctx context.Context, inv tool.Invocation, args extractEntitiesArgs) (string, error) {
if helper == nil {
return "", fmt.Errorf("extract_entities: not configured")
}
text := args.Text
if strings.TrimSpace(text) == "" {
return marshalExtractEntities(extractEntitiesResult{Error: "text is empty"}), nil
}
if len(args.Fields) == 0 {
return marshalExtractEntities(extractEntitiesResult{Error: "fields is empty"}), nil
}
// Validate each field's Type before paying for an LLM
// call.
for _, f := range args.Fields {
if strings.TrimSpace(f.Name) == "" {
return marshalExtractEntities(extractEntitiesResult{Error: "field with empty name"}), nil
}
if !validExtractTypes[strings.ToLower(strings.TrimSpace(f.Type))] {
return marshalExtractEntities(extractEntitiesResult{
Error: fmt.Sprintf("field %q has unsupported type %q (allowed: string|int|float|bool|list_of_strings)", f.Name, f.Type),
}), nil
}
}
if len(text) > extractEntitiesMaxInputBytes {
text = truncateUTF8(text, extractEntitiesMaxInputBytes)
}
// Per-run budget gate.
if budget == nil {
maxPerRun := extractEntitiesFallbackMaxPerRun
if cfg != nil {
maxPerRun = cfg.MaxPerRun(ctx)
}
budget = NewInMemorySearchBudget(map[string]int{
"extract_entities": maxPerRun,
})
}
count, max, exceeded := budget.CheckAndIncrement(ctx, inv.RunID, "extract_entities")
if exceeded {
return marshalExtractEntities(extractEntitiesResult{
Error: "extract_entities_budget_exceeded",
BudgetMsg: fmt.Sprintf("per-run extract_entities budget exceeded (%d/%d). Ask an admin to raise skills.extract_entities.max_per_run.", count, max),
}), nil
}
systemPrompt := "You extract structured data from unstructured text. Return ONLY valid JSON with the requested keys. If a value is not present in the text, omit the key. Do NOT invent values."
userPrompt := buildExtractPrompt(text, args.Fields)
res, callErr := helper.Call(ctx, llmmeta.CallSpec{
Tier: "fast",
SystemPrompt: systemPrompt,
UserPrompt: userPrompt,
MaxOutputTokens: 4096,
ResponseFormat: "json",
RetryOnMalformedJSON: true,
ToolName: "extract_entities",
RunID: inv.RunID,
SkillID: inv.SkillID,
CallerID: inv.CallerID,
})
if callErr != nil {
return "", callErr
}
if !res.Success {
kind := res.ErrorKind
if kind == "" {
kind = "llm_unavailable"
}
return marshalExtractEntities(extractEntitiesResult{Error: kind}), nil
}
// Second-failure malformed JSON (success=true but parsed
// is nil and ErrorKind=malformed_json). Surface the raw
// reply so the agent can salvage.
if res.ErrorKind == llmmeta.ErrorKindMalformedJSON || res.Parsed == nil {
return marshalExtractEntities(extractEntitiesResult{
Error: "extraction_failed",
RawReply: res.Text,
ModelUsed: res.ModelUsed,
}), nil
}
parsedMap, ok := res.Parsed.(map[string]any)
if !ok {
return marshalExtractEntities(extractEntitiesResult{
Error: "extraction_failed_not_object",
RawReply: res.Text,
ModelUsed: res.ModelUsed,
}), nil
}
entities, missing := coerceExtractedEntities(parsedMap, args.Fields)
return marshalExtractEntities(extractEntitiesResult{
Entities: entities,
MissingFields: missing,
ModelUsed: res.ModelUsed,
}), nil
},
)
}
// buildExtractPrompt composes the user message describing the schema
// + source text.
func buildExtractPrompt(text string, fields []extractField) string {
var sb strings.Builder
sb.WriteString("Extract the following fields from the text below. Return a JSON object with the field names as keys.\n\nFields:\n")
for _, f := range fields {
fmt.Fprintf(&sb, "- %s (%s): %s", f.Name, f.Type, f.Description)
if f.Required {
sb.WriteString(" [required]")
}
sb.WriteString("\n")
}
sb.WriteString("\nText:\n")
sb.WriteString(text)
return sb.String()
}
// coerceExtractedEntities walks the LLM's response, validating + (when
// possible) coercing each value to the requested type. Required fields
// missing or uncoercible land in missing[]; optional fields silently
// drop.
func coerceExtractedEntities(parsed map[string]any, fields []extractField) (map[string]any, []string) {
entities := make(map[string]any, len(fields))
var missing []string
for _, f := range fields {
raw, present := parsed[f.Name]
if !present || raw == nil {
if f.Required {
missing = append(missing, f.Name)
}
continue
}
value, ok := coerceFieldValue(raw, f.Type)
if !ok {
if f.Required {
missing = append(missing, f.Name)
}
continue
}
entities[f.Name] = value
}
return entities, missing
}
// coerceFieldValue attempts to convert raw to the requested type.
// Returns (value, true) on success or (nil, false) on failure.
//
// Why coerce (vs strict reject): LLMs frequently reply with strings
// that contain numbers ("42") or pseudo-booleans ("yes"). Strict
// rejection would force every author to clean the response themselves.
// Coercion is conservative — string "42" → int 42 succeeds; string
// "forty-two" → int 42 fails (the agent never asked for word-form
// parsing).
func coerceFieldValue(raw any, fieldType string) (any, bool) {
switch strings.ToLower(strings.TrimSpace(fieldType)) {
case "string":
switch v := raw.(type) {
case string:
return v, true
case float64:
return strconv.FormatFloat(v, 'f', -1, 64), true
case bool:
return strconv.FormatBool(v), true
}
return nil, false
case "int":
switch v := raw.(type) {
case float64:
// JSON numbers are float64 by default.
if v == float64(int64(v)) {
return int64(v), true
}
return nil, false
case string:
if n, err := strconv.ParseInt(strings.TrimSpace(v), 10, 64); err == nil {
return n, true
}
// Try float-string-with-zero-fractional ("42.0").
if f, err := strconv.ParseFloat(strings.TrimSpace(v), 64); err == nil && f == float64(int64(f)) {
return int64(f), true
}
}
return nil, false
case "float":
switch v := raw.(type) {
case float64:
return v, true
case string:
if f, err := strconv.ParseFloat(strings.TrimSpace(v), 64); err == nil {
return f, true
}
}
return nil, false
case "bool":
switch v := raw.(type) {
case bool:
return v, true
case string:
s := strings.ToLower(strings.TrimSpace(v))
switch s {
case "true", "yes", "1", "y":
return true, true
case "false", "no", "0", "n":
return false, true
}
case float64:
return v != 0, true
}
return nil, false
case "list_of_strings":
switch v := raw.(type) {
case []any:
out := make([]string, 0, len(v))
for _, e := range v {
if s, ok := e.(string); ok {
out = append(out, s)
} else {
// Mixed-type lists fail the type contract.
return nil, false
}
}
return out, true
case string:
// Single-string can be lifted into a one-element list.
return []string{v}, true
}
return nil, false
}
return nil, false
}
func marshalExtractEntities(r extractEntitiesResult) string {
b, err := json.Marshal(r)
if err != nil {
return fmt.Sprintf(`{"error":"marshal_failed: %v"}`, err)
}
return string(b)
}
+79
View File
@@ -0,0 +1,79 @@
// file_delete removes a saved file by its file_id. Decrements the
// underlying blob's refcount in storage; the blob row is removed when
// refcount hits zero.
//
// Why scope is checked POST-fetch (mirrors file_get): file_id is the
// only key the caller has; we must read the row to know the scope.
package tools
import (
"context"
"errors"
"fmt"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
type fileDeleteArgs struct {
FileID string `json:"file_id" description:"Opaque file ID returned by file_save or file_list."`
}
// NewFileDelete constructs the file_delete tool. storage nil → "not
// configured" at execute time.
func NewFileDelete(storage FileStorage) tool.Tool {
return tool.NewGatedTool[fileDeleteArgs](
"file_delete",
"Remove a saved file by file_id. Returns 'ok' on success or 'not_found' if no file matched.",
tool.Permission{
AuthoringRequirement: tool.RequirementAnyone,
OperatesOn: tool.ScopeCaller,
SafeForShare: true,
Categories: []string{"storage", "write"},
},
func(ctx context.Context, inv tool.Invocation, args fileDeleteArgs) (string, error) {
if storage == nil {
return "", fmt.Errorf("file_delete: not configured")
}
if args.FileID == "" {
return "", fmt.Errorf("file_delete: file_id required")
}
// Fetch first so we can validate scope before deleting. The
// extra read is acceptable for a write path that's not in
// the hot loop, and it preserves the cross-skill /
// cross-user safety story.
meta, _, err := storage.FileGet(ctx, args.FileID)
if err != nil {
if errors.Is(err, ErrFileNotFound) {
return "not_found", nil
}
return "", fmt.Errorf("file_delete: %w", err)
}
// Honor the descendant grant like the read tools do, so a parent
// orchestrator can clean up a worker's artifacts (gadfly flagged the
// asymmetry: delete previously rejected cross-skill outright).
grantedViaDescendant := false
if meta.SkillID != inv.SkillID {
if !descendantFileGrant(ctx, storage, inv, meta.SkillID) {
return "", fmt.Errorf("file_delete: file does not belong to this skill")
}
grantedViaDescendant = true
}
if !grantedViaDescendant {
if err := ValidateScope(inv, meta.Scope, inv.CallerIsAdmin); err != nil {
return "", fmt.Errorf("file_delete: %w", err)
}
}
if err := storage.FileDelete(ctx, args.FileID); err != nil {
if errors.Is(err, ErrFileNotFound) {
// Race: row was deleted between FileGet and
// FileDelete. Surface as a clean miss.
return "not_found", nil
}
return "", fmt.Errorf("file_delete: %w", err)
}
return "ok", nil
},
)
}
+58
View File
@@ -0,0 +1,58 @@
// file_descendant_grant.go — the cross-skill file-access escape hatch
// for parent → spawned-worker handoff.
//
// The blanket rule everywhere in this package is "a file belongs to
// the skill that saved it; cross-skill refs are rejected". That rule
// breaks the agent_spawn flow: a worker saves a chart with file_save
// under ITS ephemeral ID, returns the file_id as text, and the parent
// (which orchestrated the whole thing) can't attach, read, or host it.
// Observed live on the second spawn test — the chart never reached
// Discord; general could only apologise with the file_id.
//
// The grant: a caller may access a file whose owning skill/agent
// PRODUCED A RUN THAT DESCENDS FROM THE CALLER'S CURRENT RUN. In other
// words: you may touch the artifacts of workers you (transitively)
// dispatched in this very tree — output you were already entitled to
// see as their tool results. You may NOT touch files from siblings,
// ancestors, other trees, or unrelated skills; those still reject.
//
// Why an optional interface upgrade (vs a new constructor dep on
// every file tool): six tools enforce the ownership rule, each with
// its own narrow storage interface — threading a new dep through all
// of them churns every signature and test fake. Instead, the
// production storage adapter (mort.go's skillsFileStorageAdapter,
// which backs ALL of those interfaces) additionally implements
// DescendantRunChecker; the tools type-assert at the rejection site.
// Fakes that don't implement it keep the strict behaviour — the grant
// is fail-closed everywhere. Same pattern as KVHistoryRecorder (v7).
package tools
import (
"context"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
// DescendantRunChecker reports whether ownerSkillID (the file's owning
// skill or agent ID — e.g. a spawned worker's "eph-…" ID) produced a
// run that is a DESCENDANT of callerRunID. Production walks the audit
// parent_run_id chain; see mort_skills_storage_adapters.go.
type DescendantRunChecker interface {
IsDescendantProducer(ctx context.Context, ownerSkillID, callerRunID string) (bool, error)
}
// descendantFileGrant is called at a cross-skill rejection site with
// the tool's storage dep. Returns true only when the dep implements
// DescendantRunChecker AND the owner's run descends from the caller's
// run. Any error or missing context keeps the strict rejection.
func descendantFileGrant(ctx context.Context, dep any, inv tool.Invocation, ownerSkillID string) bool {
if ownerSkillID == "" || inv.RunID == "" {
return false
}
checker, ok := dep.(DescendantRunChecker)
if !ok || checker == nil {
return false
}
granted, err := checker.IsDescendantProducer(ctx, ownerSkillID, inv.RunID)
return err == nil && granted
}
+103
View File
@@ -0,0 +1,103 @@
// file_get fetches a previously-saved file by its opaque file_id and
// returns the metadata + base64-encoded bytes.
//
// Why scope is checked POST-fetch: file_id is the only key the caller
// knows; the scope (and therefore the authorisation envelope) is
// stored on the FileMeta row. We must read the row first to know which
// scope to validate. The trade-off is that file_id existence is
// observable (a foreign caller can probe IDs and learn that one
// exists), but the bytes themselves are still gated. file_id is a UUID,
// so the probe surface is impractical.
//
// Why base64 in the response: same reason as file_save — JSON can't
// carry arbitrary bytes natively. Callers that want a paste link or a
// direct download go through a separate path.
package tools
import (
"context"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
type fileGetArgs struct {
FileID string `json:"file_id" description:"Opaque file ID returned by file_save or file_list."`
}
type fileGetResult struct {
Name string `json:"name"`
ContentBase64 string `json:"content_base64"`
Mime string `json:"mime"`
SizeBytes int64 `json:"size_bytes"`
CreatedAt string `json:"created_at"` // RFC3339
}
// NewFileGet constructs the file_get tool. storage nil → "not
// configured" at execute time.
func NewFileGet(storage FileStorage) tool.Tool {
return tool.NewGatedTool[fileGetArgs](
"file_get",
"Fetch a saved file by its file_id. Returns name, base64 content, MIME, size, and created_at. The caller must have access to the file's scope (skill / own user: / own run:).",
tool.Permission{
AuthoringRequirement: tool.RequirementAnyone,
OperatesOn: tool.ScopeCaller,
SafeForShare: true,
Categories: []string{"storage", "read"},
},
func(ctx context.Context, inv tool.Invocation, args fileGetArgs) (string, error) {
if storage == nil {
return "", fmt.Errorf("file_get: not configured")
}
if args.FileID == "" {
return "", fmt.Errorf("file_get: file_id required")
}
meta, content, err := storage.FileGet(ctx, args.FileID)
if err != nil {
if errors.Is(err, ErrFileNotFound) {
return "", fmt.Errorf("file_get: not found")
}
return "", fmt.Errorf("file_get: %w", err)
}
// Cross-skill access check: a file's SkillID must match the
// current invocation's SkillID. Without this, a caller
// could probe another skill's file_ids and read content.
// One exception — the descendant grant (see
// file_descendant_grant.go): workers this run dispatched.
grantedViaDescendant := false
if meta.SkillID != inv.SkillID {
if !descendantFileGrant(ctx, storage, inv, meta.SkillID) {
return "", fmt.Errorf("file_get: file does not belong to this skill")
}
grantedViaDescendant = true
}
// Scope check: even within the same skill, the scope on the
// row gates access (e.g. user:bob's file is unreadable by
// alice). The descendant grant stands in for it — the file's
// scope is the WORKER's run, never the caller's.
if err := ValidateScope(inv, meta.Scope, inv.CallerIsAdmin); err != nil && !grantedViaDescendant {
return "", fmt.Errorf("file_get: %w", err)
}
res := fileGetResult{
Name: meta.Name,
ContentBase64: base64.StdEncoding.EncodeToString(content),
Mime: meta.MimeType,
SizeBytes: meta.SizeBytes,
CreatedAt: meta.CreatedAt.UTC().Format(time.RFC3339),
}
b, err := json.Marshal(res)
if err != nil {
return "", fmt.Errorf("file_get: marshal: %w", err)
}
return string(b), nil
},
)
}
+91
View File
@@ -0,0 +1,91 @@
// file_get_metadata returns metadata about a saved file (name, mime,
// size, created_at) WITHOUT loading the bytes. This is the v10
// agent-friendly companion to file_get — agents that just need to
// reason about a file's properties (size, type, name) should use
// file_get_metadata instead of pulling the full body into the context
// window.
//
// Why a separate tool (vs adding a flag to file_get): the byte-vs-
// reference principle is enforced statically — file_get_metadata's
// return shape simply does not carry bytes, so agents and tool
// authors can rely on the type signature. A flag-gated variant would
// invite "what does include_content=false mean" confusion.
package tools
import (
"context"
"encoding/json"
"errors"
"fmt"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
type fileGetMetadataArgs struct {
FileID string `json:"file_id" description:"Opaque file ID returned by file_save or file_list."`
}
type fileGetMetadataResult struct {
Name string `json:"name"`
Mime string `json:"mime"`
SizeBytes int64 `json:"size_bytes"`
CreatedAt string `json:"created_at"` // RFC3339
Scope string `json:"scope"`
}
// NewFileGetMetadata constructs the file_get_metadata tool. storage
// nil → "not configured" at execute time.
func NewFileGetMetadata(storage FileStorage) tool.Tool {
return tool.NewGatedTool[fileGetMetadataArgs](
"file_get_metadata",
"Fetch metadata for a saved file by its file_id (name, mime, size_bytes, created_at, scope). Does NOT load the file bytes — use file_get_text for text content or send_attachments to ship binary content to Discord.",
tool.Permission{
AuthoringRequirement: tool.RequirementAnyone,
OperatesOn: tool.ScopeCaller,
SafeForShare: true,
Categories: []string{"storage", "read"},
},
func(ctx context.Context, inv tool.Invocation, args fileGetMetadataArgs) (string, error) {
if storage == nil {
return "", fmt.Errorf("file_get_metadata: not configured")
}
if args.FileID == "" {
return "", fmt.Errorf("file_get_metadata: file_id required")
}
meta, _, err := storage.FileGet(ctx, args.FileID)
if err != nil {
if errors.Is(err, ErrFileNotFound) {
return "", fmt.Errorf("file_get_metadata: not found")
}
return "", fmt.Errorf("file_get_metadata: %w", err)
}
// Descendant grant: see file_descendant_grant.go — covers
// the scope check too (the file's scope is the worker's run).
grantedViaDescendant := false
if meta.SkillID != inv.SkillID {
if !descendantFileGrant(ctx, storage, inv, meta.SkillID) {
return "", fmt.Errorf("file_get_metadata: file does not belong to this skill")
}
grantedViaDescendant = true
}
if !grantedViaDescendant {
if err := ValidateScope(inv, meta.Scope, inv.CallerIsAdmin); err != nil {
return "", fmt.Errorf("file_get_metadata: %w", err)
}
}
res := fileGetMetadataResult{
Name: meta.Name,
Mime: meta.MimeType,
SizeBytes: meta.SizeBytes,
CreatedAt: meta.CreatedAt.UTC().Format(time.RFC3339),
Scope: meta.Scope,
}
b, err := json.Marshal(res)
if err != nil {
return "", fmt.Errorf("file_get_metadata: marshal: %w", err)
}
return string(b), nil
},
)
}
+119
View File
@@ -0,0 +1,119 @@
// file_get_text fetches a saved text file's content as plain text.
// Only succeeds for text/* MIMEs; binary MIMEs return an error so the
// agent knows to use a different path (file_get_metadata for
// reasoning, send_attachments for delivery).
//
// Why a 64 KiB cap: the v10 byte-vs-reference principle says inline
// text content stays under ~10KB ideally; we set the hard cap at 64
// KiB to handle reasonable text artifacts (logs, configs, small
// reports) without blowing the agent's context. Files larger than
// the cap return an error pointing to send_attachments.
//
// Why a separate tool (vs file_get): file_get returns base64 +
// metadata regardless of MIME, which agents misuse to dump 10MB PDFs
// into the context window. file_get_text is the agent-friendly
// alternative that explicitly fails fast on binary content.
package tools
import (
"context"
"encoding/json"
"errors"
"fmt"
"strings"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
const fileGetTextMaxBytes = 64 * 1024
type fileGetTextArgs struct {
FileID string `json:"file_id" description:"Opaque file ID returned by file_save or file_list."`
}
type fileGetTextResult struct {
Text string `json:"text"`
Mime string `json:"mime"`
SizeBytes int64 `json:"size_bytes"`
CreatedAt string `json:"created_at"` // RFC3339
}
// NewFileGetText constructs the file_get_text tool. storage nil →
// "not configured" at execute time.
func NewFileGetText(storage FileStorage) tool.Tool {
return tool.NewGatedTool[fileGetTextArgs](
"file_get_text",
"Fetch a saved text file's content (text/* MIMEs only, capped at 64KB). For binary content use file_get_metadata + send_attachments. Errors with 'not_text' for non-text MIMEs and 'too_large' for files > 64KB.",
tool.Permission{
AuthoringRequirement: tool.RequirementAnyone,
OperatesOn: tool.ScopeCaller,
SafeForShare: true,
Categories: []string{"storage", "read"},
},
func(ctx context.Context, inv tool.Invocation, args fileGetTextArgs) (string, error) {
if storage == nil {
return "", fmt.Errorf("file_get_text: not configured")
}
if args.FileID == "" {
return "", fmt.Errorf("file_get_text: file_id required")
}
meta, content, err := storage.FileGet(ctx, args.FileID)
if err != nil {
if errors.Is(err, ErrFileNotFound) {
return "", fmt.Errorf("file_get_text: not found")
}
return "", fmt.Errorf("file_get_text: %w", err)
}
// Descendant grant: a worker this run (transitively)
// dispatched may have produced the file — its scope is the
// WORKER's run, so the grant also stands in for the scope
// check below.
grantedViaDescendant := false
if meta.SkillID != inv.SkillID {
if !descendantFileGrant(ctx, storage, inv, meta.SkillID) {
return "", fmt.Errorf("file_get_text: file does not belong to this skill")
}
grantedViaDescendant = true
}
if !grantedViaDescendant {
if err := ValidateScope(inv, meta.Scope, inv.CallerIsAdmin); err != nil {
return "", fmt.Errorf("file_get_text: %w", err)
}
}
if !isTextMime(meta.MimeType) {
return "", fmt.Errorf("file_get_text: not_text: mime %q is not text/*", meta.MimeType)
}
if int64(len(content)) > fileGetTextMaxBytes {
return "", fmt.Errorf("file_get_text: too_large: %d bytes exceeds 64KB cap; use send_attachments to deliver this file to Discord", len(content))
}
res := fileGetTextResult{
Text: string(content),
Mime: meta.MimeType,
SizeBytes: meta.SizeBytes,
CreatedAt: meta.CreatedAt.UTC().Format(time.RFC3339),
}
b, err := json.Marshal(res)
if err != nil {
return "", fmt.Errorf("file_get_text: marshal: %w", err)
}
return string(b), nil
},
)
}
// isTextMime reports whether the given MIME is a text/* type.
// Accepts "text/plain", "text/markdown", "text/csv", "application/json"
// and "application/xml" since those are conventionally text.
func isTextMime(mime string) bool {
mime = strings.ToLower(strings.TrimSpace(mime))
if strings.HasPrefix(mime, "text/") {
return true
}
switch mime {
case "application/json", "application/xml", "application/xhtml+xml",
"application/javascript", "application/yaml", "application/x-yaml":
return true
}
return false
}
+74
View File
@@ -0,0 +1,74 @@
// file_list returns metadata for files in a scope. Blob bytes are NOT
// loaded — listing is a hot path that must stay light, and the LLM
// would burn tokens for no benefit.
package tools
import (
"context"
"encoding/json"
"fmt"
"strings"
"time"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
type fileListArgs struct {
Scope string `json:"scope" description:"Storage scope: 'skill', 'user:<your_id>', or 'run:<run_id>'."`
}
type fileListEntry struct {
FileID string `json:"file_id"`
Name string `json:"name"`
Mime string `json:"mime"`
SizeBytes int64 `json:"size_bytes"`
CreatedAt string `json:"created_at"`
}
// NewFileList constructs the file_list tool. storage nil → "not
// configured" at execute time.
func NewFileList(storage FileStorage) tool.Tool {
return tool.NewGatedTool[fileListArgs](
"file_list",
"List files in a scope. Returns a JSON array of {file_id, name, mime, size_bytes, created_at}. Does NOT include bytes — call file_get with a file_id to fetch content.",
tool.Permission{
AuthoringRequirement: tool.RequirementAnyone,
OperatesOn: tool.ScopeCaller,
SafeForShare: true,
Categories: []string{"storage", "read"},
},
func(ctx context.Context, inv tool.Invocation, args fileListArgs) (string, error) {
if storage == nil {
return "", fmt.Errorf("file_list: not configured")
}
if err := ValidateScope(inv, args.Scope, inv.CallerIsAdmin); err != nil {
return "", fmt.Errorf("file_list: %w", err)
}
// root_run is a KV-only scope (v1) — see file_save's guard.
if strings.HasPrefix(args.Scope, "root_run:") {
return "", fmt.Errorf("file_list: root_run scope is KV-only")
}
rows, err := storage.FileList(ctx, inv.SkillID, args.Scope)
if err != nil {
return "", fmt.Errorf("file_list: %w", err)
}
out := make([]fileListEntry, 0, len(rows))
for _, r := range rows {
out = append(out, fileListEntry{
FileID: r.ID,
Name: r.Name,
Mime: r.MimeType,
SizeBytes: r.SizeBytes,
CreatedAt: r.CreatedAt.UTC().Format(time.RFC3339),
})
}
b, err := json.Marshal(out)
if err != nil {
return "", fmt.Errorf("file_list: marshal: %w", err)
}
return string(b), nil
},
)
}
+171
View File
@@ -0,0 +1,171 @@
// file_save persists arbitrary bytes (base64-encoded by the caller)
// against a (scope, name) tuple within the calling skill's namespace.
// Returns the new file_id, the SHA256 content hash, and the size.
//
// Why base64 over raw bytes: the LLM's tool-call wire format is JSON,
// which can't carry arbitrary bytes natively. Base64 round-trips
// cleanly through the schema.
//
// Why hash + size in the response: agents commonly want to dedup
// across runs (same hash = same content) or build a manifest. Reporting
// these inline saves an immediate file_get round-trip just to compute
// them.
//
// Per-file cap: maxFileBytes (constructor arg) enforces an upper bound
// on individual file size. 0 falls back to defaultFileMaxBytes (10 MB).
//
// Per-skill quota (sum across all files): the constructor's QuotaProvider
// arg drives the v4 Phase 4 enforcement. nil disables enforcement
// (useful for tests and admin-only deployments). The check is:
//
// used := storage.FileUsageBytes(skill)
// if used + len(new content) > filesMax → quota_exceeded
//
// Note we do NOT subtract a "prior" value here the way kv_set does:
// file_save always inserts a new file row (content-addressable dedup
// is at the blob layer, not the row layer), so every save is additive
// to FileUsageBytes.
package tools
import (
"context"
"crypto/sha256"
"encoding/base64"
"encoding/hex"
"encoding/json"
"fmt"
"net/http"
"strings"
"time"
"github.com/google/uuid"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
const defaultFileMaxBytes = 16 * 1024 * 1024 // 10 MiB
type fileSaveArgs struct {
Scope string `json:"scope" description:"Storage scope: 'skill' (shared across all callers of this skill), 'user:<your_id>' (per-caller), or 'run:<run_id>' (this run's scratchpad)."`
Name string `json:"name" description:"Filename including extension. Used for display only — the file is identified by an opaque file_id."`
ContentBase64 string `json:"content_base64" description:"Base64-encoded file content."`
Mime string `json:"mime,omitempty" description:"Optional MIME type. If omitted, detected from the first 512 bytes of content."`
}
type fileSaveResult struct {
FileID string `json:"file_id"`
Hash string `json:"hash"`
SizeBytes int64 `json:"size_bytes"`
}
// NewFileSave constructs the file_save tool.
//
// storage nil → "not configured" at execute time.
// maxFileBytes <= 0 falls back to defaultFileMaxBytes (10 MiB).
// quota nil → per-skill quota check skipped (per-file cap still applies).
//
// Permission: anyone may author; safe for share. Scope check at handler
// entry prevents cross-user writes; per-user buckets are isolated by
// inv.CallerID.
func NewFileSave(storage FileStorage, quota QuotaProvider, maxFileBytes int) tool.Tool {
if maxFileBytes <= 0 {
maxFileBytes = defaultFileMaxBytes
}
return tool.NewGatedTool[fileSaveArgs](
"file_save",
"Save base64-encoded bytes against a (scope, name) tuple. Returns file_id (opaque), SHA256 hash, and size_bytes. Content is dedup'd by hash — multiple file_save calls with identical bytes share storage. NOTE: for files produced inside code_exec, do NOT hand-encode base64 here (it corrupts) — write them to /workspace/ in the code_exec call and use the files_out file_id it returns.",
tool.Permission{
AuthoringRequirement: tool.RequirementAnyone,
OperatesOn: tool.ScopeCaller,
SafeForShare: true,
Categories: []string{"storage", "write"},
},
func(ctx context.Context, inv tool.Invocation, args fileSaveArgs) (string, error) {
if storage == nil {
return "", fmt.Errorf("file_save: not configured")
}
if err := ValidateScope(inv, args.Scope, inv.CallerIsAdmin); err != nil {
return "", fmt.Errorf("file_save: %w", err)
}
// root_run is a KV-only scope (v1): file storage partitions
// by the calling skill, so a root_run file would silently be
// invisible to siblings AND escape the run-scope sweeper.
// Reject loudly instead.
if strings.HasPrefix(args.Scope, "root_run:") {
return "", fmt.Errorf("file_save: root_run scope is KV-only; save under run:<run_id> and share the file_id via kv_set in the root_run scope")
}
if args.Name == "" {
return "", fmt.Errorf("file_save: name required")
}
if args.ContentBase64 == "" {
return "", fmt.Errorf("file_save: content_base64 required")
}
// Decode + cap. Decoding twice (once to count, once to
// store) would waste cycles; we decode once and check size
// after.
content, err := base64.StdEncoding.DecodeString(args.ContentBase64)
if err != nil {
return "", fmt.Errorf("file_save: invalid base64: %w", err)
}
if len(content) > maxFileBytes {
return "", fmt.Errorf("file_save: file exceeds max %d bytes (got %d)", maxFileBytes, len(content))
}
// Per-skill quota gate (v4 Phase 4). Skipped when quota is nil
// (tests / admin opt-out) so the per-file cap above is the
// only line of defence in that mode.
if quota != nil {
_, filesMax, err := quota.EffectiveQuota(ctx, inv.SkillID)
if err != nil {
return "", fmt.Errorf("file_save: quota lookup: %w", err)
}
used, err := storage.FileUsageBytes(ctx, inv.SkillID)
if err != nil {
return "", fmt.Errorf("file_save: usage check: %w", err)
}
if used+int64(len(content)) > filesMax {
return "", fmt.Errorf("file_save: quota_exceeded — %d/%d bytes used; ask admin for higher quota", used, filesMax)
}
}
// SHA256 for content-addressable dedup at the storage layer.
h := sha256.Sum256(content)
hashHex := hex.EncodeToString(h[:])
mime := args.Mime
if mime == "" {
// http.DetectContentType is documented to read at most
// the first 512 bytes; passing the full slice is fine.
mime = http.DetectContentType(content)
}
meta := FileDomainMeta{
ID: uuid.NewString(),
SkillID: inv.SkillID,
Scope: args.Scope,
Name: args.Name,
ContentHash: hashHex,
MimeType: mime,
SizeBytes: int64(len(content)),
CreatedAt: time.Now(),
}
fileID, err := storage.FileSave(ctx, meta, content)
if err != nil {
return "", fmt.Errorf("file_save: %w", err)
}
res := fileSaveResult{
FileID: fileID,
Hash: hashHex,
SizeBytes: int64(len(content)),
}
b, err := json.Marshal(res)
if err != nil {
return "", fmt.Errorf("file_save: marshal result: %w", err)
}
return string(b), nil
},
)
}
+131
View File
@@ -0,0 +1,131 @@
// file_search runs a token-AND search over the per-skill (or, for
// admin authors, cross-skill) file index. Returns up to N matches with
// {file_id, name, snippet, score}.
//
// Why admin-authoring only: a public skill could otherwise probe
// other skills' file content via cross-skill search. Restricting the
// tool's authoring requirement to admins blocks shared/public skills
// from depending on file_search at all (it never appears in their
// allowed-tool catalog at save time). Within a private skill,
// admin-authored or otherwise, scope is per-call: the handler always
// pins skill_id to inv.SkillID — no matter what the LLM-supplied scope
// arg says — so a non-admin caller invoking an admin-authored public
// skill cannot escape the skill's own bucket.
//
// Why use Storage's SearchFiles directly: token logic + scoring lives
// in the skills package. The handler is a thin transcoder.
package tools
import (
"context"
"encoding/json"
"fmt"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
)
// FileSearcher is the narrow surface the file_search tool needs.
// Production wiring (mort.go) bridges *skills.System.Storage().
// nil-safe: a nil FileSearcher surfaces "not configured" at the first
// call.
type FileSearcher interface {
SearchFiles(ctx context.Context, skillID, scope, query string, limit int) ([]FileSearchDomainHit, error)
}
// FileSearchDomainHit mirrors skills.FileSearchHit (cycle-break domain
// shape). The production adapter is a struct copy.
type FileSearchDomainHit struct {
FileID string
SkillID string
Scope string
Name string
MimeType string
Snippet string
Score int
}
type fileSearchArgs struct {
Query string `json:"query" description:"Free-text search query. Tokenised, lowercased, ANDed."`
Scope string `json:"scope,omitempty" description:"Optional storage scope to restrict the search ('skill', 'user:<your_id>', 'run:<run_id>'). Empty = all scopes within this skill."`
Limit int `json:"limit,omitempty" description:"Optional max hits to return (default 25, max 100)."`
}
type fileSearchHit struct {
FileID string `json:"file_id"`
Name string `json:"name"`
Mime string `json:"mime,omitempty"`
Snippet string `json:"snippet,omitempty"`
Score int `json:"score"`
}
// NewFileSearch constructs the file_search tool. Authoring-required
// admin so non-admins can't include this tool in shared/public skills
// (the share-safety check rejects share+admin-only as private-only).
//
// Wait — if the tool is admin-authoring AND share-safe, an admin could
// author a public skill that uses it. That's the desired flow: admin
// curates the skill, but the privacy property still holds because the
// handler PINS skill_id to inv.SkillID. A non-admin caller of the
// public skill can ONLY search files within that skill's bucket, not
// cross-skill.
//
// Setting SafeForShare=false would force this tool to be private-only;
// that's needlessly restrictive. The privacy property comes from the
// per-call skill_id pin, not from share-time gating.
func NewFileSearch(searcher FileSearcher) tool.Tool {
return tool.NewGatedTool[fileSearchArgs](
"file_search",
"Full-text search over this skill's saved files. Returns array of {file_id, name, snippet, score} ordered by score desc. Tokens are lowercased + ANDed. Admin-authored only — non-admin callers of an admin-authored public skill still see only that skill's files.",
tool.Permission{
AuthoringRequirement: tool.RequirementAdmin,
OperatesOn: tool.ScopeCaller,
SafeForShare: true,
Categories: []string{"storage", "read"},
},
func(ctx context.Context, inv tool.Invocation, args fileSearchArgs) (string, error) {
if searcher == nil {
return "", fmt.Errorf("file_search: not configured")
}
if args.Query == "" {
return "", fmt.Errorf("file_search: query required")
}
limit := args.Limit
if limit <= 0 {
limit = 25
}
if limit > 100 {
limit = 100
}
scope := args.Scope
if scope != "" {
if err := ValidateScope(inv, scope, inv.CallerIsAdmin); err != nil {
return "", fmt.Errorf("file_search: %w", err)
}
}
// Pin skill_id to the invoking skill — even if the LLM
// supplies a different value somewhere, the handler always
// scopes to inv.SkillID. This is the privacy guarantee
// referenced in the package doc.
rows, err := searcher.SearchFiles(ctx, inv.SkillID, scope, args.Query, limit)
if err != nil {
return "", fmt.Errorf("file_search: %w", err)
}
out := make([]fileSearchHit, 0, len(rows))
for _, r := range rows {
out = append(out, fileSearchHit{
FileID: r.FileID,
Name: r.Name,
Mime: r.MimeType,
Snippet: r.Snippet,
Score: r.Score,
})
}
b, err := json.Marshal(out)
if err != nil {
return "", fmt.Errorf("file_search: marshal: %w", err)
}
return string(b), nil
},
)
}
+49
View File
@@ -0,0 +1,49 @@
// file_storage.go declares the narrow FileStorage interface that the
// four v4 file tools (file_save, file_get, file_list, file_delete)
// need at execute time.
//
// Why a narrow interface (vs importing pkg/logic/skills directly): same
// cycle constraint as kv_storage.go — pkg/logic/skills imports
// pkg/skilltools, so we mirror the FileMeta shape here and let
// pkg/logic/mort.go adapt at wiring time.
//
// FileDomainMeta is field-for-field with skills.FileMeta; the production
// adapter is a struct copy.
package tools
import (
"context"
"errors"
"time"
)
// FileStorage is the narrow surface file tools need from the skills
// package. Production wiring (mort.go) bridges *skills.System.Storage().
// nil-safe: tools constructed against a nil FileStorage surface "not
// configured" at the first call.
type FileStorage interface {
FileSave(ctx context.Context, meta FileDomainMeta, content []byte) (string, error)
FileGet(ctx context.Context, fileID string) (*FileDomainMeta, []byte, error)
FileList(ctx context.Context, skillID, scope string) ([]FileDomainMeta, error)
FileDelete(ctx context.Context, fileID string) error
FileUsageBytes(ctx context.Context, skillID string) (int64, error)
}
// FileDomainMeta mirrors skills.FileMeta. Field-for-field; the
// production adapter is a struct copy.
type FileDomainMeta struct {
ID string // UUID, the public file_id
SkillID string
Scope string
Name string
ContentHash string // SHA256 hex
MimeType string
SizeBytes int64
CreatedAt time.Time
}
// ErrFileNotFound mirrors skills.ErrFileNotFound. The production
// adapter returns this sentinel when wrapping a skills.ErrFileNotFound;
// tools detect it with errors.Is to surface a "not_found" string to the
// LLM rather than a generic error.
var ErrFileNotFound = errors.New("file: not found")
+73
View File
@@ -0,0 +1,73 @@
package tools_test
import (
"context"
"encoding/json"
"testing"
"gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/majordomo/provider/fake"
"gitea.stevedudenhoeffer.com/steve/executus/run"
"gitea.stevedudenhoeffer.com/steve/executus/tool"
"gitea.stevedudenhoeffer.com/steve/executus/tools"
)
// TestExecutorRunsToolUsingAgent is the end-to-end proof that a host can
// register a generic tool and the executor runs an agent that CALLS it: the
// fake model emits a `think` tool call, the executor dispatches it through the
// registered tool, then the model finalises. Exercises the full tool-dispatch
// loop + step instrumentation.
func TestExecutorRunsToolUsingAgent(t *testing.T) {
reg := tool.NewRegistry()
if err := tools.Register(reg); err != nil {
t.Fatalf("register tools: %v", err)
}
fp := fake.New("fake")
fp.Enqueue("test-model",
// Step 1: the model decides to call `think`.
fake.ReplyWith(llm.Response{
ToolCalls: []llm.ToolCall{{
ID: "call-1",
Name: "think",
Arguments: json.RawMessage(`{"thought":"plan: answer briefly"}`),
}},
}),
// Step 2: with the tool result in hand, the model finalises.
fake.Reply("all done"),
)
m, err := fp.Model("test-model")
if err != nil {
t.Fatalf("fake model: %v", err)
}
ex := run.New(run.Config{
Registry: reg,
Models: func(ctx context.Context, _ string) (context.Context, llm.Model, error) {
return ctx, m, nil
},
})
res := ex.Run(context.Background(),
run.RunnableAgent{Name: "thinker", ModelTier: "test-model", LowLevelTools: []string{"think"}},
tool.Invocation{RunID: "run-tool-1", CallerID: "c"},
"do the thing")
if res.Err != nil {
t.Fatalf("run error: %v", res.Err)
}
if res.Output != "all done" {
t.Fatalf("output = %q, want %q", res.Output, "all done")
}
// The step instrumentation should have captured the think call.
var sawThink bool
for _, s := range res.Steps {
if s.Title == "think" {
sawThink = true
}
}
if !sawThink {
t.Errorf("expected a `think` step in Result.Steps, got %d steps: %+v", len(res.Steps), res.Steps)
}
}

Some files were not shown because too many files have changed in this diff Show More