Author SHA1 Message Date
steveandClaude Fable 5 8eb0265e75 fix(reusable): thread the dispatch pr_number as a workflow_call input (hotfix lineage)
Mirrors gadfly#24 on main. Env/inputs only; image sha-ed9e946 unchanged.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-07-15 23:34:37 -04:00
steveandClaude Fable 5 3664ce858d chore(reusable): replace the retired ragnaros endpoint with netherstorm (hotfix lineage)
Mirrors gadfly#23 on main. Env-only; image sha-ed9e946 unchanged.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-07-15 23:32:30 -04:00
steveandClaude Fable 5 f542d4e1cd chore(reusable): forward GADFLY_ENDPOINT_NETHERSTORM (hotfix lineage)
Mirrors gadfly#22 on main: the var was added after the forwarding list,
so netherstorm reviewers failed with 'unknown provider' — the env never
crossed the workflow_call boundary. Env-only; image sha-ed9e946 unchanged.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-07-15 23:30:50 -04:00
steveandClaude Fable 5 b6a33dc218 chore(reusable): bump image pin to sha-ed9e946 (Gitea 1.27 workflow_call hotfix)
Same binary lineage as sha-3095ebf (5007597 + entrypoint fix only) — this
is the conservative hotfix pin for consumers; the executus re-platform on
main rolls out separately.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-07-15 23:01:30 -04:00
steveandClaude Fable 5 ed9e946cd1 fix: handle Gitea 1.27's workflow_call event name in the trigger gate
Gitea 1.27 (breaking change go-gitea#37478) runs a called (reusable)
workflow with github.event_name = 'workflow_call' instead of propagating
the caller's event. Every consumer stub forwards EVENT_NAME from
github.event_name, so since the server upgrade every review arrived as an
unhandled event and self-skipped in one second while reporting success —
mort PRs #1445-#1447 all went unreviewed.

Reclassify workflow_call from the forwarded payload: a non-empty
COMMENT_BODY can only come from issue_comment (trigger-phrase + actor
gates still apply); otherwise a PR number means a pull_request-shaped
trigger. Neither → the existing unhandled-event skip. Pre-1.27 servers
are unaffected.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-07-15 23:00:43 -04:00
25 changed files with 284 additions and 1871 deletions
-3
View File
@@ -53,6 +53,3 @@ jobs:
# 5-lens suite) from review-reusable.yml. Only the consumer-specific # 5-lens suite) from review-reusable.yml. Only the consumer-specific
# allow-list is set here. # allow-list is set here.
allowed_users: "steve,fizi,dazed" allowed_users: "steve,fizi,dazed"
# Gitea >= 1.27 does not propagate dispatch inputs into a called workflow's
# github.event — thread the PR number explicitly (empty on non-dispatch events).
pr_number: ${{ github.event.inputs.pr_number }}
+1 -7
View File
@@ -99,11 +99,7 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
timeout-minutes: ${{ inputs.timeout_minutes }} timeout-minutes: ${{ inputs.timeout_minutes }}
steps: steps:
# sha-3095ebf predated the Gitea >= 1.27 workflow_call reclassification - uses: docker://gitea.stevedudenhoeffer.com/steve/gadfly:sha-ed9e946
# (entrypoint.sh, added in 9d74cb9) — so the dogfood self-review skipped every
# PR with "event 'workflow_call' not handled". Bumped to sha-f468fe6 (current
# main HEAD) which contains that fix. Bump per Gadfly release.
- uses: docker://gitea.stevedudenhoeffer.com/steve/gadfly:sha-f468fe6
env: env:
# --- event context (from the CALLER's github.*) ------------------- # --- event context (from the CALLER's github.*) -------------------
GITEA_API: ${{ github.server_url }}/api/v1/repos/${{ github.repository }} GITEA_API: ${{ github.server_url }}/api/v1/repos/${{ github.repository }}
@@ -120,8 +116,6 @@ jobs:
COMMENT_ID: ${{ github.event.comment.id }} COMMENT_ID: ${{ github.event.comment.id }}
ACTOR: ${{ github.actor }} ACTOR: ${{ github.actor }}
# --- provider auth (forwarded workflow_call secrets; empty if the caller doesn't forward it) - # --- provider auth (forwarded workflow_call secrets; empty if the caller doesn't forward it) -
# OLLAMA_CLOUD_API_KEY powers both the ollama-cloud majordomo path AND
# the opencode engine (GADFLY_MODELS entry "opencode/<model>").
OLLAMA_CLOUD_API_KEY: ${{ secrets.OLLAMA_CLOUD_API_KEY }} OLLAMA_CLOUD_API_KEY: ${{ secrets.OLLAMA_CLOUD_API_KEY }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }} OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
+5 -32
View File
@@ -22,24 +22,15 @@ verifies each one against the actual code, and posts its findings as a comment.
4. **Provider-agnostic.** Powered by [majordomo](https://gitea.stevedudenhoeffer.com/steve/majordomo), 4. **Provider-agnostic.** Powered by [majordomo](https://gitea.stevedudenhoeffer.com/steve/majordomo),
so it can target Ollama (local/cloud), OpenAI, Anthropic, Google, or any so it can target Ollama (local/cloud), OpenAI, Anthropic, Google, or any
OpenAI/Ollama-compatible endpoint. Don't re-hardcode a single provider. OpenAI/Ollama-compatible endpoint. Don't re-hardcode a single provider.
5. **Portable & self-contained.** `cmd/gadfly` depends only on the Go stdlib, majordomo, and 5. **Portable & self-contained.** `cmd/gadfly` depends only on the Go stdlib + majordomo. Keep
[executus](https://gitea.stevedudenhoeffer.com/steve/executus) (whose *core*`run`/`compact`/ it that way — no heavyweight deps, no coupling to any one consumer repo (e.g. mort).
`model`/`fanout`/`tool` — is itself majordomo+stdlib only, so the binary stays static; do NOT
pull executus's `contrib/store` or any battery that drags in a DB driver). No heavyweight deps,
no coupling to any one consumer repo (e.g. mort). Gadfly is executus's canonical *light* consumer.
## Architecture ## Architecture
``` ```
cmd/gadfly/ the reviewer binary — pure producer of review markdown (stdout) cmd/gadfly/ the reviewer binary — pure producer of review markdown (stdout)
main.go orchestration: fan specialists out (executus/fanout), each a review pass + recheck main.go orchestration: loop specialists, each a review pass + adversarial recheck
engine.go reviewEngine abstraction: executus run.Executor (majordomo agent loop + engine.go reviewEngine abstraction: majordomo agent loop vs claude-code CLI shell-out
compaction/bounding/budget/critic) vs claude-code / opencode CLI shell-outs
opencode.go the opencode CLI engine: ollama-cloud model through the OpenCode harness
(read-only via a generated OPENCODE_CONFIG_CONTENT agent+provider); for
benchmarking the boutique harness vs a free one on the same model
executus.go executus wiring: tool.Registry over the repo tools, the run.Executor build
(compact + model context-limit threshold + per-PR budget + wrap-up critic)
specialists.go specialist lenses: built-ins, default suite, env + .gadfly.yml resolution specialists.go specialist lenses: built-ins, default suite, env + .gadfly.yml resolution
auto.go dynamic `auto` selection: a selector model picks lenses per-diff (may invent) auto.go dynamic `auto` selection: a selector model picks lenses per-diff (may invent)
delegate.go worker-tier delegate_investigation tool (cheap sub-agent does legwork) delegate.go worker-tier delegate_investigation tool (cheap sub-agent does legwork)
@@ -78,7 +69,7 @@ verdict. Verdict is one of: `No material issues found` / `Minor issues` / `Block
## Build / test ## Build / test
```sh ```sh
go build ./cmd/gadfly # needs read access to the private majordomo + executus modules go build ./cmd/gadfly # needs read access to the private majordomo module
go test ./... go test ./...
gofmt -l cmd/ # must be clean gofmt -l cmd/ # must be clean
docker build -t gadfly:dev --secret id=REGISTRY_USER,env=REGISTRY_USER --secret id=REGISTRY_PASSWORD,env=REGISTRY_PASSWORD . docker build -t gadfly:dev --secret id=REGISTRY_USER,env=REGISTRY_USER --secret id=REGISTRY_PASSWORD,env=REGISTRY_PASSWORD .
@@ -158,21 +149,3 @@ are actually exercised. OpenAI/Anthropic/Google come from majordomo's abstractio
parallel, `cap` (from `GADFLY_PROVIDER_CONCURRENCY` else `GADFLY_CONCURRENCY`, default 1) bounds parallel, `cap` (from `GADFLY_PROVIDER_CONCURRENCY` else `GADFLY_CONCURRENCY`, default 1) bounds
models-at-once within a lane. The review timeout (`GADFLY_TIMEOUT_SECS`) is **per-lens**, not models-at-once within a lane. The review timeout (`GADFLY_TIMEOUT_SECS`) is **per-lens**, not
shared across the suite — a slow model can't starve later lenses (the original timeout bug). shared across the suite — a slow model can't starve later lenses (the original timeout bug).
- **Large-PR token burn**: the agent loop re-sends the whole transcript every step, so a giant
diff (the old `get_diff` dumped it untruncated, and it was embedded in both the review and
recheck task) was re-transmitted ~steps × lenses × passes × models times — a ~250 K-token PR
could drain a metered usage block in minutes. Fixed in three size-gated layers (small PRs
untouched): paginated `get_diff` + `executus/compact` compaction in the binary; an
`entrypoint.sh` downshift above `GADFLY_HUGE_DIFF_BYTES` (one cheap model, fewer lenses/steps,
no recheck); and a swarm-wide `GADFLY_PR_BUDGET_SECS` wall-clock backstop. Compaction's threshold
is intentionally LOW (`GADFLY_COMPACT_RATIO` 0.45, not executus's 0.7) because the burning
transcript on the embedded path rarely reaches 0.7×context.
- **executus re-platform**: the in-process review path runs through `executus/run`'s `run.Executor`
(compaction, run-bounding, `Ports.Budget`, the wrap-up nudge as `Ports.Critic`), wiring it in
`cmd/gadfly/executus.go`. Gadfly KEEPS its own `model.go` resolution (so `GADFLY_ENDPOINT_<NAME>`
http aliases + the claude-code engine survive) and only hands `run.Executor` the already-resolved
model via a trivial resolver — do NOT route review-model resolution through
`model.ParseModelForContext` (it bypasses gadfly's endpoint aliases). `run.Result` exposes no
transcript, so the old transcript-based forced-finalization fallback is gone; the wrap-up critic
nudge is the remaining "always emit something" mechanism. The claude-code engine still shells out
and is unaffected.
+1 -24
View File
@@ -24,35 +24,12 @@ RUN --mount=type=cache,target=/go/pkg/mod \
go build -trimpath -ldflags="-s -w" -o /out/gadfly ./cmd/gadfly go build -trimpath -ldflags="-s -w" -o /out/gadfly ./cmd/gadfly
FROM alpine:3.20 FROM alpine:3.20
# procps provides pkill/pgrep, which entrypoint.sh's per-PR wall-clock backstop RUN apk add --no-cache bash git curl jq ca-certificates nodejs npm
# (GADFLY_PR_BUDGET_SECS) uses to stop the review subtrees — busybox's applets
# are not guaranteed to include them.
RUN apk add --no-cache bash git curl jq ca-certificates nodejs npm procps
# Bundle the Claude Code CLI so the `claude-code` review engine works out of the # Bundle the Claude Code CLI so the `claude-code` review engine works out of the
# box (GADFLY_MODELS=claude-code or claude-code/<model>). This adds Node + the # box (GADFLY_MODELS=claude-code or claude-code/<model>). This adds Node + the
# CLI to the image (notably larger); ollama-only users pay the size but nothing # CLI to the image (notably larger); ollama-only users pay the size but nothing
# else. Auth is provided at runtime via CLAUDE_CODE_OAUTH_TOKEN / ANTHROPIC_API_KEY. # else. Auth is provided at runtime via CLAUDE_CODE_OAUTH_TOKEN / ANTHROPIC_API_KEY.
RUN npm install -g @anthropic-ai/claude-code && npm cache clean --force RUN npm install -g @anthropic-ai/claude-code && npm cache clean --force
# Bundle the OpenCode CLI (opencode.ai) for the `opencode` review engine
# (GADFLY_MODELS=opencode/<model>): a freely-available agentic harness driving an
# ollama-cloud model, used to benchmark it against gadfly's own executus harness
# on the same model. Auth reuses OLLAMA_CLOUD_API_KEY at runtime. opencode ships a
# compiled (Bun) binary; it publishes musl variants (opencode-linux-*-musl) that
# npm auto-selects on alpine via the package "libc" field. libstdc++/libgcc are
# the Bun binary's runtime deps; gcompat is a belt-and-suspenders fallback in case
# npm ever resolves a glibc build here.
RUN apk add --no-cache gcompat libstdc++ libgcc \
&& npm install -g opencode-ai \
&& npm cache clean --force
# Best-effort: confirm the binary runs and pre-warm the openai-compatible provider
# package into opencode's cache so a review doesn't pay a first-run npm fetch. The
# warm-up model call intentionally fails against a dead URL. Never fail the build:
# a musl/runtime quirk here must not break the shared image for ollama/claude
# users — a broken opencode engine degrades to a normal (advisory) pass error.
RUN opencode --version >/dev/null 2>&1 \
&& OPENCODE_CONFIG_CONTENT='{"provider":{"gadfly":{"npm":"@ai-sdk/openai-compatible","options":{"baseURL":"http://127.0.0.1:9/v1"},"models":{"x":{}}}}}' \
timeout 120 opencode run --model gadfly/x "warm" >/dev/null 2>&1 \
; true
COPY --from=build /out/gadfly /usr/local/bin/gadfly COPY --from=build /out/gadfly /usr/local/bin/gadfly
COPY scripts /app/scripts COPY scripts /app/scripts
COPY entrypoint.sh /entrypoint.sh COPY entrypoint.sh /entrypoint.sh
+2 -95
View File
@@ -140,58 +140,6 @@ as an example, not wired or tested here.
> specialist selection and the `delegate_investigation` worker are majordomo-only and are skipped > specialist selection and the `delegate_investigation` worker are majordomo-only and are skipped
> with this engine (Claude Code does its own legwork). > with this engine (Claude Code does its own legwork).
### OpenCode engine (`opencode`)
The same shell-out idea, but with a **freely-available** harness: Gadfly can review through the
**[OpenCode](https://opencode.ai) CLI**, which — like Claude Code — brings its own read tools and
verifies findings against the checked-out repo, but drives an **ollama-cloud** model. The point is
to benchmark gadfly's boutique executus harness against a good open harness *on the same model*:
run `ollama-cloud/glm-5.2` (majordomo loop) and `opencode/glm-5.2` (OpenCode) side by side and
compare their findings. This is the wired, no-proxy version of the "alternate backends" comparison
described above. The CLI is bundled in the image (Node + `opencode-ai`).
Select it as a model id:
| Spec | Meaning |
|------|---------|
| `opencode/glm-5.2` | serve `glm-5.2` via ollama-cloud through OpenCode |
| `open-code/glm-5.2` | accepted alias spelling (`opencode` is canonical) |
| `opencode/qwen3-coder:480b-cloud` | model ids are taken **verbatim** — colons are preserved (no `:thinking` suffix here, unlike claude-code) |
| `opencode/<provider>/<model>` | escape hatch: pass `<provider>/<model>` straight to OpenCode's own provider registry/auth (e.g. `opencode/anthropic/claude-sonnet-4-6`) |
| `opencode` | bare: OpenCode's configured default model |
```yaml
GADFLY_MODELS: "ollama-cloud/glm-5.2,opencode/glm-5.2" # the benchmark pairing
```
Auth reuses **`OLLAMA_CLOUD_API_KEY`** (the same secret the ollama-cloud path uses; it's mapped to
`OLLAMA_API_KEY`, which the generated provider references as `{env:OLLAMA_API_KEY}` — never a literal
secret in config). Tuning knobs (all optional):
| Env | Default | Meaning |
|-----|---------|---------|
| `GADFLY_OPENCODE_MODEL` | *(from the spec suffix)* | overrides the model |
| `GADFLY_OPENCODE_BASE_URL` | `https://ollama.com/v1` | ollama-cloud endpoint; point at a local Ollama (`http://localhost:11434/v1`) or any OpenAI-compatible server |
| `GADFLY_OPENCODE_EXTRA_ARGS` | *(unset)* | extra `opencode run` args, **whitespace-split**, appended before the positional task |
| `GADFLY_OPENCODE_BIN` | `opencode` | CLI binary path |
> **Read-only is enforced through config, not a flag.** OpenCode has no `--append-system-prompt`, so
> Gadfly generates a per-lens config — the lens system prompt as a `gadfly` agent's prompt, with the
> mutating and network tools (`edit`/`bash`/`webfetch`/`websearch`/`external_directory`) denied at both
> the global and agent level — and injects it via `OPENCODE_CONFIG_CONTENT`. That env var is the
> highest-precedence config source in the container, so it **outranks any `opencode.json` a reviewed
> repo ships** — a repo can't re-enable edits on the reviewer. The subprocess runs with a **reduced
> environment**: the provider keys OpenCode needs to authenticate (`OLLAMA_API_KEY` for the primary
> path, plus `ANTHROPIC_*`/`OPENAI_*`/`GOOGLE_*`/`GEMINI_*` for the `opencode/<provider>/<model>`
> pass-through) alongside `PATH`/`HOME`/locale/`OPENCODE_*`/`GADFLY_OPENCODE_*` — but **not** gadfly's
> own secrets (the Gitea token, the findings token, or the claude-code subscription token), which the
> CLI has no use for.
> **Newly wired, lightly tested.** Like the claude-code engine, `auto` specialist selection and the
> `delegate_investigation` worker are majordomo-only and are skipped here (OpenCode does its own
> legwork). Output capture reads OpenCode's default text output, so treat the engine as new and
> sanity-check a run before trusting a benchmark.
### Endpoint aliases via env vars ### Endpoint aliases via env vars
For multiple named backends (e.g. a couple of Ollama boxes on your LAN), register them by For multiple named backends (e.g. a couple of Ollama boxes on your LAN), register them by
@@ -436,7 +384,6 @@ The reviewer binary reads these (the stub/entrypoint set sane defaults):
| `GADFLY_BASE_URL` | — | override endpoint (OpenAI/Ollama-compatible servers) | | `GADFLY_BASE_URL` | — | override endpoint (OpenAI/Ollama-compatible servers) |
| `GADFLY_API_KEY` | — | provider key; falls back to the provider's standard env | | `GADFLY_API_KEY` | — | provider key; falls back to the provider's standard env |
| `claude-code` model id | — | route a model through the bundled Claude Code CLI (`claude-code` / `claude-code/<model>`); see [Claude Code engine](#claude-code-engine-claude-code) for its `GADFLY_CLAUDE_*` knobs | | `claude-code` model id | — | route a model through the bundled Claude Code CLI (`claude-code` / `claude-code/<model>`); see [Claude Code engine](#claude-code-engine-claude-code) for its `GADFLY_CLAUDE_*` knobs |
| `opencode` model id | — | route an ollama-cloud model through the bundled OpenCode CLI (`opencode/<model>`); see [OpenCode engine](#opencode-engine-opencode) for its `GADFLY_OPENCODE_*` knobs |
| `GADFLY_SPECIALISTS` | default suite | csv of lenses, `all`, or `auto` (dynamic selection) | | `GADFLY_SPECIALISTS` | default suite | csv of lenses, `all`, or `auto` (dynamic selection) |
| `GADFLY_SELECTOR_MODEL` | review model | model that picks lenses in `auto` mode | | `GADFLY_SELECTOR_MODEL` | review model | model that picks lenses in `auto` mode |
| `GADFLY_WORKER_MODEL` | — | cheap model for `delegate_investigation`; unset = no delegation | | `GADFLY_WORKER_MODEL` | — | cheap model for `delegate_investigation`; unset = no delegation |
@@ -449,16 +396,7 @@ The reviewer binary reads these (the stub/entrypoint set sane defaults):
| `GADFLY_TIMEOUT_SECS` | 300 | deadline **per specialist lens** (review+recheck) | | `GADFLY_TIMEOUT_SECS` | 300 | deadline **per specialist lens** (review+recheck) |
| `GADFLY_RECHECK` | on | set `0`/`false` to skip the recheck pass | | `GADFLY_RECHECK` | on | set `0`/`false` to skip the recheck pass |
| `GADFLY_RECHECK_MAX_STEPS` | 16 | recheck-pass step cap | | `GADFLY_RECHECK_MAX_STEPS` | 16 | recheck-pass step cap |
| `GADFLY_MAX_DIFF_CHARS` | 60000 | diff chars embedded in the **review** prompt (the full diff is reachable via the paginated `get_diff` tool, scoped per file with its `path` arg) | | `GADFLY_MAX_DIFF_CHARS` | 60000 | diff chars embedded in the prompt (full diff via `get_diff`) |
| `GADFLY_RECHECK_DIFF_CHARS` | 20000 | diff chars embedded in the **recheck** prompt (smaller — the recheck pages `get_diff` for the hunks it verifies) |
| `GADFLY_COMPACT` | on | context compaction (via [executus](https://gitea.stevedudenhoeffer.com/steve/executus)): fold the transcript's runaway middle into a summary as it nears the model's context window, so a big diff + accumulating tool output can't balloon every step. `0` disables |
| `GADFLY_COMPACT_RATIO` | 0.45 | fraction of the model's context window at which compaction fires |
| `GADFLY_COMPACT_MODEL` | worker, else review model | cheap model the compactor uses to summarize the folded middle |
| `GADFLY_COMPACT_KEEP_RECENT` | 8 | most-recent messages kept verbatim during compaction |
| `GADFLY_COMPACT_SUMMARY_WORDS` | 200 | word cap on the compaction summary |
| `GADFLY_MODEL_CONTEXT_TOKENS` | *(auto)* | override the model's context-window size (tokens) for the compaction threshold; set it for self-hosted endpoints executus can't introspect (Ollama Cloud models resolve automatically) |
| `GADFLY_PR_TOKEN_BUDGET` | — | per-model token ceiling for this PR; once spent, remaining lenses/passes are skipped (advisory). 0 = off |
| `GADFLY_PR_TIME_BUDGET_SECS` | — | per-model wall-clock ceiling for this PR (advisory). 0 = off |
| `GADFLY_STATUS_BOARD` | on | set `0` to disable the live status-board comment | | `GADFLY_STATUS_BOARD` | on | set `0` to disable the live status-board comment |
| `GADFLY_STATUS_POLL_SECS` | 12 | how often the status board re-renders/upserts | | `GADFLY_STATUS_POLL_SECS` | 12 | how often the status board re-renders/upserts |
| `GADFLY_CONSOLIDATE` | `auto` | cross-model consensus comment: `auto` (on for ≥2 models), `1` (force on), `0` (off — one comment per model) | | `GADFLY_CONSOLIDATE` | `auto` | cross-model consensus comment: `auto` (on for ≥2 models), `1` (force on), `0` (off — one comment per model) |
@@ -470,37 +408,6 @@ The reviewer binary reads these (the stub/entrypoint set sane defaults):
| `GADFLY_REPO` | *(from `GITEA_API`)* | `owner/repo` slug stamped on emitted runs/findings (set by `entrypoint.sh`) | | `GADFLY_REPO` | *(from `GITEA_API`)* | `owner/repo` slug stamped on emitted runs/findings (set by `entrypoint.sh`) |
| `GADFLY_PR` | *(from event)* | PR number stamped on emitted runs/findings (set by `entrypoint.sh`) | | `GADFLY_PR` | *(from event)* | PR number stamped on emitted runs/findings (set by `entrypoint.sh`) |
### Large-PR cost controls
A very large diff is the one thing that can blow the budget: every review step
re-sends it, multiplied across models × lenses × passes × steps (a single
~250 K-token PR can otherwise burn a whole metered usage block). Gadfly handles
big PRs in three layers, all **size-gated so small PRs are untouched**:
1. **Paginated `get_diff` + compaction** (reviewer binary, on by default) —
`get_diff` returns a paginated, optionally per-file window instead of the whole
diff, and once a transcript nears the model's context window its middle is
folded into a summary (powered by [executus](https://gitea.stevedudenhoeffer.com/steve/executus)'s
`compact`). Tune with the `GADFLY_COMPACT_*` knobs above.
2. **Downshift** (`entrypoint.sh`) — above `GADFLY_HUGE_DIFF_BYTES` the whole fleet
collapses to a single cheap model + a focused lens subset, fewer steps, and no
recheck. A finished shallow review beats a budget-nuking one, and the posted
comment says so.
3. **Hard backstop** (`entrypoint.sh`) — `GADFLY_PR_BUDGET_SECS` is a wall-clock
ceiling across the *entire* fleet; on expiry the review is stopped and whatever
was found so far is posted. Like everything else, it never fails CI.
| Env | Default | Meaning |
|-----|---------|---------|
| `GADFLY_HUGE_DIFF_BYTES` | 600000 | downshift the fleet when the PR diff exceeds this many bytes (0 = never downshift) |
| `GADFLY_HUGE_DIFF_MODELS` | first model | model(s) to run on a downshifted huge PR |
| `GADFLY_HUGE_DIFF_SPECIALISTS` | `security,correctness,error-handling` | lenses on a downshifted huge PR |
| `GADFLY_HUGE_DIFF_MAX_STEPS` | 12 | review step cap on a huge PR |
| `GADFLY_HUGE_DIFF_RECHECK_MAX_STEPS` | 8 | recheck step cap on a huge PR |
| `GADFLY_HUGE_DIFF_RECHECK` | 0 | run the recheck pass on a huge PR (off by default) |
| `GADFLY_HUGE_DIFF_MAX_DIFF_CHARS` | 20000 | embedded review-diff chars on a huge PR |
| `GADFLY_PR_BUDGET_SECS` | — | swarm-wide wall-clock backstop; stops the whole fleet when reached (0 = off) |
## Findings telemetry (optional) ## Findings telemetry (optional)
Gadfly can record what it found so model quality can be tracked over time. It is Gadfly can record what it found so model quality can be tracked over time. It is
@@ -524,7 +431,7 @@ code.
## Building locally ## Building locally
```sh ```sh
go build ./cmd/gadfly # needs read access to the private majordomo + executus modules go build ./cmd/gadfly # needs read access to the private majordomo module
go test ./... go test ./...
``` ```
+25 -42
View File
@@ -19,35 +19,28 @@ import (
// the model's text answer. It is the one primitive both review passes use — the // the model's text answer. It is the one primitive both review passes use — the
// draft review and the adversarial recheck — so the rest of the pipeline // draft review and the adversarial recheck — so the rest of the pipeline
// (specialist composition, recheck orchestration, consolidation, emit) is // (specialist composition, recheck orchestration, consolidation, emit) is
// engine-agnostic. Three implementations: // engine-agnostic. Two implementations:
// //
// - majordomoEngine: the original path — a majordomo tool-using agent loop // - majordomoEngine: the original path — a majordomo tool-using agent loop
// (read_file/grep/… over a sandboxed repoFS). // (read_file/grep/… over a sandboxed repoFS).
// - claudeCodeEngine: shells out to the `claude` CLI in print mode, which // - claudeCodeEngine: shells out to the `claude` CLI in print mode, which
// brings its OWN repo tools; gadfly just feeds it the prompt and reads back // brings its OWN repo tools; gadfly just feeds it the prompt and reads back
// the final text. // the final text.
// - openCodeEngine (opencode.go): shells out to the `opencode` CLI likewise,
// but driving an ollama-cloud model — for benchmarking the two harnesses on
// the same model.
// //
// maxSteps is the tool-step budget for engines that have one (majordomo); the // maxSteps is the tool-step budget for engines that have one (majordomo); the
// shell-out engines manage their own loop and ignore it. // claude-code engine manages its own loop and ignores it.
type reviewEngine interface { type reviewEngine interface {
runPass(ctx context.Context, system, task string, maxSteps int) (string, error) runPass(ctx context.Context, system, task string, maxSteps int) (string, error)
} }
// majordomoEngine drives the in-process review path over the repo sandbox. It no // majordomoEngine drives the in-process majordomo agent over the repo sandbox.
// longer calls majordomo's agent loop directly: each pass runs through an
// executus run.Executor (see executus.go), which adds context compaction, run
// bounding, the per-PR budget gate, and the wrap-up critic. mdl is retained only
// so auto-select can fall back to the review model as its selector.
type majordomoEngine struct { type majordomoEngine struct {
rex *reviewExecutor mdl llm.Model
mdl llm.Model fsTools *repoFS
} }
func (e *majordomoEngine) runPass(ctx context.Context, system, task string, maxSteps int) (string, error) { func (e *majordomoEngine) runPass(ctx context.Context, system, task string, maxSteps int) (string, error) {
return e.rex.run(ctx, system, task, maxSteps) return runAgent(ctx, e.mdl, e.fsTools, system, task, maxSteps)
} }
// claudeCodeEngine reviews by shelling out to the `claude` CLI (Claude Code) in // claudeCodeEngine reviews by shelling out to the `claude` CLI (Claude Code) in
@@ -160,7 +153,16 @@ func (e *claudeCodeEngine) runPass(ctx context.Context, system, task string, _ i
// Force an extended-thinking budget for this run (a "...:max" spec). // Force an extended-thinking budget for this run (a "...:max" spec).
cmd.Env = append(cmd.Env, "MAX_THINKING_TOKENS="+strconv.Itoa(e.thinkingTokens)) cmd.Env = append(cmd.Env, "MAX_THINKING_TOKENS="+strconv.Itoa(e.thinkingTokens))
} }
killGroupOnCancel(cmd) // Put the CLI and the Node children it spawns in their own process group and
// kill the WHOLE group on context cancel, so a timed-out lens can't leave
// orphaned claude/node processes behind in the container.
cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true}
cmd.Cancel = func() error {
if cmd.Process != nil {
_ = syscall.Kill(-cmd.Process.Pid, syscall.SIGKILL)
}
return nil
}
var stdout, stderr bytes.Buffer var stdout, stderr bytes.Buffer
cmd.Stdout = &stdout cmd.Stdout = &stdout
cmd.Stderr = &stderr cmd.Stderr = &stderr
@@ -211,39 +213,13 @@ func (e *claudeCodeEngine) runPass(ctx context.Context, system, task string, _ i
return "", fmt.Errorf("claude -p produced no parseable output") return "", fmt.Errorf("claude -p produced no parseable output")
} }
// killGroupOnCancel puts cmd in its own process group and, on context cancel,
// SIGKILLs the whole group — so a timed-out shell-out CLI (claude/opencode) can't
// leave orphaned Node children behind in the container. Call before cmd.Run.
func killGroupOnCancel(cmd *exec.Cmd) {
cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true}
cmd.Cancel = func() error {
if cmd.Process != nil {
_ = syscall.Kill(-cmd.Process.Pid, syscall.SIGKILL)
}
return nil
}
}
// filterEnv returns the current process environment reduced to the variables for
// which keep returns true. Shared by the shell-out engines' minimal-env builders
// (claudeEnv, openCodeEnv), which differ only in their keep predicate.
func filterEnv(keep func(string) bool) []string {
var env []string
for _, kv := range os.Environ() {
if k, _, ok := strings.Cut(kv, "="); ok && keep(k) {
env = append(env, kv)
}
}
return env
}
// claudeEnv builds a minimal environment for the `claude` subprocess: only what // claudeEnv builds a minimal environment for the `claude` subprocess: only what
// the CLI needs (PATH/HOME, its auth tokens, locale, Node/XDG/GADFLY_CLAUDE_* // the CLI needs (PATH/HOME, its auth tokens, locale, Node/XDG/GADFLY_CLAUDE_*
// knobs), deliberately dropping the rest of the runner's secrets — GITEA_TOKEN, // knobs), deliberately dropping the rest of the runner's secrets — GITEA_TOKEN,
// GADFLY_FINDINGS_TOKEN, provider keys — so they never reach the third-party // GADFLY_FINDINGS_TOKEN, provider keys — so they never reach the third-party
// CLI. Defense in depth: the parent already holds them, but the CLI has no need. // CLI. Defense in depth: the parent already holds them, but the CLI has no need.
func claudeEnv() []string { func claudeEnv() []string {
return filterEnv(func(k string) bool { keep := func(k string) bool {
switch k { switch k {
case "PATH", "HOME", "USER", "LOGNAME", "TMPDIR", "LANG", "TERM", "SHELL", "MAX_THINKING_TOKENS": case "PATH", "HOME", "USER", "LOGNAME", "TMPDIR", "LANG", "TERM", "SHELL", "MAX_THINKING_TOKENS":
return true return true
@@ -254,7 +230,14 @@ func claudeEnv() []string {
strings.HasPrefix(k, "GADFLY_CLAUDE_") || strings.HasPrefix(k, "GADFLY_CLAUDE_") ||
strings.HasPrefix(k, "NODE_") || strings.HasPrefix(k, "NODE_") ||
strings.HasPrefix(k, "XDG_") strings.HasPrefix(k, "XDG_")
}) }
var env []string
for _, kv := range os.Environ() {
if k, _, ok := strings.Cut(kv, "="); ok && keep(k) {
env = append(env, kv)
}
}
return env
} }
// truncateForErr caps CLI error detail so a stderr dump can't bloat the comment, // truncateForErr caps CLI error detail so a stderr dump can't bloat the comment,
-371
View File
@@ -1,371 +0,0 @@
package main
// executus.go wires gadfly's agentic review path onto the executus run kernel
// (gitea.stevedudenhoeffer.com/steve/executus), layered above majordomo. The
// majordomoEngine no longer drives majordomo's agent loop directly; it builds a
// run.Executor that gives gadfly, for free:
//
// - context compaction (executus/compact): once the transcript a step would
// SEND crosses a token threshold derived from the model's real context
// window, the runaway middle is folded into a one-paragraph summary by a
// cheap summarizer model — so a big diff + accumulating read_file/grep
// results can't balloon every re-sent step (the large-PR burn).
// - run bounding + a per-PR spend budget (executus/run Ports.Budget): a hard
// token/seconds ceiling so a pathological PR can't drain the usage block.
// - the wrap-up nudge, re-expressed as an executus Critic (Ports.Critic): the
// steer that tells a step-hungry model to stop investigating and write its
// answer is now the critic seam, not a bespoke RunOption.
//
// Everything degrades to today's behavior when unconfigured: nil summarizer or a
// 0 context window disables compaction; nil budget disables the ceiling; the
// claude-code engine shells out and is unaffected by any of this.
//
// gadfly keeps its own model.go resolution (so GADFLY_ENDPOINT_<NAME> http
// aliases, failover chains, and the claude-code engine all survive) — the
// run.Executor is handed gadfly's already-resolved model via a trivial resolver,
// not routed through executus's tier table.
import (
"context"
"errors"
"fmt"
"os"
"strconv"
"strings"
"sync"
"sync/atomic"
"time"
llm "gitea.stevedudenhoeffer.com/steve/majordomo/llm"
"gitea.stevedudenhoeffer.com/steve/executus/compact"
"gitea.stevedudenhoeffer.com/steve/executus/model"
exrun "gitea.stevedudenhoeffer.com/steve/executus/run"
exectool "gitea.stevedudenhoeffer.com/steve/executus/tool"
)
const (
// defaultCompactRatio is the fraction of the model's context window at which
// compaction fires. It is deliberately LOWER than executus's own 0.7 default:
// on the large-PR burn the per-step transcript is the embedded diff (~17K) plus
// accumulating read_file results, which rarely reaches 0.7×262K≈183K — so a
// 0.7 threshold never bites. ~0.45×262K≈118K folds the runaway middle while a
// transcript is still well under the cap. Override with GADFLY_COMPACT_RATIO.
defaultCompactRatio = 0.45
// defaultCompactKeepRecent / defaultCompactSummaryWords mirror executus's own
// compactor defaults; surfaced as gadfly env knobs for tuning.
defaultCompactKeepRecent = 8
defaultCompactSummaryWords = 200
// contextTokenLookupTimeout bounds the one-shot /api/show call that resolves a
// cloud model's context window at executor-build time. Kept short so a slow or
// unreachable endpoint adds at most this to startup before degrading to
// no-compaction (rather than the provider cache's default 15s).
contextTokenLookupTimeout = 5 * time.Second
)
// runSeq mints a unique-per-process RunID suffix for each executor run so audit
// and the run kernel can tell one pass from another within a binary process.
var runSeq atomic.Uint64
// wrappedTool adapts an already-built majordomo llm.Tool (gadfly's sandboxed
// read_file/grep/get_diff/… closures over the repoFS) to executus's tool.Tool
// interface so the run kernel can build a toolbox from them by name. gadfly's
// tools need no caller/channel identity, so BuildLLM ignores the Invocation and
// returns the pre-built tool; Permission is the zero value (private, ungated).
type wrappedTool struct{ t llm.Tool }
func (w wrappedTool) Name() string { return w.t.Name }
func (w wrappedTool) Description() string { return w.t.Description }
func (w wrappedTool) Permission() exectool.Permission { return exectool.Permission{} }
func (w wrappedTool) BuildLLM(_ exectool.Invocation) llm.Tool { return w.t }
// gadflyToolRegistry registers the repo's read-only tools (plus the optional
// delegate_investigation worker tool) in a fresh executus tool.Registry and
// returns it along with the tool names for RunnableAgent.LowLevelTools.
func gadflyToolRegistry(fs *repoFS) (exectool.Registry, []string, error) {
reg := exectool.NewRegistry()
tools := fs.allTools()
names := make([]string, 0, len(tools))
for _, t := range tools {
if err := reg.Register(wrappedTool{t: t}); err != nil {
return nil, nil, fmt.Errorf("register tool %q: %w", t.Name, err)
}
names = append(names, t.Name)
}
return reg, names, nil
}
// gadflyBudget is gadfly's per-PR spend ceiling, satisfying run.Ports.Budget.
// It gates a run BEFORE it makes any model call (Check) once the process has
// spent its token or wall-clock allowance on this PR. Tokens are fed in
// out-of-band via addUsage (the Budget interface's Commit only carries seconds);
// the engine calls addUsage after each pass with run.Result.Usage. A nil
// *gadflyBudget is never installed — caps of 0 mean "unlimited", so the port is
// only wired when at least one cap is set.
//
// The guard is PASS-granular: Check runs before each pass, so it stops the NEXT
// pass once the budget is spent but cannot abort a single runaway pass mid-flight.
// The swarm-wide GADFLY_PR_BUDGET_SECS wall-clock backstop (entrypoint.sh) is what
// bounds a mid-pass runaway.
type gadflyBudget struct {
mu sync.Mutex
maxTokens int64
maxSeconds float64
tokens int64
seconds float64
}
// newPRBudget builds the per-PR budget from env, or nil when neither cap is set
// (the default — the swarm-wide ceiling lives in entrypoint.sh; this is the
// per-process belt to its suspenders).
func newPRBudget() *gadflyBudget {
toks := envInt("GADFLY_PR_TOKEN_BUDGET", 0)
secs := envInt("GADFLY_PR_TIME_BUDGET_SECS", 0)
if toks <= 0 && secs <= 0 {
return nil
}
return &gadflyBudget{maxTokens: int64(toks), maxSeconds: float64(secs)}
}
func (b *gadflyBudget) Check(_ context.Context, _ string) error {
if b == nil {
return nil
}
b.mu.Lock()
defer b.mu.Unlock()
if b.maxTokens > 0 && b.tokens >= b.maxTokens {
return fmt.Errorf("gadfly: per-PR token budget exhausted (%d/%d)", b.tokens, b.maxTokens)
}
if b.maxSeconds > 0 && b.seconds >= b.maxSeconds {
return fmt.Errorf("gadfly: per-PR time budget exhausted (%.0f/%.0fs)", b.seconds, b.maxSeconds)
}
return nil
}
func (b *gadflyBudget) Commit(_ context.Context, _ string, runtimeSeconds float64) {
if b == nil {
return
}
b.mu.Lock()
defer b.mu.Unlock()
b.seconds += runtimeSeconds
}
// addUsage records a finished pass's token spend toward the budget. Safe on nil.
func (b *gadflyBudget) addUsage(u llm.Usage) {
if b == nil {
return
}
b.mu.Lock()
defer b.mu.Unlock()
b.tokens += int64(u.InputTokens) + int64(u.OutputTokens)
}
// wrapUpCritic re-expresses gadfly's wrap-up nudge as an executus run.Critic:
// once a run comes within wrapUpReserve steps of its cap, Steer() injects the
// "stop calling tools and write your final answer" message so a thorough model
// spends its last steps finalizing instead of hard-failing empty. It sets no
// hard deadline (Deadline()==zero) and never raises the step ceiling
// (MaxSteps()==0, defer to the run's MaxIterations) — it is purely the nudge.
type wrapUpCritic struct{ reserve int }
func (c *wrapUpCritic) Monitor(_ context.Context, info exrun.RunInfo, _ time.Duration) exrun.CriticHandle {
return &wrapUpHandle{maxSteps: info.MaxIterations, reserve: c.reserve}
}
type wrapUpHandle struct {
mu sync.Mutex
maxSteps int
reserve int
done int // steps completed so far
nudged bool
}
func (h *wrapUpHandle) RecordStep(iter int, _ *llm.Response) {
h.mu.Lock()
h.done = iter + 1
h.mu.Unlock()
}
func (h *wrapUpHandle) RecordToolStart(string, string) {}
func (h *wrapUpHandle) Steer() []llm.Message {
h.mu.Lock()
defer h.mu.Unlock()
at := h.maxSteps - h.reserve
if at < 1 {
at = 1
}
if !h.nudged && h.maxSteps > 0 && h.done >= at {
h.nudged = true
return []llm.Message{llm.UserText(wrapUpInstruction)}
}
return nil
}
func (h *wrapUpHandle) Deadline() time.Time { return time.Time{} }
func (h *wrapUpHandle) MaxSteps() int { return 0 }
func (h *wrapUpHandle) KillCause() error { return nil }
func (h *wrapUpHandle) Stop() {}
// reviewExecutor bundles a run.Executor with the per-run wiring the engine needs
// for each pass (the tool names to expose, the model spec to report as the tier,
// the per-PR caller id, and the budget to feed token usage into).
type reviewExecutor struct {
ex *exrun.Executor
toolNames []string
modelSpec string
callerID string
budget *gadflyBudget
}
// newReviewExecutor builds the run.Executor for the in-process majordomo review
// path. mdl is gadfly's already-resolved review model; summarizer is the cheap
// model the compactor uses (nil disables compaction). Compaction also needs the
// model's context window (resolved once here, not per pass); a 0 window likewise
// disables it. The budget (may be nil) becomes the run.Ports.Budget gate.
func newReviewExecutor(fs *repoFS, mdl, summarizer llm.Model, modelSpec string, budget *gadflyBudget) (*reviewExecutor, error) {
reg, names, err := gadflyToolRegistry(fs)
if err != nil {
return nil, err
}
// gadfly resolves exactly one model per process; the run kernel's resolver
// just hands that model back regardless of the tier string it is asked for.
modelsResolver := func(ctx context.Context, _ string) (context.Context, llm.Model, error) {
return ctx, mdl, nil
}
var compactor compact.CompactorFactory
var ctxTokens func(string) int
if summarizer != nil && compactionEnabled() {
if window := resolveContextTokens(modelSpec); window > 0 {
sumResolver := func(ctx context.Context, _ string) (context.Context, llm.Model, error) {
return ctx, summarizer, nil
}
compactor = compact.NewCompactor(compact.CompactorConfig{
Models: sumResolver,
KeepRecent: envInt("GADFLY_COMPACT_KEEP_RECENT", defaultCompactKeepRecent),
SummaryWordCap: envInt("GADFLY_COMPACT_SUMMARY_WORDS", defaultCompactSummaryWords),
})
ctxTokens = func(string) int { return window } // memoized: one window per process
}
}
var ports exrun.Ports
ports.Critic = &wrapUpCritic{reserve: wrapUpReserve()}
if budget != nil {
ports.Budget = budget
}
cfg := exrun.Config{
Registry: reg,
Models: modelsResolver,
Compactor: compactor,
ContextTokens: ctxTokens,
Defaults: exrun.Defaults{
// MaxIterations/MaxRuntime are intentionally omitted: every pass sets its
// own per-run cap on the RunnableAgent below (the review and recheck caps
// differ), so a Defaults value here would always be overridden — dead. This
// leaves only the cross-pass guards + the compaction ratio.
MaxConsecutiveToolErrors: 4,
MaxSameToolCallRepeats: 4,
CompactionThresholdRatio: compactRatio(),
FallbackTier: modelSpec,
},
Ports: ports,
}
return &reviewExecutor{
ex: exrun.New(cfg),
toolNames: names,
modelSpec: modelSpec,
callerID: prCallerID(),
budget: budget,
}, nil
}
// run executes one agent pass (review or recheck) through the run kernel and
// returns the model's final text. An empty answer with no error is reported as
// an error so the caller (reviewWithSpecialist) renders the advisory "reviewer
// failed to complete" notice rather than a blank section.
func (r *reviewExecutor) run(ctx context.Context, system, task string, maxSteps int) (string, error) {
res := r.ex.Run(ctx, exrun.RunnableAgent{
Name: "gadfly-review",
SystemPrompt: system,
ModelTier: r.modelSpec,
MaxIterations: maxSteps,
MaxRuntime: reviewTimeout(),
LowLevelTools: r.toolNames,
Critic: exrun.CriticConfig{Enabled: true},
}, exectool.Invocation{
RunID: fmt.Sprintf("gadfly-%d", runSeq.Add(1)),
CallerID: r.callerID,
}, task)
// Feed token spend toward the per-PR budget out-of-band (Commit carries only
// seconds; the executor already called it). Safe on a nil budget.
r.budget.addUsage(res.Usage)
if res.Err != nil {
return "", res.Err
}
if out := strings.TrimSpace(res.Output); out != "" {
return out, nil
}
return "", errors.New("agent produced no output")
}
// prCallerID is the budget/audit caller key: the repo + PR, so a budget keyed on
// it is naturally per-PR. Falls back to "local" for an out-of-CI run.
func prCallerID() string {
repo := strings.TrimSpace(os.Getenv("GADFLY_REPO"))
pr := strings.TrimSpace(os.Getenv("GADFLY_PR"))
if repo == "" && pr == "" {
return "local"
}
return repo + "#" + pr
}
// compactionEnabled reports whether context compaction should be wired. On
// unless GADFLY_COMPACT is explicitly falsey.
func compactionEnabled() bool { return envBool("GADFLY_COMPACT", true) }
// compactRatio is the compaction threshold as a fraction of the model context
// window (GADFLY_COMPACT_RATIO), clamped to (0,1]; default defaultCompactRatio.
func compactRatio() float64 {
v := strings.TrimSpace(os.Getenv("GADFLY_COMPACT_RATIO"))
if v == "" {
return defaultCompactRatio
}
f, err := strconv.ParseFloat(v, 64)
if err != nil || f <= 0 || f > 1 {
return defaultCompactRatio
}
return f
}
// resolveContextTokens returns the review model's context window in tokens, used
// to set the compaction threshold. GADFLY_MODEL_CONTEXT_TOKENS overrides it
// (needed for custom/self-hosted endpoints executus can't introspect); otherwise
// it asks executus/model, which knows the static catalog and can fetch an
// Ollama Cloud model's limit via /api/show (one call, at executor-build time).
// Returns 0 — disabling compaction — for an unknown model, mirroring executus's
// "unknown ⇒ don't budget" contract.
func resolveContextTokens(modelSpec string) int {
if v := envInt("GADFLY_MODEL_CONTEXT_TOKENS", 0); v > 0 {
return v
}
key := strings.TrimSpace(os.Getenv("GADFLY_API_KEY"))
if key == "" {
key = strings.TrimSpace(os.Getenv("OLLAMA_API_KEY"))
}
cache := model.NewCloudOllamaLimitCache("", key, nil)
ctx, cancel := context.WithTimeout(context.Background(), contextTokenLookupTimeout)
defer cancel()
if n, ok := model.MaxContextTokensResolving(ctx, modelSpec, cache); ok {
return n
}
// Unknown model or a failed lookup (e.g. no key / unreachable endpoint): don't
// guess — compaction is disabled. Log it so a misconfiguration is debuggable
// rather than silently dropping the protection. Set GADFLY_MODEL_CONTEXT_TOKENS
// to force a window for an endpoint executus can't introspect.
fmt.Fprintf(os.Stderr, "gadfly: no context window resolved for %q; compaction disabled (set GADFLY_MODEL_CONTEXT_TOKENS to enable it)\n", modelSpec)
return 0
}
-140
View File
@@ -1,140 +0,0 @@
package main
import (
"context"
"testing"
llm "gitea.stevedudenhoeffer.com/steve/majordomo/llm"
)
func TestGadflyBudget(t *testing.T) {
ctx := context.Background()
// A nil budget never blocks and never panics.
var nilB *gadflyBudget
if err := nilB.Check(ctx, "pr"); err != nil {
t.Errorf("nil budget Check should be nil, got %v", err)
}
nilB.Commit(ctx, "pr", 10)
nilB.addUsage(llm.Usage{InputTokens: 5})
// Token ceiling: passes until usage crosses it.
b := &gadflyBudget{maxTokens: 100}
if err := b.Check(ctx, "pr"); err != nil {
t.Fatalf("fresh budget should pass, got %v", err)
}
b.addUsage(llm.Usage{InputTokens: 60, OutputTokens: 50}) // 110 >= 100
if err := b.Check(ctx, "pr"); err == nil {
t.Error("budget over the token cap should reject the next run")
}
// Seconds ceiling, accumulated via Commit.
s := &gadflyBudget{maxSeconds: 30}
s.Commit(ctx, "pr", 31)
if err := s.Check(ctx, "pr"); err == nil {
t.Error("budget over the time cap should reject the next run")
}
}
func TestNewPRBudget(t *testing.T) {
t.Setenv("GADFLY_PR_TOKEN_BUDGET", "")
t.Setenv("GADFLY_PR_TIME_BUDGET_SECS", "")
if newPRBudget() != nil {
t.Error("no caps set should yield a nil (disabled) budget")
}
t.Setenv("GADFLY_PR_TOKEN_BUDGET", "1000")
if b := newPRBudget(); b == nil || b.maxTokens != 1000 {
t.Errorf("token cap should build a budget with maxTokens=1000, got %+v", b)
}
}
func TestCompactRatio(t *testing.T) {
t.Setenv("GADFLY_COMPACT_RATIO", "")
if got := compactRatio(); got != defaultCompactRatio {
t.Errorf("default ratio = %v, want %v", got, defaultCompactRatio)
}
t.Setenv("GADFLY_COMPACT_RATIO", "0.6")
if got := compactRatio(); got != 0.6 {
t.Errorf("ratio override = %v, want 0.6", got)
}
for _, bad := range []string{"0", "-1", "2", "nope"} {
t.Setenv("GADFLY_COMPACT_RATIO", bad)
if got := compactRatio(); got != defaultCompactRatio {
t.Errorf("invalid ratio %q should fall back to the default, got %v", bad, got)
}
}
}
func TestResolveContextTokensOverride(t *testing.T) {
// The explicit override short-circuits any model introspection (no network).
t.Setenv("GADFLY_MODEL_CONTEXT_TOKENS", "123456")
if got := resolveContextTokens("anything"); got != 123456 {
t.Errorf("explicit context-token override = %d, want 123456", got)
}
}
func TestPRCallerID(t *testing.T) {
t.Setenv("GADFLY_REPO", "")
t.Setenv("GADFLY_PR", "")
if got := prCallerID(); got != "local" {
t.Errorf("no repo/PR should be %q, got %q", "local", got)
}
t.Setenv("GADFLY_REPO", "steve/mort")
t.Setenv("GADFLY_PR", "1367")
if got := prCallerID(); got != "steve/mort#1367" {
t.Errorf("callerID = %q, want steve/mort#1367", got)
}
}
func TestCompactionEnabled(t *testing.T) {
for _, v := range []string{"", "1", "true", "yes"} {
t.Setenv("GADFLY_COMPACT", v)
if !compactionEnabled() {
t.Errorf("GADFLY_COMPACT=%q should be enabled", v)
}
}
for _, v := range []string{"0", "false", "no", "off"} {
t.Setenv("GADFLY_COMPACT", v)
if compactionEnabled() {
t.Errorf("GADFLY_COMPACT=%q should be disabled", v)
}
}
}
func TestGadflyToolRegistry(t *testing.T) {
fs, err := newRepoFS(t.TempDir(), "diff")
if err != nil {
t.Fatal(err)
}
_, names, err := gadflyToolRegistry(fs)
if err != nil {
t.Fatalf("gadflyToolRegistry: %v", err)
}
want := map[string]bool{"read_file": true, "list_dir": true, "grep": true, "find_files": true, "get_diff": true}
has := func(n string) bool {
for _, g := range names {
if g == n {
return true
}
}
return false
}
for n := range want {
if !has(n) {
t.Errorf("registry missing tool %q (got %v)", n, names)
}
}
if has("delegate_investigation") {
t.Error("delegate_investigation must be absent without a worker model")
}
// With a worker model the delegate tool is registered too.
fs.worker = fakeModel(t, "x")
_, names, err = gadflyToolRegistry(fs)
if err != nil {
t.Fatalf("gadflyToolRegistry with worker: %v", err)
}
if !has("delegate_investigation") {
t.Errorf("delegate_investigation should be registered with a worker model, got %v", names)
}
}
+3 -3
View File
@@ -104,7 +104,7 @@ func TestRunSpecialists_FansOut(t *testing.T) {
} }
specs := threeLenses() specs := threeLenses()
results := runSpecialists(testEngine(t, mdl, fs), "sys", specs, "task", "diff") results := runSpecialists(&majordomoEngine{mdl: mdl, fsTools: fs}, "sys", specs, "task", "diff")
if got := peak(); got != 3 { if got := peak(); got != 3 {
t.Errorf("peak concurrent lenses = %d, want 3", got) t.Errorf("peak concurrent lenses = %d, want 3", got)
@@ -124,7 +124,7 @@ func TestRunSpecialists_SequentialByDefault(t *testing.T) {
} }
specs := threeLenses() specs := threeLenses()
results := runSpecialists(testEngine(t, mdl, fs), "sys", specs, "task", "diff") results := runSpecialists(&majordomoEngine{mdl: mdl, fsTools: fs}, "sys", specs, "task", "diff")
if got := peak(); got != 1 { if got := peak(); got != 1 {
t.Errorf("peak concurrent lenses = %d, want 1 (sequential by default)", got) t.Errorf("peak concurrent lenses = %d, want 1 (sequential by default)", got)
@@ -146,7 +146,7 @@ func TestRunSpecialists_PerProviderFanOut(t *testing.T) {
} }
specs := threeLenses() specs := threeLenses()
results := runSpecialists(testEngine(t, mdl, fs), "sys", specs, "task", "diff") results := runSpecialists(&majordomoEngine{mdl: mdl, fsTools: fs}, "sys", specs, "task", "diff")
if got := peak(); got != 3 { if got := peak(); got != 3 {
t.Errorf("peak concurrent lenses = %d, want 3 (m1 per-provider override)", got) t.Errorf("peak concurrent lenses = %d, want 3 (m1 per-provider override)", got)
+153 -98
View File
@@ -22,11 +22,7 @@
// //
// GADFLY_MODEL model id, or a full "provider/model" spec / majordomo // GADFLY_MODEL model id, or a full "provider/model" spec / majordomo
// alias / failover chain (required). A bare id is // alias / failover chain (required). A bare id is
// prefixed with GADFLY_PROVIDER. Two prefixes select a // prefixed with GADFLY_PROVIDER.
// shell-out CLI engine instead of the in-process loop:
// "claude-code/<model>" (Claude Code, see engine.go) and
// "opencode/<model>" (OpenCode over ollama-cloud, see
// opencode.go) — both bring their own repo tools.
// GADFLY_PROVIDER provider for bare model ids (default "ollama-cloud"; // GADFLY_PROVIDER provider for bare model ids (default "ollama-cloud";
// e.g. "ollama" for a local daemon, "openai", …). // e.g. "ollama" for a local daemon, "openai", …).
// GADFLY_BASE_URL override the backend endpoint (OpenAI/Ollama-compatible // GADFLY_BASE_URL override the backend endpoint (OpenAI/Ollama-compatible
@@ -43,9 +39,10 @@
// GADFLY_TITLE PR title (optional). // GADFLY_TITLE PR title (optional).
// GADFLY_BODY PR description (optional). // GADFLY_BODY PR description (optional).
// GADFLY_MAX_STEPS review-pass step cap (optional, default 24). // GADFLY_MAX_STEPS review-pass step cap (optional, default 24).
// GADFLY_WRAPUP_RESERVE steps before the cap at which the wrap-up critic nudges // GADFLY_WRAPUP_RESERVE steps before the cap at which the agent is told to
// the agent to stop investigating and write its answer // stop investigating and write its answer (optional,
// (optional, default 4). // default 4). Plus a tool-free finalization fallback
// guarantees a step-exhausted pass still emits output.
// GADFLY_RECHECK set to 0/false to skip the recheck pass (optional, default on). // GADFLY_RECHECK set to 0/false to skip the recheck pass (optional, default on).
// GADFLY_RECHECK_MAX_STEPS recheck-pass step cap (optional, default 16). // GADFLY_RECHECK_MAX_STEPS recheck-pass step cap (optional, default 16).
// GADFLY_TIMEOUT_SECS overall deadline in seconds, shared by both passes (optional, default 300). // GADFLY_TIMEOUT_SECS overall deadline in seconds, shared by both passes (optional, default 300).
@@ -58,8 +55,8 @@
// lanes as GADFLY_PROVIDER_CONCURRENCY (e.g. // lanes as GADFLY_PROVIDER_CONCURRENCY (e.g.
// "ollama-cloud=3,m1=1"). Wins over GADFLY_LENS_CONCURRENCY // "ollama-cloud=3,m1=1"). Wins over GADFLY_LENS_CONCURRENCY
// for the model's provider; falls back to it otherwise. // for the model's provider; falls back to it otherwise.
// GADFLY_MAX_DIFF_CHARS diff chars embedded in the review prompt (optional, default 60000; // GADFLY_MAX_DIFF_CHARS diff chars embedded in the prompt (optional, default 60000;
// the full diff is reachable via the paginated get_diff tool). // the full diff is always available via the get_diff tool).
// //
// On success it prints the review to stdout and exits 0. On a usage/config or // On success it prints the review to stdout and exits 0. On a usage/config or
// model error it prints a diagnostic to stderr and exits non-zero; run.sh then // model error it prints a diagnostic to stderr and exits non-zero; run.sh then
@@ -73,9 +70,11 @@ import (
"os" "os"
"strconv" "strconv"
"strings" "strings"
"sync"
"time" "time"
"gitea.stevedudenhoeffer.com/steve/executus/fanout" "gitea.stevedudenhoeffer.com/steve/majordomo/agent"
llm "gitea.stevedudenhoeffer.com/steve/majordomo/llm"
) )
const ( const (
@@ -108,6 +107,13 @@ const wrapUpInstruction = "⚠️ You are almost out of your investigation budge
"Do not begin any new investigation. If a finding could not be confirmed, drop it or mark it explicitly as unverified. " + "Do not begin any new investigation. If a finding could not be confirmed, drop it or mark it explicitly as unverified. " +
"Output the review in the required format right now." "Output the review in the required format right now."
// finalizeInstruction is the user message sent on the tool-free fallback pass
// when the agent exhausted its budget (or tripped a loop guard) without ever
// producing a final answer. It forces the model to synthesize whatever it has.
const finalizeInstruction = "You have run out of investigation steps. Do NOT call any tools. " +
"Based solely on what you have already gathered above, write your final answer now in the required format. " +
"If you could not confirm some findings, omit them or mark them as unverified, but produce the answer."
func main() { func main() {
if err := run(); err != nil { if err := run(); err != nil {
fmt.Fprintln(os.Stderr, "gadfly:", err) fmt.Fprintln(os.Stderr, "gadfly:", err)
@@ -151,18 +157,15 @@ func run() error {
return err return err
} }
// Resolve the review engine. The shell-out engines (claude-code, opencode) // Resolve the review engine. The claude-code engine shells out to the
// bring their OWN repo tools; every other spec is an in-process majordomo // `claude` CLI (its own repo tools); every other spec is a majordomo model.
// model. auto-selection and the delegate worker are majordomo-only — with a // auto-selection and the delegate worker are majordomo-only — with
// shell-out engine they're skipped (the CLI does its own legwork). // claude-code they're skipped (Claude Code does its own legwork).
spec := os.Getenv("GADFLY_MODEL") ccSpec := isClaudeCodeSpec(os.Getenv("GADFLY_MODEL"))
var eng reviewEngine var eng reviewEngine
switch { if ccSpec {
case isClaudeCodeSpec(spec): eng = newClaudeCodeEngine(os.Getenv("GADFLY_MODEL"), fsTools.root)
eng = newClaudeCodeEngine(spec, fsTools.root) } else {
case isOpenCodeSpec(spec):
eng = newOpenCodeEngine(spec, fsTools.root)
default:
mdl, merr := resolveModel() mdl, merr := resolveModel()
if merr != nil { if merr != nil {
return fmt.Errorf("resolve model: %w", merr) return fmt.Errorf("resolve model: %w", merr)
@@ -174,18 +177,7 @@ func run() error {
} else if worker != nil { } else if worker != nil {
fsTools.worker = worker fsTools.worker = worker
} }
// The context compactor needs a cheap summarizer; reuse the worker model eng = &majordomoEngine{mdl: mdl, fsTools: fsTools}
// when present, else the review model. A bad explicit GADFLY_COMPACT_MODEL
// just disables compaction rather than sinking the review.
summarizer, serr := resolveSummarizerModel(mdl, fsTools.worker)
if serr != nil {
fmt.Fprintln(os.Stderr, "gadfly: compaction summarizer disabled:", serr)
}
rex, rerr := newReviewExecutor(fsTools, mdl, summarizer, os.Getenv("GADFLY_MODEL"), newPRBudget())
if rerr != nil {
return fmt.Errorf("build review executor: %w", rerr)
}
eng = &majordomoEngine{rex: rex, mdl: mdl}
} }
specialists, registry, auto, serrs := resolveSpecialists(repoDir) specialists, registry, auto, serrs := resolveSpecialists(repoDir)
@@ -194,15 +186,13 @@ func run() error {
} }
// Dynamic selection: a (cheap) model picks the lenses this diff needs. // Dynamic selection: a (cheap) model picks the lenses this diff needs.
// Majordomo-only — the selector is an llm.Model, so a shell-out engine // Majordomo-only — the selector is an llm.Model.
// (claude-code, opencode) can't provide one; fall back to the default suite.
if auto { if auto {
md, ok := eng.(*majordomoEngine) if ccSpec {
if !ok { fmt.Fprintln(os.Stderr, "gadfly: auto-select is not supported with the claude-code engine; using the default suite")
fmt.Fprintln(os.Stderr, "gadfly: auto-select requires an in-process model engine; using the default suite")
specialists = suiteFromRegistry(registry, defaultSuite) specialists = suiteFromRegistry(registry, defaultSuite)
} else { } else {
selector, serr := resolveSelectorModel(md.mdl) selector, serr := resolveSelectorModel(eng.(*majordomoEngine).mdl)
if serr != nil { if serr != nil {
return fmt.Errorf("resolve selector model: %w", serr) return fmt.Errorf("resolve selector model: %w", serr)
} }
@@ -241,55 +231,55 @@ func run() error {
} }
// runSpecialists reviews the diff through each lens and returns the results in // runSpecialists reviews the diff through each lens and returns the results in
// the SAME order as specialists, regardless of finish order. It uses executus's // the SAME order as specialists, regardless of finish order. Up to
// fanout primitive: up to GADFLY_LENS_CONCURRENCY lenses run concurrently (the // GADFLY_LENS_CONCURRENCY lenses run concurrently; the default of 1 keeps the
// default of 1 keeps the suite sequential, exactly as before), and fanout.Run // suite sequential, exactly as before. Each lens already runs under its own
// returns one result per lens in input order. Each lens already runs under its // per-lens timeout (reviewWithSpecialist), so concurrency simply overlaps those
// own per-lens timeout (reviewWithSpecialist) and the lenses only read the // independent passes — and because reviewWithSpecialist builds a fresh toolbox
// immutable repoFS, so concurrency simply overlaps independent passes. // per pass and the lenses only read the immutable repoFS, they share no mutable
// state. Results are stored by index so the consolidated comment keeps the
// configured lens order.
// //
// Caution: this fans out WITHIN one model. It multiplies with entrypoint.sh's // Caution: this fans out WITHIN one model. It multiplies with entrypoint.sh's
// per-provider model concurrency, so total concurrent backend requests ≈ // per-provider model concurrency, so total concurrent backend requests ≈
// (models at once) × (lenses at once). To fan lenses out without oversubscribing // (models at once) × (lenses at once). To fan lenses out without oversubscribing
// the backend, run models one at a time (provider lane cap 1) and raise this. // the backend, run models one at a time (provider lane cap 1) and raise this.
func runSpecialists(eng reviewEngine, base string, specialists []Specialist, task, diff string) []specialistResult { func runSpecialists(eng reviewEngine, base string, specialists []Specialist, task, diff string) []specialistResult {
results := make([]specialistResult, len(specialists))
// Optional live status board: publishes this model's per-lens progress to a // Optional live status board: publishes this model's per-lens progress to a
// file the entrypoint board renders. Inert (no-op) unless GADFLY_STATUS_FILE // file the entrypoint board renders. Inert (no-op) unless GADFLY_STATUS_FILE
// is set, so plain runs are unaffected. // is set, so plain runs are unaffected.
sw := newStatusWriter(os.Getenv("GADFLY_MODEL"), modelProvider(), specialists) sw := newStatusWriter(os.Getenv("GADFLY_MODEL"), modelProvider(), specialists)
fanResults := fanout.Run(context.Background(), specialists, fanout.Options[Specialist]{ conc := min(lensConcurrency(), len(specialists))
MaxConcurrent: lensConcurrency(),
}, func(_ context.Context, sp Specialist) (res specialistResult, _ error) {
// A panic in one lens must not crash the whole binary (which would kill
// every other lens's output) or leave this lens stuck at "running" on the
// status board. fanout does not recover fn panics, so we do it here:
// record the panic as an errored result and mark the lens finished.
defer func() {
if r := recover(); r != nil {
res = specialistResult{spec: sp, out: fmt.Sprintf("⚠️ This reviewer panicked: %v", r), verdict: verdictUnknown, errored: true}
sw.set(sp.Name, lensFinished, "", true)
}
}()
sw.set(sp.Name, lensRunning, "", false)
out, errored := reviewWithSpecialist(eng, base, sp, task, diff)
v := parseVerdict(out)
sw.set(sp.Name, lensFinished, v.label(), errored)
return specialistResult{spec: sp, out: out, verdict: v, errored: errored}, nil
})
// fanout guarantees input order; its Result.Err is set only when the context sem := make(chan struct{}, conc)
// is cancelled before a lens ran (reviewWithSpecialist embeds its own failures var wg sync.WaitGroup
// in the result), so surface that as an errored lens rather than dropping it. for i, sp := range specialists {
results := make([]specialistResult, len(specialists)) wg.Add(1)
for i, r := range fanResults { sem <- struct{}{} // blocks once `conc` lenses are already in flight
if r.Err != nil { go func(i int, sp Specialist) {
results[i] = specialistResult{spec: specialists[i], out: fmt.Sprintf("⚠️ This reviewer did not run: %v", r.Err), verdict: verdictUnknown, errored: true} defer wg.Done()
sw.set(specialists[i].Name, lensFinished, "", true) defer func() { <-sem }()
continue // A panic in one lens must not crash the whole binary (which would
} // kill every other lens's output) or leave this lens stuck at
results[i] = r.Value // "running" on the status board. Recover, record it as an errored
// result, and mark the lens finished so the board can complete.
defer func() {
if r := recover(); r != nil {
results[i] = specialistResult{spec: sp, out: fmt.Sprintf("⚠️ This reviewer panicked: %v", r), verdict: verdictUnknown, errored: true}
sw.set(sp.Name, lensFinished, "", true)
}
}()
sw.set(sp.Name, lensRunning, "", false)
out, errored := reviewWithSpecialist(eng, base, sp, task, diff)
v := parseVerdict(out)
results[i] = specialistResult{spec: sp, out: out, verdict: v, errored: errored}
sw.set(sp.Name, lensFinished, v.label(), errored)
}(i, sp)
} }
wg.Wait()
return results return results
} }
@@ -330,7 +320,8 @@ func providerOverride(envName, provider string) (int, bool) {
// returned bool is true when the review pass failed (rendered as an inline // returned bool is true when the review pass failed (rendered as an inline
// notice — advisory; one lens failing never sinks the others or the job). // notice — advisory; one lens failing never sinks the others or the job).
func reviewWithSpecialist(eng reviewEngine, base string, sp Specialist, task, diff string) (string, bool) { func reviewWithSpecialist(eng reviewEngine, base string, sp Specialist, task, diff string) (string, bool) {
ctx, cancel := context.WithTimeout(context.Background(), reviewTimeout()) timeout := time.Duration(envInt("GADFLY_TIMEOUT_SECS", defaultTimeoutSecs)) * time.Second
ctx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel() defer cancel()
draft, err := eng.runPass(ctx, composeSpecialistPrompt(base, sp), task, draft, err := eng.runPass(ctx, composeSpecialistPrompt(base, sp), task,
@@ -353,6 +344,90 @@ func reviewWithSpecialist(eng reviewEngine, base string, sp Specialist, task, di
return final, false return final, false
} }
// runAgent runs one agent pass (its own fresh toolbox over the sandbox) and
// returns the final answer. An empty answer is an error — the caller decides
// whether that is fatal (review pass) or recoverable (recheck pass). A
// non-empty answer that ended on a budget/guard error is still returned: the
// model wrote its output, then ran out of steps.
//
// Two mechanisms keep a step-hungry model from hard-failing with no output:
// 1. A wrap-up steer: once the run comes within wrapUpReserve steps of the
// cap, a forceful "stop calling tools, write your final answer" message is
// injected so the model spends its remaining steps finalizing.
// 2. A finalization fallback: if the loop still ends empty (the model ignored
// the nudge, or a loop guard tripped), one tool-free model call forces a
// final answer out of the transcript already gathered.
func runAgent(ctx context.Context, mdl llm.Model, fsTools *repoFS, system, task string, maxSteps int) (string, error) {
box, err := fsTools.toolbox()
if err != nil {
return "", err
}
loop := agent.New(mdl, system,
agent.WithToolbox(box),
agent.WithMaxSteps(maxSteps),
// Guard rails: stop the model from spinning on failing or identical
// tool calls instead of writing its answer.
agent.WithToolErrorLimits(4, 4),
)
wrapUpAt := maxSteps - wrapUpReserve()
if wrapUpAt < 1 {
wrapUpAt = 1
}
var completed int // steps finished so far (updated after each step)
nudged := false
res, runErr := loop.Run(ctx, task,
agent.OnStep(func(s agent.Step) { completed = s.Index + 1 }),
agent.WithSteer(func() []llm.Message {
if !nudged && completed >= wrapUpAt {
nudged = true
return []llm.Message{llm.UserText(wrapUpInstruction)}
}
return nil
}),
)
out := ""
if res != nil {
out = strings.TrimSpace(res.Output)
}
if out != "" {
return out, nil
}
// No final answer. If we still have budget on the clock and a transcript to
// work from, force a tool-free finalization rather than losing the pass.
if res != nil && len(res.Messages) > 0 && ctx.Err() == nil {
if forced := forceFinalAnswer(ctx, mdl, system, res.Messages); forced != "" {
return forced, nil
}
}
if runErr != nil {
return "", runErr
}
return "", errors.New("agent produced no output")
}
// forceFinalAnswer makes one tool-free model call to squeeze a final answer out
// of an agent that exhausted its step budget without producing one. Tools are
// forbidden (ToolChoice "none") so the model must synthesize from the transcript
// instead of investigating further. Best-effort: any error or empty reply
// returns "" and the caller falls back to its normal empty-output handling.
func forceFinalAnswer(ctx context.Context, mdl llm.Model, system string, transcript []llm.Message) string {
msgs := append(append([]llm.Message(nil), transcript...), llm.UserText(finalizeInstruction))
resp, err := mdl.Generate(ctx, llm.Request{
System: system,
Messages: msgs,
ToolChoice: "none",
})
if err != nil || resp == nil {
return ""
}
return strings.TrimSpace(resp.Text())
}
// wrapUpReserve is how many steps before the cap the wrap-up nudge fires, // wrapUpReserve is how many steps before the cap the wrap-up nudge fires,
// overridable via GADFLY_WRAPUP_RESERVE. // overridable via GADFLY_WRAPUP_RESERVE.
func wrapUpReserve() int { func wrapUpReserve() int {
@@ -369,7 +444,7 @@ func buildTask(diff string) string {
truncNote := "" truncNote := ""
if maxDiff > 0 && len(diff) > maxDiff { if maxDiff > 0 && len(diff) > maxDiff {
diff = diff[:maxDiff] diff = diff[:maxDiff]
truncNote = fmt.Sprintf("\n\n[NOTE: diff truncated to %d chars in this message; page the full diff with get_diff (paginated; pass a `path` to scope it to one file) or read the changed files.]", maxDiff) truncNote = fmt.Sprintf("\n\n[NOTE: diff truncated to %d chars in this message; read the changed files (or call get_diff, if available) for the full text.]", maxDiff)
} }
var b strings.Builder var b strings.Builder
@@ -396,23 +471,3 @@ func envInt(name string, def int) int {
} }
return n return n
} }
// envBool reads a boolean-ish env var: def when unset, false for an explicit
// falsey value (0/false/no/off), true otherwise. The shared spelling for
// gadfly's "on unless disabled" opt-out flags (GADFLY_RECHECK, GADFLY_COMPACT).
func envBool(name string, def bool) bool {
switch strings.ToLower(strings.TrimSpace(os.Getenv(name))) {
case "":
return def
case "0", "false", "no", "off":
return false
default:
return true
}
}
// reviewTimeout is the per-specialist-lens deadline (GADFLY_TIMEOUT_SECS), shared
// across a lens's review+recheck passes and applied as each pass's run cap.
func reviewTimeout() time.Duration {
return time.Duration(envInt("GADFLY_TIMEOUT_SECS", defaultTimeoutSecs)) * time.Second
}
-21
View File
@@ -127,27 +127,6 @@ func resolveWorkerModel() (llm.Model, error) {
return majordomo.Parse(buildSpec(provider, spec)) return majordomo.Parse(buildSpec(provider, spec))
} }
// resolveSummarizerModel picks the model the context compactor uses to compress
// the runaway middle of a transcript. It should be CHEAP, since it fires once per
// compaction: GADFLY_COMPACT_MODEL if set (honoring GADFLY_PROVIDER for a bare
// id), else the delegate worker model when one is configured (already cheap by
// design), else the review model itself. Returns (nil, err) only on an explicit
// bad GADFLY_COMPACT_MODEL spec — the caller logs it and simply runs without
// compaction rather than failing the review.
func resolveSummarizerModel(review, worker llm.Model) (llm.Model, error) {
if spec := strings.TrimSpace(os.Getenv("GADFLY_COMPACT_MODEL")); spec != "" {
provider := strings.TrimSpace(os.Getenv("GADFLY_PROVIDER"))
if provider == "" {
provider = defaultProvider
}
return majordomo.Parse(buildSpec(provider, spec))
}
if worker != nil {
return worker, nil
}
return review, nil
}
// buildSpec turns (provider, model) into a majordomo spec. A model id that // buildSpec turns (provider, model) into a majordomo spec. A model id that
// already carries a "provider/" prefix (or is a multi-element failover chain) // already carries a "provider/" prefix (or is a multi-element failover chain)
// is passed through verbatim; a bare id is prefixed with the provider. // is passed through verbatim; a bare id is prefixed with the provider.
-273
View File
@@ -1,273 +0,0 @@
package main
import (
"bytes"
"context"
"encoding/json"
"fmt"
"os"
"os/exec"
"strings"
)
// openCodeEngine reviews by shelling out to the `opencode` CLI (opencode.ai) in
// non-interactive `run` mode. Like the claude-code engine it is a pure shell-out
// — OpenCode brings its OWN read tools (read/grep/glob/list) and reads the
// checked-out tree, so findings are verified against real code — but it drives an
// ollama-cloud model instead of a Claude subscription. The point is to benchmark
// gadfly's boutique executus harness against a freely-available agentic harness
// on the SAME models (e.g. "ollama-cloud/glm-5.2" vs "opencode/glm-5.2").
//
// OpenCode has no --append-system-prompt flag, so the lens system prompt AND the
// read-only discipline are delivered through a generated config injected via the
// OPENCODE_CONFIG_CONTENT env var (see config): a custom "gadfly" agent whose
// prompt is the system prompt with edit/bash denied, plus a "gadfly" provider
// pointing at ollama-cloud. OPENCODE_CONFIG_CONTENT is the highest-precedence
// config source that matters in the container — it outranks any opencode.json a
// reviewed repo might ship — so a repo can't re-enable edits on us.
type openCodeEngine struct {
bin string // CLI binary (GADFLY_OPENCODE_BIN, default "opencode")
providerModel string // ollama model id for the generated "gadfly" provider ("" = none)
modelRef string // --model value ("gadfly/<id>", a pass-through "<prov>/<id>", or "" = CLI default)
baseURL string // ollama-cloud base URL (GADFLY_OPENCODE_BASE_URL)
repoDir string // cwd for the CLI, so its tools read the checked-out tree
extraArgs []string // appended verbatim (GADFLY_OPENCODE_EXTRA_ARGS)
}
// openCodeProviderName / openCodeAgentName are the internal names of the provider
// and agent gadfly generates in the injected config. "gadfly" won't collide with
// OpenCode's models.dev provider registry.
const (
openCodeProviderName = "gadfly"
openCodeAgentName = "gadfly"
)
// defaultOpenCodeBaseURL is ollama-cloud's OpenAI-compatible endpoint.
const defaultOpenCodeBaseURL = "https://ollama.com/v1"
// isOpenCodeSpec reports whether a GADFLY_MODEL spec selects the opencode engine:
// the bare id "opencode"/"open-code" or an "opencode/<model>" form (both
// spellings accepted; "opencode" is canonical).
func isOpenCodeSpec(model string) bool {
m := strings.TrimSpace(model)
for _, p := range []string{"opencode", "open-code"} {
if m == p || strings.HasPrefix(m, p+"/") {
return true
}
}
return false
}
// newOpenCodeEngine builds the engine from the GADFLY_MODEL spec and the optional
// GADFLY_OPENCODE_* overrides. The part after the FIRST slash is the model, taken
// verbatim — no ":"-suffix parsing, because ollama model ids legitimately contain
// colons (e.g. "qwen3-coder:480b-cloud"). Three spec forms:
//
// opencode → bare: no --model, no generated provider (CLI default model)
// opencode/<model> → wrap <model> in the generated ollama-cloud "gadfly" provider
// opencode/<provider>/<model> → pass-through: --model <provider>/<model>, using OpenCode's
// own provider registry/auth (escape hatch, no generated provider)
//
// GADFLY_OPENCODE_MODEL overrides the model taken from the spec (and is itself run
// through the same slash logic). It does not verify the CLI is installed — a
// missing binary surfaces as a normal pass error (advisory, never fatal).
func newOpenCodeEngine(spec, repoDir string) *openCodeEngine {
e := &openCodeEngine{
bin: envOr("GADFLY_OPENCODE_BIN", "opencode"),
baseURL: envOr("GADFLY_OPENCODE_BASE_URL", defaultOpenCodeBaseURL),
repoDir: repoDir,
extraArgs: strings.Fields(os.Getenv("GADFLY_OPENCODE_EXTRA_ARGS")),
}
var after string
if _, a, ok := strings.Cut(strings.TrimSpace(spec), "/"); ok {
after = strings.TrimSpace(a)
}
if env := strings.TrimSpace(os.Getenv("GADFLY_OPENCODE_MODEL")); env != "" {
after = env
}
switch {
case after == "":
// bare spec: let OpenCode's configured default model apply.
case strings.Contains(after, "/"):
// "<provider>/<model>" pass-through to an OpenCode built-in provider.
e.modelRef = after
default:
// A bare model id → serve it via the generated ollama-cloud provider.
e.providerModel = after
e.modelRef = openCodeProviderName + "/" + after
}
return e
}
// args assembles the `opencode` argv for one pass. Factored out (and pure) so it
// can be unit-tested without invoking the CLI. The task is the positional message
// and MUST come last; it never begins with '-' (buildTask output starts with "PR
// title:"/"Review …"), so no "--" terminator is needed. Note: in `opencode run`,
// -p is basic-auth password, NOT the prompt — the message is positional.
func (e *openCodeEngine) args(task string) []string {
a := []string{"run", "--agent", openCodeAgentName}
if e.modelRef != "" {
a = append(a, "--model", e.modelRef)
}
a = append(a, e.extraArgs...)
return append(a, task)
}
// openCode config structs — the minimal shape gadfly generates. Marshaled to JSON
// and handed to the CLI via OPENCODE_CONFIG_CONTENT.
type openCodeConfig struct {
Schema string `json:"$schema"`
Permission openCodePermission `json:"permission"`
Provider map[string]openCodeProvider `json:"provider,omitempty"`
Agent map[string]openCodeAgent `json:"agent"`
}
// openCodePermission is OpenCode's permission map: tool key → "allow"|"ask"|"deny".
type openCodePermission map[string]string
// denyMutations denies every OpenCode permission that could change the repo, run
// commands, or reach the network / the filesystem outside the checked-out tree —
// keeping the reviewer strictly read-only. The read/search tools OpenCode gates
// separately (read/glob/grep/list/lsp) stay at their default so the agent can
// still verify findings against the code. OpenCode's permission keys are
// enumerated at https://opencode.ai/docs/agents; a key it doesn't recognize is
// simply ignored, so listing extras is safe.
func denyMutations() openCodePermission {
return openCodePermission{
"edit": "deny",
"bash": "deny",
"webfetch": "deny",
"websearch": "deny",
"external_directory": "deny",
}
}
type openCodeProvider struct {
NPM string `json:"npm"`
Name string `json:"name"`
Options openCodeProviderOptions `json:"options"`
Models map[string]struct{} `json:"models"`
}
type openCodeProviderOptions struct {
BaseURL string `json:"baseURL"`
APIKey string `json:"apiKey"`
}
type openCodeAgent struct {
Description string `json:"description"`
Mode string `json:"mode"`
Prompt string `json:"prompt"`
Permission openCodePermission `json:"permission"`
}
// config builds the OpenCode config JSON for one pass. The system prompt becomes
// the "gadfly" agent's prompt; the mutating/network tools are denied at BOTH the
// global and agent level (defense in depth — OpenCode's read tools stay
// available). For a bare or pass-through spec the generated ollama-cloud provider
// block is omitted. The API key is always the "{env:OLLAMA_API_KEY}" reference,
// never a literal secret baked into the config.
func (e *openCodeEngine) config(system string) ([]byte, error) {
deny := denyMutations()
cfg := openCodeConfig{
Schema: "https://opencode.ai/config.json",
Permission: deny,
Agent: map[string]openCodeAgent{
openCodeAgentName: {
Description: "Gadfly adversarial code-review lens (read-only).",
Mode: "primary",
Prompt: system,
Permission: deny,
},
},
}
if e.providerModel != "" {
cfg.Provider = map[string]openCodeProvider{
openCodeProviderName: {
NPM: "@ai-sdk/openai-compatible",
Name: openCodeProviderName,
Options: openCodeProviderOptions{
BaseURL: e.baseURL,
APIKey: "{env:OLLAMA_API_KEY}",
},
Models: map[string]struct{}{e.providerModel: {}},
},
}
}
return json.Marshal(cfg)
}
func (e *openCodeEngine) runPass(ctx context.Context, system, task string, _ int) (string, error) {
cfg, err := e.config(system)
if err != nil {
return "", fmt.Errorf("opencode config: %w", err)
}
cmd := exec.CommandContext(ctx, e.bin, e.args(task)...)
cmd.Dir = e.repoDir
// Inject the review config (system prompt as the agent prompt + read-only
// permissions + the ollama-cloud provider) via OPENCODE_CONFIG_CONTENT, which
// outranks any opencode.json the reviewed repo itself ships. NO_COLOR keeps the
// captured stdout free of ANSI decoration.
cmd.Env = append(openCodeEnv(), "OPENCODE_CONFIG_CONTENT="+string(cfg), "NO_COLOR=1")
killGroupOnCancel(cmd) // don't orphan the CLI's Node children on a timed-out lens
var stdout, stderr bytes.Buffer
cmd.Stdout = &stdout
cmd.Stderr = &stderr
runErr := cmd.Run()
// A cancelled/timed-out run must surface as an error, never as whatever partial
// bytes the CLI flushed before it was killed.
if ctx.Err() != nil {
return "", fmt.Errorf("opencode run %v", ctx.Err())
}
if runErr != nil {
detail := truncateForErr(stderr.String())
if detail == "" {
detail = truncateForErr(stdout.String())
}
if detail != "" {
return "", fmt.Errorf("opencode run failed: %v: %s", runErr, detail)
}
return "", fmt.Errorf("opencode run failed: %v", runErr)
}
// OpenCode's default (non-JSON) format prints the assistant's final text; trust
// it as the review. An empty result on a clean exit is an error, never "".
if out := strings.TrimSpace(stdout.String()); out != "" {
return out, nil
}
return "", fmt.Errorf("opencode run returned no output")
}
// openCodeEnv builds a minimal environment for the `opencode` subprocess. It
// forwards what the CLI needs to reach a model provider: OLLAMA_API_KEY for the
// generated ollama-cloud provider (the primary opencode/<model> path), PLUS the
// standard provider keys — ANTHROPIC_*, OPENAI_*, GOOGLE_*, GEMINI_* — so the
// opencode/<provider>/<model> pass-through form can authenticate against
// OpenCode's own built-in providers (those keys are otherwise stripped, which
// broke the documented escape hatch). It still withholds gadfly's OWN secrets —
// GITEA_TOKEN, GADFLY_API_KEY, GADFLY_FINDINGS_TOKEN, and the claude-code
// subscription token (CLAUDE_CODE_OAUTH_TOKEN, which OpenCode can't use anyway) —
// so they never reach the third-party CLI. OPENCODE_CONFIG_CONTENT is never
// inherited: runPass sets it, and a duplicate key would be ambiguous (getenv
// returns the first occurrence).
func openCodeEnv() []string {
return filterEnv(func(k string) bool {
if k == "OPENCODE_CONFIG_CONTENT" {
return false // set explicitly by runPass; never inherit a competing value
}
switch k {
case "PATH", "HOME", "USER", "LOGNAME", "TMPDIR", "LANG", "TERM", "SHELL", "OLLAMA_API_KEY":
return true
}
return strings.HasPrefix(k, "LC_") ||
strings.HasPrefix(k, "OPENCODE_") ||
strings.HasPrefix(k, "GADFLY_OPENCODE_") ||
strings.HasPrefix(k, "NODE_") ||
strings.HasPrefix(k, "XDG_") ||
strings.HasPrefix(k, "ANTHROPIC_") ||
strings.HasPrefix(k, "OPENAI_") ||
strings.HasPrefix(k, "GOOGLE_") ||
strings.HasPrefix(k, "GEMINI_")
})
}
-303
View File
@@ -1,303 +0,0 @@
package main
import (
"context"
"encoding/json"
"os"
"slices"
"strings"
"testing"
)
func TestIsOpenCodeSpec(t *testing.T) {
cases := map[string]bool{
"opencode": true,
"opencode/glm-5.2": true,
"open-code/glm-5.2": true, // accepted alias spelling
"opencode/qwen3-coder:480b-cloud": true, // colon-bearing model id
"opencode/anthropic/claude": true, // pass-through form
" opencode ": true, // trimmed
"opencode-extra": false, // not the bare id, not a "/" form
"qwen3-coder:480b-cloud": false,
"claude-code/opus": false,
"": false,
}
for spec, want := range cases {
if got := isOpenCodeSpec(spec); got != want {
t.Errorf("isOpenCodeSpec(%q) = %v, want %v", spec, got, want)
}
}
}
func TestNewOpenCodeEngineModel(t *testing.T) {
t.Setenv("GADFLY_OPENCODE_MODEL", "")
// "opencode/<model>" → wrapped in the generated "gadfly" provider.
if e := newOpenCodeEngine("opencode/glm-5.2", "/repo"); e.providerModel != "glm-5.2" || e.modelRef != "gadfly/glm-5.2" {
t.Errorf("glm-5.2: providerModel=%q modelRef=%q, want glm-5.2 / gadfly/glm-5.2", e.providerModel, e.modelRef)
}
// Colon-bearing ollama id is preserved verbatim — NOT split on ":".
if e := newOpenCodeEngine("opencode/qwen3-coder:480b-cloud", "/repo"); e.providerModel != "qwen3-coder:480b-cloud" {
t.Errorf("colon id: providerModel=%q, want qwen3-coder:480b-cloud (no split)", e.providerModel)
}
// "open-code/" spelling behaves identically.
if e := newOpenCodeEngine("open-code/glm-5.2", "/repo"); e.modelRef != "gadfly/glm-5.2" {
t.Errorf("open-code alias: modelRef=%q, want gadfly/glm-5.2", e.modelRef)
}
// Pass-through "opencode/<provider>/<model>" → no generated provider.
if e := newOpenCodeEngine("opencode/anthropic/claude-sonnet-4-6", "/repo"); e.providerModel != "" || e.modelRef != "anthropic/claude-sonnet-4-6" {
t.Errorf("pass-through: providerModel=%q modelRef=%q, want '' / anthropic/claude-sonnet-4-6", e.providerModel, e.modelRef)
}
// Bare spec → no model, no provider (CLI default applies).
if e := newOpenCodeEngine("opencode", "/repo"); e.providerModel != "" || e.modelRef != "" {
t.Errorf("bare: providerModel=%q modelRef=%q, want both empty", e.providerModel, e.modelRef)
}
// GADFLY_OPENCODE_MODEL overrides the spec suffix.
t.Setenv("GADFLY_OPENCODE_MODEL", "deepseek-v3")
if e := newOpenCodeEngine("opencode/glm-5.2", "/repo"); e.providerModel != "deepseek-v3" || e.modelRef != "gadfly/deepseek-v3" {
t.Errorf("env override: providerModel=%q modelRef=%q, want deepseek-v3 / gadfly/deepseek-v3", e.providerModel, e.modelRef)
}
}
func TestOpenCodeEngineDefaults(t *testing.T) {
t.Setenv("GADFLY_OPENCODE_BIN", "")
t.Setenv("GADFLY_OPENCODE_BASE_URL", "")
t.Setenv("GADFLY_OPENCODE_EXTRA_ARGS", "")
e := newOpenCodeEngine("opencode/glm-5.2", "/repo")
if e.bin != "opencode" {
t.Errorf("bin = %q, want opencode", e.bin)
}
if e.baseURL != defaultOpenCodeBaseURL {
t.Errorf("baseURL = %q, want %q", e.baseURL, defaultOpenCodeBaseURL)
}
if e.repoDir != "/repo" {
t.Errorf("repoDir = %q, want /repo", e.repoDir)
}
}
func TestOpenCodeArgs(t *testing.T) {
t.Setenv("GADFLY_OPENCODE_MODEL", "")
t.Setenv("GADFLY_OPENCODE_EXTRA_ARGS", "--variant reasoning")
e := newOpenCodeEngine("opencode/glm-5.2", "/repo")
args := e.args("TASK-PROMPT")
// "run" is the subcommand and must be first.
if len(args) == 0 || args[0] != "run" {
t.Fatalf("args[0] = %q, want run (args=%v)", args, args)
}
if argAfter(args, "--agent") != openCodeAgentName {
t.Errorf("--agent = %q, want %q", argAfter(args, "--agent"), openCodeAgentName)
}
if argAfter(args, "--model") != "gadfly/glm-5.2" {
t.Errorf("--model = %q, want gadfly/glm-5.2", argAfter(args, "--model"))
}
// extra args appended verbatim (split on whitespace).
if !strings.Contains(strings.Join(args, " "), "--variant reasoning") {
t.Errorf("extra args not appended: %v", args)
}
// task is the positional message and must be LAST.
if args[len(args)-1] != "TASK-PROMPT" {
t.Errorf("last arg = %q, want TASK-PROMPT (args=%v)", args[len(args)-1], args)
}
}
func TestOpenCodeArgsBareModelOmitsFlag(t *testing.T) {
t.Setenv("GADFLY_OPENCODE_MODEL", "")
t.Setenv("GADFLY_OPENCODE_EXTRA_ARGS", "")
e := newOpenCodeEngine("opencode", "/repo")
args := e.args("t")
if slices.Contains(args, "--model") {
t.Errorf("--model should be omitted for a bare opencode spec: %v", args)
}
if args[len(args)-1] != "t" {
t.Errorf("last arg = %q, want t", args[len(args)-1])
}
}
func TestOpenCodeConfig(t *testing.T) {
t.Setenv("GADFLY_OPENCODE_MODEL", "")
t.Setenv("GADFLY_OPENCODE_BASE_URL", "")
// Round-trip a system prompt containing quotes and newlines.
sys := "Line one with \"quotes\".\nLine two."
e := newOpenCodeEngine("opencode/glm-5.2", "/repo")
raw, err := e.config(sys)
if err != nil {
t.Fatalf("config: %v", err)
}
var cfg openCodeConfig
if err := json.Unmarshal(raw, &cfg); err != nil {
t.Fatalf("generated config is not valid JSON: %v\n%s", err, raw)
}
// Agent carries the system prompt verbatim and denies edit+bash.
ag, ok := cfg.Agent[openCodeAgentName]
if !ok {
t.Fatalf("agent %q missing from config", openCodeAgentName)
}
if ag.Prompt != sys {
t.Errorf("agent prompt = %q, want it to round-trip the system prompt", ag.Prompt)
}
// Mutating/network tools are denied at BOTH the agent and global level (defense
// in depth); the read/search tools stay at OpenCode's default.
for _, k := range []string{"edit", "bash", "webfetch", "websearch", "external_directory"} {
if ag.Permission[k] != "deny" {
t.Errorf("agent permission[%q] = %q, want deny", k, ag.Permission[k])
}
if cfg.Permission[k] != "deny" {
t.Errorf("global permission[%q] = %q, want deny", k, cfg.Permission[k])
}
}
// Provider block: correct npm, default baseURL, env-ref apiKey, model in map.
prov, ok := cfg.Provider[openCodeProviderName]
if !ok {
t.Fatalf("provider %q missing from config", openCodeProviderName)
}
if prov.NPM != "@ai-sdk/openai-compatible" {
t.Errorf("provider npm = %q, want @ai-sdk/openai-compatible", prov.NPM)
}
if prov.Options.BaseURL != defaultOpenCodeBaseURL {
t.Errorf("provider baseURL = %q, want %q", prov.Options.BaseURL, defaultOpenCodeBaseURL)
}
if prov.Options.APIKey != "{env:OLLAMA_API_KEY}" {
t.Errorf("provider apiKey = %q, want {env:OLLAMA_API_KEY} (never a literal secret)", prov.Options.APIKey)
}
if _, ok := prov.Models["glm-5.2"]; !ok {
t.Errorf("provider models = %v, want it to contain glm-5.2", prov.Models)
}
// GADFLY_OPENCODE_BASE_URL override reaches the provider.
t.Setenv("GADFLY_OPENCODE_BASE_URL", "http://localhost:11434/v1")
e2 := newOpenCodeEngine("opencode/glm-5.2", "/repo")
raw2, err := e2.config(sys)
if err != nil {
t.Fatalf("config override: %v", err)
}
var cfg2 openCodeConfig
if err := json.Unmarshal(raw2, &cfg2); err != nil {
t.Fatalf("override config invalid JSON: %v", err)
}
if got := cfg2.Provider[openCodeProviderName].Options.BaseURL; got != "http://localhost:11434/v1" {
t.Errorf("override baseURL = %q, want http://localhost:11434/v1", got)
}
}
func TestOpenCodeConfigNoProviderForPassThroughAndBare(t *testing.T) {
t.Setenv("GADFLY_OPENCODE_MODEL", "")
for _, spec := range []string{"opencode", "opencode/anthropic/claude-sonnet-4-6"} {
e := newOpenCodeEngine(spec, "/repo")
raw, err := e.config("sys")
if err != nil {
t.Fatalf("config(%q): %v", spec, err)
}
if strings.Contains(string(raw), "\"provider\"") {
t.Errorf("spec %q: config should omit the provider block, got %s", spec, raw)
}
}
}
func TestOpenCodeEnvFilters(t *testing.T) {
t.Setenv("GITEA_TOKEN", "secret-gitea")
t.Setenv("OLLAMA_API_KEY", "keep-ollama")
t.Setenv("GADFLY_API_KEY", "secret-gadfly")
t.Setenv("GADFLY_FINDINGS_TOKEN", "secret-findings")
t.Setenv("ANTHROPIC_API_KEY", "keep-anthropic") // pass-through provider auth
t.Setenv("OPENAI_API_KEY", "keep-openai") // pass-through provider auth
t.Setenv("CLAUDE_CODE_OAUTH_TOKEN", "secret-claude")
t.Setenv("GADFLY_OPENCODE_MODEL", "keep-knob")
t.Setenv("OPENCODE_CONFIG_CONTENT", "should-not-inherit")
env := openCodeEnv()
has := func(k string) bool {
for _, kv := range env {
if strings.HasPrefix(kv, k+"=") {
return true
}
}
return false
}
// kept: the ollama key + the standard provider keys the opencode/<provider>/<model>
// pass-through form needs + opencode knobs + PATH
for _, k := range []string{"OLLAMA_API_KEY", "ANTHROPIC_API_KEY", "OPENAI_API_KEY", "GADFLY_OPENCODE_MODEL", "PATH"} {
if !has(k) {
t.Errorf("openCodeEnv dropped %s, but it should be kept", k)
}
}
// dropped: gadfly's own secrets + the claude engine's subscription token
// (OpenCode's anthropic provider uses ANTHROPIC_API_KEY, not this OAuth token).
for _, k := range []string{"GITEA_TOKEN", "GADFLY_API_KEY", "GADFLY_FINDINGS_TOKEN", "CLAUDE_CODE_OAUTH_TOKEN"} {
if has(k) {
t.Errorf("openCodeEnv leaked %s into the subprocess env", k)
}
}
// OPENCODE_CONFIG_CONTENT must NOT be inherited — runPass sets it, and a
// duplicate key would be ambiguous.
if has("OPENCODE_CONFIG_CONTENT") {
t.Errorf("openCodeEnv inherited OPENCODE_CONFIG_CONTENT; runPass sets it explicitly")
}
}
// stubOpenCode writes an executable shell stub that prints body and exits code,
// and returns an engine pointed at it.
func stubOpenCode(t *testing.T, body string, code int) *openCodeEngine {
t.Helper()
dir := t.TempDir()
path := dir + "/opencode-stub.sh"
script := "#!/bin/sh\nprintf '%s' " + shSingleQuote(body) + "\nexit " + itoa(code) + "\n"
if err := os.WriteFile(path, []byte(script), 0o755); err != nil {
t.Fatal(err)
}
return &openCodeEngine{bin: path, repoDir: dir}
}
func TestOpenCodeRunPassCleanResult(t *testing.T) {
e := stubOpenCode(t, " REVIEW TEXT ", 0)
out, err := e.runPass(context.Background(), "sys", "task", 0)
if err != nil || out != "REVIEW TEXT" {
t.Fatalf("clean result: got (%q, %v), want (REVIEW TEXT, nil)", out, err)
}
}
func TestOpenCodeRunPassEmptyIsError(t *testing.T) {
e := stubOpenCode(t, " ", 0)
out, err := e.runPass(context.Background(), "sys", "task", 0)
if err == nil {
t.Fatalf("empty output should be an error, got out=%q", out)
}
}
func TestOpenCodeRunPassNonZero(t *testing.T) {
e := stubOpenCode(t, "fatal: provider auth failed", 1)
_, err := e.runPass(context.Background(), "sys", "task", 0)
if err == nil || !strings.Contains(err.Error(), "opencode run failed") {
t.Fatalf("non-zero exit should error with detail, got %v", err)
}
}
// TestOpenCodeRunPassInjectsConfig proves the end-to-end env plumbing: the stub
// echoes OPENCODE_CONFIG_CONTENT back, and the emitted JSON must carry the exact
// system prompt as the gadfly agent's prompt.
func TestOpenCodeRunPassInjectsConfig(t *testing.T) {
dir := t.TempDir()
stub := dir + "/opencode-stub.sh"
script := "#!/bin/sh\nprintf '%s' \"$OPENCODE_CONFIG_CONTENT\"\n"
if err := os.WriteFile(stub, []byte(script), 0o755); err != nil {
t.Fatal(err)
}
e := newOpenCodeEngine("opencode/glm-5.2", dir)
e.bin = stub
sys := "SYSTEM-PROMPT-SENTINEL\nwith a second line"
out, err := e.runPass(context.Background(), sys, "task", 0)
if err != nil {
t.Fatalf("runPass: %v", err)
}
var cfg openCodeConfig
if err := json.Unmarshal([]byte(out), &cfg); err != nil {
t.Fatalf("injected config is not valid JSON: %v\n%s", err, out)
}
if got := cfg.Agent[openCodeAgentName].Prompt; got != sys {
t.Errorf("injected agent prompt = %q, want the system prompt", got)
}
}
+11 -10
View File
@@ -2,6 +2,7 @@ package main
import ( import (
"fmt" "fmt"
"os"
"strings" "strings"
) )
@@ -10,13 +11,6 @@ import (
// than discovering them. // than discovering them.
const defaultRecheckMaxSteps = 16 const defaultRecheckMaxSteps = 16
// defaultRecheckDiffChars caps the diff embedded in the recheck task. It is much
// smaller than the review task's GADFLY_MAX_DIFF_CHARS: the recheck already has
// the draft findings to verify and can pull the exact hunks it needs via the
// paginated get_diff tool (optionally scoped to a path), so re-embedding the
// whole diff on every recheck step is pure burn. Override: GADFLY_RECHECK_DIFF_CHARS.
const defaultRecheckDiffChars = 20000
// recheckSystemPrompt drives the second, adversarial verification pass. The // recheckSystemPrompt drives the second, adversarial verification pass. The
// model is given a DRAFT review and must independently confirm each finding // model is given a DRAFT review and must independently confirm each finding
// against the real code before letting it survive — the antidote to a // against the real code before letting it survive — the antidote to a
@@ -64,7 +58,14 @@ Output rules:
// recheckEnabled reports whether the verification pass should run. On unless // recheckEnabled reports whether the verification pass should run. On unless
// GADFLY_RECHECK is explicitly a falsey value. // GADFLY_RECHECK is explicitly a falsey value.
func recheckEnabled() bool { return envBool("GADFLY_RECHECK", true) } func recheckEnabled() bool {
switch strings.ToLower(strings.TrimSpace(os.Getenv("GADFLY_RECHECK"))) {
case "0", "false", "no", "off":
return false
default:
return true
}
}
// shouldRecheck decides whether to run the verification pass for a given draft. // shouldRecheck decides whether to run the verification pass for a given draft.
// A clean "no material issues" draft has nothing to verify, so it is skipped // A clean "no material issues" draft has nothing to verify, so it is skipped
@@ -83,11 +84,11 @@ func shouldRecheck(draft string) bool {
// scrutinize, with the full diff available via get_diff (and embedded here, // scrutinize, with the full diff available via get_diff (and embedded here,
// truncated, to save a tool call). // truncated, to save a tool call).
func buildRecheckTask(draft, diff string) string { func buildRecheckTask(draft, diff string) string {
maxDiff := envInt("GADFLY_RECHECK_DIFF_CHARS", defaultRecheckDiffChars) maxDiff := envInt("GADFLY_MAX_DIFF_CHARS", defaultMaxDiffChars)
truncNote := "" truncNote := ""
if maxDiff > 0 && len(diff) > maxDiff { if maxDiff > 0 && len(diff) > maxDiff {
diff = diff[:maxDiff] diff = diff[:maxDiff]
truncNote = fmt.Sprintf("\n\n[NOTE: diff truncated to %d chars here; call get_diff (paginated; pass a `path` to scope it to one file) or read the changed files for the rest.]", maxDiff) truncNote = fmt.Sprintf("\n\n[NOTE: diff truncated to %d chars here; read the changed files (or call get_diff, if available) for the full text.]", maxDiff)
} }
var b strings.Builder var b strings.Builder
+8 -29
View File
@@ -49,7 +49,7 @@ func TestRecheckEnabled(t *testing.T) {
} }
func TestBuildRecheckTask(t *testing.T) { func TestBuildRecheckTask(t *testing.T) {
t.Setenv("GADFLY_RECHECK_DIFF_CHARS", "") t.Setenv("GADFLY_MAX_DIFF_CHARS", "")
draft := "VERDICT: Blocking issues found\n- foo.go:1 broken" draft := "VERDICT: Blocking issues found\n- foo.go:1 broken"
out := buildRecheckTask(draft, "diff --git a/x b/x\n+y\n") out := buildRecheckTask(draft, "diff --git a/x b/x\n+y\n")
if !strings.Contains(out, draft) { if !strings.Contains(out, draft) {
@@ -77,46 +77,25 @@ func fakeModel(t *testing.T, reply string) llm.Model {
return m return m
} }
// newTestReviewExecutor builds a reviewExecutor over a fake model + repo for unit func TestRunAgent_ReturnsOutput(t *testing.T) {
// tests: no compaction summarizer and no budget, so it exercises the bare agent
// loop through the executus run kernel without any network call.
func newTestReviewExecutor(t *testing.T, mdl llm.Model, fs *repoFS) *reviewExecutor {
t.Helper()
rex, err := newReviewExecutor(fs, mdl, nil, "mock", nil)
if err != nil {
t.Fatal(err)
}
return rex
}
// testEngine wraps newTestReviewExecutor in a majordomoEngine for the
// runSpecialists-level tests.
func testEngine(t *testing.T, mdl llm.Model, fs *repoFS) *majordomoEngine {
t.Helper()
return &majordomoEngine{rex: newTestReviewExecutor(t, mdl, fs), mdl: mdl}
}
func TestReviewExecutor_ReturnsOutput(t *testing.T) {
fs, err := newRepoFS(t.TempDir(), "diff") fs, err := newRepoFS(t.TempDir(), "diff")
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
mdl := fakeModel(t, " corrected review: No material issues found. ") mdl := fakeModel(t, " corrected review: No material issues found. ")
rex := newTestReviewExecutor(t, mdl, fs) out, err := runAgent(context.Background(), mdl, fs, "sys", "task", 4)
out, err := rex.run(context.Background(), "sys", "task", 4)
if err != nil { if err != nil {
t.Fatalf("run: %v", err) t.Fatalf("runAgent: %v", err)
} }
if out != "corrected review: No material issues found." { if out != "corrected review: No material issues found." {
t.Errorf("run should return trimmed model output, got %q", out) t.Errorf("runAgent should return trimmed model output, got %q", out)
} }
} }
func TestReviewExecutor_EmptyIsError(t *testing.T) { func TestRunAgent_EmptyIsError(t *testing.T) {
fs, _ := newRepoFS(t.TempDir(), "diff") fs, _ := newRepoFS(t.TempDir(), "diff")
mdl := fakeModel(t, " ") mdl := fakeModel(t, " ")
rex := newTestReviewExecutor(t, mdl, fs) if _, err := runAgent(context.Background(), mdl, fs, "sys", "task", 4); err == nil {
if _, err := rex.run(context.Background(), "sys", "task", 4); err == nil { t.Error("runAgent should error on empty model output")
t.Error("run should error on empty model output")
} }
} }
+16 -143
View File
@@ -9,7 +9,6 @@ import (
"regexp" "regexp"
"sort" "sort"
"strings" "strings"
"sync"
llm "gitea.stevedudenhoeffer.com/steve/majordomo/llm" llm "gitea.stevedudenhoeffer.com/steve/majordomo/llm"
) )
@@ -18,13 +17,11 @@ import (
// every tool caps how much it can pull in one call — a runaway read_file or // every tool caps how much it can pull in one call — a runaway read_file or
// grep would blow the window and stall the loop. // grep would blow the window and stall the loop.
const ( const (
maxFileBytes = 64 * 1024 // per read_file call maxFileBytes = 64 * 1024 // per read_file call
maxReadLines = 800 // per read_file call maxReadLines = 800 // per read_file call
maxGrepResults = 200 // per grep call maxGrepResults = 200 // per grep call
maxFindResults = 200 // per find_files call maxFindResults = 200 // per find_files call
maxLineLen = 400 // truncate any single returned line to this maxLineLen = 400 // truncate any single returned line to this
maxGetDiffLines = 800 // per get_diff call (paginated window)
maxGetDiffBytes = 64 * 1024 // per get_diff call
) )
// skipDirs are never descended into by grep / find_files — noise and bulk that // skipDirs are never descended into by grep / find_files — noise and bulk that
@@ -43,12 +40,6 @@ type repoFS struct {
root string // absolute, symlink-resolved repo root root string // absolute, symlink-resolved repo root
diff string // the full PR unified diff (served by get_diff) diff string // the full PR unified diff (served by get_diff)
worker llm.Model // optional cheap model for delegate_investigation; nil = no delegation worker llm.Model // optional cheap model for delegate_investigation; nil = no delegation
// diffLines caches the split diff so paging through get_diff doesn't re-split
// the whole (possibly large) diff on every call. The diff is immutable, so the
// cache is computed once and safe to share across concurrent lenses.
diffOnce sync.Once
diffLines []string
} }
// newRepoFS resolves root to an absolute, symlink-free path. // newRepoFS resolves root to an absolute, symlink-free path.
@@ -117,24 +108,15 @@ func (r *repoFS) fsTools() []llm.Tool {
} }
} }
// allTools is the single source of truth for the reviewer's tool set: the // toolbox builds the reviewer's toolbox: the read-only repo tools, plus the
// read-only repo tools, plus delegate_investigation when a worker model is // delegate_investigation tool when a worker model is configured.
// configured. Both the executus registry (gadflyToolRegistry, the production func (r *repoFS) toolbox() (*llm.Toolbox, error) {
// path) and toolbox() build from this list. box := llm.NewToolbox("gadfly")
func (r *repoFS) allTools() []llm.Tool {
tools := r.fsTools() tools := r.fsTools()
if r.worker != nil { if r.worker != nil {
tools = append(tools, r.delegateTool()) tools = append(tools, r.delegateTool())
} }
return tools for _, t := range tools {
}
// toolbox builds a majordomo toolbox from allTools(). The production review path
// now goes through executus's tool.Registry (see executus.go); this remains for
// the toolbox-level tests (the `call` helper).
func (r *repoFS) toolbox() (*llm.Toolbox, error) {
box := llm.NewToolbox("gadfly")
for _, t := range r.allTools() {
if err := box.Add(t); err != nil { if err := box.Add(t); err != nil {
return nil, fmt.Errorf("add tool %q: %w", t.Name, err) return nil, fmt.Errorf("add tool %q: %w", t.Name, err)
} }
@@ -414,124 +396,15 @@ func (r *repoFS) findFilesTool() llm.Tool {
) )
} }
type getDiffArgs struct {
Path string `json:"path,omitempty" description:"Optional changed-file path (e.g. pkg/foo/bar.go); returns ONLY that file's diff hunks. Omit for the whole diff. Use this on a large PR to pull just the file a finding is about."`
StartLine int `json:"start_line,omitempty" description:"Optional 1-based line to start from within the (whole or path-scoped) diff (default 1)."`
Limit int `json:"limit,omitempty" description:"Optional max number of diff lines to return (default/maximum 800)."`
}
func (r *repoFS) getDiffTool() llm.Tool { func (r *repoFS) getDiffTool() llm.Tool {
return llm.DefineTool[getDiffArgs]( return llm.DefineTool[struct{}](
"get_diff", "get_diff",
"Return the unified diff under review as a numbered, PAGINATED window (like read_file) — not the whole diff at once, so a huge PR can't blow the context window. Pass `path` to fetch just one changed file's hunks, or `start_line`/`limit` to page through. A truncated copy of the diff is also embedded in the task message.", "Return the complete unified diff under review. The diff is also included (possibly truncated) in the task message; call this to get the full, untruncated text.",
func(_ context.Context, args getDiffArgs) (any, error) { func(_ context.Context, _ struct{}) (any, error) {
scope := "the diff" if strings.TrimSpace(r.diff) == "" {
var lines []string return "(empty diff)", nil
if p := strings.TrimSpace(args.Path); p != "" {
lines = diffLinesForPath(r.diff, p)
if len(lines) == 0 {
return fmt.Sprintf("(no diff hunks for path %q; check it against the changed-files list — the path must match a file the diff touches)", p), nil
}
scope = "the diff for " + p
} else {
lines = r.diffAllLines()
if len(lines) == 0 {
return "(empty diff)", nil
}
} }
return windowDiff(lines, scope, args.StartLine, args.Limit), nil return r.diff, nil
}, },
) )
} }
// diffAllLines returns the whole diff split into lines, cached so paging never
// re-splits the (possibly large) diff.
func (r *repoFS) diffAllLines() []string {
r.diffOnce.Do(func() { r.diffLines = splitDiffLines(r.diff) })
return r.diffLines
}
// splitDiffLines splits a unified diff into lines, dropping the single trailing
// empty element a trailing newline produces — otherwise windowDiff would emit a
// blank final line and over-count the total by one.
func splitDiffLines(diff string) []string {
lines := strings.Split(diff, "\n")
if n := len(lines); n > 0 && lines[n-1] == "" {
lines = lines[:n-1]
}
return lines
}
// windowDiff returns a numbered, paginated slice of pre-split diff lines,
// mirroring read_file's caps (maxGetDiffLines / maxGetDiffBytes / maxLineLen) so
// a single get_diff call can never dump a multi-hundred-KB diff into the
// transcript — the amplifier behind the large-PR token burn. The full diff stays
// reachable by paging with start_line, or scoped per file via the path arg.
func windowDiff(lines []string, scope string, start, limit int) string {
total := len(lines)
if start < 1 {
start = 1
}
if limit <= 0 || limit > maxGetDiffLines {
limit = maxGetDiffLines
}
if start > total {
return fmt.Sprintf("(%s has %d lines; nothing at/after line %d)", scope, total, start)
}
var b strings.Builder
emitted := 0
i := start - 1
for ; i < total; i++ {
if emitted >= limit || b.Len() >= maxGetDiffBytes {
break
}
line := lines[i]
if len(line) > maxLineLen {
line = line[:maxLineLen] + "…"
}
fmt.Fprintf(&b, "%d\t%s\n", i+1, line)
emitted++
}
if i < total {
// i (0-based) is the first line NOT emitted; line i was the last shown.
fmt.Fprintf(&b, "... (%s truncated after line %d of %d; call get_diff again with start_line=%d for the rest, or pass a `path` to scope to one file)\n", scope, i, total, i+1)
}
return b.String()
}
// diffLinesForPath returns the unified-diff lines for one changed file: from the
// `diff --git` header that names path through to the next header (or end). The
// header names two path tokens (a/<old> b/<new>); a match is on a WHOLE token,
// so path "foo.go" does not pull in "barfoo.go" — and a trailing "/" scopes to a
// directory (e.g. "pkg/foo/" matches pkg/foo/bar.go).
func diffLinesForPath(diff, path string) []string {
want := strings.TrimPrefix(strings.TrimPrefix(strings.TrimSpace(path), "a/"), "b/")
var out []string
inSection := false
for _, ln := range splitDiffLines(diff) {
if strings.HasPrefix(ln, "diff --git ") {
inSection = diffHeaderNames(ln, want)
}
if inSection {
out = append(out, ln)
}
}
return out
}
// diffHeaderNames reports whether a `diff --git a/X b/Y` header names want as one
// of its (a/b-stripped) path tokens — exact whole-token match, or a directory
// prefix when want ends with "/".
func diffHeaderNames(header, want string) bool {
fields := strings.Fields(header)
if len(fields) < 3 {
return false
}
for _, f := range fields[2:] {
p := strings.TrimPrefix(strings.TrimPrefix(f, "a/"), "b/")
if p == want || (strings.HasSuffix(want, "/") && strings.HasPrefix(p, want)) {
return true
}
}
return false
}
+3 -81
View File
@@ -197,93 +197,15 @@ func TestFindFilesTool(t *testing.T) {
func TestGetDiffTool(t *testing.T) { func TestGetDiffTool(t *testing.T) {
root := buildFixtureRepo(t) root := buildFixtureRepo(t)
const diff = "diff --git a/x b/x\n--- a/x\n+++ b/x\n+added line\n" + const diff = "diff --git a/x b/x\n+added line\n"
"diff --git a/y.go b/y.go\n--- a/y.go\n+++ b/y.go\n+y change\n"
fs, _ := newRepoFS(root, diff) fs, _ := newRepoFS(root, diff)
// Default: the whole diff as a NUMBERED window (paginated), not a raw dump —
// so a huge PR can't be poured into the transcript in one call.
out, err := call(t, fs, "get_diff", map[string]any{}) out, err := call(t, fs, "get_diff", map[string]any{})
if err != nil { if err != nil {
t.Fatalf("get_diff: %v", err) t.Fatalf("get_diff: %v", err)
} }
if !strings.Contains(out, "1\tdiff --git a/x b/x") { if out != diff {
t.Errorf("get_diff should return a numbered window, got:\n%s", out) t.Errorf("get_diff returned %q, want %q", out, diff)
}
if !strings.Contains(out, "+added line") || !strings.Contains(out, "+y change") {
t.Errorf("the full (short) diff window should include every hunk, got:\n%s", out)
}
// path filter: only the named file's hunks come back.
out, err = call(t, fs, "get_diff", map[string]any{"path": "y.go"})
if err != nil {
t.Fatalf("get_diff path: %v", err)
}
if !strings.Contains(out, "y change") {
t.Errorf("get_diff path=y.go should include y's hunk, got:\n%s", out)
}
if strings.Contains(out, "added line") {
t.Errorf("get_diff path=y.go must NOT include x's hunk, got:\n%s", out)
}
// unknown path: a clear note, never an error.
out, err = call(t, fs, "get_diff", map[string]any{"path": "nope.txt"})
if err != nil {
t.Fatalf("get_diff unknown path: %v", err)
}
if !strings.Contains(out, "no diff hunks") {
t.Errorf("get_diff for an unknown path should note no hunks, got:\n%s", out)
}
}
// TestGetDiffTool_Paginates: a diff longer than the per-call line cap is returned
// as a truncated window with a paging hint, and start_line pages past it — the
// mechanism that stops get_diff from dumping a multi-hundred-KB diff at once.
func TestGetDiffTool_Paginates(t *testing.T) {
diff := "diff --git a/big b/big\n" + strings.Repeat("+line\n", maxGetDiffLines+50)
fs, _ := newRepoFS(t.TempDir(), diff)
out, err := call(t, fs, "get_diff", map[string]any{})
if err != nil {
t.Fatalf("get_diff: %v", err)
}
if !strings.Contains(out, "truncated after line") {
t.Error("a diff longer than the per-call cap should be truncated with a paging hint")
}
if strings.Contains(out, "801\t") {
t.Error("the first window must stop at the line cap, not reach line 801")
}
out, err = call(t, fs, "get_diff", map[string]any{"start_line": 805})
if err != nil {
t.Fatalf("get_diff page: %v", err)
}
if !strings.Contains(out, "805\t") {
t.Error("paging with start_line=805 should include line 805")
}
}
// TestDiffLinesForPath_Anchored: get_diff path= matches on a WHOLE path token,
// so "foo.go" never pulls in "barfoo.go" (the unanchored-substring weakness the
// swarm flagged), while a trailing "/" still scopes to a directory.
func TestDiffLinesForPath_Anchored(t *testing.T) {
diff := "diff --git a/foo.go b/foo.go\n+foo change\n" +
"diff --git a/barfoo.go b/barfoo.go\n+barfoo change\n" +
"diff --git a/pkg/x.go b/pkg/x.go\n+x change\n"
joined := strings.Join(diffLinesForPath(diff, "foo.go"), "\n")
if !strings.Contains(joined, "foo change") {
t.Errorf("foo.go should match its own hunk:\n%s", joined)
}
if strings.Contains(joined, "barfoo change") {
t.Errorf("foo.go must NOT match barfoo.go (unanchored substring regression):\n%s", joined)
}
if !strings.Contains(strings.Join(diffLinesForPath(diff, "pkg/"), "\n"), "x change") {
t.Error("a trailing-slash path should scope to the directory (pkg/ -> pkg/x.go)")
}
if len(diffLinesForPath(diff, "nope.go")) != 0 {
t.Error("an unknown path should yield no lines")
} }
} }
+51 -20
View File
@@ -37,11 +37,10 @@ func lastUserText(req llm.Request) string {
return req.Messages[len(req.Messages)-1].Text() return req.Messages[len(req.Messages)-1].Text()
} }
// TestReviewExecutor_WrapUpNudgeProducesAnswer: a model that keeps calling tools // TestRunAgent_WrapUpNudgeProducesAnswer: a model that keeps calling tools until
// until it is nudged to wrap up should still finish inside its budget — the steer // it is nudged to wrap up should still finish inside its budget — the steer
// message (delivered by the executus wrap-up critic a few steps before the cap) // message arrives a few steps before the cap and the model writes its answer.
// arrives and the model writes its answer. func TestRunAgent_WrapUpNudgeProducesAnswer(t *testing.T) {
func TestReviewExecutor_WrapUpNudgeProducesAnswer(t *testing.T) {
t.Setenv("GADFLY_WRAPUP_RESERVE", "4") t.Setenv("GADFLY_WRAPUP_RESERVE", "4")
final := "VERDICT: No material issues found." final := "VERDICT: No material issues found."
@@ -61,10 +60,9 @@ func TestReviewExecutor_WrapUpNudgeProducesAnswer(t *testing.T) {
} }
fs, _ := newRepoFS(t.TempDir(), "diff --git a/x b/x\n+y\n") fs, _ := newRepoFS(t.TempDir(), "diff --git a/x b/x\n+y\n")
rex := newTestReviewExecutor(t, mdl, fs) out, err := runAgent(context.Background(), mdl, fs, "sys", "task", 12)
out, err := rex.run(context.Background(), "sys", "task", 12)
if err != nil { if err != nil {
t.Fatalf("run should succeed via wrap-up nudge, got error: %v", err) t.Fatalf("runAgent should succeed via wrap-up nudge, got error: %v", err)
} }
if out != final { if out != final {
t.Errorf("expected final review %q, got %q", final, out) t.Errorf("expected final review %q, got %q", final, out)
@@ -74,19 +72,24 @@ func TestReviewExecutor_WrapUpNudgeProducesAnswer(t *testing.T) {
} }
} }
// TestReviewExecutor_ExhaustionWithoutAnswerIsError: a model that ignores the // TestRunAgent_FinalizationFallback: a model that ignores the wrap-up nudge and
// wrap-up nudge and spins on tools until the step cap produces no final answer. // spins on tools until the cap should NOT hard-fail — the tool-free finalization
// The transcript-based forced-finalization fallback was removed in the executus // pass forces a final answer out of the transcript.
// re-platform (run.Result does not expose the loop transcript), so the pass now func TestRunAgent_FinalizationFallback(t *testing.T) {
// surfaces an error — which reviewWithSpecialist renders as an advisory "reviewer
// failed to complete" notice rather than a phantom success.
func TestReviewExecutor_ExhaustionWithoutAnswerIsError(t *testing.T) {
t.Setenv("GADFLY_WRAPUP_RESERVE", "2") t.Setenv("GADFLY_WRAPUP_RESERVE", "2")
final := "VERDICT: Minor issues\n- something"
forcedCalled := false
n := 0 n := 0
p := fake.New("fake", fake.WithDefault(func(_ string, _ llm.Request) fake.Step { p := fake.New("fake", fake.WithDefault(func(_ string, req llm.Request) fake.Step {
// Only the tool-free finalization pass forbids tools — reply there.
if req.ToolChoice == "none" {
forcedCalled = true
return fake.Reply(final)
}
// Otherwise keep spinning, ignoring the wrap-up nudge entirely.
n++ n++
return spinToolCall(n) // spin forever, ignoring the wrap-up nudge return spinToolCall(n)
})) }))
mdl, err := p.Model("mock") mdl, err := p.Model("mock")
if err != nil { if err != nil {
@@ -94,9 +97,37 @@ func TestReviewExecutor_ExhaustionWithoutAnswerIsError(t *testing.T) {
} }
fs, _ := newRepoFS(t.TempDir(), "diff --git a/x b/x\n+y\n") fs, _ := newRepoFS(t.TempDir(), "diff --git a/x b/x\n+y\n")
rex := newTestReviewExecutor(t, mdl, fs) out, err := runAgent(context.Background(), mdl, fs, "sys", "task", 6)
if _, err := rex.run(context.Background(), "sys", "task", 6); err == nil { if err != nil {
t.Error("run should error when the model exhausts its steps without an answer") t.Fatalf("runAgent should recover via finalization fallback, got error: %v", err)
}
if !forcedCalled {
t.Error("finalization fallback was never invoked")
}
if out != final {
t.Errorf("expected forced final answer %q, got %q", final, out)
}
}
// TestRunAgent_FallbackStillEmptyIsError: if even the tool-free finalization
// yields nothing, runAgent surfaces an error rather than a phantom success.
func TestRunAgent_FallbackStillEmptyIsError(t *testing.T) {
n := 0
p := fake.New("fake", fake.WithDefault(func(_ string, req llm.Request) fake.Step {
if req.ToolChoice == "none" {
return fake.Reply(" ") // finalization produces only whitespace
}
n++
return spinToolCall(n)
}))
mdl, err := p.Model("mock")
if err != nil {
t.Fatal(err)
}
fs, _ := newRepoFS(t.TempDir(), "diff --git a/x b/x\n+y\n")
if _, err := runAgent(context.Background(), mdl, fs, "sys", "task", 4); err == nil {
t.Error("runAgent should error when the finalization fallback also yields no output")
} }
} }
+1 -78
View File
@@ -39,10 +39,6 @@
# CLAUDE_CODE_OAUTH_TOKEN auth for the claude-code engine (GADFLY_MODELS entry # CLAUDE_CODE_OAUTH_TOKEN auth for the claude-code engine (GADFLY_MODELS entry
# "claude-code"/"claude-code/<model>"); Pro/Max subscription # "claude-code"/"claude-code/<model>"); Pro/Max subscription
# token from `claude setup-token`. Else ANTHROPIC_API_KEY. # token from `claude setup-token`. Else ANTHROPIC_API_KEY.
# OLLAMA_CLOUD_API_KEY also feeds the opencode engine (GADFLY_MODELS entry
# "opencode/<model>"): the bundled `opencode` CLI drives
# that ollama-cloud model, for benchmarking the two
# harnesses on the same model. Tune via GADFLY_OPENCODE_*.
# GADFLY_TRIGGER_PHRASE comment phrase that triggers a re-review (default "@gadfly review") # GADFLY_TRIGGER_PHRASE comment phrase that triggers a re-review (default "@gadfly review")
# GADFLY_ALLOWED_USERS comma-separated usernames allowed to comment-trigger; # GADFLY_ALLOWED_USERS comma-separated usernames allowed to comment-trigger;
# empty => fall back to "is a repo collaborator" # empty => fall back to "is a repo collaborator"
@@ -207,36 +203,6 @@ export GADFLY_FINDINGS_TOKEN="${GADFLY_FINDINGS_TOKEN:-}"
MODELS="${GADFLY_MODELS:-${OLLAMA_REVIEW_MODELS:-$DEFAULT_MODELS}}" MODELS="${GADFLY_MODELS:-${OLLAMA_REVIEW_MODELS:-$DEFAULT_MODELS}}"
DEFAULT_CONC="${GADFLY_CONCURRENCY:-1}" DEFAULT_CONC="${GADFLY_CONCURRENCY:-1}"
# --- huge-PR downshift ------------------------------------------------------
# A very large diff is what burns the model budget: every review step re-sends
# it, multiplied across models × lenses × passes × steps (this is what nuked a
# whole Ollama Cloud block on one giant PR). entrypoint is the only process that
# spans the whole fleet, so the fleet-wide size decision lives here: size the PR
# diff ONCE, and above GADFLY_HUGE_DIFF_BYTES collapse to a single cheap model +
# a focused lens subset, fewer steps, no recheck, and a smaller embedded diff.
# A finished shallow review beats a budget-nuking one. All knobs override; set
# GADFLY_HUGE_DIFF_BYTES=0 to disable. Small PRs are never touched.
HUGE_PR=0
HUGE_DIFF_BYTES="${GADFLY_HUGE_DIFF_BYTES:-600000}"
if [ "$HUGE_DIFF_BYTES" -gt 0 ] 2>/dev/null; then
PR_DIFF_BYTES="$(API "${GITEA_API}/pulls/${PR}.diff" 2>/dev/null | wc -c | tr -d '[:space:]')"
[ -z "$PR_DIFF_BYTES" ] && PR_DIFF_BYTES=0
if [ "$PR_DIFF_BYTES" -gt "$HUGE_DIFF_BYTES" ] 2>/dev/null; then
HUGE_PR=1
log "huge PR: diff ${PR_DIFF_BYTES}B > ${HUGE_DIFF_BYTES}B — downshifting the fleet (advisory)"
MODELS="${GADFLY_HUGE_DIFF_MODELS:-${MODELS%%,*}}" # first model only by default
export GADFLY_SPECIALISTS="${GADFLY_HUGE_DIFF_SPECIALISTS:-security,correctness,error-handling}"
export GADFLY_MAX_STEPS="${GADFLY_HUGE_DIFF_MAX_STEPS:-12}"
export GADFLY_RECHECK_MAX_STEPS="${GADFLY_HUGE_DIFF_RECHECK_MAX_STEPS:-8}"
export GADFLY_RECHECK="${GADFLY_HUGE_DIFF_RECHECK:-0}" # skip recheck on huge PRs
# The Go-visible name directly (run.sh prefers GADFLY_MAX_DIFF_CHARS over its
# own MAX_DIFF_CHARS), so the cap is honored without relying on run.sh's alias.
export GADFLY_MAX_DIFF_CHARS="${GADFLY_HUGE_DIFF_MAX_DIFF_CHARS:-20000}"
# Surfaced on each posted comment so the shallower review is self-explaining.
export GADFLY_NOTICE="⚠️ Large PR (${PR_DIFF_BYTES} bytes): Gadfly downshifted to a focused, single-model review to stay within budget — coverage is intentionally shallower. Consider splitting the PR for a deeper review."
fi
fi
provider_of() { case "$1" in */*) echo "${1%%/*}";; *) echo "${GADFLY_PROVIDER:-ollama-cloud}";; esac; } provider_of() { case "$1" in */*) echo "${1%%/*}";; *) echo "${GADFLY_PROVIDER:-ollama-cloud}";; esac; }
# Per-model status file path for the live board. The model id can contain '/' # Per-model status file path for the live board. The model id can contain '/'
@@ -351,32 +317,6 @@ if [ "${GADFLY_STATUS_BOARD:-1}" != "0" ]; then
log "status board started (pid ${BOARD_PID})" log "status board started (pid ${BOARD_PID})"
fi fi
# --- swarm-wide hard backstop ----------------------------------------------
# A wall-clock ceiling across the WHOLE fleet, so a pathological PR can never
# drain the usage block however the models behave. entrypoint is the only
# process spanning every model, so a single "never exceed X" guard lives here.
# On expiry it stops the review subtrees (the binary + run.sh); whatever partial
# findings were gathered are still posted and the job never fails (advisory).
# GADFLY_PR_BUDGET_SECS=0 (default) disables it.
KILLER_PID=""
rm -f "${WORKDIR}/.budget_killed" "${WORKDIR}/.disarmed" 2>/dev/null || true
if [ "${GADFLY_PR_BUDGET_SECS:-0}" -gt 0 ] 2>/dev/null; then
(
sleep "${GADFLY_PR_BUDGET_SECS}"
log "PR wall-clock budget (${GADFLY_PR_BUDGET_SECS}s) reached — stopping the review fleet (advisory; partial findings still posted)"
: > "${WORKDIR}/.budget_killed"
pkill -TERM -f '/usr/local/bin/gadfly' 2>/dev/null || true
pkill -TERM -f "${SCRIPTS_DIR}/run.sh" 2>/dev/null || true
sleep 5
# Guard the delayed SIGKILL on the disarm marker: once the lanes finished and
# the watchdog was disarmed, the consolidation gadfly pass runs next, and a
# name-based KILL here must NOT catch it.
[ -f "${WORKDIR}/.disarmed" ] || pkill -KILL -f '/usr/local/bin/gadfly' 2>/dev/null || true
) &
KILLER_PID=$!
log "PR budget watchdog armed (${GADFLY_PR_BUDGET_SECS}s, pid ${KILLER_PID})"
fi
log "providers: ${PROVIDERS:-none}" log "providers: ${PROVIDERS:-none}"
# Each provider lane runs in parallel; cap is enforced within each lane. Track # Each provider lane runs in parallel; cap is enforced within each lane. Track
# the lane PIDs so we wait ONLY for the review work — not the status board, # the lane PIDs so we wait ONLY for the review work — not the status board,
@@ -388,21 +328,6 @@ for p in $PROVIDERS; do
done done
[ "${#LANE_PIDS[@]}" -gt 0 ] && wait "${LANE_PIDS[@]}" [ "${#LANE_PIDS[@]}" -gt 0 ] && wait "${LANE_PIDS[@]}"
# Reviews finished (or the watchdog killed them): disarm the watchdog so its
# delayed SIGKILL can't catch the consolidation pass that runs next. Drop the
# disarm marker FIRST so even a racing watchdog that already reached its KILL line
# skips it (the kill below also tears the watchdog subshell down during its sleep).
if [ -n "$KILLER_PID" ]; then
: > "${WORKDIR}/.disarmed"
kill "$KILLER_PID" 2>/dev/null || true
fi
# If the backstop fired, note it on the consensus comment (per-model comments
# were already posted during the run; a killed model surfaces as a failed lane).
if [ -f "${WORKDIR}/.budget_killed" ]; then
export GADFLY_NOTICE="${GADFLY_NOTICE:+${GADFLY_NOTICE} }⏱️ This review was stopped early by the per-PR time budget (GADFLY_PR_BUDGET_SECS); findings are partial."
fi
# Reviews are done: signal the board to render the final state once and exit. # Reviews are done: signal the board to render the final state once and exit.
if [ -n "$BOARD_PID" ]; then if [ -n "$BOARD_PID" ]; then
touch "${STATUS_DIR}/.done" 2>/dev/null || true touch "${STATUS_DIR}/.done" 2>/dev/null || true
@@ -426,9 +351,7 @@ if [ "$CONSOLIDATE" = "1" ]; then
CONSENSUS="$(GADFLY_CONSOLIDATE_DIR="$FINDINGS_DIR" GADFLY_DIFF_FILE="$DIFF_FILE" \ CONSENSUS="$(GADFLY_CONSOLIDATE_DIR="$FINDINGS_DIR" GADFLY_DIFF_FILE="$DIFF_FILE" \
/usr/local/bin/gadfly 2>"${WORKDIR}/consolidate.err" || true)" /usr/local/bin/gadfly 2>"${WORKDIR}/consolidate.err" || true)"
if [ -n "$CONSENSUS" ]; then if [ -n "$CONSENSUS" ]; then
NOTICE_BLOCK="" BODY="$(printf '%s\n\n<sub>Automated adversarial review by Gadfly — consensus across the model swarm. Advisory only — does not block merge.</sub>' "$CONSENSUS")"
[ -n "${GADFLY_NOTICE:-}" ] && NOTICE_BLOCK="> ${GADFLY_NOTICE}"$'\n\n'
BODY="$(printf '%s%s\n\n<sub>Automated adversarial review by Gadfly — consensus across the model swarm. Advisory only — does not block merge.</sub>' "$NOTICE_BLOCK" "$CONSENSUS")"
upsert_comment_body "<!-- gadfly-consensus -->" "$BODY" upsert_comment_body "<!-- gadfly-consensus -->" "$BODY"
log "consensus comment posted" log "consensus comment posted"
else else
-1
View File
@@ -12,7 +12,6 @@ set the secrets/vars it references. Gadfly is advisory only — it never blocks
| [`openai-compatible.yml`](openai-compatible.yml) | any **OpenAI-compatible** endpoint (local Ollama `/v1`, gateway, vLLM, OpenRouter…) | `GADFLY_BASE_URL` (+ a key for most gateways) | | [`openai-compatible.yml`](openai-compatible.yml) | any **OpenAI-compatible** endpoint (local Ollama `/v1`, gateway, vLLM, OpenRouter…) | `GADFLY_BASE_URL` (+ a key for most gateways) |
| [`endpoint-aliases.yml`](endpoint-aliases.yml) | **several named backends** at once (one comment each) | repo vars `GADFLY_ENDPOINT_<NAME>` | | [`endpoint-aliases.yml`](endpoint-aliases.yml) | **several named backends** at once (one comment each) | repo vars `GADFLY_ENDPOINT_<NAME>` |
| [`claude-code.yml`](claude-code.yml) | the bundled **Claude Code CLI** engine (`claude-code/<model>`) | secret `CLAUDE_CODE_OAUTH_TOKEN` (or `ANTHROPIC_API_KEY`) | | [`claude-code.yml`](claude-code.yml) | the bundled **Claude Code CLI** engine (`claude-code/<model>`) | secret `CLAUDE_CODE_OAUTH_TOKEN` (or `ANTHROPIC_API_KEY`) |
| [`opencode.yml`](opencode.yml) | the bundled **OpenCode CLI** engine (`opencode/<model>`) driving an ollama-cloud model — benchmark it against the majordomo loop on the same model | secret `OLLAMA_CLOUD_API_KEY` |
| [`.gadfly.yml`](.gadfly.yml) | **per-repo specialist config** (not a workflow — goes at your repo root) | — | | [`.gadfly.yml`](.gadfly.yml) | **per-repo specialist config** (not a workflow — goes at your repo root) | — |
Common to all: Common to all:
-76
View File
@@ -1,76 +0,0 @@
# Gadfly reviewing via the OpenCode CLI engine.
# Copy to .gitea/workflows/adversarial-review.yml in your repo.
#
# Instead of gadfly's own majordomo loop, each lens shells out to the bundled
# `opencode` CLI (opencode.ai) inside the checked-out repo — it uses its own read
# tools to verify findings — while driving an ollama-cloud model. Gadfly then runs
# its usual verdict + recheck + consolidate pipeline.
#
# Why: benchmark gadfly's boutique harness against a freely-available one ON THE
# SAME MODEL. List both entries to get one comment section each and compare:
# GADFLY_MODELS: "ollama-cloud/glm-5.2,opencode/glm-5.2"
#
# Auth: reuses the OLLAMA_CLOUD_API_KEY secret (same as the ollama-cloud path) —
# no OpenCode-specific credential is needed for the ollama-cloud provider.
#
# Heads-up: this engine is newly wired and lightly tested — read the README's
# "OpenCode engine" note before relying on it.
name: Adversarial Review (Gadfly)
on:
pull_request:
types: [opened, reopened, ready_for_review]
issue_comment:
types: [created]
workflow_dispatch:
inputs:
pr_number: { description: "PR number to review", required: true }
permissions:
contents: read
issues: write
pull-requests: write
concurrency:
group: gadfly-${{ github.event.issue.number || github.event.pull_request.number || github.event.inputs.pr_number }}
cancel-in-progress: true
jobs:
review:
# Security: only trusted users may trigger a secret-bearing run via a PR
# comment. Replace the username(s) below with your maintainers — keep them in
# sync with GADFLY_ALLOWED_USERS (the in-container belt-and-suspenders check).
if: >-
github.event_name != 'issue_comment'
|| github.actor == 'your-username'
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: docker://gitea.stevedudenhoeffer.com/steve/gadfly:latest
env:
GITEA_API: ${{ github.server_url }}/api/v1/repos/${{ github.repository }}
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
# --- OpenCode engine ---
# Reuses the ollama-cloud key; mapped to OLLAMA_API_KEY in-container and
# referenced by the generated provider as {env:OLLAMA_API_KEY}.
OLLAMA_CLOUD_API_KEY: ${{ secrets.OLLAMA_CLOUD_API_KEY }}
# "opencode/<model>" serves that model via ollama-cloud through OpenCode.
# Model ids are verbatim (colons preserved). List an "ollama-cloud/<model>"
# entry too to benchmark the two harnesses on the same model.
GADFLY_MODELS: "opencode/glm-5.2"
# Optional CLI tuning:
# GADFLY_OPENCODE_BASE_URL: "https://ollama.com/v1" # or a local Ollama /v1
# GADFLY_OPENCODE_MODEL: "glm-5.2" # overrides the spec suffix
# GADFLY_OPENCODE_EXTRA_ARGS: "--variant reasoning" # whitespace-split
# Escape hatch: "opencode/<provider>/<model>" passes straight to OpenCode's
# own provider registry/auth (e.g. opencode/anthropic/claude-sonnet-4-6).
GADFLY_ALLOWED_USERS: "your-username"
# --- event context (leave as-is) ---
EVENT_NAME: ${{ github.event_name }}
PR: ${{ github.event.pull_request.number || github.event.issue.number || github.event.inputs.pr_number }}
PR_BRANCH: ${{ github.head_ref }}
IS_DRAFT: ${{ github.event.pull_request.draft }}
COMMENT_BODY: ${{ github.event.comment.body }}
COMMENT_ID: ${{ github.event.comment.id }}
ACTOR: ${{ github.actor }}
-2
View File
@@ -3,7 +3,6 @@ module gitea.stevedudenhoeffer.com/steve/gadfly
go 1.26.2 go 1.26.2
require ( require (
gitea.stevedudenhoeffer.com/steve/executus v0.1.4
gitea.stevedudenhoeffer.com/steve/majordomo v0.0.0-20260627225659-aa25b2c33462 gitea.stevedudenhoeffer.com/steve/majordomo v0.0.0-20260627225659-aa25b2c33462
gopkg.in/yaml.v3 v3.0.1 gopkg.in/yaml.v3 v3.0.1
) )
@@ -18,7 +17,6 @@ require (
github.com/go-logr/stdr v1.2.2 // indirect github.com/go-logr/stdr v1.2.2 // indirect
github.com/google/go-cmp v0.7.0 // indirect github.com/google/go-cmp v0.7.0 // indirect
github.com/google/s2a-go v0.1.9 // indirect github.com/google/s2a-go v0.1.9 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.17 // indirect github.com/googleapis/enterprise-certificate-proxy v0.3.17 // indirect
github.com/googleapis/gax-go/v2 v2.22.0 // indirect github.com/googleapis/gax-go/v2 v2.22.0 // indirect
github.com/gorilla/websocket v1.5.3 // indirect github.com/gorilla/websocket v1.5.3 // indirect
-2
View File
@@ -4,8 +4,6 @@ cloud.google.com/go/auth v0.20.0 h1:kXTssoVb4azsVDoUiF8KvxAqrsQcQtB53DcSgta74CA=
cloud.google.com/go/auth v0.20.0/go.mod h1:942/yi/itH1SsmpyrbnTMDgGfdy2BUqIKyd0cyYLc5Q= cloud.google.com/go/auth v0.20.0/go.mod h1:942/yi/itH1SsmpyrbnTMDgGfdy2BUqIKyd0cyYLc5Q=
cloud.google.com/go/compute/metadata v0.9.0 h1:pDUj4QMoPejqq20dK0Pg2N4yG9zIkYGdBtwLoEkH9Zs= cloud.google.com/go/compute/metadata v0.9.0 h1:pDUj4QMoPejqq20dK0Pg2N4yG9zIkYGdBtwLoEkH9Zs=
cloud.google.com/go/compute/metadata v0.9.0/go.mod h1:E0bWwX5wTnLPedCKqk3pJmVgCBSM6qQI1yTBdEb3C10= cloud.google.com/go/compute/metadata v0.9.0/go.mod h1:E0bWwX5wTnLPedCKqk3pJmVgCBSM6qQI1yTBdEb3C10=
gitea.stevedudenhoeffer.com/steve/executus v0.1.4 h1:4F99uCV3OVaE9ITFp0FjPiYxLUQO+WpE+wU2HCnpXNM=
gitea.stevedudenhoeffer.com/steve/executus v0.1.4/go.mod h1:WQP/lH+meU06OSNF0TQO/wQLcJCrMwpi0EMj5vSpVtk=
gitea.stevedudenhoeffer.com/steve/majordomo v0.0.0-20260627225659-aa25b2c33462 h1:1crjE1YkWHLZ91tUDOxN/Y5cuOnJ56e0U9UADoFfEPY= gitea.stevedudenhoeffer.com/steve/majordomo v0.0.0-20260627225659-aa25b2c33462 h1:1crjE1YkWHLZ91tUDOxN/Y5cuOnJ56e0U9UADoFfEPY=
gitea.stevedudenhoeffer.com/steve/majordomo v0.0.0-20260627225659-aa25b2c33462/go.mod h1:UZLveG17SmENt4sne2RSLIbioix30RZbRIQUzBAnOyY= gitea.stevedudenhoeffer.com/steve/majordomo v0.0.0-20260627225659-aa25b2c33462/go.mod h1:UZLveG17SmENt4sne2RSLIbioix30RZbRIQUzBAnOyY=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs= github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
+3 -14
View File
@@ -29,13 +29,6 @@
# tuning are read straight from the inherited environment — same as the other # tuning are read straight from the inherited environment — same as the other
# provider keys (OPENAI_API_KEY, …) — so no extra wiring is needed here. # provider keys (OPENAI_API_KEY, …) — so no extra wiring is needed here.
# #
# opencode engine: when MODEL is "opencode" or "opencode/<model>" the binary
# shells out to the bundled `opencode` CLI, driving an ollama-cloud model (for
# benchmarking against the majordomo path on the same model). Its auth reuses
# OLLAMA_CLOUD_API_KEY (mapped to OLLAMA_API_KEY below, same as the ollama-cloud
# path) and GADFLY_OPENCODE_* tuning is read from the inherited environment — so
# no extra wiring is needed here either.
#
# Optional: # Optional:
# MAX_DIFF_CHARS diff truncation cap for the prompt (default 60000) # MAX_DIFF_CHARS diff truncation cap for the prompt (default 60000)
# GADFLY_STATUS_FILE per-model JSON path for the live status board (set by # GADFLY_STATUS_FILE per-model JSON path for the live status board (set by
@@ -184,7 +177,7 @@ case "$PROVIDER" in
GADFLY_SYSTEM_FILE="${SCRIPT_DIR}/system-prompt.txt" \ GADFLY_SYSTEM_FILE="${SCRIPT_DIR}/system-prompt.txt" \
GADFLY_TITLE="$TITLE" \ GADFLY_TITLE="$TITLE" \
GADFLY_BODY="$BODY" \ GADFLY_BODY="$BODY" \
GADFLY_MAX_DIFF_CHARS="${GADFLY_MAX_DIFF_CHARS:-$MAX_DIFF_CHARS}" \ GADFLY_MAX_DIFF_CHARS="$MAX_DIFF_CHARS" \
GADFLY_STATUS_FILE="${GADFLY_STATUS_FILE:-}" \ GADFLY_STATUS_FILE="${GADFLY_STATUS_FILE:-}" \
GADFLY_FINDINGS_OUT="${GADFLY_FINDINGS_OUT:-}" \ GADFLY_FINDINGS_OUT="${GADFLY_FINDINGS_OUT:-}" \
"$BIN" 2>"$ERR_FILE" "$BIN" 2>"$ERR_FILE"
@@ -237,12 +230,8 @@ fi
if [ "$CONSOLIDATE" = "1" ]; then if [ "$CONSOLIDATE" = "1" ]; then
say "done in ${DUR} (consolidated; no per-model comment)" say "done in ${DUR} (consolidated; no per-model comment)"
else else
# An optional one-line notice (e.g. entrypoint's huge-PR downshift advisory), COMMENT="$(printf '%s\n### 🪰 Gadfly review — `%s` (%s)\n\n%s\n\n<sub>Automated adversarial review by Gadfly. Advisory only — does not block merge. · ⏱️ reviewed in %s</sub>' \
# shown under the header so a shallower review is self-explaining. "$MARKER" "$MODEL" "$MODEL_PROVIDER" "$REVIEW" "$DUR")"
NOTICE_BLOCK=""
[ -n "${GADFLY_NOTICE:-}" ] && NOTICE_BLOCK="> ${GADFLY_NOTICE}"$'\n\n'
COMMENT="$(printf '%s\n### 🪰 Gadfly review — `%s` (%s)\n\n%s%s\n\n<sub>Automated adversarial review by Gadfly. Advisory only — does not block merge. · ⏱️ reviewed in %s</sub>' \
"$MARKER" "$MODEL" "$MODEL_PROVIDER" "$NOTICE_BLOCK" "$REVIEW" "$DUR")"
upsert_comment "$COMMENT" upsert_comment "$COMMENT"
say "done in ${DUR}" say "done in ${DUR}"
fi fi
+1 -3
View File
@@ -16,9 +16,7 @@ state. USE THEM to verify before you report. Do not review the diff in isolation
- list_dir([path]) — list a directory. - list_dir([path]) — list a directory.
- grep(pattern[, path, max_results]) — RE2 regex search across the repo. - grep(pattern[, path, max_results]) — RE2 regex search across the repo.
- find_files(name[, max_results]) — locate a file by path substring. - find_files(name[, max_results]) — locate a file by path substring.
- get_diff([path, start_line, limit]) — the unified diff as a paginated, numbered window; - get_diff() — the full unified diff (the task message may truncate it).
pass `path` to fetch just one changed file's hunks (do this on a big PR instead of pulling
the whole diff at once).
Mandatory verification discipline — this is the whole point of giving you tools: Mandatory verification discipline — this is the whole point of giving you tools:
- Before claiming a missing/duplicate import, an undefined symbol, a wrong signature, - Before claiming a missing/duplicate import, an undefined symbol, a wrong signature,