steveandClaude Opus 5
0f40b21d79
majordomo now ships qwen and kimi as built-ins that ARE the openai client at their own base URL, so "qwen/qwen3.8-max" works as a GADFLY_MODELS entry once the key reaches the container. This wires up the parts that key has to pass through. Two provider switches had to learn the names, not one. resolveModel's GADFLY_BASE_URL override was the obvious one; endpointProvider's GADFLY_ENDPOINT_* parser is its sibling, and I fixed the first and missed the second on the first pass — a config that resolves one way and errors the other for no reason a user could guess. TestOpenAICompatProvidersResolveOnBothPaths now asserts both from one table so the pair fails together; break-checked in both directions. QWEN_API_KEY (and KIMI_API_KEY) are declared as workflow_call secrets and forwarded to the container, with gadfly's own stub forwarding QWEN_API_KEY so a qwen entry can join the default swarm by editing GADFLY_DEFAULT_MODELS alone — no workflow edit, no re-release. The run.sh credential pre-flight is now a provider→variable table instead of an ollama-cloud special case. Without it a forgotten key surfaces as five identical per-lens agent failures naming no variable, and the operator reads a stack trace to find out which secret they missed. Google stays out of the table on purpose: it accepts either GOOGLE_API_KEY or GEMINI_API_KEY, and a one-var entry would wrongly skip a correctly-configured run. Verified across 17 provider x key-state combinations, including that a wrong-provider key never satisfies qwen (majordomo refuses cross-provider fallback) and that unkeyed providers are never blocked. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
265 lines
13 KiB
Bash
265 lines
13 KiB
Bash
#!/usr/bin/env bash
|
|
# Adversarial PR review runner.
|
|
#
|
|
# Fetches a PR's unified diff + metadata from Gitea, asks ONE model to review it
|
|
# adversarially, then upserts the result as a single labeled PR comment (so
|
|
# re-runs on new commits update the comment in place instead of stacking dupes).
|
|
#
|
|
# The ollama lane is AGENTIC: it runs the cmd/gadfly Go binary, which drives a
|
|
# tool-using agent (majordomo + Ollama Cloud) over the PR's checked-out repo so
|
|
# the model can read_file/grep/etc. to VERIFY findings instead of guessing from
|
|
# the diff alone. The antigravity lane stays a one-shot `agy` call (agy has its
|
|
# own file tools).
|
|
#
|
|
# Required env:
|
|
# GITEA_API e.g. https://gitea.stevedudenhoeffer.com/api/v1/repos/steve/mort
|
|
# GITEA_TOKEN token with repo write access (posts the comment)
|
|
# PR pull request index/number
|
|
# PROVIDER "ollama" | "antigravity"
|
|
# MODEL model id (e.g. qwen3-coder:480b-cloud, gemini-3-pro)
|
|
#
|
|
# Provider-specific env:
|
|
# ollama: OLLAMA_CLOUD_API_KEY, GADFLY_BIN (path to the built reviewer),
|
|
# GADFLY_REPO_DIR (checked-out repo; default: this script's repo)
|
|
# antigravity: `agy` on PATH with credentials already seeded (~/.gemini)
|
|
#
|
|
# claude-code engine: when MODEL is "claude-code" or "claude-code/<model>" the
|
|
# binary shells out to the bundled `claude` CLI instead of a majordomo model.
|
|
# Its auth (CLAUDE_CODE_OAUTH_TOKEN, else ANTHROPIC_API_KEY) and GADFLY_CLAUDE_*
|
|
# tuning are read straight from the inherited environment — same as the other
|
|
# provider keys (OPENAI_API_KEY, …) — so no extra wiring is needed here.
|
|
#
|
|
# opencode engine: when MODEL is "opencode" or "opencode/<model>" the binary
|
|
# shells out to the bundled `opencode` CLI, driving an ollama-cloud model (for
|
|
# benchmarking against the majordomo path on the same model). Its auth reuses
|
|
# OLLAMA_CLOUD_API_KEY (mapped to OLLAMA_API_KEY below, same as the ollama-cloud
|
|
# path) and GADFLY_OPENCODE_* tuning is read from the inherited environment — so
|
|
# no extra wiring is needed here either.
|
|
#
|
|
# Optional:
|
|
# MAX_DIFF_CHARS diff truncation cap for the prompt (default 60000)
|
|
# GADFLY_STATUS_FILE per-model JSON path for the live status board (set by
|
|
# entrypoint.sh; empty/unset disables status publishing)
|
|
#
|
|
# This script is advisory: it never fails the job for review content. It exits
|
|
# non-zero only on a usage/configuration error.
|
|
set -uo pipefail
|
|
|
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
MAX_DIFF_CHARS="${MAX_DIFF_CHARS:-60000}"
|
|
|
|
: "${GITEA_API:?GITEA_API required}"
|
|
: "${GITEA_TOKEN:?GITEA_TOKEN required}"
|
|
: "${PR:?PR required}"
|
|
: "${PROVIDER:?PROVIDER required}"
|
|
: "${MODEL:?MODEL required}"
|
|
|
|
MARKER="<!-- gadfly-review:${PROVIDER}:${MODEL} -->"
|
|
say() { echo "[gadfly-review:${PROVIDER}:${MODEL}] $*" >&2; }
|
|
|
|
# When the swarm is consolidating (GADFLY_CONSOLIDATE=1, set by entrypoint.sh for
|
|
# a multi-model run), this model does NOT post its own comment — it writes its
|
|
# findings to GADFLY_FINDINGS_OUT and a single cross-model consensus comment is
|
|
# posted after the whole swarm finishes. Live progress still shows on the status
|
|
# board. Default 0 (post a per-model comment, the standalone behavior).
|
|
CONSOLIDATE="${GADFLY_CONSOLIDATE:-0}"
|
|
|
|
# Display the model's ACTUAL backend: the provider segment of the spec
|
|
# ("m1pro/qwen3.6:35b-mlx" -> "m1pro"); a bare id uses GADFLY_PROVIDER (default
|
|
# ollama-cloud). This is what the comment header shows, not the run.sh lane.
|
|
case "$MODEL" in
|
|
*/*) MODEL_PROVIDER="${MODEL%%/*}" ;;
|
|
*) MODEL_PROVIDER="${GADFLY_PROVIDER:-ollama-cloud}" ;;
|
|
esac
|
|
|
|
# jq is required for payload building / response parsing; install if missing.
|
|
if ! command -v jq >/dev/null 2>&1; then
|
|
say "jq not found; attempting install"
|
|
{ apt-get update -qq && apt-get install -y -qq jq; } >/dev/null 2>&1 \
|
|
|| { sudo apt-get update -qq && sudo apt-get install -y -qq jq; } >/dev/null 2>&1 \
|
|
|| { say "could not install jq"; exit 1; }
|
|
fi
|
|
|
|
# curl timeouts: Gitea API calls are quick. Word-split on purpose so the flags
|
|
# expand as separate args. (The LLM call's own deadline lives in the reviewer
|
|
# binary / agy, not here.)
|
|
API_TIMEOUT="--connect-timeout 20 --max-time 30"
|
|
|
|
# upsert_comment BODY — create or update (by MARKER) this model's single comment.
|
|
upsert_comment() {
|
|
local body="$1" post_body existing_id page=1 cmts
|
|
post_body="$(jq -n --arg b "$body" '{body:$b}')"
|
|
existing_id=""
|
|
while [ "$page" -le 10 ]; do
|
|
cmts="$(curl $API_TIMEOUT -fsS -H "Authorization: token ${GITEA_TOKEN}" \
|
|
"${GITEA_API}/issues/${PR}/comments?limit=50&page=${page}" || echo '[]')"
|
|
[ "$(echo "$cmts" | jq 'length')" = "0" ] && break
|
|
existing_id="$(echo "$cmts" | jq -r --arg m "$MARKER" \
|
|
'.[] | select(.body != null and (.body | startswith($m))) | .id' | head -n1)"
|
|
[ -n "$existing_id" ] && break
|
|
page=$((page+1))
|
|
done
|
|
if [ -n "$existing_id" ]; then
|
|
curl $API_TIMEOUT -sS -X PATCH -H "Authorization: token ${GITEA_TOKEN}" -H "Content-Type: application/json" \
|
|
"${GITEA_API}/issues/comments/${existing_id}" -d "$post_body" >/dev/null
|
|
else
|
|
curl $API_TIMEOUT -sS -X POST -H "Authorization: token ${GITEA_TOKEN}" -H "Content-Type: application/json" \
|
|
"${GITEA_API}/issues/${PR}/comments" -d "$post_body" >/dev/null
|
|
fi
|
|
}
|
|
|
|
# fmt_duration SECONDS -> "1m 23s" / "45s"
|
|
fmt_duration() { if [ "$1" -ge 60 ]; then echo "$(($1/60))m $(($1%60))s"; else echo "$1s"; fi; }
|
|
|
|
# --- fetch PR context -------------------------------------------------------
|
|
say "fetching PR #${PR} context"
|
|
DIFF="$(curl $API_TIMEOUT -fsS -H "Authorization: token ${GITEA_TOKEN}" "${GITEA_API}/pulls/${PR}.diff" || true)"
|
|
META="$(curl $API_TIMEOUT -fsS -H "Authorization: token ${GITEA_TOKEN}" "${GITEA_API}/pulls/${PR}" || echo '{}')"
|
|
TITLE="$(echo "$META" | jq -r '.title // ""')"
|
|
BODY="$(echo "$META" | jq -r '.body // ""')"
|
|
|
|
if [ -z "$DIFF" ]; then
|
|
say "empty diff; nothing to review"
|
|
exit 0
|
|
fi
|
|
|
|
# Keep the FULL diff for the agentic (ollama) reviewer — it can pull the whole
|
|
# thing via the get_diff tool and embeds a truncated copy in the prompt itself.
|
|
# The truncated copy below is only for the one-shot antigravity prompt.
|
|
FULL_DIFF="$DIFF"
|
|
TRUNC_NOTE=""
|
|
if [ "${#DIFF}" -gt "$MAX_DIFF_CHARS" ]; then
|
|
DIFF="${DIFF:0:$MAX_DIFF_CHARS}"
|
|
TRUNC_NOTE=$'\n\n[NOTE: diff truncated to '"${MAX_DIFF_CHARS}"' chars for length; review the rest manually.]'
|
|
fi
|
|
|
|
SYS="$(cat "${SCRIPT_DIR}/system-prompt.txt")"
|
|
USR="$(printf 'PR #%s: %s\n\nDescription:\n%s\n\nUnified diff to review:\n```diff\n%s\n```%s' \
|
|
"$PR" "$TITLE" "$BODY" "$DIFF" "$TRUNC_NOTE")"
|
|
|
|
# --- announce start (placeholder comment) -----------------------------------
|
|
START_TS="$(date +%s)"
|
|
say "starting review with ${MODEL}"
|
|
# Skip the per-model placeholder when consolidating (the consensus comment is
|
|
# posted later; live progress is on the status board).
|
|
[ "$CONSOLIDATE" = "1" ] || upsert_comment "$(printf '%s\n### 🪰 Gadfly review — `%s` (%s)\n\n⏳ Reviewing… this comment will update with findings and run time.' \
|
|
"$MARKER" "$MODEL" "$MODEL_PROVIDER")"
|
|
|
|
# --- call the model ---------------------------------------------------------
|
|
REVIEW=""
|
|
case "$PROVIDER" in
|
|
ollama)
|
|
# Agentic lane: hand off to the cmd/gadfly binary, which runs a tool-using
|
|
# agent over the checked-out repo so it can verify findings instead of
|
|
# guessing from the diff. The reviewer is majordomo-powered, so GADFLY_PROVIDER
|
|
# selects the backend (default ollama-cloud); local Ollama, OpenAI, Anthropic,
|
|
# Google and OpenAI/Ollama-compatible endpoints all work — see the README.
|
|
|
|
# Back-compat: map the consumer's OLLAMA_CLOUD_API_KEY secret onto the
|
|
# OLLAMA_API_KEY env the ollama-cloud provider reads.
|
|
if [ -n "${OLLAMA_CLOUD_API_KEY:-}" ] && [ -z "${OLLAMA_API_KEY:-}" ]; then
|
|
export OLLAMA_API_KEY="$OLLAMA_CLOUD_API_KEY"
|
|
fi
|
|
GADFLY_PROVIDER_EFF="$MODEL_PROVIDER"
|
|
|
|
# Pre-flight the credential for providers that need one. Why bother, when
|
|
# majordomo already fails closed with a 401: without this, a missing key
|
|
# surfaces as five identical per-lens agent failures that name no variable,
|
|
# and the operator has to read a stack trace to learn which secret they
|
|
# forgot to forward. Providers absent from this table need no key (local
|
|
# ollama, llama-swap) or carry it in their endpoint/DSN (foreman), and are
|
|
# left alone. GADFLY_API_KEY overrides any of them.
|
|
KEY_ENV=""; KEY_HINT=""
|
|
case "$GADFLY_PROVIDER_EFF" in
|
|
ollama-cloud) KEY_ENV="OLLAMA_API_KEY"; KEY_HINT="OLLAMA_CLOUD_API_KEY" ;;
|
|
qwen) KEY_ENV="QWEN_API_KEY"; KEY_HINT="QWEN_API_KEY" ;;
|
|
kimi) KEY_ENV="KIMI_API_KEY"; KEY_HINT="KIMI_API_KEY" ;;
|
|
openai) KEY_ENV="OPENAI_API_KEY"; KEY_HINT="OPENAI_API_KEY" ;;
|
|
anthropic) KEY_ENV="ANTHROPIC_API_KEY"; KEY_HINT="ANTHROPIC_API_KEY" ;;
|
|
esac
|
|
# Indirect expansion (bash), so the table above stays a table. Each majordomo
|
|
# built-in reads ONLY its own variable — cross-provider fallback is refused
|
|
# by design — so the named hint is always the actual fix.
|
|
if [ -n "$KEY_ENV" ] && [ -z "${!KEY_ENV:-}" ] && [ -z "${GADFLY_API_KEY:-}" ]; then
|
|
REVIEW="⚠️ No API key configured for provider \`${GADFLY_PROVIDER_EFF}\` (set \`${KEY_HINT}\`, or \`GADFLY_API_KEY\`); this reviewer was skipped."
|
|
else
|
|
BIN="${GADFLY_BIN:-gadfly}"
|
|
if ! command -v "$BIN" >/dev/null 2>&1 && [ ! -x "$BIN" ]; then
|
|
REVIEW="⚠️ Agentic reviewer binary not found (\`GADFLY_BIN=${BIN}\`); the workflow build step may have failed."
|
|
else
|
|
REPO_DIR="${GADFLY_REPO_DIR:-$(cd "${SCRIPT_DIR}/../../.." && pwd)}"
|
|
DIFF_FILE="$(mktemp)"
|
|
ERR_FILE="${DIFF_FILE}.err"
|
|
printf '%s' "$FULL_DIFF" > "$DIFF_FILE"
|
|
# GADFLY_PROVIDER / GADFLY_BASE_URL / GADFLY_API_KEY and provider key
|
|
# envs (OPENAI_API_KEY, …) are inherited from the process environment.
|
|
REVIEW="$(
|
|
GADFLY_MODEL="$MODEL" \
|
|
GADFLY_REPO_DIR="$REPO_DIR" \
|
|
GADFLY_DIFF_FILE="$DIFF_FILE" \
|
|
GADFLY_SYSTEM_FILE="${SCRIPT_DIR}/system-prompt.txt" \
|
|
GADFLY_TITLE="$TITLE" \
|
|
GADFLY_BODY="$BODY" \
|
|
GADFLY_MAX_DIFF_CHARS="${GADFLY_MAX_DIFF_CHARS:-$MAX_DIFF_CHARS}" \
|
|
GADFLY_STATUS_FILE="${GADFLY_STATUS_FILE:-}" \
|
|
GADFLY_FINDINGS_OUT="${GADFLY_FINDINGS_OUT:-}" \
|
|
"$BIN" 2>"$ERR_FILE"
|
|
)"
|
|
rc=$?
|
|
if [ "$rc" -ne 0 ] || [ -z "$REVIEW" ]; then
|
|
REVIEW="⚠️ Agentic reviewer for \`${MODEL}\` failed (exit ${rc}):
|
|
\`\`\`
|
|
$(tail -c 1500 "$ERR_FILE" 2>/dev/null)
|
|
\`\`\`"
|
|
fi
|
|
rm -f "$DIFF_FILE" "$ERR_FILE"
|
|
fi
|
|
fi
|
|
;;
|
|
antigravity)
|
|
if ! command -v agy >/dev/null 2>&1; then
|
|
REVIEW="⚠️ Antigravity CLI (\`agy\`) not found on PATH."
|
|
else
|
|
FULL="$(printf '%s\n\n%s' "$SYS" "$USR")"
|
|
if ! REVIEW="$(agy -p "$FULL" --model "$MODEL" 2>agy.err)"; then
|
|
REVIEW="⚠️ Antigravity CLI failed:
|
|
\`\`\`
|
|
$(tail -c 1500 agy.err 2>/dev/null)
|
|
\`\`\`"
|
|
fi
|
|
[ -z "$REVIEW" ] && REVIEW="⚠️ Antigravity CLI returned no output (auth/quota?)."
|
|
fi
|
|
;;
|
|
*)
|
|
say "unknown provider: ${PROVIDER}"; exit 1 ;;
|
|
esac
|
|
|
|
# --- assemble + post final comment (with run time) --------------------------
|
|
ELAPSED="$(( $(date +%s) - START_TS ))"
|
|
DUR="$(fmt_duration "$ELAPSED")"
|
|
|
|
# Consolidating: the binary writes its findings file on success. If it failed or
|
|
# was skipped (no file, or an empty one), write a stub so this model still shows
|
|
# up in the consensus (as failed) and an all-models-fail run still posts a
|
|
# comment — never silently drop a model or the whole review.
|
|
if [ "$CONSOLIDATE" = "1" ] && [ -n "${GADFLY_FINDINGS_OUT:-}" ] && [ ! -s "${GADFLY_FINDINGS_OUT}" ]; then
|
|
jq -n --arg model "$MODEL" --arg provider "$MODEL_PROVIDER" --arg md "$REVIEW" \
|
|
'{model:$model, provider:$provider, verdict:"reviewer failed", errored:true, markdown:$md, findings:[]}' \
|
|
> "${GADFLY_FINDINGS_OUT}" 2>/dev/null || true
|
|
fi
|
|
# When consolidating, the binary has written this model's findings to
|
|
# GADFLY_FINDINGS_OUT; the consensus comment is posted by entrypoint.sh after the
|
|
# whole swarm finishes, so this model posts no comment of its own.
|
|
if [ "$CONSOLIDATE" = "1" ]; then
|
|
say "done in ${DUR} (consolidated; no per-model comment)"
|
|
else
|
|
# An optional one-line notice (e.g. entrypoint's huge-PR downshift advisory),
|
|
# shown under the header so a shallower review is self-explaining.
|
|
NOTICE_BLOCK=""
|
|
[ -n "${GADFLY_NOTICE:-}" ] && NOTICE_BLOCK="> ${GADFLY_NOTICE}"$'\n\n'
|
|
COMMENT="$(printf '%s\n### 🪰 Gadfly review — `%s` (%s)\n\n%s%s\n\n<sub>Automated adversarial review by Gadfly. Advisory only — does not block merge. · ⏱️ reviewed in %s</sub>' \
|
|
"$MARKER" "$MODEL" "$MODEL_PROVIDER" "$NOTICE_BLOCK" "$REVIEW" "$DUR")"
|
|
upsert_comment "$COMMENT"
|
|
say "done in ${DUR}"
|
|
fi
|