commit fae89a1ed0027d833a61a780b7683dcb0056c213 Author: Steve Dudenhoeffer Date: Sat Jul 4 23:06:25 2026 -0400 feat(gif): animated GIF/MP4 skill pack for Discord Adapts mort's builtin animate skill into an executus/skillpack: - plans scenes + a reusable cast so recurring people/props stay consistent (built for funny bits 'about ourselves'), incl. multi-minute pieces - bundled scripts/encode.py owns the fragile encode + format decision: GIF for short loops, H.264 MP4 (yuv420p, even dims, +faststart — plays inline in Discord) for long pieces OR when a GIF comes out too big, with a gifski→ffmpeg-palette fallback - keeps the workspace discipline, text-readability pacing, and <=3-pass vision self-critique from the original; adds a frame budget for long renders encode.py verified end-to-end (short->gif, long->mp4 confirmed h264/yuv420p/ even/faststart); SKILL.md parses cleanly via the executus skillpack loader. Co-Authored-By: Claude Opus 4.8 diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..b908d4c --- /dev/null +++ b/.gitignore @@ -0,0 +1,3 @@ +__pycache__/ +*.pyc +.DS_Store diff --git a/README.md b/README.md new file mode 100644 index 0000000..478fe8c --- /dev/null +++ b/README.md @@ -0,0 +1,46 @@ +# executus-skills + +Skill packs for [executus](https://gitea.stevedudenhoeffer.com/steve/executus) +harnesses (mort, gadfly, …), in the SKILL.md "agent-skills" format. Each pack is +its own subfolder with a `SKILL.md` manifest plus any bundled helper files. + +``` +executus-skills/ + gif/ + SKILL.md # instructions the agent loads on demand + scripts/encode.py # bundled helper the agent runs in its sandbox +``` + +## Packs + +- **`gif`** — make a funny animated GIF or MP4 for Discord from a description. + Plans scenes, keeps recurring people/props consistent via a reusable cast, + renders frames in Python, and lets a bundled `encode.py` pick the format: + a GIF for short loops, or an H.264 MP4 (that Discord plays inline) for long + bits or when a GIF comes out too big. Adapted from mort's builtin `animate` + skill. Handles multi-minute pieces. + +## Using a pack from mort + +Each subfolder is a separate subscription (pinned by content digest): + +``` +.set skillpacks.enabled true # then restart mort (boot-read) +.set skillpacks.allowed_sources git:https://gitea.stevedudenhoeffer.com/steve/ +.skillpack subscribe git https://gitea.stevedudenhoeffer.com/steve/executus-skills subpath=gif ref=main +``` + +Then attach the pack to an agent that already has the tools the pack uses (the +pack supplies instructions, not tools). For `gif`, the agent needs +`code_exec`, `image_describe`, `send_attachments`, `file_get_metadata`, and +`think` in its toolset, a generous `max_runtime` with `critic_enabled: true` +(long renders run for minutes), and a sandbox image with `ffmpeg` (plus +`gifski`/`gifsicle` for nicer GIFs). Add it in the agent's YAML: + +```yaml +low_level_tools: [code_exec, image_describe, send_attachments, file_get_metadata, think] +skill_packs: [gif] +``` + +When the pack is updated upstream, `.skillpack check` reports it and +`.skillpack apply gif` re-pins — nothing an agent runs changes until you apply. diff --git a/gif/SKILL.md b/gif/SKILL.md new file mode 100644 index 0000000..78865e4 --- /dev/null +++ b/gif/SKILL.md @@ -0,0 +1,179 @@ +--- +name: gif +description: Make a funny animated GIF (or MP4) for Discord from a description — plans scenes, keeps recurring people/props consistent via a reusable cast, renders frames in Python, encodes cleanly, and automatically hands Discord an H.264 MP4 it can play when the piece is long or a GIF comes out too big. Use whenever someone asks to draw/animate/gif something, including multi-minute bits about people or things that happened. +license: MIT +allowed-tools: [code_exec, image_describe, send_attachments, file_get_metadata, think] +metadata: + version: "1.0.0" + adapted-from: mort/skills/animate +--- + +# Make an animated GIF / MP4 for Discord + +You make HIGH-QUALITY, funny animations from a description — often caricatures +of the people in the channel or a bit about something that just happened. +Naive PIL GIF export looks bad (256-colour banding, muddy dithering) and a +multi-minute GIF is physically absurd (hundreds of MB), so follow this recipe: +render frames in Python, let the bundled `encode.py` choose GIF vs MP4, and +self-critique the result. **Work by calling tools — do not introduce yourself +or list capabilities.** + +## 1. Plan first: length, scenes, and cast + +Before drawing, decide (use `think` for anything non-trivial): + +- **How long / how many scenes?** Count the distinct beats the prompt actually + describes, and render exactly that many. **ASSUME ONE.** A single continuous + action ("a bouncing ball", "Steve slipping on a banana") is ONE scene — don't + chop it into cuts. Start a new scene only when the prompt moves to a new beat + (new action, place, or time jump). There is **no upper limit** — a bit that + genuinely describes twelve beats gets twelve scenes and runs for minutes — but + **never pad a simple request with invented beats**, and never compress beats + the user clearly separated. When in doubt, fewer scenes. +- **Who/what is the cast?** List every recurring character and prop (a person's + persona, "the dog", a logo). For a bit "about ", pick a simple, + recognizable caricature — one or two signature features (glasses, hair colour, + a hat) — and pin each cast member's look ONCE as a reusable Python helper (see + §4). That is what keeps the SAME person drawn identically across every scene + instead of re-invented each cut. A one-scene GIF with a single subject needs + no cast block. + +## 2. Pick the format from the length (encode.py does this for you) + +- **Short & loopable (≤ ~20 s):** a GIF — fun, loops, autoplays. +- **Long (tens of seconds to minutes):** an **MP4**. A 3–10 minute GIF is + hundreds of MB and unusable; H.264 is the right tool and Discord plays it + inline. +- You don't hand-pick this: `encode.py`'s `encode_animation(...)` takes a + `seconds_hint` and (a) goes straight to MP4 for long pieces, and (b) for short + ones, makes the GIF but **auto-re-encodes to an MP4 if the GIF is too big for + Discord to render nicely.** It returns `{"path", "mime", "bytes"}` — deliver + whatever it hands back. + +## 3. code_exec workspace — READ THIS (it is why draws fail) + +The `code_exec` sandbox has three hard rules; breaking them is the #1 cause of +"vanished" files and wasted budget: + +1. **`/workspace` is WIPED before every `code_exec` call.** Nothing survives + between calls. So render → encode → write your critique still **all in ONE + call**. Never write frames in one call and read them in another. +2. **Only TOP-LEVEL files in `/workspace/` come back as `files_out` file_ids.** + A file in a subdir (`/workspace/frames/x.png`) is NOT returned. Write the + final artifact and the ONE critique still directly at the top: + `encode.py` already writes `/workspace/final.gif` or `/workspace/final.mp4`; + you add `/workspace/still.png`. Render scratch frames under `/tmp`, never + `/workspace`. +3. **Don't dump many files into `/workspace`.** Hundreds of frame PNGs blow the + `files_out` size cap and get dropped. Keep `/workspace` to just the final + artifact + one still; everything else lives under `/tmp`. + +`image_describe`, `send_attachments`, `file_get_metadata` take a **file_id from +`files_out` — never a path.** Passing `"/workspace/final.mp4"` fails +`file_not_found`. Read the file_id out of `files_out` and pass THAT. + +## 4. Render → encode (ONE code_exec call, using encode.py) + +This skill bundles `scripts/encode.py`. Load it into the render call with +`files_in` (its file_id is listed when you open this skill), then in a SINGLE +`code_exec` call: + +1. **Clear scratch** so stale frames can't leak into this encode: + `import shutil, os; shutil.rmtree('/tmp/frames', ignore_errors=True); os.makedirs('/tmp/frames')` +2. **Render frames** with Python (matplotlib / Pillow / numpy) into `/tmp/frames` + as `frame_00000.png, frame_00001.png, …` — **fixed 5-digit zero-pad, ONE + continuous counter across all scenes** (do NOT restart numbering per scene, or + the encoder reorders/overwrites frames). +3. **Encode** — let the helper choose GIF vs MP4: + ```python + from encode import encode_animation + art = encode_animation("/tmp/frames", fps=15, seconds_hint=) + print(art) # {'path': '/workspace/final.mp4', 'mime': 'video/mp4', 'bytes': ...} + ``` +4. **Write ONE critique still** to `/workspace/still.png` in the same call — a + representative frame, or for a multi-scene piece a COMPACT contact sheet with + one small (~200–320px) tile per scene, labelled by scene index, so the + critique can check every scene and cast consistency. Keep it low-res and + legible — it's an internal aid, not the deliverable. + +The call ends with the final artifact + `still.png` at the top of `/workspace`; +`files_out` returns a file_id for each. Give `code_exec` a generous +`timeout_seconds` (e.g. 120+) — encoding is CPU-bound, and a long piece is real +work, not a hang. + +### Multi-scene: define the cast ONCE, reuse it everywhere + +It all still happens in the one call — structure the program as: +1. **Cast block at the top, defined ONCE:** helpers like + `def draw_steve(ax, x, y, pose): …` and shared constants (`PALETTE`, `SKY`). +2. **One function per scene** (`def scene_0(...): …`) that draws its beat by + CALLING the cast helpers — never by re-describing a character inline. +3. **A timeline** that runs the scenes in order into `/tmp/frames` with the one + continuous counter, then encodes the whole sequence. + +Because every scene calls the same `draw_*` helpers, recurring people/props come +out pixel-identical scene to scene — the entire point of the cast block. If you +catch yourself re-describing a character inside a scene, hoist it into a cast +helper. Carry shared state (positions, score, time of day) in plain variables +between scene calls so cuts stay continuous. + +### Budget for LONG pieces (minutes) + +A 3–10 minute animation is thousands of frames. Keep it renderable: **fps +10–12**, cap the long edge ~**480–540px** (`long_edge=` in `encode_animation`), +and favour procedural motion that's cheap per frame — don't render 9000 frames +of heavy matplotlib if a lighter draw reads the same. **If code_exec times out +or errors mid-render you get nothing back**, so on the retry SCALE DOWN +decisively (lower fps, smaller canvas, trim hold frames, reduce the longest +scenes) — keep every beat, just render it leaner. Each retry counts toward the +3-pass cap; never burn all three on an over-ambitious render and leave the user +with nothing. + +## 5. Text in GIFs — make it READABLE + +Adding captions/labels/punchlines is fine, but pace them for a human. Motion can +stay 10–20 fps; what matters is how long each distinct TEXT STATE holds — short +phrases ~1.5–2 s, full sentences longer (budget by word count). At 15 fps that's +~25–30 repeated frames per text state, not a handful. Hold by duplicating frames, +or with PIL pass a per-frame `duration=[…]` list (long on text, short on motion). +Text flashing past unreadably is the #1 complaint — check it in the critique. + +## 6. Self-critique (AT MOST 3 render passes) + +Before delivering, LOOK at what you made: call `image_describe` on the +`still.png` **file_id** (from `files_out`, not a path) and judge it against the +request — motion, colour banding, artifacts, framing, "does it depict what was +asked", and text readability. For a multi-scene piece, judge scene by scene off +the contact sheet AND check each recurring character/prop is the SAME across +scenes (drift means that scene drew it ad hoc — fix the cast helper and +re-render). If it falls short, change something concrete and re-run the WHOLE +pipeline in one fresh `code_exec` call (workspace was wiped — rebuild from +scratch). **At most 3 render passes total**; for a long piece be more decisive +(1–2 passes) since each re-render is expensive. After the 3rd, deliver the best +version with a one-line note on any trade-off. + +## 7. Deliver + +Deliver the final artifact with a SINGLE `send_attachments` call using +`channel_id` from your platform context and the **file_id that +`encode_animation` produced** (read it from that render call's `files_out` by +matching `final.gif` / `final.mp4` — NOT the still, NOT a path). Finish with a +one-line caption. Critique aids (contact sheets, stills) are INTERNAL — never +attach them unless the user explicitly asked. + +## Forbidden + +- NEVER pass a `/workspace/...` path to image_describe / send_attachments / + file_get_metadata — they take a file_id from `files_out`. +- Do not write the final artifact into a `/workspace` subdir — top level only. +- Do not split render and read across separate code_exec calls — `/workspace` is + wiped between them. +- Do not hand-roll the encode/format decision — call `encode_animation`; it + guarantees a Discord-playable MP4 (yuv420p, even dims, faststart) when needed. +- Do not restart the frame counter per scene — one continuous 5-digit counter. +- Do not re-describe a recurring character inline per scene — one shared cast + helper so it stays identical. +- Do not invent beats the user didn't ask for — assume ONE scene, add more only + for beats the prompt describes (no cap, but no padding). +- Do not exceed 3 render passes; do not ship an empty/broken result without + saying so. diff --git a/gif/scripts/encode.py b/gif/scripts/encode.py new file mode 100644 index 0000000..1af0b56 --- /dev/null +++ b/gif/scripts/encode.py @@ -0,0 +1,115 @@ +""" +encode.py — turn a directory of PNG frames into a Discord-friendly GIF or MP4. + +Use it INSIDE a single code_exec call. Render your frames to a scratch dir as +frame_00000.png, frame_00001.png, … (fixed 5-digit width, ONE continuous +counter across all scenes), then: + + from encode import encode_animation + art = encode_animation("/tmp/frames", fps=15, seconds_hint=8) + # art == {"path": "/workspace/final.mp4", "mime": "video/mp4", "bytes": 1234567} + +It writes the finished artifact to the TOP of /workspace (so code_exec returns a +file_id for it) and picks the format for you: + + * short & light -> animated GIF (fun, loops, autoplays) + * long (many seconds+), + OR a GIF too big to post -> H.264 MP4: yuv420p, even dimensions, +faststart + — the combination Discord needs to PLAY it inline + (a video whose dimensions Discord can't read is + shown as a dead download, not a player). + +Requires ffmpeg (always present in the sandbox). Uses gifski + gifsicle for a +cleaner GIF when they're installed, falling back to ffmpeg's palette pipeline. +This helper only ENCODES — you draw the frames. +""" + +import glob +import os +import shutil +import subprocess + +# A GIF longer than this balloons past what's worth shipping as a GIF — encode +# straight to MP4 instead. Tune down for chattier channels. +GIF_MAX_SECONDS = 20 +# Keep a GIF comfortably under Discord's ~10 MiB inline limit; over this we +# re-encode to an MP4 and drop the GIF. +GIF_MAX_BYTES = 9 * 1024 * 1024 + + +def _which(prog): + return shutil.which(prog) is not None + + +def _run(cmd): + p = subprocess.run(cmd, capture_output=True, text=True) + if p.returncode != 0: + raise RuntimeError(f"{cmd[0]} failed:\n{p.stderr[-2000:]}") + + +def _frames(frames_dir): + fs = sorted(glob.glob(os.path.join(frames_dir, "frame_*.png"))) + if not fs: + raise FileNotFoundError( + f"no frame_*.png in {frames_dir} — render frames as " + f"{frames_dir}/frame_00000.png, frame_00001.png, … first" + ) + return fs + + +def _encode_gif(frames_dir, fps, out, long_edge): + fs = _frames(frames_dir) + if _which("gifski"): + # gifski takes an explicit, sorted file list — no numbering-gap surprises. + _run(["gifski", "-o", out, "--fps", str(fps), "--quality", "90"] + fs) + else: + vf = (f"fps={fps},scale='min({long_edge},iw)':-1:flags=lanczos," + f"split[s0][s1];[s0]palettegen=stats_mode=diff[p];" + f"[s1][p]paletteuse=dither=bayer:bayer_scale=3") + _run(["ffmpeg", "-y", "-pattern_type", "glob", + "-i", os.path.join(frames_dir, "frame_*.png"), "-vf", vf, out]) + if _which("gifsicle"): + _run(["gifsicle", "-O3", "--lossy=60", out, "-o", out]) + + +def _encode_mp4(frames_dir, fps, out, long_edge): + _frames(frames_dir) # validate presence + # -2 forces EVEN width/height (H.264 requires it); yuv420p + faststart make + # Discord/browsers play it inline instead of offering a download. + vf = f"scale='min({long_edge},iw)':-2:flags=lanczos" + _run(["ffmpeg", "-y", "-framerate", str(fps), "-pattern_type", "glob", + "-i", os.path.join(frames_dir, "frame_*.png"), + "-c:v", "libx264", "-pix_fmt", "yuv420p", "-crf", "23", + "-vf", vf, "-movflags", "+faststart", out]) + + +def encode_animation(frames_dir, fps=15, seconds_hint=0, long_edge=640): + """Encode frames_dir → /workspace/final.{gif,mp4}. Returns {path, mime, bytes}. + + seconds_hint: your intended runtime in seconds (falls back to frame_count/fps + when 0). Pass it for long pieces so the format decision doesn't depend on how + many frames you happened to render. + long_edge: cap for the output's long side in px (keeps size/encode sane). + """ + n = len(_frames(frames_dir)) + seconds = seconds_hint or (n / max(fps, 1)) + gif, mp4 = "/workspace/final.gif", "/workspace/final.mp4" + + if seconds > GIF_MAX_SECONDS: + _encode_mp4(frames_dir, fps, mp4, long_edge) + return _art(mp4, "video/mp4") + + _encode_gif(frames_dir, fps, gif, long_edge) + if os.path.getsize(gif) <= GIF_MAX_BYTES: + return _art(gif, "image/gif") + + # Too heavy as a GIF — hand Discord an MP4 it can actually render. + _encode_mp4(frames_dir, fps, mp4, long_edge) + os.remove(gif) + return _art(mp4, "video/mp4") + + +def _art(path, mime): + b = os.path.getsize(path) + print(f"[encode] {path} {mime} {b / 1024 / 1024:.2f} MiB") + return {"path": path, "mime": mime, "bytes": b}