feat(gif): animated GIF/MP4 skill pack for Discord
Adapts mort's builtin animate skill into an executus/skillpack: - plans scenes + a reusable cast so recurring people/props stay consistent (built for funny bits 'about ourselves'), incl. multi-minute pieces - bundled scripts/encode.py owns the fragile encode + format decision: GIF for short loops, H.264 MP4 (yuv420p, even dims, +faststart — plays inline in Discord) for long pieces OR when a GIF comes out too big, with a gifski→ffmpeg-palette fallback - keeps the workspace discipline, text-readability pacing, and <=3-pass vision self-critique from the original; adds a frame budget for long renders encode.py verified end-to-end (short->gif, long->mp4 confirmed h264/yuv420p/ even/faststart); SKILL.md parses cleanly via the executus skillpack loader. Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
@@ -0,0 +1,3 @@
|
||||
__pycache__/
|
||||
*.pyc
|
||||
.DS_Store
|
||||
@@ -0,0 +1,46 @@
|
||||
# executus-skills
|
||||
|
||||
Skill packs for [executus](https://gitea.stevedudenhoeffer.com/steve/executus)
|
||||
harnesses (mort, gadfly, …), in the SKILL.md "agent-skills" format. Each pack is
|
||||
its own subfolder with a `SKILL.md` manifest plus any bundled helper files.
|
||||
|
||||
```
|
||||
executus-skills/
|
||||
gif/
|
||||
SKILL.md # instructions the agent loads on demand
|
||||
scripts/encode.py # bundled helper the agent runs in its sandbox
|
||||
```
|
||||
|
||||
## Packs
|
||||
|
||||
- **`gif`** — make a funny animated GIF or MP4 for Discord from a description.
|
||||
Plans scenes, keeps recurring people/props consistent via a reusable cast,
|
||||
renders frames in Python, and lets a bundled `encode.py` pick the format:
|
||||
a GIF for short loops, or an H.264 MP4 (that Discord plays inline) for long
|
||||
bits or when a GIF comes out too big. Adapted from mort's builtin `animate`
|
||||
skill. Handles multi-minute pieces.
|
||||
|
||||
## Using a pack from mort
|
||||
|
||||
Each subfolder is a separate subscription (pinned by content digest):
|
||||
|
||||
```
|
||||
.set skillpacks.enabled true # then restart mort (boot-read)
|
||||
.set skillpacks.allowed_sources git:https://gitea.stevedudenhoeffer.com/steve/
|
||||
.skillpack subscribe git https://gitea.stevedudenhoeffer.com/steve/executus-skills subpath=gif ref=main
|
||||
```
|
||||
|
||||
Then attach the pack to an agent that already has the tools the pack uses (the
|
||||
pack supplies instructions, not tools). For `gif`, the agent needs
|
||||
`code_exec`, `image_describe`, `send_attachments`, `file_get_metadata`, and
|
||||
`think` in its toolset, a generous `max_runtime` with `critic_enabled: true`
|
||||
(long renders run for minutes), and a sandbox image with `ffmpeg` (plus
|
||||
`gifski`/`gifsicle` for nicer GIFs). Add it in the agent's YAML:
|
||||
|
||||
```yaml
|
||||
low_level_tools: [code_exec, image_describe, send_attachments, file_get_metadata, think]
|
||||
skill_packs: [gif]
|
||||
```
|
||||
|
||||
When the pack is updated upstream, `.skillpack check` reports it and
|
||||
`.skillpack apply gif` re-pins — nothing an agent runs changes until you apply.
|
||||
+179
@@ -0,0 +1,179 @@
|
||||
---
|
||||
name: gif
|
||||
description: Make a funny animated GIF (or MP4) for Discord from a description — plans scenes, keeps recurring people/props consistent via a reusable cast, renders frames in Python, encodes cleanly, and automatically hands Discord an H.264 MP4 it can play when the piece is long or a GIF comes out too big. Use whenever someone asks to draw/animate/gif something, including multi-minute bits about people or things that happened.
|
||||
license: MIT
|
||||
allowed-tools: [code_exec, image_describe, send_attachments, file_get_metadata, think]
|
||||
metadata:
|
||||
version: "1.0.0"
|
||||
adapted-from: mort/skills/animate
|
||||
---
|
||||
|
||||
# Make an animated GIF / MP4 for Discord
|
||||
|
||||
You make HIGH-QUALITY, funny animations from a description — often caricatures
|
||||
of the people in the channel or a bit about something that just happened.
|
||||
Naive PIL GIF export looks bad (256-colour banding, muddy dithering) and a
|
||||
multi-minute GIF is physically absurd (hundreds of MB), so follow this recipe:
|
||||
render frames in Python, let the bundled `encode.py` choose GIF vs MP4, and
|
||||
self-critique the result. **Work by calling tools — do not introduce yourself
|
||||
or list capabilities.**
|
||||
|
||||
## 1. Plan first: length, scenes, and cast
|
||||
|
||||
Before drawing, decide (use `think` for anything non-trivial):
|
||||
|
||||
- **How long / how many scenes?** Count the distinct beats the prompt actually
|
||||
describes, and render exactly that many. **ASSUME ONE.** A single continuous
|
||||
action ("a bouncing ball", "Steve slipping on a banana") is ONE scene — don't
|
||||
chop it into cuts. Start a new scene only when the prompt moves to a new beat
|
||||
(new action, place, or time jump). There is **no upper limit** — a bit that
|
||||
genuinely describes twelve beats gets twelve scenes and runs for minutes — but
|
||||
**never pad a simple request with invented beats**, and never compress beats
|
||||
the user clearly separated. When in doubt, fewer scenes.
|
||||
- **Who/what is the cast?** List every recurring character and prop (a person's
|
||||
persona, "the dog", a logo). For a bit "about <someone>", pick a simple,
|
||||
recognizable caricature — one or two signature features (glasses, hair colour,
|
||||
a hat) — and pin each cast member's look ONCE as a reusable Python helper (see
|
||||
§4). That is what keeps the SAME person drawn identically across every scene
|
||||
instead of re-invented each cut. A one-scene GIF with a single subject needs
|
||||
no cast block.
|
||||
|
||||
## 2. Pick the format from the length (encode.py does this for you)
|
||||
|
||||
- **Short & loopable (≤ ~20 s):** a GIF — fun, loops, autoplays.
|
||||
- **Long (tens of seconds to minutes):** an **MP4**. A 3–10 minute GIF is
|
||||
hundreds of MB and unusable; H.264 is the right tool and Discord plays it
|
||||
inline.
|
||||
- You don't hand-pick this: `encode.py`'s `encode_animation(...)` takes a
|
||||
`seconds_hint` and (a) goes straight to MP4 for long pieces, and (b) for short
|
||||
ones, makes the GIF but **auto-re-encodes to an MP4 if the GIF is too big for
|
||||
Discord to render nicely.** It returns `{"path", "mime", "bytes"}` — deliver
|
||||
whatever it hands back.
|
||||
|
||||
## 3. code_exec workspace — READ THIS (it is why draws fail)
|
||||
|
||||
The `code_exec` sandbox has three hard rules; breaking them is the #1 cause of
|
||||
"vanished" files and wasted budget:
|
||||
|
||||
1. **`/workspace` is WIPED before every `code_exec` call.** Nothing survives
|
||||
between calls. So render → encode → write your critique still **all in ONE
|
||||
call**. Never write frames in one call and read them in another.
|
||||
2. **Only TOP-LEVEL files in `/workspace/` come back as `files_out` file_ids.**
|
||||
A file in a subdir (`/workspace/frames/x.png`) is NOT returned. Write the
|
||||
final artifact and the ONE critique still directly at the top:
|
||||
`encode.py` already writes `/workspace/final.gif` or `/workspace/final.mp4`;
|
||||
you add `/workspace/still.png`. Render scratch frames under `/tmp`, never
|
||||
`/workspace`.
|
||||
3. **Don't dump many files into `/workspace`.** Hundreds of frame PNGs blow the
|
||||
`files_out` size cap and get dropped. Keep `/workspace` to just the final
|
||||
artifact + one still; everything else lives under `/tmp`.
|
||||
|
||||
`image_describe`, `send_attachments`, `file_get_metadata` take a **file_id from
|
||||
`files_out` — never a path.** Passing `"/workspace/final.mp4"` fails
|
||||
`file_not_found`. Read the file_id out of `files_out` and pass THAT.
|
||||
|
||||
## 4. Render → encode (ONE code_exec call, using encode.py)
|
||||
|
||||
This skill bundles `scripts/encode.py`. Load it into the render call with
|
||||
`files_in` (its file_id is listed when you open this skill), then in a SINGLE
|
||||
`code_exec` call:
|
||||
|
||||
1. **Clear scratch** so stale frames can't leak into this encode:
|
||||
`import shutil, os; shutil.rmtree('/tmp/frames', ignore_errors=True); os.makedirs('/tmp/frames')`
|
||||
2. **Render frames** with Python (matplotlib / Pillow / numpy) into `/tmp/frames`
|
||||
as `frame_00000.png, frame_00001.png, …` — **fixed 5-digit zero-pad, ONE
|
||||
continuous counter across all scenes** (do NOT restart numbering per scene, or
|
||||
the encoder reorders/overwrites frames).
|
||||
3. **Encode** — let the helper choose GIF vs MP4:
|
||||
```python
|
||||
from encode import encode_animation
|
||||
art = encode_animation("/tmp/frames", fps=15, seconds_hint=<intended seconds>)
|
||||
print(art) # {'path': '/workspace/final.mp4', 'mime': 'video/mp4', 'bytes': ...}
|
||||
```
|
||||
4. **Write ONE critique still** to `/workspace/still.png` in the same call — a
|
||||
representative frame, or for a multi-scene piece a COMPACT contact sheet with
|
||||
one small (~200–320px) tile per scene, labelled by scene index, so the
|
||||
critique can check every scene and cast consistency. Keep it low-res and
|
||||
legible — it's an internal aid, not the deliverable.
|
||||
|
||||
The call ends with the final artifact + `still.png` at the top of `/workspace`;
|
||||
`files_out` returns a file_id for each. Give `code_exec` a generous
|
||||
`timeout_seconds` (e.g. 120+) — encoding is CPU-bound, and a long piece is real
|
||||
work, not a hang.
|
||||
|
||||
### Multi-scene: define the cast ONCE, reuse it everywhere
|
||||
|
||||
It all still happens in the one call — structure the program as:
|
||||
1. **Cast block at the top, defined ONCE:** helpers like
|
||||
`def draw_steve(ax, x, y, pose): …` and shared constants (`PALETTE`, `SKY`).
|
||||
2. **One function per scene** (`def scene_0(...): …`) that draws its beat by
|
||||
CALLING the cast helpers — never by re-describing a character inline.
|
||||
3. **A timeline** that runs the scenes in order into `/tmp/frames` with the one
|
||||
continuous counter, then encodes the whole sequence.
|
||||
|
||||
Because every scene calls the same `draw_*` helpers, recurring people/props come
|
||||
out pixel-identical scene to scene — the entire point of the cast block. If you
|
||||
catch yourself re-describing a character inside a scene, hoist it into a cast
|
||||
helper. Carry shared state (positions, score, time of day) in plain variables
|
||||
between scene calls so cuts stay continuous.
|
||||
|
||||
### Budget for LONG pieces (minutes)
|
||||
|
||||
A 3–10 minute animation is thousands of frames. Keep it renderable: **fps
|
||||
10–12**, cap the long edge ~**480–540px** (`long_edge=` in `encode_animation`),
|
||||
and favour procedural motion that's cheap per frame — don't render 9000 frames
|
||||
of heavy matplotlib if a lighter draw reads the same. **If code_exec times out
|
||||
or errors mid-render you get nothing back**, so on the retry SCALE DOWN
|
||||
decisively (lower fps, smaller canvas, trim hold frames, reduce the longest
|
||||
scenes) — keep every beat, just render it leaner. Each retry counts toward the
|
||||
3-pass cap; never burn all three on an over-ambitious render and leave the user
|
||||
with nothing.
|
||||
|
||||
## 5. Text in GIFs — make it READABLE
|
||||
|
||||
Adding captions/labels/punchlines is fine, but pace them for a human. Motion can
|
||||
stay 10–20 fps; what matters is how long each distinct TEXT STATE holds — short
|
||||
phrases ~1.5–2 s, full sentences longer (budget by word count). At 15 fps that's
|
||||
~25–30 repeated frames per text state, not a handful. Hold by duplicating frames,
|
||||
or with PIL pass a per-frame `duration=[…]` list (long on text, short on motion).
|
||||
Text flashing past unreadably is the #1 complaint — check it in the critique.
|
||||
|
||||
## 6. Self-critique (AT MOST 3 render passes)
|
||||
|
||||
Before delivering, LOOK at what you made: call `image_describe` on the
|
||||
`still.png` **file_id** (from `files_out`, not a path) and judge it against the
|
||||
request — motion, colour banding, artifacts, framing, "does it depict what was
|
||||
asked", and text readability. For a multi-scene piece, judge scene by scene off
|
||||
the contact sheet AND check each recurring character/prop is the SAME across
|
||||
scenes (drift means that scene drew it ad hoc — fix the cast helper and
|
||||
re-render). If it falls short, change something concrete and re-run the WHOLE
|
||||
pipeline in one fresh `code_exec` call (workspace was wiped — rebuild from
|
||||
scratch). **At most 3 render passes total**; for a long piece be more decisive
|
||||
(1–2 passes) since each re-render is expensive. After the 3rd, deliver the best
|
||||
version with a one-line note on any trade-off.
|
||||
|
||||
## 7. Deliver
|
||||
|
||||
Deliver the final artifact with a SINGLE `send_attachments` call using
|
||||
`channel_id` from your platform context and the **file_id that
|
||||
`encode_animation` produced** (read it from that render call's `files_out` by
|
||||
matching `final.gif` / `final.mp4` — NOT the still, NOT a path). Finish with a
|
||||
one-line caption. Critique aids (contact sheets, stills) are INTERNAL — never
|
||||
attach them unless the user explicitly asked.
|
||||
|
||||
## Forbidden
|
||||
|
||||
- NEVER pass a `/workspace/...` path to image_describe / send_attachments /
|
||||
file_get_metadata — they take a file_id from `files_out`.
|
||||
- Do not write the final artifact into a `/workspace` subdir — top level only.
|
||||
- Do not split render and read across separate code_exec calls — `/workspace` is
|
||||
wiped between them.
|
||||
- Do not hand-roll the encode/format decision — call `encode_animation`; it
|
||||
guarantees a Discord-playable MP4 (yuv420p, even dims, faststart) when needed.
|
||||
- Do not restart the frame counter per scene — one continuous 5-digit counter.
|
||||
- Do not re-describe a recurring character inline per scene — one shared cast
|
||||
helper so it stays identical.
|
||||
- Do not invent beats the user didn't ask for — assume ONE scene, add more only
|
||||
for beats the prompt describes (no cap, but no padding).
|
||||
- Do not exceed 3 render passes; do not ship an empty/broken result without
|
||||
saying so.
|
||||
@@ -0,0 +1,115 @@
|
||||
"""
|
||||
encode.py — turn a directory of PNG frames into a Discord-friendly GIF or MP4.
|
||||
|
||||
Use it INSIDE a single code_exec call. Render your frames to a scratch dir as
|
||||
frame_00000.png, frame_00001.png, … (fixed 5-digit width, ONE continuous
|
||||
counter across all scenes), then:
|
||||
|
||||
from encode import encode_animation
|
||||
art = encode_animation("/tmp/frames", fps=15, seconds_hint=8)
|
||||
# art == {"path": "/workspace/final.mp4", "mime": "video/mp4", "bytes": 1234567}
|
||||
|
||||
It writes the finished artifact to the TOP of /workspace (so code_exec returns a
|
||||
file_id for it) and picks the format for you:
|
||||
|
||||
* short & light -> animated GIF (fun, loops, autoplays)
|
||||
* long (many seconds+),
|
||||
OR a GIF too big to post -> H.264 MP4: yuv420p, even dimensions, +faststart
|
||||
— the combination Discord needs to PLAY it inline
|
||||
(a video whose dimensions Discord can't read is
|
||||
shown as a dead download, not a player).
|
||||
|
||||
Requires ffmpeg (always present in the sandbox). Uses gifski + gifsicle for a
|
||||
cleaner GIF when they're installed, falling back to ffmpeg's palette pipeline.
|
||||
This helper only ENCODES — you draw the frames.
|
||||
"""
|
||||
|
||||
import glob
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
|
||||
# A GIF longer than this balloons past what's worth shipping as a GIF — encode
|
||||
# straight to MP4 instead. Tune down for chattier channels.
|
||||
GIF_MAX_SECONDS = 20
|
||||
# Keep a GIF comfortably under Discord's ~10 MiB inline limit; over this we
|
||||
# re-encode to an MP4 and drop the GIF.
|
||||
GIF_MAX_BYTES = 9 * 1024 * 1024
|
||||
|
||||
|
||||
def _which(prog):
|
||||
return shutil.which(prog) is not None
|
||||
|
||||
|
||||
def _run(cmd):
|
||||
p = subprocess.run(cmd, capture_output=True, text=True)
|
||||
if p.returncode != 0:
|
||||
raise RuntimeError(f"{cmd[0]} failed:\n{p.stderr[-2000:]}")
|
||||
|
||||
|
||||
def _frames(frames_dir):
|
||||
fs = sorted(glob.glob(os.path.join(frames_dir, "frame_*.png")))
|
||||
if not fs:
|
||||
raise FileNotFoundError(
|
||||
f"no frame_*.png in {frames_dir} — render frames as "
|
||||
f"{frames_dir}/frame_00000.png, frame_00001.png, … first"
|
||||
)
|
||||
return fs
|
||||
|
||||
|
||||
def _encode_gif(frames_dir, fps, out, long_edge):
|
||||
fs = _frames(frames_dir)
|
||||
if _which("gifski"):
|
||||
# gifski takes an explicit, sorted file list — no numbering-gap surprises.
|
||||
_run(["gifski", "-o", out, "--fps", str(fps), "--quality", "90"] + fs)
|
||||
else:
|
||||
vf = (f"fps={fps},scale='min({long_edge},iw)':-1:flags=lanczos,"
|
||||
f"split[s0][s1];[s0]palettegen=stats_mode=diff[p];"
|
||||
f"[s1][p]paletteuse=dither=bayer:bayer_scale=3")
|
||||
_run(["ffmpeg", "-y", "-pattern_type", "glob",
|
||||
"-i", os.path.join(frames_dir, "frame_*.png"), "-vf", vf, out])
|
||||
if _which("gifsicle"):
|
||||
_run(["gifsicle", "-O3", "--lossy=60", out, "-o", out])
|
||||
|
||||
|
||||
def _encode_mp4(frames_dir, fps, out, long_edge):
|
||||
_frames(frames_dir) # validate presence
|
||||
# -2 forces EVEN width/height (H.264 requires it); yuv420p + faststart make
|
||||
# Discord/browsers play it inline instead of offering a download.
|
||||
vf = f"scale='min({long_edge},iw)':-2:flags=lanczos"
|
||||
_run(["ffmpeg", "-y", "-framerate", str(fps), "-pattern_type", "glob",
|
||||
"-i", os.path.join(frames_dir, "frame_*.png"),
|
||||
"-c:v", "libx264", "-pix_fmt", "yuv420p", "-crf", "23",
|
||||
"-vf", vf, "-movflags", "+faststart", out])
|
||||
|
||||
|
||||
def encode_animation(frames_dir, fps=15, seconds_hint=0, long_edge=640):
|
||||
"""Encode frames_dir → /workspace/final.{gif,mp4}. Returns {path, mime, bytes}.
|
||||
|
||||
seconds_hint: your intended runtime in seconds (falls back to frame_count/fps
|
||||
when 0). Pass it for long pieces so the format decision doesn't depend on how
|
||||
many frames you happened to render.
|
||||
long_edge: cap for the output's long side in px (keeps size/encode sane).
|
||||
"""
|
||||
n = len(_frames(frames_dir))
|
||||
seconds = seconds_hint or (n / max(fps, 1))
|
||||
gif, mp4 = "/workspace/final.gif", "/workspace/final.mp4"
|
||||
|
||||
if seconds > GIF_MAX_SECONDS:
|
||||
_encode_mp4(frames_dir, fps, mp4, long_edge)
|
||||
return _art(mp4, "video/mp4")
|
||||
|
||||
_encode_gif(frames_dir, fps, gif, long_edge)
|
||||
if os.path.getsize(gif) <= GIF_MAX_BYTES:
|
||||
return _art(gif, "image/gif")
|
||||
|
||||
# Too heavy as a GIF — hand Discord an MP4 it can actually render.
|
||||
_encode_mp4(frames_dir, fps, mp4, long_edge)
|
||||
os.remove(gif)
|
||||
return _art(mp4, "video/mp4")
|
||||
|
||||
|
||||
def _art(path, mime):
|
||||
b = os.path.getsize(path)
|
||||
print(f"[encode] {path} {mime} {b / 1024 / 1024:.2f} MiB")
|
||||
return {"path": path, "mime": mime, "bytes": b}
|
||||
Reference in New Issue
Block a user