steve/executus-skills
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request 'feat(gif): staging rails + bundled render harness' (#1) from feat/gif-staging-rails-and-harness into main

Browse Source
This commit was merged in pull request #1.
This commit is contained in:
steve
2026-07-24 23:12:20 +00:00
parent fae89a1ed0 f16ac83790
commit 0c05ec8c60
3 changed files with 568 additions and 251 deletions
Download Patch File Download Diff File Expand all files Collapse all files
gif/SKILL.md
+147 -136
View File
@@ -1,10 +1,10 @@
--- ---
name: gif name: gif
description: Make a funny animated GIF (or MP4) for Discord from a description — plans scenes, keeps recurring people/props consistent via a reusable cast, renders frames in Python, encodes cleanly, and automatically hands Discord an H.264 MP4 it can play when the piece is long or a GIF comes out too big. Use whenever someone asks to draw/animate/gif something, including multi-minute bits about people or things that happened. description: Make a funny animated GIF (or MP4) for Discord from a description — plans scenes, keeps recurring people/props consistent via a reusable cast, renders frames in Python through a bundled harness that owns the scratch/encode plumbing, and automatically hands Discord an H.264 MP4 it can play when the piece is long or a GIF comes out too big. Use whenever someone asks to draw/animate/gif something, including multi-minute bits about people or things that happened.
license: MIT license: MIT
allowed-tools: [code_exec, image_describe, send_attachments, file_get_metadata, think] allowed-tools: [code_exec, image_describe, send_attachments, file_get_metadata, think]
metadata: metadata:
version: "1.0.0" version: "3.0.0"
adapted-from: mort/skills/animate adapted-from: mort/skills/animate
--- ---
@@ -12,168 +12,179 @@ metadata:
You make HIGH-QUALITY, funny animations from a description — often caricatures You make HIGH-QUALITY, funny animations from a description — often caricatures
of the people in the channel or a bit about something that just happened. of the people in the channel or a bit about something that just happened.
Naive PIL GIF export looks bad (256-colour banding, muddy dithering) and a **Work by calling tools — do not introduce yourself or list capabilities.**
multi-minute GIF is physically absurd (hundreds of MB), so follow this recipe:
render frames in Python, let the bundled `encode.py` choose GIF vs MP4, and This skill bundles `scripts/gifsmith.py`, a render harness that owns everything
self-critique the result. **Work by calling tools — do not introduce yourself that used to break these renders: the scratch directory and its disk budget,
or list capabilities.** frame numbering, a locked canvas size, per-scene crash isolation, the GIF-vs-MP4
decision, and the critique contact sheet. **You write only the drawing.**
## 1. Plan first: length, scenes, and cast ## 1. Plan first: length, scenes, and cast
Before drawing, decide (use `think` for anything non-trivial): Decide before drawing (use `think` for anything non-trivial):
- **How long / how many scenes?** Count the distinct beats the prompt actually - **How many scenes?** Count the distinct beats the prompt actually describes and
describes, and render exactly that many. **ASSUME ONE.** A single continuous render exactly that many. **ASSUME ONE.** A single continuous action ("a
action ("a bouncing ball", "Steve slipping on a banana") is ONE scene — don't bouncing ball", "Steve slipping on a banana") is ONE scene — don't chop it into
chop it into cuts. Start a new scene only when the prompt moves to a new beat cuts. Start a new scene only when the prompt moves to a new beat (new action,
(new action, place, or time jump). There is **no upper limit** — a bit that place, or time jump). There is **no upper limit** — a bit that genuinely
genuinely describes twelve beats gets twelve scenes and runs for minutes — but describes twelve beats gets twelve scenes and runs for minutes — but **never
**never pad a simple request with invented beats**, and never compress beats pad a simple request with invented beats**, and never compress beats the user
the user clearly separated. When in doubt, fewer scenes. clearly separated. When in doubt, fewer scenes.
- **Who/what is the cast?** List every recurring character and prop (a person's - **Who is the cast?** Every recurring character and prop. For a bit "about
persona, "the dog", a logo). For a bit "about <someone>", pick a simple, <someone>", pick a simple recognizable caricature — one or two signature
recognizable caricature — one or two signature features (glasses, hair colour, features (glasses, hair colour, a hat) — and pin each cast member's look ONCE
a hat) — and pin each cast member's look ONCE as a reusable Python helper (see as a `draw_<name>(ax, ...)` helper. Every scene CALLS that helper; never
§4). That is what keeps the SAME person drawn identically across every scene re-describe a character inline, or they drift scene to scene.
instead of re-invented each cut. A one-scene GIF with a single subject needs
no cast block.
## 2. Pick the format from the length (encode.py does this for you) ## 2. STAGING — this is what separates a good GIF from a bad one
- **Short & loopable (≤ ~20 s):** a GIF — fun, loops, autoplays. The harness guarantees a *working* animation. What makes it *good* is staging,
- **Long (tens of seconds to minutes):** an **MP4**. A 3–10 minute GIF is and that is where these renders most often fail. Four rules, all checked in §5:
hundreds of MB and unusable; H.264 is the right tool and Discord plays it
inline.
- You don't hand-pick this: `encode.py`'s `encode_animation(...)` takes a
`seconds_hint` and (a) goes straight to MP4 for long pieces, and (b) for short
ones, makes the GIF but **auto-re-encodes to an MP4 if the GIF is too big for
Discord to render nicely.** It returns `{"path", "mime", "bytes"}` — deliver
whatever it hands back.
## 3. code_exec workspace — READ THIS (it is why draws fail) - **FILL THE FRAME.** The subject of a beat must be at least **one third of the
frame height** — for a close reaction, half or more. A character drawn 5% tall
in a corner under a big empty sky is the single most common failure: the viewer
can't tell what it is. Crop in. Empty sky/floor is wasted frame; if a region has
nothing happening in it, the camera is in the wrong place. Put the action in the
middle two thirds, and let characters break the frame edge rather than shrink.
- **EVERY FRAME MOVES.** Nothing sits pixel-identical for more than ~0.3 s unless
you are deliberately holding on a text beat (use `hold=` for that). Even a
"still" character breathes, sways, blinks; even a still scene has drifting
clouds or a flickering light. Long stretches of identical frames read as a
broken GIF. Animate the SUBJECT, not just one prop next to it.
- **SHOW IT, DON'T CAPTION IT.** A caption is for a punchline or a line of
dialogue — never a narration of what the animation should be depicting. If you
find yourself writing `"*CRASH* garbage everywhere!"`, draw the garbage flying
instead. Every noun in the request should appear on screen, not in text.
- **GIVE IT DEPTH.** Flat single-tone shapes look like a diagram. Cheap wins that
land every time: a soft elliptical **shadow on the ground under every
character**, a second darker tone on the shaded side of each big shape, a
background that is a gradient or has a couple of out-of-focus shapes, and clear
foreground/midground/background separation. Outline characters in a dark colour
so they pop.
The `code_exec` sandbox has three hard rules; breaking them is the #1 cause of Motion quality, in order of payoff: **ease in and out** of every move
"vanished" files and wasted budget: (`t*t*(3-2*t)`, never linear), **anticipation** (wind up before a big action),
**overshoot/settle** on impacts, **squash & stretch** on anything that bounces or
lands. A held pose that eases beats three extra scenes.
1. **`/workspace` is WIPED before every `code_exec` call.** Nothing survives ## 3. Render with the harness (ONE code_exec call)
between calls. So render → encode → write your critique still **all in ONE
call**. Never write frames in one call and read them in another.
2. **Only TOP-LEVEL files in `/workspace/` come back as `files_out` file_ids.**
A file in a subdir (`/workspace/frames/x.png`) is NOT returned. Write the
final artifact and the ONE critique still directly at the top:
`encode.py` already writes `/workspace/final.gif` or `/workspace/final.mp4`;
you add `/workspace/still.png`. Render scratch frames under `/tmp`, never
`/workspace`.
3. **Don't dump many files into `/workspace`.** Hundreds of frame PNGs blow the
`files_out` size cap and get dropped. Keep `/workspace` to just the final
artifact + one still; everything else lives under `/tmp`.
`image_describe`, `send_attachments`, `file_get_metadata` take a **file_id from Pass `scripts/gifsmith.py` in with `files_in` (its file_id is listed at the
`files_out` — never a path.** Passing `"/workspace/final.mp4"` fails bottom of this skill) as `{"name": "gifsmith.py", "file_id": "<that id>"}`, then
`file_not_found`. Read the file_id out of `files_out` and pass THAT. write ONE `code_exec` call:
## 4. Render → encode (ONE code_exec call, using encode.py) ```python
# gifsmith re-exports the drawing vocabulary, so this one import is enough:
from gifsmith import Animation, Circle, Ellipse, Rectangle, Polygon, Wedge, np
anim = Animation(width=640, height=480, fps=15) # canvas locked here
This skill bundles `scripts/encode.py`. Load it into the render call with # ---- cast: defined ONCE, called from every scene ----
`files_in` (its file_id is listed when you open this skill), then in a SINGLE def draw_steve(ax, x, y, rage=0.0):
`code_exec` call: ax.add_patch(Ellipse((x, y - 9), 14, 3, fc="#00000033", zorder=1)) # shadow
...
1. **Clear scratch** so stale frames can't leak into this encode: @anim.scene("Steve watches the printer", seconds=2.5)
`import shutil, os; shutil.rmtree('/tmp/frames', ignore_errors=True); os.makedirs('/tmp/frames')` def s0(ax, t): # t runs 0.0 -> 1.0 across the scene
2. **Render frames** with Python (matplotlib / Pillow / numpy) into `/tmp/frames` ease = t * t * (3 - 2 * t)
as `frame_00000.png, frame_00001.png, …` — **fixed 5-digit zero-pad, ONE draw_steve(ax, 30 + 8 * ease, 40, rage=t)
continuous counter across all scenes** (do NOT restart numbering per scene, or
the encoder reorders/overwrites frames).
3. **Encode** — let the helper choose GIF vs MP4:
```python
from encode import encode_animation
art = encode_animation("/tmp/frames", fps=15, seconds_hint=<intended seconds>)
print(art) # {'path': '/workspace/final.mp4', 'mime': 'video/mp4', 'bytes': ...}
```
4. **Write ONE critique still** to `/workspace/still.png` in the same call — a
representative frame, or for a multi-scene piece a COMPACT contact sheet with
one small (~200–320px) tile per scene, labelled by scene index, so the
critique can check every scene and cast consistency. Keep it low-res and
legible — it's an internal aid, not the deliverable.
The call ends with the final artifact + `still.png` at the top of `/workspace`; @anim.scene("it sprays him", seconds=2.0, hold=0.8)
`files_out` returns a file_id for each. Give `code_exec` a generous def s1(ax, t):
`timeout_seconds` (e.g. 120+) — encoding is CPU-bound, and a long piece is real draw_steve(ax, 38, 40, rage=1.0)
work, not a hang. anim.text(ax, "every single time") # boxed, bold, legible caption
### Multi-scene: define the cast ONCE, reuse it everywhere art = anim.render() # encodes, writes final.* + still.png, prints a report
print(art)
```
It all still happens in the one call — structure the program as: - `ax` is a fresh matplotlib Axes at exactly the locked pixel size, with
1. **Cast block at the top, defined ONCE:** helpers like `x in [0, 100]` and `y in [0, anim.ymax]`, origin bottom-left, no margins.
`def draw_steve(ax, x, y, pose): …` and shared constants (`PALETTE`, `SKY`). Draw in those units and every frame lines up.
2. **One function per scene** (`def scene_0(...): …`) that draws its beat by - `hold=<seconds>` freezes the scene's last frame — that is how you make a
CALLING the cast helpers — never by re-describing a character inline. caption readable (short phrases ~1.5–2 s, sentences longer).
3. **A timeline** that runs the scenes in order into `/tmp/frames` with the one - PIL instead: `@anim.scene("...", seconds=2, mode="pil")` gives
continuous counter, then encodes the whole sequence. `(img, draw, t)`. `anim.font(size)` returns a real bold TrueType face.
- Hand-built frames: `anim.add_frame(pil_image, times=n)`.
- `anim.render(long_edge=..., force="gif"|"mp4", audio="/workspace/track.wav")`.
It picks GIF vs MP4 by length, and re-encodes to MP4 if the GIF is too big for
Discord. It returns `{"path", "mime", "bytes"}`.
Because every scene calls the same `draw_*` helpers, recurring people/props come **Everything happens in ONE call** — `/workspace` is wiped before every
out pixel-identical scene to scene — the entire point of the cast block. If you `code_exec` call, so a render in one call and a read in the next finds nothing.
catch yourself re-describing a character inside a scene, hoist it into a cast That also means **`gifsmith.py` must be in `files_in` on EVERY `code_exec` call
helper. Carry shared state (positions, score, time of day) in plain variables that imports it**, including every retry — it does not survive from the last one.
between scene calls so cuts stay continuous. Give the call a generous `timeout_seconds` (120+); encoding is CPU-bound.
### Budget for LONG pieces (minutes) **A SyntaxError costs you the whole pass** — it happens before the harness can
isolate anything, and it is the one failure it cannot absorb. You are emitting a
lot of code in one shot, so keep it boring: one statement per line, short helper
signatures, no dense one-liners packing three drawing calls into a single
`add_patch`, and the same keyword names everywhere you call a helper. Read back
any line with more than about four arguments before you send it.
A 3–10 minute animation is thousands of frames. Keep it renderable: **fps Read the report `render()` prints. It tells you the frame count, the scratch
10–12**, cap the long edge ~**480–540px** (`long_edge=` in `encode_animation`), peak, and — importantly — **which scenes failed**. A scene that raises is
and favour procedural motion that's cheap per frame — don't render 9000 frames skipped, the rest still render and ship, and you fix just that scene next pass
of heavy matplotlib if a lighter draw reads the same. **If code_exec times out instead of re-deriving the whole program.
or errors mid-render you get nothing back**, so on the retry SCALE DOWN
decisively (lower fps, smaller canvas, trim hold frames, reduce the longest
scenes) — keep every beat, just render it leaner. Each retry counts toward the
3-pass cap; never burn all three on an over-ambitious render and leave the user
with nothing.
## 5. Text in GIFs — make it READABLE Long pieces (minutes): `fps=10..12`, `width=480..540`, and favour procedural
motion that is cheap per frame. If a render times out you get nothing back, so on
a retry scale down decisively — keep every beat, render it leaner.
Adding captions/labels/punchlines is fine, but pace them for a human. Motion can ## 4. Files: file_ids, never paths
stay 10–20 fps; what matters is how long each distinct TEXT STATE holds — short
phrases ~1.5–2 s, full sentences longer (budget by word count). At 15 fps that's
~25–30 repeated frames per text state, not a handful. Hold by duplicating frames,
or with PIL pass a per-frame `duration=[…]` list (long on text, short on motion).
Text flashing past unreadably is the #1 complaint — check it in the critique.
## 6. Self-critique (AT MOST 3 render passes) `render()` leaves `/workspace/final.gif` (or `.mp4`) and `/workspace/still.png` at
the TOP of `/workspace`, and `files_out` returns a **file_id** for each.
`image_describe`, `send_attachments` and `file_get_metadata` take that file_id —
passing `"/workspace/final.gif"` fails `file_not_found`. **Copy the file_id
character-for-character out of the `files_out` JSON**; do not retype it from
memory. One wrong digit is a `file_not_found` that silently costs you the whole
critique pass.
Nothing here needs `pip install` — matplotlib, PIL, numpy, ffmpeg, gifski and
gifsicle are already in the sandbox.
## 5. Self-critique (AT MOST 3 render passes)
Before delivering, LOOK at what you made: call `image_describe` on the Before delivering, LOOK at what you made: call `image_describe` on the
`still.png` **file_id** (from `files_out`, not a path) and judge it against the **`still.png` file_id** — the harness already built it as a labelled contact
request — motion, colour banding, artifacts, framing, "does it depict what was sheet, one tile per scene. Ask specifically about subject size, caption
asked", and text readability. For a multi-scene piece, judge scene by scene off readability and whether the depicted action matches the request. Then answer:
the contact sheet AND check each recurring character/prop is the SAME across
scenes (drift means that scene drew it ad hoc — fix the cast helper and
re-render). If it falls short, change something concrete and re-run the WHOLE
pipeline in one fresh `code_exec` call (workspace was wiped — rebuild from
scratch). **At most 3 render passes total**; for a long piece be more decisive
(1–2 passes) since each re-render is expensive. After the 3rd, deliver the best
version with a one-line note on any trade-off.
## 7. Deliver 1. Does it depict what was asked — every named character, prop and action?
2. Is the subject **big enough to read** (≥ ⅓ frame height)?
3. Is any caption legible, and does it hold long enough?
4. Does every recurring character look the SAME across scenes?
5. Any scene the report flagged as failed, colour banding, clipped shapes,
characters floating with no shadow?
Deliver the final artifact with a SINGLE `send_attachments` call using **A "no" on 1, 2 or 3 means you re-render** — those three are what makes a GIF
`channel_id` from your platform context and the **file_id that land or flop, and "it's basically fine" is how a bad one ships. Change something
`encode_animation` produced** (read it from that render call's `files_out` by concrete and re-run the whole program in one fresh `code_exec` call. **At most 3
matching `final.gif` / `final.mp4` — NOT the still, NOT a path). Finish with a render passes**; for a long piece be decisive (1–2). After the 3rd, deliver the
one-line caption. Critique aids (contact sheets, stills) are INTERNAL — never best version with a one-line note on the trade-off.
attach them unless the user explicitly asked.
## 6. Deliver
ONE `send_attachments` call with `channel_id` from your platform context and the
file_id of `final.gif` / `final.mp4` — NOT the still, NOT a path. Finish with a
one-line caption. The contact sheet is INTERNAL; never attach it unless asked.
## Forbidden ## Forbidden
- NEVER pass a `/workspace/...` path to image_describe / send_attachments / - Never pass a `/workspace/...` path where a file_id is wanted, and never retype
file_get_metadata — they take a file_id from `files_out`. a file_id from memory — copy it from `files_out`.
- Do not write the final artifact into a `/workspace` subdir — top level only. - Do not hand-roll the scratch dir, the frame loop, the frame numbering, the
- Do not split render and read across separate code_exec calls — `/workspace` is contact sheet, or the encode — that is what the harness is for. In particular
wiped between them. do not render frames into `/tmp` (a 16 MiB tmpfs) or into `/workspace` itself.
- Do not hand-roll the encode/format decision — call `encode_animation`; it - Do not split render and read across separate code_exec calls, and do not
guarantees a Discord-playable MP4 (yuv420p, even dims, faststart) when needed. assume `gifsmith.py` is still in `/workspace` from the last call — pass it in
- Do not restart the frame counter per scene — one continuous 5-digit counter. `files_in` every time.
- Do not re-describe a recurring character inline per scene — one shared cast - Do not re-describe a recurring character inline per scene — one cast helper.
helper so it stays identical. - Do not invent beats the user didn't ask for — assume ONE scene.
- Do not invent beats the user didn't ask for — assume ONE scene, add more only - Do not narrate the action in a caption instead of animating it.
for beats the prompt describes (no cap, but no padding). - Do not ship a subject too small to recognise, or a stretch of identical frames.
- Do not exceed 3 render passes; do not ship an empty/broken result without - Do not exceed 3 render passes; do not ship an empty/broken result without
saying so. saying so.
gif/scripts/encode.py
-115
View File
@@ -1,115 +0,0 @@
"""
encode.py — turn a directory of PNG frames into a Discord-friendly GIF or MP4.
Use it INSIDE a single code_exec call. Render your frames to a scratch dir as
frame_00000.png, frame_00001.png, … (fixed 5-digit width, ONE continuous
counter across all scenes), then:
from encode import encode_animation
art = encode_animation("/tmp/frames", fps=15, seconds_hint=8)
# art == {"path": "/workspace/final.mp4", "mime": "video/mp4", "bytes": 1234567}
It writes the finished artifact to the TOP of /workspace (so code_exec returns a
file_id for it) and picks the format for you:
* short & light -> animated GIF (fun, loops, autoplays)
* long (many seconds+),
OR a GIF too big to post -> H.264 MP4: yuv420p, even dimensions, +faststart
— the combination Discord needs to PLAY it inline
(a video whose dimensions Discord can't read is
shown as a dead download, not a player).
Requires ffmpeg (always present in the sandbox). Uses gifski + gifsicle for a
cleaner GIF when they're installed, falling back to ffmpeg's palette pipeline.
This helper only ENCODES — you draw the frames.
"""
import glob
import os
import shutil
import subprocess
# A GIF longer than this balloons past what's worth shipping as a GIF — encode
# straight to MP4 instead. Tune down for chattier channels.
GIF_MAX_SECONDS = 20
# Keep a GIF comfortably under Discord's ~10 MiB inline limit; over this we
# re-encode to an MP4 and drop the GIF.
GIF_MAX_BYTES = 9 * 1024 * 1024
def _which(prog):
return shutil.which(prog) is not None
def _run(cmd):
p = subprocess.run(cmd, capture_output=True, text=True)
if p.returncode != 0:
raise RuntimeError(f"{cmd[0]} failed:\n{p.stderr[-2000:]}")
def _frames(frames_dir):
fs = sorted(glob.glob(os.path.join(frames_dir, "frame_*.png")))
if not fs:
raise FileNotFoundError(
f"no frame_*.png in {frames_dir} — render frames as "
f"{frames_dir}/frame_00000.png, frame_00001.png, … first"
)
return fs
def _encode_gif(frames_dir, fps, out, long_edge):
fs = _frames(frames_dir)
if _which("gifski"):
# gifski takes an explicit, sorted file list — no numbering-gap surprises.
_run(["gifski", "-o", out, "--fps", str(fps), "--quality", "90"] + fs)
else:
vf = (f"fps={fps},scale='min({long_edge},iw)':-1:flags=lanczos,"
f"split[s0][s1];[s0]palettegen=stats_mode=diff[p];"
f"[s1][p]paletteuse=dither=bayer:bayer_scale=3")
_run(["ffmpeg", "-y", "-pattern_type", "glob",
"-i", os.path.join(frames_dir, "frame_*.png"), "-vf", vf, out])
if _which("gifsicle"):
_run(["gifsicle", "-O3", "--lossy=60", out, "-o", out])
def _encode_mp4(frames_dir, fps, out, long_edge):
_frames(frames_dir) # validate presence
# -2 forces EVEN width/height (H.264 requires it); yuv420p + faststart make
# Discord/browsers play it inline instead of offering a download.
vf = f"scale='min({long_edge},iw)':-2:flags=lanczos"
_run(["ffmpeg", "-y", "-framerate", str(fps), "-pattern_type", "glob",
"-i", os.path.join(frames_dir, "frame_*.png"),
"-c:v", "libx264", "-pix_fmt", "yuv420p", "-crf", "23",
"-vf", vf, "-movflags", "+faststart", out])
def encode_animation(frames_dir, fps=15, seconds_hint=0, long_edge=640):
"""Encode frames_dir → /workspace/final.{gif,mp4}. Returns {path, mime, bytes}.
seconds_hint: your intended runtime in seconds (falls back to frame_count/fps
when 0). Pass it for long pieces so the format decision doesn't depend on how
many frames you happened to render.
long_edge: cap for the output's long side in px (keeps size/encode sane).
"""
n = len(_frames(frames_dir))
seconds = seconds_hint or (n / max(fps, 1))
gif, mp4 = "/workspace/final.gif", "/workspace/final.mp4"
if seconds > GIF_MAX_SECONDS:
_encode_mp4(frames_dir, fps, mp4, long_edge)
return _art(mp4, "video/mp4")
_encode_gif(frames_dir, fps, gif, long_edge)
if os.path.getsize(gif) <= GIF_MAX_BYTES:
return _art(gif, "image/gif")
# Too heavy as a GIF — hand Discord an MP4 it can actually render.
_encode_mp4(frames_dir, fps, mp4, long_edge)
os.remove(gif)
return _art(mp4, "video/mp4")
def _art(path, mime):
b = os.path.getsize(path)
print(f"[encode] {path} {mime} {b / 1024 / 1024:.2f} MiB")
return {"path": path, "mime": mime, "bytes": b}
gif/scripts/gifsmith.py
+421
View File
@@ -0,0 +1,421 @@
"""
gifsmith.py — the render harness for the `gif` skill.
You write ONLY the drawing. This module owns every part that has historically
broken renders: the scratch directory and its disk budget, frame naming and the
continuous counter, a LOCKED canvas size (frames that differ by one pixel break
the encoder), per-scene crash isolation, the GIF-vs-MP4 decision, and the
critique contact sheet.
from gifsmith import Animation
anim = Animation(width=640, height=480, fps=15)
# --- cast: define each recurring character ONCE ---
def draw_steve(ax, x, y, angry=0.0):
...
@anim.scene("Steve watches the printer", seconds=2.5)
def s0(ax, t): # t goes 0.0 -> 1.0 across the scene
draw_steve(ax, 2, 1, angry=t)
art = anim.render() # -> {"path": "/workspace/final.gif", "mime": ..., "bytes": ...}
`ax` is a fresh matplotlib Axes at exactly (width, height) pixels, x in [0,100],
y in [0, 100*height/width], origin bottom-left, no margins. Draw in those units
and every frame lines up.
PIL instead of matplotlib:
@anim.scene("title card", seconds=1.5, mode="pil")
def s1(img, draw, t): # img: RGB Image at the locked size; draw: ImageDraw
draw.text((40, 40), "THE END", fill=(255, 255, 255), font=anim.font(48))
render() prints a report and returns the artifact dict. If one scene raises, the
OTHER scenes still render and ship — the report names the broken one so you can
fix just that scene. It also writes /workspace/still.png: a labelled contact
sheet, one tile per scene, for the critique pass.
"""
import glob
import math
import os
import shutil
import subprocess
import sys
import traceback
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from PIL import Image, ImageDraw, ImageFont # noqa: E402
# Re-exported so one import line covers the whole drawing vocabulary — a missing
# `from matplotlib.patches import ...` is otherwise a NameError that costs a pass.
from matplotlib.patches import ( # noqa: E402,F401
Arc, Circle, Ellipse, FancyBboxPatch, Polygon, Rectangle, Wedge)
import numpy as np # noqa: E402,F401
# A GIF longer than this balloons past what's worth shipping as a GIF.
GIF_MAX_SECONDS = 20
# Keep a GIF comfortably under Discord's ~10 MiB inline limit.
GIF_MAX_BYTES = 9 * 1024 * 1024
# Measured on busy 640x480 matplotlib frames in this sandbox: ~86 KiB per PNG at
# compress_level=1, i.e. 88064 / 307200 px. This decides whether a render streams
# instead of staging frames on disk, so it must not read low — erring low
# reintroduces the ENOSPC this harness exists to prevent, and erring high only
# turns a borderline short piece into an MP4.
PNG_BYTES_PER_PIXEL = 0.29
_FONT_CANDIDATES = [
"/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf",
"/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf",
]
def _which(prog):
return shutil.which(prog) is not None
def _free_mb(path):
try:
st = os.statvfs(path)
return st.f_bavail * st.f_frsize / 1024 / 1024
except OSError:
return 0.0
def _pick_scratch():
"""Largest writable scratch dir. /tmp is a small tmpfs in this sandbox; a
SUBDIR of /workspace is bigger and is not returned in files_out (only
TOP-LEVEL /workspace files are), so it is safe to fill with frames."""
best, best_free = None, -1.0
for base in ("/workspace/.frames", "/tmp/frames", "/var/tmp/frames"):
parent = os.path.dirname(base)
if not os.path.isdir(parent) or not os.access(parent, os.W_OK):
continue
free = _free_mb(parent)
if free > best_free:
best, best_free = base, free
if best is None:
raise RuntimeError("no writable scratch directory found")
shutil.rmtree(best, ignore_errors=True)
os.makedirs(best, exist_ok=True)
return best, best_free
class _Scene:
def __init__(self, fn, title, seconds, mode, hold):
self.fn, self.title, self.seconds = fn, title, float(seconds),
self.mode, self.hold = mode, float(hold)
self.frames = 0
self.error = None
self.first_frame = None
class Animation:
"""Collects scenes, renders them to a locked-size frame sequence, encodes."""
def __init__(self, width=640, height=480, fps=15, dpi=100, bg="white"):
# H.264 needs even dimensions; lock them here so the encoder never fails.
self.width = int(width) // 2 * 2
self.height = int(height) // 2 * 2
self.fps = int(fps)
self.dpi = int(dpi)
self.bg = bg
self.xmax = 100.0
self.ymax = 100.0 * self.height / self.width
self._scenes = []
self._count = 0
self._scratch = None
self._peak_mb = 0.0
self._streaming = False
self._pipe = None
# ---------------------------------------------------------------- authoring
def scene(self, title, seconds=2.0, mode="mpl", hold=0.0):
"""Decorator: register a scene. `hold` freezes the last frame for that
many extra seconds — use it to let a caption stay readable."""
def deco(fn):
self._scenes.append(_Scene(fn, title, seconds, mode, hold))
return fn
return deco
def font(self, size):
for p in _FONT_CANDIDATES:
if os.path.exists(p):
return ImageFont.truetype(p, size)
return ImageFont.load_default()
def text(self, ax, s, size=18, y=None, color="white", box="black"):
"""Caption helper: centred, boxed, legible at GIF resolution."""
ax.text(self.xmax / 2, self.ymax * 0.06 if y is None else y, s,
ha="center", va="center", fontsize=size, color=color, zorder=999,
bbox=dict(boxstyle="round,pad=0.4", fc=box, ec="none", alpha=0.75))
def add_frame(self, img, times=1):
"""Escape hatch for hand-built frames (PIL compositing, pasted photos,
anything the scene decorators can't express). Still size-locked."""
for _ in range(max(1, int(times))):
self._emit(img)
# ------------------------------------------------------------------ drawing
def _new_axes(self):
fig = plt.figure(figsize=(self.width / self.dpi, self.height / self.dpi),
dpi=self.dpi)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, self.xmax)
ax.set_ylim(0, self.ymax)
ax.set_axis_off()
ax.set_facecolor(self.bg)
fig.patch.set_facecolor(self.bg)
return fig, ax
def _emit(self, img):
"""Emit one frame at exactly the locked size. Never bbox_inches='tight'.
Short pieces go to PNG files (gifski wants a file list). Long pieces are
piped straight into ffmpeg so a multi-minute render never touches disk —
the sandbox scratch would otherwise cap you at well under a minute."""
if img.size != (self.width, self.height):
img = img.resize((self.width, self.height), Image.LANCZOS)
img = img.convert("RGB")
if self._pipe is not None:
img.save(self._pipe.stdin, format="PNG", compress_level=1)
else:
img.save(os.path.join(self._scratch, f"frame_{self._count:05d}.png"),
compress_level=1)
self._count += 1
def _render_mpl(self, sc, t):
fig, ax = self._new_axes()
try:
sc.fn(ax, t)
fig.canvas.draw()
img = Image.frombuffer(
"RGBA", fig.canvas.get_width_height(),
fig.canvas.buffer_rgba(), "raw", "RGBA", 0, 1).convert("RGB")
finally:
plt.close(fig)
return img
def _render_pil(self, sc, t):
img = Image.new("RGB", (self.width, self.height),
self.bg if isinstance(self.bg, (str, tuple)) else "white")
sc.fn(img, ImageDraw.Draw(img), t)
return img
# ------------------------------------------------------------------- render
def render(self, long_edge=None, force=None, audio=None):
if not self._scenes:
raise RuntimeError("no scenes registered — decorate at least one "
"function with @anim.scene(...)")
self._scratch, free_mb = _pick_scratch()
long_edge = long_edge or max(self.width, self.height)
total_s = sum(s.seconds + s.hold for s in self._scenes)
est_frames = int(total_s * self.fps)
est_mb = est_frames * self.width * self.height * PNG_BYTES_PER_PIXEL / 1048576
# A piece too long for a GIF is an MP4 anyway, and an MP4 can be encoded
# in one streaming pass — so pipe the frames into ffmpeg and keep NOTHING
# on disk. That is what makes multi-minute pieces possible at all: the
# sandbox scratch tops out around a minute of busy frames.
self._streaming = (force == "mp4" or audio is not None
or (force != "gif" and (total_s > GIF_MAX_SECONDS
or est_mb > free_mb * 0.6)))
self._pipe = None
if self._streaming:
self._pipe = self._open_pipe("/workspace/final.mp4", long_edge, audio)
print(f"[gifsmith] streaming to MP4 (projected {est_mb:.0f} MiB of "
f"frames vs {free_mb:.0f} MiB scratch) — nothing hits disk",
flush=True)
elif est_mb > free_mb * 0.6:
print(f"[gifsmith] WARNING projected frames ~{est_mb:.0f} MiB vs "
f"{free_mb:.0f} MiB free in {os.path.dirname(self._scratch)} — "
f"dropping fps or canvas would be safer", file=sys.stderr)
print(f"[gifsmith] {len(self._scenes)} scene(s), {total_s:.1f}s at "
f"{self.fps}fps, canvas {self.width}x{self.height}", flush=True)
for i, sc in enumerate(self._scenes):
start = self._count
n = max(1, int(round(sc.seconds * self.fps)))
try:
for k in range(n):
t = k / max(n - 1, 1)
img = (self._render_pil if sc.mode == "pil" else self._render_mpl)(sc, t)
if sc.first_frame is None and k >= n // 2:
sc.first_frame = img.copy()
self._emit(img)
for _ in range(int(round(sc.hold * self.fps))):
self._emit(img)
except Exception: # noqa: BLE001
sc.error = traceback.format_exc(limit=6)
if not self._streaming:
# Drop this scene's partial frames so it can't ship half-drawn.
# (Streamed frames are already in the encoder — a broken scene
# leaves its partial beat in the MP4; the report says so.)
for p in sorted(glob.glob(
os.path.join(self._scratch, "frame_*.png")))[start:]:
os.remove(p)
self._count = start
print(f"[gifsmith] SCENE {i} ({sc.title!r}) FAILED — skipped, "
f"other scenes continue:\n{sc.error}", file=sys.stderr, flush=True)
continue
sc.frames = self._count - start
print(f"[gifsmith] scene {i}: {sc.frames} frames {sc.title!r}", flush=True)
self._renumber_if_needed()
used = sum(os.path.getsize(p) for p in
glob.glob(os.path.join(self._scratch, "frame_*.png")))
self._peak_mb = used / 1024 / 1024
if self._count == 0:
raise RuntimeError(
"every scene failed — see the per-scene tracebacks above; "
"nothing was encoded. Fix the error they all share (usually one "
"cast helper called with a keyword it does not take, or a name "
"that differs by a character) and re-run the whole program in a "
"fresh code_exec call, remembering gifsmith.py in files_in.")
try:
art = self._encode(total_s, long_edge, force, audio)
except Exception: # noqa: BLE001
# A half-written final.* at the top of /workspace still comes back as
# a files_out file_id, and a broken artifact that LOOKS deliverable is
# worse than none — the agent would ship it. Take it with us.
for leftover in ("/workspace/final.mp4", "/workspace/final.gif"):
if os.path.exists(leftover):
os.remove(leftover)
shutil.rmtree(self._scratch, ignore_errors=True)
raise
self._contact_sheet()
broken = [f"{i}:{s.title!r}" for i, s in enumerate(self._scenes) if s.error]
print(f"[gifsmith] {self._count} frames, scratch "
+ ("streamed (0 MiB on disk)" if self._streaming
else f"peak {self._peak_mb:.1f} MiB"))
print(f"[gifsmith] scene errors: {', '.join(broken) if broken else 'none'}")
print(f"[gifsmith] ART {art}")
print("[gifsmith] still: /workspace/still.png (contact sheet — call "
"image_describe on ITS file_id from files_out)")
shutil.rmtree(self._scratch, ignore_errors=True)
return art
def _renumber_if_needed(self):
"""A skipped scene leaves no gaps because _count rewinds; kept as a
no-op hook so the counter contract is explicit."""
return
# ------------------------------------------------------------------ encoding
def _open_pipe(self, out, long_edge, audio):
"""ffmpeg reading a PNG stream on stdin — the streaming MP4 path.
stderr goes to a FILE, not a pipe: nobody drains a stderr pipe until the
encode is done, so a chatty ffmpeg would fill the 64 KiB pipe buffer,
block, stop reading stdin, and deadlock the render halfway through."""
cmd = ["ffmpeg", "-y", "-loglevel", "error",
"-f", "image2pipe", "-framerate", str(self.fps), "-i", "-"]
if audio:
cmd += ["-i", audio, "-c:a", "aac", "-b:a", "128k", "-shortest"]
cmd += ["-c:v", "libx264", "-pix_fmt", "yuv420p", "-crf", "23",
"-vf", f"scale='min({long_edge},iw)':-2:flags=lanczos",
"-movflags", "+faststart", out]
self._pipe_err = open(os.path.join(self._scratch, "ffmpeg.log"), "w+b")
return subprocess.Popen(cmd, stdin=subprocess.PIPE,
stdout=subprocess.DEVNULL, stderr=self._pipe_err)
def _close_pipe(self):
try:
self._pipe.stdin.close()
except OSError:
# ffmpeg already died; its exit code + log below say why.
pass
rc = self._pipe.wait()
self._pipe_err.seek(0)
err = self._pipe_err.read().decode(errors="replace")
self._pipe_err.close()
self._pipe = None
if rc != 0:
raise RuntimeError(f"ffmpeg (streaming) exited {rc}:\n{err[-2000:]}")
def _encode(self, seconds, long_edge, force, audio=None):
gif, mp4 = "/workspace/final.gif", "/workspace/final.mp4"
if self._streaming:
self._close_pipe()
return _art(mp4, "video/mp4")
fs = sorted(glob.glob(os.path.join(self._scratch, "frame_*.png")))
want_mp4 = (force == "mp4" or audio is not None
or (force != "gif" and seconds > GIF_MAX_SECONDS))
if want_mp4:
self._mp4(mp4, long_edge, audio)
return _art(mp4, "video/mp4")
self._gif(fs, gif, long_edge)
if os.path.getsize(gif) <= GIF_MAX_BYTES:
return _art(gif, "image/gif")
print(f"[gifsmith] GIF {os.path.getsize(gif)/1048576:.1f} MiB is too big "
f"for Discord — re-encoding as MP4", flush=True)
self._mp4(mp4, long_edge, audio)
os.remove(gif)
return _art(mp4, "video/mp4")
def _gif(self, fs, out, long_edge):
if _which("gifski"):
_run(["gifski", "-o", out, "--fps", str(self.fps), "--quality", "90",
"--width", str(min(long_edge, self.width))] + fs)
else:
vf = (f"fps={self.fps},scale='min({long_edge},iw)':-1:flags=lanczos,"
f"split[s0][s1];[s0]palettegen=stats_mode=diff[p];"
f"[s1][p]paletteuse=dither=bayer:bayer_scale=3")
_run(["ffmpeg", "-y", "-loglevel", "error", "-pattern_type", "glob",
"-i", os.path.join(self._scratch, "frame_*.png"), "-vf", vf, out])
if _which("gifsicle"):
_run(["gifsicle", "-O3", "--lossy=60", out, "-o", out])
def _mp4(self, out, long_edge, audio=None):
cmd = ["ffmpeg", "-y", "-loglevel", "error", "-framerate", str(self.fps),
"-pattern_type", "glob",
"-i", os.path.join(self._scratch, "frame_*.png")]
if audio:
cmd += ["-i", audio, "-c:a", "aac", "-b:a", "128k", "-shortest"]
cmd += ["-c:v", "libx264", "-pix_fmt", "yuv420p", "-crf", "23",
"-vf", f"scale='min({long_edge},iw)':-2:flags=lanczos",
"-movflags", "+faststart", out]
_run(cmd)
# ------------------------------------------------------------- critique aid
def _contact_sheet(self, path="/workspace/still.png"):
tiles = [(i, s) for i, s in enumerate(self._scenes) if s.first_frame is not None]
if not tiles:
return
if len(tiles) == 1:
tiles[0][1].first_frame.save(path)
return
cols = min(3, len(tiles))
rows = math.ceil(len(tiles) / cols)
tw = 300
th = max(1, round(self.height * tw / self.width))
sheet = Image.new("RGB", (cols * tw, rows * (th + 20)), (20, 20, 24))
d = ImageDraw.Draw(sheet)
f = self.font(14)
for k, (i, sc) in enumerate(tiles):
x, y = (k % cols) * tw, (k // cols) * (th + 20)
sheet.paste(sc.first_frame.resize((tw, th), Image.LANCZOS), (x, y + 20))
d.text((x + 4, y + 3), f"scene {i}: {sc.title[:38]}", fill=(235, 235, 240), font=f)
sheet.save(path)
def _run(cmd):
p = subprocess.run(cmd, capture_output=True, text=True)
if p.returncode != 0:
raise RuntimeError(f"{cmd[0]} failed:\n{p.stderr[-2000:]}")
def _art(path, mime):
b = os.path.getsize(path)
print(f"[gifsmith] encoded {path} {mime} {b / 1024 / 1024:.2f} MiB")
return {"path": path, "mime": mime, "bytes": b}