steve/executus-skills
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(gif): staging rails + bundled render harness

Browse Source 
Two changes, both driven by measurement against 128 production gifsmith runs
and a local replica of the agent loop (docs/audits/2026-07-24-gifsmith-quality.md
in mort).

Staging rules. The renders that "worked" still read badly, always the same four
ways: the subject drawn 5-10% of the frame height under an empty sky, long runs
of pixel-identical frames, captions narrating action that was never drawn, and
flat untextured shapes. The recipe never asked for anything else. SKILL.md now
states four checkable rules — fill the frame, every frame moves, show it don't
caption it, give it depth — plus easing/anticipation/squash-and-stretch, and the
self-critique makes a "no" on depiction, subject size or caption legibility a
mandatory re-render instead of a suggestion.

Render harness. 26% of production code_exec calls end in a traceback, and every
crash class lives in plumbing the agent re-derives each run: scratch dir, frame
loop, numbering, canvas size, contact sheet, encode. scripts/gifsmith.py owns all
of it and the agent writes only @anim.scene drawing functions. A scene that
raises is now skipped while the others still render and ship, the canvas is
size-locked so ffmpeg cannot fail on jittering frames, and the drawing vocabulary
is re-exported so a missing import cannot NameError.

The harness also unblocks the multi-minute pieces this skill advertises but could
never produce: /tmp is a 16 MiB tmpfs (~13s of frames) and /workspace 64 MiB
(~51s), which is why "No space left on device" is 15% of all production crashes.
Anything past the GIF length threshold is an MP4, and an MP4 encodes in one
streaming pass, so those frames now go straight into ffmpeg's stdin and never
touch disk — verified at 1350 frames / 90 s with 0 MiB on disk.

encode.py is removed; the harness supersedes it.
This commit is contained in:
steve
2026-07-24 18:49:11 -04:00
parent fae89a1ed0
commit 7906a1ab4c
3 changed files with 541 additions and 251 deletions
Download Patch File Download Diff File Expand all files Collapse all files
gif/SKILL.md
+147 -136
View File
@@ -1,10 +1,10 @@
---
name: gif
description: Make a funny animated GIF (or MP4) for Discord from a description — plans scenes, keeps recurring people/props consistent via a reusable cast, renders frames in Python, encodes cleanly, and automatically hands Discord an H.264 MP4 it can play when the piece is long or a GIF comes out too big. Use whenever someone asks to draw/animate/gif something, including multi-minute bits about people or things that happened.
description: Make a funny animated GIF (or MP4) for Discord from a description — plans scenes, keeps recurring people/props consistent via a reusable cast, renders frames in Python through a bundled harness that owns the scratch/encode plumbing, and automatically hands Discord an H.264 MP4 it can play when the piece is long or a GIF comes out too big. Use whenever someone asks to draw/animate/gif something, including multi-minute bits about people or things that happened.
license: MIT
allowed-tools: [code_exec, image_describe, send_attachments, file_get_metadata, think]
metadata:
version: "1.0.0"
version: "3.0.0"
adapted-from: mort/skills/animate
---
@@ -12,168 +12,179 @@ metadata:
You make HIGH-QUALITY, funny animations from a description — often caricatures
of the people in the channel or a bit about something that just happened.
Naive PIL GIF export looks bad (256-colour banding, muddy dithering) and a
multi-minute GIF is physically absurd (hundreds of MB), so follow this recipe:
render frames in Python, let the bundled `encode.py` choose GIF vs MP4, and
self-critique the result. **Work by calling tools — do not introduce yourself
or list capabilities.**
**Work by calling tools — do not introduce yourself or list capabilities.**
This skill bundles `scripts/gifsmith.py`, a render harness that owns everything
that used to break these renders: the scratch directory and its disk budget,
frame numbering, a locked canvas size, per-scene crash isolation, the GIF-vs-MP4
decision, and the critique contact sheet. **You write only the drawing.**
## 1. Plan first: length, scenes, and cast
Before drawing, decide (use `think` for anything non-trivial):
Decide before drawing (use `think` for anything non-trivial):
- **How long / how many scenes?** Count the distinct beats the prompt actually
describes, and render exactly that many. **ASSUME ONE.** A single continuous
action ("a bouncing ball", "Steve slipping on a banana") is ONE scene — don't
chop it into cuts. Start a new scene only when the prompt moves to a new beat
(new action, place, or time jump). There is **no upper limit** — a bit that
genuinely describes twelve beats gets twelve scenes and runs for minutes — but
**never pad a simple request with invented beats**, and never compress beats
the user clearly separated. When in doubt, fewer scenes.
- **Who/what is the cast?** List every recurring character and prop (a person's
persona, "the dog", a logo). For a bit "about <someone>", pick a simple,
recognizable caricature — one or two signature features (glasses, hair colour,
a hat) — and pin each cast member's look ONCE as a reusable Python helper (see
§4). That is what keeps the SAME person drawn identically across every scene
instead of re-invented each cut. A one-scene GIF with a single subject needs
no cast block.
- **How many scenes?** Count the distinct beats the prompt actually describes and
render exactly that many. **ASSUME ONE.** A single continuous action ("a
bouncing ball", "Steve slipping on a banana") is ONE scene — don't chop it into
cuts. Start a new scene only when the prompt moves to a new beat (new action,
place, or time jump). There is **no upper limit** — a bit that genuinely
describes twelve beats gets twelve scenes and runs for minutes — but **never
pad a simple request with invented beats**, and never compress beats the user
clearly separated. When in doubt, fewer scenes.
- **Who is the cast?** Every recurring character and prop. For a bit "about
<someone>", pick a simple recognizable caricature — one or two signature
features (glasses, hair colour, a hat) — and pin each cast member's look ONCE
as a `draw_<name>(ax, ...)` helper. Every scene CALLS that helper; never
re-describe a character inline, or they drift scene to scene.
## 2. Pick the format from the length (encode.py does this for you)
## 2. STAGING — this is what separates a good GIF from a bad one
- **Short & loopable (≤ ~20 s):** a GIF — fun, loops, autoplays.
- **Long (tens of seconds to minutes):** an **MP4**. A 3–10 minute GIF is
hundreds of MB and unusable; H.264 is the right tool and Discord plays it
inline.
- You don't hand-pick this: `encode.py`'s `encode_animation(...)` takes a
`seconds_hint` and (a) goes straight to MP4 for long pieces, and (b) for short
ones, makes the GIF but **auto-re-encodes to an MP4 if the GIF is too big for
Discord to render nicely.** It returns `{"path", "mime", "bytes"}` — deliver
whatever it hands back.
The harness guarantees a *working* animation. What makes it *good* is staging,
and that is where these renders most often fail. Four rules, all checked in §5:
## 3. code_exec workspace — READ THIS (it is why draws fail)
- **FILL THE FRAME.** The subject of a beat must be at least **one third of the
frame height** — for a close reaction, half or more. A character drawn 5% tall
in a corner under a big empty sky is the single most common failure: the viewer
can't tell what it is. Crop in. Empty sky/floor is wasted frame; if a region has
nothing happening in it, the camera is in the wrong place. Put the action in the
middle two thirds, and let characters break the frame edge rather than shrink.
- **EVERY FRAME MOVES.** Nothing sits pixel-identical for more than ~0.3 s unless
you are deliberately holding on a text beat (use `hold=` for that). Even a
"still" character breathes, sways, blinks; even a still scene has drifting
clouds or a flickering light. Long stretches of identical frames read as a
broken GIF. Animate the SUBJECT, not just one prop next to it.
- **SHOW IT, DON'T CAPTION IT.** A caption is for a punchline or a line of
dialogue — never a narration of what the animation should be depicting. If you
find yourself writing `"*CRASH* garbage everywhere!"`, draw the garbage flying
instead. Every noun in the request should appear on screen, not in text.
- **GIVE IT DEPTH.** Flat single-tone shapes look like a diagram. Cheap wins that
land every time: a soft elliptical **shadow on the ground under every
character**, a second darker tone on the shaded side of each big shape, a
background that is a gradient or has a couple of out-of-focus shapes, and clear
foreground/midground/background separation. Outline characters in a dark colour
so they pop.
The `code_exec` sandbox has three hard rules; breaking them is the #1 cause of
"vanished" files and wasted budget:
Motion quality, in order of payoff: **ease in and out** of every move
(`t*t*(3-2*t)`, never linear), **anticipation** (wind up before a big action),
**overshoot/settle** on impacts, **squash & stretch** on anything that bounces or
lands. A held pose that eases beats three extra scenes.
1. **`/workspace` is WIPED before every `code_exec` call.** Nothing survives
between calls. So render → encode → write your critique still **all in ONE
call**. Never write frames in one call and read them in another.
2. **Only TOP-LEVEL files in `/workspace/` come back as `files_out` file_ids.**
A file in a subdir (`/workspace/frames/x.png`) is NOT returned. Write the
final artifact and the ONE critique still directly at the top:
`encode.py` already writes `/workspace/final.gif` or `/workspace/final.mp4`;
you add `/workspace/still.png`. Render scratch frames under `/tmp`, never
`/workspace`.
3. **Don't dump many files into `/workspace`.** Hundreds of frame PNGs blow the
`files_out` size cap and get dropped. Keep `/workspace` to just the final
artifact + one still; everything else lives under `/tmp`.
## 3. Render with the harness (ONE code_exec call)
`image_describe`, `send_attachments`, `file_get_metadata` take a **file_id from
`files_out` — never a path.** Passing `"/workspace/final.mp4"` fails
`file_not_found`. Read the file_id out of `files_out` and pass THAT.
Pass `scripts/gifsmith.py` in with `files_in` (its file_id is listed at the
bottom of this skill) as `{"name": "gifsmith.py", "file_id": "<that id>"}`, then
write ONE `code_exec` call:
## 4. Render → encode (ONE code_exec call, using encode.py)
```python
# gifsmith re-exports the drawing vocabulary, so this one import is enough:
from gifsmith import Animation, Circle, Ellipse, Rectangle, Polygon, Wedge, np
anim = Animation(width=640, height=480, fps=15) # canvas locked here
This skill bundles `scripts/encode.py`. Load it into the render call with
`files_in` (its file_id is listed when you open this skill), then in a SINGLE
`code_exec` call:
# ---- cast: defined ONCE, called from every scene ----
def draw_steve(ax, x, y, rage=0.0):
ax.add_patch(Ellipse((x, y - 9), 14, 3, fc="#00000033", zorder=1)) # shadow
...
1. **Clear scratch** so stale frames can't leak into this encode:
`import shutil, os; shutil.rmtree('/tmp/frames', ignore_errors=True); os.makedirs('/tmp/frames')`
2. **Render frames** with Python (matplotlib / Pillow / numpy) into `/tmp/frames`
as `frame_00000.png, frame_00001.png, …` — **fixed 5-digit zero-pad, ONE
continuous counter across all scenes** (do NOT restart numbering per scene, or
the encoder reorders/overwrites frames).
3. **Encode** — let the helper choose GIF vs MP4:
```python
from encode import encode_animation
art = encode_animation("/tmp/frames", fps=15, seconds_hint=<intended seconds>)
print(art) # {'path': '/workspace/final.mp4', 'mime': 'video/mp4', 'bytes': ...}
```
4. **Write ONE critique still** to `/workspace/still.png` in the same call — a
representative frame, or for a multi-scene piece a COMPACT contact sheet with
one small (~200–320px) tile per scene, labelled by scene index, so the
critique can check every scene and cast consistency. Keep it low-res and
legible — it's an internal aid, not the deliverable.
@anim.scene("Steve watches the printer", seconds=2.5)
def s0(ax, t): # t runs 0.0 -> 1.0 across the scene
ease = t * t * (3 - 2 * t)
draw_steve(ax, 30 + 8 * ease, 40, rage=t)
The call ends with the final artifact + `still.png` at the top of `/workspace`;
`files_out` returns a file_id for each. Give `code_exec` a generous
`timeout_seconds` (e.g. 120+) — encoding is CPU-bound, and a long piece is real
work, not a hang.
@anim.scene("it sprays him", seconds=2.0, hold=0.8)
def s1(ax, t):
draw_steve(ax, 38, 40, rage=1.0)
anim.text(ax, "every single time") # boxed, bold, legible caption
### Multi-scene: define the cast ONCE, reuse it everywhere
art = anim.render() # encodes, writes final.* + still.png, prints a report
print(art)
```
It all still happens in the one call — structure the program as:
1. **Cast block at the top, defined ONCE:** helpers like
`def draw_steve(ax, x, y, pose): …` and shared constants (`PALETTE`, `SKY`).
2. **One function per scene** (`def scene_0(...): …`) that draws its beat by
CALLING the cast helpers — never by re-describing a character inline.
3. **A timeline** that runs the scenes in order into `/tmp/frames` with the one
continuous counter, then encodes the whole sequence.
- `ax` is a fresh matplotlib Axes at exactly the locked pixel size, with
`x in [0, 100]` and `y in [0, anim.ymax]`, origin bottom-left, no margins.
Draw in those units and every frame lines up.
- `hold=<seconds>` freezes the scene's last frame — that is how you make a
caption readable (short phrases ~1.5–2 s, sentences longer).
- PIL instead: `@anim.scene("...", seconds=2, mode="pil")` gives
`(img, draw, t)`. `anim.font(size)` returns a real bold TrueType face.
- Hand-built frames: `anim.add_frame(pil_image, times=n)`.
- `anim.render(long_edge=..., force="gif"|"mp4", audio="/workspace/track.wav")`.
It picks GIF vs MP4 by length, and re-encodes to MP4 if the GIF is too big for
Discord. It returns `{"path", "mime", "bytes"}`.
Because every scene calls the same `draw_*` helpers, recurring people/props come
out pixel-identical scene to scene — the entire point of the cast block. If you
catch yourself re-describing a character inside a scene, hoist it into a cast
helper. Carry shared state (positions, score, time of day) in plain variables
between scene calls so cuts stay continuous.
**Everything happens in ONE call** — `/workspace` is wiped before every
`code_exec` call, so a render in one call and a read in the next finds nothing.
That also means **`gifsmith.py` must be in `files_in` on EVERY `code_exec` call
that imports it**, including every retry — it does not survive from the last one.
Give the call a generous `timeout_seconds` (120+); encoding is CPU-bound.
### Budget for LONG pieces (minutes)
**A SyntaxError costs you the whole pass** — it happens before the harness can
isolate anything, and it is the one failure it cannot absorb. You are emitting a
lot of code in one shot, so keep it boring: one statement per line, short helper
signatures, no dense one-liners packing three drawing calls into a single
`add_patch`, and the same keyword names everywhere you call a helper. Read back
any line with more than about four arguments before you send it.
A 3–10 minute animation is thousands of frames. Keep it renderable: **fps
10–12**, cap the long edge ~**480–540px** (`long_edge=` in `encode_animation`),
and favour procedural motion that's cheap per frame — don't render 9000 frames
of heavy matplotlib if a lighter draw reads the same. **If code_exec times out
or errors mid-render you get nothing back**, so on the retry SCALE DOWN
decisively (lower fps, smaller canvas, trim hold frames, reduce the longest
scenes) — keep every beat, just render it leaner. Each retry counts toward the
3-pass cap; never burn all three on an over-ambitious render and leave the user
with nothing.
Read the report `render()` prints. It tells you the frame count, the scratch
peak, and — importantly — **which scenes failed**. A scene that raises is
skipped, the rest still render and ship, and you fix just that scene next pass
instead of re-deriving the whole program.
## 5. Text in GIFs — make it READABLE
Long pieces (minutes): `fps=10..12`, `width=480..540`, and favour procedural
motion that is cheap per frame. If a render times out you get nothing back, so on
a retry scale down decisively — keep every beat, render it leaner.
Adding captions/labels/punchlines is fine, but pace them for a human. Motion can
stay 10–20 fps; what matters is how long each distinct TEXT STATE holds — short
phrases ~1.5–2 s, full sentences longer (budget by word count). At 15 fps that's
~25–30 repeated frames per text state, not a handful. Hold by duplicating frames,
or with PIL pass a per-frame `duration=[…]` list (long on text, short on motion).
Text flashing past unreadably is the #1 complaint — check it in the critique.
## 4. Files: file_ids, never paths
## 6. Self-critique (AT MOST 3 render passes)
`render()` leaves `/workspace/final.gif` (or `.mp4`) and `/workspace/still.png` at
the TOP of `/workspace`, and `files_out` returns a **file_id** for each.
`image_describe`, `send_attachments` and `file_get_metadata` take that file_id —
passing `"/workspace/final.gif"` fails `file_not_found`. **Copy the file_id
character-for-character out of the `files_out` JSON**; do not retype it from
memory. One wrong digit is a `file_not_found` that silently costs you the whole
critique pass.
Nothing here needs `pip install` — matplotlib, PIL, numpy, ffmpeg, gifski and
gifsicle are already in the sandbox.
## 5. Self-critique (AT MOST 3 render passes)
Before delivering, LOOK at what you made: call `image_describe` on the
`still.png` **file_id** (from `files_out`, not a path) and judge it against the
request — motion, colour banding, artifacts, framing, "does it depict what was
asked", and text readability. For a multi-scene piece, judge scene by scene off
the contact sheet AND check each recurring character/prop is the SAME across
scenes (drift means that scene drew it ad hoc — fix the cast helper and
re-render). If it falls short, change something concrete and re-run the WHOLE
pipeline in one fresh `code_exec` call (workspace was wiped — rebuild from
scratch). **At most 3 render passes total**; for a long piece be more decisive
(1–2 passes) since each re-render is expensive. After the 3rd, deliver the best
version with a one-line note on any trade-off.
**`still.png` file_id** — the harness already built it as a labelled contact
sheet, one tile per scene. Ask specifically about subject size, caption
readability and whether the depicted action matches the request. Then answer:
## 7. Deliver
1. Does it depict what was asked — every named character, prop and action?
2. Is the subject **big enough to read** (≥ ⅓ frame height)?
3. Is any caption legible, and does it hold long enough?
4. Does every recurring character look the SAME across scenes?
5. Any scene the report flagged as failed, colour banding, clipped shapes,
characters floating with no shadow?
Deliver the final artifact with a SINGLE `send_attachments` call using
`channel_id` from your platform context and the **file_id that
`encode_animation` produced** (read it from that render call's `files_out` by
matching `final.gif` / `final.mp4` — NOT the still, NOT a path). Finish with a
one-line caption. Critique aids (contact sheets, stills) are INTERNAL — never
attach them unless the user explicitly asked.
**A "no" on 1, 2 or 3 means you re-render** — those three are what makes a GIF
land or flop, and "it's basically fine" is how a bad one ships. Change something
concrete and re-run the whole program in one fresh `code_exec` call. **At most 3
render passes**; for a long piece be decisive (1–2). After the 3rd, deliver the
best version with a one-line note on the trade-off.
## 6. Deliver
ONE `send_attachments` call with `channel_id` from your platform context and the
file_id of `final.gif` / `final.mp4` — NOT the still, NOT a path. Finish with a
one-line caption. The contact sheet is INTERNAL; never attach it unless asked.
## Forbidden
- NEVER pass a `/workspace/...` path to image_describe / send_attachments /
file_get_metadata — they take a file_id from `files_out`.
- Do not write the final artifact into a `/workspace` subdir — top level only.
- Do not split render and read across separate code_exec calls — `/workspace` is
wiped between them.
- Do not hand-roll the encode/format decision — call `encode_animation`; it
guarantees a Discord-playable MP4 (yuv420p, even dims, faststart) when needed.
- Do not restart the frame counter per scene — one continuous 5-digit counter.
- Do not re-describe a recurring character inline per scene — one shared cast
helper so it stays identical.
- Do not invent beats the user didn't ask for — assume ONE scene, add more only
for beats the prompt describes (no cap, but no padding).
- Never pass a `/workspace/...` path where a file_id is wanted, and never retype
a file_id from memory — copy it from `files_out`.
- Do not hand-roll the scratch dir, the frame loop, the frame numbering, the
contact sheet, or the encode — that is what the harness is for. In particular
do not render frames into `/tmp` (a 16 MiB tmpfs) or into `/workspace` itself.
- Do not split render and read across separate code_exec calls, and do not
assume `gifsmith.py` is still in `/workspace` from the last call — pass it in
`files_in` every time.
- Do not re-describe a recurring character inline per scene — one cast helper.
- Do not invent beats the user didn't ask for — assume ONE scene.
- Do not narrate the action in a caption instead of animating it.
- Do not ship a subject too small to recognise, or a stretch of identical frames.
- Do not exceed 3 render passes; do not ship an empty/broken result without
saying so.
gif/scripts/encode.py
-115
View File
@@ -1,115 +0,0 @@
"""
encode.py — turn a directory of PNG frames into a Discord-friendly GIF or MP4.
Use it INSIDE a single code_exec call. Render your frames to a scratch dir as
frame_00000.png, frame_00001.png, … (fixed 5-digit width, ONE continuous
counter across all scenes), then:
from encode import encode_animation
art = encode_animation("/tmp/frames", fps=15, seconds_hint=8)
# art == {"path": "/workspace/final.mp4", "mime": "video/mp4", "bytes": 1234567}
It writes the finished artifact to the TOP of /workspace (so code_exec returns a
file_id for it) and picks the format for you:
* short & light -> animated GIF (fun, loops, autoplays)
* long (many seconds+),
OR a GIF too big to post -> H.264 MP4: yuv420p, even dimensions, +faststart
— the combination Discord needs to PLAY it inline
(a video whose dimensions Discord can't read is
shown as a dead download, not a player).
Requires ffmpeg (always present in the sandbox). Uses gifski + gifsicle for a
cleaner GIF when they're installed, falling back to ffmpeg's palette pipeline.
This helper only ENCODES — you draw the frames.
"""
import glob
import os
import shutil
import subprocess
# A GIF longer than this balloons past what's worth shipping as a GIF — encode
# straight to MP4 instead. Tune down for chattier channels.
GIF_MAX_SECONDS = 20
# Keep a GIF comfortably under Discord's ~10 MiB inline limit; over this we
# re-encode to an MP4 and drop the GIF.
GIF_MAX_BYTES = 9 * 1024 * 1024
def _which(prog):
return shutil.which(prog) is not None
def _run(cmd):
p = subprocess.run(cmd, capture_output=True, text=True)
if p.returncode != 0:
raise RuntimeError(f"{cmd[0]} failed:\n{p.stderr[-2000:]}")
def _frames(frames_dir):
fs = sorted(glob.glob(os.path.join(frames_dir, "frame_*.png")))
if not fs:
raise FileNotFoundError(
f"no frame_*.png in {frames_dir} — render frames as "
f"{frames_dir}/frame_00000.png, frame_00001.png, … first"
)
return fs
def _encode_gif(frames_dir, fps, out, long_edge):
fs = _frames(frames_dir)
if _which("gifski"):
# gifski takes an explicit, sorted file list — no numbering-gap surprises.
_run(["gifski", "-o", out, "--fps", str(fps), "--quality", "90"] + fs)
else:
vf = (f"fps={fps},scale='min({long_edge},iw)':-1:flags=lanczos,"
f"split[s0][s1];[s0]palettegen=stats_mode=diff[p];"
f"[s1][p]paletteuse=dither=bayer:bayer_scale=3")
_run(["ffmpeg", "-y", "-pattern_type", "glob",
"-i", os.path.join(frames_dir, "frame_*.png"), "-vf", vf, out])
if _which("gifsicle"):
_run(["gifsicle", "-O3", "--lossy=60", out, "-o", out])
def _encode_mp4(frames_dir, fps, out, long_edge):
_frames(frames_dir) # validate presence
# -2 forces EVEN width/height (H.264 requires it); yuv420p + faststart make
# Discord/browsers play it inline instead of offering a download.
vf = f"scale='min({long_edge},iw)':-2:flags=lanczos"
_run(["ffmpeg", "-y", "-framerate", str(fps), "-pattern_type", "glob",
"-i", os.path.join(frames_dir, "frame_*.png"),
"-c:v", "libx264", "-pix_fmt", "yuv420p", "-crf", "23",
"-vf", vf, "-movflags", "+faststart", out])
def encode_animation(frames_dir, fps=15, seconds_hint=0, long_edge=640):
"""Encode frames_dir → /workspace/final.{gif,mp4}. Returns {path, mime, bytes}.
seconds_hint: your intended runtime in seconds (falls back to frame_count/fps
when 0). Pass it for long pieces so the format decision doesn't depend on how
many frames you happened to render.
long_edge: cap for the output's long side in px (keeps size/encode sane).
"""
n = len(_frames(frames_dir))
seconds = seconds_hint or (n / max(fps, 1))
gif, mp4 = "/workspace/final.gif", "/workspace/final.mp4"
if seconds > GIF_MAX_SECONDS:
_encode_mp4(frames_dir, fps, mp4, long_edge)
return _art(mp4, "video/mp4")
_encode_gif(frames_dir, fps, gif, long_edge)
if os.path.getsize(gif) <= GIF_MAX_BYTES:
return _art(gif, "image/gif")
# Too heavy as a GIF — hand Discord an MP4 it can actually render.
_encode_mp4(frames_dir, fps, mp4, long_edge)
os.remove(gif)
return _art(mp4, "video/mp4")
def _art(path, mime):
b = os.path.getsize(path)
print(f"[encode] {path} {mime} {b / 1024 / 1024:.2f} MiB")
return {"path": path, "mime": mime, "bytes": b}
gif/scripts/gifsmith.py
+394
View File
@@ -0,0 +1,394 @@
"""
gifsmith.py — the render harness for the `gif` skill.
You write ONLY the drawing. This module owns every part that has historically
broken renders: the scratch directory and its disk budget, frame naming and the
continuous counter, a LOCKED canvas size (frames that differ by one pixel break
the encoder), per-scene crash isolation, the GIF-vs-MP4 decision, and the
critique contact sheet.
from gifsmith import Animation
anim = Animation(width=640, height=480, fps=15)
# --- cast: define each recurring character ONCE ---
def draw_steve(ax, x, y, angry=0.0):
...
@anim.scene("Steve watches the printer", seconds=2.5)
def s0(ax, t): # t goes 0.0 -> 1.0 across the scene
draw_steve(ax, 2, 1, angry=t)
art = anim.render() # -> {"path": "/workspace/final.gif", "mime": ..., "bytes": ...}
`ax` is a fresh matplotlib Axes at exactly (width, height) pixels, x in [0,100],
y in [0, 100*height/width], origin bottom-left, no margins. Draw in those units
and every frame lines up.
PIL instead of matplotlib:
@anim.scene("title card", seconds=1.5, mode="pil")
def s1(img, draw, t): # img: RGB Image at the locked size; draw: ImageDraw
draw.text((40, 40), "THE END", fill=(255, 255, 255), font=anim.font(48))
render() prints a report and returns the artifact dict. If one scene raises, the
OTHER scenes still render and ship — the report names the broken one so you can
fix just that scene. It also writes /workspace/still.png: a labelled contact
sheet, one tile per scene, for the critique pass.
"""
import glob
import math
import os
import shutil
import subprocess
import sys
import traceback
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from PIL import Image, ImageDraw, ImageFont # noqa: E402
# Re-exported so one import line covers the whole drawing vocabulary — a missing
# `from matplotlib.patches import ...` is otherwise a NameError that costs a pass.
from matplotlib.patches import ( # noqa: E402,F401
Arc, Circle, Ellipse, FancyBboxPatch, Polygon, Rectangle, Wedge)
import numpy as np # noqa: E402,F401
# A GIF longer than this balloons past what's worth shipping as a GIF.
GIF_MAX_SECONDS = 20
# Keep a GIF comfortably under Discord's ~10 MiB inline limit.
GIF_MAX_BYTES = 9 * 1024 * 1024
_FONT_CANDIDATES = [
"/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf",
"/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf",
]
def _which(prog):
return shutil.which(prog) is not None
def _free_mb(path):
try:
st = os.statvfs(path)
return st.f_bavail * st.f_frsize / 1024 / 1024
except OSError:
return 0.0
def _pick_scratch():
"""Largest writable scratch dir. /tmp is a small tmpfs in this sandbox; a
SUBDIR of /workspace is bigger and is not returned in files_out (only
TOP-LEVEL /workspace files are), so it is safe to fill with frames."""
best, best_free = None, -1.0
for base in ("/workspace/.frames", "/tmp/frames", "/var/tmp/frames"):
parent = os.path.dirname(base)
if not os.path.isdir(parent) or not os.access(parent, os.W_OK):
continue
free = _free_mb(parent)
if free > best_free:
best, best_free = base, free
if best is None:
raise RuntimeError("no writable scratch directory found")
shutil.rmtree(best, ignore_errors=True)
os.makedirs(best, exist_ok=True)
return best, best_free
class _Scene:
def __init__(self, fn, title, seconds, mode, hold):
self.fn, self.title, self.seconds = fn, title, float(seconds),
self.mode, self.hold = mode, float(hold)
self.frames = 0
self.error = None
self.first_frame = None
class Animation:
"""Collects scenes, renders them to a locked-size frame sequence, encodes."""
def __init__(self, width=640, height=480, fps=15, dpi=100, bg="white"):
# H.264 needs even dimensions; lock them here so the encoder never fails.
self.width = int(width) // 2 * 2
self.height = int(height) // 2 * 2
self.fps = int(fps)
self.dpi = int(dpi)
self.bg = bg
self.xmax = 100.0
self.ymax = 100.0 * self.height / self.width
self._scenes = []
self._count = 0
self._scratch = None
self._peak_mb = 0.0
self._streaming = False
self._pipe = None
# ---------------------------------------------------------------- authoring
def scene(self, title, seconds=2.0, mode="mpl", hold=0.0):
"""Decorator: register a scene. `hold` freezes the last frame for that
many extra seconds — use it to let a caption stay readable."""
def deco(fn):
self._scenes.append(_Scene(fn, title, seconds, mode, hold))
return fn
return deco
def font(self, size):
for p in _FONT_CANDIDATES:
if os.path.exists(p):
return ImageFont.truetype(p, size)
return ImageFont.load_default()
def text(self, ax, s, size=18, y=None, color="white", box="black"):
"""Caption helper: centred, boxed, legible at GIF resolution."""
ax.text(self.xmax / 2, self.ymax * 0.06 if y is None else y, s,
ha="center", va="center", fontsize=size, color=color, zorder=999,
bbox=dict(boxstyle="round,pad=0.4", fc=box, ec="none", alpha=0.75))
def add_frame(self, img, times=1):
"""Escape hatch for hand-built frames (PIL compositing, pasted photos,
anything the scene decorators can't express). Still size-locked."""
for _ in range(max(1, int(times))):
self._emit(img)
# ------------------------------------------------------------------ drawing
def _new_axes(self):
fig = plt.figure(figsize=(self.width / self.dpi, self.height / self.dpi),
dpi=self.dpi)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, self.xmax)
ax.set_ylim(0, self.ymax)
ax.set_axis_off()
ax.set_facecolor(self.bg)
fig.patch.set_facecolor(self.bg)
return fig, ax
def _emit(self, img):
"""Emit one frame at exactly the locked size. Never bbox_inches='tight'.
Short pieces go to PNG files (gifski wants a file list). Long pieces are
piped straight into ffmpeg so a multi-minute render never touches disk —
the sandbox scratch would otherwise cap you at well under a minute."""
if img.size != (self.width, self.height):
img = img.resize((self.width, self.height), Image.LANCZOS)
img = img.convert("RGB")
if self._pipe is not None:
img.save(self._pipe.stdin, format="PNG", compress_level=1)
else:
img.save(os.path.join(self._scratch, f"frame_{self._count:05d}.png"),
compress_level=1)
self._count += 1
def _render_mpl(self, sc, t):
fig, ax = self._new_axes()
try:
sc.fn(ax, t)
fig.canvas.draw()
img = Image.frombuffer(
"RGBA", fig.canvas.get_width_height(),
fig.canvas.buffer_rgba(), "raw", "RGBA", 0, 1).convert("RGB")
finally:
plt.close(fig)
return img
def _render_pil(self, sc, t):
img = Image.new("RGB", (self.width, self.height),
self.bg if isinstance(self.bg, (str, tuple)) else "white")
sc.fn(img, ImageDraw.Draw(img), t)
return img
# ------------------------------------------------------------------- render
def render(self, long_edge=None, force=None, audio=None):
if not self._scenes:
raise RuntimeError("no scenes registered — decorate at least one "
"function with @anim.scene(...)")
self._scratch, free_mb = _pick_scratch()
long_edge = long_edge or max(self.width, self.height)
total_s = sum(s.seconds + s.hold for s in self._scenes)
est_frames = int(total_s * self.fps)
# ~90 KiB per 640x480 PNG at compress_level=1; budget 60% of free space.
est_mb = est_frames * (self.width * self.height * 9e-5)
# A piece too long for a GIF is an MP4 anyway, and an MP4 can be encoded
# in one streaming pass — so pipe the frames into ffmpeg and keep NOTHING
# on disk. That is what makes multi-minute pieces possible at all: the
# sandbox scratch tops out around a minute of busy frames.
self._streaming = (force == "mp4" or audio is not None
or (force != "gif" and (total_s > GIF_MAX_SECONDS
or est_mb > free_mb * 0.6)))
self._pipe = None
if self._streaming:
self._pipe = self._open_pipe("/workspace/final.mp4", long_edge, audio)
print(f"[gifsmith] streaming to MP4 (projected {est_mb:.0f} MiB of "
f"frames vs {free_mb:.0f} MiB scratch) — nothing hits disk",
flush=True)
elif est_mb > free_mb * 0.6:
print(f"[gifsmith] WARNING projected frames ~{est_mb:.0f} MiB vs "
f"{free_mb:.0f} MiB free in {os.path.dirname(self._scratch)} — "
f"dropping fps or canvas would be safer", file=sys.stderr)
print(f"[gifsmith] {len(self._scenes)} scene(s), {total_s:.1f}s at "
f"{self.fps}fps, canvas {self.width}x{self.height}", flush=True)
for i, sc in enumerate(self._scenes):
start = self._count
n = max(1, int(round(sc.seconds * self.fps)))
try:
for k in range(n):
t = k / max(n - 1, 1)
img = (self._render_pil if sc.mode == "pil" else self._render_mpl)(sc, t)
if sc.first_frame is None and k >= n // 2:
sc.first_frame = img.copy()
self._emit(img)
for _ in range(int(round(sc.hold * self.fps))):
self._emit(img)
except Exception: # noqa: BLE001
sc.error = traceback.format_exc(limit=6)
if not self._streaming:
# Drop this scene's partial frames so it can't ship half-drawn.
# (Streamed frames are already in the encoder — a broken scene
# leaves its partial beat in the MP4; the report says so.)
for p in sorted(glob.glob(
os.path.join(self._scratch, "frame_*.png")))[start:]:
os.remove(p)
self._count = start
print(f"[gifsmith] SCENE {i} ({sc.title!r}) FAILED — skipped, "
f"other scenes continue:\n{sc.error}", file=sys.stderr, flush=True)
continue
sc.frames = self._count - start
print(f"[gifsmith] scene {i}: {sc.frames} frames {sc.title!r}", flush=True)
self._renumber_if_needed()
used = sum(os.path.getsize(p) for p in
glob.glob(os.path.join(self._scratch, "frame_*.png")))
self._peak_mb = used / 1024 / 1024
if self._count == 0:
raise RuntimeError(
"every scene failed — see the per-scene tracebacks above; "
"nothing was encoded. Fix the error they all share (usually one "
"cast helper called with a keyword it does not take, or a name "
"that differs by a character) and re-run the whole program in a "
"fresh code_exec call, remembering gifsmith.py in files_in.")
art = self._encode(total_s, long_edge, force, audio)
self._contact_sheet()
broken = [f"{i}:{s.title!r}" for i, s in enumerate(self._scenes) if s.error]
print(f"[gifsmith] {self._count} frames, scratch "
+ ("streamed (0 MiB on disk)" if self._streaming
else f"peak {self._peak_mb:.1f} MiB"))
print(f"[gifsmith] scene errors: {', '.join(broken) if broken else 'none'}")
print(f"[gifsmith] ART {art}")
print("[gifsmith] still: /workspace/still.png (contact sheet — call "
"image_describe on ITS file_id from files_out)")
shutil.rmtree(self._scratch, ignore_errors=True)
return art
def _renumber_if_needed(self):
"""A skipped scene leaves no gaps because _count rewinds; kept as a
no-op hook so the counter contract is explicit."""
return
# ------------------------------------------------------------------ encoding
def _open_pipe(self, out, long_edge, audio):
"""ffmpeg reading a PNG stream on stdin — the streaming MP4 path."""
cmd = ["ffmpeg", "-y", "-loglevel", "error",
"-f", "image2pipe", "-framerate", str(self.fps), "-i", "-"]
if audio:
cmd += ["-i", audio, "-c:a", "aac", "-b:a", "128k", "-shortest"]
cmd += ["-c:v", "libx264", "-pix_fmt", "yuv420p", "-crf", "23",
"-vf", f"scale='min({long_edge},iw)':-2:flags=lanczos",
"-movflags", "+faststart", out]
return subprocess.Popen(cmd, stdin=subprocess.PIPE,
stdout=subprocess.DEVNULL, stderr=subprocess.PIPE)
def _close_pipe(self):
self._pipe.stdin.close()
err = self._pipe.stderr.read().decode(errors="replace")
if self._pipe.wait() != 0:
raise RuntimeError(f"ffmpeg (streaming) failed:\n{err[-2000:]}")
self._pipe = None
def _encode(self, seconds, long_edge, force, audio=None):
gif, mp4 = "/workspace/final.gif", "/workspace/final.mp4"
if self._streaming:
self._close_pipe()
return _art(mp4, "video/mp4")
fs = sorted(glob.glob(os.path.join(self._scratch, "frame_*.png")))
want_mp4 = (force == "mp4" or audio is not None
or (force != "gif" and seconds > GIF_MAX_SECONDS))
if want_mp4:
self._mp4(mp4, long_edge, audio)
return _art(mp4, "video/mp4")
self._gif(fs, gif, long_edge)
if os.path.getsize(gif) <= GIF_MAX_BYTES:
return _art(gif, "image/gif")
print(f"[gifsmith] GIF {os.path.getsize(gif)/1048576:.1f} MiB is too big "
f"for Discord — re-encoding as MP4", flush=True)
self._mp4(mp4, long_edge, audio)
os.remove(gif)
return _art(mp4, "video/mp4")
def _gif(self, fs, out, long_edge):
if _which("gifski"):
_run(["gifski", "-o", out, "--fps", str(self.fps), "--quality", "90",
"--width", str(min(long_edge, self.width))] + fs)
else:
vf = (f"fps={self.fps},scale='min({long_edge},iw)':-1:flags=lanczos,"
f"split[s0][s1];[s0]palettegen=stats_mode=diff[p];"
f"[s1][p]paletteuse=dither=bayer:bayer_scale=3")
_run(["ffmpeg", "-y", "-loglevel", "error", "-pattern_type", "glob",
"-i", os.path.join(self._scratch, "frame_*.png"), "-vf", vf, out])
if _which("gifsicle"):
_run(["gifsicle", "-O3", "--lossy=60", out, "-o", out])
def _mp4(self, out, long_edge, audio=None):
cmd = ["ffmpeg", "-y", "-loglevel", "error", "-framerate", str(self.fps),
"-pattern_type", "glob",
"-i", os.path.join(self._scratch, "frame_*.png")]
if audio:
cmd += ["-i", audio, "-c:a", "aac", "-b:a", "128k", "-shortest"]
cmd += ["-c:v", "libx264", "-pix_fmt", "yuv420p", "-crf", "23",
"-vf", f"scale='min({long_edge},iw)':-2:flags=lanczos",
"-movflags", "+faststart", out]
_run(cmd)
# ------------------------------------------------------------- critique aid
def _contact_sheet(self, path="/workspace/still.png"):
tiles = [(i, s) for i, s in enumerate(self._scenes) if s.first_frame is not None]
if not tiles:
return
if len(tiles) == 1:
tiles[0][1].first_frame.save(path)
return
cols = min(3, len(tiles))
rows = math.ceil(len(tiles) / cols)
tw = 300
th = max(1, round(self.height * tw / self.width))
sheet = Image.new("RGB", (cols * tw, rows * (th + 20)), (20, 20, 24))
d = ImageDraw.Draw(sheet)
f = self.font(14)
for k, (i, sc) in enumerate(tiles):
x, y = (k % cols) * tw, (k // cols) * (th + 20)
sheet.paste(sc.first_frame.resize((tw, th), Image.LANCZOS), (x, y + 20))
d.text((x + 4, y + 3), f"scene {i}: {sc.title[:38]}", fill=(235, 235, 240), font=f)
sheet.save(path)
def _run(cmd):
p = subprocess.run(cmd, capture_output=True, text=True)
if p.returncode != 0:
raise RuntimeError(f"{cmd[0]} failed:\n{p.stderr[-2000:]}")
def _art(path, mime):
b = os.path.getsize(path)
print(f"[gifsmith] encoded {path} {mime} {b / 1024 / 1024:.2f} MiB")
return {"path": path, "mime": mime, "bytes": b}