diff --git a/gif/SKILL.md b/gif/SKILL.md index 78865e4..604afb5 100644 --- a/gif/SKILL.md +++ b/gif/SKILL.md @@ -1,10 +1,10 @@ --- name: gif -description: Make a funny animated GIF (or MP4) for Discord from a description — plans scenes, keeps recurring people/props consistent via a reusable cast, renders frames in Python, encodes cleanly, and automatically hands Discord an H.264 MP4 it can play when the piece is long or a GIF comes out too big. Use whenever someone asks to draw/animate/gif something, including multi-minute bits about people or things that happened. +description: Make a funny animated GIF (or MP4) for Discord from a description — plans scenes, keeps recurring people/props consistent via a reusable cast, renders frames in Python through a bundled harness that owns the scratch/encode plumbing, and automatically hands Discord an H.264 MP4 it can play when the piece is long or a GIF comes out too big. Use whenever someone asks to draw/animate/gif something, including multi-minute bits about people or things that happened. license: MIT allowed-tools: [code_exec, image_describe, send_attachments, file_get_metadata, think] metadata: - version: "1.0.0" + version: "3.0.0" adapted-from: mort/skills/animate --- @@ -12,168 +12,179 @@ metadata: You make HIGH-QUALITY, funny animations from a description — often caricatures of the people in the channel or a bit about something that just happened. -Naive PIL GIF export looks bad (256-colour banding, muddy dithering) and a -multi-minute GIF is physically absurd (hundreds of MB), so follow this recipe: -render frames in Python, let the bundled `encode.py` choose GIF vs MP4, and -self-critique the result. **Work by calling tools — do not introduce yourself -or list capabilities.** +**Work by calling tools — do not introduce yourself or list capabilities.** + +This skill bundles `scripts/gifsmith.py`, a render harness that owns everything +that used to break these renders: the scratch directory and its disk budget, +frame numbering, a locked canvas size, per-scene crash isolation, the GIF-vs-MP4 +decision, and the critique contact sheet. **You write only the drawing.** ## 1. Plan first: length, scenes, and cast -Before drawing, decide (use `think` for anything non-trivial): +Decide before drawing (use `think` for anything non-trivial): -- **How long / how many scenes?** Count the distinct beats the prompt actually - describes, and render exactly that many. **ASSUME ONE.** A single continuous - action ("a bouncing ball", "Steve slipping on a banana") is ONE scene — don't - chop it into cuts. Start a new scene only when the prompt moves to a new beat - (new action, place, or time jump). There is **no upper limit** — a bit that - genuinely describes twelve beats gets twelve scenes and runs for minutes — but - **never pad a simple request with invented beats**, and never compress beats - the user clearly separated. When in doubt, fewer scenes. -- **Who/what is the cast?** List every recurring character and prop (a person's - persona, "the dog", a logo). For a bit "about ", pick a simple, - recognizable caricature — one or two signature features (glasses, hair colour, - a hat) — and pin each cast member's look ONCE as a reusable Python helper (see - §4). That is what keeps the SAME person drawn identically across every scene - instead of re-invented each cut. A one-scene GIF with a single subject needs - no cast block. +- **How many scenes?** Count the distinct beats the prompt actually describes and + render exactly that many. **ASSUME ONE.** A single continuous action ("a + bouncing ball", "Steve slipping on a banana") is ONE scene — don't chop it into + cuts. Start a new scene only when the prompt moves to a new beat (new action, + place, or time jump). There is **no upper limit** — a bit that genuinely + describes twelve beats gets twelve scenes and runs for minutes — but **never + pad a simple request with invented beats**, and never compress beats the user + clearly separated. When in doubt, fewer scenes. +- **Who is the cast?** Every recurring character and prop. For a bit "about + ", pick a simple recognizable caricature — one or two signature + features (glasses, hair colour, a hat) — and pin each cast member's look ONCE + as a `draw_(ax, ...)` helper. Every scene CALLS that helper; never + re-describe a character inline, or they drift scene to scene. -## 2. Pick the format from the length (encode.py does this for you) +## 2. STAGING — this is what separates a good GIF from a bad one -- **Short & loopable (≤ ~20 s):** a GIF — fun, loops, autoplays. -- **Long (tens of seconds to minutes):** an **MP4**. A 3–10 minute GIF is - hundreds of MB and unusable; H.264 is the right tool and Discord plays it - inline. -- You don't hand-pick this: `encode.py`'s `encode_animation(...)` takes a - `seconds_hint` and (a) goes straight to MP4 for long pieces, and (b) for short - ones, makes the GIF but **auto-re-encodes to an MP4 if the GIF is too big for - Discord to render nicely.** It returns `{"path", "mime", "bytes"}` — deliver - whatever it hands back. +The harness guarantees a *working* animation. What makes it *good* is staging, +and that is where these renders most often fail. Four rules, all checked in §5: -## 3. code_exec workspace — READ THIS (it is why draws fail) +- **FILL THE FRAME.** The subject of a beat must be at least **one third of the + frame height** — for a close reaction, half or more. A character drawn 5% tall + in a corner under a big empty sky is the single most common failure: the viewer + can't tell what it is. Crop in. Empty sky/floor is wasted frame; if a region has + nothing happening in it, the camera is in the wrong place. Put the action in the + middle two thirds, and let characters break the frame edge rather than shrink. +- **EVERY FRAME MOVES.** Nothing sits pixel-identical for more than ~0.3 s unless + you are deliberately holding on a text beat (use `hold=` for that). Even a + "still" character breathes, sways, blinks; even a still scene has drifting + clouds or a flickering light. Long stretches of identical frames read as a + broken GIF. Animate the SUBJECT, not just one prop next to it. +- **SHOW IT, DON'T CAPTION IT.** A caption is for a punchline or a line of + dialogue — never a narration of what the animation should be depicting. If you + find yourself writing `"*CRASH* garbage everywhere!"`, draw the garbage flying + instead. Every noun in the request should appear on screen, not in text. +- **GIVE IT DEPTH.** Flat single-tone shapes look like a diagram. Cheap wins that + land every time: a soft elliptical **shadow on the ground under every + character**, a second darker tone on the shaded side of each big shape, a + background that is a gradient or has a couple of out-of-focus shapes, and clear + foreground/midground/background separation. Outline characters in a dark colour + so they pop. -The `code_exec` sandbox has three hard rules; breaking them is the #1 cause of -"vanished" files and wasted budget: +Motion quality, in order of payoff: **ease in and out** of every move +(`t*t*(3-2*t)`, never linear), **anticipation** (wind up before a big action), +**overshoot/settle** on impacts, **squash & stretch** on anything that bounces or +lands. A held pose that eases beats three extra scenes. -1. **`/workspace` is WIPED before every `code_exec` call.** Nothing survives - between calls. So render → encode → write your critique still **all in ONE - call**. Never write frames in one call and read them in another. -2. **Only TOP-LEVEL files in `/workspace/` come back as `files_out` file_ids.** - A file in a subdir (`/workspace/frames/x.png`) is NOT returned. Write the - final artifact and the ONE critique still directly at the top: - `encode.py` already writes `/workspace/final.gif` or `/workspace/final.mp4`; - you add `/workspace/still.png`. Render scratch frames under `/tmp`, never - `/workspace`. -3. **Don't dump many files into `/workspace`.** Hundreds of frame PNGs blow the - `files_out` size cap and get dropped. Keep `/workspace` to just the final - artifact + one still; everything else lives under `/tmp`. +## 3. Render with the harness (ONE code_exec call) -`image_describe`, `send_attachments`, `file_get_metadata` take a **file_id from -`files_out` — never a path.** Passing `"/workspace/final.mp4"` fails -`file_not_found`. Read the file_id out of `files_out` and pass THAT. +Pass `scripts/gifsmith.py` in with `files_in` (its file_id is listed at the +bottom of this skill) as `{"name": "gifsmith.py", "file_id": ""}`, then +write ONE `code_exec` call: -## 4. Render → encode (ONE code_exec call, using encode.py) +```python +# gifsmith re-exports the drawing vocabulary, so this one import is enough: +from gifsmith import Animation, Circle, Ellipse, Rectangle, Polygon, Wedge, np +anim = Animation(width=640, height=480, fps=15) # canvas locked here -This skill bundles `scripts/encode.py`. Load it into the render call with -`files_in` (its file_id is listed when you open this skill), then in a SINGLE -`code_exec` call: +# ---- cast: defined ONCE, called from every scene ---- +def draw_steve(ax, x, y, rage=0.0): + ax.add_patch(Ellipse((x, y - 9), 14, 3, fc="#00000033", zorder=1)) # shadow + ... -1. **Clear scratch** so stale frames can't leak into this encode: - `import shutil, os; shutil.rmtree('/tmp/frames', ignore_errors=True); os.makedirs('/tmp/frames')` -2. **Render frames** with Python (matplotlib / Pillow / numpy) into `/tmp/frames` - as `frame_00000.png, frame_00001.png, …` — **fixed 5-digit zero-pad, ONE - continuous counter across all scenes** (do NOT restart numbering per scene, or - the encoder reorders/overwrites frames). -3. **Encode** — let the helper choose GIF vs MP4: - ```python - from encode import encode_animation - art = encode_animation("/tmp/frames", fps=15, seconds_hint=) - print(art) # {'path': '/workspace/final.mp4', 'mime': 'video/mp4', 'bytes': ...} - ``` -4. **Write ONE critique still** to `/workspace/still.png` in the same call — a - representative frame, or for a multi-scene piece a COMPACT contact sheet with - one small (~200–320px) tile per scene, labelled by scene index, so the - critique can check every scene and cast consistency. Keep it low-res and - legible — it's an internal aid, not the deliverable. +@anim.scene("Steve watches the printer", seconds=2.5) +def s0(ax, t): # t runs 0.0 -> 1.0 across the scene + ease = t * t * (3 - 2 * t) + draw_steve(ax, 30 + 8 * ease, 40, rage=t) -The call ends with the final artifact + `still.png` at the top of `/workspace`; -`files_out` returns a file_id for each. Give `code_exec` a generous -`timeout_seconds` (e.g. 120+) — encoding is CPU-bound, and a long piece is real -work, not a hang. +@anim.scene("it sprays him", seconds=2.0, hold=0.8) +def s1(ax, t): + draw_steve(ax, 38, 40, rage=1.0) + anim.text(ax, "every single time") # boxed, bold, legible caption -### Multi-scene: define the cast ONCE, reuse it everywhere +art = anim.render() # encodes, writes final.* + still.png, prints a report +print(art) +``` -It all still happens in the one call — structure the program as: -1. **Cast block at the top, defined ONCE:** helpers like - `def draw_steve(ax, x, y, pose): …` and shared constants (`PALETTE`, `SKY`). -2. **One function per scene** (`def scene_0(...): …`) that draws its beat by - CALLING the cast helpers — never by re-describing a character inline. -3. **A timeline** that runs the scenes in order into `/tmp/frames` with the one - continuous counter, then encodes the whole sequence. +- `ax` is a fresh matplotlib Axes at exactly the locked pixel size, with + `x in [0, 100]` and `y in [0, anim.ymax]`, origin bottom-left, no margins. + Draw in those units and every frame lines up. +- `hold=` freezes the scene's last frame — that is how you make a + caption readable (short phrases ~1.5–2 s, sentences longer). +- PIL instead: `@anim.scene("...", seconds=2, mode="pil")` gives + `(img, draw, t)`. `anim.font(size)` returns a real bold TrueType face. +- Hand-built frames: `anim.add_frame(pil_image, times=n)`. +- `anim.render(long_edge=..., force="gif"|"mp4", audio="/workspace/track.wav")`. + It picks GIF vs MP4 by length, and re-encodes to MP4 if the GIF is too big for + Discord. It returns `{"path", "mime", "bytes"}`. -Because every scene calls the same `draw_*` helpers, recurring people/props come -out pixel-identical scene to scene — the entire point of the cast block. If you -catch yourself re-describing a character inside a scene, hoist it into a cast -helper. Carry shared state (positions, score, time of day) in plain variables -between scene calls so cuts stay continuous. +**Everything happens in ONE call** — `/workspace` is wiped before every +`code_exec` call, so a render in one call and a read in the next finds nothing. +That also means **`gifsmith.py` must be in `files_in` on EVERY `code_exec` call +that imports it**, including every retry — it does not survive from the last one. +Give the call a generous `timeout_seconds` (120+); encoding is CPU-bound. -### Budget for LONG pieces (minutes) +**A SyntaxError costs you the whole pass** — it happens before the harness can +isolate anything, and it is the one failure it cannot absorb. You are emitting a +lot of code in one shot, so keep it boring: one statement per line, short helper +signatures, no dense one-liners packing three drawing calls into a single +`add_patch`, and the same keyword names everywhere you call a helper. Read back +any line with more than about four arguments before you send it. -A 3–10 minute animation is thousands of frames. Keep it renderable: **fps -10–12**, cap the long edge ~**480–540px** (`long_edge=` in `encode_animation`), -and favour procedural motion that's cheap per frame — don't render 9000 frames -of heavy matplotlib if a lighter draw reads the same. **If code_exec times out -or errors mid-render you get nothing back**, so on the retry SCALE DOWN -decisively (lower fps, smaller canvas, trim hold frames, reduce the longest -scenes) — keep every beat, just render it leaner. Each retry counts toward the -3-pass cap; never burn all three on an over-ambitious render and leave the user -with nothing. +Read the report `render()` prints. It tells you the frame count, the scratch +peak, and — importantly — **which scenes failed**. A scene that raises is +skipped, the rest still render and ship, and you fix just that scene next pass +instead of re-deriving the whole program. -## 5. Text in GIFs — make it READABLE +Long pieces (minutes): `fps=10..12`, `width=480..540`, and favour procedural +motion that is cheap per frame. If a render times out you get nothing back, so on +a retry scale down decisively — keep every beat, render it leaner. -Adding captions/labels/punchlines is fine, but pace them for a human. Motion can -stay 10–20 fps; what matters is how long each distinct TEXT STATE holds — short -phrases ~1.5–2 s, full sentences longer (budget by word count). At 15 fps that's -~25–30 repeated frames per text state, not a handful. Hold by duplicating frames, -or with PIL pass a per-frame `duration=[…]` list (long on text, short on motion). -Text flashing past unreadably is the #1 complaint — check it in the critique. +## 4. Files: file_ids, never paths -## 6. Self-critique (AT MOST 3 render passes) +`render()` leaves `/workspace/final.gif` (or `.mp4`) and `/workspace/still.png` at +the TOP of `/workspace`, and `files_out` returns a **file_id** for each. +`image_describe`, `send_attachments` and `file_get_metadata` take that file_id — +passing `"/workspace/final.gif"` fails `file_not_found`. **Copy the file_id +character-for-character out of the `files_out` JSON**; do not retype it from +memory. One wrong digit is a `file_not_found` that silently costs you the whole +critique pass. + +Nothing here needs `pip install` — matplotlib, PIL, numpy, ffmpeg, gifski and +gifsicle are already in the sandbox. + +## 5. Self-critique (AT MOST 3 render passes) Before delivering, LOOK at what you made: call `image_describe` on the -`still.png` **file_id** (from `files_out`, not a path) and judge it against the -request — motion, colour banding, artifacts, framing, "does it depict what was -asked", and text readability. For a multi-scene piece, judge scene by scene off -the contact sheet AND check each recurring character/prop is the SAME across -scenes (drift means that scene drew it ad hoc — fix the cast helper and -re-render). If it falls short, change something concrete and re-run the WHOLE -pipeline in one fresh `code_exec` call (workspace was wiped — rebuild from -scratch). **At most 3 render passes total**; for a long piece be more decisive -(1–2 passes) since each re-render is expensive. After the 3rd, deliver the best -version with a one-line note on any trade-off. +**`still.png` file_id** — the harness already built it as a labelled contact +sheet, one tile per scene. Ask specifically about subject size, caption +readability and whether the depicted action matches the request. Then answer: -## 7. Deliver +1. Does it depict what was asked — every named character, prop and action? +2. Is the subject **big enough to read** (≥ ⅓ frame height)? +3. Is any caption legible, and does it hold long enough? +4. Does every recurring character look the SAME across scenes? +5. Any scene the report flagged as failed, colour banding, clipped shapes, + characters floating with no shadow? -Deliver the final artifact with a SINGLE `send_attachments` call using -`channel_id` from your platform context and the **file_id that -`encode_animation` produced** (read it from that render call's `files_out` by -matching `final.gif` / `final.mp4` — NOT the still, NOT a path). Finish with a -one-line caption. Critique aids (contact sheets, stills) are INTERNAL — never -attach them unless the user explicitly asked. +**A "no" on 1, 2 or 3 means you re-render** — those three are what makes a GIF +land or flop, and "it's basically fine" is how a bad one ships. Change something +concrete and re-run the whole program in one fresh `code_exec` call. **At most 3 +render passes**; for a long piece be decisive (1–2). After the 3rd, deliver the +best version with a one-line note on the trade-off. + +## 6. Deliver + +ONE `send_attachments` call with `channel_id` from your platform context and the +file_id of `final.gif` / `final.mp4` — NOT the still, NOT a path. Finish with a +one-line caption. The contact sheet is INTERNAL; never attach it unless asked. ## Forbidden -- NEVER pass a `/workspace/...` path to image_describe / send_attachments / - file_get_metadata — they take a file_id from `files_out`. -- Do not write the final artifact into a `/workspace` subdir — top level only. -- Do not split render and read across separate code_exec calls — `/workspace` is - wiped between them. -- Do not hand-roll the encode/format decision — call `encode_animation`; it - guarantees a Discord-playable MP4 (yuv420p, even dims, faststart) when needed. -- Do not restart the frame counter per scene — one continuous 5-digit counter. -- Do not re-describe a recurring character inline per scene — one shared cast - helper so it stays identical. -- Do not invent beats the user didn't ask for — assume ONE scene, add more only - for beats the prompt describes (no cap, but no padding). +- Never pass a `/workspace/...` path where a file_id is wanted, and never retype + a file_id from memory — copy it from `files_out`. +- Do not hand-roll the scratch dir, the frame loop, the frame numbering, the + contact sheet, or the encode — that is what the harness is for. In particular + do not render frames into `/tmp` (a 16 MiB tmpfs) or into `/workspace` itself. +- Do not split render and read across separate code_exec calls, and do not + assume `gifsmith.py` is still in `/workspace` from the last call — pass it in + `files_in` every time. +- Do not re-describe a recurring character inline per scene — one cast helper. +- Do not invent beats the user didn't ask for — assume ONE scene. +- Do not narrate the action in a caption instead of animating it. +- Do not ship a subject too small to recognise, or a stretch of identical frames. - Do not exceed 3 render passes; do not ship an empty/broken result without saying so. diff --git a/gif/scripts/encode.py b/gif/scripts/encode.py deleted file mode 100644 index 1af0b56..0000000 --- a/gif/scripts/encode.py +++ /dev/null @@ -1,115 +0,0 @@ -""" -encode.py — turn a directory of PNG frames into a Discord-friendly GIF or MP4. - -Use it INSIDE a single code_exec call. Render your frames to a scratch dir as -frame_00000.png, frame_00001.png, … (fixed 5-digit width, ONE continuous -counter across all scenes), then: - - from encode import encode_animation - art = encode_animation("/tmp/frames", fps=15, seconds_hint=8) - # art == {"path": "/workspace/final.mp4", "mime": "video/mp4", "bytes": 1234567} - -It writes the finished artifact to the TOP of /workspace (so code_exec returns a -file_id for it) and picks the format for you: - - * short & light -> animated GIF (fun, loops, autoplays) - * long (many seconds+), - OR a GIF too big to post -> H.264 MP4: yuv420p, even dimensions, +faststart - — the combination Discord needs to PLAY it inline - (a video whose dimensions Discord can't read is - shown as a dead download, not a player). - -Requires ffmpeg (always present in the sandbox). Uses gifski + gifsicle for a -cleaner GIF when they're installed, falling back to ffmpeg's palette pipeline. -This helper only ENCODES — you draw the frames. -""" - -import glob -import os -import shutil -import subprocess - -# A GIF longer than this balloons past what's worth shipping as a GIF — encode -# straight to MP4 instead. Tune down for chattier channels. -GIF_MAX_SECONDS = 20 -# Keep a GIF comfortably under Discord's ~10 MiB inline limit; over this we -# re-encode to an MP4 and drop the GIF. -GIF_MAX_BYTES = 9 * 1024 * 1024 - - -def _which(prog): - return shutil.which(prog) is not None - - -def _run(cmd): - p = subprocess.run(cmd, capture_output=True, text=True) - if p.returncode != 0: - raise RuntimeError(f"{cmd[0]} failed:\n{p.stderr[-2000:]}") - - -def _frames(frames_dir): - fs = sorted(glob.glob(os.path.join(frames_dir, "frame_*.png"))) - if not fs: - raise FileNotFoundError( - f"no frame_*.png in {frames_dir} — render frames as " - f"{frames_dir}/frame_00000.png, frame_00001.png, … first" - ) - return fs - - -def _encode_gif(frames_dir, fps, out, long_edge): - fs = _frames(frames_dir) - if _which("gifski"): - # gifski takes an explicit, sorted file list — no numbering-gap surprises. - _run(["gifski", "-o", out, "--fps", str(fps), "--quality", "90"] + fs) - else: - vf = (f"fps={fps},scale='min({long_edge},iw)':-1:flags=lanczos," - f"split[s0][s1];[s0]palettegen=stats_mode=diff[p];" - f"[s1][p]paletteuse=dither=bayer:bayer_scale=3") - _run(["ffmpeg", "-y", "-pattern_type", "glob", - "-i", os.path.join(frames_dir, "frame_*.png"), "-vf", vf, out]) - if _which("gifsicle"): - _run(["gifsicle", "-O3", "--lossy=60", out, "-o", out]) - - -def _encode_mp4(frames_dir, fps, out, long_edge): - _frames(frames_dir) # validate presence - # -2 forces EVEN width/height (H.264 requires it); yuv420p + faststart make - # Discord/browsers play it inline instead of offering a download. - vf = f"scale='min({long_edge},iw)':-2:flags=lanczos" - _run(["ffmpeg", "-y", "-framerate", str(fps), "-pattern_type", "glob", - "-i", os.path.join(frames_dir, "frame_*.png"), - "-c:v", "libx264", "-pix_fmt", "yuv420p", "-crf", "23", - "-vf", vf, "-movflags", "+faststart", out]) - - -def encode_animation(frames_dir, fps=15, seconds_hint=0, long_edge=640): - """Encode frames_dir → /workspace/final.{gif,mp4}. Returns {path, mime, bytes}. - - seconds_hint: your intended runtime in seconds (falls back to frame_count/fps - when 0). Pass it for long pieces so the format decision doesn't depend on how - many frames you happened to render. - long_edge: cap for the output's long side in px (keeps size/encode sane). - """ - n = len(_frames(frames_dir)) - seconds = seconds_hint or (n / max(fps, 1)) - gif, mp4 = "/workspace/final.gif", "/workspace/final.mp4" - - if seconds > GIF_MAX_SECONDS: - _encode_mp4(frames_dir, fps, mp4, long_edge) - return _art(mp4, "video/mp4") - - _encode_gif(frames_dir, fps, gif, long_edge) - if os.path.getsize(gif) <= GIF_MAX_BYTES: - return _art(gif, "image/gif") - - # Too heavy as a GIF — hand Discord an MP4 it can actually render. - _encode_mp4(frames_dir, fps, mp4, long_edge) - os.remove(gif) - return _art(mp4, "video/mp4") - - -def _art(path, mime): - b = os.path.getsize(path) - print(f"[encode] {path} {mime} {b / 1024 / 1024:.2f} MiB") - return {"path": path, "mime": mime, "bytes": b} diff --git a/gif/scripts/gifsmith.py b/gif/scripts/gifsmith.py new file mode 100644 index 0000000..a7ecc10 --- /dev/null +++ b/gif/scripts/gifsmith.py @@ -0,0 +1,421 @@ +""" +gifsmith.py — the render harness for the `gif` skill. + +You write ONLY the drawing. This module owns every part that has historically +broken renders: the scratch directory and its disk budget, frame naming and the +continuous counter, a LOCKED canvas size (frames that differ by one pixel break +the encoder), per-scene crash isolation, the GIF-vs-MP4 decision, and the +critique contact sheet. + + from gifsmith import Animation + + anim = Animation(width=640, height=480, fps=15) + + # --- cast: define each recurring character ONCE --- + def draw_steve(ax, x, y, angry=0.0): + ... + + @anim.scene("Steve watches the printer", seconds=2.5) + def s0(ax, t): # t goes 0.0 -> 1.0 across the scene + draw_steve(ax, 2, 1, angry=t) + + art = anim.render() # -> {"path": "/workspace/final.gif", "mime": ..., "bytes": ...} + +`ax` is a fresh matplotlib Axes at exactly (width, height) pixels, x in [0,100], +y in [0, 100*height/width], origin bottom-left, no margins. Draw in those units +and every frame lines up. + +PIL instead of matplotlib: + + @anim.scene("title card", seconds=1.5, mode="pil") + def s1(img, draw, t): # img: RGB Image at the locked size; draw: ImageDraw + draw.text((40, 40), "THE END", fill=(255, 255, 255), font=anim.font(48)) + +render() prints a report and returns the artifact dict. If one scene raises, the +OTHER scenes still render and ship — the report names the broken one so you can +fix just that scene. It also writes /workspace/still.png: a labelled contact +sheet, one tile per scene, for the critique pass. +""" + +import glob +import math +import os +import shutil +import subprocess +import sys +import traceback + +import matplotlib +matplotlib.use("Agg") +import matplotlib.pyplot as plt # noqa: E402 +from PIL import Image, ImageDraw, ImageFont # noqa: E402 + +# Re-exported so one import line covers the whole drawing vocabulary — a missing +# `from matplotlib.patches import ...` is otherwise a NameError that costs a pass. +from matplotlib.patches import ( # noqa: E402,F401 + Arc, Circle, Ellipse, FancyBboxPatch, Polygon, Rectangle, Wedge) +import numpy as np # noqa: E402,F401 + +# A GIF longer than this balloons past what's worth shipping as a GIF. +GIF_MAX_SECONDS = 20 +# Keep a GIF comfortably under Discord's ~10 MiB inline limit. +GIF_MAX_BYTES = 9 * 1024 * 1024 +# Measured on busy 640x480 matplotlib frames in this sandbox: ~86 KiB per PNG at +# compress_level=1, i.e. 88064 / 307200 px. This decides whether a render streams +# instead of staging frames on disk, so it must not read low — erring low +# reintroduces the ENOSPC this harness exists to prevent, and erring high only +# turns a borderline short piece into an MP4. +PNG_BYTES_PER_PIXEL = 0.29 + +_FONT_CANDIDATES = [ + "/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf", + "/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf", +] + + +def _which(prog): + return shutil.which(prog) is not None + + +def _free_mb(path): + try: + st = os.statvfs(path) + return st.f_bavail * st.f_frsize / 1024 / 1024 + except OSError: + return 0.0 + + +def _pick_scratch(): + """Largest writable scratch dir. /tmp is a small tmpfs in this sandbox; a + SUBDIR of /workspace is bigger and is not returned in files_out (only + TOP-LEVEL /workspace files are), so it is safe to fill with frames.""" + best, best_free = None, -1.0 + for base in ("/workspace/.frames", "/tmp/frames", "/var/tmp/frames"): + parent = os.path.dirname(base) + if not os.path.isdir(parent) or not os.access(parent, os.W_OK): + continue + free = _free_mb(parent) + if free > best_free: + best, best_free = base, free + if best is None: + raise RuntimeError("no writable scratch directory found") + shutil.rmtree(best, ignore_errors=True) + os.makedirs(best, exist_ok=True) + return best, best_free + + +class _Scene: + def __init__(self, fn, title, seconds, mode, hold): + self.fn, self.title, self.seconds = fn, title, float(seconds), + self.mode, self.hold = mode, float(hold) + self.frames = 0 + self.error = None + self.first_frame = None + + +class Animation: + """Collects scenes, renders them to a locked-size frame sequence, encodes.""" + + def __init__(self, width=640, height=480, fps=15, dpi=100, bg="white"): + # H.264 needs even dimensions; lock them here so the encoder never fails. + self.width = int(width) // 2 * 2 + self.height = int(height) // 2 * 2 + self.fps = int(fps) + self.dpi = int(dpi) + self.bg = bg + self.xmax = 100.0 + self.ymax = 100.0 * self.height / self.width + self._scenes = [] + self._count = 0 + self._scratch = None + self._peak_mb = 0.0 + self._streaming = False + self._pipe = None + + # ---------------------------------------------------------------- authoring + + def scene(self, title, seconds=2.0, mode="mpl", hold=0.0): + """Decorator: register a scene. `hold` freezes the last frame for that + many extra seconds — use it to let a caption stay readable.""" + def deco(fn): + self._scenes.append(_Scene(fn, title, seconds, mode, hold)) + return fn + return deco + + def font(self, size): + for p in _FONT_CANDIDATES: + if os.path.exists(p): + return ImageFont.truetype(p, size) + return ImageFont.load_default() + + def text(self, ax, s, size=18, y=None, color="white", box="black"): + """Caption helper: centred, boxed, legible at GIF resolution.""" + ax.text(self.xmax / 2, self.ymax * 0.06 if y is None else y, s, + ha="center", va="center", fontsize=size, color=color, zorder=999, + bbox=dict(boxstyle="round,pad=0.4", fc=box, ec="none", alpha=0.75)) + + def add_frame(self, img, times=1): + """Escape hatch for hand-built frames (PIL compositing, pasted photos, + anything the scene decorators can't express). Still size-locked.""" + for _ in range(max(1, int(times))): + self._emit(img) + + # ------------------------------------------------------------------ drawing + + def _new_axes(self): + fig = plt.figure(figsize=(self.width / self.dpi, self.height / self.dpi), + dpi=self.dpi) + ax = fig.add_axes([0, 0, 1, 1]) + ax.set_xlim(0, self.xmax) + ax.set_ylim(0, self.ymax) + ax.set_axis_off() + ax.set_facecolor(self.bg) + fig.patch.set_facecolor(self.bg) + return fig, ax + + def _emit(self, img): + """Emit one frame at exactly the locked size. Never bbox_inches='tight'. + + Short pieces go to PNG files (gifski wants a file list). Long pieces are + piped straight into ffmpeg so a multi-minute render never touches disk — + the sandbox scratch would otherwise cap you at well under a minute.""" + if img.size != (self.width, self.height): + img = img.resize((self.width, self.height), Image.LANCZOS) + img = img.convert("RGB") + if self._pipe is not None: + img.save(self._pipe.stdin, format="PNG", compress_level=1) + else: + img.save(os.path.join(self._scratch, f"frame_{self._count:05d}.png"), + compress_level=1) + self._count += 1 + + def _render_mpl(self, sc, t): + fig, ax = self._new_axes() + try: + sc.fn(ax, t) + fig.canvas.draw() + img = Image.frombuffer( + "RGBA", fig.canvas.get_width_height(), + fig.canvas.buffer_rgba(), "raw", "RGBA", 0, 1).convert("RGB") + finally: + plt.close(fig) + return img + + def _render_pil(self, sc, t): + img = Image.new("RGB", (self.width, self.height), + self.bg if isinstance(self.bg, (str, tuple)) else "white") + sc.fn(img, ImageDraw.Draw(img), t) + return img + + # ------------------------------------------------------------------- render + + def render(self, long_edge=None, force=None, audio=None): + if not self._scenes: + raise RuntimeError("no scenes registered — decorate at least one " + "function with @anim.scene(...)") + self._scratch, free_mb = _pick_scratch() + long_edge = long_edge or max(self.width, self.height) + + total_s = sum(s.seconds + s.hold for s in self._scenes) + est_frames = int(total_s * self.fps) + est_mb = est_frames * self.width * self.height * PNG_BYTES_PER_PIXEL / 1048576 + + # A piece too long for a GIF is an MP4 anyway, and an MP4 can be encoded + # in one streaming pass — so pipe the frames into ffmpeg and keep NOTHING + # on disk. That is what makes multi-minute pieces possible at all: the + # sandbox scratch tops out around a minute of busy frames. + self._streaming = (force == "mp4" or audio is not None + or (force != "gif" and (total_s > GIF_MAX_SECONDS + or est_mb > free_mb * 0.6))) + self._pipe = None + if self._streaming: + self._pipe = self._open_pipe("/workspace/final.mp4", long_edge, audio) + print(f"[gifsmith] streaming to MP4 (projected {est_mb:.0f} MiB of " + f"frames vs {free_mb:.0f} MiB scratch) — nothing hits disk", + flush=True) + elif est_mb > free_mb * 0.6: + print(f"[gifsmith] WARNING projected frames ~{est_mb:.0f} MiB vs " + f"{free_mb:.0f} MiB free in {os.path.dirname(self._scratch)} — " + f"dropping fps or canvas would be safer", file=sys.stderr) + + print(f"[gifsmith] {len(self._scenes)} scene(s), {total_s:.1f}s at " + f"{self.fps}fps, canvas {self.width}x{self.height}", flush=True) + + for i, sc in enumerate(self._scenes): + start = self._count + n = max(1, int(round(sc.seconds * self.fps))) + try: + for k in range(n): + t = k / max(n - 1, 1) + img = (self._render_pil if sc.mode == "pil" else self._render_mpl)(sc, t) + if sc.first_frame is None and k >= n // 2: + sc.first_frame = img.copy() + self._emit(img) + for _ in range(int(round(sc.hold * self.fps))): + self._emit(img) + except Exception: # noqa: BLE001 + sc.error = traceback.format_exc(limit=6) + if not self._streaming: + # Drop this scene's partial frames so it can't ship half-drawn. + # (Streamed frames are already in the encoder — a broken scene + # leaves its partial beat in the MP4; the report says so.) + for p in sorted(glob.glob( + os.path.join(self._scratch, "frame_*.png")))[start:]: + os.remove(p) + self._count = start + print(f"[gifsmith] SCENE {i} ({sc.title!r}) FAILED — skipped, " + f"other scenes continue:\n{sc.error}", file=sys.stderr, flush=True) + continue + sc.frames = self._count - start + print(f"[gifsmith] scene {i}: {sc.frames} frames {sc.title!r}", flush=True) + self._renumber_if_needed() + + used = sum(os.path.getsize(p) for p in + glob.glob(os.path.join(self._scratch, "frame_*.png"))) + self._peak_mb = used / 1024 / 1024 + if self._count == 0: + raise RuntimeError( + "every scene failed — see the per-scene tracebacks above; " + "nothing was encoded. Fix the error they all share (usually one " + "cast helper called with a keyword it does not take, or a name " + "that differs by a character) and re-run the whole program in a " + "fresh code_exec call, remembering gifsmith.py in files_in.") + + try: + art = self._encode(total_s, long_edge, force, audio) + except Exception: # noqa: BLE001 + # A half-written final.* at the top of /workspace still comes back as + # a files_out file_id, and a broken artifact that LOOKS deliverable is + # worse than none — the agent would ship it. Take it with us. + for leftover in ("/workspace/final.mp4", "/workspace/final.gif"): + if os.path.exists(leftover): + os.remove(leftover) + shutil.rmtree(self._scratch, ignore_errors=True) + raise + self._contact_sheet() + broken = [f"{i}:{s.title!r}" for i, s in enumerate(self._scenes) if s.error] + print(f"[gifsmith] {self._count} frames, scratch " + + ("streamed (0 MiB on disk)" if self._streaming + else f"peak {self._peak_mb:.1f} MiB")) + print(f"[gifsmith] scene errors: {', '.join(broken) if broken else 'none'}") + print(f"[gifsmith] ART {art}") + print("[gifsmith] still: /workspace/still.png (contact sheet — call " + "image_describe on ITS file_id from files_out)") + shutil.rmtree(self._scratch, ignore_errors=True) + return art + + def _renumber_if_needed(self): + """A skipped scene leaves no gaps because _count rewinds; kept as a + no-op hook so the counter contract is explicit.""" + return + + # ------------------------------------------------------------------ encoding + + def _open_pipe(self, out, long_edge, audio): + """ffmpeg reading a PNG stream on stdin — the streaming MP4 path. + + stderr goes to a FILE, not a pipe: nobody drains a stderr pipe until the + encode is done, so a chatty ffmpeg would fill the 64 KiB pipe buffer, + block, stop reading stdin, and deadlock the render halfway through.""" + cmd = ["ffmpeg", "-y", "-loglevel", "error", + "-f", "image2pipe", "-framerate", str(self.fps), "-i", "-"] + if audio: + cmd += ["-i", audio, "-c:a", "aac", "-b:a", "128k", "-shortest"] + cmd += ["-c:v", "libx264", "-pix_fmt", "yuv420p", "-crf", "23", + "-vf", f"scale='min({long_edge},iw)':-2:flags=lanczos", + "-movflags", "+faststart", out] + self._pipe_err = open(os.path.join(self._scratch, "ffmpeg.log"), "w+b") + return subprocess.Popen(cmd, stdin=subprocess.PIPE, + stdout=subprocess.DEVNULL, stderr=self._pipe_err) + + def _close_pipe(self): + try: + self._pipe.stdin.close() + except OSError: + # ffmpeg already died; its exit code + log below say why. + pass + rc = self._pipe.wait() + self._pipe_err.seek(0) + err = self._pipe_err.read().decode(errors="replace") + self._pipe_err.close() + self._pipe = None + if rc != 0: + raise RuntimeError(f"ffmpeg (streaming) exited {rc}:\n{err[-2000:]}") + + def _encode(self, seconds, long_edge, force, audio=None): + gif, mp4 = "/workspace/final.gif", "/workspace/final.mp4" + if self._streaming: + self._close_pipe() + return _art(mp4, "video/mp4") + fs = sorted(glob.glob(os.path.join(self._scratch, "frame_*.png"))) + want_mp4 = (force == "mp4" or audio is not None + or (force != "gif" and seconds > GIF_MAX_SECONDS)) + if want_mp4: + self._mp4(mp4, long_edge, audio) + return _art(mp4, "video/mp4") + self._gif(fs, gif, long_edge) + if os.path.getsize(gif) <= GIF_MAX_BYTES: + return _art(gif, "image/gif") + print(f"[gifsmith] GIF {os.path.getsize(gif)/1048576:.1f} MiB is too big " + f"for Discord — re-encoding as MP4", flush=True) + self._mp4(mp4, long_edge, audio) + os.remove(gif) + return _art(mp4, "video/mp4") + + def _gif(self, fs, out, long_edge): + if _which("gifski"): + _run(["gifski", "-o", out, "--fps", str(self.fps), "--quality", "90", + "--width", str(min(long_edge, self.width))] + fs) + else: + vf = (f"fps={self.fps},scale='min({long_edge},iw)':-1:flags=lanczos," + f"split[s0][s1];[s0]palettegen=stats_mode=diff[p];" + f"[s1][p]paletteuse=dither=bayer:bayer_scale=3") + _run(["ffmpeg", "-y", "-loglevel", "error", "-pattern_type", "glob", + "-i", os.path.join(self._scratch, "frame_*.png"), "-vf", vf, out]) + if _which("gifsicle"): + _run(["gifsicle", "-O3", "--lossy=60", out, "-o", out]) + + def _mp4(self, out, long_edge, audio=None): + cmd = ["ffmpeg", "-y", "-loglevel", "error", "-framerate", str(self.fps), + "-pattern_type", "glob", + "-i", os.path.join(self._scratch, "frame_*.png")] + if audio: + cmd += ["-i", audio, "-c:a", "aac", "-b:a", "128k", "-shortest"] + cmd += ["-c:v", "libx264", "-pix_fmt", "yuv420p", "-crf", "23", + "-vf", f"scale='min({long_edge},iw)':-2:flags=lanczos", + "-movflags", "+faststart", out] + _run(cmd) + + # ------------------------------------------------------------- critique aid + + def _contact_sheet(self, path="/workspace/still.png"): + tiles = [(i, s) for i, s in enumerate(self._scenes) if s.first_frame is not None] + if not tiles: + return + if len(tiles) == 1: + tiles[0][1].first_frame.save(path) + return + cols = min(3, len(tiles)) + rows = math.ceil(len(tiles) / cols) + tw = 300 + th = max(1, round(self.height * tw / self.width)) + sheet = Image.new("RGB", (cols * tw, rows * (th + 20)), (20, 20, 24)) + d = ImageDraw.Draw(sheet) + f = self.font(14) + for k, (i, sc) in enumerate(tiles): + x, y = (k % cols) * tw, (k // cols) * (th + 20) + sheet.paste(sc.first_frame.resize((tw, th), Image.LANCZOS), (x, y + 20)) + d.text((x + 4, y + 3), f"scene {i}: {sc.title[:38]}", fill=(235, 235, 240), font=f) + sheet.save(path) + + +def _run(cmd): + p = subprocess.run(cmd, capture_output=True, text=True) + if p.returncode != 0: + raise RuntimeError(f"{cmd[0]} failed:\n{p.stderr[-2000:]}") + + +def _art(path, mime): + b = os.path.getsize(path) + print(f"[gifsmith] encoded {path} {mime} {b / 1024 / 1024:.2f} MiB") + return {"path": path, "mime": mime, "bytes": b}