gif: the drawing coordinate space is undocumented, so agents draw in pixels and render a blank GIF that reports success #5

Open
opened 2026-07-27 00:32:42 +00:00 by steve · 0 comments
Owner

Found monitoring production run cff9e803 (the seventh "pickle musical", 2026-07-27).

The trap

Animation(width=1280, height=720) names two numbers in pixels. The matplotlib
axes the scene function actually draws into are data range x∈[0,100],
y∈[0,56.25]
. Nothing in the skill doc says so.

I searched the 3993-char SKILL.md that the worker received:

term hits
coordinate 0
axes 0
xlim / ylim 0
0,100 / 56.25 0
data range 0

The only route to this fact is reading gifsmith.py's source.

What it cost, verbatim

Worker 701eb3d1 (scene: "Pickle Rambo charge") drew its whole scene in pixel
coordinates. The render succeeded:

[gifsmith] 1 scene(s), 4.5s at 15fps, canvas 1280x720
[gifsmith]   scene 0: 68 frames  'Pickle Rambo charge'
[gifsmith] encoded /workspace/final.gif  image/gif  1.11 MiB
[gifsmith] scene errors: none

Exit code 0. scene errors: none. A real 1.11 MiB GIF. Every pixel of it blank,
because the drawing was entirely off-axes.

It was caught only by the critique loop — image_describe came back:

Looking at the 3×3 contact sheet, every single tile is completely blank.

The worker then spent five consecutive code_exec calls reverse-engineering
the harness:

seq call
15 print(inspect.signature(Animation.text))
18 print(inspect.getsource(Animation.text))
27 inspect.getsource(Animation)__init__
30 inspect.getsource(Animation)_render_mpl
33 inspect.getsource(Animation)_new_axes

and concluded at iteration 11:

Found it — the harness axes use data range x∈[0,100], y∈[0,56.25], but I drew
in raw pixel coordinates (0–1280, 0–720), so everything was off-canvas.

Total cost: 1 wasted render + 1 wasted image_describe + 5 introspection calls
≈ 7 turns, on a worker whose whole job takes ~18.

The same misconception shows up in a second worker: ac395484 called
anim.text(ax, "AAAARRRGH!!", x=820, y=560, ...) — 820,560 are pixel
coordinates for a 1280×720 canvas.

Suggested fixes (either alone would have prevented it)

  1. Document it in SKILL.md. One line: scene functions draw in data
    coordinates x∈[0,100], y∈[0,100/aspect] (56.25 at 16:9) — not pixels.
    Aspect-dependence is exactly why an agent cannot guess it.
  2. Make the blank render loud. A scene whose artists all fall outside the
    axes limits is never intentional. Checking the drawn artists' bbox against
    the axes after the t=0 dry run and emitting
    [gifsmith] SCENE 0 WARNING: all artists are outside the axes (drew x∈[0,1280] vs axes x∈[0,100]) — are you drawing in pixels?
    turns a silent success into a self-diagnosing one. This follows the rule the
    repo already applies elsewhere: the harness may refuse or discard, but never
    quietly.

Related, same family, from the same run

Both are "a keyword that reads as obviously right and is not", which is the
class PR #4 (closes mort#1522) already fixed for Animation(force=…) and
Animation(background=…) — that difflib handler IS live in this run, so this is
about the surfaces it does not cover:

  • Animation.scene(force="gif")TypeError: Animation.scene() got an unexpected keyword argument 'force' (701eb3d1 seq=10). Same force
    confusion the Animation() path now catches by name, one constructor over.
  • z= instead of zorder= on re-exported matplotlib patches
    Ellipse(..., z=zleg+1) then Polygon(..., z=9), two separate turns in
    ac395484 (seq 10, 13). matplotlib's own error (Ellipse.set() got an unexpected keyword argument 'z') names the symptom but not the fix, and the
    agent hit it twice because fixing one occurrence didn't teach it the rule.
  • Animation.text(x=…, y=…) still raises TypeError: ... unexpected keyword argument 'x'. Did you mean 'ax'? (701eb3d1 seq=13). Python's own suggestion
    here is actively wrong — ax is the positional axes handle already being
    passed, not a position. The real signature is
    text(self, ax, s, size=18, y=None, color='white', box='black'), i.e. there
    is no x at all and y is in data units — which loops back to the main
    issue above.

Run: cff9e803-c952-4306-9725-0cf5d3db4d30, worker 701eb3d1.

Found monitoring production run `cff9e803` (the seventh "pickle musical", 2026-07-27). ## The trap `Animation(width=1280, height=720)` names two numbers in pixels. The matplotlib axes the scene function actually draws into are **data range `x∈[0,100]`, `y∈[0,56.25]`**. Nothing in the skill doc says so. I searched the 3993-char `SKILL.md` that the worker received: | term | hits | |---|---| | `coordinate` | 0 | | `axes` | 0 | | `xlim` / `ylim` | 0 | | `0,100` / `56.25` | 0 | | `data range` | 0 | The only route to this fact is reading `gifsmith.py`'s source. ## What it cost, verbatim Worker `701eb3d1` (scene: "Pickle Rambo charge") drew its whole scene in pixel coordinates. The render **succeeded**: ``` [gifsmith] 1 scene(s), 4.5s at 15fps, canvas 1280x720 [gifsmith] scene 0: 68 frames 'Pickle Rambo charge' [gifsmith] encoded /workspace/final.gif image/gif 1.11 MiB [gifsmith] scene errors: none ``` Exit code 0. `scene errors: none`. A real 1.11 MiB GIF. Every pixel of it blank, because the drawing was entirely off-axes. It was caught only by the critique loop — `image_describe` came back: > Looking at the 3×3 contact sheet, **every single tile is completely blank.** The worker then spent **five consecutive `code_exec` calls** reverse-engineering the harness: | seq | call | |---|---| | 15 | `print(inspect.signature(Animation.text))` | | 18 | `print(inspect.getsource(Animation.text))` | | 27 | `inspect.getsource(Animation)` → `__init__` | | 30 | `inspect.getsource(Animation)` → `_render_mpl` | | 33 | `inspect.getsource(Animation)` → `_new_axes` | and concluded at iteration 11: > Found it — the harness axes use data range `x∈[0,100], y∈[0,56.25]`, but I drew > in raw pixel coordinates (0–1280, 0–720), so everything was off-canvas. Total cost: 1 wasted render + 1 wasted `image_describe` + 5 introspection calls ≈ 7 turns, on a worker whose whole job takes ~18. The same misconception shows up in a second worker: `ac395484` called `anim.text(ax, "AAAARRRGH!!", x=820, y=560, ...)` — 820,560 are pixel coordinates for a 1280×720 canvas. ## Suggested fixes (either alone would have prevented it) 1. **Document it in `SKILL.md`.** One line: scene functions draw in data coordinates `x∈[0,100]`, `y∈[0,100/aspect]` (56.25 at 16:9) — *not* pixels. Aspect-dependence is exactly why an agent cannot guess it. 2. **Make the blank render loud.** A scene whose artists all fall outside the axes limits is never intentional. Checking the drawn artists' bbox against the axes after the t=0 dry run and emitting `[gifsmith] SCENE 0 WARNING: all artists are outside the axes (drew x∈[0,1280] vs axes x∈[0,100]) — are you drawing in pixels?` turns a silent success into a self-diagnosing one. This follows the rule the repo already applies elsewhere: the harness may refuse or discard, but never quietly. ## Related, same family, from the same run Both are "a keyword that reads as obviously right and is not", which is the class PR #4 (closes mort#1522) already fixed for `Animation(force=…)` and `Animation(background=…)` — that difflib handler IS live in this run, so this is about the surfaces it does not cover: - **`Animation.scene(force="gif")`** → `TypeError: Animation.scene() got an unexpected keyword argument 'force'` (`701eb3d1` seq=10). Same `force` confusion the `Animation()` path now catches by name, one constructor over. - **`z=` instead of `zorder=` on re-exported matplotlib patches** — `Ellipse(..., z=zleg+1)` then `Polygon(..., z=9)`, two separate turns in `ac395484` (seq 10, 13). matplotlib's own error (`Ellipse.set() got an unexpected keyword argument 'z'`) names the symptom but not the fix, and the agent hit it twice because fixing one occurrence didn't teach it the rule. - **`Animation.text(x=…, y=…)`** still raises `TypeError: ... unexpected keyword argument 'x'. Did you mean 'ax'?` (`701eb3d1` seq=13). Python's own suggestion here is actively wrong — `ax` is the positional axes handle already being passed, not a position. The real signature is `text(self, ax, s, size=18, y=None, color='white', box='black')`, i.e. there is no `x` at all and `y` is in *data* units — which loops back to the main issue above. Run: `cff9e803-c952-4306-9725-0cf5d3db4d30`, worker `701eb3d1`.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: steve/executus-skills#5