feat(videogen): LastImage — pin the trailing keyframe (first-last-frame-to-video)
Gadfly review (reusable) / review (pull_request) Successful in 3m41s
Adversarial Review (Gadfly) / review (pull_request) Successful in 3m41s
CI / Tidy (pull_request) Successful in 9m26s
CI / Build & Test (pull_request) Successful in 9m48s

videogen.Request gains LastImage alongside InitImage, so one Request covers
t2v, i2v and FL2V without a mode flag. With InitImage it pins both ends of the
clip; alone it pins the destination and lets the backend invent the approach.

The llamaswap provider sends it as a SEPARATE `input_reference_last` part
rather than a second `input_reference`. Multipart permits repeated names, but
then which frame is first and which is last depends on part ORDER — an
ordering contract invisible in the payload, that nothing notices breaking. A
backend that does not know the new name ignores the part, the same degradation
as any other unknown field.

Both parts go through one writeImagePart helper so their encoding cannot
drift, and an empty LastImage is rejected up front exactly as InitImage
already is.

Support is per-model and deliberately NOT advertised in this contract: a
backend that ignores a trailing keyframe returns an ordinary clip, which is
indistinguishable from success. The doc comment says so, because a caller that
needs to know whether the pin took effect has to establish that out of band —
and the mort side gates on a convar for exactly this reason.

Motivated by mort's #1567 (long-form video): with both ends pinned, drift
becomes structurally bounded inside each shot instead of compounding across an
autoregressive chain.

Tests break-checked: sending the last frame under the shared name fails both
the distinct-name assertion and the last-alone case.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01PLjgrxvHjm1sJgUu9zBPH9
This commit is contained in:
2026-08-08 02:47:39 -04:00
co-authored by Claude Opus 5
parent 203895696c
commit 44fcfbb273
3 changed files with 163 additions and 5 deletions
+29 -5
View File
@@ -56,6 +56,9 @@ func (m *videoModel) Generate(ctx context.Context, req videogen.Request, opts ..
if req.InitImage != nil && len(req.InitImage.Data) == 0 {
return nil, fmt.Errorf("%w: video init image has no bytes", llm.ErrUnsupported)
}
if req.LastImage != nil && len(req.LastImage.Data) == 0 {
return nil, fmt.Errorf("%w: video last image has no bytes", llm.ErrUnsupported)
}
width, height, err := parseSize(req.Size)
if err != nil {
return nil, fmt.Errorf("%w: %v", llm.ErrUnsupported, err)
@@ -83,12 +86,19 @@ func (m *videoModel) Generate(ctx context.Context, req videogen.Request, opts ..
return nil, err
}
if req.InitImage != nil {
fw, err := w.CreateFormFile("input_reference", initImageFilename(req.InitImage.MIME))
if err != nil {
return nil, fmt.Errorf("llama-swap: build video form: %w", err)
if err := writeImagePart(w, "input_reference", req.InitImage); err != nil {
return nil, err
}
if _, err := fw.Write(req.InitImage.Data); err != nil {
return nil, fmt.Errorf("llama-swap: build video form: %w", err)
}
// The trailing keyframe rides a SEPARATE part rather than a second
// `input_reference`: multipart permits repeated names, but the receiving
// end would then have to rely on part ORDER to tell first from last, and
// an ordering contract that is invisible in the field name is one nobody
// can see they have broken. A backend that does not know the name ignores
// the part, which is the same degradation as any other unknown field.
if req.LastImage != nil {
if err := writeImagePart(w, "input_reference_last", req.LastImage); err != nil {
return nil, err
}
}
if err := w.Close(); err != nil {
@@ -141,6 +151,20 @@ func initImageFilename(mimeType string) string {
return imageFilename(mimeType, "frame")
}
// writeImagePart attaches one conditioning frame under the given field name.
// Shared by the first- and last-frame parts so the two cannot drift in how
// they encode, which is the usual way a second copy of a block goes wrong.
func writeImagePart(w *multipart.Writer, field string, img *videogen.Image) error {
fw, err := w.CreateFormFile(field, initImageFilename(img.MIME))
if err != nil {
return fmt.Errorf("llama-swap: build video form: %w", err)
}
if _, err := fw.Write(img.Data); err != nil {
return fmt.Errorf("llama-swap: build video form: %w", err)
}
return nil
}
// formatInt renders an optional int pointer for a form field; nil = "" (omit).
func formatInt(v *int) string {
if v == nil {