# ADR-0025: Wave-3 video surfaces (lipsync, video matte, video upscale, chain jobs)

Status: Accepted (2026-07-16)

## Context

The llama-swap host is gaining wave-3 video capabilities (spec: mort
docs/specs/2026-07-16-llamaswap-wave3.md): SadTalker talking heads, Robust
Video Matting and per-frame Real-ESRGAN on the mediautils shim, and a
videoutils orchestrator that generates LONG videos as a chain of i2v
segments (generate → extract last frame → continue → concat → optional RIFE
smoothing), exposed as an async job API following the ACE-Step precedent.

## Decision

1. **Three new optional `videogen` interfaces**, ADR-0016→0024 conventions:
   - `videogen.LipSyncer` / `LipsyncProvider` — `LipsyncRequest{Image,
     Audio, AudioMIME, AudioFilename, Still, Enhance, Preprocess}` →
     `POST /upstream/<id>/v1/talking_head` (multipart `image` + `audio`
     file parts + optional `still`/`enhance`/`preprocess` fields) → mp4.
     Sync and minutes-slow (Hunyuan precedent); context deadline is the
     budget.
   - `videogen.VideoBackgroundRemover` / `VideoBackgroundRemovalProvider` —
     `VideoBackgroundRemovalRequest{Video, MIME, Filename, Output}` →
     `POST /upstream/<id>/v1/video/matte`. `Output` ∈
     `greenscreen_mp4` (universally playable) | `alpha_webm` (true
     transparency); "" = backend default.
   - `videogen.VideoUpscaler` / `VideoUpscaleProvider` —
     `VideoUpscaleRequest{Video, MIME, Filename, Scale}` (2 or 4) →
     `POST /upstream/<id>/v1/video/upscale` → mp4.
2. **The chain client is deliberately ASYNC** (`videogen.Chainer` /
   `ChainerProvider`), unlike musicgen's blocking Generate (ADR-0021): a
   chain holds the GPU through multiple model swaps for many minutes, and
   the caller must be able to poll progress AND fetch completed segments
   after a mid-chain failure — partial delivery is mandatory (never discard
   multi-minute GPU output), which a blocking one-call contract cannot
   express.
   - `SubmitChain(ctx, ChainRequest{Segments[{Prompt,Seconds}], InitImage,
     SmoothJoins, Size}) (jobID, error)` — JSON
     `POST /upstream/<id>/v1/video/chain`, init image as base64
     `init_image_b64` (JSON submit, not multipart, per the pinned host
     contract).
   - `ChainStatus(ctx, jobID) (*ChainJob{Status, Segment, Total,
     SegmentIDs, Raw})` — `GET /v1/jobs/{id}`; polling doubles as a
     liveness signal for the shim's idle TTL. Segment entries are tolerated
     as strings or `{id|segment_id}` objects.
   - `ChainResult(ctx, jobID)` / `ChainSegmentResult(ctx, jobID, n)` —
     `GET /v1/jobs/{id}/result` and `/v1/jobs/{id}/segments/{n}`.
   - Job ids are echoed server input: job paths reject ids carrying path
     structure (`/?#`, `..`), the upstreamPath smuggling rule.
3. **Binary success bodies are validated before wrapping** (ADR-0020 rule):
   all four clip-returning calls go through a shared `singleVideoResult`
   (positive evidence of video-ness — declared video/* Content-Type or
   sniffed mp4/webm magic — so a JSON status page or proxy error can never
   become "the clip").

## Consequences

- videogen grows from two surfaces (Model, Interpolator) to six; the
  chain client is the package's first async surface — the job-API shape
  deferred in ADR-0019/0021 now exists where the workload actually
  demands it.
- The wire shapes are pinned by the netherstorm videoutils/mediautils/
  SadTalker image builds; host smoke tests are the drift defence.
- mort's long-video tool owns the poll loop, timeout budget, and
  partial-result envelope; majordomo only guarantees the artifacts stay
  fetchable.