feat: wave-3 image + document surfaces — segmentation, colorize, face restore, ocr (ADR-0023)
CI / Tidy (pull_request) Successful in 9m25s
CI / Build & Test (pull_request) Successful in 9m48s
Gadfly review (reusable) / review (pull_request) Successful in 47m4s
Adversarial Review (Gadfly) / review (pull_request) Successful in 47m4s

- imagegen.Segmenter/SegmentationProvider: prompted mask via
  POST /upstream/<id>/v1/segment (file, prompt[, threshold], output=mask);
  white = prompted region, EditRequest.Mask polarity.
- imagegen.Colorizer/ColorizeProvider: POST /upstream/<id>/v1/colorize.
- imagegen.FaceRestorer/FaceRestoreProvider:
  POST /upstream/<id>/v1/restore_faces (upscale 1|2).
- New ocr leaf package (Request/Page/Result, Recognize) + llamaswap
  OCRModel: POST /upstream/<id>/v1/ocr (file[, langs, max_pages]),
  tolerant per-page decode (join lines when page text absent), Raw
  escape hatch.
- httptest contract tests per surface; ADR-0023; ADR index backfilled
  (0020-0022 rows were missing).

Co-Authored-By: Claude Fable 5 <[email protected]>
This commit is contained in:
2026-07-16 16:54:30 -04:00
co-authored by Claude Fable 5
parent cd43009672
commit e6987f54b2
11 changed files with 1155 additions and 0 deletions
+59
View File
@@ -0,0 +1,59 @@
# ADR-0023: Wave-3 image + document surfaces (segmentation, colorize, face restore, OCR)
Status: Accepted (2026-07-16)
## Context
The llama-swap host is gaining four wave-3 capabilities (spec:
mort docs/specs/2026-07-16-llamaswap-wave3.md): promptable segmentation
(GroundingDINO + SAM 2.1), photo colorization (DDColor), face restoration
(GFPGAN), and document OCR (Surya). All ride the `/upstream/<model>/<path>`
passthrough (ADR-0020); none of their native APIs are OpenAI-shaped.
## Decision
1. **Grow `imagegen` with three optional interfaces**, ADR-0016→0022
conventions (functional options, zero value = backend default, bytes-only
I/O, provider mints model):
- `imagegen.Segmenter` / `SegmentationProvider` — `SegmentationRequest
{Image, Prompt, Threshold}` → one grayscale mask image, WHITE = the
prompted region. Same polarity as `EditRequest.Mask` (white = repaint),
so the mask feeds inpainting directly; cutouts are derived client-side
(mask-as-alpha), one host call serving both. The client always sends
`output=mask` — the shim's `cutout`/`boxes` modes are not exposed.
- `imagegen.Colorizer` / `ColorizeProvider` — `ColorizeRequest{Image}`,
no knobs (the reference backend takes none; options reserved).
- `imagegen.FaceRestorer` / `FaceRestoreProvider` —
`FaceRestoreRequest{Image, Upscale}` (1 or 2, 0 = backend default).
2. **New `ocr` leaf package** rather than a method on an existing surface:
OCR is document-shaped (multi-page, PDFs), not image-generation-shaped.
`Request{Document, MIME, Filename, Languages, MaxPages}` →
`Result{Text, Pages []Page{Number, Text}, Raw}`. `Result.Text` is the
pages joined with a blank line; the per-line bbox/confidence/layout
detail stays in `Raw` (json.RawMessage) — exact text is the contract,
geometry is the escape hatch.
3. **provider/llamaswap wire shapes** (pinned by the netherstorm image
builds; host smoke tests are the drift defence):
- `POST /upstream/<id>/v1/segment` multipart `file`,`prompt`
[,`threshold`],`output=mask` → mask PNG.
- `POST /upstream/<id>/v1/colorize` multipart `file` → PNG.
- `POST /upstream/<id>/v1/restore_faces` multipart `file`[,`upscale`] → PNG.
- `POST /upstream/<id>/v1/ocr` multipart `file`[,`langs` (comma-joined),
`max_pages`] → JSON `{pages:[{number,text,lines,layout}]}`. The decode
is tolerant: a page without aggregate `text` joins its line texts; a
missing page `number` defaults to position. Zero pages is an error
(a blank page still arrives as a page), mirroring the "no transcript"
honesty rule.
4. **Binary success bodies are validated before wrapping** (ADR-0020 rule):
the three image surfaces reuse `singleImageResult` (positive evidence of
image-ness required), OCR requires decodable JSON.
## Consequences
- imagegen grows from five to eight optional surfaces; consumers
type-assert or use the provider methods directly, as before.
- `ocr` is the seventh leaf media package (imagegen, audio, videogen,
meshgen, musicgen, embeddings, ocr); the conventions have held across all
of them.
- PDF handling lives host-side (the shim rasterizes via pypdfium2);
majordomo ships bytes and never needs a PDF dependency.
+4
View File
@@ -23,3 +23,7 @@ One decision per file, append-only; supersede rather than rewrite.
| [0017](0017-audio-interfaces.md) | audio — canonical speech synthesis + transcription interfaces | Accepted |
| [0018](0018-imagegen-editor.md) | imagegen.Editor — image-to-image as a separate optional interface | Accepted |
| [0019](0019-videogen-interface.md) | videogen — canonical video-generation surface | Accepted |
| [0020](0020-upstream-passthrough-media-surfaces.md) | Upstream-passthrough media surfaces (mask, upscale, background removal, interpolation, diarization, meshgen) | Accepted |
| [0021](0021-musicgen-interface.md) | musicgen — blocking Generate over an async job queue | Accepted |
| [0022](0022-embeddings-rerank-interface.md) | embeddings + rerank interface | Accepted |
| [0023](0023-image-doc-surfaces.md) | Wave-3 image + document surfaces (segmentation, colorize, face restore, OCR) | Accepted |