feat: wave-3 image + document surfaces — segmentation, colorize, face restore, ocr (ADR-0023)
- imagegen.Segmenter/SegmentationProvider: prompted mask via POST /upstream/<id>/v1/segment (file, prompt[, threshold], output=mask); white = prompted region, EditRequest.Mask polarity. - imagegen.Colorizer/ColorizeProvider: POST /upstream/<id>/v1/colorize. - imagegen.FaceRestorer/FaceRestoreProvider: POST /upstream/<id>/v1/restore_faces (upscale 1|2). - New ocr leaf package (Request/Page/Result, Recognize) + llamaswap OCRModel: POST /upstream/<id>/v1/ocr (file[, langs, max_pages]), tolerant per-page decode (join lines when page text absent), Raw escape hatch. - httptest contract tests per surface; ADR-0023; ADR index backfilled (0020-0022 rows were missing). Co-Authored-By: Claude Fable 5 <[email protected]>
This commit is contained in:
@@ -0,0 +1,59 @@
|
||||
# ADR-0023: Wave-3 image + document surfaces (segmentation, colorize, face restore, OCR)
|
||||
|
||||
Status: Accepted (2026-07-16)
|
||||
|
||||
## Context
|
||||
|
||||
The llama-swap host is gaining four wave-3 capabilities (spec:
|
||||
mort docs/specs/2026-07-16-llamaswap-wave3.md): promptable segmentation
|
||||
(GroundingDINO + SAM 2.1), photo colorization (DDColor), face restoration
|
||||
(GFPGAN), and document OCR (Surya). All ride the `/upstream/<model>/<path>`
|
||||
passthrough (ADR-0020); none of their native APIs are OpenAI-shaped.
|
||||
|
||||
## Decision
|
||||
|
||||
1. **Grow `imagegen` with three optional interfaces**, ADR-0016→0022
|
||||
conventions (functional options, zero value = backend default, bytes-only
|
||||
I/O, provider mints model):
|
||||
- `imagegen.Segmenter` / `SegmentationProvider` — `SegmentationRequest
|
||||
{Image, Prompt, Threshold}` → one grayscale mask image, WHITE = the
|
||||
prompted region. Same polarity as `EditRequest.Mask` (white = repaint),
|
||||
so the mask feeds inpainting directly; cutouts are derived client-side
|
||||
(mask-as-alpha), one host call serving both. The client always sends
|
||||
`output=mask` — the shim's `cutout`/`boxes` modes are not exposed.
|
||||
- `imagegen.Colorizer` / `ColorizeProvider` — `ColorizeRequest{Image}`,
|
||||
no knobs (the reference backend takes none; options reserved).
|
||||
- `imagegen.FaceRestorer` / `FaceRestoreProvider` —
|
||||
`FaceRestoreRequest{Image, Upscale}` (1 or 2, 0 = backend default).
|
||||
2. **New `ocr` leaf package** rather than a method on an existing surface:
|
||||
OCR is document-shaped (multi-page, PDFs), not image-generation-shaped.
|
||||
`Request{Document, MIME, Filename, Languages, MaxPages}` →
|
||||
`Result{Text, Pages []Page{Number, Text}, Raw}`. `Result.Text` is the
|
||||
pages joined with a blank line; the per-line bbox/confidence/layout
|
||||
detail stays in `Raw` (json.RawMessage) — exact text is the contract,
|
||||
geometry is the escape hatch.
|
||||
3. **provider/llamaswap wire shapes** (pinned by the netherstorm image
|
||||
builds; host smoke tests are the drift defence):
|
||||
- `POST /upstream/<id>/v1/segment` multipart `file`,`prompt`
|
||||
[,`threshold`],`output=mask` → mask PNG.
|
||||
- `POST /upstream/<id>/v1/colorize` multipart `file` → PNG.
|
||||
- `POST /upstream/<id>/v1/restore_faces` multipart `file`[,`upscale`] → PNG.
|
||||
- `POST /upstream/<id>/v1/ocr` multipart `file`[,`langs` (comma-joined),
|
||||
`max_pages`] → JSON `{pages:[{number,text,lines,layout}]}`. The decode
|
||||
is tolerant: a page without aggregate `text` joins its line texts; a
|
||||
missing page `number` defaults to position. Zero pages is an error
|
||||
(a blank page still arrives as a page), mirroring the "no transcript"
|
||||
honesty rule.
|
||||
4. **Binary success bodies are validated before wrapping** (ADR-0020 rule):
|
||||
the three image surfaces reuse `singleImageResult` (positive evidence of
|
||||
image-ness required), OCR requires decodable JSON.
|
||||
|
||||
## Consequences
|
||||
|
||||
- imagegen grows from five to eight optional surfaces; consumers
|
||||
type-assert or use the provider methods directly, as before.
|
||||
- `ocr` is the seventh leaf media package (imagegen, audio, videogen,
|
||||
meshgen, musicgen, embeddings, ocr); the conventions have held across all
|
||||
of them.
|
||||
- PDF handling lives host-side (the shim rasterizes via pypdfium2);
|
||||
majordomo ships bytes and never needs a PDF dependency.
|
||||
Reference in New Issue
Block a user