steveandClaude Fable 5
e6987f54b2
- imagegen.Segmenter/SegmentationProvider: prompted mask via POST /upstream/<id>/v1/segment (file, prompt[, threshold], output=mask); white = prompted region, EditRequest.Mask polarity. - imagegen.Colorizer/ColorizeProvider: POST /upstream/<id>/v1/colorize. - imagegen.FaceRestorer/FaceRestoreProvider: POST /upstream/<id>/v1/restore_faces (upscale 1|2). - New ocr leaf package (Request/Page/Result, Recognize) + llamaswap OCRModel: POST /upstream/<id>/v1/ocr (file[, langs, max_pages]), tolerant per-page decode (join lines when page text absent), Raw escape hatch. - httptest contract tests per surface; ADR-0023; ADR index backfilled (0020-0022 rows were missing). Co-Authored-By: Claude Fable 5 <[email protected]>
3.2 KiB
3.2 KiB
ADR-0023: Wave-3 image + document surfaces (segmentation, colorize, face restore, OCR)
Status: Accepted (2026-07-16)
Context
The llama-swap host is gaining four wave-3 capabilities (spec:
mort docs/specs/2026-07-16-llamaswap-wave3.md): promptable segmentation
(GroundingDINO + SAM 2.1), photo colorization (DDColor), face restoration
(GFPGAN), and document OCR (Surya). All ride the /upstream/<model>/<path>
passthrough (ADR-0020); none of their native APIs are OpenAI-shaped.
Decision
- Grow
imagegenwith three optional interfaces, ADR-0016→0022 conventions (functional options, zero value = backend default, bytes-only I/O, provider mints model):imagegen.Segmenter/SegmentationProvider—SegmentationRequest {Image, Prompt, Threshold}→ one grayscale mask image, WHITE = the prompted region. Same polarity asEditRequest.Mask(white = repaint), so the mask feeds inpainting directly; cutouts are derived client-side (mask-as-alpha), one host call serving both. The client always sendsoutput=mask— the shim'scutout/boxesmodes are not exposed.imagegen.Colorizer/ColorizeProvider—ColorizeRequest{Image}, no knobs (the reference backend takes none; options reserved).imagegen.FaceRestorer/FaceRestoreProvider—FaceRestoreRequest{Image, Upscale}(1 or 2, 0 = backend default).
- New
ocrleaf package rather than a method on an existing surface: OCR is document-shaped (multi-page, PDFs), not image-generation-shaped.Request{Document, MIME, Filename, Languages, MaxPages}→Result{Text, Pages []Page{Number, Text}, Raw}.Result.Textis the pages joined with a blank line; the per-line bbox/confidence/layout detail stays inRaw(json.RawMessage) — exact text is the contract, geometry is the escape hatch. - provider/llamaswap wire shapes (pinned by the netherstorm image
builds; host smoke tests are the drift defence):
POST /upstream/<id>/v1/segmentmultipartfile,prompt[,threshold],output=mask→ mask PNG.POST /upstream/<id>/v1/colorizemultipartfile→ PNG.POST /upstream/<id>/v1/restore_facesmultipartfile[,upscale] → PNG.POST /upstream/<id>/v1/ocrmultipartfile[,langs(comma-joined),max_pages] → JSON{pages:[{number,text,lines,layout}]}. The decode is tolerant: a page without aggregatetextjoins its line texts; a missing pagenumberdefaults to position. Zero pages is an error (a blank page still arrives as a page), mirroring the "no transcript" honesty rule.
- Binary success bodies are validated before wrapping (ADR-0020 rule):
the three image surfaces reuse
singleImageResult(positive evidence of image-ness required), OCR requires decodable JSON.
Consequences
- imagegen grows from five to eight optional surfaces; consumers type-assert or use the provider methods directly, as before.
ocris the seventh leaf media package (imagegen, audio, videogen, meshgen, musicgen, embeddings, ocr); the conventions have held across all of them.- PDF handling lives host-side (the shim rasterizes via pypdfium2); majordomo ships bytes and never needs a PDF dependency.