Measured against the instruction-edit models on 2026-07-31: asking a diffusion
model to put a SPECIFIC person's face into a photo does not work by any route.
qwen-image-edit returns the picture essentially unchanged whether asked by
name, by attribute, or by supplying the portrait as a second reference image;
flux-kontext replaces the face with a different generic person. Identity
transfer is a detect/align/blend pipeline, not a better prompt, so it gets its
own interface rather than more Edit options.
imagegen.FaceSwapper is optional and type-asserted, like Editor — a provider
that cannot do this must not have Edit quietly stand in for it.
ListFaces is part of the interface, not a convenience: a caller asked to
change "the man on the right" needs a stable way to NAME one face, and pixel
boxes let it check its own choice. The llamaswap shim orders faces left to
right for exactly that reason (insightface's own order is score-ranked and
unstable between near-identical images), and a malformed box is a protocol
error rather than a zero-filled struct, because a wrong box aims the swap at
the wrong person.
The provider is the first here to POST more than one file, so buildMultipart
gained buildMultipartFiles and now delegates to it — one writer loop, so the
two cannot drift in how they escape names or terminate the body.
index and all are mutually exclusive ON THE WIRE: the shim ignores index under
all=true, and sending both would imply a precedence the caller cannot see.
A JSON body is refused rather than returned as image bytes — the shim answers
JSON on a semantic miss (no face in the source), and handing that back as a
picture would report success while delivering a file that is not one.
FLUX.1 Kontext and Qwen-Image-Edit are a different kind of edit from img2img
and reach sd-server by a different path, and nothing in imagegen could
express it: EditRequest only had Init, which is noised and denoised back
under the prompt.
Measured against FLUX.1-Kontext on the netherstorm host 2026-07-30, on a
synthetic scene with a red rectangle, a blue rectangle and a flat background,
prompted "change the blue rectangle on the right to bright green, keep
everything else exactly the same":
via init_images (the only path that existed)
right rect (60,60,200) -> (47,82,228) still blue, instruction ignored
left rect (200,60,60) -> (229,43,50) drifted
background (150,200,240) -> (154,211,229) drifted
via extra_images (this change)
right rect (60,60,200) -> (70,254,4) green, as asked
left rect (200,60,60) -> (204,57,57) intact
background (150,200,240) -> (151,202,247) intact
No mask, no strength, no compositing — the model is handed the picture as
conditioning and the prompt as an instruction about it.
EditRequest.RefImages selects the path; when set, Init/Mask/Strength are
ignored rather than rejected, so a caller handing the same request to
whichever model is configured gets the better result on a Kontext-class model
instead of an error. The provider posts /sdapi/v1/txt2img with extra_images
(sd-server reads that field on both routes into gen_params.ref_images, where
the CLI's -r/--ref-image also lands); there is no init latent to denoise, so
sending one would only add noise to a pipeline that does not want any.
An all-empty reference set is refused: it would otherwise degrade into a
plain txt2img and render the prompt from scratch, which is not the request.
- upstreamPath rejects '..' in model ids AND in the rest path — the rest
can embed SERVER-SUPPLIED components (ACE-Step result file URLs), so
dot-dot/scheme smuggling toward other proxy endpoints is refused
- singleImageResult requires positive image evidence (sniffed magic OR
declared image/*): an empty-Content-Type error page can no longer pass
as 'the image' via sniffImageMIME's PNG-default labelling
- upscale/background responses get a dedicated 256MB cap (the 64MB cap
is JSON-sized; a 4x PNG legitimately exceeds it)
- mesh JSON-detection widened (512-byte whitespace-tolerant peek + reject
declared application/json)
- Transcribe now reuses buildMultipart; transcriptionFilename takes
(filename, mime) so diarize shares it without a fake request struct;
truncateForError stops shadowing builtin cap; OnlyMask doc de-ambiguated
Co-Authored-By: Claude Fable 5 <[email protected]>
- New leaf package `audio` (ADR-0017): SpeechModel/SpeechProvider and
TranscriptionModel/TranscriptionProvider with imagegen conventions
(zero value = backend default, functional options + Apply, bytes
in/out, never URLs). Root re-exports added.
- imagegen.Editor (ADR-0018): optional image-to-image interface —
EditRequest carries the generation knobs plus Init image and
denoising Strength; separate interface so existing Models keep
compiling.
- provider/llamaswap implements all of it: POST /v1/audio/speech (JSON,
raw-audio response, MIME from Content-Type with format fallback),
POST /v1/audio/transcriptions (multipart, response_format=json),
ListVoices (GET /v1/audio/voices?model=, tolerant of string-list and
object-list shapes), POST /sdapi/v1/img2img (txt2img wire +
init_images/denoising_strength, shared image decode), and Health(ctx)
(GET /health) — a cheap liveness probe for often-offline hosts.
- Hermetic httptest coverage for every new wire shape and validation
path; README sections + support-matrix footnote updated in the same
commit (also corrects the stale /v1/images/generations claim — the
image path has been SDAPI since the seed fix).
First consumer: mort's llamaswap media tool cluster (status / image /
TTS / STT agent tools against the netherstorm host).
Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01AXQxVhXBw8PwFAtsVrXSmj
Add Steps, CFGScale, NegativePrompt, Sampler, Seed to imagegen.Request
(pointer/empty = leave the backend's per-model default), with mirror
options, and forward them in the llamaswap wire payload as the
stable-diffusion.cpp fields (steps/cfg_scale/negative_prompt/
sample_method/seed). Unset fields are omitted so sd-server keeps its
baked defaults.
Lets callers (e.g. mort drawbots) override only what they explicitly set.
Add provider/llamaswap, a tailored provider for llama-swap (the model-swapping
proxy over llama.cpp / stable-diffusion.cpp). Its chat path delegates to
provider/openai at {base}/v1 — no duplicated wire client (ADR-0007) — with
legacy max_tokens, a Bearer no-key placeholder for keyless local instances, and
a timeout-free client so cold model swaps rely on context deadlines. The
"tailored" surface is concrete management methods (ListModels / Running /
Unload) that don't belong on the canonical llm.Provider interface. The
llama-swap:// DSN scheme builds an http base URL (local-first); a no-URL
built-in errors clearly on use, mirroring foreman.
Add imagegen, a new canonical text-to-image interface separate from llm
(Request/Result/Model/Provider; Image = llm.ImagePart so generated images feed
straight back into chat). First backend is llama-swap via OpenAI
/v1/images/generations (b64_json, bytes-only). Re-exported from the root. v1 is
txt2img only.
Hermetic httptest coverage for chat delegation, management endpoints, image
decode, and scheme wiring. ADR-0015 + ADR-0016, README support matrix +
image-gen section, CLAUDE.md package map, and progress.md updated in the same
commit.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
steve
