372bf826aa23c52a0db0da0a9d7fa32ff6ae2ced
9
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 steve
|
372bf826aa |
fix(llamaswap): a headerless non-image response was returned as a PNG
Gadfly on #23, blocking, 2/2 agreement — and it is the exact defect this whole line of work has been about: a call that succeeds while handing back the wrong bytes. sniffImageMIME falls back to image/png when detection is inconclusive, and the guard only consulted Content-Type. A response with NO Content-Type therefore skipped the check entirely and was labelled a PNG. The shim answers JSON on a semantic miss (no face found in the source or target), which is precisely the body that would have sailed through as a successful image. The check now validates the BYTES — http.DetectContentType must say image/ — and the reported MIME prefers the server's own label only when that label is itself an image type. Break-checked by restoring the header-only condition, which fails the new test. Also from that review: - index is documented as ignored under all=true, so a negative one is no longer rejected there; it is still rejected when it would actually be sent, and both halves are tested. - initImageFilename (video.go) was imageFilename with the base fixed to "frame" and now delegates to it — two copies of one extension table is how they drift. - DetectedFace carried Width/Height alongside Box, two sources of truth for one fact that can disagree after any transform. Now a Size() method derived from Box. - a dead `apiErr` in the test (declared, then `_ = apiErr`) was an abandoned errors.As check; it is wired up and now asserts callers can classify the error. - swapImg duplicated editInit verbatim; removed. Not taken: adding a FaceSwapProvider/ModelOption surface to match the other optional imagegen capabilities (single-model finding). There are no options to carry yet, and inventing an empty option type to look symmetrical would be API surface with nothing behind it. Worth revisiting when a real knob exists. |
||
|
steve
|
0ff90d80f6 |
feat(imagegen): face swap (identity transfer), a separate operation from Edit
Measured against the instruction-edit models on 2026-07-31: asking a diffusion model to put a SPECIFIC person's face into a photo does not work by any route. qwen-image-edit returns the picture essentially unchanged whether asked by name, by attribute, or by supplying the portrait as a second reference image; flux-kontext replaces the face with a different generic person. Identity transfer is a detect/align/blend pipeline, not a better prompt, so it gets its own interface rather than more Edit options. imagegen.FaceSwapper is optional and type-asserted, like Editor — a provider that cannot do this must not have Edit quietly stand in for it. ListFaces is part of the interface, not a convenience: a caller asked to change "the man on the right" needs a stable way to NAME one face, and pixel boxes let it check its own choice. The llamaswap shim orders faces left to right for exactly that reason (insightface's own order is score-ranked and unstable between near-identical images), and a malformed box is a protocol error rather than a zero-filled struct, because a wrong box aims the swap at the wrong person. The provider is the first here to POST more than one file, so buildMultipart gained buildMultipartFiles and now delegates to it — one writer loop, so the two cannot drift in how they escape names or terminate the body. index and all are mutually exclusive ON THE WIRE: the shim ignores index under all=true, and sending both would imply a precedence the caller cannot see. A JSON body is refused rather than returned as image bytes — the shim answers JSON on a semantic miss (no face in the source), and handing that back as a picture would report success while delivering a file that is not one. |
||
|
steve
|
2c70d32fd4 |
feat(imagegen): reference-image editing for instruction-edit models
FLUX.1 Kontext and Qwen-Image-Edit are a different kind of edit from img2img
and reach sd-server by a different path, and nothing in imagegen could
express it: EditRequest only had Init, which is noised and denoised back
under the prompt.
Measured against FLUX.1-Kontext on the netherstorm host 2026-07-30, on a
synthetic scene with a red rectangle, a blue rectangle and a flat background,
prompted "change the blue rectangle on the right to bright green, keep
everything else exactly the same":
via init_images (the only path that existed)
right rect (60,60,200) -> (47,82,228) still blue, instruction ignored
left rect (200,60,60) -> (229,43,50) drifted
background (150,200,240) -> (154,211,229) drifted
via extra_images (this change)
right rect (60,60,200) -> (70,254,4) green, as asked
left rect (200,60,60) -> (204,57,57) intact
background (150,200,240) -> (151,202,247) intact
No mask, no strength, no compositing — the model is handed the picture as
conditioning and the prompt as an instruction about it.
EditRequest.RefImages selects the path; when set, Init/Mask/Strength are
ignored rather than rejected, so a caller handing the same request to
whichever model is configured gets the better result on a Kontext-class model
instead of an error. The provider posts /sdapi/v1/txt2img with extra_images
(sd-server reads that field on both routes into gen_params.ref_images, where
the CLI's -r/--ref-image also lands); there is no init latent to denoise, so
sending one would only add noise to a pipeline that does not want any.
An all-empty reference set is refused: it would otherwise degrade into a
plain txt2img and render the prompt from scratch, which is not the request.
|
||
 steveandClaude Fable 5
|
e6987f54b2 |
feat: wave-3 image + document surfaces — segmentation, colorize, face restore, ocr (ADR-0023)
- imagegen.Segmenter/SegmentationProvider: prompted mask via POST /upstream/<id>/v1/segment (file, prompt[, threshold], output=mask); white = prompted region, EditRequest.Mask polarity. - imagegen.Colorizer/ColorizeProvider: POST /upstream/<id>/v1/colorize. - imagegen.FaceRestorer/FaceRestoreProvider: POST /upstream/<id>/v1/restore_faces (upscale 1|2). - New ocr leaf package (Request/Page/Result, Recognize) + llamaswap OCRModel: POST /upstream/<id>/v1/ocr (file[, langs, max_pages]), tolerant per-page decode (join lines when page text absent), Raw escape hatch. - httptest contract tests per surface; ADR-0023; ADR index backfilled (0020-0022 rows were missing). Co-Authored-By: Claude Fable 5 <[email protected]> |
||
|
steveandClaude Fable 5
|
4a752ffa2a |
fix: harden upstream path + binary-body validation (gadfly round 1)
- upstreamPath rejects '..' in model ids AND in the rest path — the rest can embed SERVER-SUPPLIED components (ACE-Step result file URLs), so dot-dot/scheme smuggling toward other proxy endpoints is refused - singleImageResult requires positive image evidence (sniffed magic OR declared image/*): an empty-Content-Type error page can no longer pass as 'the image' via sniffImageMIME's PNG-default labelling - upscale/background responses get a dedicated 256MB cap (the 64MB cap is JSON-sized; a 4x PNG legitimately exceeds it) - mesh JSON-detection widened (512-byte whitespace-tolerant peek + reject declared application/json) - Transcribe now reuses buildMultipart; transcriptionFilename takes (filename, mime) so diarize shares it without a fake request struct; truncateForError stops shadowing builtin cap; OnlyMask doc de-ambiguated Co-Authored-By: Claude Fable 5 <[email protected]> |
||
|
steveandClaude Fable 5
|
e98493bcfb |
feat: media expansion surfaces — edit mask, upscale, background removal, interpolation, diarization, meshgen (ADR-0020)
- imagegen.EditRequest.Mask -> sd-server img2img inpainting (white=repaint) - imagegen.Upscaler + BackgroundRemover, videogen.Interpolator, audio.DiarizationModel: new optional provider-minted surfaces - NEW meshgen leaf package (image->3D, glb/stl/obj) - provider/llamaswap: all five via the /upstream/<model>/<path> passthrough (upstreamPath helper, shared one-file multipart builder); binary success bodies validated (non-image, non-video, JSON-mesh rejection); diarization pins output=json (vtt/srt drop speaker labels) Co-Authored-By: Claude Fable 5 <[email protected]> |
||
|
steveandClaude Fable 5
|
434d721b99 |
feat: audio surfaces (TTS + transcription), imagegen.Editor, llamaswap health probe
- New leaf package `audio` (ADR-0017): SpeechModel/SpeechProvider and TranscriptionModel/TranscriptionProvider with imagegen conventions (zero value = backend default, functional options + Apply, bytes in/out, never URLs). Root re-exports added. - imagegen.Editor (ADR-0018): optional image-to-image interface — EditRequest carries the generation knobs plus Init image and denoising Strength; separate interface so existing Models keep compiling. - provider/llamaswap implements all of it: POST /v1/audio/speech (JSON, raw-audio response, MIME from Content-Type with format fallback), POST /v1/audio/transcriptions (multipart, response_format=json), ListVoices (GET /v1/audio/voices?model=, tolerant of string-list and object-list shapes), POST /sdapi/v1/img2img (txt2img wire + init_images/denoising_strength, shared image decode), and Health(ctx) (GET /health) — a cheap liveness probe for often-offline hosts. - Hermetic httptest coverage for every new wire shape and validation path; README sections + support-matrix footnote updated in the same commit (also corrects the stale /v1/images/generations claim — the image path has been SDAPI since the seed fix). First consumer: mort's llamaswap media tool cluster (status / image / TTS / STT agent tools against the netherstorm host). Co-Authored-By: Claude Fable 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01AXQxVhXBw8PwFAtsVrXSmj |
||
|
steve
|
a744cdc335 |
feat(imagegen): optional per-request generation settings
Add Steps, CFGScale, NegativePrompt, Sampler, Seed to imagegen.Request (pointer/empty = leave the backend's per-model default), with mirror options, and forward them in the llamaswap wire payload as the stable-diffusion.cpp fields (steps/cfg_scale/negative_prompt/ sample_method/seed). Unset fields are omitted so sd-server keeps its baked defaults. Lets callers (e.g. mort drawbots) override only what they explicitly set. |
||
|
steveandClaude Opus 4.8
|
96c612e707 |
feat(llamaswap): add llama-swap provider + canonical imagegen interface
Add provider/llamaswap, a tailored provider for llama-swap (the model-swapping
proxy over llama.cpp / stable-diffusion.cpp). Its chat path delegates to
provider/openai at {base}/v1 — no duplicated wire client (ADR-0007) — with
legacy max_tokens, a Bearer no-key placeholder for keyless local instances, and
a timeout-free client so cold model swaps rely on context deadlines. The
"tailored" surface is concrete management methods (ListModels / Running /
Unload) that don't belong on the canonical llm.Provider interface. The
llama-swap:// DSN scheme builds an http base URL (local-first); a no-URL
built-in errors clearly on use, mirroring foreman.
Add imagegen, a new canonical text-to-image interface separate from llm
(Request/Result/Model/Provider; Image = llm.ImagePart so generated images feed
straight back into chat). First backend is llama-swap via OpenAI
/v1/images/generations (b64_json, bytes-only). Re-exported from the root. v1 is
txt2img only.
Hermetic httptest coverage for chat delegation, management endpoints, image
decode, and scheme wiring. ADR-0015 + ADR-0016, README support matrix +
image-gen section, CLAUDE.md package map, and progress.md updated in the same
commit.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|