main
11
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 steve
|
6995a8dee1 |
feat(faceswap): expose yaw on enumeration too
ListFaces now carries head yaw, so a caller choosing WHICH face to swap can see the thing that decides whether the swap will read — not only learn it afterwards from the swap report. In the run that prompted this the target's three faces sat at -82, -8 and -11 degrees; only the first was hopeless, and nothing in a bounding box said so. |
||
|
steve
|
ae2615ca68 |
feat(faceswap): carry the measured outcome, not just the image
A face swap always returns an image and always looks like success. Whether the likeness actually transferred is a different question, and until now nothing in the response answered it — so a caller wanting to know went and asked a vision model instead. That is wrong in precisely the cases that matter: shown a jogger in a Georgetown cap holding McDonald's cups, a VLM answers "Bill Clinton" whoever's face is on him. In the run that prompted this it reported failure on six consecutive CORRECT swaps (measured afterwards at 0.79-0.84 cosine), and the caller burned 21 minutes chasing a problem that did not exist. Result.SwappedFaces now carries, per replaced face: pixel size, the target image's dimensions, head yaw, and cosine similarity between the source face and the face actually present in the output. Yaw and FractionOfImage are the two that explain the complaint. The swap in question replaced a 138px face in a 1010px-wide photo — 14% of the width, correct and invisible at a glance — and elsewhere a face turned -82 degrees, where the features carrying identity are edge-on and any swap reads as a generic person. Same code on a 168px face in a 385px picture (44%, yaw 2) is unmistakable. None of that was inferable from a bounding box. Typed on Result rather than stuffed into Raw: a caller has to act on this, and a value reachable only by type-asserting an `any` is one nobody finds in time. doRawHeaders is doRaw with the whole header instead of only Content-Type; doRaw delegates to it, so the other 25 call sites are untouched and there is still one place where the status check and the size cap live. A missing or malformed header yields nil, not an error — an older shim sends no header, and a swap that produced a good image must not fail because the diagnostics beside it were unreadable. Covered for absent/garbage/wrong-type, and the parse is break-checked. |
||
|
steve
|
372bf826aa |
fix(llamaswap): a headerless non-image response was returned as a PNG
Gadfly on #23, blocking, 2/2 agreement — and it is the exact defect this whole line of work has been about: a call that succeeds while handing back the wrong bytes. sniffImageMIME falls back to image/png when detection is inconclusive, and the guard only consulted Content-Type. A response with NO Content-Type therefore skipped the check entirely and was labelled a PNG. The shim answers JSON on a semantic miss (no face found in the source or target), which is precisely the body that would have sailed through as a successful image. The check now validates the BYTES — http.DetectContentType must say image/ — and the reported MIME prefers the server's own label only when that label is itself an image type. Break-checked by restoring the header-only condition, which fails the new test. Also from that review: - index is documented as ignored under all=true, so a negative one is no longer rejected there; it is still rejected when it would actually be sent, and both halves are tested. - initImageFilename (video.go) was imageFilename with the base fixed to "frame" and now delegates to it — two copies of one extension table is how they drift. - DetectedFace carried Width/Height alongside Box, two sources of truth for one fact that can disagree after any transform. Now a Size() method derived from Box. - a dead `apiErr` in the test (declared, then `_ = apiErr`) was an abandoned errors.As check; it is wired up and now asserts callers can classify the error. - swapImg duplicated editInit verbatim; removed. Not taken: adding a FaceSwapProvider/ModelOption surface to match the other optional imagegen capabilities (single-model finding). There are no options to carry yet, and inventing an empty option type to look symmetrical would be API surface with nothing behind it. Worth revisiting when a real knob exists. |
||
|
steve
|
0ff90d80f6 |
feat(imagegen): face swap (identity transfer), a separate operation from Edit
Measured against the instruction-edit models on 2026-07-31: asking a diffusion model to put a SPECIFIC person's face into a photo does not work by any route. qwen-image-edit returns the picture essentially unchanged whether asked by name, by attribute, or by supplying the portrait as a second reference image; flux-kontext replaces the face with a different generic person. Identity transfer is a detect/align/blend pipeline, not a better prompt, so it gets its own interface rather than more Edit options. imagegen.FaceSwapper is optional and type-asserted, like Editor — a provider that cannot do this must not have Edit quietly stand in for it. ListFaces is part of the interface, not a convenience: a caller asked to change "the man on the right" needs a stable way to NAME one face, and pixel boxes let it check its own choice. The llamaswap shim orders faces left to right for exactly that reason (insightface's own order is score-ranked and unstable between near-identical images), and a malformed box is a protocol error rather than a zero-filled struct, because a wrong box aims the swap at the wrong person. The provider is the first here to POST more than one file, so buildMultipart gained buildMultipartFiles and now delegates to it — one writer loop, so the two cannot drift in how they escape names or terminate the body. index and all are mutually exclusive ON THE WIRE: the shim ignores index under all=true, and sending both would imply a precedence the caller cannot see. A JSON body is refused rather than returned as image bytes — the shim answers JSON on a semantic miss (no face in the source), and handing that back as a picture would report success while delivering a file that is not one. |
||
|
steve
|
2c70d32fd4 |
feat(imagegen): reference-image editing for instruction-edit models
FLUX.1 Kontext and Qwen-Image-Edit are a different kind of edit from img2img
and reach sd-server by a different path, and nothing in imagegen could
express it: EditRequest only had Init, which is noised and denoised back
under the prompt.
Measured against FLUX.1-Kontext on the netherstorm host 2026-07-30, on a
synthetic scene with a red rectangle, a blue rectangle and a flat background,
prompted "change the blue rectangle on the right to bright green, keep
everything else exactly the same":
via init_images (the only path that existed)
right rect (60,60,200) -> (47,82,228) still blue, instruction ignored
left rect (200,60,60) -> (229,43,50) drifted
background (150,200,240) -> (154,211,229) drifted
via extra_images (this change)
right rect (60,60,200) -> (70,254,4) green, as asked
left rect (200,60,60) -> (204,57,57) intact
background (150,200,240) -> (151,202,247) intact
No mask, no strength, no compositing — the model is handed the picture as
conditioning and the prompt as an instruction about it.
EditRequest.RefImages selects the path; when set, Init/Mask/Strength are
ignored rather than rejected, so a caller handing the same request to
whichever model is configured gets the better result on a Kontext-class model
instead of an error. The provider posts /sdapi/v1/txt2img with extra_images
(sd-server reads that field on both routes into gen_params.ref_images, where
the CLI's -r/--ref-image also lands); there is no init latent to denoise, so
sending one would only add noise to a pipeline that does not want any.
An all-empty reference set is refused: it would otherwise degrade into a
plain txt2img and render the prompt from scratch, which is not the request.
|
||
 steveandClaude Fable 5
|
e6987f54b2 |
feat: wave-3 image + document surfaces — segmentation, colorize, face restore, ocr (ADR-0023)
- imagegen.Segmenter/SegmentationProvider: prompted mask via POST /upstream/<id>/v1/segment (file, prompt[, threshold], output=mask); white = prompted region, EditRequest.Mask polarity. - imagegen.Colorizer/ColorizeProvider: POST /upstream/<id>/v1/colorize. - imagegen.FaceRestorer/FaceRestoreProvider: POST /upstream/<id>/v1/restore_faces (upscale 1|2). - New ocr leaf package (Request/Page/Result, Recognize) + llamaswap OCRModel: POST /upstream/<id>/v1/ocr (file[, langs, max_pages]), tolerant per-page decode (join lines when page text absent), Raw escape hatch. - httptest contract tests per surface; ADR-0023; ADR index backfilled (0020-0022 rows were missing). Co-Authored-By: Claude Fable 5 <[email protected]> |
||
|
steveandClaude Fable 5
|
4a752ffa2a |
fix: harden upstream path + binary-body validation (gadfly round 1)
- upstreamPath rejects '..' in model ids AND in the rest path — the rest can embed SERVER-SUPPLIED components (ACE-Step result file URLs), so dot-dot/scheme smuggling toward other proxy endpoints is refused - singleImageResult requires positive image evidence (sniffed magic OR declared image/*): an empty-Content-Type error page can no longer pass as 'the image' via sniffImageMIME's PNG-default labelling - upscale/background responses get a dedicated 256MB cap (the 64MB cap is JSON-sized; a 4x PNG legitimately exceeds it) - mesh JSON-detection widened (512-byte whitespace-tolerant peek + reject declared application/json) - Transcribe now reuses buildMultipart; transcriptionFilename takes (filename, mime) so diarize shares it without a fake request struct; truncateForError stops shadowing builtin cap; OnlyMask doc de-ambiguated Co-Authored-By: Claude Fable 5 <[email protected]> |
||
|
steveandClaude Fable 5
|
e98493bcfb |
feat: media expansion surfaces — edit mask, upscale, background removal, interpolation, diarization, meshgen (ADR-0020)
- imagegen.EditRequest.Mask -> sd-server img2img inpainting (white=repaint) - imagegen.Upscaler + BackgroundRemover, videogen.Interpolator, audio.DiarizationModel: new optional provider-minted surfaces - NEW meshgen leaf package (image->3D, glb/stl/obj) - provider/llamaswap: all five via the /upstream/<model>/<path> passthrough (upstreamPath helper, shared one-file multipart builder); binary success bodies validated (non-image, non-video, JSON-mesh rejection); diarization pins output=json (vtt/srt drop speaker labels) Co-Authored-By: Claude Fable 5 <[email protected]> |
||
|
steveandClaude Fable 5
|
434d721b99 |
feat: audio surfaces (TTS + transcription), imagegen.Editor, llamaswap health probe
- New leaf package `audio` (ADR-0017): SpeechModel/SpeechProvider and TranscriptionModel/TranscriptionProvider with imagegen conventions (zero value = backend default, functional options + Apply, bytes in/out, never URLs). Root re-exports added. - imagegen.Editor (ADR-0018): optional image-to-image interface — EditRequest carries the generation knobs plus Init image and denoising Strength; separate interface so existing Models keep compiling. - provider/llamaswap implements all of it: POST /v1/audio/speech (JSON, raw-audio response, MIME from Content-Type with format fallback), POST /v1/audio/transcriptions (multipart, response_format=json), ListVoices (GET /v1/audio/voices?model=, tolerant of string-list and object-list shapes), POST /sdapi/v1/img2img (txt2img wire + init_images/denoising_strength, shared image decode), and Health(ctx) (GET /health) — a cheap liveness probe for often-offline hosts. - Hermetic httptest coverage for every new wire shape and validation path; README sections + support-matrix footnote updated in the same commit (also corrects the stale /v1/images/generations claim — the image path has been SDAPI since the seed fix). First consumer: mort's llamaswap media tool cluster (status / image / TTS / STT agent tools against the netherstorm host). Co-Authored-By: Claude Fable 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01AXQxVhXBw8PwFAtsVrXSmj |
||
|
steve
|
a744cdc335 |
feat(imagegen): optional per-request generation settings
Add Steps, CFGScale, NegativePrompt, Sampler, Seed to imagegen.Request (pointer/empty = leave the backend's per-model default), with mirror options, and forward them in the llamaswap wire payload as the stable-diffusion.cpp fields (steps/cfg_scale/negative_prompt/ sample_method/seed). Unset fields are omitted so sd-server keeps its baked defaults. Lets callers (e.g. mort drawbots) override only what they explicitly set. |
||
|
steveandClaude Opus 4.8
|
96c612e707 |
feat(llamaswap): add llama-swap provider + canonical imagegen interface
Add provider/llamaswap, a tailored provider for llama-swap (the model-swapping
proxy over llama.cpp / stable-diffusion.cpp). Its chat path delegates to
provider/openai at {base}/v1 — no duplicated wire client (ADR-0007) — with
legacy max_tokens, a Bearer no-key placeholder for keyless local instances, and
a timeout-free client so cold model swaps rely on context deadlines. The
"tailored" surface is concrete management methods (ListModels / Running /
Unload) that don't belong on the canonical llm.Provider interface. The
llama-swap:// DSN scheme builds an http base URL (local-first); a no-URL
built-in errors clearly on use, mirroring foreman.
Add imagegen, a new canonical text-to-image interface separate from llm
(Request/Result/Model/Provider; Image = llm.ImagePart so generated images feed
straight back into chat). First backend is llama-swap via OpenAI
/v1/images/generations (b64_json, bytes-only). Re-exported from the root. v1 is
txt2img only.
Hermetic httptest coverage for chat delegation, management endpoints, image
decode, and scheme wiring. ADR-0015 + ADR-0016, README support matrix +
image-gen section, CLAUDE.md package map, and progress.md updated in the same
commit.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|