37 Commits
Author SHA1 Message Date
steveandClaude Opus 5 5994d96921 refactor(llamaswap): drop initImageFilename — one caller left, and it was a rename of imageFilename
CI / Tidy (pull_request) Successful in 9m40s
CI / Build & Test (pull_request) Successful in 11m38s
The 2/4 finding is right: after writeImagePart started passing an explicit
filename stem, initImageFilename had no caller in video.go, and its
"conditioning frame" doc no longer described its one remaining user
(lipsync.go's avatar image). A one-line wrapper that survives only to be
misdescribed is not indirection worth keeping.

lipsync now calls imageFilename(mime, "frame") directly, and imageFilename's
doc lists the real bases — including WHY the video keyframes need distinct
ones: a backend that stages uploads by filename would otherwise have the
second overwrite the first.

Not taken: consolidating the first/last-frame rationale to a single canonical
site. The copies address different readers — the wire encoding (provider), the
contract's undetectable-support caveat (videogen), and the mode table (README)
— and last round's finding was a doc pointing at a note that did not exist.
Trading duplication for cross-references is what produced that.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01PLjgrxvHjm1sJgUu9zBPH9
2026-08-08 03:06:21 -04:00
steveandClaude Opus 5 588e092465 docs(videogen): gadfly — README FL2V section, and stop pointing at a note that does not exist
CI / Tidy (pull_request) Successful in 9m25s
CI / Build & Test (pull_request) Successful in 9m50s
- README documented only t2v/i2v. Now a table of the four keyframe
  combinations, plus the undetectable-support caveat, which is the one thing a
  caller cannot work out for itself.
- The LastImage doc comment said "see the note on LastImage support in
  provider/llamaswap" — there was no such note. A pointer to something that
  does not exist is worse than no pointer; the comment is now self-contained.
- Generate's doc described only input_reference; it now names
  input_reference_last and explains why an unsupporting backend returns a clip
  rather than an error.

The 2/4 finding (writeImagePart reusing the "frame" base for both parts) was
already fixed in dbc9689 — from the receiving end, where the consequence is
concrete rather than stylistic: ComfyUI stages uploads by FILENAME with
overwrite=true, so a shared name means the second clobbers the first and both
keyframes resolve to one image.

Not taken: initImageFilename's name is no longer misleading (writeImagePart
stopped calling it), and it is still used by lipsync.go so it is not dead.
The empty-LastImage test stays standalone — it mirrors the existing standalone
empty-InitImage coverage rather than a table this file does not have.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01PLjgrxvHjm1sJgUu9zBPH9
2026-08-08 02:59:12 -04:00
steveandClaude Opus 5 dbc96898ab fix(videogen): distinct FILENAMES for the two keyframes, not just distinct field names
CI / Tidy (pull_request) Successful in 9m25s
CI / Build & Test (pull_request) Successful in 10m28s
Caught while writing the receiving end. Distinct multipart field names are not
sufficient: backends stage an uploaded frame under a name derived from the
FILENAME, and our own ComfyUI shim posts to /upload/image with overwrite=true.
Both parts were sending initImageFilename(mime) — literally "frame.png" for
each — so the second upload would have clobbered the first and BOTH keyframe
inputs would have resolved to the same stored image.

The failure mode is the worst kind: a clip pinned at both ends to the same
frame renders cleanly, returns 200, and looks like the feature not working
rather than like a bug. Nothing upstream or downstream would report a fault.

writeImagePart now takes the filename stem (frame / frame_last), and the test
asserts the two arrive under different filenames.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01PLjgrxvHjm1sJgUu9zBPH9
2026-08-08 02:50:43 -04:00
steveandClaude Opus 5 44fcfbb273 feat(videogen): LastImage — pin the trailing keyframe (first-last-frame-to-video)
Gadfly review (reusable) / review (pull_request) Successful in 3m41s
Adversarial Review (Gadfly) / review (pull_request) Successful in 3m41s
CI / Tidy (pull_request) Successful in 9m26s
CI / Build & Test (pull_request) Successful in 9m48s
videogen.Request gains LastImage alongside InitImage, so one Request covers
t2v, i2v and FL2V without a mode flag. With InitImage it pins both ends of the
clip; alone it pins the destination and lets the backend invent the approach.

The llamaswap provider sends it as a SEPARATE `input_reference_last` part
rather than a second `input_reference`. Multipart permits repeated names, but
then which frame is first and which is last depends on part ORDER — an
ordering contract invisible in the payload, that nothing notices breaking. A
backend that does not know the new name ignores the part, the same degradation
as any other unknown field.

Both parts go through one writeImagePart helper so their encoding cannot
drift, and an empty LastImage is rejected up front exactly as InitImage
already is.

Support is per-model and deliberately NOT advertised in this contract: a
backend that ignores a trailing keyframe returns an ordinary clip, which is
indistinguishable from success. The doc comment says so, because a caller that
needs to know whether the pin took effect has to establish that out of band —
and the mort side gates on a convar for exactly this reason.

Motivated by mort's #1567 (long-form video): with both ends pinned, drift
becomes structurally bounded inside each shot instead of compounding across an
autoregressive chain.

Tests break-checked: sending the last frame under the shared name fails both
the distinct-name assertion and the last-alone case.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01PLjgrxvHjm1sJgUu9zBPH9
2026-08-08 02:47:39 -04:00
steve 6995a8dee1 feat(faceswap): expose yaw on enumeration too
Gadfly review (reusable) / review (pull_request) Successful in 4m54s
Adversarial Review (Gadfly) / review (pull_request) Successful in 4m55s
CI / Tidy (pull_request) Successful in 9m23s
CI / Build & Test (pull_request) Successful in 9m48s
ListFaces now carries head yaw, so a caller choosing WHICH face to swap can
see the thing that decides whether the swap will read — not only learn it
afterwards from the swap report. In the run that prompted this the target's
three faces sat at -82, -8 and -11 degrees; only the first was hopeless, and
nothing in a bounding box said so.
2026-07-31 17:50:27 -04:00
steve ae2615ca68 feat(faceswap): carry the measured outcome, not just the image
A face swap always returns an image and always looks like success. Whether the
likeness actually transferred is a different question, and until now nothing in
the response answered it — so a caller wanting to know went and asked a vision
model instead. That is wrong in precisely the cases that matter: shown a jogger
in a Georgetown cap holding McDonald's cups, a VLM answers "Bill Clinton"
whoever's face is on him. In the run that prompted this it reported failure on
six consecutive CORRECT swaps (measured afterwards at 0.79-0.84 cosine), and
the caller burned 21 minutes chasing a problem that did not exist.

Result.SwappedFaces now carries, per replaced face: pixel size, the target
image's dimensions, head yaw, and cosine similarity between the source face and
the face actually present in the output.

Yaw and FractionOfImage are the two that explain the complaint. The swap in
question replaced a 138px face in a 1010px-wide photo — 14% of the width,
correct and invisible at a glance — and elsewhere a face turned -82 degrees,
where the features carrying identity are edge-on and any swap reads as a
generic person. Same code on a 168px face in a 385px picture (44%, yaw 2) is
unmistakable. None of that was inferable from a bounding box.

Typed on Result rather than stuffed into Raw: a caller has to act on this, and
a value reachable only by type-asserting an `any` is one nobody finds in time.

doRawHeaders is doRaw with the whole header instead of only Content-Type; doRaw
delegates to it, so the other 25 call sites are untouched and there is still
one place where the status check and the size cap live.

A missing or malformed header yields nil, not an error — an older shim sends no
header, and a swap that produced a good image must not fail because the
diagnostics beside it were unreadable. Covered for absent/garbage/wrong-type,
and the parse is break-checked.
2026-07-31 17:49:27 -04:00
steve 372bf826aa fix(llamaswap): a headerless non-image response was returned as a PNG
CI / Tidy (pull_request) Successful in 9m26s
CI / Build & Test (pull_request) Successful in 10m4s
Gadfly on #23, blocking, 2/2 agreement — and it is the exact defect this
whole line of work has been about: a call that succeeds while handing back
the wrong bytes.

sniffImageMIME falls back to image/png when detection is inconclusive, and
the guard only consulted Content-Type. A response with NO Content-Type
therefore skipped the check entirely and was labelled a PNG. The shim answers
JSON on a semantic miss (no face found in the source or target), which is
precisely the body that would have sailed through as a successful image.

The check now validates the BYTES — http.DetectContentType must say image/ —
and the reported MIME prefers the server's own label only when that label is
itself an image type. Break-checked by restoring the header-only condition,
which fails the new test.

Also from that review:
  - index is documented as ignored under all=true, so a negative one is no
    longer rejected there; it is still rejected when it would actually be
    sent, and both halves are tested.
  - initImageFilename (video.go) was imageFilename with the base fixed to
    "frame" and now delegates to it — two copies of one extension table is
    how they drift.
  - DetectedFace carried Width/Height alongside Box, two sources of truth for
    one fact that can disagree after any transform. Now a Size() method
    derived from Box.
  - a dead `apiErr` in the test (declared, then `_ = apiErr`) was an
    abandoned errors.As check; it is wired up and now asserts callers can
    classify the error.
  - swapImg duplicated editInit verbatim; removed.

Not taken: adding a FaceSwapProvider/ModelOption surface to match the other
optional imagegen capabilities (single-model finding). There are no options
to carry yet, and inventing an empty option type to look symmetrical would be
API surface with nothing behind it. Worth revisiting when a real knob exists.
2026-07-31 12:34:58 -04:00
steve 0ff90d80f6 feat(imagegen): face swap (identity transfer), a separate operation from Edit
Gadfly review (reusable) / review (pull_request) Successful in 5m7s
Adversarial Review (Gadfly) / review (pull_request) Successful in 5m7s
CI / Tidy (pull_request) Successful in 9m24s
CI / Build & Test (pull_request) Successful in 9m52s
Measured against the instruction-edit models on 2026-07-31: asking a diffusion
model to put a SPECIFIC person's face into a photo does not work by any route.
qwen-image-edit returns the picture essentially unchanged whether asked by
name, by attribute, or by supplying the portrait as a second reference image;
flux-kontext replaces the face with a different generic person. Identity
transfer is a detect/align/blend pipeline, not a better prompt, so it gets its
own interface rather than more Edit options.

imagegen.FaceSwapper is optional and type-asserted, like Editor — a provider
that cannot do this must not have Edit quietly stand in for it.

ListFaces is part of the interface, not a convenience: a caller asked to
change "the man on the right" needs a stable way to NAME one face, and pixel
boxes let it check its own choice. The llamaswap shim orders faces left to
right for exactly that reason (insightface's own order is score-ranked and
unstable between near-identical images), and a malformed box is a protocol
error rather than a zero-filled struct, because a wrong box aims the swap at
the wrong person.

The provider is the first here to POST more than one file, so buildMultipart
gained buildMultipartFiles and now delegates to it — one writer loop, so the
two cannot drift in how they escape names or terminate the body.

index and all are mutually exclusive ON THE WIRE: the shim ignores index under
all=true, and sending both would imply a precedence the caller cannot see.
A JSON body is refused rather than returned as image bytes — the shim answers
JSON on a semantic miss (no face in the source), and handing that back as a
picture would report success while delivering a file that is not one.
2026-07-31 12:25:12 -04:00
steve 2c70d32fd4 feat(imagegen): reference-image editing for instruction-edit models
Gadfly review (reusable) / review (pull_request) Successful in 4m24s
Adversarial Review (Gadfly) / review (pull_request) Successful in 4m24s
CI / Tidy (pull_request) Successful in 9m40s
CI / Build & Test (pull_request) Successful in 11m17s
FLUX.1 Kontext and Qwen-Image-Edit are a different kind of edit from img2img
and reach sd-server by a different path, and nothing in imagegen could
express it: EditRequest only had Init, which is noised and denoised back
under the prompt.

Measured against FLUX.1-Kontext on the netherstorm host 2026-07-30, on a
synthetic scene with a red rectangle, a blue rectangle and a flat background,
prompted "change the blue rectangle on the right to bright green, keep
everything else exactly the same":

  via init_images (the only path that existed)
      right rect (60,60,200) -> (47,82,228)   still blue, instruction ignored
      left rect  (200,60,60) -> (229,43,50)   drifted
      background (150,200,240) -> (154,211,229) drifted

  via extra_images (this change)
      right rect (60,60,200) -> (70,254,4)    green, as asked
      left rect  (200,60,60) -> (204,57,57)   intact
      background (150,200,240) -> (151,202,247) intact

No mask, no strength, no compositing — the model is handed the picture as
conditioning and the prompt as an instruction about it.

EditRequest.RefImages selects the path; when set, Init/Mask/Strength are
ignored rather than rejected, so a caller handing the same request to
whichever model is configured gets the better result on a Kontext-class model
instead of an error. The provider posts /sdapi/v1/txt2img with extra_images
(sd-server reads that field on both routes into gen_params.ref_images, where
the CLI's -r/--ref-image also lands); there is no init latent to denoise, so
sending one would only add noise to a pipeline that does not want any.

An all-empty reference set is refused: it would otherwise degrade into a
plain txt2img and render the prompt from scratch, which is not the request.
2026-07-30 21:21:35 -04:00
steveandClaude Opus 4.8 2bfffff47a feat: kimi (Moonshot AI) built-in provider (ADR-0026)
CI / Tidy (pull_request) Successful in 9m29s
CI / Build & Test (pull_request) Successful in 10m23s
Gadfly review (reusable) / review (pull_request) Successful in 18m38s
Adversarial Review (Gadfly) / review (pull_request) Successful in 18m38s
Add a first-class `kimi` provider and `kimi://` DSN scheme for Moonshot AI's
OpenAI-compatible Chat Completions API. Both reuse provider/openai (no new
client, mirroring llama-swap's chat path). Default endpoint is the
international host; the China endpoint is reachable via a kimi:// LLM_* DSN.

- Credential is KIMI_API_KEY, read through the registry's injected envLookup
  so it stays hermetically testable. WithAPIKey is passed unconditionally so
  an unset KIMI_API_KEY can never fall through to the openai client's
  OPENAI_API_KEY default.
- New openai.WithAPIKeyName option customizes the missing-key error hint
  (default OPENAI_API_KEY); kimi names KIMI_API_KEY.
- Hermetic tests: built-in base URL + bearer, missing-key hint names
  KIMI_API_KEY with no OPENAI fallthrough and no network hit, kimi:// scheme
  round-trips against the China host.
- Docs in sync: README built-in table + DSN scheme list + support matrix,
  .env.example, env.go DSN doc, ADR-0026 (+ index, backfilling 0024/0025),
  progress.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-07-18 02:59:30 -04:00
steve 95147f7582 Merge pull request 'feat: wave-3 video surfaces — lipsync, video matte, video upscale, chain jobs (ADR-0025)' (#19) from feat/wave3-video-surfaces into main
CI / Tidy (push) Successful in 9m24s
CI / Build & Test (push) Successful in 9m46s
2026-07-16 23:25:18 +00:00
steve 036406b221 Merge pull request 'feat: wave-3 audio surfaces — stems, SFX, speech enhance, voice clone, translate (ADR-0024)' (#18) from feat/wave3-audio-surfaces into main
CI / Tidy (push) Successful in 9m22s
CI / Build & Test (push) Successful in 9m44s
2026-07-16 23:24:42 +00:00
steveandClaude Fable 5 56b5b000a6 fix: review findings — chain NaN/Inf + id hygiene, percent-escape jobPath, shared singleVideoResult
CI / Tidy (pull_request) Successful in 9m25s
CI / Build & Test (pull_request) Successful in 9m46s
- SubmitChain rejects NaN/±Inf segment seconds with ErrUnsupported
  (previously an obscure json.Marshal error; NaN fails every comparison
  and +Inf passed the >= 0 check).
- ChainStatus skips segment entries with no usable id — JSON null
  (which no-op-unmarshals into a string, previously appending ""),
  empty strings, and id-less objects; the unfiltered list survives in
  Raw. ChainJob.SegmentIDs doc now also says ChainSegmentResult takes
  the segment index, not an id string.
- jobPath rejects '%' in job ids — %2F/%2E%2E percent-escapes decode
  back into path structure server-side, bypassing the literal check on
  this upstream-echoed value.
- singleVideoResult moves to video.go next to videoMIME, and the two
  remaining hand-rolled copies of the video-result validation
  (videoModel.Generate, Interpolate) now use it — one validation, one
  message shape.
- videogen.LipSyncer renamed to videogen.Lipsyncer for consistency with
  the rest of the surface's Lipsync* naming (LipsyncProvider,
  LipsyncModel, LipsyncRequest); not yet consumed downstream, so the
  rename is free now and never again.

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01WWCQcYStWXBYUy5sZWnbLT
2026-07-16 19:12:42 -04:00
steveandClaude Fable 5 31bccf6e19 fix: review findings — clone-route audio sniffing, zip caps, filename sanitization, WAV response caps
CI / Tidy (pull_request) Successful in 9m27s
CI / Build & Test (pull_request) Successful in 9m46s
- speakWithReference now validates the response IS audio via the same
  audioResultMIME sniff the sfx/enhance surfaces use — a 2xx JSON soft
  error or HTML proxy page was previously wrapped up as audio/wav bytes.
- audioResultMIME moves to audio.go (next to speechMIME; it was defined
  in sfx.go but shared by enhance/clone) and learns the Ogg container
  normalization (application/ogg -> audio/ogg).
- Stems zip unpack gains entry-count (16) and total-decompressed (1GB)
  caps on top of the existing per-entry cap — the per-entry bound alone
  still let a many-entry bomb multiply up.
- sanitizeFilename drops NUL and both path separators too, so upload
  metadata can never smuggle directory structure to a file-writing shim.
- New maxAudioResponseBytes (256MB) for bodies that ARE one audio clip
  (clone, enhance, sfx): a long WAV legitimately passes the 64MB JSON
  cap.
- speakWithReference local renamed path -> upPath (naming parity with
  stems/enhance).

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01WWCQcYStWXBYUy5sZWnbLT
2026-07-16 19:07:37 -04:00
steveandClaude Fable 5 966ea16166 fix: review findings — NaN threshold guard, percent-escape rejection in upstream model ids
CI / Tidy (pull_request) Successful in 9m31s
CI / Build & Test (pull_request) Successful in 9m48s
- Segment: reject NaN thresholds (NaN fails every comparison, so it
  passed the [0,1] range check and reached the shim as the literal
  string "NaN"); ±Inf were already caught by the range comparisons,
  now covered by tests too.
- upstreamPath: reject '%' in model ids — %2F/%2E%2E percent-escapes
  decode back into path structure server-side, bypassing the literal
  /?#/.. rejection. Ids never legitimately contain '%'.

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01WWCQcYStWXBYUy5sZWnbLT
2026-07-16 19:00:11 -04:00
steveandClaude Fable 5 5f175ecf82 feat: wave-3 video surfaces — lipsync, video matte, video upscale, chain jobs (ADR-0025)
CI / Tidy (pull_request) Successful in 9m27s
CI / Build & Test (pull_request) Successful in 10m43s
Gadfly review (reusable) / review (pull_request) Successful in 10m9s
Adversarial Review (Gadfly) / review (pull_request) Successful in 10m9s
- videogen.LipSyncer/LipsyncProvider: SadTalker talking heads via
  POST /upstream/<id>/v1/talking_head (multipart image+audio parts,
  still/enhance/preprocess flags) -> mp4.
- videogen.VideoBackgroundRemover/VideoBackgroundRemovalProvider:
  POST /upstream/<id>/v1/video/matte (output greenscreen_mp4|alpha_webm).
- videogen.VideoUpscaler/VideoUpscaleProvider:
  POST /upstream/<id>/v1/video/upscale (scale 2|4).
- videogen.Chainer/ChainerProvider: async long-video chain-job client —
  SubmitChain (JSON POST /v1/video/chain, init_image_b64), ChainStatus
  (GET /v1/jobs/{id}, tolerant segment-id decode), ChainResult,
  ChainSegmentResult (partial delivery after mid-chain failure); hostile
  job-id path rejection.
- Shared singleVideoResult validation (positive video evidence) + a
  videoInputFilename hint helper; httptest contract tests per surface;
  ADR-0025 (index row deferred — MJ-A backfills the ADR index table and
  parallel edits would conflict).

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-07-16 17:07:27 -04:00
steveandClaude Fable 5 b3a172a053 feat: wave-3 audio surfaces — stems, sfx, speech enhance, voice clone, translate (ADR-0024)
CI / Tidy (pull_request) Successful in 9m27s
CI / Build & Test (pull_request) Successful in 10m37s
Gadfly review (reusable) / review (pull_request) Successful in 41m14s
Adversarial Review (Gadfly) / review (pull_request) Successful in 41m14s
- audio.StemSeparator/StemSeparationProvider: Demucs zip transport via
  POST /upstream/<id>/v1/stems (Mode two -> two_stems=vocals; model +
  format fields); bounded zip unpack, entry name -> stem, ext -> MIME.
- SFXModel reuses musicgen against the sync /upstream/<id>/v1/sfx route
  (JSON prompt/seconds/steps/cfg_scale/seed -> WAV); musicgen.Request
  gains CFGScale.
- audio.SpeechEnhancer/SpeechEnhancementProvider:
  POST /upstream/<id>/v1/enhance -> WAV (result reuses SpeechResult).
- SpeechRequest.ReferenceAudio/ReferenceMIME (+WithReferenceAudio):
  llamaswap switches to the chatterbox clone route
  POST /upstream/<id>/v1/audio/speech/upload (input + voice_file),
  wav MIME fallback.
- TranscriptionRequest.Translate (+WithTranslate): translate=true form
  field, language=auto forced when no explicit hint (whisper.cpp default
  en would skip translation).
- httptest contract tests (zip unpack, clone-route switch, translate +
  auto-language injection); ADR-0024 (index row deferred — MJ-A backfills
  the ADR index table and parallel edits would conflict).

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-07-16 17:01:11 -04:00
steveandClaude Fable 5 e6987f54b2 feat: wave-3 image + document surfaces — segmentation, colorize, face restore, ocr (ADR-0023)
CI / Tidy (pull_request) Successful in 9m25s
CI / Build & Test (pull_request) Successful in 9m48s
Gadfly review (reusable) / review (pull_request) Successful in 47m4s
Adversarial Review (Gadfly) / review (pull_request) Successful in 47m4s
- imagegen.Segmenter/SegmentationProvider: prompted mask via
  POST /upstream/<id>/v1/segment (file, prompt[, threshold], output=mask);
  white = prompted region, EditRequest.Mask polarity.
- imagegen.Colorizer/ColorizeProvider: POST /upstream/<id>/v1/colorize.
- imagegen.FaceRestorer/FaceRestoreProvider:
  POST /upstream/<id>/v1/restore_faces (upscale 1|2).
- New ocr leaf package (Request/Page/Result, Recognize) + llamaswap
  OCRModel: POST /upstream/<id>/v1/ocr (file[, langs, max_pages]),
  tolerant per-page decode (join lines when page text absent), Raw
  escape hatch.
- httptest contract tests per surface; ADR-0023; ADR index backfilled
  (0020-0022 rows were missing).

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-07-16 16:54:30 -04:00
steveandClaude Fable 5 fd8d56c3c1 fix: live-API corrections — music result shapes, mesh format honesty, mesh conversion
Gadfly review (reusable) / review (pull_request) Successful in 15s
Adversarial Review (Gadfly) / review (pull_request) Successful in 15s
CI / Tidy (pull_request) Successful in 9m26s
CI / Build & Test (pull_request) Successful in 10m0s
Round 2 from live smokes on netherstorm (2026-07-14):

- ACE-Step result blob is an ARRAY of objects and carries RAW control
  characters inside string values (literal newlines) — strict JSON
  rejected it. parseMusicResult sanitizes control chars (only legal
  inside string values in the double-encoded blob) and accepts array or
  object shapes. Regression test uses the live payload shape.
- Hunyuan3D GenerationRequest has NO output-format field (the documented
  type param is fiction) — it always returns GLB. Results are now
  labelled by sniffed magic bytes, never by the requested format.
- NEW meshgen.Converter/ConverterProvider optional surface + llamaswap
  impl over the mediautils shim POST /v1/convert_mesh — the STL hop for
  the printer pipeline.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-07-14 12:22:06 -04:00
steveandClaude Fable 5 cf2d83f157 fix: music poll resilience + hostile-URL guard + embed dup-index (gadfly round 1)
- pollResult tolerates up to 5 CONSECUTIVE bad polls (transport blip,
  unparseable payload, task momentarily absent) instead of killing a
  multi-minute exclusive-GPU job on the first hiccup; only status=2, a
  failure run, or ctx deadline aborts
- server-supplied result.File must be server-relative; combined with the
  upstreamPath dot-dot/scheme rejection this stops a hostile upstream
  from steering the follow-up GET at other proxy endpoints (test:
  ../../api/models/unload refused)
- WithSteps(<=0) rejected; embed responses repeating an index rejected;
  musicFormatMIME now wraps speechMIME (one format table, wav32
  normalized); poll interval is a test-shrinkable var (CI no longer
  burns 2s+ per music test); parens + comments per review

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-07-13 00:52:06 -04:00
steveandClaude Fable 5 cacce61ddc feat: musicgen + embeddings/rerank surfaces (ADR-0021, ADR-0022)
- NEW musicgen leaf package: blocking Generate over ACE-Step's async job
  queue (release_task -> poll query_result -> fetch file, all via
  /upstream); tolerant envelope parsing, double-encoded result handled
- NEW embeddings leaf package: EmbedModel + RerankModel as separate mints
  (two server instances on the host, llama.cpp #20085); InstructedQuery
  helper for Qwen3-style query/document asymmetry
- provider/llamaswap: /v1/embeddings + /v1/rerank clients with strict
  validation (index-ordered vectors, count mismatch and out-of-range
  index are hard errors; rerank sorted descending, minimal parser)

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-07-13 00:50:36 -04:00
steveandClaude Fable 5 4a752ffa2a fix: harden upstream path + binary-body validation (gadfly round 1)
CI / Tidy (pull_request) Successful in 9m24s
CI / Build & Test (pull_request) Successful in 10m17s
- upstreamPath rejects '..' in model ids AND in the rest path — the rest
  can embed SERVER-SUPPLIED components (ACE-Step result file URLs), so
  dot-dot/scheme smuggling toward other proxy endpoints is refused
- singleImageResult requires positive image evidence (sniffed magic OR
  declared image/*): an empty-Content-Type error page can no longer pass
  as 'the image' via sniffImageMIME's PNG-default labelling
- upscale/background responses get a dedicated 256MB cap (the 64MB cap
  is JSON-sized; a 4x PNG legitimately exceeds it)
- mesh JSON-detection widened (512-byte whitespace-tolerant peek + reject
  declared application/json)
- Transcribe now reuses buildMultipart; transcriptionFilename takes
  (filename, mime) so diarize shares it without a fake request struct;
  truncateForError stops shadowing builtin cap; OnlyMask doc de-ambiguated

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-07-13 00:50:21 -04:00
steveandClaude Fable 5 e98493bcfb feat: media expansion surfaces — edit mask, upscale, background removal, interpolation, diarization, meshgen (ADR-0020)
CI / Tidy (pull_request) Successful in 9m23s
CI / Build & Test (pull_request) Successful in 10m24s
Adversarial Review (Gadfly) / review (pull_request) Successful in 15m37s
- imagegen.EditRequest.Mask -> sd-server img2img inpainting (white=repaint)
- imagegen.Upscaler + BackgroundRemover, videogen.Interpolator,
  audio.DiarizationModel: new optional provider-minted surfaces
- NEW meshgen leaf package (image->3D, glb/stl/obj)
- provider/llamaswap: all five via the /upstream/<model>/<path> passthrough
  (upstreamPath helper, shared one-file multipart builder); binary success
  bodies validated (non-image, non-video, JSON-mesh rejection); diarization
  pins output=json (vtt/srt drop speaker labels)

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-07-12 23:52:34 -04:00
steveandClaude Fable 5 776ef6fda9 fix: gadfly review — video-sized response cap, stale comment
CI / Tidy (pull_request) Successful in 9m31s
CI / Build & Test (pull_request) Successful in 10m23s
maxVideoResponseBytes (512MB) replaces the shared 64MB JSON cap on the
/v1/videos/sync read path (doRaw now takes the cap per call) — 3/6
models flagged that a legitimate long/high-bitrate clip would be
discarded after minutes of GPU work. Plus a stale stable-diffusion
comment in initImageFilename and a test-handler early return.

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01AXQxVhXBw8PwFAtsVrXSmj
2026-07-12 09:57:53 -04:00
steveandClaude Fable 5 89210346b8 fix: address review — fail loud on non-video bodies, dedupe form plumbing, doc parity
CI / Tidy (pull_request) Successful in 9m28s
CI / Build & Test (pull_request) Successful in 10m0s
- videoMIME no longer hard-falls-back to video/mp4: a 2xx body that is
  neither declared nor sniffable as video (JSON job envelope, HTML error
  page) is now an APIError instead of a 'successful' corrupt clip.
- Resolution rides the wire as width/height AND the OpenAI-style size
  string, so either upstream convention honors an explicit request.
- writeFormFields + mimeFromContentType shared helpers replace the
  copied multipart loop (audio.go/video.go) and Content-Type branch.
- ADR-0019 indexed in docs/adr/README.md; README gains the videogen
  section + support-matrix mention (docs-parity rule).

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01AXQxVhXBw8PwFAtsVrXSmj
2026-07-12 09:53:34 -04:00
steveandClaude Fable 5 4629c6af0f feat(videogen): canonical video-generation surface + llama-swap client
Adversarial Review (Gadfly) / review (pull_request) Successful in 8m43s
CI / Tidy (pull_request) Successful in 9m28s
CI / Build & Test (pull_request) Successful in 10m11s
New videogen/ contract package (ADR-0019): Request/Result/Model/Provider
with the imagegen conventions. Text-to-video and image-to-video are one
surface (Request.InitImage, nil = t2v) since hybrid checkpoints like
Wan 2.2 TI2V serve both from one model; Result carries a single clip.

provider/llamaswap gains VideoModel(id) targeting the blocking
POST {base}/v1/videos/sync (multipart, model-routed by the fork's new
video routes): vLLM-Omni parameter names, OpenAI-style input_reference
file part, optional fields stay off the wire so per-model launch-flag
defaults apply. CLAUDE.md package map picks up audio/ (missed in #12)
and videogen/.

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01AXQxVhXBw8PwFAtsVrXSmj
2026-07-12 09:31:08 -04:00
steveandClaude Fable 5 9c0ac1d60b fix: address gadfly review — filename sanitization, truncation guard, shared plumbing
CI / Tidy (pull_request) Successful in 10m2s
CI / Build & Test (pull_request) Successful in 10m24s
- Transcribe: sanitize the caller-supplied multipart filename (CR/LF
  would inject Content-Disposition headers; upload metadata is
  untrusted), always send the required model/response_format fields,
  parse MIME parameters before extension matching, and give audio/opus
  its own .opus extension.
- doRaw: a response larger than maxResponseBytes is now an error, not a
  silent truncation.
- Shared plumbing: requireBaseURL() + newRequest() helpers replace the
  7x-duplicated guard/error string and the triplicated request
  building across doJSON/doRaw/Health.
- Health: non-2xx now returns *llm.APIError (package convention,
  programmatically distinguishable from transport failure) instead of
  a one-off unexported error type.
- Speak: reject negative Speed; speechMIME no longer accepts video/*
  Content-Types.
- image.go: Generate/Edit share one sdWire validate+map helper.

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01AXQxVhXBw8PwFAtsVrXSmj
2026-07-11 23:46:23 -04:00
steveandClaude Fable 5 434d721b99 feat: audio surfaces (TTS + transcription), imagegen.Editor, llamaswap health probe
CI / Tidy (pull_request) Successful in 10m9s
CI / Build & Test (pull_request) Successful in 11m16s
Adversarial Review (Gadfly) / review (pull_request) Successful in 14m51s
- New leaf package `audio` (ADR-0017): SpeechModel/SpeechProvider and
  TranscriptionModel/TranscriptionProvider with imagegen conventions
  (zero value = backend default, functional options + Apply, bytes
  in/out, never URLs). Root re-exports added.
- imagegen.Editor (ADR-0018): optional image-to-image interface —
  EditRequest carries the generation knobs plus Init image and
  denoising Strength; separate interface so existing Models keep
  compiling.
- provider/llamaswap implements all of it: POST /v1/audio/speech (JSON,
  raw-audio response, MIME from Content-Type with format fallback),
  POST /v1/audio/transcriptions (multipart, response_format=json),
  ListVoices (GET /v1/audio/voices?model=, tolerant of string-list and
  object-list shapes), POST /sdapi/v1/img2img (txt2img wire +
  init_images/denoising_strength, shared image decode), and Health(ctx)
  (GET /health) — a cheap liveness probe for often-offline hosts.
- Hermetic httptest coverage for every new wire shape and validation
  path; README sections + support-matrix footnote updated in the same
  commit (also corrects the stale /v1/images/generations claim — the
  image path has been SDAPI since the seed fix).

First consumer: mort's llamaswap media tool cluster (status / image /
TTS / STT agent tools against the netherstorm host).

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01AXQxVhXBw8PwFAtsVrXSmj
2026-07-11 23:24:14 -04:00
steve a213c18263 fix(llamaswap): use A1111 /sdapi/v1/txt2img so seed is honored
CI / Tidy (pull_request) Successful in 9m24s
CI / Build & Test (pull_request) Successful in 9m45s
Adversarial Review (Gadfly) / review (pull_request) Successful in 11m30s
The OpenAI /v1/images/generations endpoint ignores `seed` on our
stable-diffusion.cpp build — every render of a given prompt comes back
byte-identical, so a drawbot batch of N collapsed to one image. Switch the
image provider to sd-server's A1111 /sdapi/v1/txt2img endpoint, which honors
`seed` (verified live: distinct seeds -> distinct images on SDXL and
Qwen-Image). Size is split into width/height; llama-swap still routes by the
`model` field. Tests + ADR-0016 updated.
2026-06-28 22:56:25 -04:00
steve a744cdc335 feat(imagegen): optional per-request generation settings
CI / Tidy (pull_request) Successful in 9m27s
CI / Build & Test (pull_request) Successful in 9m47s
Add Steps, CFGScale, NegativePrompt, Sampler, Seed to imagegen.Request
(pointer/empty = leave the backend's per-model default), with mirror
options, and forward them in the llamaswap wire payload as the
stable-diffusion.cpp fields (steps/cfg_scale/negative_prompt/
sample_method/seed). Unset fields are omitted so sd-server keeps its
baked defaults.

Lets callers (e.g. mort drawbots) override only what they explicitly set.
2026-06-28 19:05:49 -04:00
steveandClaude Opus 4.8 64642c43c4 fix(llamaswap): address Gadfly review findings
CI / Tidy (pull_request) Successful in 9m25s
CI / Build & Test (pull_request) Successful in 10m15s
- Unload: reject model ids containing path separators (/?#) so a model name
  can't redirect the request to another endpoint; ":" (common in ids) stays
  verbatim.
- doJSON: take a model arg so image/management HTTP errors carry the target id
  (was always ""); add a base-URL guard so management methods fail clearly
  instead of building a bare-path request; cap the success-path JSON decode with
  io.LimitReader (64 MiB) and drain the body when out is nil for conn reuse.
- image: reject negative Request.N before sending.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-27 16:04:23 -04:00
steveandClaude Opus 4.8 96c612e707 feat(llamaswap): add llama-swap provider + canonical imagegen interface
CI / Tidy (pull_request) Successful in 9m25s
CI / Build & Test (pull_request) Successful in 10m15s
Add provider/llamaswap, a tailored provider for llama-swap (the model-swapping
proxy over llama.cpp / stable-diffusion.cpp). Its chat path delegates to
provider/openai at {base}/v1 — no duplicated wire client (ADR-0007) — with
legacy max_tokens, a Bearer no-key placeholder for keyless local instances, and
a timeout-free client so cold model swaps rely on context deadlines. The
"tailored" surface is concrete management methods (ListModels / Running /
Unload) that don't belong on the canonical llm.Provider interface. The
llama-swap:// DSN scheme builds an http base URL (local-first); a no-URL
built-in errors clearly on use, mirroring foreman.

Add imagegen, a new canonical text-to-image interface separate from llm
(Request/Result/Model/Provider; Image = llm.ImagePart so generated images feed
straight back into chat). First backend is llama-swap via OpenAI
/v1/images/generations (b64_json, bytes-only). Re-exported from the root. v1 is
txt2img only.

Hermetic httptest coverage for chat delegation, management endpoints, image
decode, and scheme wiring. ADR-0015 + ADR-0016, README support matrix +
image-gen section, CLAUDE.md package map, and progress.md updated in the same
commit.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-27 15:01:54 -04:00
steveandClaude Fable 5 0147a79d18 feat: conversion-driven extensions — resolvers, DefineTool, hooks, ops controls
CI / Tidy (push) Successful in 9m31s
CI / Build & Test (push) Successful in 10m13s
Phase 9a (ADR-0014): Registry.RegisterResolver for dynamic tiers;
DefineTool[Args] typed tools; Usage cache/reasoning detail fields wired
through anthropic/openai/google; WithPromptCaching (Anthropic
cache_control); agent supervision hooks (WithMaxStepsFunc, WithSteer,
WithCompactor, WithToolErrorLimits + ErrToolLoop); health
Bench/Unbench/Snapshot; ChainConfig.Observer failover events.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-06-10 13:30:06 +02:00
steveandClaude Fable 5 04b21fdad2 feat: live-validated against Ollama Cloud; schema instruction fallback for cloud
Phase 8: all six live checks pass (tier aliases, thinking-tier chat, real
tool invocation, structured Generate[T], forced failover with bench+skip,
skill agent). Discovery: ollama.com ignores the format field — the
provider now also states the schema as a system instruction (constrained
decoding locally, instruction-guided JSON on cloud), with hermetic test.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-06-10 13:22:54 +02:00
steveandClaude Fable 5 1ca607906d feat: Google (Gemini) provider on the official Gen AI SDK
Phase 4: provider/google on google.golang.org/genai v1.59.0 — lazy cached
client, FunctionResponse tool loop, raw-JSON-schema tools and structured
output, ThinkingLevel reasoning mapping, iter.Pull2 streaming, hermetic
httptest suite via HTTPOptions.BaseURL. Registry wires google + gemini
schemes to the real client; stub machinery deleted (all built-ins real).
ADR-0011; README matrix + CLAUDE.md synced.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-06-10 13:04:28 +02:00
steveandClaude Fable 5 043249e0e1 feat: OpenAI, Anthropic, and native-Ollama providers + media pipeline
Phase 3:
- provider/openai: Chat Completions for OpenAI + compat endpoints (SSE
  streaming with by-index tool-call assembly, response_format json_schema,
  legacy max_tokens option, reasoning_effort)
- provider/anthropic: Messages API (tool_use/tool_result, GA structured
  output via output_config.format, full SSE event parser, 529 transient)
- provider/ollama: one native /api/chat client behind the ollama,
  ollama-cloud, and foreman built-ins (presets; NDJSON streaming tolerant
  of foreman's buffered single-object responses; object tool arguments;
  format-schema structured output; think mapping)
- media/: capability normalization (sniff, downscale, transcode, byte
  ladder, ErrUnsupported), wired into the chain executor per target with
  penalty-free advance past incapable elements
- registry: real provider + scheme wiring, WithHTTPClient option, required
  env-foreman TLS chat round-trip test
- ADR-0009 multimodal strategy, ADR-0010 tools/structured mapping; README
  matrix + CLAUDE.md synced

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-06-10 12:58:08 +02:00
steveandClaude Fable 5 dcd004289f feat: foundations — canonical types, Parse grammar, env DSNs, health, chains
Phase 1 of the majordomo build:
- llm/ canonical contract (messages, parts, tools, capabilities, streaming,
  Model/Provider, error classification)
- health/ clock-injected tracker (threshold bench, exponential capped
  cooldown, reset-on-success)
- root Registry + Parse (verbatim model ids, inline recursive alias
  expansion with cycle detection, chain dedup), LLM_* env-DSN providers
  (go-llm parity: lazy fallback + eager LoadEnv), health-aware chain
  executor behind the Model interface
- provider/fake scriptable test provider; hermetic test suite incl. the
  trailing-thinking chain and foreman:// env loading
- ADRs 0001-0008, CLAUDE.md, README (honest matrix), CI workflow,
  docs/phase-1-design.md

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-06-10 12:35:34 +02:00