fix(llamaswap): a headerless non-image response was returned as a PNG
CI / Tidy (pull_request) Successful in 9m26s
CI / Build & Test (pull_request) Successful in 10m4s

Gadfly on #23, blocking, 2/2 agreement — and it is the exact defect this
whole line of work has been about: a call that succeeds while handing back
the wrong bytes.

sniffImageMIME falls back to image/png when detection is inconclusive, and
the guard only consulted Content-Type. A response with NO Content-Type
therefore skipped the check entirely and was labelled a PNG. The shim answers
JSON on a semantic miss (no face found in the source or target), which is
precisely the body that would have sailed through as a successful image.

The check now validates the BYTES — http.DetectContentType must say image/ —
and the reported MIME prefers the server's own label only when that label is
itself an image type. Break-checked by restoring the header-only condition,
which fails the new test.

Also from that review:
  - index is documented as ignored under all=true, so a negative one is no
    longer rejected there; it is still rejected when it would actually be
    sent, and both halves are tested.
  - initImageFilename (video.go) was imageFilename with the base fixed to
    "frame" and now delegates to it — two copies of one extension table is
    how they drift.
  - DetectedFace carried Width/Height alongside Box, two sources of truth for
    one fact that can disagree after any transform. Now a Size() method
    derived from Box.
  - a dead `apiErr` in the test (declared, then `_ = apiErr`) was an
    abandoned errors.As check; it is wired up and now asserts callers can
    classify the error.
  - swapImg duplicated editInit verbatim; removed.

Not taken: adding a FaceSwapProvider/ModelOption surface to match the other
optional imagegen capabilities (single-model finding). There are no options
to carry yet, and inventing an empty option type to look symmetrical would be
API surface with nothing behind it. Worth revisiting when a real knob exists.
This commit is contained in:
2026-07-31 12:34:58 -04:00
parent 0ff90d80f6
commit 372bf826aa
4 changed files with 96 additions and 47 deletions
+25 -15
View File
@@ -74,7 +74,7 @@ func (m *faceSwapModel) ListFaces(ctx context.Context, img imagegen.Image) ([]im
}
out := make([]imagegen.DetectedFace, 0, len(parsed.Faces))
for _, f := range parsed.Faces {
df := imagegen.DetectedFace{Index: f.Index, Score: f.Score, Width: f.Width, Height: f.Height}
df := imagegen.DetectedFace{Index: f.Index, Score: f.Score}
// A short box would silently index out of range below; treat a
// malformed entry as a protocol error rather than zero-filling it,
// because a wrong box sends the caller at the wrong face.
@@ -97,7 +97,10 @@ func (m *faceSwapModel) FaceSwap(ctx context.Context, req imagegen.FaceSwapReque
if len(req.Source.Data) == 0 {
return nil, fmt.Errorf("%w: face swap requires a source image", llm.ErrUnsupported)
}
if req.Index != nil && *req.Index < 0 {
// Only when it will actually be sent: under All the index is documented
// as ignored, so rejecting a negative one there would fail a request that
// is perfectly well formed.
if !req.All && req.Index != nil && *req.Index < 0 {
return nil, fmt.Errorf("%w: face index must be >= 0, got %d", llm.ErrUnsupported, *req.Index)
}
path, err := upstreamPath(m.id, "/v1/faceswap")
@@ -130,20 +133,34 @@ func (m *faceSwapModel) FaceSwap(ctx context.Context, req imagegen.FaceSwapReque
if len(raw) == 0 {
return nil, &llm.APIError{Provider: m.p.name, Model: m.id, Message: "face swap response contained no image"}
}
mime := sniffImageMIME(raw)
if respType != "" && !isImageContentType(respType) {
// A JSON error body sniffs as text, not image — say what came back
// rather than handing the caller bytes that are not a picture.
// Validate the BYTES, not the header. sniffImageMIME falls back to
// image/png when detection is inconclusive, so trusting it here would
// label a JSON error body as a PNG and return it as a successful image —
// and a header check alone misses the case where the response carries no
// Content-Type at all. The shim answers JSON on a semantic miss (no face
// found), which is exactly the body that would sail through.
detected := http.DetectContentType(raw)
if !strings.HasPrefix(detected, "image/") {
return nil, &llm.APIError{Provider: m.p.name, Model: m.id,
Message: fmt.Sprintf("face swap response is not an image (Content-Type %q): %s", respType, truncateForError(raw))}
Message: fmt.Sprintf("face swap response is not an image (sniffed %q, Content-Type %q): %s",
detected, respType, truncateForError(raw))}
}
return &imagegen.Result{Images: []llm.ImagePart{{MIME: mime, Data: raw}}}, nil
// Prefer the server's own label when it is an image type (it knows
// subtypes the sniffer does not), else what the bytes actually are.
mimeType := detected
if hdr := mimeFromContentType(respType, "image/"); hdr != "" {
mimeType = hdr
}
return &imagegen.Result{Images: []llm.ImagePart{{MIME: mimeType, Data: raw}}}, nil
}
// imageFilename picks a multipart filename for an image part. The shim reads
// bytes, not names, but a plausible extension keeps server-side sniffing and
// request logs honest. base distinguishes the parts of a multi-file form
// ("target"/"source") so a log line says which one was malformed.
//
// initImageFilename (video.go) is this function with base fixed to "frame"
// and delegates here — two copies of one extension table is how they drift.
func imageFilename(mimeType, base string) string {
if base == "" {
base = "image"
@@ -167,10 +184,3 @@ func imageFilename(mimeType, base string) string {
return base + ".png"
}
}
// isImageContentType reports whether a response Content-Type is an image.
// Used to tell a returned picture apart from a JSON error body, which would
// otherwise be handed back as "image" bytes.
func isImageContentType(contentType string) bool {
return mimeFromContentType(contentType, "image/") != ""
}