Add a sixth image entry that runs ComfyUI, unlisted: true, leaving the five sd-server entries untouched and serving traffic. Nothing to lose: if it is worse, delete three lines of yaml.
images/comfyui-image/, modelled on the existing images/comfyui-h3/:
Dockerfile — pytorch/pytorch:2.8.0-cuda12.8-cudnn9-runtime base, ComfyUI pinned to a specific commit (ARG COMFYUI_REF, bump deliberately — same rule as comfyui-h3), plus city96/ComfyUI-GGUF pinned into custom_nodes/. Weights are not baked in — repo invariant.
extra_model_paths.yaml — points ComfyUI's unet/diffusion_models, clip/text_encoders, and vae folders at the mounted /models, which is the host's existing /srv/models/sd. That directory is already in ComfyUI's taxonomy; nothing moves and nothing is re-downloaded.
shim.py — aiohttp server on :8080 speaking the A1111 contract majordomo actually calls.
entrypoint.sh — start ComfyUI headless on 127.0.0.1:8188, then the shim.
Shim contract — this is the part that has to be exactly right
majordomo (provider/llamaswap/image.go) calls POST /sdapi/v1/txt2img and POST /sdapi/v1/img2img, not the OpenAI routes, because sd-server ignores seed on /v1/images/generations — every render of a prompt comes back byte-identical and a batch of N collapses to one image.
Request fields that must be honoured:
Field
Meaning
model
picks the graph — imagegen-flux-dev, -schnell, -kontext, imagegen-qwen-image, -qwen-image-edit
prompt, negative_prompt
Flux runs CFG 1 / no negative branch; Qwen-Image uses CFG 2.5. Log-and-ignore where the graph has no slot, same as the H3 shim does
seed
must vary output. The whole reason this route exists
steps, cfg_scale, sample_method
optional; unset falls back to the per-model default that is currently baked into the docker cmd
width, height
batch_count
N images in one response
extra_images (img2img)
ref-image conditioning for Kontext / Qwen-Image-Edit. These two are instruction-edit models — they take the source as conditioning and the prompt as an instruction about it. Sending them a plain img2img request silently does the wrong thing
Response: {"images": ["<base64>", ...]}.
/health returns 503 until ComfyUI's HTTP is up (proxy /system_stats), so llama-swap's checkEndpoint keeps waiting through interpreter start and node import — copy the comfyui-h3 pattern.
Per-model defaults to carry over verbatim
From the current cmd lines. sd-server defaults to cfg 7, which burns every model here — these are not style, they are correctness:
flux-schnell / flux-kontext: same family, own step counts
qwen-image / qwen-image-edit: cfg 2.5, Qwen2.5-VL-7B-Instruct text encoder, qwen_image_vae.safetensors
llama-swap entry
"comfyui-image":unlisted:truename:"ComfyUI image (A/B candidate)"cmd:| docker run --name ${MODEL_ID} --rm --init --network web --gpus all --ipc=host
-v /srv/models/sd:/models
comfyui-image:localcmdStop:docker stop ${MODEL_ID}proxy:http://${MODEL_ID}:8080checkEndpoint:/healthttl:300
Add it to the all group. unlisted: true keeps it out of /v1/models so mort never routes to it by accident — drive it with curl for the A/B.
Acceptance
All five models render through the shim with correct sampler/cfg defaults
Two calls with different seed produce different images; two with the same seed produce the same one
batch_count: 3 returns three images
Kontext and Qwen-Image-Edit take extra_images and actually follow the instruction
Timings recorded, cold and warm, against the sd-server baseline: warm flux-dev 10–33s, cold-with-swap 45s–3m
RAM watched during back-to-back model switches — 46 GB total, and ComfyUI's default model cache will thrash without --cache-none or an LRU cap
Notes
sd_xl_base_1.0.safetensors (6.9 GB) is still in /srv/models/sd/checkpoints although imagegen-sdxl was removed. Not in scope; flagging it as reclaimable.
Keep image and video as separate ComfyUI images. One process is one crash domain, and a wedged mega-comfy takes both down.
Part of #3.
Add a **sixth** image entry that runs ComfyUI, `unlisted: true`, leaving the five `sd-server` entries untouched and serving traffic. Nothing to lose: if it is worse, delete three lines of yaml.
## Deliverables (steveternet `azeroth/outland/netherstorm/llama-swap/`)
`images/comfyui-image/`, modelled on the existing `images/comfyui-h3/`:
- **`Dockerfile`** — `pytorch/pytorch:2.8.0-cuda12.8-cudnn9-runtime` base, ComfyUI pinned to a specific commit (`ARG COMFYUI_REF`, bump deliberately — same rule as comfyui-h3), plus `city96/ComfyUI-GGUF` pinned into `custom_nodes/`. Weights are **not** baked in — repo invariant.
- **`extra_model_paths.yaml`** — points ComfyUI's `unet`/`diffusion_models`, `clip`/`text_encoders`, and `vae` folders at the mounted `/models`, which is the host's existing `/srv/models/sd`. That directory is *already* in ComfyUI's taxonomy; nothing moves and nothing is re-downloaded.
- **`shim.py`** — aiohttp server on :8080 speaking the A1111 contract majordomo actually calls.
- **`entrypoint.sh`** — start ComfyUI headless on 127.0.0.1:8188, then the shim.
## Shim contract — this is the part that has to be exactly right
majordomo (`provider/llamaswap/image.go`) calls `POST /sdapi/v1/txt2img` and `POST /sdapi/v1/img2img`, **not** the OpenAI routes, because sd-server ignores `seed` on `/v1/images/generations` — every render of a prompt comes back byte-identical and a batch of N collapses to one image.
Request fields that must be honoured:
| Field | Meaning |
|---|---|
| `model` | picks the graph — `imagegen-flux-dev`, `-schnell`, `-kontext`, `imagegen-qwen-image`, `-qwen-image-edit` |
| `prompt`, `negative_prompt` | Flux runs CFG 1 / no negative branch; Qwen-Image uses CFG 2.5. Log-and-ignore where the graph has no slot, same as the H3 shim does |
| `seed` | **must vary output.** The whole reason this route exists |
| `steps`, `cfg_scale`, `sample_method` | optional; unset falls back to the per-model default that is currently baked into the docker `cmd` |
| `width`, `height` | |
| `batch_count` | N images in one response |
| `extra_images` (img2img) | ref-image conditioning for Kontext / Qwen-Image-Edit. **These two are instruction-edit models** — they take the source as conditioning and the prompt as an instruction about it. Sending them a plain img2img request silently does the wrong thing |
Response: `{"images": ["<base64>", ...]}`.
`/health` returns 503 until ComfyUI's HTTP is up (proxy `/system_stats`), so llama-swap's `checkEndpoint` keeps waiting through interpreter start and node import — copy the comfyui-h3 pattern.
## Per-model defaults to carry over verbatim
From the current `cmd` lines. sd-server defaults to cfg 7, which burns every model here — these are not style, they are correctness:
- flux-dev: `--cfg-scale 1 --guidance 3.5 --steps 20 --sampling-method euler`, t5xxl + clip_l, `ae.safetensors`
- flux-schnell / flux-kontext: same family, own step counts
- qwen-image / qwen-image-edit: cfg 2.5, `Qwen2.5-VL-7B-Instruct` text encoder, `qwen_image_vae.safetensors`
## llama-swap entry
```yaml
"comfyui-image":
unlisted: true
name: "ComfyUI image (A/B candidate)"
cmd: |
docker run --name ${MODEL_ID} --rm --init --network web --gpus all --ipc=host
-v /srv/models/sd:/models
comfyui-image:local
cmdStop: docker stop ${MODEL_ID}
proxy: http://${MODEL_ID}:8080
checkEndpoint: /health
ttl: 300
```
Add it to the `all` group. `unlisted: true` keeps it out of `/v1/models` so mort never routes to it by accident — drive it with `curl` for the A/B.
## Acceptance
- All five models render through the shim with correct sampler/cfg defaults
- Two calls with different `seed` produce different images; two with the same `seed` produce the same one
- `batch_count: 3` returns three images
- Kontext and Qwen-Image-Edit take `extra_images` and actually follow the instruction
- Timings recorded, cold and warm, against the sd-server baseline: **warm flux-dev 10–33s, cold-with-swap 45s–3m**
- RAM watched during back-to-back model switches — 46 GB total, and ComfyUI's default model cache will thrash without `--cache-none` or an LRU cap
## Notes
- `sd_xl_base_1.0.safetensors` (6.9 GB) is still in `/srv/models/sd/checkpoints` although `imagegen-sdxl` was removed. Not in scope; flagging it as reclaimable.
- Keep image and video as separate ComfyUI images. One process is one crash domain, and a wedged mega-comfy takes both down.
steveternet 6acf5c6. images/comfyui-image/ + the unlisted: true llama-swap entry are live on netherstorm; the five sd-server entries are untouched and still serving.
Weight re-use confirmed as designed: UnetLoaderGGUF's enum lists all five existing GGUFs straight out of /srv/models/sd/diffusion_models, and VAELoader sees both VAEs. Nothing was moved and nothing was re-downloaded except one new text encoder (below). Image build was 47s — every layer up to the custom node is shared with comfyui-h3.
Every node class the shim emits was checked against this ComfyUI pin's /object_info before writing the graphs, rather than guessed: UnetLoaderGGUF, CLIPLoaderGGUF, DualCLIPLoaderGGUF, VAELoader, CLIPTextEncode, FluxGuidance, EmptySD3LatentImage, KSampler, VAEDecode, SaveImage, LoadImage, VAEEncode, ImageToMask, SetLatentNoiseMask, ModelSamplingAuraFlow, TextEncodeQwenImageEditPlus, FluxKontextImageScale, ReferenceLatent, ImageStitch — all present, all input names matching.
Measured (direct to the shim, netherstorm)
path
result
flux-schnell t2i
21s cold / 8s warm
flux-dev t2i
24s cold / 18s warm
flux-kontext instruction edit
44s; 33s with 2 refs stitched
qwen-image t2i (40 steps)
113s → 106s on the new encoder
qwen-image-edit instruction edit
166s cold / 154s warm
flux-dev img2img @ denoise 0.6
29s
flux-dev inpaint
19s; mask polarity confirmed white = repaint
seed
same seed → byte-identical; different seed → different
batch_count: 3
→ 3 images
sd-server baseline from the live logs for comparison: flux-dev warm 10–33s, cold-with-swap 45s–3m.
Two real defects the A/B caught
1. Kontext conditioning order.CLIPTextEncode → ReferenceLatent → FluxGuidance, in that order. Building guidance first and rerouting its input creates a dependency cycle and ComfyUI rejects the graph outright (dependency_cycle: pos_guided (FluxGuidance) -> refl (ReferenceLatent) -> pos_guided). Fixed by splitting encode_text() from apply_guidance() so the caller controls where ReferenceLatent goes.
2. Qwen-Image-Edit needs a different text encoder than sd-server does — the exact inverse of the existing rule. TextEncodeQwenImageEditPlus pushes the reference image through Qwen2.5-VL's vision tower. The unsloth GGUF sd-server passes as --llm carries only the language half, so the request dies in the projector:
mat1 and mat2 shapes cannot be multiplied (784x1280 and 3840x1280)
Fix: Comfy-Org/Qwen-Image_ComfyUI → qwen_2.5_vl_7b_fp8_scaled.safetensors (9.4 GB), now fetched by fetch-models.sh. fp8 rather than the nvfp4 build of the same weights, because fp8 is native on Ada and nvfp4 is emulated.
Note this contradictsfetch-models.sh's existing sd-server rule — "Encoder MUST be GGUF; the ComfyUI fp8 one silently produces blank images". Both are true, of different engines. Both encoders stay on disk until one engine is retired in #5. Qwen-Image t2i works off either and points at the new one so there is only one thing to be right about.
llama-swap integration verified
The all group evicted imagegen-flux-dev and started comfyui-image, whose /health gate passed — so cmd, cmdStop, checkEndpoint and group membership all behave. /v1/models still lists exactly the five imagegen-* ids and notcomfyui-image, which is what unlisted: true is for.
Memory
--cache-none disables the intermediate node-result cache, not the model cache — worth stating because the first version of the comment here claimed otherwise. It is set because every request is a fresh graph, so the cache buys nothing and costs headroom. Measured after the first renders: 3 GB resident, 43 GB page cache — the GGUFs come off reclaimable page cache, not a Python-side cache that needs bounding. Peak during the Qwen-Image-Edit run: 20.9 GB VRAM, 11 GB RSS.
Head-to-head sd-server vs ComfyUI on identical prompt + seed, for output quality rather than wall clock — in progress.
Qwen-Image-Edit at 154s warm is the one number that does not obviously beat sd-server. Needs a like-for-like comparison before the alias flip; if it is genuinely slower, that is an argument for keeping imagegen-qwen-image-edit on sd-server while the other four move.
## Built, deployed, all five models rendering
steveternet `6acf5c6`. `images/comfyui-image/` + the `unlisted: true` llama-swap entry are live on netherstorm; the five sd-server entries are untouched and still serving.
Weight re-use confirmed as designed: `UnetLoaderGGUF`'s enum lists all five existing GGUFs straight out of `/srv/models/sd/diffusion_models`, and `VAELoader` sees both VAEs. Nothing was moved and nothing was re-downloaded except one new text encoder (below). Image build was 47s — every layer up to the custom node is shared with `comfyui-h3`.
Every node class the shim emits was checked against this ComfyUI pin's `/object_info` before writing the graphs, rather than guessed: `UnetLoaderGGUF`, `CLIPLoaderGGUF`, `DualCLIPLoaderGGUF`, `VAELoader`, `CLIPTextEncode`, `FluxGuidance`, `EmptySD3LatentImage`, `KSampler`, `VAEDecode`, `SaveImage`, `LoadImage`, `VAEEncode`, `ImageToMask`, `SetLatentNoiseMask`, `ModelSamplingAuraFlow`, `TextEncodeQwenImageEditPlus`, `FluxKontextImageScale`, `ReferenceLatent`, `ImageStitch` — all present, all input names matching.
### Measured (direct to the shim, netherstorm)
| path | result |
|---|---|
| flux-schnell t2i | 21s cold / **8s warm** |
| flux-dev t2i | 24s cold / **18s warm** |
| flux-kontext instruction edit | 44s; 33s with 2 refs stitched |
| qwen-image t2i (40 steps) | 113s → 106s on the new encoder |
| qwen-image-edit instruction edit | 166s cold / 154s warm |
| flux-dev img2img @ denoise 0.6 | 29s |
| flux-dev inpaint | 19s; mask polarity confirmed **white = repaint** |
| seed | same seed → byte-identical; different seed → different |
| `batch_count: 3` | → 3 images |
sd-server baseline from the live logs for comparison: flux-dev warm 10–33s, cold-with-swap 45s–3m.
### Two real defects the A/B caught
**1. Kontext conditioning order.** `CLIPTextEncode → ReferenceLatent → FluxGuidance`, in that order. Building guidance first and rerouting its input creates a dependency cycle and ComfyUI rejects the graph outright (`dependency_cycle: pos_guided (FluxGuidance) -> refl (ReferenceLatent) -> pos_guided`). Fixed by splitting `encode_text()` from `apply_guidance()` so the caller controls where ReferenceLatent goes.
**2. Qwen-Image-Edit needs a different text encoder than sd-server does — the exact inverse of the existing rule.**
`TextEncodeQwenImageEditPlus` pushes the reference image through Qwen2.5-VL's **vision** tower. The unsloth GGUF sd-server passes as `--llm` carries only the language half, so the request dies in the projector:
```
mat1 and mat2 shapes cannot be multiplied (784x1280 and 3840x1280)
```
Fix: `Comfy-Org/Qwen-Image_ComfyUI` → `qwen_2.5_vl_7b_fp8_scaled.safetensors` (9.4 GB), now fetched by `fetch-models.sh`. fp8 rather than the nvfp4 build of the same weights, because fp8 is native on Ada and nvfp4 is emulated.
Note this **contradicts** `fetch-models.sh`'s existing sd-server rule — "Encoder MUST be GGUF; the ComfyUI fp8 one silently produces blank images". Both are true, of different engines. Both encoders stay on disk until one engine is retired in #5. Qwen-Image t2i works off either and points at the new one so there is only one thing to be right about.
### llama-swap integration verified
The `all` group evicted `imagegen-flux-dev` and started `comfyui-image`, whose `/health` gate passed — so `cmd`, `cmdStop`, `checkEndpoint` and group membership all behave. `/v1/models` still lists exactly the five `imagegen-*` ids and **not** `comfyui-image`, which is what `unlisted: true` is for.
### Memory
`--cache-none` disables the intermediate **node-result** cache, not the model cache — worth stating because the first version of the comment here claimed otherwise. It is set because every request is a fresh graph, so the cache buys nothing and costs headroom. Measured after the first renders: 3 GB resident, 43 GB page cache — the GGUFs come off reclaimable page cache, not a Python-side cache that needs bounding. Peak during the Qwen-Image-Edit run: 20.9 GB VRAM, 11 GB RSS.
### Still open before #5
- Head-to-head sd-server vs ComfyUI on identical prompt + seed, for output quality rather than wall clock — in progress.
- **Qwen-Image-Edit at 154s warm is the one number that does not obviously beat sd-server.** Needs a like-for-like comparison before the alias flip; if it is genuinely slower, that is an argument for keeping `imagegen-qwen-image-edit` on sd-server while the other four move.
A/B results — and a measurement trap worth recording
The trap
The first head-to-head made ComfyUI look 5x slower on qwen-image (20s vs 111s). It was not slower. sd-server rendered 512×512 and the shim rendered 1024×1024 — 4x the pixels. Neither harness sent an explicit size.
That is not just a harness bug, it is a behaviour trap in the migration: mort's llamaswap_generate_image documents size as "omit for the model's default", and majordomo's parseSize("") sends no width/height at all — so whatever the engine defaults to is the contract for every size-less generation. sd.cpp defaults to 512. The shim was defaulting to 1024, which would have silently quadrupled the pixels and the latency of those calls the moment the aliases flipped in #5, with nothing in the diff saying so. Fixed in bf75dcb: DEFAULT_SIZE = 512.
Separately worth deciding: 512 is a poor default for these models, all of which are trained at 1024, so a size-less Flux render today is worse than it needs to be. That is a real finding but a product decision — it belongs in the tool description on mort's side, not smuggled into an engine swap.
Warm, same model, repeated (1024×1024, the number that decides #5)
call 1 (cold)
call 2
call 3
sd-server flux-schnell
15s
10s
10s
ComfyUI flux-schnell
23s
8s
8s
ComfyUI is ~20% faster warm. Its cold call is ~13s more expensive because a Python interpreter plus custom-node import costs more to boot than an sd-server binary — and every call in the alternating A/B below paid that boot, which is what made ComfyUI look bad.
Cold-to-cold, alternating engines (1024×1024, seed 4242, both swapping every call)
model
sd-server
ComfyUI
flux-dev
35s
34s
flux-schnell
15s
23s
qwen-image
66s
111s
The phase-1 payoff, measured
Switching models inside a warm ComfyUI: schnell → dev = 26s, then dev → schnell = 8s. The container never restarts, so the second switch costs nothing but the DiT load, and going back to an already-touched model is free. Today the same switch is docker stop + docker run + engine boot + weight load every time.
Output quality
Parity. Both engines render "NETHERSTORM" correctly on qwen-image (that model's whole point) and both flux-dev foxes are clean 1024² images. Same seed gives different images across engines — expected, the samplers and noise schedules differ — and seed reproducibility within an engine was verified byte-identical.
The one real regression: qwen-image is ~2x slower on ComfyUI
~106s warm vs ~55s. Cause is almost certainly ComfyUI-GGUF dequantizing Q5_K_M weights on the fly in the forward pass, where sd.cpp has native GGUF kernels. Flux is barely affected because Q8_0 is nearly trivial to dequantize; a Q5_K_M 20B-class model at 40 steps is not.
Recommended experiment before #5 (not run — phase 0's job was to surface this, and a 20 GB download plus another test cycle is phase-1 prep): swap the Qwen-Image DiT for Comfy-Org/Qwen-Image_ComfyUI → split_files/diffusion_models/qwen_image_fp8_e4m3fn.safetensors (20.4 GB). fp8 is native on Ada so there is no dequant step at all. VRAM goes 14.9 → 20.4 GB, which fits the 4090 given ComfyUI offloads the text encoder after encoding, but it is tight and wants measuring. Do not use the nvfp4 build — emulated on Ada.
If that does not close the gap, the fallback is simply to leave imagegen-qwen-image and imagegen-qwen-image-edit on sd-server and alias only the three Flux models in #5. The alias mechanism is per-model, so a partial flip is expressible.
Phase 0 status: done
Built, deployed, all five models rendering correctly through both the shim directly and through llama-swap's /upstream route; cmd/cmdStop/checkEndpoint/group eviction all verified; unlisted: true confirmed keeping it out of /v1/models. steveternet at bf75dcb, host repo fast-forwarded, image rebuilt and byte-identical to the committed shim.
## A/B results — and a measurement trap worth recording
### The trap
The first head-to-head made ComfyUI look 5x slower on qwen-image (20s vs 111s). It was not slower. **sd-server rendered 512×512 and the shim rendered 1024×1024** — 4x the pixels. Neither harness sent an explicit size.
That is not just a harness bug, it is a **behaviour trap in the migration**: mort's `llamaswap_generate_image` documents `size` as "omit for the model's default", and majordomo's `parseSize("")` sends no width/height at all — so whatever the *engine* defaults to is the contract for every size-less generation. sd.cpp defaults to 512. The shim was defaulting to 1024, which would have silently quadrupled the pixels and the latency of those calls the moment the aliases flipped in #5, with nothing in the diff saying so. Fixed in `bf75dcb`: `DEFAULT_SIZE = 512`.
Separately worth deciding: **512 is a poor default for these models**, all of which are trained at 1024, so a size-less Flux render today is worse than it needs to be. That is a real finding but a product decision — it belongs in the tool description on mort's side, not smuggled into an engine swap.
### Warm, same model, repeated (1024×1024, the number that decides #5)
| | call 1 (cold) | call 2 | call 3 |
|---|---|---|---|
| sd-server flux-schnell | 15s | **10s** | **10s** |
| ComfyUI flux-schnell | 23s | **8s** | **8s** |
**ComfyUI is ~20% faster warm.** Its cold call is ~13s more expensive because a Python interpreter plus custom-node import costs more to boot than an sd-server binary — and every call in the alternating A/B below paid that boot, which is what made ComfyUI look bad.
### Cold-to-cold, alternating engines (1024×1024, seed 4242, both swapping every call)
| model | sd-server | ComfyUI |
|---|---|---|
| flux-dev | 35s | 34s |
| flux-schnell | 15s | 23s |
| qwen-image | 66s | 111s |
### The phase-1 payoff, measured
Switching models *inside* a warm ComfyUI: schnell → dev = 26s, then dev → schnell = **8s**. The container never restarts, so the second switch costs nothing but the DiT load, and going back to an already-touched model is free. Today the same switch is `docker stop` + `docker run` + engine boot + weight load every time.
### Output quality
Parity. Both engines render "NETHERSTORM" correctly on qwen-image (that model's whole point) and both flux-dev foxes are clean 1024² images. Same seed gives *different* images across engines — expected, the samplers and noise schedules differ — and seed reproducibility *within* an engine was verified byte-identical.
### The one real regression: qwen-image is ~2x slower on ComfyUI
~106s warm vs ~55s. Cause is almost certainly `ComfyUI-GGUF` dequantizing Q5_K_M weights on the fly in the forward pass, where sd.cpp has native GGUF kernels. Flux is barely affected because Q8_0 is nearly trivial to dequantize; a Q5_K_M 20B-class model at 40 steps is not.
**Recommended experiment before #5** (not run — phase 0's job was to surface this, and a 20 GB download plus another test cycle is phase-1 prep): swap the Qwen-Image DiT for `Comfy-Org/Qwen-Image_ComfyUI` → `split_files/diffusion_models/qwen_image_fp8_e4m3fn.safetensors` (20.4 GB). fp8 is native on Ada so there is no dequant step at all. VRAM goes 14.9 → 20.4 GB, which fits the 4090 given ComfyUI offloads the text encoder after encoding, but it is tight and wants measuring. Do not use the nvfp4 build — emulated on Ada.
If that does not close the gap, the fallback is simply to leave `imagegen-qwen-image` and `imagegen-qwen-image-edit` on sd-server and alias only the three Flux models in #5. The alias mechanism is per-model, so a partial flip is expressible.
### Phase 0 status: done
Built, deployed, all five models rendering correctly through both the shim directly and through llama-swap's `/upstream` route; `cmd`/`cmdStop`/`checkEndpoint`/group eviction all verified; `unlisted: true` confirmed keeping it out of `/v1/models`. steveternet at `bf75dcb`, host repo fast-forwarded, image rebuilt and byte-identical to the committed shim.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Part of #3.
Add a sixth image entry that runs ComfyUI,
unlisted: true, leaving the fivesd-serverentries untouched and serving traffic. Nothing to lose: if it is worse, delete three lines of yaml.Deliverables (steveternet
azeroth/outland/netherstorm/llama-swap/)images/comfyui-image/, modelled on the existingimages/comfyui-h3/:Dockerfile—pytorch/pytorch:2.8.0-cuda12.8-cudnn9-runtimebase, ComfyUI pinned to a specific commit (ARG COMFYUI_REF, bump deliberately — same rule as comfyui-h3), pluscity96/ComfyUI-GGUFpinned intocustom_nodes/. Weights are not baked in — repo invariant.extra_model_paths.yaml— points ComfyUI'sunet/diffusion_models,clip/text_encoders, andvaefolders at the mounted/models, which is the host's existing/srv/models/sd. That directory is already in ComfyUI's taxonomy; nothing moves and nothing is re-downloaded.shim.py— aiohttp server on :8080 speaking the A1111 contract majordomo actually calls.entrypoint.sh— start ComfyUI headless on 127.0.0.1:8188, then the shim.Shim contract — this is the part that has to be exactly right
majordomo (
provider/llamaswap/image.go) callsPOST /sdapi/v1/txt2imgandPOST /sdapi/v1/img2img, not the OpenAI routes, because sd-server ignoresseedon/v1/images/generations— every render of a prompt comes back byte-identical and a batch of N collapses to one image.Request fields that must be honoured:
modelimagegen-flux-dev,-schnell,-kontext,imagegen-qwen-image,-qwen-image-editprompt,negative_promptseedsteps,cfg_scale,sample_methodcmdwidth,heightbatch_countextra_images(img2img)Response:
{"images": ["<base64>", ...]}./healthreturns 503 until ComfyUI's HTTP is up (proxy/system_stats), so llama-swap'scheckEndpointkeeps waiting through interpreter start and node import — copy the comfyui-h3 pattern.Per-model defaults to carry over verbatim
From the current
cmdlines. sd-server defaults to cfg 7, which burns every model here — these are not style, they are correctness:--cfg-scale 1 --guidance 3.5 --steps 20 --sampling-method euler, t5xxl + clip_l,ae.safetensorsQwen2.5-VL-7B-Instructtext encoder,qwen_image_vae.safetensorsllama-swap entry
Add it to the
allgroup.unlisted: truekeeps it out of/v1/modelsso mort never routes to it by accident — drive it withcurlfor the A/B.Acceptance
seedproduce different images; two with the sameseedproduce the same onebatch_count: 3returns three imagesextra_imagesand actually follow the instruction--cache-noneor an LRU capNotes
sd_xl_base_1.0.safetensors(6.9 GB) is still in/srv/models/sd/checkpointsalthoughimagegen-sdxlwas removed. Not in scope; flagging it as reclaimable.Built, deployed, all five models rendering
steveternet
6acf5c6.images/comfyui-image/+ theunlisted: truellama-swap entry are live on netherstorm; the five sd-server entries are untouched and still serving.Weight re-use confirmed as designed:
UnetLoaderGGUF's enum lists all five existing GGUFs straight out of/srv/models/sd/diffusion_models, andVAELoadersees both VAEs. Nothing was moved and nothing was re-downloaded except one new text encoder (below). Image build was 47s — every layer up to the custom node is shared withcomfyui-h3.Every node class the shim emits was checked against this ComfyUI pin's
/object_infobefore writing the graphs, rather than guessed:UnetLoaderGGUF,CLIPLoaderGGUF,DualCLIPLoaderGGUF,VAELoader,CLIPTextEncode,FluxGuidance,EmptySD3LatentImage,KSampler,VAEDecode,SaveImage,LoadImage,VAEEncode,ImageToMask,SetLatentNoiseMask,ModelSamplingAuraFlow,TextEncodeQwenImageEditPlus,FluxKontextImageScale,ReferenceLatent,ImageStitch— all present, all input names matching.Measured (direct to the shim, netherstorm)
batch_count: 3sd-server baseline from the live logs for comparison: flux-dev warm 10–33s, cold-with-swap 45s–3m.
Two real defects the A/B caught
1. Kontext conditioning order.
CLIPTextEncode → ReferenceLatent → FluxGuidance, in that order. Building guidance first and rerouting its input creates a dependency cycle and ComfyUI rejects the graph outright (dependency_cycle: pos_guided (FluxGuidance) -> refl (ReferenceLatent) -> pos_guided). Fixed by splittingencode_text()fromapply_guidance()so the caller controls where ReferenceLatent goes.2. Qwen-Image-Edit needs a different text encoder than sd-server does — the exact inverse of the existing rule.
TextEncodeQwenImageEditPluspushes the reference image through Qwen2.5-VL's vision tower. The unsloth GGUF sd-server passes as--llmcarries only the language half, so the request dies in the projector:Fix:
Comfy-Org/Qwen-Image_ComfyUI→qwen_2.5_vl_7b_fp8_scaled.safetensors(9.4 GB), now fetched byfetch-models.sh. fp8 rather than the nvfp4 build of the same weights, because fp8 is native on Ada and nvfp4 is emulated.Note this contradicts
fetch-models.sh's existing sd-server rule — "Encoder MUST be GGUF; the ComfyUI fp8 one silently produces blank images". Both are true, of different engines. Both encoders stay on disk until one engine is retired in #5. Qwen-Image t2i works off either and points at the new one so there is only one thing to be right about.llama-swap integration verified
The
allgroup evictedimagegen-flux-devand startedcomfyui-image, whose/healthgate passed — socmd,cmdStop,checkEndpointand group membership all behave./v1/modelsstill lists exactly the fiveimagegen-*ids and notcomfyui-image, which is whatunlisted: trueis for.Memory
--cache-nonedisables the intermediate node-result cache, not the model cache — worth stating because the first version of the comment here claimed otherwise. It is set because every request is a fresh graph, so the cache buys nothing and costs headroom. Measured after the first renders: 3 GB resident, 43 GB page cache — the GGUFs come off reclaimable page cache, not a Python-side cache that needs bounding. Peak during the Qwen-Image-Edit run: 20.9 GB VRAM, 11 GB RSS.Still open before #5
imagegen-qwen-image-editon sd-server while the other four move.A/B results — and a measurement trap worth recording
The trap
The first head-to-head made ComfyUI look 5x slower on qwen-image (20s vs 111s). It was not slower. sd-server rendered 512×512 and the shim rendered 1024×1024 — 4x the pixels. Neither harness sent an explicit size.
That is not just a harness bug, it is a behaviour trap in the migration: mort's
llamaswap_generate_imagedocumentssizeas "omit for the model's default", and majordomo'sparseSize("")sends no width/height at all — so whatever the engine defaults to is the contract for every size-less generation. sd.cpp defaults to 512. The shim was defaulting to 1024, which would have silently quadrupled the pixels and the latency of those calls the moment the aliases flipped in #5, with nothing in the diff saying so. Fixed inbf75dcb:DEFAULT_SIZE = 512.Separately worth deciding: 512 is a poor default for these models, all of which are trained at 1024, so a size-less Flux render today is worse than it needs to be. That is a real finding but a product decision — it belongs in the tool description on mort's side, not smuggled into an engine swap.
Warm, same model, repeated (1024×1024, the number that decides #5)
ComfyUI is ~20% faster warm. Its cold call is ~13s more expensive because a Python interpreter plus custom-node import costs more to boot than an sd-server binary — and every call in the alternating A/B below paid that boot, which is what made ComfyUI look bad.
Cold-to-cold, alternating engines (1024×1024, seed 4242, both swapping every call)
The phase-1 payoff, measured
Switching models inside a warm ComfyUI: schnell → dev = 26s, then dev → schnell = 8s. The container never restarts, so the second switch costs nothing but the DiT load, and going back to an already-touched model is free. Today the same switch is
docker stop+docker run+ engine boot + weight load every time.Output quality
Parity. Both engines render "NETHERSTORM" correctly on qwen-image (that model's whole point) and both flux-dev foxes are clean 1024² images. Same seed gives different images across engines — expected, the samplers and noise schedules differ — and seed reproducibility within an engine was verified byte-identical.
The one real regression: qwen-image is ~2x slower on ComfyUI
~106s warm vs ~55s. Cause is almost certainly
ComfyUI-GGUFdequantizing Q5_K_M weights on the fly in the forward pass, where sd.cpp has native GGUF kernels. Flux is barely affected because Q8_0 is nearly trivial to dequantize; a Q5_K_M 20B-class model at 40 steps is not.Recommended experiment before #5 (not run — phase 0's job was to surface this, and a 20 GB download plus another test cycle is phase-1 prep): swap the Qwen-Image DiT for
Comfy-Org/Qwen-Image_ComfyUI→split_files/diffusion_models/qwen_image_fp8_e4m3fn.safetensors(20.4 GB). fp8 is native on Ada so there is no dequant step at all. VRAM goes 14.9 → 20.4 GB, which fits the 4090 given ComfyUI offloads the text encoder after encoding, but it is tight and wants measuring. Do not use the nvfp4 build — emulated on Ada.If that does not close the gap, the fallback is simply to leave
imagegen-qwen-imageandimagegen-qwen-image-editon sd-server and alias only the three Flux models in #5. The alias mechanism is per-model, so a partial flip is expressible.Phase 0 status: done
Built, deployed, all five models rendering correctly through both the shim directly and through llama-swap's
/upstreamroute;cmd/cmdStop/checkEndpoint/group eviction all verified;unlisted: trueconfirmed keeping it out of/v1/models. steveternet atbf75dcb, host repo fast-forwarded, image rebuilt and byte-identical to the committed shim.