Part of #3. Orthogonal to the ComfyUI work — depends on nothing, can land any time.
What exists today
FIFO already has a priority queue. routing.scheduler.settings.fifo.priority is a map[modelID]int (internal/config/config.go:173, validated at internal/config/load.go:372), and FIFO.enqueue() (internal/router/scheduler/fifo.go:318) inserts a request just before the first queued item of strictly lower priority — so higher priority is serviced first while equal-priority requests keep arrival order.
It is keyed on model, not request. That is the wrong grain: the same model serves both a user typing in Discord and a background batch job, and they should not rank the same.
What to add
A per-request priority, carried on an X-Priority: <int> header, that takes effect only when the request has to wait.
Explicitly NOT preemption
Decided 2026-08-05: an in-progress low-priority job keeps the machine until it finishes. Priority answers exactly one question — of the requests currently waiting, which do we pick up next? No killing in-flight work, no requeueing, no draining a running render to let a small job in. This is both the desired semantics and by far the smaller change.
Sketch
ReqContextData (internal/shared/http.go:26) gains Priority int, parsed from the X-Priority header in FetchContext. Absent or unparseable → 0. Parse it in both branches — the /upstream/<model>/... path and the body-model path — so /upstream callers can set it too.
scheduler.HandlerReq (internal/router/scheduler/scheduler.go) gains Priority int; baseRouter.ServeHTTP (internal/router/base.go:422) populates it from data.Priority.
FIFO.enqueue() ranks on max(s.cfg.Priority[req.Model], req.Priority) instead of the config value alone. Model priority becomes a floor, not the whole story — an existing config that pins a model high keeps working unchanged.
That is the entire behavioural change. drainQueue() already walks s.queued in order and preserves relative order for items that stay blocked, so the ordering established at enqueue survives.
Things that correctly do NOT change
Fast-path requests (model ready, nothing to evict) never enqueue, so priority is irrelevant to them — right answer.
Requests joining an in-flight swap for the same model land in sw.waiters, not the queue. Same model, already loading; no ranking to do.
The serial scheduler ignores groups entirely and is not used in production here. Leave it alone, or return an explicit error if a priority is set while serial is active rather than silently dropping it.
mort side
Map the three tiers onto integers and send the header from majordomo's llama-swap provider:
tier
value
elevated
10
normal
0 (omit the header)
low
-10
Interactive Discord-triggered work is elevated; scheduled skills, standing queries, and long-video chain segments are low. Wire it as a plumbed-through request option rather than a global, so a single agent run can mark individual calls.
Tests
Two queued requests for different models, low enqueued first, high enqueued second → high is granted first
Equal priority preserves arrival order (existing FIFO guarantee must not regress)
A running low-priority job is not interrupted by an arriving high-priority request — it finishes, and the high-priority one goes first out of the queue afterwards
Config model-priority still works with no header present
max() semantics: a header of -10 on a model configured at +5 still ranks +5
Watch
broadcastQueuePositions(s.queued) reports queue position to waiters. With priorities in play a caller's position can now go up as higher-priority work arrives. Check nothing downstream assumes position is monotonically decreasing.
Part of #3. **Orthogonal to the ComfyUI work** — depends on nothing, can land any time.
## What exists today
`FIFO` already has a priority queue. `routing.scheduler.settings.fifo.priority` is a `map[modelID]int` (`internal/config/config.go:173`, validated at `internal/config/load.go:372`), and `FIFO.enqueue()` (`internal/router/scheduler/fifo.go:318`) inserts a request just before the first queued item of strictly lower priority — so higher priority is serviced first while equal-priority requests keep arrival order.
It is keyed on **model**, not **request**. That is the wrong grain: the same model serves both a user typing in Discord and a background batch job, and they should not rank the same.
## What to add
A per-request priority, carried on an `X-Priority: <int>` header, that takes effect **only when the request has to wait**.
### Explicitly NOT preemption
Decided 2026-08-05: an in-progress low-priority job keeps the machine until it finishes. Priority answers exactly one question — *of the requests currently waiting, which do we pick up next?* No killing in-flight work, no requeueing, no draining a running render to let a small job in. This is both the desired semantics and by far the smaller change.
### Sketch
1. `ReqContextData` (`internal/shared/http.go:26`) gains `Priority int`, parsed from the `X-Priority` header in `FetchContext`. Absent or unparseable → 0. Parse it in both branches — the `/upstream/<model>/...` path and the body-model path — so `/upstream` callers can set it too.
2. `scheduler.HandlerReq` (`internal/router/scheduler/scheduler.go`) gains `Priority int`; `baseRouter.ServeHTTP` (`internal/router/base.go:422`) populates it from `data.Priority`.
3. `FIFO.enqueue()` ranks on `max(s.cfg.Priority[req.Model], req.Priority)` instead of the config value alone. Model priority becomes a floor, not the whole story — an existing config that pins a model high keeps working unchanged.
That is the entire behavioural change. `drainQueue()` already walks `s.queued` in order and preserves relative order for items that stay blocked, so the ordering established at enqueue survives.
### Things that correctly do NOT change
- Fast-path requests (model ready, nothing to evict) never enqueue, so priority is irrelevant to them — right answer.
- Requests joining an in-flight swap for the *same* model land in `sw.waiters`, not the queue. Same model, already loading; no ranking to do.
- The `serial` scheduler ignores groups entirely and is not used in production here. Leave it alone, or return an explicit error if a priority is set while serial is active rather than silently dropping it.
## mort side
Map the three tiers onto integers and send the header from majordomo's llama-swap provider:
| tier | value |
|---|---|
| elevated | 10 |
| normal | 0 (omit the header) |
| low | -10 |
Interactive Discord-triggered work is elevated; scheduled skills, standing queries, and long-video chain segments are low. Wire it as a plumbed-through request option rather than a global, so a single agent run can mark individual calls.
## Tests
- Two queued requests for different models, low enqueued first, high enqueued second → high is granted first
- Equal priority preserves arrival order (existing FIFO guarantee must not regress)
- A running low-priority job is **not** interrupted by an arriving high-priority request — it finishes, and the high-priority one goes first out of the queue afterwards
- Config model-priority still works with no header present
- `max()` semantics: a header of -10 on a model configured at +5 still ranks +5
## Watch
`broadcastQueuePositions(s.queued)` reports queue position to waiters. With priorities in play a caller's position can now go **up** as higher-priority work arrives. Check nothing downstream assumes position is monotonically decreasing.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Part of #3. Orthogonal to the ComfyUI work — depends on nothing, can land any time.
What exists today
FIFOalready has a priority queue.routing.scheduler.settings.fifo.priorityis amap[modelID]int(internal/config/config.go:173, validated atinternal/config/load.go:372), andFIFO.enqueue()(internal/router/scheduler/fifo.go:318) inserts a request just before the first queued item of strictly lower priority — so higher priority is serviced first while equal-priority requests keep arrival order.It is keyed on model, not request. That is the wrong grain: the same model serves both a user typing in Discord and a background batch job, and they should not rank the same.
What to add
A per-request priority, carried on an
X-Priority: <int>header, that takes effect only when the request has to wait.Explicitly NOT preemption
Decided 2026-08-05: an in-progress low-priority job keeps the machine until it finishes. Priority answers exactly one question — of the requests currently waiting, which do we pick up next? No killing in-flight work, no requeueing, no draining a running render to let a small job in. This is both the desired semantics and by far the smaller change.
Sketch
ReqContextData(internal/shared/http.go:26) gainsPriority int, parsed from theX-Priorityheader inFetchContext. Absent or unparseable → 0. Parse it in both branches — the/upstream/<model>/...path and the body-model path — so/upstreamcallers can set it too.scheduler.HandlerReq(internal/router/scheduler/scheduler.go) gainsPriority int;baseRouter.ServeHTTP(internal/router/base.go:422) populates it fromdata.Priority.FIFO.enqueue()ranks onmax(s.cfg.Priority[req.Model], req.Priority)instead of the config value alone. Model priority becomes a floor, not the whole story — an existing config that pins a model high keeps working unchanged.That is the entire behavioural change.
drainQueue()already walkss.queuedin order and preserves relative order for items that stay blocked, so the ordering established at enqueue survives.Things that correctly do NOT change
sw.waiters, not the queue. Same model, already loading; no ranking to do.serialscheduler ignores groups entirely and is not used in production here. Leave it alone, or return an explicit error if a priority is set while serial is active rather than silently dropping it.mort side
Map the three tiers onto integers and send the header from majordomo's llama-swap provider:
Interactive Discord-triggered work is elevated; scheduled skills, standing queries, and long-video chain segments are low. Wire it as a plumbed-through request option rather than a global, so a single agent run can mark individual calls.
Tests
max()semantics: a header of -10 on a model configured at +5 still ranks +5Watch
broadcastQueuePositions(s.queued)reports queue position to waiters. With priorities in play a caller's position can now go up as higher-priority work arrives. Check nothing downstream assumes position is monotonically decreasing.