internal/router: priority queues so batch jobs yield to interactive requests

The GPU is a size-1 resource, so a single long job monopolises the box for its
whole duration and every interactive request queues behind it. Callers can now
declare intent with an X-LlamaSwap-Priority header and the serial scheduler
dispatches by score instead of by arrival.

- X-LlamaSwap-Priority: signed integer, 0 default, absent/unparseable means 0.
  interactive/normal/batch aliases resolve to +100/0/-100. Values are not
  clamped: the caller composes band and any per-user offset itself.
- serial dispatch score = priority + swap affinity + aging. Bands sit 100 apart
  so a small caller offset orders work inside a band without crossing one;
  aging is unbounded so low-priority work cannot starve.
- routing.scheduler.settings.serial.{agingDivisor,swapAffinityBonus}, defaulting
  to 60s/point and +10. swapAffinityBonus is capped at 99 so it can never
  promote a request into the next band.
- fifo adds the header to its per-model priority, so the header is not silently
  ignored under that scheduler.
- /metrics exports per-band queue depth, oldest wait and dispatch counts, plus
  counters for how often aging or swap affinity changed the pick. Each request
  records its priority, band, score and queue wait in the activity log.

Note swapAffinityBonus defaults to 10, so equal-priority requests for the
already-loaded model now run before older requests that need a swap. Set it to
0 for the previous strict arrival order.

fixes #9

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01WUyhZBgv8BBCC5MduX88gE

config.example.yaml

@@ -559,23 +559,62 @@ routing:
   # scheduler: how queued requests are ordered and run.
   # - optional, default on this fork: "serial"
   # - valid values:
-  #   - "serial": strict one-model-at-a-time. Requests run in exact arrival
-  #     order; only one request runs at a time; switching to a different model
-  #     evicts every other running model first so a single model occupies memory
-  #     at a time. This ignores group/matrix co-residency entirely. The "fifo"
-  #     settings below (priority) do not apply.
+  #   - "serial": strict one-model-at-a-time. Only one request runs at a time;
+  #     switching to a different model evicts every other running model first so
+  #     a single model occupies memory at a time. This ignores group/matrix
+  #     co-residency entirely. The "fifo" settings below do not apply.
   #   - "fifo": throughput-oriented. Same-model requests are batched to reduce
   #     swaps and a model serves up to its concurrencyLimit in parallel; models
   #     in non-exclusive groups can run concurrently. Requests may be reordered.
+  #
+  # Callers set per-request priority with the X-LlamaSwap-Priority header
+  # (a signed integer; "interactive" = 100, "normal" = 0, "batch" = -100;
+  # absent or unparseable means 0). Both schedulers honour it. Scheduling is
+  # non-preemptive: priority applies when a job finishes, so an interactive
+  # request can still wait up to one full job duration.
   scheduler:
     use: serial
     settings:
+      # serial settings only apply when use: serial
+      #
+      # At each dispatch the serial scheduler runs the highest scoring waiting
+      # request, where:
+      #
+      #   score = X-LlamaSwap-Priority + swap affinity + aging
+      #
+      # Priority bands sit 100 apart, which leaves room for a caller to add a
+      # small per-user offset (a subscription tier, say) without crossing a
+      # band: a "max member" batch job at -98 beats other batch work but still
+      # loses to every normal request at 0.
+      serial:
+        # agingDivisor: seconds a request must wait to gain one priority point
+        # - optional, default: 60 (one point per minute)
+        # - 0 disables aging
+        # - unbounded on purpose: aging is the only term allowed to promote a
+        #   request across a band, which is what stops low-priority work from
+        #   starving. At 60, a batch job at -100 overtakes normal traffic after
+        #   ~100 minutes of waiting.
+        agingDivisor: 60
+
+        # swapAffinityBonus: bonus for a request that needs no model swap
+        # - optional, default: 10
+        # - must be 0..99, so it can only break near-ties and can never promote
+        #   a request into the next band
+        # - 0 disables it, giving strict priority-then-arrival order
+        # - cold loads are expensive, so this keeps a run of same-model requests
+        #   together instead of forcing a reload on every dispatch. Note this
+        #   means equal-priority requests are NOT served in strict arrival
+        #   order; set it to 0 if you need that.
+        swapAffinityBonus: 10
+
       # fifo settings only apply when use: fifo
       fifo:
         # priority: a dictionary of model ID -> priority
         # - optional, default: empty dictionary
         # - models default to priority 0
         # - higher priority requests are serviced first in the queue
+        # - added to the caller's X-LlamaSwap-Priority, so a model priority of
+        #   10 and a header of "batch" (-100) queue at -90
         priority:
           A: 10
           B: 5