internal/router: priority queues so batch jobs yield to interactive requests

The GPU is a size-1 resource, so a single long job monopolises the box for its
whole duration and every interactive request queues behind it. Callers can now
declare intent with an X-LlamaSwap-Priority header and the serial scheduler
dispatches by score instead of by arrival.

- X-LlamaSwap-Priority: signed integer, 0 default, absent/unparseable means 0.
  interactive/normal/batch aliases resolve to +100/0/-100. Values are not
  clamped: the caller composes band and any per-user offset itself.
- serial dispatch score = priority + swap affinity + aging. Bands sit 100 apart
  so a small caller offset orders work inside a band without crossing one;
  aging is unbounded so low-priority work cannot starve.
- routing.scheduler.settings.serial.{agingDivisor,swapAffinityBonus}, defaulting
  to 60s/point and +10. swapAffinityBonus is capped at 99 so it can never
  promote a request into the next band.
- fifo adds the header to its per-model priority, so the header is not silently
  ignored under that scheduler.
- /metrics exports per-band queue depth, oldest wait and dispatch counts, plus
  counters for how often aging or swap affinity changed the pick. Each request
  records its priority, band, score and queue wait in the activity log.

Note swapAffinityBonus defaults to 10, so equal-priority requests for the
already-loaded model now run before older requests that need a swap. Set it to
0 for the previous strict arrival order.

fixes #9

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01WUyhZBgv8BBCC5MduX88gE

internal/router/scheduler/scheduler.go

@@ -95,7 +95,7 @@ type Effects interface {
 // New returns a Scheduler selected by conf.Routing.Scheduler.Use, configured from
 // conf and bound to the given planner and effects. Supported values are "fifo"
 // (throughput-oriented, batches same-model requests) and "serial" (strict
-// one-model-at-a-time, exact arrival order).
+// one-model-at-a-time, highest-priority-first).
 //
 // The deployment default is applied by config loading (LoadConfig sets Use to
 // "serial" when unset). The "" fallback here is the library default and remains
@@ -110,7 +110,7 @@ func New(conf config.Config, name string, logger *logmon.Monitor, planner Swappe
 		return NewFIFO(name, logger, planner, conf.Routing.Scheduler.Settings.Fifo, conf.Models, eff), nil
 	case "serial":
 		// Serial ignores the group planner: it always evicts every other model.
-		return NewSerial(name, logger, eff), nil
+		return NewSerial(name, logger, conf.Routing.Scheduler.Settings.Serial, eff), nil
 	default:
 		return nil, fmt.Errorf("unsupported scheduler type: %q", use)
 	}
@@ -118,12 +118,50 @@ func New(conf config.Config, name string, logger *logmon.Monitor, planner Swappe
 
 // HandlerReq is one in-flight ServeHTTP request waiting for a routing decision.
 type HandlerReq struct {
-	Model      string
+	Model string
+	// Priority is the caller's scheduling priority from the
+	// X-LlamaSwap-Priority header. Higher is more urgent, 0 is normal, and
+	// negative is batch work. Schedulers use it to order their queues.
+	Priority   int
 	Ctx        context.Context
 	Respond    chan HandlerResp
 	PositionCh chan int
 }
 
+// BandStats is the queue state for one priority band.
+type BandStats struct {
+	// Depth is how many requests are waiting in this band.
+	Depth int
+	// OldestWait is how long the longest-waiting request in this band has
+	// been queued, or 0 when the band is empty.
+	OldestWait time.Duration
+	// Dispatched counts requests in this band handed off since start.
+	Dispatched uint64
+}
+
+// QueueStats is a point-in-time snapshot of a scheduler's queue, for metrics.
+// The reorder counters answer "did the scoring function actually change
+// anything?" — without them, tuning aging and swap affinity is guesswork.
+type QueueStats struct {
+	// Bands is keyed by shared.PriorityBand and always carries an entry for
+	// every band in shared.PriorityBands, so metric series stay stable.
+	Bands map[string]BandStats
+	// AgingReorders counts dispatches where the aging term changed which
+	// request was picked.
+	AgingReorders uint64
+	// AffinityReorders counts dispatches where the swap-affinity term changed
+	// which request was picked.
+	AffinityReorders uint64
+}
+
+// StatsReporter is implemented by schedulers that can report queue statistics.
+// It is optional: callers type-assert and skip schedulers that do not provide
+// it. Unlike the Scheduler methods, QueueStats is called from arbitrary
+// goroutines (the /metrics handler) and must be safe for concurrent use.
+type StatsReporter interface {
+	QueueStats() QueueStats
+}
+
 // HandlerResp is the routing decision returned to a HandlerReq's caller: either
 // a handler to serve with, or an error.
 type HandlerResp struct {