internal/router/scheduler: add serial scheduler, default on this fork

Add a strict one-model-at-a-time scheduler. Requests run in exact arrival order; at most one runs at a time; switching to a different model evicts every other running model first so a single model occupies memory at a time. Unlike fifo it never reorders or batches same-model requests, and it ignores group/matrix co-residency entirely, making the single-model guarantee a property of the scheduler rather than the config. - new Serial scheduler implementing the Scheduler interface - register "serial" in scheduler.New; default routing.scheduler.use to "serial" at config load (fifo still selectable for upstream behavior) - update config schema, example config, and config defaults tests Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 12:17:32 -04:00
parent 0a25b3bd31
commit 542b79dacf
9 changed files with 683 additions and 15 deletions
@@ -601,10 +601,11 @@
                        "use": {
                            "type": "string",
                            "enum": [
+                                "serial",
                                "fifo"
                            ],
-                            "default": "fifo",
-                            "description": "Scheduler to use. Only 'fifo' is currently supported."
+                            "default": "serial",
+                            "description": "Scheduler to use. 'serial' (default on this fork): strict one-model-at-a-time, requests run in exact arrival order, switching models evicts every other model first. 'fifo': throughput-oriented, batches same-model requests and allows parallel/co-resident models."
                        },
                        "settings": {
                            "type": "object",