Compare commits

..
Author SHA1 Message Date
steve c87893ba6f Merge pull request #10 from feat/request-priority
Build CUDA image (fork) / build (push) Successful in 1m25s
internal/router: priority queues so batch jobs yield to interactive requests

Callers declare intent with X-LlamaSwap-Priority (signed int, 0 default,
interactive/normal/batch aliases at +100/0/-100). serial dispatches by
score = priority + swap affinity + aging; fifo adds the header to the
model's configured priority so it is not silently ignored there.

Closes #9. Consumer side: steve/mort#1576.
2026-08-08 01:23:14 +00:00

Diff Content Not Available