go.mod,ui/package-lock.json: dependency and security updates (#418 )

proxy: add support for anthropic v1/messages api (#417 )
* proxy: add support for anthropic v1/messages api * proxy: restrict loading message to /v1/chat/completions
2025-11-29 22:27:22 -08:00 · 2025-11-29 22:09:07 -08:00 · 2025-11-29 20:59:15 -08:00 · 2025-11-25 10:44:13 -08:00 · 2025-11-24 21:39:56 -08:00 · 2025-11-17 10:43:47 -08:00
29 changed files with 1373 additions and 204 deletions
@@ -0,0 +1,41 @@
 name: Validate JSON Schema
 on:
  pull_request:
    paths:
      - "config-schema.json"
  push:
    branches:
      - main
    paths:
      - "config-schema.json"
  workflow_dispatch:
 jobs:
  validate-schema:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
      - name: Validate JSON Schema
        run: |
          # Check if the file is valid JSON
          if ! jq empty config-schema.json 2>/dev/null; then
            echo "Error: config-schema.json is not valid JSON"
            exit 1
          fi
          # Validate that it's a valid JSON Schema
          # Check for required $schema field
          if ! jq -e '."$schema"' config-schema.json > /dev/null; then
            echo "Warning: config-schema.json should have a \$schema field"
          fi
          # Check that it has either properties or definitions
          if ! jq -e '.properties or .definitions or ."$defs"' config-schema.json > /dev/null; then
            echo "Warning: JSON Schema should contain properties, definitions, or \$defs"
          fi
          echo "✓ config-schema.json is valid"
@@ -11,7 +11,7 @@ llama-swap is a light weight, transparent proxy server that provides automatic m
 ## Testing
- `make test-dev` - Use this when making iterative changes. Runs `go test` and `staticcheck`. Fix any static checking errors.
+- `make test-dev` - Use this when making iterative changes. Runs `go test` and `staticcheck`. Fix any static checking errors. Use this only when changes are made to any code under the `proxy/` directory
 - `make test-all` - runs at the end before completing work. Includes long running concurrency tests.
 ## Workflow Tasks
@@ -42,7 +42,8 @@ Built in Go for performance and simplicity, llama-swap has zero dependencies and
 llama-swap includes a real time web interface for monitoring logs and controlling models:
-<img width="1360" height="963" alt="image" src="https://github.com/user-attachments/assets/adef4a8e-de0b-49db-885a-8f6dedae6799" />
+<img width="1164" height="745" alt="image" src="https://github.com/user-attachments/assets/bacf3f9d-819f-430b-9ed2-1bfaa8d54579" />
 The Activity Page shows recent requests:
@@ -0,0 +1,64 @@
 <!DOCTYPE html>
 <html>
 <head>
 <meta charset="utf-8">
 <title>Loading...</title>
 <style>
 body {
    font-family: sans-serif;
    display: flex;
    justify-content: center;
    align-items: center;
    height: 100vh;
    margin: 0;
    background: #f5f5f5;
 }
 .loader {
    text-align: center;
 }
 .stats {
    font-size: 18px;
    color: #333;
    margin: 20px 0;
 }
 .stats-label {
    color: #666;
    font-size: 14px;
 }
 </style>
 </head>
 <body>
 <div class="loader">
    <p>Waking up upstream server...</p>
    <div class="stats">
        <div><span class="stats-label">Time elapsed:</span> <span id="elapsed">0s</span></div>
        <div><span id="attempts">&nbsp;</span></div>
    </div>
 </div>
 <script>
 var startTime = Date.now();
 var attempts = 0;
 setInterval(function() {
    var elapsed = (Date.now() - startTime) / 1000;
    document.getElementById('elapsed').textContent = elapsed.toFixed(1) + 's';
 }, 100);
 // Check status every second
 setInterval(function() {
    attempts++;
    var dots = '.'.repeat((attempts % 10) || 10);
    document.getElementById('attempts').textContent = dots;
    fetch('/status')
        .then(function(r) { return r.text(); })
        .then(function(t) {
            if (t.indexOf('status: ready') !== -1) {
                location.reload();
            }
        })
        .catch(function() {});
 }, 1000);
 </script>
 </body>
 </html>
@@ -3,6 +3,7 @@ package main
 import (
 	"bufio"
 	"context"
 	_ "embed"
 	"errors"
 	"flag"
 	"fmt"
@@ -19,6 +20,9 @@ import (
 	"time"
 )
 //go:embed index.html
 var loadingPageHTML string
 var (
 	flagMac      = flag.String("mac", "", "mac address to send WoL packet to")
 	flagUpstream = flag.String("upstream", "", "upstream proxy address to send requests to")
@@ -230,6 +234,16 @@ func (p *proxyServer) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 		if err := sendMagicPacket(*flagMac); err != nil {
 			slog.Warn("failed to send magic WoL packet", "error", err)
 		}
 		// For root or UI path requests, return loading page with status polling
 		// the web page will do the polling and redirect when ready
 		if path == "/" || strings.HasPrefix(path, "/ui/") {
 			w.Header().Set("Content-Type", "text/html; charset=utf-8")
 			w.WriteHeader(http.StatusOK)
 			fmt.Fprint(w, loadingPageHTML)
 			return
 		}
 		ticker := time.NewTicker(250 * time.Millisecond)
 		timeout, cancel := context.WithTimeout(context.Background(), time.Duration(*flagTimeout)*time.Second)
 		defer cancel()
@@ -0,0 +1,278 @@
 {
    "$schema": "https://json-schema.org/draft-07/schema#",
    "$id": "llama-swap-config-schema.json",
    "title": "llama-swap configuration",
    "description": "Configuration file for llama-swap",
    "type": "object",
    "required": [
        "models"
    ],
    "definitions": {
        "macros": {
            "type": "object",
            "additionalProperties": {
                "oneOf": [
                    {
                        "type": "string",
                        "minLength": 0,
                        "maxLength": 1024
                    },
                    {
                        "type": "number"
                    },
                    {
                        "type": "boolean"
                    }
                ]
            },
            "propertyNames": {
                "type": "string",
                "minLength": 1,
                "maxLength": 64,
                "pattern": "^[a-zA-Z0-9_-]+$",
                "not": {
                    "enum": [
                        "PORT",
                        "MODEL_ID"
                    ]
                }
            },
            "default": {},
            "description": "A dictionary of string substitutions. Macros are reusable snippets used in model cmd, cmdStop, proxy, checkEndpoint, filters.stripParams. Macro names must be <64 chars, match ^[a-zA-Z0-9_-]+$, and not be PORT or MODEL_ID. Values can be string, number, or boolean. Macros can reference other macros defined before them."
        }
    },
    "properties": {
        "healthCheckTimeout": {
            "type": "integer",
            "minimum": 15,
            "default": 120,
            "description": "Number of seconds to wait for a model to be ready to serve requests."
        },
        "logLevel": {
            "type": "string",
            "enum": [
                "debug",
                "info",
                "warn",
                "error"
            ],
            "default": "info",
            "description": "Sets the logging value. Valid values: debug, info, warn, error."
        },
        "logTimeFormat": {
            "type": "string",
            "enum": [
                "",
                "ansic",
                "unixdate",
                "rubydate",
                "rfc822",
                "rfc822z",
                "rfc850",
                "rfc1123",
                "rfc1123z",
                "rfc3339",
                "rfc3339nano",
                "kitchen",
                "stamp",
                "stampmilli",
                "stampmicro",
                "stampnano"
            ],
            "default": "",
            "description": "Enables and sets the logging timestamp format. Valid values: \"\", \"ansic\", \"unixdate\", \"rubydate\", \"rfc822\", \"rfc822z\", \"rfc850\", \"rfc1123\", \"rfc1123z\", \"rfc3339\", \"rfc3339nano\", \"kitchen\", \"stamp\", \"stampmilli\", \"stampmicro\", and \"stampnano\". For more info, read: https://pkg.go.dev/time#pkg-constants"
        },
        "metricsMaxInMemory": {
            "type": "integer",
            "default": 1000,
            "description": "Maximum number of metrics to keep in memory. Controls how many metrics are stored before older ones are discarded."
        },
        "startPort": {
            "type": "integer",
            "default": 5800,
            "description": "Starting port number for the automatic ${PORT} macro. The ${PORT} macro is incremented for every model that uses it."
        },
        "sendLoadingState": {
            "type": "boolean",
            "default": false,
            "description": "Inject loading status updates into the reasoning field. When true, a stream of loading messages will be sent to the client."
        },
        "includeAliasesInList": {
            "type": "boolean",
            "default": false,
            "description": "Present aliases within the /v1/models OpenAI API listing. when true, model aliases will be output to the API model listing duplicating all fields except for Id so chat UIs can use the alias equivalent to the original."
        },
        "macros": {
            "$ref": "#/definitions/macros"
        },
        "models": {
            "type": "object",
            "description": "A dictionary of model configurations. Each key is a model's ID. Model settings have defaults if not defined. The model's ID is available as ${MODEL_ID}.",
            "additionalProperties": {
                "type": "object",
                "required": [
                    "cmd"
                ],
                "properties": {
                    "macros": {
                        "$ref": "#/definitions/macros"
                    },
                    "cmd": {
                        "type": "string",
                        "minLength": 1,
                        "description": "Command to run to start the inference server. Macros can be used. Comments allowed with |."
                    },
                    "cmdStop": {
                        "type": "string",
                        "default": "",
                        "description": "Command to run to stop the model gracefully. Uses ${PID} macro for upstream process id. If empty, default shutdown behavior is used."
                    },
                    "name": {
                        "type": "string",
                        "default": "",
                        "maxLength": 128,
                        "description": "Display name for the model. Used in v1/models API response."
                    },
                    "description": {
                        "type": "string",
                        "default": "",
                        "maxLength": 1024,
                        "description": "Description for the model. Used in v1/models API response."
                    },
                    "env": {
                        "type": "array",
                        "items": {
                            "type": "string",
                            "pattern": "^[A-Z_][A-Z0-9_]*=.*$"
                        },
                        "default": [],
                        "description": "Array of environment variables to inject into cmd's environment. Each value is a string in ENV_NAME=value format."
                    },
                    "proxy": {
                        "type": "string",
                        "default": "http://localhost:${PORT}",
                        "format": "uri",
                        "description": "URL where llama-swap routes API requests. If custom port is used in cmd, this must be set."
                    },
                    "aliases": {
                        "type": "array",
                        "items": {
                            "type": "string",
                            "minLength": 1
                        },
                        "default": [],
                        "description": "Alternative model names for this configuration. Must be unique globally."
                    },
                    "checkEndpoint": {
                        "type": "string",
                        "default": "/health",
                        "pattern": "^/.*$|^none$",
                        "description": "URL path to check if the server is ready. Use 'none' to skip health checking."
                    },
                    "ttl": {
                        "type": "integer",
                        "minimum": 0,
                        "default": 0,
                        "description": "Automatically unload the model after ttl seconds. 0 disables unloading. Must be >0 to enable."
                    },
                    "useModelName": {
                        "type": "string",
                        "default": "",
                        "description": "Override the model name sent to upstream server. Useful if upstream expects a different name."
                    },
                    "filters": {
                        "type": "object",
                        "properties": {
                            "stripParams": {
                                "type": "string",
                                "default": "",
                                "pattern": "^[a-zA-Z0-9_, ]*$",
                                "description": "Comma separated list of parameters to remove from the request. Used for server-side enforcement of sampling parameters."
                            }
                        },
                        "additionalProperties": false,
                        "default": {},
                        "description": "Dictionary of filter settings. Only stripParams is supported."
                    },
                    "metadata": {
                        "type": "object",
                        "additionalProperties": true,
                        "default": {},
                        "description": "Dictionary of arbitrary values included in /v1/models. Can contain complex types. Only passed through in /v1/models responses."
                    },
                    "concurrencyLimit": {
                        "type": "integer",
                        "minimum": 0,
                        "default": 0,
                        "description": "Overrides allowed number of active parallel requests to a model. 0 uses internal default of 10. >0 overrides default. Requests exceeding limit get HTTP 429."
                    },
                    "sendLoadingState": {
                        "type": "boolean",
                        "description": "Overrides the global sendLoadingState for this model. Ommitting this property will use the global setting."
                    },
                    "unlisted": {
                        "type": "boolean",
                        "default": false,
                        "description": "If true the model will not show up in /v1/models responses. It can still be used as normal in API requests."
                    }
                }
            }
        },
        "groups": {
            "type": "object",
            "additionalProperties": {
                "type": "object",
                "required": [
                    "members"
                ],
                "properties": {
                    "swap": {
                        "type": "boolean",
                        "default": true,
                        "description": "Controls model swapping behaviour within the group. True: only one model runs at a time. False: all models can run together."
                    },
                    "exclusive": {
                        "type": "boolean",
                        "default": true,
                        "description": "Controls how the group affects other groups. True: causes all other groups to unload when this group runs a model. False: does not affect other groups."
                    },
                    "persistent": {
                        "type": "boolean",
                        "default": false,
                        "description": "Prevents other groups from unloading the models in this group. Does not affect individual model behaviour."
                    },
                    "members": {
                        "type": "array",
                        "items": {
                            "type": "string"
                        },
                        "description": "Array of model IDs that are members of this group. Model IDs must be defined in models."
                    }
                }
            },
            "description": "A dictionary of group settings. Provides advanced controls over model swapping behaviour. Model IDs must be defined in models. A model can only be a member of one group. Behaviour controlled via swap, exclusive, persistent."
        },
        "hooks": {
            "type": "object",
            "properties": {
                "on_startup": {
                    "type": "object",
                    "properties": {
                        "preload": {
                            "type": "array",
                            "items": {
                                "type": "string"
                            },
                            "default": [],
                            "description": "List of model IDs to load on startup. Model names must match keys in models. When preloading multiple models, define a group to prevent swapping."
                        }
                    },
                    "additionalProperties": false,
                    "description": "Actions to perform on startup. Only supported action is preload."
                }
            },
            "additionalProperties": false,
            "description": "A dictionary of event triggers and actions. Only supported hook is on_startup."
        }
    }
 }
@@ -1,3 +1,6 @@
 # add this modeline for validation in vscode
 # yaml-language-server: $schema=https://raw.githubusercontent.com/mostlygeek/llama-swap/refs/heads/main/config-schema.json
 #
 # llama-swap YAML configuration example
 # -------------------------------------
 #
@@ -23,6 +26,14 @@ healthCheckTimeout: 500
 # - Valid log levels: debug, info, warn, error
 logLevel: info
 # logTimeFormat: enables and sets the logging timestamp format
 # - optional, default (disabled): ""
 # - Valid values: "", "ansic", "unixdate", "rubydate", "rfc822", "rfc822z",
 #   "rfc850", "rfc1123", "rfc1123z", "rfc3339", "rfc3339nano", "kitchen",
 #   "stamp", "stampmilli", "stampmicro", and "stampnano".
 # - For more info, read: https://pkg.go.dev/time#pkg-constants
 logTimeFormat: ""
 # metricsMaxInMemory: maximum number of metrics to keep in memory
 # - optional, default: 1000
 # - controls how many metrics are stored in memory before older ones are discarded
@@ -35,6 +46,20 @@ metricsMaxInMemory: 1000
 # - it is automatically incremented for every model that uses it
 startPort: 10001
 # sendLoadingState: inject loading status updates into the reasoning (thinking)
 # field
 # - optional, default: false
 # - when true, a stream of loading messages will be sent to the client in the
 #   reasoning field so chat UIs can show that loading is in progress.
 # - see #366 for more details
 sendLoadingState: true
 # includeAliasesInList: present aliases within the /v1/models OpenAI API listing
 # - optional, default: false
 # - when true, model aliases will be output to the API model listing duplicating
 #   all fields except for Id so chat UIs can use the alias equivalent to the original.
 includeAliasesInList: false
 # macros: a dictionary of string substitutions
 # - optional, default: empty dictionary
 # - macros are reusable snippets
@@ -64,7 +89,6 @@ macros:
 # - the model's ID is available in the ${MODEL_ID} macro, also available in macros defined above
 # - below are examples of the all the settings a model can have
 models:
  # keys are the model names used in API requests
  "llama":
    # macros: a dictionary of string substitutions specific to this model
@@ -184,6 +208,10 @@ models:
    # - recommended to be omitted and the default used
    concurrencyLimit: 0
    # sendLoadingState: overrides the global sendLoadingState setting for this model
    # - optional, default: undefined (use global setting)
    sendLoadingState: false
  # Unlisted model example:
  "qwen-unlisted":
    # unlisted: boolean, true or false
@@ -286,10 +314,10 @@ hooks:
  # - optional, default: empty dictionary
  # - the only supported action is preload
  on_startup:
-        # preload: a list of model ids to load on startup
+    # preload: a list of model ids to load on startup
-        # - optional, default: empty list
+    # - optional, default: empty list
-        # - model names must match keys in the models sections
+    # - model names must match keys in the models sections
-        # - when preloading multiple models at once, define a group
+    # - when preloading multiple models at once, define a group
-        #   otherwise models will be loaded and swapped out
+    #   otherwise models will be loaded and swapped out
    preload:
-      - "llama"
+      - "llama"
@@ -20,36 +20,60 @@ if [[ -z "$GITHUB_TOKEN" ]]; then
  exit 1
 fi
 # Set llama.cpp base image, customizable using the BASE_LLAMACPP_IMAGE environment
 # variable, this permits testing with forked llama.cpp repositories
 BASE_IMAGE=${BASE_LLAMACPP_IMAGE:-ghcr.io/ggml-org/llama.cpp}
 # Set llama-swap repository, automatically uses GITHUB_REPOSITORY variable
 # to enable easy container builds on forked repos
 LS_REPO=${GITHUB_REPOSITORY:-mostlygeek/llama-swap}
 # the most recent llama-swap tag
 # have to strip out the 'v' due to .tar.gz file naming
-LS_VER=$(curl -s https://api.github.com/repos/mostlygeek/llama-swap/releases/latest | jq -r .tag_name | sed 's/v//')
+LS_VER=$(curl -s https://api.github.com/repos/${LS_REPO}/releases/latest | jq -r .tag_name | sed 's/v//')
 if [ "$ARCH" == "cpu" ]; then
-    # cpu only containers just use the latest available
+    # cpu only containers just use the server tag
-    CONTAINER_LATEST="ghcr.io/mostlygeek/llama-swap:cpu"
+    LCPP_TAG=$(curl -s -H "Authorization: Bearer $GITHUB_TOKEN" \
-    echo "Building ${CONTAINER_LATEST} $LS_VER"
+        "https://api.github.com/users/ggml-org/packages/container/llama.cpp/versions" \
-    docker build -f llama-swap.Containerfile --build-arg BASE_TAG=server --build-arg LS_VER=${LS_VER} -t ${CONTAINER_LATEST} .
+        | jq -r '.[] | select(.metadata.container.tags[] | startswith("server")) | .metadata.container.tags[]' \
-    if [ "$PUSH_IMAGES" == "true" ]; then
+        | sort -r | head -n1 | awk -F '-' '{print $3}')
-      docker push ${CONTAINER_LATEST}
+    BASE_TAG=server-${LCPP_TAG}
    fi
 else
    LCPP_TAG=$(curl -s -H "Authorization: Bearer $GITHUB_TOKEN" \
        "https://api.github.com/users/ggml-org/packages/container/llama.cpp/versions" \
        | jq -r --arg arch "$ARCH" '.[] | select(.metadata.container.tags[] | startswith("server-\($arch)")) | .metadata.container.tags[]' \
        | sort -r | head -n1 | awk -F '-' '{print $3}')
    BASE_TAG=server-${ARCH}-${LCPP_TAG}
 fi
-    # Abort if LCPP_TAG is empty.
+# Abort if LCPP_TAG is empty.
-    if [[ -z "$LCPP_TAG" ]]; then
+if [[ -z "$LCPP_TAG" ]]; then
-        echo "Abort: Could not find llama-server container for arch: $ARCH"
+    echo "Abort: Could not find llama-server container for arch: $ARCH"
-        exit 1
+    exit 1
-    fi
+fi
-    CONTAINER_TAG="ghcr.io/mostlygeek/llama-swap:v${LS_VER}-${ARCH}-${LCPP_TAG}"
+for CONTAINER_TYPE in non-root root; do
-    CONTAINER_LATEST="ghcr.io/mostlygeek/llama-swap:${ARCH}"
+  CONTAINER_TAG="ghcr.io/${LS_REPO}:v${LS_VER}-${ARCH}-${LCPP_TAG}"
-    echo "Building ${CONTAINER_TAG} $LS_VER"
+  CONTAINER_LATEST="ghcr.io/${LS_REPO}:${ARCH}"
-    docker build -f llama-swap.Containerfile --build-arg BASE_TAG=server-${ARCH}-${LCPP_TAG} --build-arg LS_VER=${LS_VER} -t ${CONTAINER_TAG} -t ${CONTAINER_LATEST} .
+  USER_UID=0
-    if [ "$PUSH_IMAGES" == "true" ]; then
+  USER_GID=0
-      docker push ${CONTAINER_TAG}
+  USER_HOME=/root
-      docker push ${CONTAINER_LATEST}
+
-    fi
+  if [ "$CONTAINER_TYPE" == "non-root" ]; then
-fi
+    CONTAINER_TAG="${CONTAINER_TAG}-non-root"
    CONTAINER_LATEST="${CONTAINER_LATEST}-non-root"
    USER_UID=10001
    USER_GID=10001
    USER_HOME=/app
  fi
  echo "Building $CONTAINER_TYPE $CONTAINER_TAG $LS_VER"
  docker build -f llama-swap.Containerfile --build-arg BASE_TAG=${BASE_TAG} --build-arg LS_VER=${LS_VER} --build-arg UID=${USER_UID} \
    --build-arg LS_REPO=${LS_REPO} --build-arg GID=${USER_GID} --build-arg USER_HOME=${USER_HOME} -t ${CONTAINER_TAG} -t ${CONTAINER_LATEST} \
    --build-arg BASE_IMAGE=${BASE_IMAGE} .
  if [ "$PUSH_IMAGES" == "true" ]; then
    docker push ${CONTAINER_TAG}
    docker push ${CONTAINER_LATEST}
  fi
 done
@@ -1,16 +1,40 @@
 ARG BASE_IMAGE=ghcr.io/ggml-org/llama.cpp
 ARG BASE_TAG=server-cuda
-FROM ghcr.io/ggml-org/llama.cpp:${BASE_TAG}
+FROM ${BASE_IMAGE}:${BASE_TAG}
 # has to be after the FROM
-ARG LS_VER=89
+ARG LS_VER=170
 ARG LS_REPO=mostlygeek/llama-swap
 # Set default UID/GID arguments
 ARG UID=10001
 ARG GID=10001
 ARG USER_HOME=/app
 # Add user/group
 ENV HOME=$USER_HOME
 RUN if [ $UID -ne 0 ]; then \
      if [ $GID -ne 0 ]; then \
        groupadd --system --gid $GID app; \
      fi; \
      useradd --system --uid $UID --gid $GID \
      --home $USER_HOME app; \
    fi
 # Handle paths
 RUN mkdir --parents $HOME /app
 RUN chown --recursive $UID:$GID $HOME /app
 # Switch user
 USER $UID:$GID
 WORKDIR /app
 RUN \
-    curl -LO https://github.com/mostlygeek/llama-swap/releases/download/v"${LS_VER}"/llama-swap_"${LS_VER}"_linux_amd64.tar.gz && \
+    curl -LO "https://github.com/${LS_REPO}/releases/download/v${LS_VER}/llama-swap_${LS_VER}_linux_amd64.tar.gz" && \
-    tar -zxf llama-swap_"${LS_VER}"_linux_amd64.tar.gz && \
+    tar -zxf "llama-swap_${LS_VER}_linux_amd64.tar.gz" && \
-    rm llama-swap_"${LS_VER}"_linux_amd64.tar.gz
+    rm "llama-swap_${LS_VER}_linux_amd64.tar.gz"
-COPY config.example.yaml /app/config.yaml
+COPY --chown=$UID:$GID config.example.yaml /app/config.yaml
 HEALTHCHECK CMD curl -f http://localhost:8080/ || exit 1
-ENTRYPOINT [ "/app/llama-swap", "-config", "/app/config.yaml" ]
+ENTRYPOINT [ "/app/llama-swap", "-config", "/app/config.yaml" ]
@@ -1,6 +1,6 @@
 module github.com/mostlygeek/llama-swap
-go 1.23.0
+go 1.25.4
 require (
 	github.com/billziss-gh/golib v0.2.0
@@ -37,9 +37,9 @@ require (
 	github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
 	github.com/ugorji/go/codec v1.2.12 // indirect
 	golang.org/x/arch v0.8.0 // indirect
-	golang.org/x/crypto v0.36.0 // indirect
+	golang.org/x/crypto v0.45.0 // indirect
-	golang.org/x/net v0.38.0 // indirect
+	golang.org/x/net v0.47.0 // indirect
-	golang.org/x/sys v0.31.0 // indirect
+	golang.org/x/sys v0.38.0 // indirect
-	golang.org/x/text v0.23.0 // indirect
+	golang.org/x/text v0.31.0 // indirect
 	google.golang.org/protobuf v1.34.1 // indirect
 )
@@ -80,16 +80,16 @@ github.com/ugorji/go/codec v1.2.12/go.mod h1:UNopzCgEMSXjBc6AOMqYvWC1ktqTAfzJZUZ
 golang.org/x/arch v0.0.0-20210923205945-b76863e36670/go.mod h1:5om86z9Hs0C8fWVUuoMHwpExlXzs5Tkyp9hOrfG7pp8=
 golang.org/x/arch v0.8.0 h1:3wRIsP3pM4yUptoR96otTUOXI367OS0+c9eeRi9doIc=
 golang.org/x/arch v0.8.0/go.mod h1:FEVrYAQjsQXMVJ1nsMoVVXPZg6p2JE2mx8psSWTDQys=
-golang.org/x/crypto v0.36.0 h1:AnAEvhDddvBdpY+uR+MyHmuZzzNqXSe/GvuDeob5L34=
+golang.org/x/crypto v0.45.0 h1:jMBrvKuj23MTlT0bQEOBcAE0mjg8mK9RXFhRH6nyF3Q=
-golang.org/x/crypto v0.36.0/go.mod h1:Y4J0ReaxCR1IMaabaSMugxJES1EpwhBHhv2bDHklZvc=
+golang.org/x/crypto v0.45.0/go.mod h1:XTGrrkGJve7CYK7J8PEww4aY7gM3qMCElcJQ8n8JdX4=
-golang.org/x/net v0.38.0 h1:vRMAPTMaeGqVhG5QyLJHqNDwecKTomGeqbnfZyKlBI8=
+golang.org/x/net v0.47.0 h1:Mx+4dIFzqraBXUugkia1OOvlD6LemFo1ALMHjrXDOhY=
-golang.org/x/net v0.38.0/go.mod h1:ivrbrMbzFq5J41QOQh0siUuly180yBYtLp+CKbEaFx8=
+golang.org/x/net v0.47.0/go.mod h1:/jNxtkgq5yWUGYkaZGqo27cfGZ1c5Nen03aYrrKpVRU=
 golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.31.0 h1:ioabZlmFYtWhL+TRYpcnNlLwhyxaM9kWTDEmfnprqik=
+golang.org/x/sys v0.38.0 h1:3yZWxaJjBmCWXqhN1qh02AkOnCQ1poK6oF+a7xWL6Gc=
-golang.org/x/sys v0.31.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
+golang.org/x/sys v0.38.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
-golang.org/x/text v0.23.0 h1:D71I7dUrlY+VX0gQShAThNGHFxZ13dGLBHQLVl1mJlY=
+golang.org/x/text v0.31.0 h1:aC8ghyu4JhP8VojJ2lEHBnochRno1sgL6nEi9WGFGMM=
-golang.org/x/text v0.23.0/go.mod h1:/BLNzu4aZCJ1+kcD0DNRotWKage4q2rGVAg4o22unh4=
+golang.org/x/text v0.31.0/go.mod h1:tKRAlv61yKIjGGHX/4tP1LTbc13YSec1pxVEWXzfoeM=
 golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 h1:E7g+9GITq07hpfrRu66IVDexMakfv52eLZ2CXBWiKr4=
 golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 google.golang.org/protobuf v1.34.1 h1:9ddQBjfCyZPOHPUiPxpYESBLc+T8P3E+Vo4IbKZgFWg=
@@ -95,7 +95,9 @@ func main() {
 			fmt.Println("Configuration Changed")
 			currentPM.Shutdown()
-			srv.Handler = proxy.New(conf)
+			newPM := proxy.New(conf)
 			newPM.SetVersion(date, commit, version)
 			srv.Handler = newPM
 			fmt.Println("Configuration Reloaded")
 			// wait a few seconds and tell any UI to reload
@@ -110,7 +112,9 @@ func main() {
 				fmt.Printf("Error, unable to load configuration: %v\n", err)
 				os.Exit(1)
 			}
-			srv.Handler = proxy.New(conf)
+			newPM := proxy.New(conf)
 			newPM.SetVersion(date, commit, version)
 			srv.Handler = newPM
 		}
 	}
@@ -113,6 +113,7 @@ type Config struct {
 	HealthCheckTimeout int                    `yaml:"healthCheckTimeout"`
 	LogRequests        bool                   `yaml:"logRequests"`
 	LogLevel           string                 `yaml:"logLevel"`
 	LogTimeFormat      string                 `yaml:"logTimeFormat"`
 	MetricsMaxInMemory int                    `yaml:"metricsMaxInMemory"`
 	Models             map[string]ModelConfig `yaml:"models"` /* key is model ID */
 	Profiles           map[string][]string    `yaml:"profiles"`
@@ -129,6 +130,12 @@ type Config struct {
 	// hooks, see: #209
 	Hooks HooksConfig `yaml:"hooks"`
 	// send loading state in reasoning
 	SendLoadingState bool `yaml:"sendLoadingState"`
 	// present aliases to /v1/models OpenAI API listing
 	IncludeAliasesInList bool `yaml:"includeAliasesInList"`
 }
 func (c *Config) RealModelName(search string) (string, bool) {
@@ -169,6 +176,7 @@ func LoadConfigFromReader(r io.Reader) (Config, error) {
 		HealthCheckTimeout: 120,
 		StartPort:          5800,
 		LogLevel:           "info",
 		LogTimeFormat:      "",
 		MetricsMaxInMemory: 1000,
 	}
 	err = yaml.Unmarshal(data, &config)
@@ -350,6 +358,13 @@ func LoadConfigFromReader(r io.Reader) (Config, error) {
 			)
 		}
 		// if sendLoadingState is nil, set it to the global config value
 		// see #366
 		if modelConfig.SendLoadingState == nil {
 			v := config.SendLoadingState // copy it
 			modelConfig.SendLoadingState = &v
 		}
 		config.Models[modelId] = modelConfig
 	}
@@ -58,6 +58,7 @@ models:
 	assert.Equal(t, 120, config.HealthCheckTimeout)
 	assert.Equal(t, 5800, config.StartPort)
 	assert.Equal(t, "info", config.LogLevel)
 	assert.Equal(t, "", config.LogTimeFormat)
 	// Test default group exists
 	defaultGroup, exists := config.Groups["(default)"]
@@ -160,9 +161,12 @@ groups:
 		t.Fatalf("Failed to load config: %v", err)
 	}
 	modelLoadingState := false
 	expected := Config{
-		LogLevel:  "info",
+		LogLevel:      "info",
-		StartPort: 5800,
+		LogTimeFormat: "",
 		StartPort:     5800,
 		Macros: MacroList{
 			{"svr-path", "path/to/server"},
 		},
@@ -171,36 +175,41 @@ groups:
 				Preload: []string{"model1", "model2"},
 			},
 		},
 		SendLoadingState: false,
 		Models: map[string]ModelConfig{
 			"model1": {
-				Cmd:           "path/to/cmd --arg1 one",
+				Cmd:              "path/to/cmd --arg1 one",
-				Proxy:         "http://localhost:8080",
+				Proxy:            "http://localhost:8080",
-				Aliases:       []string{"m1", "model-one"},
+				Aliases:          []string{"m1", "model-one"},
-				Env:           []string{"VAR1=value1", "VAR2=value2"},
+				Env:              []string{"VAR1=value1", "VAR2=value2"},
-				CheckEndpoint: "/health",
+				CheckEndpoint:    "/health",
-				Name:          "Model 1",
+				Name:             "Model 1",
-				Description:   "This is model 1",
+				Description:      "This is model 1",
 				SendLoadingState: &modelLoadingState,
 			},
 			"model2": {
-				Cmd:           "path/to/server --arg1 one",
+				Cmd:              "path/to/server --arg1 one",
-				Proxy:         "http://localhost:8081",
+				Proxy:            "http://localhost:8081",
-				Aliases:       []string{"m2"},
+				Aliases:          []string{"m2"},
-				Env:           []string{},
+				Env:              []string{},
-				CheckEndpoint: "/",
+				CheckEndpoint:    "/",
 				SendLoadingState: &modelLoadingState,
 			},
 			"model3": {
-				Cmd:           "path/to/cmd --arg1 one",
+				Cmd:              "path/to/cmd --arg1 one",
-				Proxy:         "http://localhost:8081",
+				Proxy:            "http://localhost:8081",
-				Aliases:       []string{"mthree"},
+				Aliases:          []string{"mthree"},
-				Env:           []string{},
+				Env:              []string{},
-				CheckEndpoint: "/",
+				CheckEndpoint:    "/",
 				SendLoadingState: &modelLoadingState,
 			},
 			"model4": {
-				Cmd:           "path/to/cmd --arg1 one",
+				Cmd:              "path/to/cmd --arg1 one",
-				Proxy:         "http://localhost:8082",
+				Proxy:            "http://localhost:8082",
-				CheckEndpoint: "/",
+				CheckEndpoint:    "/",
-				Aliases:       []string{},
+				Aliases:          []string{},
-				Env:           []string{},
+				Env:              []string{},
 				SendLoadingState: &modelLoadingState,
 			},
 		},
 		HealthCheckTimeout: 15,
@@ -55,6 +55,7 @@ models:
 	assert.Equal(t, 120, config.HealthCheckTimeout)
 	assert.Equal(t, 5800, config.StartPort)
 	assert.Equal(t, "info", config.LogLevel)
 	assert.Equal(t, "", config.LogTimeFormat)
 	// Test default group exists
 	defaultGroup, exists := config.Groups["(default)"]
@@ -152,44 +153,52 @@ groups:
 		t.Fatalf("Failed to load config: %v", err)
 	}
 	modelLoadingState := false
 	expected := Config{
-		LogLevel:  "info",
+		LogLevel:      "info",
-		StartPort: 5800,
+		LogTimeFormat: "",
 		StartPort:     5800,
 		Macros: MacroList{
 			{"svr-path", "path/to/server"},
 		},
 		SendLoadingState: false,
 		Models: map[string]ModelConfig{
 			"model1": {
-				Cmd:           "path/to/cmd --arg1 one",
+				Cmd:              "path/to/cmd --arg1 one",
-				CmdStop:       "taskkill /f /t /pid ${PID}",
+				CmdStop:          "taskkill /f /t /pid ${PID}",
-				Proxy:         "http://localhost:8080",
+				Proxy:            "http://localhost:8080",
-				Aliases:       []string{"m1", "model-one"},
+				Aliases:          []string{"m1", "model-one"},
-				Env:           []string{"VAR1=value1", "VAR2=value2"},
+				Env:              []string{"VAR1=value1", "VAR2=value2"},
-				CheckEndpoint: "/health",
+				CheckEndpoint:    "/health",
 				SendLoadingState: &modelLoadingState,
 			},
 			"model2": {
-				Cmd:           "path/to/server --arg1 one",
+				Cmd:              "path/to/server --arg1 one",
-				CmdStop:       "taskkill /f /t /pid ${PID}",
+				CmdStop:          "taskkill /f /t /pid ${PID}",
-				Proxy:         "http://localhost:8081",
+				Proxy:            "http://localhost:8081",
-				Aliases:       []string{"m2"},
+				Aliases:          []string{"m2"},
-				Env:           []string{},
+				Env:              []string{},
-				CheckEndpoint: "/",
+				CheckEndpoint:    "/",
 				SendLoadingState: &modelLoadingState,
 			},
 			"model3": {
-				Cmd:           "path/to/cmd --arg1 one",
+				Cmd:              "path/to/cmd --arg1 one",
-				CmdStop:       "taskkill /f /t /pid ${PID}",
+				CmdStop:          "taskkill /f /t /pid ${PID}",
-				Proxy:         "http://localhost:8081",
+				Proxy:            "http://localhost:8081",
-				Aliases:       []string{"mthree"},
+				Aliases:          []string{"mthree"},
-				Env:           []string{},
+				Env:              []string{},
-				CheckEndpoint: "/",
+				CheckEndpoint:    "/",
 				SendLoadingState: &modelLoadingState,
 			},
 			"model4": {
-				Cmd:           "path/to/cmd --arg1 one",
+				Cmd:              "path/to/cmd --arg1 one",
-				CmdStop:       "taskkill /f /t /pid ${PID}",
+				CmdStop:          "taskkill /f /t /pid ${PID}",
-				Proxy:         "http://localhost:8082",
+				Proxy:            "http://localhost:8082",
-				CheckEndpoint: "/",
+				CheckEndpoint:    "/",
-				Aliases:       []string{},
+				Aliases:          []string{},
-				Env:           []string{},
+				Env:              []string{},
 				SendLoadingState: &modelLoadingState,
 			},
 		},
 		HealthCheckTimeout: 15,
@@ -35,6 +35,9 @@ type ModelConfig struct {
 	// Metadata: see #264
 	// Arbitrary metadata that can be exposed through the API
 	Metadata map[string]any `yaml:"metadata"`
 	// override global setting
 	SendLoadingState *bool `yaml:"sendLoadingState"`
 }
 func (m *ModelConfig) UnmarshalYAML(unmarshal func(interface{}) error) error {
@@ -50,5 +50,25 @@ models:
 			}
 		})
 	}
-
+}
 func TestConfig_ModelSendLoadingState(t *testing.T) {
 	content := `
 sendLoadingState: true
 models:
  model1:
    cmd: path/to/cmd --port ${PORT}
    sendLoadingState: false
  model2:
    cmd: path/to/cmd --port ${PORT}
 `
 	config, err := LoadConfigFromReader(strings.NewReader(content))
 	assert.NoError(t, err)
 	assert.True(t, config.SendLoadingState)
 	if assert.NotNil(t, config.Models["model1"].SendLoadingState) {
 		assert.False(t, *config.Models["model1"].SendLoadingState)
 	}
 	if assert.NotNil(t, config.Models["model2"].SendLoadingState) {
 		assert.True(t, *config.Models["model2"].SendLoadingState)
 	}
 }
@@ -7,6 +7,7 @@ import (
 	"io"
 	"os"
 	"sync"
 	"time"
 	"github.com/mostlygeek/llama-swap/event"
 )
@@ -32,6 +33,9 @@ type LogMonitor struct {
 	// logging levels
 	level  LogLevel
 	prefix string
 	// timestamps
 	timeFormat string
 }
 func NewLogMonitor() *LogMonitor {
@@ -40,11 +44,12 @@ func NewLogMonitor() *LogMonitor {
 func NewLogMonitorWriter(stdout io.Writer) *LogMonitor {
 	return &LogMonitor{
-		eventbus: event.NewDispatcherConfig(1000),
+		eventbus:   event.NewDispatcherConfig(1000),
-		buffer:   ring.New(10 * 1024), // keep 10KB of buffered logs
+		buffer:     ring.New(10 * 1024), // keep 10KB of buffered logs
-		stdout:   stdout,
+		stdout:     stdout,
-		level:    LevelInfo,
+		level:      LevelInfo,
-		prefix:   "",
+		prefix:     "",
 		timeFormat: "",
 	}
 }
@@ -106,12 +111,22 @@ func (w *LogMonitor) SetLogLevel(level LogLevel) {
 	w.level = level
 }
 func (w *LogMonitor) SetLogTimeFormat(timeFormat string) {
 	w.mu.Lock()
 	defer w.mu.Unlock()
 	w.timeFormat = timeFormat
 }
 func (w *LogMonitor) formatMessage(level string, msg string) []byte {
 	prefix := ""
 	if w.prefix != "" {
 		prefix = fmt.Sprintf("[%s] ", w.prefix)
 	}
-	return []byte(fmt.Sprintf("%s[%s] %s\n", prefix, level, msg))
+	timestamp := ""
 	if w.timeFormat != "" {
 		timestamp = fmt.Sprintf("%s ", time.Now().Format(w.timeFormat))
 	}
 	return []byte(fmt.Sprintf("%s%s[%s] %s\n", timestamp, prefix, level, msg))
 }
 func (w *LogMonitor) log(level LogLevel, msg string) {
@@ -3,8 +3,10 @@ package proxy
 import (
 	"bytes"
 	"io"
 	"strings"
 	"sync"
 	"testing"
 	"time"
 )
 func TestLogMonitor(t *testing.T) {
@@ -84,3 +86,30 @@ func TestWrite_ImmutableBuffer(t *testing.T) {
 		t.Errorf("Expected history to be %q, got %q", expected, history)
 	}
 }
 func TestWrite_LogTimeFormat(t *testing.T) {
 	// Create a new LogMonitor instance
 	lm := NewLogMonitorWriter(io.Discard)
 	// Enable timestamps
 	lm.timeFormat = time.RFC3339
 	// Write the message to the LogMonitor
 	lm.Info("Hello, World!")
 	// Get the history from the LogMonitor
 	history := lm.GetHistory()
 	timestamp := ""
 	fields := strings.Fields(string(history))
 	if len(fields) > 0 {
 		timestamp = fields[0]
 	} else {
 		t.Fatalf("Cannot extract string from history")
 	}
 	_, err := time.Parse(time.RFC3339, timestamp)
 	if err != nil {
 		t.Fatalf("Cannot find timestamp: %v", err)
 	}
 }
@@ -2,8 +2,10 @@ package proxy
 import (
 	"context"
 	"encoding/json"
 	"errors"
 	"fmt"
 	"math/rand"
 	"net"
 	"net/http"
 	"net/http/httputil"
@@ -254,6 +256,7 @@ func (p *Process) start() error {
 	p.cmd.Env = append(p.cmd.Environ(), p.config.Env...)
 	p.cmd.Cancel = p.cmdStopUpstreamProcess
 	p.cmd.WaitDelay = p.gracefulStopTimeout
 	setProcAttributes(p.cmd)
 	p.cmdMutex.Lock()
 	p.cancelUpstream = ctxCancelUpstream
@@ -462,6 +465,12 @@ func (p *Process) checkHealthEndpoint(healthURL string) error {
 }
 func (p *Process) ProxyRequest(w http.ResponseWriter, r *http.Request) {
 	if p.reverseProxy == nil {
 		http.Error(w, fmt.Sprintf("No reverse proxy available for %s", p.ID), http.StatusInternalServerError)
 		return
 	}
 	requestBeginTime := time.Now()
 	var startDuration time.Duration
@@ -488,21 +497,68 @@ func (p *Process) ProxyRequest(w http.ResponseWriter, r *http.Request) {
 		p.inFlightRequests.Done()
 	}()
 	// for #366
 	// - extract streaming param from request context, should have been set by proxymanager
 	var srw *statusResponseWriter
 	swapCtx, cancelLoadCtx := context.WithCancel(r.Context())
 	// start the process on demand
 	if p.CurrentState() != StateReady {
 		// start a goroutine to stream loading status messages into the response writer
 		// add a sync so the streaming client only runs when the goroutine has exited
 		isStreaming, _ := r.Context().Value(proxyCtxKey("streaming")).(bool)
 		// PR #417 (no support for anthropic v1/messages yet)
 		isChatCompletions := strings.HasPrefix(r.URL.Path, "/v1/chat/completions")
 		if p.config.SendLoadingState != nil && *p.config.SendLoadingState && isStreaming && isChatCompletions {
 			srw = newStatusResponseWriter(p, w)
 			go srw.statusUpdates(swapCtx)
 		} else {
 			p.proxyLogger.Debugf("<%s> SendLoadingState is nil or false, not streaming loading state", p.ID)
 		}
 		beginStartTime := time.Now()
 		if err := p.start(); err != nil {
 			errstr := fmt.Sprintf("unable to start process: %s", err)
-			http.Error(w, errstr, http.StatusBadGateway)
+			cancelLoadCtx()
 			if srw != nil {
 				srw.sendData(fmt.Sprintf("Unable to swap model err: %s\n", errstr))
 				// Wait for statusUpdates goroutine to finish writing its deferred "Done!" messages
 				// before closing the connection. Without this, the connection would close before
 				// the goroutine can write its cleanup messages, causing incomplete SSE output.
 				srw.waitForCompletion(100 * time.Millisecond)
 			} else {
 				http.Error(w, errstr, http.StatusBadGateway)
 			}
 			return
 		}
 		startDuration = time.Since(beginStartTime)
 	}
-	if p.reverseProxy != nil {
+	// should trigger srw to stop sending loading events ...
-		p.reverseProxy.ServeHTTP(w, r)
+	cancelLoadCtx()
 	// recover from http.ErrAbortHandler panics that can occur when the client
 	// disconnects before the response is sent
 	defer func() {
 		if r := recover(); r != nil {
 			if r == http.ErrAbortHandler {
 				p.proxyLogger.Infof("<%s> recovered from client disconnection during streaming", p.ID)
 			} else {
 				p.proxyLogger.Infof("<%s> recovered from panic: %v", p.ID, r)
 			}
 		}
 	}()
 	if srw != nil {
 		// Wait for the goroutine to finish writing its final messages
 		const completionTimeout = 1 * time.Second
 		if !srw.waitForCompletion(completionTimeout) {
 			p.proxyLogger.Warnf("<%s> status updates goroutine did not complete within %v, proceeding with proxy request", p.ID, completionTimeout)
 		}
 		p.reverseProxy.ServeHTTP(srw, r)
 	} else {
-		http.Error(w, fmt.Sprintf("No reverse proxy available for %s", p.ID), http.StatusInternalServerError)
+		p.reverseProxy.ServeHTTP(w, r)
 	}
 	totalTime := time.Since(requestBeginTime)
@@ -573,6 +629,7 @@ func (p *Process) cmdStopUpstreamProcess() error {
 		stopCmd := exec.Command(stopArgs[0], stopArgs[1:]...)
 		stopCmd.Stdout = p.processLogger
 		stopCmd.Stderr = p.processLogger
 		setProcAttributes(stopCmd)
 		stopCmd.Env = p.cmd.Env
 		if err := stopCmd.Run(); err != nil {
@@ -588,3 +645,228 @@ func (p *Process) cmdStopUpstreamProcess() error {
 	return nil
 }
 var loadingRemarks = []string{
 	"Still faster than your last standup meeting...",
 	"Reticulating splines...",
 	"Waking up the hamsters...",
 	"Teaching the model manners...",
 	"Convincing the GPU to participate...",
 	"Loading weights (they're heavy)...",
 	"Herding electrons...",
 	"Compiling excuses for the delay...",
 	"Downloading more RAM...",
 	"Asking the model nicely to boot up...",
 	"Bribing CUDA with cookies...",
 	"Still loading (blame VRAM)...",
 	"The model is fashionably late...",
 	"Warming up those tensors...",
 	"Making the neural net do push-ups...",
 	"Your patience is appreciated (really)...",
 	"Almost there (probably)...",
 	"Loading like it's 1999...",
 	"The model forgot where it put its keys...",
 	"Quantum tunneling through layers...",
 	"Negotiating with the PCIe bus...",
 	"Defrosting frozen parameters...",
 	"Teaching attention heads to focus...",
 	"Running the matrix (slowly)...",
 	"Untangling transformer blocks...",
 	"Calibrating the flux capacitor...",
 	"Spinning up the probability wheels...",
 	"Waiting for the GPU to wake from its nap...",
 	"Converting caffeine to compute...",
 	"Allocating virtual patience...",
 	"Performing arcane CUDA rituals...",
 	"The model is stuck in traffic...",
 	"Inflating embeddings...",
 	"Summoning computational demons...",
 	"Pleading with the OOM killer...",
 	"Calculating the meaning of life (still at 42)...",
 	"Training the training wheels...",
 	"Optimizing the optimizer...",
 	"Bootstrapping the bootstrapper...",
 	"Loading loading screen...",
 	"Processing processing logs...",
 	"Buffering buffer overflow jokes...",
 	"The model hit snooze...",
 	"Debugging the debugger...",
 	"Compiling the compiler...",
 	"Parsing the parser (meta)...",
 	"Tokenizing tokens...",
 	"Encoding the encoder...",
 	"Hashing hash browns...",
 	"Forking spoons (not forks)...",
 	"The model is contemplating existence...",
 	"Transcending dimensional barriers...",
 	"Invoking elder tensor gods...",
 	"Unfurling probability clouds...",
 	"Synchronizing parallel universes...",
 	"The GPU is having second thoughts...",
 	"Recalibrating reality matrices...",
 	"Time is an illusion, loading doubly so...",
 	"Convincing bits to flip themselves...",
 	"The model is reading its own documentation...",
 }
 type statusResponseWriter struct {
 	hasWritten bool
 	writer     http.ResponseWriter
 	process    *Process
 	wg         sync.WaitGroup // Track goroutine completion
 	start      time.Time
 }
 func newStatusResponseWriter(p *Process, w http.ResponseWriter) *statusResponseWriter {
 	s := &statusResponseWriter{
 		writer:  w,
 		process: p,
 		start:   time.Now(),
 	}
 	s.Header().Set("Content-Type", "text/event-stream") // SSE
 	s.Header().Set("Cache-Control", "no-cache")         // no-cache
 	s.Header().Set("Connection", "keep-alive")          // keep-alive
 	s.WriteHeader(http.StatusOK)                        // send status code 200
 	s.sendLine("━━━━━")
 	s.sendLine(fmt.Sprintf("llama-swap loading model: %s", p.ID))
 	return s
 }
 // statusUpdates sends status updates to the client while the model is loading
 func (s *statusResponseWriter) statusUpdates(ctx context.Context) {
 	s.wg.Add(1)
 	defer s.wg.Done()
 	// Recover from panics caused by client disconnection
 	// Note: recover() only works within the same goroutine, so we need it here
 	defer func() {
 		if r := recover(); r != nil {
 			s.process.proxyLogger.Debugf("<%s> statusUpdates recovered from panic (likely client disconnect): %v", s.process.ID, r)
 		}
 	}()
 	defer func() {
 		duration := time.Since(s.start)
 		s.sendLine(fmt.Sprintf("\nDone! (%.2fs)", duration.Seconds()))
 		s.sendLine("━━━━━")
 		s.sendLine(" ")
 	}()
 	// Create a shuffled copy of loadingRemarks
 	remarks := make([]string, len(loadingRemarks))
 	copy(remarks, loadingRemarks)
 	rand.Shuffle(len(remarks), func(i, j int) {
 		remarks[i], remarks[j] = remarks[j], remarks[i]
 	})
 	ri := 0
 	// Pick a random duration to send a remark
 	nextRemarkIn := time.Duration(2+rand.Intn(4)) * time.Second
 	lastRemarkTime := time.Now()
 	ticker := time.NewTicker(time.Second)
 	defer ticker.Stop() // Ensure ticker is stopped to prevent resource leak
 	for {
 		select {
 		case <-ctx.Done():
 			return
 		case <-ticker.C:
 			if s.process.CurrentState() == StateReady {
 				return
 			}
 			// Check if it's time for a snarky remark
 			if time.Since(lastRemarkTime) >= nextRemarkIn {
 				remark := remarks[ri%len(remarks)]
 				ri++
 				s.sendLine(fmt.Sprintf("\n%s", remark))
 				lastRemarkTime = time.Now()
 				// Pick a new random duration for the next remark
 				nextRemarkIn = time.Duration(5+rand.Intn(5)) * time.Second
 			} else {
 				s.sendData(".")
 			}
 		}
 	}
 }
 // waitForCompletion waits for the statusUpdates goroutine to finish
 func (s *statusResponseWriter) waitForCompletion(timeout time.Duration) bool {
 	done := make(chan struct{})
 	go func() {
 		s.wg.Wait()
 		close(done)
 	}()
 	select {
 	case <-done:
 		return true
 	case <-time.After(timeout):
 		return false
 	}
 }
 func (s *statusResponseWriter) sendLine(line string) {
 	s.sendData(line + "\n")
 }
 func (s *statusResponseWriter) sendData(data string) {
 	// Create the proper SSE JSON structure
 	type Delta struct {
 		ReasoningContent string `json:"reasoning_content"`
 	}
 	type Choice struct {
 		Delta Delta `json:"delta"`
 	}
 	type SSEMessage struct {
 		Choices []Choice `json:"choices"`
 	}
 	msg := SSEMessage{
 		Choices: []Choice{
 			{
 				Delta: Delta{
 					ReasoningContent: data,
 				},
 			},
 		},
 	}
 	jsonData, err := json.Marshal(msg)
 	if err != nil {
 		s.process.proxyLogger.Errorf("<%s> Failed to marshal SSE message: %v", s.process.ID, err)
 		return
 	}
 	// Write SSE formatted data, panic if not able to write
 	_, err = fmt.Fprintf(s.writer, "data: %s\n\n", jsonData)
 	if err != nil {
 		panic(fmt.Sprintf("<%s> Failed to write SSE data: %v", s.process.ID, err))
 	}
 	s.Flush()
 }
 func (s *statusResponseWriter) Header() http.Header {
 	return s.writer.Header()
 }
 func (s *statusResponseWriter) Write(data []byte) (int, error) {
 	return s.writer.Write(data)
 }
 func (s *statusResponseWriter) WriteHeader(statusCode int) {
 	if s.hasWritten {
 		return
 	}
 	s.hasWritten = true
 	s.writer.WriteHeader(statusCode)
 	s.Flush()
 }
 // Add Flush method
 func (s *statusResponseWriter) Flush() {
 	if flusher, ok := s.writer.(http.Flusher); ok {
 		flusher.Flush()
 	}
 }
@@ -494,3 +494,74 @@ func TestProcess_EnvironmentSetCorrectly(t *testing.T) {
 	assert.Equal(t, len(process1.cmd.Environ())+2, len(process2.cmd.Environ()), "process2 should have 2 more environment variables than process1")
 }
 // TestProcess_ReverseProxyPanicIsHandled tests that panics from
 // httputil.ReverseProxy in Process.ProxyRequest(w, r) do not bubble up and are
 // handled appropriately.
 //
 // httputil.ReverseProxy will panic with http.ErrAbortHandler when it has sent headers
 // can't copy the body. This can be caused by a client disconnecting before the full
 // response is sent from some reason.
 //
 // bug: https://github.com/mostlygeek/llama-swap/issues/362
 // see: https://github.com/golang/go/issues/23643 (where panic was added to httputil.ReverseProxy)
 func TestProcess_ReverseProxyPanicIsHandled(t *testing.T) {
 	// Add defer/recover to catch any panics that aren't handled by ProxyRequest
 	// If this recover() is hit, it means ProxyRequest didn't handle the panic properly
 	defer func() {
 		if r := recover(); r != nil {
 			t.Fatalf("ProxyRequest should handle panics from reverseProxy.ServeHTTP, but panic was not caught: %v", r)
 		}
 	}()
 	expectedMessage := "panic_test"
 	config := getTestSimpleResponderConfig(expectedMessage)
 	process := NewProcess("panic-test", 5, config, debugLogger, debugLogger)
 	defer process.Stop()
 	// Start the process
 	err := process.start()
 	assert.Nil(t, err)
 	assert.Equal(t, StateReady, process.CurrentState())
 	// Create a custom ResponseWriter that simulates a client disconnect
 	// by panicking when Write is called after headers are sent
 	panicWriter := &panicOnWriteResponseWriter{
 		ResponseRecorder: httptest.NewRecorder(),
 		shouldPanic:      true,
 	}
 	// Make a request that will trigger the panic
 	req := httptest.NewRequest("GET", "/slow-respond?echo=test&delay=100ms", nil)
 	// This should panic inside reverseProxy.ServeHTTP when the panicWriter.Write() is called.
 	// ProxyRequest should catch and handle this panic gracefully.
 	process.ProxyRequest(panicWriter, req)
 	// If we get here, the panic was properly recovered in ProxyRequest
 	// The process should still be in a ready state
 	assert.Equal(t, StateReady, process.CurrentState())
 }
 // panicOnWriteResponseWriter is a ResponseWriter that panics on Write
 // to simulate a client disconnect after headers are sent
 // used by: TestProcess_ReverseProxyPanicIsHandled
 type panicOnWriteResponseWriter struct {
 	*httptest.ResponseRecorder
 	shouldPanic   bool
 	headerWritten bool
 }
 func (w *panicOnWriteResponseWriter) WriteHeader(statusCode int) {
 	w.headerWritten = true
 	w.ResponseRecorder.WriteHeader(statusCode)
 }
 func (w *panicOnWriteResponseWriter) Write(b []byte) (int, error) {
 	if w.shouldPanic && w.headerWritten {
 		// Simulate the panic that httputil.ReverseProxy throws
 		panic(http.ErrAbortHandler)
 	}
 	return w.ResponseRecorder.Write(b)
 }
@@ -0,0 +1,12 @@
 //go:build !windows
 package proxy
 import (
 	"os/exec"
 )
 // setProcAttributes sets platform-specific process attributes
 func setProcAttributes(cmd *exec.Cmd) {
 	// No-op on Unix systems
 }
@@ -0,0 +1,16 @@
 //go:build windows
 package proxy
 import (
 	"os/exec"
 	"syscall"
 )
 // setProcAttributes sets platform-specific process attributes
 func setProcAttributes(cmd *exec.Cmd) {
 	cmd.SysProcAttr = &syscall.SysProcAttr{
 		HideWindow:    true,
 		CreationFlags: 0x08000000, // CREATE_NO_WINDOW
 	}
 }
@@ -25,6 +25,8 @@ const (
 	PROFILE_SPLIT_CHAR = ":"
 )
 type proxyCtxKey string
 type ProxyManager struct {
 	sync.Mutex
@@ -43,6 +45,11 @@ type ProxyManager struct {
 	// shutdown signaling
 	shutdownCtx    context.Context
 	shutdownCancel context.CancelFunc
 	// version info
 	buildDate string
 	commit    string
 	version   string
 }
 func New(config config.Config) *ProxyManager {
@@ -73,6 +80,30 @@ func New(config config.Config) *ProxyManager {
 		upstreamLogger.SetLogLevel(LevelInfo)
 	}
 	// see: https://go.dev/src/time/format.go
 	timeFormats := map[string]string{
 		"ansic":       time.ANSIC,
 		"unixdate":    time.UnixDate,
 		"rubydate":    time.RubyDate,
 		"rfc822":      time.RFC822,
 		"rfc822z":     time.RFC822Z,
 		"rfc850":      time.RFC850,
 		"rfc1123":     time.RFC1123,
 		"rfc1123z":    time.RFC1123Z,
 		"rfc3339":     time.RFC3339,
 		"rfc3339nano": time.RFC3339Nano,
 		"kitchen":     time.Kitchen,
 		"stamp":       time.Stamp,
 		"stampmilli":  time.StampMilli,
 		"stampmicro":  time.StampMicro,
 		"stampnano":   time.StampNano,
 	}
 	if timeFormat, ok := timeFormats[strings.ToLower(strings.TrimSpace(config.LogTimeFormat))]; ok {
 		proxyLogger.SetLogTimeFormat(timeFormat)
 		upstreamLogger.SetLogTimeFormat(timeFormat)
 	}
 	shutdownCtx, shutdownCancel := context.WithCancel(context.Background())
 	var maxMetrics int
@@ -96,6 +127,10 @@ func New(config config.Config) *ProxyManager {
 		shutdownCtx:    shutdownCtx,
 		shutdownCancel: shutdownCancel,
 		buildDate: "unknown",
 		commit:    "abcd1234",
 		version:   "0",
 	}
 	// create the process groups
@@ -201,27 +236,29 @@ func (pm *ProxyManager) setupGinEngine() {
 	})
 	// Set up routes using the Gin engine
-	pm.ginEngine.POST("/v1/chat/completions", pm.proxyOAIHandler)
+	pm.ginEngine.POST("/v1/chat/completions", pm.proxyInferenceHandler)
 	// Support legacy /v1/completions api, see issue #12
-	pm.ginEngine.POST("/v1/completions", pm.proxyOAIHandler)
+	pm.ginEngine.POST("/v1/completions", pm.proxyInferenceHandler)
 	// Support anthropic /v1/messages (added https://github.com/ggml-org/llama.cpp/pull/17570)
 	pm.ginEngine.POST("/v1/messages", pm.proxyInferenceHandler)
 	// Support embeddings and reranking
-	pm.ginEngine.POST("/v1/embeddings", pm.proxyOAIHandler)
+	pm.ginEngine.POST("/v1/embeddings", pm.proxyInferenceHandler)
 	// llama-server's /reranking endpoint + aliases
-	pm.ginEngine.POST("/reranking", pm.proxyOAIHandler)
+	pm.ginEngine.POST("/reranking", pm.proxyInferenceHandler)
-	pm.ginEngine.POST("/rerank", pm.proxyOAIHandler)
+	pm.ginEngine.POST("/rerank", pm.proxyInferenceHandler)
-	pm.ginEngine.POST("/v1/rerank", pm.proxyOAIHandler)
+	pm.ginEngine.POST("/v1/rerank", pm.proxyInferenceHandler)
-	pm.ginEngine.POST("/v1/reranking", pm.proxyOAIHandler)
+	pm.ginEngine.POST("/v1/reranking", pm.proxyInferenceHandler)
 	// llama-server's /infill endpoint for code infilling
-	pm.ginEngine.POST("/infill", pm.proxyOAIHandler)
+	pm.ginEngine.POST("/infill", pm.proxyInferenceHandler)
 	// llama-server's /completion endpoint
-	pm.ginEngine.POST("/completion", pm.proxyOAIHandler)
+	pm.ginEngine.POST("/completion", pm.proxyInferenceHandler)
 	// Support audio/speech endpoint
-	pm.ginEngine.POST("/v1/audio/speech", pm.proxyOAIHandler)
+	pm.ginEngine.POST("/v1/audio/speech", pm.proxyInferenceHandler)
 	pm.ginEngine.POST("/v1/audio/transcriptions", pm.proxyOAIPostFormHandler)
 	pm.ginEngine.GET("/v1/models", pm.listModelsHandler)
@@ -374,28 +411,40 @@ func (pm *ProxyManager) listModelsHandler(c *gin.Context) {
 			continue
 		}
-		record := gin.H{
+		newRecord := func(modelId string) gin.H {
-			"id":       id,
+			record := gin.H{
-			"object":   "model",
+				"id":       modelId,
-			"created":  createdTime,
+				"object":   "model",
-			"owned_by": "llama-swap",
+				"created":  createdTime,
 				"owned_by": "llama-swap",
 			}
 			if name := strings.TrimSpace(modelConfig.Name); name != "" {
 				record["name"] = name
 			}
 			if desc := strings.TrimSpace(modelConfig.Description); desc != "" {
 				record["description"] = desc
 			}
 			// Add metadata if present
 			if len(modelConfig.Metadata) > 0 {
 				record["meta"] = gin.H{
 					"llamaswap": modelConfig.Metadata,
 				}
 			}
 			return record
 		}
-		if name := strings.TrimSpace(modelConfig.Name); name != "" {
+		data = append(data, newRecord(id))
 			record["name"] = name
 		}
 		if desc := strings.TrimSpace(modelConfig.Description); desc != "" {
 			record["description"] = desc
 		}
-		// Add metadata if present
+		// Include aliases
-		if len(modelConfig.Metadata) > 0 {
+		if pm.config.IncludeAliasesInList {
-			record["meta"] = gin.H{
+			for _, alias := range modelConfig.Aliases {
-				"llamaswap": modelConfig.Metadata,
+				if alias := strings.TrimSpace(alias); alias != "" {
 					data = append(data, newRecord(alias))
 				}
 			}
 		}
 		data = append(data, record)
 	}
 	// Sort by the "id" key
@@ -498,7 +547,7 @@ func (pm *ProxyManager) proxyToUpstream(c *gin.Context) {
 	}
 }
-func (pm *ProxyManager) proxyOAIHandler(c *gin.Context) {
+func (pm *ProxyManager) proxyInferenceHandler(c *gin.Context) {
 	bodyBytes, err := io.ReadAll(c.Request.Body)
 	if err != nil {
 		pm.sendErrorResponse(c, http.StatusBadRequest, "could not ready request body")
@@ -555,6 +604,12 @@ func (pm *ProxyManager) proxyOAIHandler(c *gin.Context) {
 	c.Request.Header.Set("content-length", strconv.Itoa(len(bodyBytes)))
 	c.Request.ContentLength = int64(len(bodyBytes))
 	// issue #366 extract values that downstream handlers may need
 	isStreaming := gjson.GetBytes(bodyBytes, "stream").Bool()
 	ctx := context.WithValue(c.Request.Context(), proxyCtxKey("streaming"), isStreaming)
 	ctx = context.WithValue(ctx, proxyCtxKey("model"), realModelName)
 	c.Request = c.Request.WithContext(ctx)
 	if pm.metricsMonitor != nil && c.Request.Method == "POST" {
 		if err := pm.metricsMonitor.wrapHandler(realModelName, c.Writer, c.Request, processGroup.ProxyRequest); err != nil {
 			pm.sendErrorResponse(c, http.StatusInternalServerError, fmt.Sprintf("error proxying metrics wrapped request: %s", err.Error()))
@@ -726,3 +781,11 @@ func (pm *ProxyManager) findGroupByModelName(modelName string) *ProcessGroup {
 	}
 	return nil
 }
 func (pm *ProxyManager) SetVersion(buildDate string, commit string, version string) {
 	pm.Lock()
 	defer pm.Unlock()
 	pm.buildDate = buildDate
 	pm.commit = commit
 	pm.version = version
 }
@@ -28,6 +28,7 @@ func addApiHandlers(pm *ProxyManager) {
 		apiGroup.POST("/models/unload/*model", pm.apiUnloadSingleModelHandler)
 		apiGroup.GET("/events", pm.apiSendEvents)
 		apiGroup.GET("/metrics", pm.apiGetMetrics)
 		apiGroup.GET("/version", pm.apiGetVersion)
 	}
 }
@@ -227,3 +228,11 @@ func (pm *ProxyManager) apiUnloadSingleModelHandler(c *gin.Context) {
 		c.String(http.StatusOK, "OK")
 	}
 }
 func (pm *ProxyManager) apiGetVersion(c *gin.Context) {
 	c.JSON(http.StatusOK, map[string]string{
 		"version":    pm.version,
 		"commit":     pm.commit,
 		"build_date": pm.buildDate,
 	})
 }
@@ -437,6 +437,70 @@ func TestProxyManager_ListModelsHandler_SortedByID(t *testing.T) {
 	}
 }
 func TestProxyManager_ListModelsHandler_IncludeAliasesInList(t *testing.T) {
 	// Configure alias
 	config := config.Config{
 		HealthCheckTimeout:   15,
 		IncludeAliasesInList: true,
 		Models: map[string]config.ModelConfig{
 			"model1": func() config.ModelConfig {
 				mc := getTestSimpleResponderConfig("model1")
 				mc.Name = "Model 1"
 				mc.Aliases = []string{"alias1"}
 				return mc
 			}(),
 		},
 		LogLevel: "error",
 	}
 	proxy := New(config)
 	// Request models list
 	req := httptest.NewRequest("GET", "/v1/models", nil)
 	w := CreateTestResponseRecorder()
 	proxy.ServeHTTP(w, req)
 	assert.Equal(t, http.StatusOK, w.Code)
 	var response struct {
 		Data []map[string]interface{} `json:"data"`
 	}
 	if err := json.Unmarshal(w.Body.Bytes(), &response); err != nil {
 		t.Fatalf("Failed to parse JSON response: %v", err)
 	}
 	// We expect both base id and alias
 	var model1Data, alias1Data map[string]any
 	for _, model := range response.Data {
 		if model["id"] == "model1" {
 			model1Data = model
 		} else if model["id"] == "alias1" {
 			alias1Data = model
 		}
 	}
 	// Verify model1 has name
 	assert.NotNil(t, model1Data)
 	_, exists := model1Data["name"]
 	if !assert.True(t, exists, "model1 should have name key") {
 		t.FailNow()
 	}
 	name1, ok := model1Data["name"].(string)
 	assert.True(t, ok, "name1 should be a string")
 	// Verify alias1 has name
 	assert.NotNil(t, alias1Data)
 	_, exists = alias1Data["name"]
 	if !assert.True(t, exists, "alias1 should have name key") {
 		t.FailNow()
 	}
 	name2, ok := alias1Data["name"].(string)
 	assert.True(t, ok, "name2 should be a string")
 	// Name keys should match
 	assert.Equal(t, name1, name2)
 }
 func TestProxyManager_Shutdown(t *testing.T) {
 	// make broken model configurations
 	model1Config := getTestSimpleResponderConfigPort("model1", 9991)
@@ -1083,3 +1147,41 @@ func TestProxyManager_ProxiedStreamingEndpointReturnsNoBufferingHeader(t *testin
 	assert.Equal(t, "no", rec.Header().Get("X-Accel-Buffering"))
 	assert.Contains(t, rec.Header().Get("Content-Type"), "text/event-stream")
 }
 func TestProxyManager_ApiGetVersion(t *testing.T) {
 	config := config.AddDefaultGroupToConfig(config.Config{
 		HealthCheckTimeout: 15,
 		Models: map[string]config.ModelConfig{
 			"model1": getTestSimpleResponderConfig("model1"),
 		},
 		LogLevel: "error",
 	})
 	// Version test map
 	versionTest := map[string]string{
 		"build_date": "1970-01-01T00:00:00Z",
 		"commit":     "cc915ddb6f04a42d9cd1f524e1d46ec6ed069fdc",
 		"version":    "v001",
 	}
 	proxy := New(config)
 	proxy.SetVersion(versionTest["build_date"], versionTest["commit"], versionTest["version"])
 	defer proxy.StopProcesses(StopWaitForInflightRequest)
 	req := httptest.NewRequest("GET", "/api/version", nil)
 	w := CreateTestResponseRecorder()
 	proxy.ServeHTTP(w, req)
 	assert.Equal(t, http.StatusOK, w.Code)
 	// Ensure json response
 	assert.Equal(t, "application/json; charset=utf-8", w.Header().Get("Content-Type"))
 	// Check for attributes
 	response := map[string]string{}
 	assert.NoError(t, json.Unmarshal(w.Body.Bytes(), &response))
 	for key, value := range versionTest {
 		assert.Equal(t, value, response[key], "%s value %s should match response %s", key, value, response[key])
 	}
 }
@@ -752,9 +752,9 @@
      }
    },
    "node_modules/@eslint-community/eslint-utils": {
-      "version": "4.7.0",
+      "version": "4.9.0",
-      "resolved": "https://registry.npmjs.org/@eslint-community/eslint-utils/-/eslint-utils-4.7.0.tgz",
+      "resolved": "https://registry.npmjs.org/@eslint-community/eslint-utils/-/eslint-utils-4.9.0.tgz",
-      "integrity": "sha512-dyybb3AcajC7uha6CvhdVRJqaKyn7w2YKqKyAN37NKYgZT36w+iRb0Dymmc5qEJ549c/S31cMMSFd75bteCpCw==",
+      "integrity": "sha512-ayVFHdtZ+hsq1t2Dy24wCmGXGe4q9Gu3smhLYALJrr473ZH27MsnSL+LKUlimp4BWJqMDMLmPpx/Q9R3OAlL4g==",
      "dev": true,
      "license": "MIT",
      "dependencies": {
@@ -794,13 +794,13 @@
      }
    },
    "node_modules/@eslint/config-array": {
-      "version": "0.20.0",
+      "version": "0.21.1",
-      "resolved": "https://registry.npmjs.org/@eslint/config-array/-/config-array-0.20.0.tgz",
+      "resolved": "https://registry.npmjs.org/@eslint/config-array/-/config-array-0.21.1.tgz",
-      "integrity": "sha512-fxlS1kkIjx8+vy2SjuCB94q3htSNrufYTXubwiBFeaQHbH6Ipi43gFJq2zCMt6PHhImH3Xmr0NksKDvchWlpQQ==",
+      "integrity": "sha512-aw1gNayWpdI/jSYVgzN5pL0cfzU02GT3NBpeT/DXbx1/1x7ZKxFPd9bwrzygx/qiwIQiJ1sw/zD8qY/kRvlGHA==",
      "dev": true,
      "license": "Apache-2.0",
      "dependencies": {
-        "@eslint/object-schema": "^2.1.6",
+        "@eslint/object-schema": "^2.1.7",
        "debug": "^4.3.1",
        "minimatch": "^3.1.2"
      },
@@ -809,19 +809,22 @@
      }
    },
    "node_modules/@eslint/config-helpers": {
-      "version": "0.2.2",
+      "version": "0.4.2",
-      "resolved": "https://registry.npmjs.org/@eslint/config-helpers/-/config-helpers-0.2.2.tgz",
+      "resolved": "https://registry.npmjs.org/@eslint/config-helpers/-/config-helpers-0.4.2.tgz",
-      "integrity": "sha512-+GPzk8PlG0sPpzdU5ZvIRMPidzAnZDl/s9L+y13iodqvb8leL53bTannOrQ/Im7UkpsmFU5Ily5U60LWixnmLg==",
+      "integrity": "sha512-gBrxN88gOIf3R7ja5K9slwNayVcZgK6SOUORm2uBzTeIEfeVaIhOpCtTox3P6R7o2jLFwLFTLnC7kU/RGcYEgw==",
      "dev": true,
      "license": "Apache-2.0",
      "dependencies": {
        "@eslint/core": "^0.17.0"
      },
      "engines": {
        "node": "^18.18.0 || ^20.9.0 || >=21.1.0"
      }
    },
    "node_modules/@eslint/core": {
-      "version": "0.14.0",
+      "version": "0.17.0",
-      "resolved": "https://registry.npmjs.org/@eslint/core/-/core-0.14.0.tgz",
+      "resolved": "https://registry.npmjs.org/@eslint/core/-/core-0.17.0.tgz",
-      "integrity": "sha512-qIbV0/JZr7iSDjqAc60IqbLdsj9GDt16xQtWD+B78d/HAlvysGdZZ6rpJHGAc2T0FQx1X6thsSPdnoiGKdNtdg==",
+      "integrity": "sha512-yL/sLrpmtDaFEiUj1osRP4TI2MDz1AddJL+jZ7KSqvBuliN4xqYY54IfdN8qD8Toa6g1iloph1fxQNkjOxrrpQ==",
      "dev": true,
      "license": "Apache-2.0",
      "dependencies": {
@@ -869,9 +872,9 @@
      }
    },
    "node_modules/@eslint/js": {
-      "version": "9.28.0",
+      "version": "9.39.1",
-      "resolved": "https://registry.npmjs.org/@eslint/js/-/js-9.28.0.tgz",
+      "resolved": "https://registry.npmjs.org/@eslint/js/-/js-9.39.1.tgz",
-      "integrity": "sha512-fnqSjGWd/CoIp4EXIxWVK/sHA6DOHN4+8Ix2cX5ycOY7LG0UY8nHCU5pIp2eaE1Mc7Qd8kHspYNzYXT2ojPLzg==",
+      "integrity": "sha512-S26Stp4zCy88tH94QbBv3XCuzRQiZ9yXofEILmglYTh/Ug/a9/umqvgFtYBAo3Lp0nsI/5/qH1CCrbdK3AP1Tw==",
      "dev": true,
      "license": "MIT",
      "engines": {
@@ -882,9 +885,9 @@
      }
    },
    "node_modules/@eslint/object-schema": {
-      "version": "2.1.6",
+      "version": "2.1.7",
-      "resolved": "https://registry.npmjs.org/@eslint/object-schema/-/object-schema-2.1.6.tgz",
+      "resolved": "https://registry.npmjs.org/@eslint/object-schema/-/object-schema-2.1.7.tgz",
-      "integrity": "sha512-RBMg5FRL0I0gs51M/guSAj5/e14VQ4tpZnQNWwuDT66P14I43ItmPfIZRhO9fUVIPOAQXU47atlywZ/czoqFPA==",
+      "integrity": "sha512-VtAOaymWVfZcmZbp6E2mympDIHvyjXs/12LqWYjVw6qjrfF+VK+fyG33kChz3nnK+SU5/NeHOqrTEHS8sXO3OA==",
      "dev": true,
      "license": "Apache-2.0",
      "engines": {
@@ -892,13 +895,13 @@
      }
    },
    "node_modules/@eslint/plugin-kit": {
-      "version": "0.3.1",
+      "version": "0.4.1",
-      "resolved": "https://registry.npmjs.org/@eslint/plugin-kit/-/plugin-kit-0.3.1.tgz",
+      "resolved": "https://registry.npmjs.org/@eslint/plugin-kit/-/plugin-kit-0.4.1.tgz",
-      "integrity": "sha512-0J+zgWxHN+xXONWIyPWKFMgVuJoZuGiIFu8yxk7RJjxkzpGmyja5wRFqZIVtjDVOQpV+Rw0iOAjYPE2eQyjr0w==",
+      "integrity": "sha512-43/qtrDUokr7LJqoF2c3+RInu/t4zfrpYdoSDfYyhg52rwLV6TnOvdG4fXm7IkSB3wErkcmJS9iEhjVtOSEjjA==",
      "dev": true,
      "license": "Apache-2.0",
      "dependencies": {
-        "@eslint/core": "^0.14.0",
+        "@eslint/core": "^0.17.0",
        "levn": "^0.4.1"
      },
      "engines": {
@@ -1908,9 +1911,9 @@
      }
    },
    "node_modules/@typescript-eslint/typescript-estree/node_modules/brace-expansion": {
-      "version": "2.0.1",
+      "version": "2.0.2",
-      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.0.1.tgz",
+      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.0.2.tgz",
-      "integrity": "sha512-XnAIvQ8eM+kC6aULx6wuQiwVsnzsi9d3WxzV3FpWTGA19F621kwdbsAcFKXgKUHZWsy+mY6iL1sHTxWEFCytDA==",
+      "integrity": "sha512-Jt0vHyM+jmUBqojB7E1NIYadt0vI0Qxjxd2TErW94wDz+E2LAm5vKMXXwg6ZZBTHPuUlDgQHKXvjGBdfcF1ZDQ==",
      "dev": true,
      "license": "MIT",
      "dependencies": {
@@ -2010,9 +2013,9 @@
      }
    },
    "node_modules/acorn": {
-      "version": "8.14.1",
+      "version": "8.15.0",
-      "resolved": "https://registry.npmjs.org/acorn/-/acorn-8.14.1.tgz",
+      "resolved": "https://registry.npmjs.org/acorn/-/acorn-8.15.0.tgz",
-      "integrity": "sha512-OvQ/2pUDKmgfCg++xsTX1wGxfTaszcHVcTctW4UJB4hibJx2HXxxO5UmVgyjMa+ZDsiaf5wWLXYpRWMmBI0QHg==",
+      "integrity": "sha512-NZyJarBfL7nWwIq+FDL6Zp/yHEhePMNnnJ0y3qfieCrmNvYct8uvtiV41UvlSe6apAfk0fY1FbWx+NwfmpvtTg==",
      "dev": true,
      "license": "MIT",
      "bin": {
@@ -2080,9 +2083,9 @@
      "license": "MIT"
    },
    "node_modules/brace-expansion": {
-      "version": "1.1.11",
+      "version": "1.1.12",
-      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.11.tgz",
+      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.12.tgz",
-      "integrity": "sha512-iCuPHDFgrHX7H2vEI/5xpz07zSHB00TpugqhmYtVmMO6518mCuRMoOYFldEBl0g187ufozdaHgWKcYFb61qGiA==",
+      "integrity": "sha512-9T9UjW3r0UW5c1Q7GTwllptXwhvYmEzFhzMfZ9H7FQWt+uZePjZPjBP/W1ZEyZ1twGWom5/56TF4lPcqjnDHcg==",
      "dev": true,
      "license": "MIT",
      "dependencies": {
@@ -2380,33 +2383,32 @@
      }
    },
    "node_modules/eslint": {
-      "version": "9.28.0",
+      "version": "9.39.1",
-      "resolved": "https://registry.npmjs.org/eslint/-/eslint-9.28.0.tgz",
+      "resolved": "https://registry.npmjs.org/eslint/-/eslint-9.39.1.tgz",
-      "integrity": "sha512-ocgh41VhRlf9+fVpe7QKzwLj9c92fDiqOj8Y3Sd4/ZmVA4Btx4PlUYPq4pp9JDyupkf1upbEXecxL2mwNV7jPQ==",
+      "integrity": "sha512-BhHmn2yNOFA9H9JmmIVKJmd288g9hrVRDkdoIgRCRuSySRUHH7r/DI6aAXW9T1WwUuY3DFgrcaqB+deURBLR5g==",
      "dev": true,
      "license": "MIT",
      "dependencies": {
-        "@eslint-community/eslint-utils": "^4.2.0",
+        "@eslint-community/eslint-utils": "^4.8.0",
        "@eslint-community/regexpp": "^4.12.1",
-        "@eslint/config-array": "^0.20.0",
+        "@eslint/config-array": "^0.21.1",
-        "@eslint/config-helpers": "^0.2.1",
+        "@eslint/config-helpers": "^0.4.2",
-        "@eslint/core": "^0.14.0",
+        "@eslint/core": "^0.17.0",
        "@eslint/eslintrc": "^3.3.1",
-        "@eslint/js": "9.28.0",
+        "@eslint/js": "9.39.1",
-        "@eslint/plugin-kit": "^0.3.1",
+        "@eslint/plugin-kit": "^0.4.1",
        "@humanfs/node": "^0.16.6",
        "@humanwhocodes/module-importer": "^1.0.1",
        "@humanwhocodes/retry": "^0.4.2",
        "@types/estree": "^1.0.6",
        "@types/json-schema": "^7.0.15",
        "ajv": "^6.12.4",
        "chalk": "^4.0.0",
        "cross-spawn": "^7.0.6",
        "debug": "^4.3.2",
        "escape-string-regexp": "^4.0.0",
-        "eslint-scope": "^8.3.0",
+        "eslint-scope": "^8.4.0",
-        "eslint-visitor-keys": "^4.2.0",
+        "eslint-visitor-keys": "^4.2.1",
-        "espree": "^10.3.0",
+        "espree": "^10.4.0",
        "esquery": "^1.5.0",
        "esutils": "^2.0.2",
        "fast-deep-equal": "^3.1.3",
@@ -2464,9 +2466,9 @@
      }
    },
    "node_modules/eslint-scope": {
-      "version": "8.3.0",
+      "version": "8.4.0",
-      "resolved": "https://registry.npmjs.org/eslint-scope/-/eslint-scope-8.3.0.tgz",
+      "resolved": "https://registry.npmjs.org/eslint-scope/-/eslint-scope-8.4.0.tgz",
-      "integrity": "sha512-pUNxi75F8MJ/GdeKtVLSbYg4ZI34J6C0C7sbL4YOp2exGwen7ZsuBqKzUhXd0qMQ362yET3z+uPwKeg/0C2XCQ==",
+      "integrity": "sha512-sNXOfKCn74rt8RICKMvJS7XKV/Xk9kA7DyJr8mJik3S7Cwgy3qlkkmyS2uQB3jiJg6VNdZd/pDBJu0nvG2NlTg==",
      "dev": true,
      "license": "BSD-2-Clause",
      "dependencies": {
@@ -2481,9 +2483,9 @@
      }
    },
    "node_modules/eslint-visitor-keys": {
-      "version": "4.2.0",
+      "version": "4.2.1",
-      "resolved": "https://registry.npmjs.org/eslint-visitor-keys/-/eslint-visitor-keys-4.2.0.tgz",
+      "resolved": "https://registry.npmjs.org/eslint-visitor-keys/-/eslint-visitor-keys-4.2.1.tgz",
-      "integrity": "sha512-UyLnSehNt62FFhSwjZlHmeokpRK59rcz29j+F1/aDgbkbRTk7wIc9XzdoasMUbRNKDM0qQt/+BJ4BrpFeABemw==",
+      "integrity": "sha512-Uhdk5sfqcee/9H/rCOJikYz67o0a2Tw2hGRPOG2Y1R2dg7brRe1uG0yaNQDHu+TO/uQPF/5eCapvYSmHUjt7JQ==",
      "dev": true,
      "license": "Apache-2.0",
      "engines": {
@@ -2494,15 +2496,15 @@
      }
    },
    "node_modules/espree": {
-      "version": "10.3.0",
+      "version": "10.4.0",
-      "resolved": "https://registry.npmjs.org/espree/-/espree-10.3.0.tgz",
+      "resolved": "https://registry.npmjs.org/espree/-/espree-10.4.0.tgz",
-      "integrity": "sha512-0QYC8b24HWY8zjRnDTL6RiHfDbAWn63qb4LMj1Z4b076A4une81+z03Kg7l7mn/48PUTqoLptSXez8oknU8Clg==",
+      "integrity": "sha512-j6PAQ2uUr79PZhBjP5C5fhl8e39FmRnOjsD5lGnWrFU8i2G776tBK7+nP8KuQUTTyAZUwfQqXAgrVH5MbH9CYQ==",
      "dev": true,
      "license": "BSD-2-Clause",
      "dependencies": {
-        "acorn": "^8.14.0",
+        "acorn": "^8.15.0",
        "acorn-jsx": "^5.3.2",
-        "eslint-visitor-keys": "^4.2.0"
+        "eslint-visitor-keys": "^4.2.1"
      },
      "engines": {
        "node": "^18.18.0 || ^20.9.0 || >=21.1.0"
@@ -2852,9 +2854,9 @@
      "license": "MIT"
    },
    "node_modules/js-yaml": {
-      "version": "4.1.0",
+      "version": "4.1.1",
-      "resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.1.0.tgz",
+      "resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.1.1.tgz",
-      "integrity": "sha512-wpxZs9NoxZaJESJGIZTyDEaYpl0FKSA+FB9aJiyemKhMwkxQg63h4T1KJgUGHpTqPDNRcmmYLugrRjJlBtWvRA==",
+      "integrity": "sha512-qQKT4zQxXl8lLwBtHMWwaTcGfFOZviOJet3Oy/xmGk2gZH677CJM9EvtfdSkgWcATZhj/55JZ0rmy3myCT5lsA==",
      "dev": true,
      "license": "MIT",
      "dependencies": {
@@ -2,7 +2,7 @@ import { useAPI } from "../contexts/APIProvider";
 import { useMemo } from "react";
 const ConnectionStatusIcon = () => {
-  const { connectionStatus } = useAPI();
+  const { connectionStatus, versionInfo } = useAPI();
  const eventStatusColor = useMemo(() => {
    switch (connectionStatus) {
@@ -17,7 +17,7 @@ const ConnectionStatusIcon = () => {
  }, [connectionStatus]);
  return (
-    <div className="flex items-center" title={`event stream: ${connectionStatus}`}>
+    <div className="flex items-center" title={`Event Stream: ${connectionStatus ?? 'unknown'}\nAPI Version: ${versionInfo?.version ?? 'unknown'}\nCommit Hash: ${versionInfo?.commit?.substring(0,7) ?? 'unknown'}\nBuild Date: ${versionInfo?.build_date ?? 'unknown'}`}>
      <span className={`inline-block w-3 h-3 rounded-full ${eventStatusColor} mr-2`}></span>
    </div>
  );
@@ -23,6 +23,7 @@ interface APIProviderType {
  upstreamLogs: string;
  metrics: Metrics[];
  connectionStatus: ConnectionState;
  versionInfo: VersionInfo;
 }
 interface Metrics {
@@ -41,11 +42,18 @@ interface LogData {
  source: "upstream" | "proxy";
  data: string;
 }
 interface APIEventEnvelope {
  type: "modelStatus" | "logData" | "metrics";
  data: string;
 }
 interface VersionInfo {
  build_date: string;
  commit: string;
  version: string;
 }
 const APIContext = createContext<APIProviderType | undefined>(undefined);
 type APIProviderProps = {
  children: ReactNode;
@@ -59,6 +67,11 @@ export function APIProvider({ children, autoStartAPIEvents = true }: APIProvider
  const [upstreamLogs, setUpstreamLogs] = useState("");
  const [metrics, setMetrics] = useState<Metrics[]>([]);
  const [connectionStatus, setConnectionState] = useState<ConnectionState>("disconnected");
  const [versionInfo, setVersionInfo] = useState<VersionInfo>({
    build_date: "unknown",
    commit: "unknown",
    version: "unknown"
  });
  //const apiEventSource = useRef<EventSource | null>(null);
  const [models, setModels] = useState<Model[]>([]);
@@ -152,6 +165,26 @@ export function APIProvider({ children, autoStartAPIEvents = true }: APIProvider
    connect();
  }, []);
  useEffect(() => {
    // fetch version 
    const fetchVersion = async () => {
      try {
        const response = await fetch("/api/version");
        if (!response.ok) {
          throw new Error(`HTTP error! status: ${response.status}`);
        }
        const data: VersionInfo = await response.json();
        setVersionInfo(data);
      } catch (error) {
        console.error(error);
      }
    };
    if (connectionStatus === 'connected') {
      fetchVersion();
    }
  }, [connectionStatus]);
  useEffect(() => {
    if (autoStartAPIEvents) {
      enableAPIEvents(true);
@@ -230,8 +263,9 @@ export function APIProvider({ children, autoStartAPIEvents = true }: APIProvider
      upstreamLogs,
      metrics,
      connectionStatus,
      versionInfo,
    }),
-    [models, listModels, unloadAllModels, loadModel, enableAPIEvents, proxyLogs, upstreamLogs, metrics]
+    [models, listModels, unloadAllModels, unloadSingleModel, loadModel, enableAPIEvents, proxyLogs, upstreamLogs, metrics, connectionStatus, versionInfo]
  );
  return <APIContext.Provider value={value}>{children}</APIContext.Provider>;
Author	SHA1	Message	Date
Benson Wong	bccce5fa19	go.mod,ui/package-lock.json: dependency and security updates (#418 )	2025-11-29 22:27:22 -08:00
Benson Wong	c968da1b73	proxy: add support for anthropic v1/messages api (#417 ) * proxy: add support for anthropic v1/messages api * proxy: restrict loading message to /v1/chat/completions	2025-11-29 22:09:07 -08:00
Ryan Steed	a883d68d4f	feat: Add support for custom llama.cpp base image and forked llama-swap repositories (#396 ) * feat: Add support for custom llama.cpp base image and forked llama-swap repositories - Introduce BASE_LLAMACPP_IMAGE env var to customize llama.cpp base image - Introduce LS_REPO env var to customize llama-swap source - Use GITHUB_REPOSITORY env var to automatically detect forked repos - Update container tagging to use dynamic repo paths - Pass build args for BASE_IMAGE and LS_REPO to Containerfile - Enable flexible release downloads from forked repositories * chore: quote entire curl options, appease coderabbitai	2025-11-29 20:59:15 -08:00
Ryan Steed	b1dec8b735	docker: build both root and non-root container images (#412 ) Change the user back to root for containers. Additionally, built a "non-root" labeled container for users who wish to have the additional security of running llama-swap as a lower privileged user.	2025-11-25 10:44:13 -08:00
Nikesh Parajuli	06523d8c1e	feat: add platform-specific process attributes support (#411 ) Fixes issues on Windows showing new windows for every process llama-swap spawns.	2025-11-24 21:39:56 -08:00
Ryan Steed	86e9b93c37	proxy,ui: add version endpoint and display version info in UI (#395 ) - Add /api/version endpoint to ProxyManager that returns build date, commit hash, and version - Implement SetVersion method to configure version info in ProxyManager - Add version info fetching to APIProvider and display in ConnectionStatus component - Include version info in UI context and update dependencies - Add tests for version endpoint functionality	2025-11-17 10:43:47 -08:00
Ryan Steed	3acace810f	proxy: add configurable logging timestamp format (#401 ) introduces a new configuration option logTimeFormat that allows customizing the timestamp in log messages using golang's built in time format constants. The default remains no timestamp.	2025-11-16 10:21:59 -08:00
Ryan Steed	554d29e87d	feat: enhance model listing to include aliases (#400 ) introduce includeAliasesInList as a new configuration setting (default false) that includes aliases in v1/models Fixes #399	2025-11-15 14:35:26 -08:00
Benson Wong	3567b7df08	Update image in README.md for web UI section	2025-11-08 15:29:37 -08:00
Benson Wong	38738525c9	config.example.yaml: add modeline for schema validation	2025-11-08 15:08:55 -08:00
Benson Wong	c0fc858193	Add configuration file JSON schema (#393 ) * add json schema for configuration * add GH action to validate schema	2025-11-08 15:04:14 -08:00
Benson Wong	b429349e8a	add /ui/ to wol-proxy polling (#388 )	2025-11-08 14:16:12 -08:00
Ryan Steed	eab2efd7b5	feat: improve llama.cpp base image tag for cpu (#391 ) Refactor the container build script to resolve llama.cpp base image for CPU, also tag these builds accordingly. - For CPU containers, now fetch the latest 'server' tagged llama.cpp image instead of using a generic 'server' tag - Cleans up the docker build command to use dynamic BASE_TAG variable - Maintains existing push functionality for built images	2025-11-08 09:56:49 -08:00
Benson Wong	6aedbe121a	cmd/wol-proxy: show a loading page for / (#381 ) When requesting / wol-proxy will show a loading page that polls /status every second. When the upstream server is ready the loading page will refresh causing the actual root page to be displayed	2025-11-03 19:37:06 -08:00
Ryan Steed	b24467ab89	fix: update containerfile user/group management commands (#379 ) - Replace `addgroup` with `groupadd` for system group creation - Replace `adduser` with `useradd` for system user creation - Maintain same functionality while using more standard POSIX commands	2025-11-03 17:17:40 -05:00
Benson Wong	12b69fb718	proxy: recover from panic in Process.statusUpdate (#378 ) Process.statusUpdate() panics when it can not write data, usually from a client disconnect. Since it runs in a goroutine and did not have a recover() the result was a crash. ref: https://github.com/mostlygeek/llama-swap/discussions/326#discussioncomment-14856197	2025-11-03 05:30:09 -08:00
Ryan Steed	f91a8b2462	refactor: update Containerfile to support non-root user execution and improve security (#368 ) Set default container user/group to lower privilege app user * refactor: update Containerfile to support non-root user execution and improve security - Updated LS_VER argument from 89 to 170 to use the latest version - Added UID/GID arguments with default values of 0 (root) for backward compatibility - Added USER_HOME environment variable set to /root - Implemented conditional user/group creation logic that only runs when UID/GID are not 0 - Created necessary directory structure with proper ownership using mkdir and chown commands - Switched to non-root user execution for improved security posture - Updated COPY instruction to use --chown flag for proper file ownership * chore: update containerfile to use non-root user with proper UID/GID - Changed default UID and GID from 0 (root) to 10001 for security best practices - Updated USER_HOME from /root to /app to avoid running as root user	2025-10-31 17:01:04 -07:00
Benson Wong	a89b803d4a	Stream loading state when swapping models (#371 ) Swapping models can take a long time and leave a lot of silence while the model is loading. Rather than silently load the model in the background, this PR allows llama-swap to send status updates in the reasoning_content of a streaming chat response. Fixes: #366	2025-10-29 00:09:39 -07:00
Benson Wong	f852689104	proxy: add panic recovery to Process.ProxyRequest (#363 ) Switching to use httputil.ReverseProxy in #342 introduced a possible panic if a client disconnects while streaming the body. Since llama-swap does not use http.Server the recover() is not automatically there. - introduce a recover() in Process.ProxyRequest to recover and log the event - add TestProcess_ReverseProxyPanicIsHandled to reproduce and test the fix fixes: #362	2025-10-25 20:40:05 -07:00