proxy: add support for user defined metadata in model configs (#333)

Changes: 

- add Metadata key to ModelConfig
- include metadata in /v1/models under meta.llamaswap key
- add recursive macro substitution into Metadata
- change macros at global and model level to be any scalar type

Note: 

This is the first mostly AI generated change to llama-swap. See #333 for notes about the workflow and approach to AI going forward.

CLAUDE.md

@@ -0,0 +1,43 @@
+# Project: llama-swap
+
+## Project Description:
+
+llama-swap is a light weight, transparent proxy server that provides automatic model swapping to llama.cpp's server.
+
+## Tech stack
+
+- golang
+- typescript, vite and react for UI (ui/)
+
+## Testing
+
+- `make test-dev` - Use this when making iterative changes. Runs `go test` and `staticcheck`. Fix any static checking errors.
+- `make test-all` - runs at the end before completing work. Includes long running concurrency tests.
+
+## Workflow Tasks
+
+### Plan Improvements
+
+Work plans are located in ai-plans/. Plans written by the user may be incomplete, contain inconsistencies or errors.
+
+When the user asks to improve a plan follow these guidelines for expanding and improving it.
+
+- Identify any inconsistencies.
+- Expand plans out to be detailed specification of requirements and changes to be made.
+- Plans should have at least these sections:
+  - Title - very short, describes changes
+  - Overview: A more detailed summary of goal and outcomes desired
+  - Design Requirements: Detailed descriptions of what needs to be done
+  - Testing Plan: Tests to be implemented
+  - Checklist: A detailed list of changes to be made
+
+Look for "plan expansion" as explicit instructions to improve a plan.
+
+### Implementation of plans
+
+When the user says "paint it", respond with "commencing automated assembly". Then implement the changes as described by the plan. Update the checklist as you complete items.
+
+## General Rules
+
+- when summarizing changes only include details that require further action (action items)
+- when there are no action items, just say "Done."

Makefile

@@ -23,11 +23,17 @@ proxy/ui_dist/placeholder.txt:
 	mkdir -p proxy/ui_dist
 	touch $@
 
-test: proxy/ui_dist/placeholder.txt
-	go test -short -v -count=1 ./proxy
+# use cached test results while developing
+test-dev: proxy/ui_dist/placeholder.txt
+	go test -short ./proxy/...
+	staticcheck ./proxy/... || true
 
+test: proxy/ui_dist/placeholder.txt
+	go test -short -count=1 ./proxy/...
+
+# for CI - full test (takes longer)
 test-all: proxy/ui_dist/placeholder.txt
-	go test -v -count=1 ./proxy
+	go test -count=1 ./proxy/...
 
 ui/node_modules:
 	cd ui && npm install
@@ -81,4 +87,4 @@ release:
 	git tag "$$new_tag";
 
 # Phony targets
-.PHONY: all clean ui mac linux windows simple-responder
+.PHONY: all clean ui mac linux windows simple-responder test test-all test-dev

README.md

@@ -7,7 +7,7 @@
 
 llama-swap is a light weight, transparent proxy server that provides automatic model swapping to llama.cpp's server.
 
-Written in golang, it is very easy to install (single binary with no dependencies) and configure (single yaml file). To get started, download a pre-built binary or use the provided docker images.
+Written in golang, it is very easy to install (single binary with no dependencies) and configure (single yaml file). To get started, download a pre-built binary, a provided docker images or Homebrew.
 
 ## Features:
 
@@ -18,9 +18,12 @@ Written in golang, it is very easy to install (single binary with no dependencie
   - `v1/completions`
   - `v1/chat/completions`
   - `v1/embeddings`
-  - `v1/rerank`, `v1/reranking`, `rerank`
   - `v1/audio/speech` ([#36](https://github.com/mostlygeek/llama-swap/issues/36))
   - `v1/audio/transcriptions` ([docs](https://github.com/mostlygeek/llama-swap/issues/41#issuecomment-2722637867))
+- ✅ llama-server (llama.cpp) supported endpoints:
+  - `v1/rerank`, `v1/reranking`, `/rerank`
+  - `/infill` - for code infilling
+  - `/completion` - for completion endpoint
 - ✅ llama-swap custom API endpoints
   - `/ui` - web UI
   - `/log` - remote log monitoring
@@ -31,7 +34,7 @@ Written in golang, it is very easy to install (single binary with no dependencie
 - ✅ Run multiple models at once with `Groups` ([#107](https://github.com/mostlygeek/llama-swap/issues/107))
 - ✅ Automatic unloading of models after timeout by setting a `ttl`
 - ✅ Use any local OpenAI compatible server (llama.cpp, vllm, tabbyAPI, etc)
-- ✅ Reliable Docker and Podman support with `cmdStart` and `cmdStop`
+- ✅ Reliable Docker and Podman support using `cmd` and `cmdStop` together
 - ✅ Full control over server settings per model
 - ✅ Preload models on startup with `hooks` ([#235](https://github.com/mostlygeek/llama-swap/pull/235))
 
@@ -70,6 +73,30 @@ However, there are many more capabilities that llama-swap supports:
 
 See the [configuration documentation](https://github.com/mostlygeek/llama-swap/wiki/Configuration) in the wiki all options and examples.
 
+## Reverse Proxy Configuration (nginx)
+
+If you deploy llama-swap behind nginx, disable response buffering for streaming endpoints. By default, nginx buffers responses which breaks Server‑Sent Events (SSE) and streaming chat completion. ([#236](https://github.com/mostlygeek/llama-swap/issues/236))
+
+Recommended nginx configuration snippets:
+
+```nginx
+# SSE for UI events/logs
+location /api/events {
+    proxy_pass http://your-llama-swap-backend;
+    proxy_buffering off;
+    proxy_cache off;
+}
+
+# Streaming chat completions (stream=true)
+location /v1/chat/completions {
+    proxy_pass http://your-llama-swap-backend;
+    proxy_buffering off;
+    proxy_cache off;
+}
+```
+
+As a safeguard, llama-swap also sets `X-Accel-Buffering: no` on SSE responses. However, explicitly disabling `proxy_buffering` at your reverse proxy is still recommended for reliable streaming behavior.
+
 ## Web UI
 
 llama-swap includes a real time web interface for monitoring logs and models:
@@ -204,4 +231,7 @@ For Python based inference servers like vllm or tabbyAPI it is recommended to ru
 
 ## Star History
 
+> [!NOTE]
+> ⭐️ Star this project to help others discover it! 
+
 [![Star History Chart](https://api.star-history.com/svg?repos=mostlygeek/llama-swap&type=Date)](https://www.star-history.com/#mostlygeek/llama-swap&Date)

ai-plans/issue-264-add-metadata.md

@@ -0,0 +1,292 @@
+# Add Model Metadata Support with Typed Macros
+
+## Overview
+
+Implement support for arbitrary metadata on model configurations that can be exposed through the `/v1/models` API endpoint. This feature extends the existing macro system to support scalar types (string, int, float, bool) instead of only strings, enabling type-safe metadata values.
+
+The metadata will be schemaless, allowing users to define any key-value pairs they need. Macro substitution will work within metadata values, preserving types when macros are used directly and converting to strings when macros are interpolated within strings.
+
+## Design Requirements
+
+### 1. Enhanced Macro System
+
+**Current State:**
+
+- Macros are defined as `map[string]string` at both global and model levels
+- Only string substitution is supported
+- Macros are replaced in: `cmd`, `cmdStop`, `proxy`, `checkEndpoint`, `filters.stripParams`
+
+**Required Changes:**
+
+- Change `MacroList` type from `map[string]string` to `map[string]any`
+- Support scalar types: `string`, `int`, `float64`, `bool`
+- Implement type-preserving macro substitution:
+  - Direct macro usage (`key: ${macro}`) preserves the macro's type
+  - Interpolated usage (`key: "text ${macro}"`) converts to string
+- Add validation to ensure macro values are scalar types only
+- Update existing macro substitution logic in [proxy/config/config.go](proxy/config/config.go) to handle `any` types
+
+**Implementation Details:**
+
+- Create a generic helper function to perform macro substitution that:
+  - Takes a value of type `any`
+  - Recursively processes maps, slices, and scalar values
+  - Replaces `${macro_name}` patterns with macro values
+  - Preserves types for direct substitution
+  - Converts to strings for interpolated substitution
+- Update `validateMacro()` function to accept `any` type and validate scalar types
+- Maintain backward compatibility with existing string-only macros
+
+### 2. Metadata Field in ModelConfig
+
+**Location:** [proxy/config/model_config.go](proxy/config/model_config.go)
+
+**Required Changes:**
+
+- Add `Metadata map[string]any` field to `ModelConfig` struct
+- Support YAML unmarshaling of arbitrary structures (maps, arrays, scalars)
+- Apply macro substitution to metadata values during config loading
+
+**Schema Requirements:**
+
+- Metadata is optional (default: empty/nil map)
+- Supports nested structures (objects within objects, arrays, etc.)
+- All string values within metadata undergo macro substitution
+- Type preservation rules apply as described above
+
+### 3. Macro Substitution in Metadata
+
+**Location:** [proxy/config/config.go](proxy/config/config.go) in `LoadConfigFromReader()`
+
+**Process Flow:**
+
+1. After loading YAML configuration
+2. After model-level and global macro merging
+3. Apply macro substitution to `ModelConfig.Metadata` field
+4. Use the same merged macros available to `cmd`, `proxy`, etc.
+5. Process recursively through all nested structures
+
+**Substitution Rules:**
+
+- `port: ${PORT}` → keeps integer type from PORT macro
+- `temperature: ${temp}` → keeps float type from temp macro
+- `note: "Running on ${PORT}"` → converts to string `"Running on 10001"`
+- Arrays and nested objects are processed recursively
+- Unknown macros should cause configuration load error (consistent with existing behavior)
+
+### 4. API Response Updates
+
+**Location:** [proxy/proxymanager.go:350](proxy/proxymanager.go#L350) `listModelsHandler()`
+
+**Current Behavior:**
+
+- Returns model records with: `id`, `object`, `created`, `owned_by`
+- Optionally includes: `name`, `description`
+
+**Required Changes:**
+
+- Add metadata to each model record under the key `llamaswap_meta`
+- Only include `llamaswap_meta` if metadata is non-empty
+- Preserve all types when marshaling to JSON
+- Maintain existing sorting by model ID
+
+**Example Response:**
+
+```json
+{
+  "object": "list",
+  "data": [
+    {
+      "id": "llama",
+      "object": "model",
+      "created": 1234567890,
+      "owned_by": "llama-swap",
+      "name": "llama 3.1 8B",
+      "description": "A small but capable model",
+      "llamaswap_meta": {
+        "port": 10001,
+        "temperature": 0.7,
+        "note": "The llama is running on port 10001 temp=0.7, context=16384",
+        "a_list": [1, 1.23, "macros are OK in list and dictionary types: llama"],
+        "an_obj": {
+          "a": "1",
+          "b": 2,
+          "c": [0.7, false, "model: llama"]
+        }
+      }
+    }
+  ]
+}
+```
+
+### 5. Validation and Error Handling
+
+**Macro Validation:**
+
+- Extend `validateMacro()` to accept values of type `any`
+- Verify macro values are scalar types: `string`, `int`, `float64`, `bool`
+- Reject complex types (maps, slices, structs) as macro values
+- Maintain existing validation for macro names and lengths
+
+**Configuration Loading:**
+
+- Fail fast if unknown macros are found in metadata
+- Provide clear error messages indicating which model and field contains errors
+- Ensure macros in metadata follow same rules as macros in cmd/proxy fields
+
+## Testing Plan
+
+### Test 1: Model-Level Macros with Different Types
+
+**File:** [proxy/config/model_config_test.go](proxy/config/model_config_test.go)
+
+**Test Cases:**
+
+- Define model with macros of each scalar type
+- Verify metadata correctly substitutes and preserves types
+- Test direct substitution (`port: ${PORT}`)
+- Test string interpolation (`note: "Port is ${PORT}"`)
+- Verify nested objects and arrays work correctly
+
+### Test 2: Global and Model Macro Precedence
+
+**File:** [proxy/config/config_test.go](proxy/config/config_test.go)
+
+**Test Cases:**
+
+- Define same macro at global and model level with different types
+- Verify model-level macro takes precedence
+- Test metadata uses correct macro value
+- Verify type is preserved from the winning macro
+
+### Test 3: Macro Validation
+
+**File:** [proxy/config/config_test.go](proxy/config/config_test.go)
+
+**Test Cases:**
+
+- Test that complex types (maps, arrays) are rejected as macro values
+  - Verify error message includes: macro name and type that was rejected
+- Test that scalar types (string, int, float, bool) are accepted
+  - Each type should load without error
+- Test macro name validation still works with `any` types
+  - Invalid characters, reserved names, length limits should still be enforced
+
+### Test 4: Metadata in API Response
+
+**File:** [proxy/proxymanager_test.go](proxy/proxymanager_test.go)
+
+**Existing Test:** `TestProxyManager_ListModelsHandler`
+
+**Test Cases:**
+
+- Model with metadata → verify `llamaswap_meta` key appears
+- Model without metadata → verify `llamaswap_meta` key is absent
+- Verify all types are correctly marshaled to JSON
+- Verify nested structures are preserved
+- Verify macro substitution has occurred before serialization
+
+### Test 5: Unknown Macros in Metadata
+
+**File:** [proxy/config/config_test.go](proxy/config/config_test.go)
+
+**Test Cases:**
+
+- Use undefined macro in metadata
+- Verify configuration loading fails with clear error
+- Error should indicate model name and that macro is undefined
+
+### Test 6: Recursive Substitution
+
+**File:** [proxy/config/config_test.go](proxy/config/config_test.go)
+
+**Test Cases:**
+
+- Metadata with deeply nested structures
+- Arrays containing objects with macros
+- Objects containing arrays with macros
+- Mixed string interpolation and direct substitution at various nesting levels
+
+## Checklist
+
+### Configuration Schema Changes
+
+- [x] Change `MacroList` type from `map[string]string` to `map[string]any` in [proxy/config/config.go:19](proxy/config/config.go#L19)
+- [x] Add `Metadata map[string]any` field to `ModelConfig` struct in [proxy/config/model_config.go:37](proxy/config/model_config.go#L37)
+- [x] Update `validateMacro()` function signature to accept `any` type for values
+- [x] Add validation logic to ensure macro values are scalar types only
+
+### Macro Substitution Logic
+
+- [x] Create generic recursive function `substituteMetadataMacros()` to handle `any` types
+- [x] Implement type-preserving direct substitution logic
+- [x] Implement string interpolation with type conversion
+- [x] Handle maps: recursively process all values
+- [x] Handle slices: recursively process all elements
+- [x] Handle scalar types: perform string-based macro substitution if value is string
+- [x] Integrate macro substitution into `LoadConfigFromReader()` after existing macro expansion
+- [x] Update existing macro substitution calls to use merged macros with correct types
+
+### API Response Changes
+
+- [x] Modify `listModelsHandler()` in [proxy/proxymanager.go:350](proxy/proxymanager.go#L350)
+- [x] Add `llamaswap_meta` field to model records when metadata exists
+- [x] Ensure empty metadata results in omitted `llamaswap_meta` key
+- [x] Verify JSON marshaling preserves all types correctly
+
+### Testing - Config Package
+
+- [x] Add test for string macros in metadata: [proxy/config/config_test.go](proxy/config/config_test.go)
+- [x] Add test for int macros in metadata: [proxy/config/config_test.go](proxy/config/config_test.go)
+- [x] Add test for float macros in metadata: [proxy/config/config_test.go](proxy/config/config_test.go)
+- [x] Add test for bool macros in metadata: [proxy/config/config_test.go](proxy/config/config_test.go)
+- [x] Add test for string interpolation in metadata: [proxy/config/config_test.go](proxy/config/config_test.go)
+- [x] Add test for model-level macro precedence: [proxy/config/config_test.go](proxy/config/config_test.go)
+- [x] Add test for nested structures in metadata: [proxy/config/config_test.go](proxy/config/config_test.go)
+- [x] Add test for unknown macro in metadata (should error): [proxy/config/config_test.go](proxy/config/config_test.go)
+- [x] Add test for invalid macro type validation: [proxy/config/config_test.go](proxy/config/config_test.go)
+
+### Testing - Model Config Package
+
+- [x] Add test cases to [proxy/config/model_config_test.go](proxy/config/model_config_test.go) for metadata unmarshaling
+- [x] Test metadata with various scalar types
+- [x] Test metadata with nested objects and arrays
+
+### Testing - Proxy Manager
+
+- [x] Update `TestProxyManager_ListModelsHandler` in [proxy/proxymanager_test.go](proxy/proxymanager_test.go)
+- [x] Add test case for model with metadata
+- [x] Add test case for model without metadata
+- [x] Verify `llamaswap_meta` key presence/absence
+- [x] Verify type preservation in JSON output
+- [x] Verify macro substitution has occurred
+
+### Documentation
+
+- [x] Verify [config.example.yaml](config.example.yaml) already has complete metadata examples (lines 149-171)
+- [x] No additional documentation needed per project instructions
+
+## Known Issues and Considerations
+
+### Inconsistencies
+
+None identified. The plan references the correct existing example in [config.example.yaml:149-171](config.example.yaml#L149-L171).
+
+### Design Decisions
+
+1. **Why `llamaswap_meta` instead of merging into record?**
+
+   - Avoids potential collisions with OpenAI API standard fields
+   - Makes it clear this is llama-swap specific metadata
+   - Easier for clients to distinguish standard vs. custom fields
+
+2. **Why support nested structures?**
+
+   - Provides maximum flexibility for users
+   - Aligns with the schemaless design principle
+   - Example config already demonstrates this capability
+
+3. **Why validate macro types?**
+   - Prevents confusing behavior (e.g., substituting a map)
+   - Makes configuration errors explicit at load time
+   - Simpler implementation and testing

config.example.yaml

@@ -3,14 +3,15 @@
 #
 # 💡 Tip - Use an LLM with this file!
 # ====================================
-#  This example configuration is written to be LLM friendly! Try
+#  This example configuration is written to be LLM friendly. Try
 #  copying this file into an LLM and asking it to explain or generate
 #  sections for you.
 # ====================================
-#
+
+# Usage notes:
 # - Below are all the available configuration options for llama-swap.
-# - Settings with a default value, or noted as optional can be omitted.
-# - Settings that are marked required must be in your configuration file
+# - Settings noted as "required" must be in your configuration file
+# - Settings noted as "optional" can be omitted
 
 # healthCheckTimeout: number of seconds to wait for a model to be ready to serve requests
 # - optional, default: 120
@@ -34,26 +35,41 @@ metricsMaxInMemory: 1000
 # - it is automatically incremented for every model that uses it
 startPort: 10001
 
-# macros: sets a dictionary of string:string pairs
+# macros: a dictionary of string substitutions
 # - optional, default: empty dictionary
-# - these are reusable snippets
-# - used in a model's cmd, cmdStop, proxy and checkEndpoint
+# - macros are reusable snippets
+# - used in a model's cmd, cmdStop, proxy, checkEndpoint, filters.stripParams
 # - useful for reducing common configuration settings
+# - macro names are strings and must be less than 64 characters
+# - macro names must match the regex ^[a-zA-Z0-9_-]+$
+# - macro names must not be a reserved name: PORT or MODEL_ID
+# - macro values must be less than 1024 characters
+#
+# Important: do not nest macros inside other macros; expansion is single-pass
 macros:
   "latest-llama": >
     /path/to/llama-server/llama-server-ec9e0301
     --port ${PORT}
+  "default_ctx": "4096"
 
 # models: a dictionary of model configurations
 # - required
 # - each key is the model's ID, used in API requests
 # - model settings have default values that are used if they are not defined here
-# - below are examples of the various settings a model can have:
-# - available model settings: env, cmd, cmdStop, proxy, aliases, checkEndpoint, ttl, unlisted
+# - the model's ID is available in the ${MODEL_ID} macro, also available in macros defined above
+# - below are examples of the all the settings a model can have
 models:
 
   # keys are the model names used in API requests
   "llama":
+    # macros: a dictionary of string substitutions specific to this model
+    # - optional, default: empty dictionary
+    # - macros defined here override macros defined in the global macros section
+    # - model level macros follow the same rules as global macros
+    macros:
+      "default_ctx": 16384
+      "temp": 0.7
+
     # cmd: the command to run to start the inference server.
     # - required
     # - it is just a string, similar to what you would run on the CLI
@@ -63,6 +79,8 @@ models:
       # ${latest-llama} is a macro that is defined above
       ${latest-llama}
       --model path/to/llama-8B-Q4_K_M.gguf
+      --ctx-size ${default_ctx}
+      --temperature ${temp}
 
     # name: a display name for the model
     # - optional, default: empty string
@@ -99,49 +117,84 @@ models:
 
     # checkEndpoint: URL path to check if the server is ready
     # - optional, default: /health
-    # - use "none" to skip endpoint ready checking
     # - endpoint is expected to return an HTTP 200 response
-    # - all requests wait until the endpoint is ready (or fails)
+    # - all requests wait until the endpoint is ready or fails
+    # - use "none" to skip endpoint health checking
     checkEndpoint: /custom-endpoint
 
-    # ttl: automatically unload the model after this many seconds
+    # ttl: automatically unload the model after ttl seconds
     # - optional, default: 0
     # - ttl values must be a value greater than 0
     # - a value of 0 disables automatic unloading of the model
     ttl: 60
 
-    # useModelName: overrides the model name that is sent to upstream server
+    # useModelName: override the model name that is sent to upstream server
     # - optional, default: ""
-    # - useful when the upstream server expects a specific model name or format
+    # - useful for when the upstream server expects a specific model name that
+    #   is different from the model's ID
     useModelName: "qwen:qwq"
 
     # filters: a dictionary of filter settings
     # - optional, default: empty dictionary
+    # - only stripParams is currently supported
     filters:
-      # strip_params: a comma separated list of parameters to remove from the request
+      # stripParams: a comma separated list of parameters to remove from the request
       # - optional, default: ""
-      # - useful for preventing overriding of default server params by requests
-      # - `model` parameter is never removed
+      # - useful for server side enforcement of sampling parameters
+      # - the `model` parameter can never be removed
       # - can be any JSON key in the request body
       # - recommended to stick to sampling parameters
-      strip_params: "temperature, top_p, top_k"
+      stripParams: "temperature, top_p, top_k"
+
+    # metadata: a dictionary of arbitrary values that are included in /v1/models
+    # - optional, default: empty dictionary
+    # - while metadata can contains complex types it is recommended to keep it simple
+    # - metadata is only passed through in /v1/models responses
+    metadata:
+      # port will remain an integer
+      port: ${PORT}
+
+      # the ${temp} macro will remain a float
+      temperature: ${temp}
+      note: "The ${MODEL_ID} is running on port ${PORT} temp=${temp}, context=${default_ctx}"
+
+      a_list:
+        - 1
+        - 1.23
+        - "macros are OK in list and dictionary types: ${MODEL_ID}"
+
+      an_obj:
+        a: "1"
+        b: 2
+        # objects can contain complex types with macro substitution
+        # becomes: c: [0.7, false, "model: llama"]
+        c: ["${temp}", false, "model: ${MODEL_ID}"]
+
+    # concurrencyLimit: overrides the allowed number of active parallel requests to a model
+    # - optional, default: 0
+    # - useful for limiting the number of active parallel requests a model can process
+    # - must be set per model
+    # - any number greater than 0 will override the internal default value of 10
+    # - any requests that exceeds the limit will receive an HTTP 429 Too Many Requests response
+    # - recommended to be omitted and the default used
+    concurrencyLimit: 0
 
   # Unlisted model example:
   "qwen-unlisted":
-    # unlisted: true or false
+    # unlisted: boolean, true or false
     # - optional, default: false
-    # - unlisted models do not show up in /v1/models or /upstream lists
+    # - unlisted models do not show up in /v1/models api requests
     # - can be requested as normal through all apis
     unlisted: true
     cmd: llama-server --port ${PORT} -m Llama-3.2-1B-Instruct-Q4_K_M.gguf -ngl 0
 
   # Docker example:
-  # container run times like Docker and Podman can also be used with a
-  # a combination of cmd and cmdStop.
+  # container runtimes like Docker and Podman can be used reliably with
+  # a combination of cmd, cmdStop, and ${MODEL_ID}
   "docker-llama":
     proxy: "http://127.0.0.1:${PORT}"
     cmd: |
-      docker run --name dockertest
+      docker run --name ${MODEL_ID}
       --init --rm -p ${PORT}:8080 -v /mnt/nvme/models:/models
       ghcr.io/ggml-org/llama.cpp:server
       --model '/models/Qwen2.5-Coder-0.5B-Instruct-Q4_K_M.gguf'
@@ -149,24 +202,26 @@ models:
     # cmdStop: command to run to stop the model gracefully
     # - optional, default: ""
     # - useful for stopping commands managed by another system
-    # - on POSIX systems: a SIGTERM is sent for graceful shutdown
-    # - on Windows, taskkill is used
-    # - processes are given 5 seconds to shutdown until they are forcefully killed
     # - the upstream's process id is available in the ${PID} macro
-    cmdStop: docker stop dockertest
+    #
+    # When empty, llama-swap has this default behaviour:
+    # - on POSIX systems: a SIGTERM signal is sent
+    # - on Windows, calls taskkill to stop the process
+    # - processes have 5 seconds to shutdown until forceful termination is attempted
+    cmdStop: docker stop ${MODEL_ID}
 
 # groups: a dictionary of group settings
 # - optional, default: empty dictionary
-# - provide advanced controls over model swapping behaviour.
-# - Using groups some models can be kept loaded indefinitely, while others are swapped out.
-# - model ids must be defined in the Models section
+# - provides advanced controls over model swapping behaviour
+# - using groups some models can be kept loaded indefinitely, while others are swapped out
+# - model IDs must be defined in the Models section
 # - a model can only be a member of one group
 # - group behaviour is controlled via the `swap`, `exclusive` and `persistent` fields
 # - see issue #109 for details
 #
 # NOTE: the example below uses model names that are not defined above for demonstration purposes
 groups:
-  # group1 is same as the default behaviour of llama-swap where only one model is allowed
+  # group1 works the same as the default behaviour of llama-swap where only one model is allowed
   # to run a time across the whole llama-swap instance
   "group1":
     # swap: controls the model swapping behaviour in within the group
@@ -188,10 +243,13 @@ groups:
       - "qwen-unlisted"
 
   # Example:
-  # - in this group all the models can run at the same time
-  # - when a different group loads all running models in this group are unloaded
+  # - in group2 all models can run at the same time
+  # - when a different group is loaded it causes all running models in this group to unload
   "group2":
     swap: false
+
+    # exclusive: false does not unload other groups when a model in group2 is requested
+    # - the models in group2 will be loaded but will not unload any other groups
     exclusive: false
     members:
       - "docker-llama"
@@ -220,7 +278,7 @@ groups:
 # - the only supported hook is on_startup
 hooks:
   # on_startup: a dictionary of actions to perform on startup
-  # - optional, default: empty dictionar
+  # - optional, default: empty dictionary
   # - the only supported action is preload
   on_startup:
         # preload: a list of model ids to load on startup
@@ -229,4 +287,4 @@ hooks:
         # - when preloading multiple models at once, define a group
         #   otherwise models will be loaded and swapped out
     preload:
-      - "llama"
+      - "llama"

llama-swap.go

@@ -16,6 +16,7 @@ import (
 	"github.com/gin-gonic/gin"
 	"github.com/mostlygeek/llama-swap/event"
 	"github.com/mostlygeek/llama-swap/proxy"
+	"github.com/mostlygeek/llama-swap/proxy/config"
 )
 
 var (
@@ -38,13 +39,13 @@ func main() {
 		os.Exit(0)
 	}
 
-	config, err := proxy.LoadConfig(*configPath)
+	conf, err := config.LoadConfig(*configPath)
 	if err != nil {
 		fmt.Printf("Error loading config: %v\n", err)
 		os.Exit(1)
 	}
 
-	if len(config.Profiles) > 0 {
+	if len(conf.Profiles) > 0 {
 		fmt.Println("WARNING: Profile functionality has been removed in favor of Groups. See the README for more information.")
 	}
 
@@ -67,7 +68,7 @@ func main() {
 	// Support for watching config and reloading when it changes
 	reloadProxyManager := func() {
 		if currentPM, ok := srv.Handler.(*proxy.ProxyManager); ok {
-			config, err = proxy.LoadConfig(*configPath)
+			conf, err = config.LoadConfig(*configPath)
 			if err != nil {
 				fmt.Printf("Warning, unable to reload configuration: %v\n", err)
 				return
@@ -75,7 +76,7 @@ func main() {
 
 			fmt.Println("Configuration Changed")
 			currentPM.Shutdown()
-			srv.Handler = proxy.New(config)
+			srv.Handler = proxy.New(conf)
 			fmt.Println("Configuration Reloaded")
 
 			// wait a few seconds and tell any UI to reload
@@ -85,12 +86,12 @@ func main() {
 				})
 			})
 		} else {
-			config, err = proxy.LoadConfig(*configPath)
+			conf, err = config.LoadConfig(*configPath)
 			if err != nil {
 				fmt.Printf("Error, unable to load configuration: %v\n", err)
 				os.Exit(1)
 			}
-			srv.Handler = proxy.New(config)
+			srv.Handler = proxy.New(conf)
 		}
 	}
 

misc/simple-responder/simple-responder.go

@@ -153,6 +153,19 @@ func main() {
 
 	})
 
+	// llama-server compatibility: /completion
+	r.POST("/completion", func(c *gin.Context) {
+		c.Header("Content-Type", "application/json")
+		c.JSON(http.StatusOK, gin.H{
+			"responseMessage": *responseMessage,
+			"usage": gin.H{
+				"completion_tokens": 10,
+				"prompt_tokens":     25,
+				"total_tokens":      35,
+			},
+		})
+	})
+
 	// issue #41
 	r.POST("/v1/audio/transcriptions", func(c *gin.Context) {
 		// Parse the multipart form

proxy/config/config.go

@@ -1,4 +1,4 @@
-package proxy
+package config
 
 import (
 	"fmt"
@@ -6,7 +6,6 @@ import (
 	"os"
 	"regexp"
 	"runtime"
-	"slices"
 	"sort"
 	"strconv"
 	"strings"
@@ -17,101 +16,7 @@ import (
 
 const DEFAULT_GROUP_ID = "(default)"
 
-type ModelConfig struct {
-	Cmd           string   `yaml:"cmd"`
-	CmdStop       string   `yaml:"cmdStop"`
-	Proxy         string   `yaml:"proxy"`
-	Aliases       []string `yaml:"aliases"`
-	Env           []string `yaml:"env"`
-	CheckEndpoint string   `yaml:"checkEndpoint"`
-	UnloadAfter   int      `yaml:"ttl"`
-	Unlisted      bool     `yaml:"unlisted"`
-	UseModelName  string   `yaml:"useModelName"`
-
-	// #179 for /v1/models
-	Name        string `yaml:"name"`
-	Description string `yaml:"description"`
-
-	// Limit concurrency of HTTP requests to process
-	ConcurrencyLimit int `yaml:"concurrencyLimit"`
-
-	// Model filters see issue #174
-	Filters ModelFilters `yaml:"filters"`
-}
-
-func (m *ModelConfig) UnmarshalYAML(unmarshal func(interface{}) error) error {
-	type rawModelConfig ModelConfig
-	defaults := rawModelConfig{
-		Cmd:              "",
-		CmdStop:          "",
-		Proxy:            "http://localhost:${PORT}",
-		Aliases:          []string{},
-		Env:              []string{},
-		CheckEndpoint:    "/health",
-		UnloadAfter:      0,
-		Unlisted:         false,
-		UseModelName:     "",
-		ConcurrencyLimit: 0,
-		Name:             "",
-		Description:      "",
-	}
-
-	// the default cmdStop to taskkill /f /t /pid ${PID}
-	if runtime.GOOS == "windows" {
-		defaults.CmdStop = "taskkill /f /t /pid ${PID}"
-	}
-
-	if err := unmarshal(&defaults); err != nil {
-		return err
-	}
-
-	*m = ModelConfig(defaults)
-	return nil
-}
-
-func (m *ModelConfig) SanitizedCommand() ([]string, error) {
-	return SanitizeCommand(m.Cmd)
-}
-
-// ModelFilters see issue #174
-type ModelFilters struct {
-	StripParams string `yaml:"strip_params"`
-}
-
-func (m *ModelFilters) UnmarshalYAML(unmarshal func(interface{}) error) error {
-	type rawModelFilters ModelFilters
-	defaults := rawModelFilters{
-		StripParams: "",
-	}
-
-	if err := unmarshal(&defaults); err != nil {
-		return err
-	}
-
-	*m = ModelFilters(defaults)
-	return nil
-}
-
-func (f ModelFilters) SanitizedStripParams() ([]string, error) {
-	if f.StripParams == "" {
-		return nil, nil
-	}
-
-	params := strings.Split(f.StripParams, ",")
-	cleaned := make([]string, 0, len(params))
-
-	for _, param := range params {
-		trimmed := strings.TrimSpace(param)
-		if trimmed == "model" || trimmed == "" {
-			continue
-		}
-		cleaned = append(cleaned, trimmed)
-	}
-
-	// sort cleaned
-	slices.Sort(cleaned)
-	return cleaned, nil
-}
+type MacroList map[string]any
 
 type GroupConfig struct {
 	Swap       bool     `yaml:"swap"`
@@ -120,6 +25,11 @@ type GroupConfig struct {
 	Members    []string `yaml:"members"`
 }
 
+var (
+	macroNameRegex    = regexp.MustCompile(`^[a-zA-Z0-9_-]+$`)
+	macroPatternRegex = regexp.MustCompile(`\$\{([a-zA-Z0-9_-]+)\}`)
+)
+
 // set default values for GroupConfig
 func (c *GroupConfig) UnmarshalYAML(unmarshal func(interface{}) error) error {
 	type rawGroupConfig GroupConfig
@@ -156,7 +66,7 @@ type Config struct {
 	Groups             map[string]GroupConfig `yaml:"groups"` /* key is group ID */
 
 	// for key/value replacements in model's cmd, cmdStop, proxy, checkEndPoint
-	Macros map[string]string `yaml:"macros"`
+	Macros MacroList `yaml:"macros"`
 
 	// map aliases to actual model IDs
 	aliases map[string]string
@@ -237,23 +147,12 @@ func LoadConfigFromReader(r io.Reader) (Config, error) {
 
 	- name must fit the regex ^[a-zA-Z0-9_-]+$
 	- names must be less than 64 characters (no reason, just cause)
-	- name can not be any reserved macros: PORT
+	- name can not be any reserved macros: PORT, MODEL_ID
 	- macro values must be less than 1024 characters
 	*/
-	macroNameRegex := regexp.MustCompile(`^[a-zA-Z0-9_-]+$`)
 	for macroName, macroValue := range config.Macros {
-		if len(macroName) >= 64 {
-			return Config{}, fmt.Errorf("macro name '%s' exceeds maximum length of 63 characters", macroName)
-		}
-		if !macroNameRegex.MatchString(macroName) {
-			return Config{}, fmt.Errorf("macro name '%s' contains invalid characters, must match pattern ^[a-zA-Z0-9_-]+$", macroName)
-		}
-		if len(macroValue) >= 1024 {
-			return Config{}, fmt.Errorf("macro value for '%s' exceeds maximum length of 1024 characters", macroName)
-		}
-		switch macroName {
-		case "PORT":
-			return Config{}, fmt.Errorf("macro name '%s' is reserved and cannot be used", macroName)
+		if err = validateMacro(macroName, macroValue); err != nil {
+			return Config{}, err
 		}
 	}
 
@@ -272,14 +171,34 @@ func LoadConfigFromReader(r io.Reader) (Config, error) {
 		modelConfig.Cmd = StripComments(modelConfig.Cmd)
 		modelConfig.CmdStop = StripComments(modelConfig.CmdStop)
 
+		// validate model macros
+		for macroName, macroValue := range modelConfig.Macros {
+			if err = validateMacro(macroName, macroValue); err != nil {
+				return Config{}, fmt.Errorf("model %s: %s", modelId, err.Error())
+			}
+		}
+
+		// Merge global config and model macros. Model macros take precedence
+		mergedMacros := make(MacroList)
+		for k, v := range config.Macros {
+			mergedMacros[k] = v
+		}
+		for k, v := range modelConfig.Macros {
+			mergedMacros[k] = v
+		}
+
+		mergedMacros["MODEL_ID"] = modelId
+
 		// go through model config fields: cmd, cmdStop, proxy, checkEndPoint and replace macros with macro values
-		for macroName, macroValue := range config.Macros {
+		for macroName, macroValue := range mergedMacros {
 			macroSlug := fmt.Sprintf("${%s}", macroName)
-			modelConfig.Cmd = strings.ReplaceAll(modelConfig.Cmd, macroSlug, macroValue)
-			modelConfig.CmdStop = strings.ReplaceAll(modelConfig.CmdStop, macroSlug, macroValue)
-			modelConfig.Proxy = strings.ReplaceAll(modelConfig.Proxy, macroSlug, macroValue)
-			modelConfig.CheckEndpoint = strings.ReplaceAll(modelConfig.CheckEndpoint, macroSlug, macroValue)
-			modelConfig.Filters.StripParams = strings.ReplaceAll(modelConfig.Filters.StripParams, macroSlug, macroValue)
+			// Convert macro value to string for command/string field substitution
+			macroStr := fmt.Sprintf("%v", macroValue)
+			modelConfig.Cmd = strings.ReplaceAll(modelConfig.Cmd, macroSlug, macroStr)
+			modelConfig.CmdStop = strings.ReplaceAll(modelConfig.CmdStop, macroSlug, macroStr)
+			modelConfig.Proxy = strings.ReplaceAll(modelConfig.Proxy, macroSlug, macroStr)
+			modelConfig.CheckEndpoint = strings.ReplaceAll(modelConfig.CheckEndpoint, macroSlug, macroStr)
+			modelConfig.Filters.StripParams = strings.ReplaceAll(modelConfig.Filters.StripParams, macroSlug, macroStr)
 		}
 
 		// enforce ${PORT} used in both cmd and proxy
@@ -293,20 +212,24 @@ func LoadConfigFromReader(r io.Reader) (Config, error) {
 			modelConfig.Cmd = strings.ReplaceAll(modelConfig.Cmd, "${PORT}", nextPortStr)
 			modelConfig.CmdStop = strings.ReplaceAll(modelConfig.CmdStop, "${PORT}", nextPortStr)
 			modelConfig.Proxy = strings.ReplaceAll(modelConfig.Proxy, "${PORT}", nextPortStr)
+
+			// add port to merged macros so it can be used in metadata
+			mergedMacros["PORT"] = nextPort
+
 			nextPort++
 		}
 
 		// make sure there are no unknown macros that have not been replaced
-		macroPattern := regexp.MustCompile(`\$\{([a-zA-Z0-9_-]+)\}`)
 		fieldMap := map[string]string{
-			"cmd":           modelConfig.Cmd,
-			"cmdStop":       modelConfig.CmdStop,
-			"proxy":         modelConfig.Proxy,
-			"checkEndpoint": modelConfig.CheckEndpoint,
+			"cmd":                 modelConfig.Cmd,
+			"cmdStop":             modelConfig.CmdStop,
+			"proxy":               modelConfig.Proxy,
+			"checkEndpoint":       modelConfig.CheckEndpoint,
+			"filters.stripParams": modelConfig.Filters.StripParams,
 		}
 
 		for fieldName, fieldValue := range fieldMap {
-			matches := macroPattern.FindAllStringSubmatch(fieldValue, -1)
+			matches := macroPatternRegex.FindAllStringSubmatch(fieldValue, -1)
 			for _, match := range matches {
 				macroName := match[1]
 				if macroName == "PID" && fieldName == "cmdStop" {
@@ -318,6 +241,15 @@ func LoadConfigFromReader(r io.Reader) (Config, error) {
 			}
 		}
 
+		// Apply macro substitution to metadata
+		if len(modelConfig.Metadata) > 0 {
+			substitutedMetadata, err := substituteMetadataMacros(modelConfig.Metadata, mergedMacros)
+			if err != nil {
+				return Config{}, fmt.Errorf("model %s metadata: %s", modelId, err.Error())
+			}
+			modelConfig.Metadata = substitutedMetadata.(map[string]any)
+		}
+
 		config.Models[modelId] = modelConfig
 	}
 
@@ -380,7 +312,7 @@ func AddDefaultGroupToConfig(config Config) Config {
 		}
 	} else {
 		// iterate over existing group members and add non-grouped models into the default group
-		for modelName, _ := range config.Models {
+		for modelName := range config.Models {
 			foundModel := false
 		found:
 			// search for the model in existing groups
@@ -452,3 +384,92 @@ func StripComments(cmdStr string) string {
 	}
 	return strings.Join(cleanedLines, "\n")
 }
+
+// validateMacro validates macro name and value constraints
+func validateMacro(name string, value any) error {
+	if len(name) >= 64 {
+		return fmt.Errorf("macro name '%s' exceeds maximum length of 63 characters", name)
+	}
+	if !macroNameRegex.MatchString(name) {
+		return fmt.Errorf("macro name '%s' contains invalid characters, must match pattern ^[a-zA-Z0-9_-]+$", name)
+	}
+
+	// Validate that value is a scalar type
+	switch v := value.(type) {
+	case string:
+		if len(v) >= 1024 {
+			return fmt.Errorf("macro value for '%s' exceeds maximum length of 1024 characters", name)
+		}
+	case int, int8, int16, int32, int64, uint, uint8, uint16, uint32, uint64, float32, float64, bool:
+		// These types are allowed
+	default:
+		return fmt.Errorf("macro '%s' has invalid type %T, must be a scalar type (string, int, float, or bool)", name, value)
+	}
+
+	switch name {
+	case "PORT", "MODEL_ID":
+		return fmt.Errorf("macro name '%s' is reserved", name)
+	}
+
+	return nil
+}
+
+// substituteMetadataMacros recursively substitutes macros in metadata structures
+// Direct substitution (key: ${macro}) preserves the macro's type
+// Interpolated substitution (key: "text ${macro}") converts to string
+func substituteMetadataMacros(value any, macros MacroList) (any, error) {
+	switch v := value.(type) {
+	case string:
+		// Check if this is a direct macro substitution
+		if strings.HasPrefix(v, "${") && strings.HasSuffix(v, "}") && strings.Count(v, "${") == 1 {
+			macroName := v[2 : len(v)-1]
+			if macroValue, exists := macros[macroName]; exists {
+				return macroValue, nil
+			}
+			return nil, fmt.Errorf("unknown macro '${%s}' in metadata", macroName)
+		}
+
+		// Handle string interpolation
+		matches := macroPatternRegex.FindAllStringSubmatch(v, -1)
+		result := v
+		for _, match := range matches {
+			macroName := match[1]
+			macroValue, exists := macros[macroName]
+			if !exists {
+				return nil, fmt.Errorf("unknown macro '${%s}' in metadata", macroName)
+			}
+			// Convert macro value to string for interpolation
+			macroStr := fmt.Sprintf("%v", macroValue)
+			result = strings.ReplaceAll(result, match[0], macroStr)
+		}
+		return result, nil
+
+	case map[string]any:
+		// Recursively process map values
+		newMap := make(map[string]any)
+		for key, val := range v {
+			newVal, err := substituteMetadataMacros(val, macros)
+			if err != nil {
+				return nil, err
+			}
+			newMap[key] = newVal
+		}
+		return newMap, nil
+
+	case []any:
+		// Recursively process slice elements
+		newSlice := make([]any, len(v))
+		for i, val := range v {
+			newVal, err := substituteMetadataMacros(val, macros)
+			if err != nil {
+				return nil, err
+			}
+			newSlice[i] = newVal
+		}
+		return newSlice, nil
+
+	default:
+		// Return scalar types as-is
+		return value, nil
+	}
+}

proxy/config/config_posix_test.go

@@ -1,6 +1,6 @@
 //go:build !windows
 
-package proxy
+package config
 
 import (
 	"os"
@@ -163,7 +163,7 @@ groups:
 	expected := Config{
 		LogLevel:  "info",
 		StartPort: 5800,
-		Macros: map[string]string{
+		Macros: MacroList{
 			"svr-path": "path/to/server",
 		},
 		Hooks: HooksConfig{

proxy/config/config_test.go

@@ -1,4 +1,4 @@
-package proxy
+package config
 
 import (
 	"slices"
@@ -65,18 +65,6 @@ models:
 	assert.Contains(t, err.Error(), "duplicate alias m1 found in model: model")
 }
 
-func TestConfig_ModelConfigSanitizedCommand(t *testing.T) {
-	config := &ModelConfig{
-		Cmd: `python model1.py \
-    --arg1 value1 \
-    --arg2 value2`,
-	}
-
-	args, err := config.SanitizedCommand()
-	assert.NoError(t, err)
-	assert.Equal(t, []string{"python", "model1.py", "--arg1", "value1", "--arg2", "value2"}, args)
-}
-
 func TestConfig_FindConfig(t *testing.T) {
 
 	// TODO?
@@ -207,15 +195,19 @@ macros:
   argOne: "--arg1"
   argTwo: "--arg2"
   autoPort: "--port ${PORT}"
+  overriddenByModelMacro: failed
 
 models:
   model1:
+    macros:
+      overriddenByModelMacro: success
     cmd: |
       ${svr-path} ${argTwo}
       # the automatic ${PORT} is replaced
       ${autoPort}
       ${argOne}
       --arg3 three
+      --overridden ${overriddenByModelMacro}
     cmdStop: |
       /path/to/stop.sh --port ${PORT} ${argTwo}
 `
@@ -224,13 +216,68 @@ models:
 	assert.NoError(t, err)
 	sanitizedCmd, err := SanitizeCommand(config.Models["model1"].Cmd)
 	assert.NoError(t, err)
-	assert.Equal(t, "path/to/server --arg2 --port 9990 --arg1 --arg3 three", strings.Join(sanitizedCmd, " "))
+	assert.Equal(t, "path/to/server --arg2 --port 9990 --arg1 --arg3 three --overridden success", strings.Join(sanitizedCmd, " "))
 
 	sanitizedCmdStop, err := SanitizeCommand(config.Models["model1"].CmdStop)
 	assert.NoError(t, err)
 	assert.Equal(t, "/path/to/stop.sh --port 9990 --arg2", strings.Join(sanitizedCmdStop, " "))
 }
 
+func TestConfig_MacroReservedNames(t *testing.T) {
+
+	tests := []struct {
+		name          string
+		config        string
+		expectedError string
+	}{
+		{
+			name: "global macro named PORT",
+			config: `
+macros:
+  PORT: "1111"
+`,
+			expectedError: "macro name 'PORT' is reserved",
+		},
+		{
+			name: "global macro named MODEL_ID",
+			config: `
+macros:
+  MODEL_ID: model1
+`,
+			expectedError: "macro name 'MODEL_ID' is reserved",
+		},
+		{
+			name: "model macro named PORT",
+			config: `
+models:
+  model1:
+    macros:
+      PORT: 1111
+`,
+			expectedError: "model model1: macro name 'PORT' is reserved",
+		},
+
+		{
+			name: "model macro named MODEL_ID",
+			config: `
+models:
+  model1:
+    macros:
+      MODEL_ID: model1
+`,
+			expectedError: "model model1: macro name 'MODEL_ID' is reserved",
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			_, err := LoadConfigFromReader(strings.NewReader(tt.config))
+			assert.NotNil(t, err)
+			assert.Equal(t, tt.expectedError, err.Error())
+		})
+	}
+}
+
 func TestConfig_MacroErrorOnUnknownMacros(t *testing.T) {
 	tests := []struct {
 		name    string
@@ -288,6 +335,20 @@ models:
   model1:
     cmd: "${svr-path} --port ${PORT}"
     checkEndpoint: "http://localhost:${unknownMacro}/health"
+`,
+		},
+		{
+			name:  "unknown macro in filters.stripParams",
+			field: "filters.stripParams",
+			content: `
+startPort: 9990
+macros:
+  svr-path: "path/to/server"
+models:
+  model1:
+    cmd: "${svr-path} --port ${PORT}"
+    filters:
+      stripParams: "model,${unknownMacro}"
 `,
 		},
 	}
@@ -295,38 +356,13 @@ models:
 	for _, tt := range tests {
 		t.Run(tt.name, func(t *testing.T) {
 			_, err := LoadConfigFromReader(strings.NewReader(tt.content))
-			assert.Error(t, err)
-			assert.Contains(t, err.Error(), "unknown macro '${unknownMacro}' found in model1."+tt.field)
+			if assert.Error(t, err) {
+				assert.Contains(t, err.Error(), "unknown macro '${unknownMacro}' found in model1."+tt.field)
+			}
 			//t.Log(err)
 		})
 	}
 }
-
-func TestConfig_ModelFilters(t *testing.T) {
-	content := `
-macros:
-  default_strip: "temperature, top_p"
-models:
-  model1:
-    cmd: path/to/cmd --port ${PORT}
-    filters:
-      strip_params: "model, top_k, ${default_strip}, , ,"
-`
-	config, err := LoadConfigFromReader(strings.NewReader(content))
-	assert.NoError(t, err)
-	modelConfig, ok := config.Models["model1"]
-	if !assert.True(t, ok) {
-		t.FailNow()
-	}
-
-	// make sure `model` and enmpty strings are not in the list
-	assert.Equal(t, "model, top_k, temperature, top_p, , ,", modelConfig.Filters.StripParams)
-	sanitized, err := modelConfig.Filters.SanitizedStripParams()
-	if assert.NoError(t, err) {
-		assert.Equal(t, []string{"temperature", "top_k", "top_p"}, sanitized)
-	}
-}
-
 func TestStripComments(t *testing.T) {
 	tests := []struct {
 		name     string
@@ -440,3 +476,284 @@ models:
 	expectedCmd := "/user/llama.cpp/build/bin/llama-server --port 9990 --model /path/to/model.gguf -ngl 99"
 	assert.Equal(t, expectedCmd, cmdStr, "Final command does not match expected structure")
 }
+
+func TestConfig_MacroModelId(t *testing.T) {
+	content := `
+startPort: 9000
+macros:
+  "docker-llama": docker run --name ${MODEL_ID} -p ${PORT}:8080 docker_img
+  "docker-stop": docker stop ${MODEL_ID}
+
+models:
+  model1:
+    cmd: /path/to/server -p ${PORT} -hf ${MODEL_ID}
+
+  model2:
+    cmd: ${docker-llama}
+    cmdStop: ${docker-stop}
+
+  author/model:F16:
+    cmd: /path/to/server -p ${PORT} -hf ${MODEL_ID}
+    cmdStop: stop
+`
+
+	config, err := LoadConfigFromReader(strings.NewReader(content))
+	assert.NoError(t, err)
+	sanitizedCmd, err := SanitizeCommand(config.Models["model1"].Cmd)
+	assert.NoError(t, err)
+	assert.Equal(t, "/path/to/server -p 9001 -hf model1", strings.Join(sanitizedCmd, " "))
+
+	assert.Equal(t, "docker stop ${MODEL_ID}", config.Macros["docker-stop"])
+
+	sanitizedCmd2, err := SanitizeCommand(config.Models["model2"].Cmd)
+	assert.NoError(t, err)
+	assert.Equal(t, "docker run --name model2 -p 9002:8080 docker_img", strings.Join(sanitizedCmd2, " "))
+
+	sanitizedCmdStop, err := SanitizeCommand(config.Models["model2"].CmdStop)
+	assert.NoError(t, err)
+	assert.Equal(t, "docker stop model2", strings.Join(sanitizedCmdStop, " "))
+
+	sanitizedCmd3, err := SanitizeCommand(config.Models["author/model:F16"].Cmd)
+	assert.NoError(t, err)
+	assert.Equal(t, "/path/to/server -p 9000 -hf author/model:F16", strings.Join(sanitizedCmd3, " "))
+}
+
+func TestConfig_TypedMacrosInMetadata(t *testing.T) {
+	content := `
+startPort: 10000
+macros:
+  PORT_NUM: 10001
+  TEMP: 0.7
+  ENABLED: true
+  NAME: "llama model"
+  CTX: 16384
+
+models:
+  test-model:
+    cmd: /path/to/server -p ${PORT}
+    metadata:
+      port: ${PORT_NUM}
+      temperature: ${TEMP}
+      enabled: ${ENABLED}
+      model_name: ${NAME}
+      context: ${CTX}
+      note: "Running on port ${PORT_NUM} with temp ${TEMP} and context ${CTX}"
+`
+
+	config, err := LoadConfigFromReader(strings.NewReader(content))
+	assert.NoError(t, err)
+
+	meta := config.Models["test-model"].Metadata
+	assert.NotNil(t, meta)
+
+	// Verify direct substitution preserves types
+	assert.Equal(t, 10001, meta["port"])
+	assert.Equal(t, 0.7, meta["temperature"])
+	assert.Equal(t, true, meta["enabled"])
+	assert.Equal(t, "llama model", meta["model_name"])
+	assert.Equal(t, 16384, meta["context"])
+
+	// Verify string interpolation converts to string
+	assert.Equal(t, "Running on port 10001 with temp 0.7 and context 16384", meta["note"])
+}
+
+func TestConfig_NestedStructuresInMetadata(t *testing.T) {
+	content := `
+startPort: 10000
+macros:
+  TEMP: 0.7
+
+models:
+  test-model:
+    cmd: /path/to/server -p ${PORT}
+    metadata:
+      config:
+        port: ${PORT}
+        temperature: ${TEMP}
+      tags: ["model:${MODEL_ID}", "port:${PORT}"]
+      nested:
+        deep:
+          value: ${TEMP}
+`
+
+	config, err := LoadConfigFromReader(strings.NewReader(content))
+	assert.NoError(t, err)
+
+	meta := config.Models["test-model"].Metadata
+	assert.NotNil(t, meta)
+
+	// Verify nested objects
+	configMap := meta["config"].(map[string]any)
+	assert.Equal(t, 10000, configMap["port"])
+	assert.Equal(t, 0.7, configMap["temperature"])
+
+	// Verify arrays
+	tags := meta["tags"].([]any)
+	assert.Equal(t, "model:test-model", tags[0])
+	assert.Equal(t, "port:10000", tags[1])
+
+	// Verify deeply nested structures
+	nested := meta["nested"].(map[string]any)
+	deep := nested["deep"].(map[string]any)
+	assert.Equal(t, 0.7, deep["value"])
+}
+
+func TestConfig_ModelLevelMacroPrecedenceInMetadata(t *testing.T) {
+	content := `
+startPort: 10000
+macros:
+  TEMP: 0.5
+  GLOBAL_VAL: "global"
+
+models:
+  test-model:
+    cmd: /path/to/server -p ${PORT}
+    macros:
+      TEMP: 0.9
+      LOCAL_VAL: "local"
+    metadata:
+      temperature: ${TEMP}
+      global: ${GLOBAL_VAL}
+      local: ${LOCAL_VAL}
+`
+
+	config, err := LoadConfigFromReader(strings.NewReader(content))
+	assert.NoError(t, err)
+
+	meta := config.Models["test-model"].Metadata
+	assert.NotNil(t, meta)
+
+	// Model-level macro should override global
+	assert.Equal(t, 0.9, meta["temperature"])
+	// Global macro should be accessible
+	assert.Equal(t, "global", meta["global"])
+	// Model-level macro should be accessible
+	assert.Equal(t, "local", meta["local"])
+}
+
+func TestConfig_UnknownMacroInMetadata(t *testing.T) {
+	content := `
+startPort: 10000
+models:
+  test-model:
+    cmd: /path/to/server -p ${PORT}
+    metadata:
+      value: ${UNKNOWN_MACRO}
+`
+
+	_, err := LoadConfigFromReader(strings.NewReader(content))
+	assert.Error(t, err)
+	assert.Contains(t, err.Error(), "test-model")
+	assert.Contains(t, err.Error(), "UNKNOWN_MACRO")
+}
+
+func TestConfig_InvalidMacroType(t *testing.T) {
+	content := `
+startPort: 10000
+macros:
+  INVALID:
+    nested: value
+
+models:
+  test-model:
+    cmd: /path/to/server -p ${PORT}
+`
+
+	_, err := LoadConfigFromReader(strings.NewReader(content))
+	assert.Error(t, err)
+	assert.Contains(t, err.Error(), "INVALID")
+	assert.Contains(t, err.Error(), "must be a scalar type")
+}
+
+func TestConfig_MacroTypeValidation(t *testing.T) {
+	tests := []struct {
+		name      string
+		yaml      string
+		shouldErr bool
+	}{
+		{
+			name: "string macro",
+			yaml: `
+startPort: 10000
+macros:
+  STR: "test"
+models:
+  test-model:
+    cmd: /path/to/server -p ${PORT}
+`,
+			shouldErr: false,
+		},
+		{
+			name: "int macro",
+			yaml: `
+startPort: 10000
+macros:
+  NUM: 42
+models:
+  test-model:
+    cmd: /path/to/server -p ${PORT}
+`,
+			shouldErr: false,
+		},
+		{
+			name: "float macro",
+			yaml: `
+startPort: 10000
+macros:
+  FLOAT: 3.14
+models:
+  test-model:
+    cmd: /path/to/server -p ${PORT}
+`,
+			shouldErr: false,
+		},
+		{
+			name: "bool macro",
+			yaml: `
+startPort: 10000
+macros:
+  BOOL: true
+models:
+  test-model:
+    cmd: /path/to/server -p ${PORT}
+`,
+			shouldErr: false,
+		},
+		{
+			name: "array macro (invalid)",
+			yaml: `
+startPort: 10000
+macros:
+  ARR: [1, 2, 3]
+models:
+  test-model:
+    cmd: /path/to/server -p ${PORT}
+`,
+			shouldErr: true,
+		},
+		{
+			name: "map macro (invalid)",
+			yaml: `
+startPort: 10000
+macros:
+  MAP:
+    key: value
+models:
+  test-model:
+    cmd: /path/to/server -p ${PORT}
+`,
+			shouldErr: true,
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			_, err := LoadConfigFromReader(strings.NewReader(tt.yaml))
+			if tt.shouldErr {
+				assert.Error(t, err)
+			} else {
+				assert.NoError(t, err)
+			}
+		})
+	}
+}

proxy/config/config_windows_test.go

@@ -1,6 +1,6 @@
 //go:build windows
 
-package proxy
+package config
 
 import (
 	"os"
@@ -155,7 +155,7 @@ groups:
 	expected := Config{
 		LogLevel:  "info",
 		StartPort: 5800,
-		Macros: map[string]string{
+		Macros: MacroList{
 			"svr-path": "path/to/server",
 		},
 		Models: map[string]ModelConfig{

proxy/config/model_config.go

@@ -0,0 +1,125 @@
+package config
+
+import (
+	"errors"
+	"runtime"
+	"slices"
+	"strings"
+)
+
+type ModelConfig struct {
+	Cmd           string   `yaml:"cmd"`
+	CmdStop       string   `yaml:"cmdStop"`
+	Proxy         string   `yaml:"proxy"`
+	Aliases       []string `yaml:"aliases"`
+	Env           []string `yaml:"env"`
+	CheckEndpoint string   `yaml:"checkEndpoint"`
+	UnloadAfter   int      `yaml:"ttl"`
+	Unlisted      bool     `yaml:"unlisted"`
+	UseModelName  string   `yaml:"useModelName"`
+
+	// #179 for /v1/models
+	Name        string `yaml:"name"`
+	Description string `yaml:"description"`
+
+	// Limit concurrency of HTTP requests to process
+	ConcurrencyLimit int `yaml:"concurrencyLimit"`
+
+	// Model filters see issue #174
+	Filters ModelFilters `yaml:"filters"`
+
+	// Macros: see #264
+	// Model level macros take precedence over the global macros
+	Macros MacroList `yaml:"macros"`
+
+	// Metadata: see #264
+	// Arbitrary metadata that can be exposed through the API
+	Metadata map[string]any `yaml:"metadata"`
+}
+
+func (m *ModelConfig) UnmarshalYAML(unmarshal func(interface{}) error) error {
+	type rawModelConfig ModelConfig
+	defaults := rawModelConfig{
+		Cmd:              "",
+		CmdStop:          "",
+		Proxy:            "http://localhost:${PORT}",
+		Aliases:          []string{},
+		Env:              []string{},
+		CheckEndpoint:    "/health",
+		UnloadAfter:      0,
+		Unlisted:         false,
+		UseModelName:     "",
+		ConcurrencyLimit: 0,
+		Name:             "",
+		Description:      "",
+	}
+
+	// the default cmdStop to taskkill /f /t /pid ${PID}
+	if runtime.GOOS == "windows" {
+		defaults.CmdStop = "taskkill /f /t /pid ${PID}"
+	}
+
+	if err := unmarshal(&defaults); err != nil {
+		return err
+	}
+
+	*m = ModelConfig(defaults)
+	return nil
+}
+
+func (m *ModelConfig) SanitizedCommand() ([]string, error) {
+	return SanitizeCommand(m.Cmd)
+}
+
+// ModelFilters see issue #174
+type ModelFilters struct {
+	StripParams string `yaml:"stripParams"`
+}
+
+func (m *ModelFilters) UnmarshalYAML(unmarshal func(interface{}) error) error {
+	type rawModelFilters ModelFilters
+	defaults := rawModelFilters{
+		StripParams: "",
+	}
+
+	if err := unmarshal(&defaults); err != nil {
+		return err
+	}
+
+	// Try to unmarshal with the old field name for backwards compatibility
+	if defaults.StripParams == "" {
+		var legacy struct {
+			StripParams string `yaml:"strip_params"`
+		}
+		if legacyErr := unmarshal(&legacy); legacyErr != nil {
+			return errors.New("failed to unmarshal legacy filters.strip_params: " + legacyErr.Error())
+		}
+		defaults.StripParams = legacy.StripParams
+	}
+
+	*m = ModelFilters(defaults)
+	return nil
+}
+
+func (f ModelFilters) SanitizedStripParams() ([]string, error) {
+	if f.StripParams == "" {
+		return nil, nil
+	}
+
+	params := strings.Split(f.StripParams, ",")
+	cleaned := make([]string, 0, len(params))
+	seen := make(map[string]bool)
+
+	for _, param := range params {
+		trimmed := strings.TrimSpace(param)
+		if trimmed == "model" || trimmed == "" || seen[trimmed] {
+			continue
+		}
+		seen[trimmed] = true
+		cleaned = append(cleaned, trimmed)
+	}
+
+	// sort cleaned
+	slices.Sort(cleaned)
+	return cleaned, nil
+}

proxy/config/model_config_test.go

@@ -0,0 +1,54 @@
+package config
+
+import (
+	"fmt"
+	"strings"
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+)
+
+func TestConfig_ModelConfigSanitizedCommand(t *testing.T) {
+	config := &ModelConfig{
+		Cmd: `python model1.py \
+    --arg1 value1 \
+    --arg2 value2`,
+	}
+
+	args, err := config.SanitizedCommand()
+	assert.NoError(t, err)
+	assert.Equal(t, []string{"python", "model1.py", "--arg1", "value1", "--arg2", "value2"}, args)
+}
+
+func TestConfig_ModelFilters(t *testing.T) {
+	content := `
+macros:
+  default_strip: "temperature, top_p"
+models:
+  model1:
+    cmd: path/to/cmd --port ${PORT}
+    filters:
+      # macros inserted and list is cleaned of duplicates and empty strings
+      stripParams: "model, top_k, top_k, temperature, ${default_strip}, , ,"
+  # check for strip_params (legacy field name) compatibility
+  legacy:
+    cmd: path/to/cmd --port ${PORT}
+    filters:
+      strip_params: "model, top_k, top_k, temperature, ${default_strip}, , ,"
+`
+	config, err := LoadConfigFromReader(strings.NewReader(content))
+	assert.NoError(t, err)
+	for modelId, modelConfig := range config.Models {
+		t.Run(fmt.Sprintf("Testing macros in filters for model %s", modelId), func(t *testing.T) {
+			assert.Equal(t, "model, top_k, top_k, temperature, temperature, top_p, , ,", modelConfig.Filters.StripParams)
+			sanitized, err := modelConfig.Filters.SanitizedStripParams()
+			if assert.NoError(t, err) {
+				// model has been removed
+				// empty strings have been removed
+				// duplicates have been removed
+				assert.Equal(t, []string{"temperature", "top_k", "top_p"}, sanitized)
+			}
+		})
+	}
+
+}

proxy/helpers_test.go

@@ -9,6 +9,7 @@ import (
 	"testing"
 
 	"github.com/gin-gonic/gin"
+	"github.com/mostlygeek/llama-swap/proxy/config"
 	"gopkg.in/yaml.v3"
 )
 
@@ -65,18 +66,18 @@ func getTestPort() int {
 	return port
 }
 
-func getTestSimpleResponderConfig(expectedMessage string) ModelConfig {
+func getTestSimpleResponderConfig(expectedMessage string) config.ModelConfig {
 	return getTestSimpleResponderConfigPort(expectedMessage, getTestPort())
 }
 
-func getTestSimpleResponderConfigPort(expectedMessage string, port int) ModelConfig {
+func getTestSimpleResponderConfigPort(expectedMessage string, port int) config.ModelConfig {
 	// Create a YAML string with just the values we want to set
 	yamlStr := fmt.Sprintf(`
 cmd: '%s --port %d --silent --respond %s'
 proxy: "http://127.0.0.1:%d"
 `, simpleResponderPath, port, expectedMessage, port)
 
-	var cfg ModelConfig
+	var cfg config.ModelConfig
 	if err := yaml.Unmarshal([]byte(yamlStr), &cfg); err != nil {
 		panic(fmt.Sprintf("failed to unmarshal test config: %v in [%s]", err, yamlStr))
 	}

proxy/metrics_middleware.go

@@ -5,12 +5,20 @@ import (
 	"fmt"
 	"io"
 	"net/http"
+	"strings"
 	"time"
 
 	"github.com/gin-gonic/gin"
 	"github.com/tidwall/gjson"
 )
 
+type MetricsRecorder struct {
+	metricsMonitor *MetricsMonitor
+	realModelName  string
+	//	isStreaming    bool
+	startTime time.Time
+}
+
 // MetricsMiddleware sets up the MetricsResponseWriter for capturing upstream requests
 func MetricsMiddleware(pm *ProxyManager) gin.HandlerFunc {
 	return func(c *gin.Context) {
@@ -41,57 +49,60 @@ func MetricsMiddleware(pm *ProxyManager) gin.HandlerFunc {
 			metricsRecorder: &MetricsRecorder{
 				metricsMonitor: pm.metricsMonitor,
 				realModelName:  realModelName,
-				isStreaming:    gjson.GetBytes(bodyBytes, "stream").Bool(),
 				startTime:      time.Now(),
 			},
 		}
 		c.Writer = writer
 		c.Next()
 
-		rec := writer.metricsRecorder
-		rec.processBody(writer.body)
-	}
-}
-
-type MetricsRecorder struct {
-	metricsMonitor *MetricsMonitor
-	realModelName  string
-	isStreaming    bool
-	startTime      time.Time
-}
-
-// processBody handles response processing after request completes
-func (rec *MetricsRecorder) processBody(body []byte) {
-	if rec.isStreaming {
-		rec.processStreamingResponse(body)
-	} else {
-		rec.processNonStreamingResponse(body)
+		// check for streaming response
+		if strings.Contains(c.Writer.Header().Get("Content-Type"), "text/event-stream") {
+			writer.metricsRecorder.processStreamingResponse(writer.body)
+		} else {
+			writer.metricsRecorder.processNonStreamingResponse(writer.body)
+		}
 	}
 }
 
 func (rec *MetricsRecorder) parseAndRecordMetrics(jsonData gjson.Result) bool {
 	usage := jsonData.Get("usage")
-	if !usage.Exists() {
+	timings := jsonData.Get("timings")
+	if !usage.Exists() && !timings.Exists() {
 		return false
 	}
 
 	// default values
-	outputTokens := int(jsonData.Get("usage.completion_tokens").Int())
-	inputTokens := int(jsonData.Get("usage.prompt_tokens").Int())
+	cachedTokens := -1 // unknown or missing data
+	outputTokens := 0
+	inputTokens := 0
+
+	// timings data
 	tokensPerSecond := -1.0
 	promptPerSecond := -1.0
 	durationMs := int(time.Since(rec.startTime).Milliseconds())
 
+	if usage.Exists() {
+		outputTokens = int(jsonData.Get("usage.completion_tokens").Int())
+		inputTokens = int(jsonData.Get("usage.prompt_tokens").Int())
+	}
+
 	// use llama-server's timing data for tok/sec and duration as it is more accurate
-	if timings := jsonData.Get("timings"); timings.Exists() {
+	if timings.Exists() {
+		inputTokens = int(jsonData.Get("timings.prompt_n").Int())
+		outputTokens = int(jsonData.Get("timings.predicted_n").Int())
 		promptPerSecond = jsonData.Get("timings.prompt_per_second").Float()
 		tokensPerSecond = jsonData.Get("timings.predicted_per_second").Float()
 		durationMs = int(jsonData.Get("timings.prompt_ms").Float() + jsonData.Get("timings.predicted_ms").Float())
+
+		if cachedValue := jsonData.Get("timings.cache_n"); cachedValue.Exists() {
+			cachedTokens = int(cachedValue.Int())
+		}
 	}
 
 	rec.metricsMonitor.addMetrics(TokenMetrics{
 		Timestamp:       time.Now(),
 		Model:           rec.realModelName,
+		CachedTokens:    cachedTokens,
 		InputTokens:     inputTokens,
 		OutputTokens:    outputTokens,
 		PromptPerSecond: promptPerSecond,

proxy/metrics_monitor.go

@@ -6,6 +6,7 @@ import (
 	"time"
 
 	"github.com/mostlygeek/llama-swap/event"
+	"github.com/mostlygeek/llama-swap/proxy/config"
 )
 
 // TokenMetrics represents parsed token statistics from llama-server logs
@@ -13,6 +14,7 @@ type TokenMetrics struct {
 	ID              int       `json:"id"`
 	Timestamp       time.Time `json:"timestamp"`
 	Model           string    `json:"model"`
+	CachedTokens    int       `json:"cache_tokens"`
 	InputTokens     int       `json:"input_tokens"`
 	OutputTokens    int       `json:"output_tokens"`
 	PromptPerSecond float64   `json:"prompt_per_second"`
@@ -37,7 +39,7 @@ type MetricsMonitor struct {
 	nextID     int
 }
 
-func NewMetricsMonitor(config *Config) *MetricsMonitor {
+func NewMetricsMonitor(config *config.Config) *MetricsMonitor {
 	maxMetrics := config.MetricsMaxInMemory
 	if maxMetrics <= 0 {
 		maxMetrics = 1000 // Default fallback
@@ -61,7 +63,6 @@ func (mp *MetricsMonitor) addMetrics(metric TokenMetrics) {
 	if len(mp.metrics) > mp.maxMetrics {
 		mp.metrics = mp.metrics[len(mp.metrics)-mp.maxMetrics:]
 	}
-
 	event.Emit(TokenMetricsEvent{Metrics: metric})
 }
 

proxy/process.go

@@ -5,6 +5,7 @@ import (
 	"errors"
 	"fmt"
 	"io"
+	"net"
 	"net/http"
 	"net/url"
 	"os/exec"
@@ -15,6 +16,7 @@ import (
 	"time"
 
 	"github.com/mostlygeek/llama-swap/event"
+	"github.com/mostlygeek/llama-swap/proxy/config"
 )
 
 type ProcessState string
@@ -38,7 +40,7 @@ const (
 
 type Process struct {
 	ID     string
-	config ModelConfig
+	config config.ModelConfig
 	cmd    *exec.Cmd
 
 	// PR #155 called to cancel the upstream process
@@ -73,7 +75,7 @@ type Process struct {
 	failedStartCount int
 }
 
-func NewProcess(ID string, healthCheckTimeout int, config ModelConfig, processLogger *LogMonitor, proxyLogger *LogMonitor) *Process {
+func NewProcess(ID string, healthCheckTimeout int, config config.ModelConfig, processLogger *LogMonitor, proxyLogger *LogMonitor) *Process {
 	concurrentLimit := 10
 	if config.ConcurrencyLimit > 0 {
 		concurrentLimit = config.ConcurrencyLimit
@@ -363,8 +365,18 @@ func (p *Process) stopCommand() {
 }
 
 func (p *Process) checkHealthEndpoint(healthURL string) error {
+
 	client := &http.Client{
-		Timeout: 500 * time.Millisecond,
+		// wait a short time for a tcp connection to be established
+		Transport: &http.Transport{
+			DialContext: (&net.Dialer{
+				Timeout: 500 * time.Millisecond,
+			}).DialContext,
+		},
+
+		// give a long time to respond to the health check endpoint
+		// after the connection is established. See issue: 276
+		Timeout: 5000 * time.Millisecond,
 	}
 
 	req, err := http.NewRequest("GET", healthURL, nil)
@@ -447,6 +459,10 @@ func (p *Process) ProxyRequest(w http.ResponseWriter, r *http.Request) {
 			w.Header().Add(k, v)
 		}
 	}
+	// prevent nginx from buffering streaming responses (e.g., SSE)
+	if strings.Contains(strings.ToLower(resp.Header.Get("Content-Type")), "text/event-stream") {
+		w.Header().Set("X-Accel-Buffering", "no")
+	}
 	w.WriteHeader(resp.StatusCode)
 
 	// faster than io.Copy when streaming
@@ -524,7 +540,7 @@ func (p *Process) cmdStopUpstreamProcess() error {
 
 	if p.config.CmdStop != "" {
 		// replace ${PID} with the pid of the process
-		stopArgs, err := SanitizeCommand(strings.ReplaceAll(p.config.CmdStop, "${PID}", fmt.Sprintf("%d", p.cmd.Process.Pid)))
+		stopArgs, err := config.SanitizeCommand(strings.ReplaceAll(p.config.CmdStop, "${PID}", fmt.Sprintf("%d", p.cmd.Process.Pid)))
 		if err != nil {
 			p.proxyLogger.Errorf("<%s> Failed to sanitize stop command: %v", p.ID, err)
 			return err

proxy/process_test.go

@@ -10,6 +10,7 @@ import (
 	"testing"
 	"time"
 
+	"github.com/mostlygeek/llama-swap/proxy/config"
 	"github.com/stretchr/testify/assert"
 )
 
@@ -90,7 +91,7 @@ func TestProcess_WaitOnMultipleStarts(t *testing.T) {
 // test that the automatic start returns the expected error type
 func TestProcess_BrokenModelConfig(t *testing.T) {
 	// Create a process configuration
-	config := ModelConfig{
+	config := config.ModelConfig{
 		Cmd:           "nonexistent-command",
 		Proxy:         "http://127.0.0.1:9913",
 		CheckEndpoint: "/health",
@@ -325,7 +326,7 @@ func TestProcess_ExitInterruptsHealthCheck(t *testing.T) {
 
 	// should run and exit but interrupt the long checkHealthTimeout
 	checkHealthTimeout := 5
-	config := ModelConfig{
+	config := config.ModelConfig{
 		Cmd:           "sleep 1",
 		Proxy:         "http://127.0.0.1:9913",
 		CheckEndpoint: "/health",
@@ -402,7 +403,7 @@ func TestProcess_ForceStopWithKill(t *testing.T) {
 	binaryPath := getSimpleResponderPath()
 	port := getTestPort()
 
-	config := ModelConfig{
+	conf := config.ModelConfig{
 		// note --ignore-sig-term which ignores the SIGTERM signal so a SIGKILL must be sent
 		// to force the process to exit
 		Cmd:           fmt.Sprintf("%s --port %d --respond %s --silent --ignore-sig-term", binaryPath, port, expectedMessage),
@@ -410,7 +411,7 @@ func TestProcess_ForceStopWithKill(t *testing.T) {
 		CheckEndpoint: "/health",
 	}
 
-	process := NewProcess("stop_immediate", 2, config, debugLogger, debugLogger)
+	process := NewProcess("stop_immediate", 2, conf, debugLogger, debugLogger)
 	defer process.Stop()
 
 	// reduce to make testing go faster
@@ -450,15 +451,15 @@ func TestProcess_ForceStopWithKill(t *testing.T) {
 }
 
 func TestProcess_StopCmd(t *testing.T) {
-	config := getTestSimpleResponderConfig("test_stop_cmd")
+	conf := getTestSimpleResponderConfig("test_stop_cmd")
 
 	if runtime.GOOS == "windows" {
-		config.CmdStop = "taskkill /f /t /pid ${PID}"
+		conf.CmdStop = "taskkill /f /t /pid ${PID}"
 	} else {
-		config.CmdStop = "kill -TERM ${PID}"
+		conf.CmdStop = "kill -TERM ${PID}"
 	}
 
-	process := NewProcess("testStopCmd", 2, config, debugLogger, debugLogger)
+	process := NewProcess("testStopCmd", 2, conf, debugLogger, debugLogger)
 	defer process.Stop()
 
 	err := process.start()
@@ -470,15 +471,15 @@ func TestProcess_StopCmd(t *testing.T) {
 
 func TestProcess_EnvironmentSetCorrectly(t *testing.T) {
 	expectedMessage := "test_env_not_emptied"
-	config := getTestSimpleResponderConfig(expectedMessage)
+	conf := getTestSimpleResponderConfig(expectedMessage)
 
 	// ensure that the the default config does not blank out the inherited environment
-	configWEnv := config
+	configWEnv := conf
 
 	// ensure the additiona variables are appended to the process' environment
 	configWEnv.Env = append(configWEnv.Env, "TEST_ENV1=1", "TEST_ENV2=2")
 
-	process1 := NewProcess("env_test", 2, config, debugLogger, debugLogger)
+	process1 := NewProcess("env_test", 2, conf, debugLogger, debugLogger)
 	process2 := NewProcess("env_test", 2, configWEnv, debugLogger, debugLogger)
 
 	process1.start()

proxy/processgroup.go

@@ -5,12 +5,14 @@ import (
 	"net/http"
 	"slices"
 	"sync"
+
+	"github.com/mostlygeek/llama-swap/proxy/config"
 )
 
 type ProcessGroup struct {
 	sync.Mutex
 
-	config     Config
+	config     config.Config
 	id         string
 	swap       bool
 	exclusive  bool
@@ -24,7 +26,7 @@ type ProcessGroup struct {
 	lastUsedProcess string
 }
 
-func NewProcessGroup(id string, config Config, proxyLogger *LogMonitor, upstreamLogger *LogMonitor) *ProcessGroup {
+func NewProcessGroup(id string, config config.Config, proxyLogger *LogMonitor, upstreamLogger *LogMonitor) *ProcessGroup {
 	groupConfig, ok := config.Groups[id]
 	if !ok {
 		panic("Unable to find configuration for group id: " + id)
@@ -60,10 +62,20 @@ func (pg *ProcessGroup) ProxyRequest(modelID string, writer http.ResponseWriter,
 	if pg.swap {
 		pg.Lock()
 		if pg.lastUsedProcess != modelID {
+
+			// is there something already running?
 			if pg.lastUsedProcess != "" {
 				pg.processes[pg.lastUsedProcess].Stop()
 			}
+
+			// wait for the request to the new model to be fully handled
+			// and prevent race conditions see issue #277
+			pg.processes[modelID].ProxyRequest(writer, request)
 			pg.lastUsedProcess = modelID
+
+			// short circuit and exit
+			pg.Unlock()
+			return nil
 		}
 		pg.Unlock()
 	}
@@ -76,6 +88,29 @@ func (pg *ProcessGroup) HasMember(modelName string) bool {
 	return slices.Contains(pg.config.Groups[pg.id].Members, modelName)
 }
 
+func (pg *ProcessGroup) StopProcess(modelID string, strategy StopStrategy) error {
+	pg.Lock()
+
+	process, exists := pg.processes[modelID]
+	if !exists {
+		pg.Unlock()
+		return fmt.Errorf("process not found for %s", modelID)
+	}
+
+	if pg.lastUsedProcess == modelID {
+		pg.lastUsedProcess = ""
+	}
+	pg.Unlock()
+
+	switch strategy {
+	case StopImmediately:
+		process.StopImmediately()
+	default:
+		process.Stop()
+	}
+	return nil
+}
+
 func (pg *ProcessGroup) StopProcesses(strategy StopStrategy) {
 	pg.Lock()
 	defer pg.Unlock()

proxy/processgroup_test.go

@@ -4,21 +4,23 @@ import (
 	"bytes"
 	"net/http"
 	"net/http/httptest"
+	"sync"
 	"testing"
 
+	"github.com/mostlygeek/llama-swap/proxy/config"
 	"github.com/stretchr/testify/assert"
 )
 
-var processGroupTestConfig = AddDefaultGroupToConfig(Config{
+var processGroupTestConfig = config.AddDefaultGroupToConfig(config.Config{
 	HealthCheckTimeout: 15,
-	Models: map[string]ModelConfig{
+	Models: map[string]config.ModelConfig{
 		"model1": getTestSimpleResponderConfig("model1"),
 		"model2": getTestSimpleResponderConfig("model2"),
 		"model3": getTestSimpleResponderConfig("model3"),
 		"model4": getTestSimpleResponderConfig("model4"),
 		"model5": getTestSimpleResponderConfig("model5"),
 	},
-	Groups: map[string]GroupConfig{
+	Groups: map[string]config.GroupConfig{
 		"G1": {
 			Swap:      true,
 			Exclusive: true,
@@ -33,7 +35,7 @@ var processGroupTestConfig = AddDefaultGroupToConfig(Config{
 })
 
 func TestProcessGroup_DefaultHasCorrectModel(t *testing.T) {
-	pg := NewProcessGroup(DEFAULT_GROUP_ID, processGroupTestConfig, testLogger, testLogger)
+	pg := NewProcessGroup(config.DEFAULT_GROUP_ID, processGroupTestConfig, testLogger, testLogger)
 	assert.True(t, pg.HasMember("model5"))
 }
 
@@ -44,32 +46,49 @@ func TestProcessGroup_HasMember(t *testing.T) {
 	assert.False(t, pg.HasMember("model3"))
 }
 
-func TestProcessGroup_ProxyRequestSwapIsTrue(t *testing.T) {
+// TestProcessGroup_ProxyRequestSwapIsTrueParallel tests that when swap is true
+// and multiple requests are made in parallel, only one process is running at a time.
+func TestProcessGroup_ProxyRequestSwapIsTrueParallel(t *testing.T) {
+	var processGroupTestConfig = config.AddDefaultGroupToConfig(config.Config{
+		HealthCheckTimeout: 15,
+		Models: map[string]config.ModelConfig{
+			// use the same listening so if a model is already running, it will fail
+			// this is a way to test that swap isolation is working
+			// properly when there are parallel requests made at the
+			// same time.
+			"model1": getTestSimpleResponderConfigPort("model1", 9832),
+			"model2": getTestSimpleResponderConfigPort("model2", 9832),
+			"model3": getTestSimpleResponderConfigPort("model3", 9832),
+			"model4": getTestSimpleResponderConfigPort("model4", 9832),
+			"model5": getTestSimpleResponderConfigPort("model5", 9832),
+		},
+		Groups: map[string]config.GroupConfig{
+			"G1": {
+				Swap:    true,
+				Members: []string{"model1", "model2", "model3", "model4", "model5"},
+			},
+		},
+	})
+
 	pg := NewProcessGroup("G1", processGroupTestConfig, testLogger, testLogger)
 	defer pg.StopProcesses(StopWaitForInflightRequest)
 
-	tests := []string{"model1", "model2"}
+	tests := []string{"model1", "model2", "model3", "model4", "model5"}
 
+	var wg sync.WaitGroup
+
+	wg.Add(len(tests))
 	for _, modelName := range tests {
-		t.Run(modelName, func(t *testing.T) {
-			reqBody := `{"x", "y"}`
-			req := httptest.NewRequest("POST", "/v1/chat/completions", bytes.NewBufferString(reqBody))
+		go func(modelName string) {
+			defer wg.Done()
+			req := httptest.NewRequest("POST", "/v1/chat/completions", nil)
 			w := httptest.NewRecorder()
-
 			assert.NoError(t, pg.ProxyRequest(modelName, w, req))
 			assert.Equal(t, http.StatusOK, w.Code)
 			assert.Contains(t, w.Body.String(), modelName)
-
-			// make sure only one process is in the running state
-			count := 0
-			for _, process := range pg.processes {
-				if process.CurrentState() == StateReady {
-					count++
-				}
-			}
-			assert.Equal(t, 1, count)
-		})
+		}(modelName)
 	}
+	wg.Wait()
 }
 
 func TestProcessGroup_ProxyRequestSwapIsFalse(t *testing.T) {

proxy/proxymanager.go

@@ -16,6 +16,7 @@ import (
 
 	"github.com/gin-gonic/gin"
 	"github.com/mostlygeek/llama-swap/event"
+	"github.com/mostlygeek/llama-swap/proxy/config"
 	"github.com/tidwall/gjson"
 	"github.com/tidwall/sjson"
 )
@@ -27,7 +28,7 @@ const (
 type ProxyManager struct {
 	sync.Mutex
 
-	config    Config
+	config    config.Config
 	ginEngine *gin.Engine
 
 	// logging
@@ -44,7 +45,7 @@ type ProxyManager struct {
 	shutdownCancel context.CancelFunc
 }
 
-func New(config Config) *ProxyManager {
+func New(config config.Config) *ProxyManager {
 	// set up loggers
 	stdoutLogger := NewLogMonitorWriter(os.Stdout)
 	upstreamLogger := NewLogMonitorWriter(stdoutLogger)
@@ -191,11 +192,20 @@ func (pm *ProxyManager) setupGinEngine() {
 	// Support legacy /v1/completions api, see issue #12
 	pm.ginEngine.POST("/v1/completions", mm, pm.proxyOAIHandler)
 
-	// Support embeddings
+	// Support embeddings and reranking
 	pm.ginEngine.POST("/v1/embeddings", mm, pm.proxyOAIHandler)
+
+	// llama-server's /reranking endpoint + aliases
+	pm.ginEngine.POST("/reranking", mm, pm.proxyOAIHandler)
+	pm.ginEngine.POST("/rerank", mm, pm.proxyOAIHandler)
 	pm.ginEngine.POST("/v1/rerank", mm, pm.proxyOAIHandler)
 	pm.ginEngine.POST("/v1/reranking", mm, pm.proxyOAIHandler)
-	pm.ginEngine.POST("/rerank", mm, pm.proxyOAIHandler)
+
+	// llama-server's /infill endpoint for code infilling
+	pm.ginEngine.POST("/infill", mm, pm.proxyOAIHandler)
+
+	// llama-server's /completion endpoint
+	pm.ginEngine.POST("/completion", mm, pm.proxyOAIHandler)
 
 	// Support audio/speech endpoint
 	pm.ginEngine.POST("/v1/audio/speech", pm.proxyOAIHandler)
@@ -218,8 +228,7 @@ func (pm *ProxyManager) setupGinEngine() {
 	pm.ginEngine.GET("/upstream", func(c *gin.Context) {
 		c.Redirect(http.StatusFound, "/ui/models")
 	})
-	pm.ginEngine.Any("/upstream/:model_id/*upstreamPath", pm.proxyToUpstream)
-
+	pm.ginEngine.Any("/upstream/*upstreamPath", pm.proxyToUpstream)
 	pm.ginEngine.GET("/unload", pm.unloadAllModelsHandler)
 	pm.ginEngine.GET("/running", pm.listRunningProcessesHandler)
 	pm.ginEngine.GET("/health", func(c *gin.Context) {
@@ -361,6 +370,13 @@ func (pm *ProxyManager) listModelsHandler(c *gin.Context) {
 			record["description"] = desc
 		}
 
+		// Add metadata if present
+		if len(modelConfig.Metadata) > 0 {
+			record["meta"] = gin.H{
+				"llamaswap": modelConfig.Metadata,
+			}
+		}
+
 		data = append(data, record)
 	}
 
@@ -384,21 +400,68 @@ func (pm *ProxyManager) listModelsHandler(c *gin.Context) {
 }
 
 func (pm *ProxyManager) proxyToUpstream(c *gin.Context) {
-	requestedModel := c.Param("model_id")
+	upstreamPath := c.Param("upstreamPath")
 
-	if requestedModel == "" {
+	// split the upstream path by / and search for the model name
+	parts := strings.Split(strings.TrimSpace(upstreamPath), "/")
+	if len(parts) == 0 {
 		pm.sendErrorResponse(c, http.StatusBadRequest, "model id required in path")
 		return
 	}
 
-	processGroup, realModelName, err := pm.swapProcessGroup(requestedModel)
+	modelFound := false
+	searchModelName := ""
+	var modelName, remainingPath string
+	for i, part := range parts {
+		if parts[i] == "" {
+			continue
+		}
+
+		if searchModelName == "" {
+			searchModelName = part
+		} else {
+			searchModelName = searchModelName + "/" + parts[i]
+		}
+
+		if real, ok := pm.config.RealModelName(searchModelName); ok {
+			modelName = real
+			remainingPath = "/" + strings.Join(parts[i+1:], "/")
+			modelFound = true
+
+			// Check if this is exactly a model name with no additional path
+			// and doesn't end with a trailing slash
+			if remainingPath == "/" && !strings.HasSuffix(upstreamPath, "/") {
+				// Build new URL with query parameters preserved
+				newPath := "/upstream/" + searchModelName + "/"
+				if c.Request.URL.RawQuery != "" {
+					newPath += "?" + c.Request.URL.RawQuery
+				}
+
+				// Use 308 for non-GET/HEAD requests to preserve method
+				if c.Request.Method == http.MethodGet || c.Request.Method == http.MethodHead {
+					c.Redirect(http.StatusMovedPermanently, newPath)
+				} else {
+					c.Redirect(http.StatusPermanentRedirect, newPath)
+				}
+				return
+			}
+			break
+		}
+	}
+
+	if !modelFound {
+		pm.sendErrorResponse(c, http.StatusBadRequest, "model id required in path")
+		return
+	}
+
+	processGroup, realModelName, err := pm.swapProcessGroup(modelName)
 	if err != nil {
 		pm.sendErrorResponse(c, http.StatusInternalServerError, fmt.Sprintf("error swapping process group: %s", err.Error()))
 		return
 	}
 
 	// rewrite the path
-	c.Request.URL.Path = c.Param("upstreamPath")
+	c.Request.URL.Path = remainingPath
 	processGroup.ProxyRequest(realModelName, c.Writer, c.Request)
 }
 

proxy/proxymanager_api.go

@@ -3,8 +3,10 @@ package proxy
 import (
 	"context"
 	"encoding/json"
+	"fmt"
 	"net/http"
 	"sort"
+	"strings"
 
 	"github.com/gin-gonic/gin"
 	"github.com/mostlygeek/llama-swap/event"
@@ -23,6 +25,7 @@ func addApiHandlers(pm *ProxyManager) {
 	apiGroup := pm.ginEngine.Group("/api")
 	{
 		apiGroup.POST("/models/unload", pm.apiUnloadAllModels)
+		apiGroup.POST("/models/unload/*model", pm.apiUnloadSingleModelHandler)
 		apiGroup.GET("/events", pm.apiSendEvents)
 		apiGroup.GET("/metrics", pm.apiGetMetrics)
 	}
@@ -100,6 +103,8 @@ func (pm *ProxyManager) apiSendEvents(c *gin.Context) {
 	c.Header("Cache-Control", "no-cache")
 	c.Header("Connection", "keep-alive")
 	c.Header("X-Content-Type-Options", "nosniff")
+	// prevent nginx from buffering SSE
+	c.Header("X-Accel-Buffering", "no")
 
 	sendBuffer := make(chan messageEnvelope, 25)
 	ctx, cancel := context.WithCancel(c.Request.Context())
@@ -200,3 +205,25 @@ func (pm *ProxyManager) apiGetMetrics(c *gin.Context) {
 	}
 	c.Data(http.StatusOK, "application/json", jsonData)
 }
+
+func (pm *ProxyManager) apiUnloadSingleModelHandler(c *gin.Context) {
+	requestedModel := strings.TrimPrefix(c.Param("model"), "/")
+	realModelName, found := pm.config.RealModelName(requestedModel)
+	if !found {
+		pm.sendErrorResponse(c, http.StatusNotFound, "Model not found")
+		return
+	}
+
+	processGroup := pm.findGroupByModelName(realModelName)
+	if processGroup == nil {
+		pm.sendErrorResponse(c, http.StatusInternalServerError, fmt.Sprintf("process group not found for model %s", requestedModel))
+		return
+	}
+
+	if err := processGroup.StopProcess(realModelName, StopImmediately); err != nil {
+		pm.sendErrorResponse(c, http.StatusInternalServerError, fmt.Sprintf("error stopping process: %s", err.Error()))
+		return
+	} else {
+		c.String(http.StatusOK, "OK")
+	}
+}

proxy/proxymanager_loghandlers.go

@@ -28,6 +28,8 @@ func (pm *ProxyManager) streamLogsHandler(c *gin.Context) {
 	c.Header("Content-Type", "text/plain")
 	c.Header("Transfer-Encoding", "chunked")
 	c.Header("X-Content-Type-Options", "nosniff")
+	// prevent nginx from buffering streamed logs
+	c.Header("X-Accel-Buffering", "no")
 
 	logMonitorId := c.Param("logMonitorID")
 	logger, err := pm.getLogger(logMonitorId)

proxy/proxymanager_test.go

@@ -2,6 +2,7 @@ package proxy
 
 import (
 	"bytes"
+	"context"
 	"encoding/json"
 	"fmt"
 	"math/rand"
@@ -15,14 +16,15 @@ import (
 	"time"
 
 	"github.com/mostlygeek/llama-swap/event"
+	"github.com/mostlygeek/llama-swap/proxy/config"
 	"github.com/stretchr/testify/assert"
 	"github.com/tidwall/gjson"
 )
 
 func TestProxyManager_SwapProcessCorrectly(t *testing.T) {
-	config := AddDefaultGroupToConfig(Config{
+	config := config.AddDefaultGroupToConfig(config.Config{
 		HealthCheckTimeout: 15,
-		Models: map[string]ModelConfig{
+		Models: map[string]config.ModelConfig{
 			"model1": getTestSimpleResponderConfig("model1"),
 			"model2": getTestSimpleResponderConfig("model2"),
 		},
@@ -42,16 +44,15 @@ func TestProxyManager_SwapProcessCorrectly(t *testing.T) {
 		assert.Contains(t, w.Body.String(), modelName)
 	}
 }
-
 func TestProxyManager_SwapMultiProcess(t *testing.T) {
-	config := AddDefaultGroupToConfig(Config{
+	config := config.AddDefaultGroupToConfig(config.Config{
 		HealthCheckTimeout: 15,
-		Models: map[string]ModelConfig{
+		Models: map[string]config.ModelConfig{
 			"model1": getTestSimpleResponderConfig("model1"),
 			"model2": getTestSimpleResponderConfig("model2"),
 		},
 		LogLevel: "error",
-		Groups: map[string]GroupConfig{
+		Groups: map[string]config.GroupConfig{
 			"G1": {
 				Swap:      true,
 				Exclusive: false,
@@ -89,14 +90,14 @@ func TestProxyManager_SwapMultiProcess(t *testing.T) {
 // Test that a persistent group is not affected by the swapping behaviour of
 // other groups.
 func TestProxyManager_PersistentGroupsAreNotSwapped(t *testing.T) {
-	config := AddDefaultGroupToConfig(Config{
+	config := config.AddDefaultGroupToConfig(config.Config{
 		HealthCheckTimeout: 15,
-		Models: map[string]ModelConfig{
+		Models: map[string]config.ModelConfig{
 			"model1": getTestSimpleResponderConfig("model1"), // goes into the default group
 			"model2": getTestSimpleResponderConfig("model2"),
 		},
 		LogLevel: "error",
-		Groups: map[string]GroupConfig{
+		Groups: map[string]config.GroupConfig{
 			// the forever group is persistent and should not be affected by model1
 			"forever": {
 				Swap:       true,
@@ -133,9 +134,9 @@ func TestProxyManager_SwapMultiProcessParallelRequests(t *testing.T) {
 		t.Skip("skipping slow test")
 	}
 
-	config := AddDefaultGroupToConfig(Config{
+	config := config.AddDefaultGroupToConfig(config.Config{
 		HealthCheckTimeout: 15,
-		Models: map[string]ModelConfig{
+		Models: map[string]config.ModelConfig{
 			"model1": getTestSimpleResponderConfig("model1"),
 			"model2": getTestSimpleResponderConfig("model2"),
 			"model3": getTestSimpleResponderConfig("model3"),
@@ -196,9 +197,9 @@ func TestProxyManager_ListModelsHandler(t *testing.T) {
 	model2Config.Name = "     " // empty whitespace only strings will get ignored
 	model2Config.Description = "  "
 
-	config := Config{
+	config := config.Config{
 		HealthCheckTimeout: 15,
-		Models: map[string]ModelConfig{
+		Models: map[string]config.ModelConfig{
 			"model1": model1Config,
 			"model2": model2Config,
 			"model3": getTestSimpleResponderConfig("model3"),
@@ -281,15 +282,99 @@ func TestProxyManager_ListModelsHandler(t *testing.T) {
 	assert.Empty(t, expectedModels, "not all expected models were returned")
 }
 
+func TestProxyManager_ListModelsHandler_WithMetadata(t *testing.T) {
+	// Process config through LoadConfigFromReader to apply macro substitution
+	configYaml := `
+healthCheckTimeout: 15
+logLevel: error
+startPort: 10000
+models:
+  model1:
+    cmd: /path/to/server -p ${PORT}
+    macros:
+      PORT_NUM: 10001
+      TEMP: 0.7
+      NAME: "llama"
+    metadata:
+      port: ${PORT_NUM}
+      temperature: ${TEMP}
+      enabled: true
+      note: "Running on port ${PORT_NUM}"
+      nested:
+        value: ${TEMP}
+  model2:
+    cmd: /path/to/server -p ${PORT}
+`
+	processedConfig, err := config.LoadConfigFromReader(strings.NewReader(configYaml))
+	assert.NoError(t, err)
+
+	proxy := New(processedConfig)
+
+	req := httptest.NewRequest("GET", "/v1/models", nil)
+	w := httptest.NewRecorder()
+	proxy.ServeHTTP(w, req)
+
+	assert.Equal(t, http.StatusOK, w.Code)
+
+	var response struct {
+		Data []map[string]any `json:"data"`
+	}
+
+	err = json.Unmarshal(w.Body.Bytes(), &response)
+	assert.NoError(t, err)
+	assert.Len(t, response.Data, 2)
+
+	// Find model1 and model2 in response
+	var model1Data, model2Data map[string]any
+	for _, model := range response.Data {
+		if model["id"] == "model1" {
+			model1Data = model
+		} else if model["id"] == "model2" {
+			model2Data = model
+		}
+	}
+
+	// Verify model1 has llamaswap_meta
+	assert.NotNil(t, model1Data)
+	meta, exists := model1Data["meta"]
+	if !assert.True(t, exists, "model1 should have meta key") {
+		t.FailNow()
+	}
+
+	metaMap := meta.(map[string]any)
+
+	lsmeta, exists := metaMap["llamaswap"]
+	if !assert.True(t, exists, "model1 should have meta.llamaswap key") {
+		t.FailNow()
+	}
+
+	lsmetamap := lsmeta.(map[string]any)
+
+	// Verify type preservation
+	assert.Equal(t, float64(10001), lsmetamap["port"]) // JSON numbers are float64
+	assert.Equal(t, 0.7, lsmetamap["temperature"])
+	assert.Equal(t, true, lsmetamap["enabled"])
+	// Verify string interpolation
+	assert.Equal(t, "Running on port 10001", lsmetamap["note"])
+	// Verify nested structure
+	nested := lsmetamap["nested"].(map[string]any)
+	assert.Equal(t, 0.7, nested["value"])
+
+	// Verify model2 does NOT have llamaswap_meta
+	assert.NotNil(t, model2Data)
+	_, exists = model2Data["llamaswap_meta"]
+	assert.False(t, exists, "model2 should not have llamaswap_meta")
+}
+
 func TestProxyManager_ListModelsHandler_SortedByID(t *testing.T) {
 	// Intentionally add models in non-sorted order and with an unlisted model
-	config := Config{
+	config := config.Config{
 		HealthCheckTimeout: 15,
-		Models: map[string]ModelConfig{
+		Models: map[string]config.ModelConfig{
 			"zeta":  getTestSimpleResponderConfig("zeta"),
 			"alpha": getTestSimpleResponderConfig("alpha"),
 			"beta":  getTestSimpleResponderConfig("beta"),
-			"hidden": func() ModelConfig {
+			"hidden": func() config.ModelConfig {
 				mc := getTestSimpleResponderConfig("hidden")
 				mc.Unlisted = true
 				return mc
@@ -337,15 +422,15 @@ func TestProxyManager_Shutdown(t *testing.T) {
 	model3Config := getTestSimpleResponderConfigPort("model3", 9993)
 	model3Config.Proxy = "http://localhost:10003/"
 
-	config := AddDefaultGroupToConfig(Config{
+	config := config.AddDefaultGroupToConfig(config.Config{
 		HealthCheckTimeout: 15,
-		Models: map[string]ModelConfig{
+		Models: map[string]config.ModelConfig{
 			"model1": model1Config,
 			"model2": model2Config,
 			"model3": model3Config,
 		},
 		LogLevel: "error",
-		Groups: map[string]GroupConfig{
+		Groups: map[string]config.GroupConfig{
 			"test": {
 				Swap:    false,
 				Members: []string{"model1", "model2", "model3"},
@@ -380,38 +465,92 @@ func TestProxyManager_Shutdown(t *testing.T) {
 }
 
 func TestProxyManager_Unload(t *testing.T) {
-	config := AddDefaultGroupToConfig(Config{
+	conf := config.AddDefaultGroupToConfig(config.Config{
 		HealthCheckTimeout: 15,
-		Models: map[string]ModelConfig{
+		Models: map[string]config.ModelConfig{
 			"model1": getTestSimpleResponderConfig("model1"),
 		},
 		LogLevel: "error",
 	})
 
-	proxy := New(config)
+	proxy := New(conf)
 	reqBody := fmt.Sprintf(`{"model":"%s"}`, "model1")
 	req := httptest.NewRequest("POST", "/v1/chat/completions", bytes.NewBufferString(reqBody))
 	w := httptest.NewRecorder()
 	proxy.ServeHTTP(w, req)
 
-	assert.Equal(t, proxy.processGroups[DEFAULT_GROUP_ID].processes["model1"].CurrentState(), StateReady)
+	assert.Equal(t, proxy.processGroups[config.DEFAULT_GROUP_ID].processes["model1"].CurrentState(), StateReady)
 	req = httptest.NewRequest("GET", "/unload", nil)
 	w = httptest.NewRecorder()
 	proxy.ServeHTTP(w, req)
 	assert.Equal(t, http.StatusOK, w.Code)
 	assert.Equal(t, w.Body.String(), "OK")
 
-	// give it a bit of time to stop
-	<-time.After(time.Millisecond * 250)
-	assert.Equal(t, proxy.processGroups[DEFAULT_GROUP_ID].processes["model1"].CurrentState(), StateStopped)
+	select {
+	case <-proxy.processGroups[config.DEFAULT_GROUP_ID].processes["model1"].cmdWaitChan:
+		// good
+	case <-time.After(2 * time.Second):
+		t.Fatal("timeout waiting for model1 to stop")
+	}
+	assert.Equal(t, proxy.processGroups[config.DEFAULT_GROUP_ID].processes["model1"].CurrentState(), StateStopped)
+}
+
+func TestProxyManager_UnloadSingleModel(t *testing.T) {
+	const testGroupId = "testGroup"
+	config := config.AddDefaultGroupToConfig(config.Config{
+		HealthCheckTimeout: 15,
+		Models: map[string]config.ModelConfig{
+			"model1": getTestSimpleResponderConfig("model1"),
+			"model2": getTestSimpleResponderConfig("model2"),
+		},
+		Groups: map[string]config.GroupConfig{
+			testGroupId: {
+				Swap:    false,
+				Members: []string{"model1", "model2"},
+			},
+		},
+		LogLevel: "error",
+	})
+
+	proxy := New(config)
+	defer proxy.StopProcesses(StopImmediately)
+
+	// start both model
+	for _, modelName := range []string{"model1", "model2"} {
+		reqBody := fmt.Sprintf(`{"model":"%s"}`, modelName)
+		req := httptest.NewRequest("POST", "/v1/chat/completions", bytes.NewBufferString(reqBody))
+		w := httptest.NewRecorder()
+		proxy.ServeHTTP(w, req)
+	}
+
+	assert.Equal(t, StateReady, proxy.processGroups[testGroupId].processes["model1"].CurrentState())
+	assert.Equal(t, StateReady, proxy.processGroups[testGroupId].processes["model2"].CurrentState())
+
+	req := httptest.NewRequest("POST", "/api/models/unload/model1", nil)
+	w := httptest.NewRecorder()
+	proxy.ServeHTTP(w, req)
+	assert.Equal(t, http.StatusOK, w.Code)
+	if !assert.Equal(t, w.Body.String(), "OK") {
+		t.FailNow()
+	}
+
+	select {
+	case <-proxy.processGroups[testGroupId].processes["model1"].cmdWaitChan:
+		// good
+	case <-time.After(2 * time.Second):
+		t.Fatal("timeout waiting for model1 to stop")
+	}
+
+	assert.Equal(t, proxy.processGroups[testGroupId].processes["model1"].CurrentState(), StateStopped)
+	assert.Equal(t, proxy.processGroups[testGroupId].processes["model2"].CurrentState(), StateReady)
 }
 
 // Test issue #61 `Listing the current list of models and the loaded model.`
 func TestProxyManager_RunningEndpoint(t *testing.T) {
 	// Shared configuration
-	config := AddDefaultGroupToConfig(Config{
+	config := config.AddDefaultGroupToConfig(config.Config{
 		HealthCheckTimeout: 15,
-		Models: map[string]ModelConfig{
+		Models: map[string]config.ModelConfig{
 			"model1": getTestSimpleResponderConfig("model1"),
 			"model2": getTestSimpleResponderConfig("model2"),
 		},
@@ -474,9 +613,9 @@ func TestProxyManager_RunningEndpoint(t *testing.T) {
 }
 
 func TestProxyManager_AudioTranscriptionHandler(t *testing.T) {
-	config := AddDefaultGroupToConfig(Config{
+	config := config.AddDefaultGroupToConfig(config.Config{
 		HealthCheckTimeout: 15,
-		Models: map[string]ModelConfig{
+		Models: map[string]config.ModelConfig{
 			"TheExpectedModel": getTestSimpleResponderConfig("TheExpectedModel"),
 		},
 		LogLevel: "error",
@@ -527,15 +666,15 @@ func TestProxyManager_UseModelName(t *testing.T) {
 	modelConfig := getTestSimpleResponderConfig(upstreamModelName)
 	modelConfig.UseModelName = upstreamModelName
 
-	config := AddDefaultGroupToConfig(Config{
+	conf := config.AddDefaultGroupToConfig(config.Config{
 		HealthCheckTimeout: 15,
-		Models: map[string]ModelConfig{
+		Models: map[string]config.ModelConfig{
 			"model1": modelConfig,
 		},
 		LogLevel: "error",
 	})
 
-	proxy := New(config)
+	proxy := New(conf)
 	defer proxy.StopProcesses(StopWaitForInflightRequest)
 
 	requestedModel := "model1"
@@ -590,9 +729,9 @@ func TestProxyManager_UseModelName(t *testing.T) {
 }
 
 func TestProxyManager_CORSOptionsHandler(t *testing.T) {
-	config := AddDefaultGroupToConfig(Config{
+	config := config.AddDefaultGroupToConfig(config.Config{
 		HealthCheckTimeout: 15,
-		Models: map[string]ModelConfig{
+		Models: map[string]config.ModelConfig{
 			"model1": getTestSimpleResponderConfig("model1"),
 		},
 		LogLevel: "error",
@@ -666,7 +805,7 @@ models:
     aliases: [model-alias]
 `, getSimpleResponderPath())
 
-	config, err := LoadConfigFromReader(strings.NewReader(configStr))
+	config, err := config.LoadConfigFromReader(strings.NewReader(configStr))
 	assert.NoError(t, err)
 
 	proxy := New(config)
@@ -689,9 +828,9 @@ models:
 }
 
 func TestProxyManager_ChatContentLength(t *testing.T) {
-	config := AddDefaultGroupToConfig(Config{
+	config := config.AddDefaultGroupToConfig(config.Config{
 		HealthCheckTimeout: 15,
-		Models: map[string]ModelConfig{
+		Models: map[string]config.ModelConfig{
 			"model1": getTestSimpleResponderConfig("model1"),
 		},
 		LogLevel: "error",
@@ -714,14 +853,14 @@ func TestProxyManager_ChatContentLength(t *testing.T) {
 
 func TestProxyManager_FiltersStripParams(t *testing.T) {
 	modelConfig := getTestSimpleResponderConfig("model1")
-	modelConfig.Filters = ModelFilters{
+	modelConfig.Filters = config.ModelFilters{
 		StripParams: "temperature, model, stream",
 	}
 
-	config := AddDefaultGroupToConfig(Config{
+	config := config.AddDefaultGroupToConfig(config.Config{
 		HealthCheckTimeout: 15,
 		LogLevel:           "error",
-		Models: map[string]ModelConfig{
+		Models: map[string]config.ModelConfig{
 			"model1": modelConfig,
 		},
 	})
@@ -747,9 +886,9 @@ func TestProxyManager_FiltersStripParams(t *testing.T) {
 }
 
 func TestProxyManager_MiddlewareWritesMetrics_NonStreaming(t *testing.T) {
-	config := AddDefaultGroupToConfig(Config{
+	config := config.AddDefaultGroupToConfig(config.Config{
 		HealthCheckTimeout: 15,
-		Models: map[string]ModelConfig{
+		Models: map[string]config.ModelConfig{
 			"model1": getTestSimpleResponderConfig("model1"),
 		},
 		LogLevel: "error",
@@ -782,9 +921,9 @@ func TestProxyManager_MiddlewareWritesMetrics_NonStreaming(t *testing.T) {
 }
 
 func TestProxyManager_MiddlewareWritesMetrics_Streaming(t *testing.T) {
-	config := AddDefaultGroupToConfig(Config{
+	config := config.AddDefaultGroupToConfig(config.Config{
 		HealthCheckTimeout: 15,
-		Models: map[string]ModelConfig{
+		Models: map[string]config.ModelConfig{
 			"model1": getTestSimpleResponderConfig("model1"),
 		},
 		LogLevel: "error",
@@ -817,9 +956,9 @@ func TestProxyManager_MiddlewareWritesMetrics_Streaming(t *testing.T) {
 }
 
 func TestProxyManager_HealthEndpoint(t *testing.T) {
-	config := AddDefaultGroupToConfig(Config{
+	config := config.AddDefaultGroupToConfig(config.Config{
 		HealthCheckTimeout: 15,
-		Models: map[string]ModelConfig{
+		Models: map[string]config.ModelConfig{
 			"model1": getTestSimpleResponderConfig("model1"),
 		},
 		LogLevel: "error",
@@ -834,6 +973,28 @@ func TestProxyManager_HealthEndpoint(t *testing.T) {
 	assert.Equal(t, "OK", rec.Body.String())
 }
 
+// Ensure the custom llama-server /completion endpoint proxies correctly
+func TestProxyManager_CompletionEndpoint(t *testing.T) {
+	config := config.AddDefaultGroupToConfig(config.Config{
+		HealthCheckTimeout: 15,
+		Models: map[string]config.ModelConfig{
+			"model1": getTestSimpleResponderConfig("model1"),
+		},
+		LogLevel: "error",
+	})
+
+	proxy := New(config)
+	defer proxy.StopProcesses(StopWaitForInflightRequest)
+
+	reqBody := `{"model":"model1"}`
+	req := httptest.NewRequest("POST", "/completion", bytes.NewBufferString(reqBody))
+	w := httptest.NewRecorder()
+
+	proxy.ServeHTTP(w, req)
+	assert.Equal(t, http.StatusOK, w.Code)
+	assert.Contains(t, w.Body.String(), "model1")
+}
+
 func TestProxyManager_StartupHooks(t *testing.T) {
 
 	// using real YAML as the configuration has gotten more complex
@@ -860,7 +1021,7 @@ models:
 `, "${simpleresponderpath}", simpleResponderPath, -1)
 
 	// Create a test model configuration
-	config, err := LoadConfigFromReader(strings.NewReader(configStr))
+	config, err := config.LoadConfigFromReader(strings.NewReader(configStr))
 	if !assert.NoError(t, err, "Invalid configuration") {
 		return
 	}
@@ -892,3 +1053,67 @@ models:
 	assert.Equal(t, StateReady, proxy.processGroups["preloadTestGroup"].processes["model1"].CurrentState())
 	assert.Equal(t, StateReady, proxy.processGroups["preloadTestGroup"].processes["model2"].CurrentState())
 }
+
+func TestProxyManager_StreamingEndpointsReturnNoBufferingHeader(t *testing.T) {
+	config := config.AddDefaultGroupToConfig(config.Config{
+		HealthCheckTimeout: 15,
+		Models: map[string]config.ModelConfig{
+			"model1": getTestSimpleResponderConfig("model1"),
+		},
+		LogLevel: "error",
+	})
+
+	proxy := New(config)
+	defer proxy.StopProcesses(StopWaitForInflightRequest)
+
+	endpoints := []string{
+		"/api/events",
+		"/logs/stream",
+		"/logs/stream/proxy",
+		"/logs/stream/upstream",
+	}
+
+	for _, endpoint := range endpoints {
+		t.Run(endpoint, func(t *testing.T) {
+			ctx, cancel := context.WithCancel(context.Background())
+			defer cancel()
+
+			req := httptest.NewRequest("GET", endpoint, nil)
+			req = req.WithContext(ctx)
+			rec := httptest.NewRecorder()
+
+			// We don't need the handler to fully complete, just to set the headers
+			// so run it in a goroutine and check the headers after a short delay
+			go proxy.ServeHTTP(rec, req)
+			time.Sleep(10 * time.Millisecond) // give it time to start and write headers
+
+			assert.Equal(t, http.StatusOK, rec.Code)
+			assert.Equal(t, "no", rec.Header().Get("X-Accel-Buffering"))
+		})
+	}
+}
+
+func TestProxyManager_ProxiedStreamingEndpointReturnsNoBufferingHeader(t *testing.T) {
+	config := config.AddDefaultGroupToConfig(config.Config{
+		HealthCheckTimeout: 15,
+		Models: map[string]config.ModelConfig{
+			"streaming-model": getTestSimpleResponderConfig("streaming-model"),
+		},
+		LogLevel: "error",
+	})
+
+	proxy := New(config)
+	defer proxy.StopProcesses(StopWaitForInflightRequest)
+
+	// Make a streaming request
+	reqBody := `{"model":"streaming-model"}`
+	// simple-responder will return text/event-stream when stream=true is in the query
+	req := httptest.NewRequest("POST", "/v1/chat/completions?stream=true", bytes.NewBufferString(reqBody))
+	rec := httptest.NewRecorder()
+
+	proxy.ServeHTTP(rec, req)
+
+	assert.Equal(t, http.StatusOK, rec.Code)
+	assert.Equal(t, "no", rec.Header().Get("X-Accel-Buffering"))
+	assert.Contains(t, rec.Header().Get("Content-Type"), "text/event-stream")
+}

ui/package-lock.json

@@ -8,17 +8,15 @@
       "name": "ui",
       "version": "0.0.0",
       "dependencies": {
-        "@tailwindcss/vite": "^4.1.8",
-        "@tanstack/react-query": "^5.80.6",
         "react": "^19.1.0",
         "react-dom": "^19.1.0",
         "react-icons": "^5.5.0",
         "react-resizable-panels": "^3.0.4",
-        "react-router-dom": "^7.6.2",
-        "tailwindcss": "^4.1.8"
+        "react-router-dom": "^7.6.2"
       },
       "devDependencies": {
         "@eslint/js": "^9.25.0",
+        "@tailwindcss/vite": "^4.1.8",
         "@types/react": "^19.1.2",
         "@types/react-dom": "^19.1.2",
         "@vitejs/plugin-react": "^4.4.1",
@@ -26,6 +24,7 @@
         "eslint-plugin-react-hooks": "^5.2.0",
         "eslint-plugin-react-refresh": "^0.4.19",
         "globals": "^16.0.0",
+        "tailwindcss": "^4.1.8",
         "typescript": "~5.8.3",
         "typescript-eslint": "^8.30.1",
         "vite": "^6.3.5"
@@ -35,6 +34,7 @@
       "version": "2.3.0",
       "resolved": "https://registry.npmjs.org/@ampproject/remapping/-/remapping-2.3.0.tgz",
       "integrity": "sha512-30iZtAPgz+LTIYoeivqYo853f02jBYSd5uGnGpkFV0M3xOt9aN73erkgYAmZU43x4VfqcnLxW9Kpg3R5LC4YYw==",
+      "dev": true,
       "license": "Apache-2.0",
       "dependencies": {
         "@jridgewell/gen-mapping": "^0.3.5",
@@ -333,6 +333,7 @@
       "cpu": [
         "ppc64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -349,6 +350,7 @@
       "cpu": [
         "arm"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -365,6 +367,7 @@
       "cpu": [
         "arm64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -381,6 +384,7 @@
       "cpu": [
         "x64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -397,6 +401,7 @@
       "cpu": [
         "arm64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -413,6 +418,7 @@
       "cpu": [
         "x64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -429,6 +435,7 @@
       "cpu": [
         "arm64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -445,6 +452,7 @@
       "cpu": [
         "x64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -461,6 +469,7 @@
       "cpu": [
         "arm"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -477,6 +486,7 @@
       "cpu": [
         "arm64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -493,6 +503,7 @@
       "cpu": [
         "ia32"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -509,6 +520,7 @@
       "cpu": [
         "loong64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -525,6 +537,7 @@
       "cpu": [
         "mips64el"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -541,6 +554,7 @@
       "cpu": [
         "ppc64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -557,6 +571,7 @@
       "cpu": [
         "riscv64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -573,6 +588,7 @@
       "cpu": [
         "s390x"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -589,6 +605,7 @@
       "cpu": [
         "x64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -605,6 +622,7 @@
       "cpu": [
         "arm64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -621,6 +639,7 @@
       "cpu": [
         "x64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -637,6 +656,7 @@
       "cpu": [
         "arm64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -653,6 +673,7 @@
       "cpu": [
         "x64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -669,6 +690,7 @@
       "cpu": [
         "x64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -685,6 +707,7 @@
       "cpu": [
         "arm64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -701,6 +724,7 @@
       "cpu": [
         "ia32"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -717,6 +741,7 @@
       "cpu": [
         "x64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -950,6 +975,7 @@
       "version": "4.0.1",
       "resolved": "https://registry.npmjs.org/@isaacs/fs-minipass/-/fs-minipass-4.0.1.tgz",
       "integrity": "sha512-wgm9Ehl2jpeqP3zw/7mo3kRHFp5MEDhqAdwy1fTGkHAwnkGOVsgpvQhL8B5n1qlb01jV3n/bI0ZfZp5lWA1k4w==",
+      "dev": true,
       "license": "ISC",
       "dependencies": {
         "minipass": "^7.0.4"
@@ -962,6 +988,7 @@
       "version": "0.3.8",
       "resolved": "https://registry.npmjs.org/@jridgewell/gen-mapping/-/gen-mapping-0.3.8.tgz",
       "integrity": "sha512-imAbBGkb+ebQyxKgzv5Hu2nmROxoDOXHh80evxdoXNOrvAnVx7zimzc1Oo5h9RlfV4vPXaE2iM5pOFbvOCClWA==",
+      "dev": true,
       "license": "MIT",
       "dependencies": {
         "@jridgewell/set-array": "^1.2.1",
@@ -976,6 +1003,7 @@
       "version": "3.1.2",
       "resolved": "https://registry.npmjs.org/@jridgewell/resolve-uri/-/resolve-uri-3.1.2.tgz",
       "integrity": "sha512-bRISgCIjP20/tbWSPWMEi54QVPRZExkuD9lJL+UIxUKtwVJA8wW1Trb1jMs1RFXo1CBTNZ/5hpC9QvmKWdopKw==",
+      "dev": true,
       "license": "MIT",
       "engines": {
         "node": ">=6.0.0"
@@ -985,6 +1013,7 @@
       "version": "1.2.1",
       "resolved": "https://registry.npmjs.org/@jridgewell/set-array/-/set-array-1.2.1.tgz",
       "integrity": "sha512-R8gLRTZeyp03ymzP/6Lil/28tGeGEzhx1q2k703KGWRAI1VdvPIXdG70VJc2pAMw3NA6JKL5hhFu1sJX0Mnn/A==",
+      "dev": true,
       "license": "MIT",
       "engines": {
         "node": ">=6.0.0"
@@ -994,12 +1023,14 @@
       "version": "1.5.0",
       "resolved": "https://registry.npmjs.org/@jridgewell/sourcemap-codec/-/sourcemap-codec-1.5.0.tgz",
       "integrity": "sha512-gv3ZRaISU3fjPAgNsriBRqGWQL6quFx04YMPW/zD8XMLsU32mhCCbfbO6KZFLjvYpCZ8zyDEgqsgf+PwPaM7GQ==",
+      "dev": true,
       "license": "MIT"
     },
     "node_modules/@jridgewell/trace-mapping": {
       "version": "0.3.25",
       "resolved": "https://registry.npmjs.org/@jridgewell/trace-mapping/-/trace-mapping-0.3.25.tgz",
       "integrity": "sha512-vNk6aEwybGtawWmy/PzwnGDOjCkLWSD2wqvjGGAgOAwCGWySYXfYoxt00IJkTF+8Lb57DwOb3Aa0o9CApepiYQ==",
+      "dev": true,
       "license": "MIT",
       "dependencies": {
         "@jridgewell/resolve-uri": "^3.1.0",
@@ -1058,6 +1089,7 @@
       "cpu": [
         "arm"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -1071,6 +1103,7 @@
       "cpu": [
         "arm64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -1084,6 +1117,7 @@
       "cpu": [
         "arm64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -1097,6 +1131,7 @@
       "cpu": [
         "x64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -1110,6 +1145,7 @@
       "cpu": [
         "arm64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -1123,6 +1159,7 @@
       "cpu": [
         "x64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -1136,6 +1173,7 @@
       "cpu": [
         "arm"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -1149,6 +1187,7 @@
       "cpu": [
         "arm"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -1162,6 +1201,7 @@
       "cpu": [
         "arm64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -1175,6 +1215,7 @@
       "cpu": [
         "arm64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -1188,6 +1229,7 @@
       "cpu": [
         "loong64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -1201,6 +1243,7 @@
       "cpu": [
         "ppc64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -1214,6 +1257,7 @@
       "cpu": [
         "riscv64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -1227,6 +1271,7 @@
       "cpu": [
         "riscv64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -1240,6 +1285,7 @@
       "cpu": [
         "s390x"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -1253,6 +1299,7 @@
       "cpu": [
         "x64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -1266,6 +1313,7 @@
       "cpu": [
         "x64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -1279,6 +1327,7 @@
       "cpu": [
         "arm64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -1292,6 +1341,7 @@
       "cpu": [
         "ia32"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -1305,6 +1355,7 @@
       "cpu": [
         "x64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -1315,6 +1366,7 @@
       "version": "4.1.8",
       "resolved": "https://registry.npmjs.org/@tailwindcss/node/-/node-4.1.8.tgz",
       "integrity": "sha512-OWwBsbC9BFAJelmnNcrKuf+bka2ZxCE2A4Ft53Tkg4uoiE67r/PMEYwCsourC26E+kmxfwE0hVzMdxqeW+xu7Q==",
+      "dev": true,
       "license": "MIT",
       "dependencies": {
         "@ampproject/remapping": "^2.3.0",
@@ -1330,6 +1382,7 @@
       "version": "4.1.8",
       "resolved": "https://registry.npmjs.org/@tailwindcss/oxide/-/oxide-4.1.8.tgz",
       "integrity": "sha512-d7qvv9PsM5N3VNKhwVUhpK6r4h9wtLkJ6lz9ZY9aeZgrUWk1Z8VPyqyDT9MZlem7GTGseRQHkeB1j3tC7W1P+A==",
+      "dev": true,
       "hasInstallScript": true,
       "license": "MIT",
       "dependencies": {
@@ -1361,6 +1414,7 @@
       "cpu": [
         "arm64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -1377,6 +1431,7 @@
       "cpu": [
         "arm64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -1393,6 +1448,7 @@
       "cpu": [
         "x64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -1409,6 +1465,7 @@
       "cpu": [
         "x64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -1425,6 +1482,7 @@
       "cpu": [
         "arm"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -1441,6 +1499,7 @@
       "cpu": [
         "arm64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -1457,6 +1516,7 @@
       "cpu": [
         "arm64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -1473,6 +1533,7 @@
       "cpu": [
         "x64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -1489,6 +1550,7 @@
       "cpu": [
         "x64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -1513,6 +1575,7 @@
       "cpu": [
         "wasm32"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "dependencies": {
@@ -1534,6 +1597,7 @@
       "cpu": [
         "arm64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -1550,6 +1614,7 @@
       "cpu": [
         "x64"
       ],
+      "dev": true,
       "license": "MIT",
       "optional": true,
       "os": [
@@ -1563,6 +1628,7 @@
       "version": "4.1.8",
       "resolved": "https://registry.npmjs.org/@tailwindcss/vite/-/vite-4.1.8.tgz",
       "integrity": "sha512-CQ+I8yxNV5/6uGaJjiuymgw0kEQiNKRinYbZXPdx1fk5WgiyReG0VaUx/Xq6aVNSUNJFzxm6o8FNKS5aMaim5A==",
+      "dev": true,
       "license": "MIT",
       "dependencies": {
         "@tailwindcss/node": "4.1.8",
@@ -1573,32 +1639,6 @@
         "vite": "^5.2.0 || ^6"
       }
     },
-    "node_modules/@tanstack/query-core": {
-      "version": "5.80.6",
-      "resolved": "https://registry.npmjs.org/@tanstack/query-core/-/query-core-5.80.6.tgz",
-      "integrity": "sha512-nl7YxT/TAU+VTf+e2zTkObGTyY8YZBMnbgeA1ee66lIVqzKlYursAII6z5t0e6rXgwUMJSV4dshBTNacNpZHbQ==",
-      "license": "MIT",
-      "funding": {
-        "type": "github",
-        "url": "https://github.com/sponsors/tannerlinsley"
-      }
-    },
-    "node_modules/@tanstack/react-query": {
-      "version": "5.80.6",
-      "resolved": "https://registry.npmjs.org/@tanstack/react-query/-/react-query-5.80.6.tgz",
-      "integrity": "sha512-izX+5CnkpON3NQGcEm3/d7LfFQNo9ZpFtX2QsINgCYK9LT2VCIdi8D3bMaMSNhrAJCznRoAkFic76uvLroALBw==",
-      "license": "MIT",
-      "dependencies": {
-        "@tanstack/query-core": "5.80.6"
-      },
-      "funding": {
-        "type": "github",
-        "url": "https://github.com/sponsors/tannerlinsley"
-      },
-      "peerDependencies": {
-        "react": "^18 || ^19"
-      }
-    },
     "node_modules/@types/babel__core": {
       "version": "7.20.5",
       "resolved": "https://registry.npmjs.org/@types/babel__core/-/babel__core-7.20.5.tgz",
@@ -2148,6 +2188,7 @@
       "version": "3.0.0",
       "resolved": "https://registry.npmjs.org/chownr/-/chownr-3.0.0.tgz",
       "integrity": "sha512-+IxzY9BZOQd/XuYPRmrvEVjF/nqj5kgT4kEq7VofrDoM1MxoRjEWkrCC3EtLi59TVawxTAn+orJwFQcrqEN1+g==",
+      "dev": true,
       "license": "BlueOak-1.0.0",
       "engines": {
         "node": ">=18"
@@ -2247,6 +2288,7 @@
       "version": "2.0.4",
       "resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-2.0.4.tgz",
       "integrity": "sha512-3UDv+G9CsCKO1WKMGw9fwq/SWJYbI0c5Y7LU1AXYoDdbhE2AHQ6N6Nb34sG8Fj7T5APy8qXDCKuuIHd1BR0tVA==",
+      "dev": true,
       "license": "Apache-2.0",
       "engines": {
         "node": ">=8"
@@ -2263,6 +2305,7 @@
       "version": "5.18.1",
       "resolved": "https://registry.npmjs.org/enhanced-resolve/-/enhanced-resolve-5.18.1.tgz",
       "integrity": "sha512-ZSW3ma5GkcQBIpwZTSRAI8N71Uuwgs93IezB7mf7R60tC8ZbJideoDNKjHn2O9KIlx6rkGTTEk1xUCK2E1Y2Yg==",
+      "dev": true,
       "license": "MIT",
       "dependencies": {
         "graceful-fs": "^4.2.4",
@@ -2276,6 +2319,7 @@
       "version": "0.25.5",
       "resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.25.5.tgz",
       "integrity": "sha512-P8OtKZRv/5J5hhz0cUAdu/cLuPIKXpQl1R9pZtvmHWQvrAUVd0UNIPT4IB4W3rNOqVO0rlqHmCIbSwxh/c9yUQ==",
+      "dev": true,
       "hasInstallScript": true,
       "license": "MIT",
       "bin": {
@@ -2642,6 +2686,7 @@
       "version": "2.3.3",
       "resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz",
       "integrity": "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==",
+      "dev": true,
       "hasInstallScript": true,
       "license": "MIT",
       "optional": true,
@@ -2692,6 +2737,7 @@
       "version": "4.2.11",
       "resolved": "https://registry.npmjs.org/graceful-fs/-/graceful-fs-4.2.11.tgz",
       "integrity": "sha512-RbJ5/jmFcNNCcDV5o9eTnBLJ/HszWV0P73bc+Ff4nS/rJj+YaS6IGyiOL0VoBYX+l1Wrl3k63h/KrH+nhJ0XvQ==",
+      "dev": true,
       "license": "ISC"
     },
     "node_modules/graphemer": {
@@ -2792,6 +2838,7 @@
       "version": "2.4.2",
       "resolved": "https://registry.npmjs.org/jiti/-/jiti-2.4.2.tgz",
       "integrity": "sha512-rg9zJN+G4n2nfJl5MW3BMygZX56zKPNVEYYqq7adpmMh4Jn2QNEwhvQlFy6jPVdcod7txZtKHWnyZiA3a0zP7A==",
+      "dev": true,
       "license": "MIT",
       "bin": {
         "jiti": "lib/jiti-cli.mjs"
@@ -2892,6 +2939,7 @@
       "version": "1.30.1",
       "resolved": "https://registry.npmjs.org/lightningcss/-/lightningcss-1.30.1.tgz",
       "integrity": "sha512-xi6IyHML+c9+Q3W0S4fCQJOym42pyurFiJUHEcEyHS0CeKzia4yZDEsLlqOFykxOdHpNy0NmvVO31vcSqAxJCg==",
+      "dev": true,
       "license": "MPL-2.0",
       "dependencies": {
         "detect-libc": "^2.0.3"
@@ -2923,6 +2971,7 @@
       "cpu": [
         "arm64"
       ],
+      "dev": true,
       "license": "MPL-2.0",
       "optional": true,
       "os": [
@@ -2943,6 +2992,7 @@
       "cpu": [
         "x64"
       ],
+      "dev": true,
       "license": "MPL-2.0",
       "optional": true,
       "os": [
@@ -2963,6 +3013,7 @@
       "cpu": [
         "x64"
       ],
+      "dev": true,
       "license": "MPL-2.0",
       "optional": true,
       "os": [
@@ -2983,6 +3034,7 @@
       "cpu": [
         "arm"
       ],
+      "dev": true,
       "license": "MPL-2.0",
       "optional": true,
       "os": [
@@ -3003,6 +3055,7 @@
       "cpu": [
         "arm64"
       ],
+      "dev": true,
       "license": "MPL-2.0",
       "optional": true,
       "os": [
@@ -3023,6 +3076,7 @@
       "cpu": [
         "arm64"
       ],
+      "dev": true,
       "license": "MPL-2.0",
       "optional": true,
       "os": [
@@ -3043,6 +3097,7 @@
       "cpu": [
         "x64"
       ],
+      "dev": true,
       "license": "MPL-2.0",
       "optional": true,
       "os": [
@@ -3063,6 +3118,7 @@
       "cpu": [
         "x64"
       ],
+      "dev": true,
       "license": "MPL-2.0",
       "optional": true,
       "os": [
@@ -3083,6 +3139,7 @@
       "cpu": [
         "arm64"
       ],
+      "dev": true,
       "license": "MPL-2.0",
       "optional": true,
       "os": [
@@ -3103,6 +3160,7 @@
       "cpu": [
         "x64"
       ],
+      "dev": true,
       "license": "MPL-2.0",
       "optional": true,
       "os": [
@@ -3153,6 +3211,7 @@
       "version": "0.30.17",
       "resolved": "https://registry.npmjs.org/magic-string/-/magic-string-0.30.17.tgz",
       "integrity": "sha512-sNPKHvyjVf7gyjwS4xGTaW/mCnF8wnjtifKBEhxfZ7E/S8tQ0rssrwGNn6q8JH/ohItJfSQp9mBtQYuTlH5QnA==",
+      "dev": true,
       "license": "MIT",
       "dependencies": {
         "@jridgewell/sourcemap-codec": "^1.5.0"
@@ -3199,6 +3258,7 @@
       "version": "7.1.2",
       "resolved": "https://registry.npmjs.org/minipass/-/minipass-7.1.2.tgz",
       "integrity": "sha512-qOOzS1cBTWYF4BH8fVePDBOO9iptMnGUEZwNc/cMWnTV2nVLZ7VoNWEPHkYczZA0pdoA7dl6e7FL659nX9S2aw==",
+      "dev": true,
       "license": "ISC",
       "engines": {
         "node": ">=16 || 14 >=14.17"
@@ -3208,6 +3268,7 @@
       "version": "3.0.2",
       "resolved": "https://registry.npmjs.org/minizlib/-/minizlib-3.0.2.tgz",
       "integrity": "sha512-oG62iEk+CYt5Xj2YqI5Xi9xWUeZhDI8jjQmC5oThVH5JGCTgIjr7ciJDzC7MBzYd//WvR1OTmP5Q38Q8ShQtVA==",
+      "dev": true,
       "license": "MIT",
       "dependencies": {
         "minipass": "^7.1.2"
@@ -3220,6 +3281,7 @@
       "version": "3.0.1",
       "resolved": "https://registry.npmjs.org/mkdirp/-/mkdirp-3.0.1.tgz",
       "integrity": "sha512-+NsyUUAZDmo6YVHzL/stxSu3t9YS1iljliy3BSDrXJ/dkn1KYdmtZODGGjLcc9XLgVVpH4KshHB8XmZgMhaBXg==",
+      "dev": true,
       "license": "MIT",
       "bin": {
         "mkdirp": "dist/cjs/src/bin.js"
@@ -3242,6 +3304,7 @@
       "version": "3.3.11",
       "resolved": "https://registry.npmjs.org/nanoid/-/nanoid-3.3.11.tgz",
       "integrity": "sha512-N8SpfPUnUp1bK+PMYW8qSWdl9U+wwNWI4QKxOYDy9JAro3WMX7p2OeVRF9v+347pnakNevPmiHhNmZ2HbFA76w==",
+      "dev": true,
       "funding": [
         {
           "type": "github",
@@ -3357,6 +3420,7 @@
       "version": "1.1.1",
       "resolved": "https://registry.npmjs.org/picocolors/-/picocolors-1.1.1.tgz",
       "integrity": "sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA==",
+      "dev": true,
       "license": "ISC"
     },
     "node_modules/picomatch": {
@@ -3376,6 +3440,7 @@
       "version": "8.5.4",
       "resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.4.tgz",
       "integrity": "sha512-QSa9EBe+uwlGTFmHsPKokv3B/oEMQZxfqW0QqNCyhpa6mB1afzulwn8hihglqAb2pOw+BJgNlmXQ8la2VeHB7w==",
+      "dev": true,
       "funding": [
         {
           "type": "opencollective",
@@ -3554,6 +3619,7 @@
       "version": "4.42.0",
       "resolved": "https://registry.npmjs.org/rollup/-/rollup-4.42.0.tgz",
       "integrity": "sha512-LW+Vse3BJPyGJGAJt1j8pWDKPd73QM8cRXYK1IxOBgL2AGLu7Xd2YOW0M2sLUBCkF5MshXXtMApyEAEzMVMsnw==",
+      "dev": true,
       "license": "MIT",
       "dependencies": {
         "@types/estree": "1.0.7"
@@ -3593,6 +3659,7 @@
       "version": "1.0.7",
       "resolved": "https://registry.npmjs.org/@types/estree/-/estree-1.0.7.tgz",
       "integrity": "sha512-w28IoSUCJpidD/TGviZwwMJckNESJZXFu7NBZ5YJ4mEUnNraUn9Pm8HSZm/jDF1pDWYKspWE7oVphigUPRakIQ==",
+      "dev": true,
       "license": "MIT"
     },
     "node_modules/run-parallel": {
@@ -3668,6 +3735,7 @@
       "version": "1.2.1",
       "resolved": "https://registry.npmjs.org/source-map-js/-/source-map-js-1.2.1.tgz",
       "integrity": "sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA==",
+      "dev": true,
       "license": "BSD-3-Clause",
       "engines": {
         "node": ">=0.10.0"
@@ -3703,12 +3771,14 @@
       "version": "4.1.8",
       "resolved": "https://registry.npmjs.org/tailwindcss/-/tailwindcss-4.1.8.tgz",
       "integrity": "sha512-kjeW8gjdxasbmFKpVGrGd5T4i40mV5J2Rasw48QARfYeQ8YS9x02ON9SFWax3Qf616rt4Cp3nVNIj6Hd1mP3og==",
+      "dev": true,
       "license": "MIT"
     },
     "node_modules/tapable": {
       "version": "2.2.2",
       "resolved": "https://registry.npmjs.org/tapable/-/tapable-2.2.2.tgz",
       "integrity": "sha512-Re10+NauLTMCudc7T5WLFLAwDhQ0JWdrMK+9B2M8zR5hRExKmsRDCBA7/aV/pNJFltmBFO5BAMlQFi/vq3nKOg==",
+      "dev": true,
       "license": "MIT",
       "engines": {
         "node": ">=6"
@@ -3718,6 +3788,7 @@
       "version": "7.4.3",
       "resolved": "https://registry.npmjs.org/tar/-/tar-7.4.3.tgz",
       "integrity": "sha512-5S7Va8hKfV7W5U6g3aYxXmlPoZVAwUMy9AOKyF2fVuZa2UD3qZjg578OrLRt8PcNN1PleVaL/5/yYATNL0ICUw==",
+      "dev": true,
       "license": "ISC",
       "dependencies": {
         "@isaacs/fs-minipass": "^4.0.0",
@@ -3735,6 +3806,7 @@
       "version": "5.0.0",
       "resolved": "https://registry.npmjs.org/yallist/-/yallist-5.0.0.tgz",
       "integrity": "sha512-YgvUTfwqyc7UXVMrB+SImsVYSmTS8X/tSrtdNZMImM+n7+QTriRXyXim0mBrTXNeqzVF0KWGgHPeiyViFFrNDw==",
+      "dev": true,
       "license": "BlueOak-1.0.0",
       "engines": {
         "node": ">=18"
@@ -3744,6 +3816,7 @@
       "version": "0.2.14",
       "resolved": "https://registry.npmjs.org/tinyglobby/-/tinyglobby-0.2.14.tgz",
       "integrity": "sha512-tX5e7OM1HnYr2+a2C/4V0htOcSQcoSTH9KgJnVvNm5zm/cyEWKJ7j7YutsH9CxMdtOkkLFy2AHrMci9IM8IPZQ==",
+      "dev": true,
       "license": "MIT",
       "dependencies": {
         "fdir": "^6.4.4",
@@ -3760,6 +3833,7 @@
       "version": "6.4.5",
       "resolved": "https://registry.npmjs.org/fdir/-/fdir-6.4.5.tgz",
       "integrity": "sha512-4BG7puHpVsIYxZUbiUE3RqGloLaSSwzYie5jvasC4LWuBWzZawynvYouhjbQKw2JuIGYdm0DzIxl8iVidKlUEw==",
+      "dev": true,
       "license": "MIT",
       "peerDependencies": {
         "picomatch": "^3 || ^4"
@@ -3774,6 +3848,7 @@
       "version": "4.0.2",
       "resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.2.tgz",
       "integrity": "sha512-M7BAV6Rlcy5u+m6oPhAPFgJTzAioX/6B0DxyvDlo9l8+T3nLKbrczg2WLUyzd45L8RqfUMyGPzekbMvX2Ldkwg==",
+      "dev": true,
       "license": "MIT",
       "engines": {
         "node": ">=12"
@@ -3903,6 +3978,7 @@
       "version": "6.3.5",
       "resolved": "https://registry.npmjs.org/vite/-/vite-6.3.5.tgz",
       "integrity": "sha512-cZn6NDFE7wdTpINgs++ZJ4N49W2vRp8LCKrn3Ob1kYNtOo21vfDoaV5GzBfLU4MovSAB8uNRm4jgzVQZ+mBzPQ==",
+      "dev": true,
       "license": "MIT",
       "dependencies": {
         "esbuild": "^0.25.0",
@@ -3977,6 +4053,7 @@
       "version": "6.4.5",
       "resolved": "https://registry.npmjs.org/fdir/-/fdir-6.4.5.tgz",
       "integrity": "sha512-4BG7puHpVsIYxZUbiUE3RqGloLaSSwzYie5jvasC4LWuBWzZawynvYouhjbQKw2JuIGYdm0DzIxl8iVidKlUEw==",
+      "dev": true,
       "license": "MIT",
       "peerDependencies": {
         "picomatch": "^3 || ^4"
@@ -3991,6 +4068,7 @@
       "version": "4.0.2",
       "resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.2.tgz",
       "integrity": "sha512-M7BAV6Rlcy5u+m6oPhAPFgJTzAioX/6B0DxyvDlo9l8+T3nLKbrczg2WLUyzd45L8RqfUMyGPzekbMvX2Ldkwg==",
+      "dev": true,
       "license": "MIT",
       "engines": {
         "node": ">=12"

ui/package.json

@@ -4,23 +4,21 @@
   "version": "0.0.0",
   "type": "module",
   "scripts": {
-    "dev": "vite",
+    "start": "vite",
     "build": "tsc -b && vite build --emptyOutDir",
     "lint": "eslint .",
     "preview": "vite preview"
   },
   "dependencies": {
-    "@tailwindcss/vite": "^4.1.8",
-    "@tanstack/react-query": "^5.80.6",
     "react": "^19.1.0",
     "react-dom": "^19.1.0",
     "react-icons": "^5.5.0",
     "react-resizable-panels": "^3.0.4",
-    "react-router-dom": "^7.6.2",
-    "tailwindcss": "^4.1.8"
+    "react-router-dom": "^7.6.2"
   },
   "devDependencies": {
     "@eslint/js": "^9.25.0",
+    "@tailwindcss/vite": "^4.1.8",
     "@types/react": "^19.1.2",
     "@types/react-dom": "^19.1.2",
     "@vitejs/plugin-react": "^4.4.1",
@@ -28,6 +26,7 @@
     "eslint-plugin-react-hooks": "^5.2.0",
     "eslint-plugin-react-refresh": "^0.4.19",
     "globals": "^16.0.0",
+    "tailwindcss": "^4.1.8",
     "typescript": "~5.8.3",
     "typescript-eslint": "^8.30.1",
     "vite": "^6.3.5"

ui/src/App.tsx

@@ -1,52 +1,36 @@
-import { BrowserRouter as Router, Routes, Route, Navigate, NavLink } from "react-router-dom";
+import { useEffect } from "react";
+import { Navigate, Route, BrowserRouter as Router, Routes } from "react-router-dom";
+import { Header } from "./components/Header";
+import { useAPI } from "./contexts/APIProvider";
 import { useTheme } from "./contexts/ThemeProvider";
-import { APIProvider } from "./contexts/APIProvider";
+import ActivityPage from "./pages/Activity";
 import LogViewerPage from "./pages/LogViewer";
 import ModelPage from "./pages/Models";
-import ActivityPage from "./pages/Activity";
-import ConnectionStatus from "./components/ConnectionStatus";
-import { RiSunFill, RiMoonFill } from "react-icons/ri";
 
 function App() {
-  const { isNarrow, toggleTheme, isDarkMode } = useTheme();
+  const { setConnectionState } = useTheme();
+
+  const { connectionStatus } = useAPI();
+
+  // Synchronize the window.title connections state with the actual connection state
+  useEffect(() => {
+    setConnectionState(connectionStatus);
+  }, [connectionStatus]);
 
   return (
     <Router basename="/ui/">
-      <APIProvider>
-        <div className="flex flex-col h-screen">
-          <nav className="bg-surface border-b border-border p-2 h-[75px]">
-            <div className="flex items-center justify-between mx-auto px-4 h-full">
-              {!isNarrow && <h1 className="flex items-center p-0">llama-swap</h1>}
-              <div className="flex items-center space-x-4">
-                <NavLink to="/" className={({ isActive }) => (isActive ? "navlink active" : "navlink")}>
-                  Logs
-                </NavLink>
+      <div className="flex flex-col h-screen">
+        <Header />
 
-                <NavLink to="/models" className={({ isActive }) => (isActive ? "navlink active" : "navlink")}>
-                  Models
-                </NavLink>
-
-                <NavLink to="/activity" className={({ isActive }) => (isActive ? "navlink active" : "navlink")}>
-                  Activity
-                </NavLink>
-                <button className="" onClick={toggleTheme}>
-                  {isDarkMode ? <RiMoonFill /> : <RiSunFill />}
-                </button>
-                <ConnectionStatus />
-              </div>
-            </div>
-          </nav>
-
-          <main className="flex-1 overflow-auto p-4">
-            <Routes>
-              <Route path="/" element={<LogViewerPage />} />
-              <Route path="/models" element={<ModelPage />} />
-              <Route path="/activity" element={<ActivityPage />} />
-              <Route path="*" element={<Navigate to="/" replace />} />
-            </Routes>
-          </main>
-        </div>
-      </APIProvider>
+        <main className="flex-1 overflow-auto p-4">
+          <Routes>
+            <Route path="/" element={<LogViewerPage />} />
+            <Route path="/models" element={<ModelPage />} />
+            <Route path="/activity" element={<ActivityPage />} />
+            <Route path="*" element={<Navigate to="/" replace />} />
+          </Routes>
+        </main>
+      </div>
     </Router>
   );
 }

ui/src/components/ConnectionStatus.tsx

@@ -1,36 +1,26 @@
 import { useAPI } from "../contexts/APIProvider";
-import { useEffect, useState, useMemo } from "react";
+import { useMemo } from "react";
 
-type ConnectionStatus = "disconnected" | "connecting" | "connected";
-
-const ConnectionStatus = () => {
-  const { getConnectionStatus } = useAPI();
-  const [eventStreamStatus, setEventStreamStatus] = useState<ConnectionStatus>("disconnected");
-
-  useEffect(() => {
-    const interval = setInterval(() => {
-      setEventStreamStatus(getConnectionStatus());
-    }, 1000);
-    return () => clearInterval(interval);
-  });
+const ConnectionStatusIcon = () => {
+  const { connectionStatus } = useAPI();
 
   const eventStatusColor = useMemo(() => {
-    switch (eventStreamStatus) {
+    switch (connectionStatus) {
       case "connected":
-        return "bg-green-500";
+        return "bg-emerald-500";
       case "connecting":
-        return "bg-yellow-500";
+        return "bg-amber-500";
       case "disconnected":
       default:
         return "bg-red-500";
     }
-  }, [eventStreamStatus]);
+  }, [connectionStatus]);
 
   return (
-    <div className="flex items-center" title={`event stream: ${eventStreamStatus}`}>
+    <div className="flex items-center" title={`event stream: ${connectionStatus}`}>
       <span className={`inline-block w-3 h-3 rounded-full ${eventStatusColor} mr-2`}></span>
     </div>
   );
 };
 
-export default ConnectionStatus;
+export default ConnectionStatusIcon;

ui/src/components/Header.tsx

@@ -0,0 +1,56 @@
+import { useCallback } from "react";
+import { RiMoonFill, RiSunFill } from "react-icons/ri";
+import { NavLink, type NavLinkRenderProps } from "react-router-dom";
+import { useTheme } from "../contexts/ThemeProvider";
+import ConnectionStatusIcon from "./ConnectionStatus";
+
+export function Header() {
+  const { screenWidth, toggleTheme, isDarkMode, appTitle, setAppTitle } = useTheme();
+  const handleTitleChange = useCallback(
+    (newTitle: string) => {
+      setAppTitle(newTitle.replace(/\n/g, "").trim().substring(0, 64) || "llama-swap");
+    },
+    [setAppTitle]
+  );
+
+  const navLinkClass = ({ isActive }: NavLinkRenderProps) =>
+    `text-gray-600 hover:text-black dark:text-gray-300 dark:hover:text-gray-100 p-1 ${isActive ? "font-semibold" : ""}`;
+
+  return (
+    <header className="flex items-center justify-between bg-surface border-b border-border p-2 px-4 h-[75px]">
+      {screenWidth !== "xs" && screenWidth !== "sm" && (
+        <h1
+          contentEditable
+          suppressContentEditableWarning
+          className="p-0 outline-none hover:bg-gray-100 dark:hover:bg-gray-700 rounded"
+          onBlur={(e) => handleTitleChange(e.currentTarget.textContent || "(set title)")}
+          onKeyDown={(e) => {
+            if (e.key === "Enter") {
+              e.preventDefault();
+              handleTitleChange(e.currentTarget.textContent || "(set title)");
+              e.currentTarget.blur();
+            }
+          }}
+        >
+          {appTitle}
+        </h1>
+      )}
+
+      <menu className="flex items-center gap-4">
+        <NavLink to="/" className={navLinkClass} type="button">
+          Logs
+        </NavLink>
+        <NavLink to="/models" className={navLinkClass} type="button">
+          Models
+        </NavLink>
+        <NavLink to="/activity" className={navLinkClass} type="button">
+          Activity
+        </NavLink>
+        <button className="" onClick={toggleTheme}>
+          {isDarkMode ? <RiMoonFill /> : <RiSunFill />}
+        </button>
+        <ConnectionStatusIcon />
+      </menu>
+    </header>
+  );
+}

ui/src/contexts/APIProvider.tsx

@@ -1,4 +1,5 @@
-import { useRef, createContext, useState, useContext, useEffect, useCallback, useMemo, type ReactNode } from "react";
+import { createContext, useState, useContext, useEffect, useCallback, useMemo, type ReactNode } from "react";
+import type { ConnectionState } from "../lib/types";
 
 type ModelStatus = "ready" | "starting" | "stopping" | "stopped" | "shutdown" | "unknown";
 const LOG_LENGTH_LIMIT = 1024 * 100; /* 100KB of log data */
@@ -15,18 +16,20 @@ interface APIProviderType {
   models: Model[];
   listModels: () => Promise<Model[]>;
   unloadAllModels: () => Promise<void>;
+  unloadSingleModel: (model: string) => Promise<void>;
   loadModel: (model: string) => Promise<void>;
   enableAPIEvents: (enabled: boolean) => void;
   proxyLogs: string;
   upstreamLogs: string;
   metrics: Metrics[];
-  getConnectionStatus: () => "connected" | "connecting" | "disconnected";
+  connectionStatus: ConnectionState;
 }
 
 interface Metrics {
   id: number;
   timestamp: string;
   model: string;
+  cache_tokens: number;
   input_tokens: number;
   output_tokens: number;
   prompt_per_second: number;
@@ -49,11 +52,14 @@ type APIProviderProps = {
   autoStartAPIEvents?: boolean;
 };
 
+let apiEventSource: EventSource | null = null;
+
 export function APIProvider({ children, autoStartAPIEvents = true }: APIProviderProps) {
   const [proxyLogs, setProxyLogs] = useState("");
   const [upstreamLogs, setUpstreamLogs] = useState("");
   const [metrics, setMetrics] = useState<Metrics[]>([]);
-  const apiEventSource = useRef<EventSource | null>(null);
+  const [connectionStatus, setConnectionState] = useState<ConnectionState>("disconnected");
+  //const apiEventSource = useRef<EventSource | null>(null);
 
   const [models, setModels] = useState<Model[]>([]);
 
@@ -64,20 +70,10 @@ export function APIProvider({ children, autoStartAPIEvents = true }: APIProvider
     });
   }, []);
 
-  const getConnectionStatus = useCallback(() => {
-    if (apiEventSource.current?.readyState === EventSource.OPEN) {
-      return "connected";
-    } else if (apiEventSource.current?.readyState === EventSource.CONNECTING) {
-      return "connecting";
-    } else {
-      return "disconnected";
-    }
-  }, []);
-
   const enableAPIEvents = useCallback((enabled: boolean) => {
     if (!enabled) {
-      apiEventSource.current?.close();
-      apiEventSource.current = null;
+      apiEventSource?.close();
+      apiEventSource = null;
       setMetrics([]);
       return;
     }
@@ -86,17 +82,22 @@ export function APIProvider({ children, autoStartAPIEvents = true }: APIProvider
     const initialDelay = 1000; // 1 second
 
     const connect = () => {
-      const eventSource = new EventSource("/api/events");
+      apiEventSource?.close();
+      apiEventSource = new EventSource("/api/events");
 
-      eventSource.onopen = () => {
+      setConnectionState("connecting");
+
+      apiEventSource.onopen = () => {
         // clear everything out on connect to keep things in sync
         setProxyLogs("");
         setUpstreamLogs("");
         setMetrics([]); // clear metrics on reconnect
         setModels([]); // clear models on reconnect
+        retryCount = 0;
+        setConnectionState("connected");
       };
 
-      eventSource.onmessage = (e: MessageEvent) => {
+      apiEventSource.onmessage = (e: MessageEvent) => {
         try {
           const message = JSON.parse(e.data) as APIEventEnvelope;
           switch (message.type) {
@@ -138,14 +139,14 @@ export function APIProvider({ children, autoStartAPIEvents = true }: APIProvider
           console.error(e.data, err);
         }
       };
-      eventSource.onerror = () => {
-        eventSource.close();
+
+      apiEventSource.onerror = () => {
+        apiEventSource?.close();
         retryCount++;
         const delay = Math.min(initialDelay * Math.pow(2, retryCount - 1), 5000);
+        setConnectionState("disconnected");
         setTimeout(connect, delay);
       };
-
-      apiEventSource.current = eventSource;
     };
 
     connect();
@@ -177,7 +178,7 @@ export function APIProvider({ children, autoStartAPIEvents = true }: APIProvider
 
   const unloadAllModels = useCallback(async () => {
     try {
-      const response = await fetch(`/api/models/unload/`, {
+      const response = await fetch(`/api/models/unload`, {
         method: "POST",
       });
       if (!response.ok) {
@@ -189,6 +190,20 @@ export function APIProvider({ children, autoStartAPIEvents = true }: APIProvider
     }
   }, []);
 
+  const unloadSingleModel = useCallback(async (model: string) => {
+    try {
+      const response = await fetch(`/api/models/unload/${model}`, {
+        method: "POST",
+      });
+      if (!response.ok) {
+        throw new Error(`Failed to unload model: ${response.status}`);
+      }
+    } catch (error) {
+      console.error("Failed to unload model", model, error);
+      throw error;
+    }
+  }, []);
+
   const loadModel = useCallback(async (model: string) => {
     try {
       const response = await fetch(`/upstream/${model}/`, {
@@ -208,12 +223,13 @@ export function APIProvider({ children, autoStartAPIEvents = true }: APIProvider
       models,
       listModels,
       unloadAllModels,
+      unloadSingleModel,
       loadModel,
       enableAPIEvents,
       proxyLogs,
       upstreamLogs,
       metrics,
-      getConnectionStatus,
+      connectionStatus,
     }),
     [models, listModels, unloadAllModels, loadModel, enableAPIEvents, proxyLogs, upstreamLogs, metrics]
   );

ui/src/contexts/ThemeProvider.tsx

@@ -1,5 +1,6 @@
 import { createContext, useContext, useEffect, type ReactNode, useMemo, useState } from "react";
 import { usePersistentState } from "../hooks/usePersistentState";
+import type { ConnectionState } from "../lib/types";
 
 type ScreenWidth = "xs" | "sm" | "md" | "lg" | "xl" | "2xl";
 type ThemeContextType = {
@@ -7,6 +8,11 @@ type ThemeContextType = {
   screenWidth: ScreenWidth;
   isNarrow: boolean;
   toggleTheme: () => void;
+
+  // for managing the window title and connection state information
+  appTitle: string;
+  setAppTitle: (title: string) => void;
+  setConnectionState: (state: ConnectionState) => void;
 };
 
 const ThemeContext = createContext<ThemeContextType | undefined>(undefined);
@@ -16,6 +22,17 @@ type ThemeProviderProps = {
 };
 
 export function ThemeProvider({ children }: ThemeProviderProps) {
+  const [appTitle, setAppTitle] = usePersistentState("app-title", "llama-swap");
+  const [connectionState, setConnectionState] = useState<ConnectionState>("disconnected");
+
+  /**
+   * Set the document.title with informative information
+   */
+  useEffect(() => {
+    const connectionIcon = connectionState === "connecting" ? "🟡" : connectionState === "connected" ? "🟢" : "🔴";
+    document.title = connectionIcon + " " + appTitle; // Set initial title
+  }, [appTitle, connectionState]);
+
   const [isDarkMode, setIsDarkMode] = usePersistentState<boolean>("theme", false);
   const [screenWidth, setScreenWidth] = useState<ScreenWidth>("md"); // Default to md
 
@@ -55,7 +72,19 @@ export function ThemeProvider({ children }: ThemeProviderProps) {
   }, [screenWidth]);
 
   return (
-    <ThemeContext.Provider value={{ isDarkMode, toggleTheme, screenWidth, isNarrow }}>{children}</ThemeContext.Provider>
+    <ThemeContext.Provider
+      value={{
+        isDarkMode,
+        toggleTheme,
+        screenWidth,
+        isNarrow,
+        appTitle,
+        setAppTitle,
+        setConnectionState,
+      }}
+    >
+      {children}
+    </ThemeContext.Provider>
   );
 }
 

ui/src/index.css

@@ -93,6 +93,14 @@
     @apply px-4;
   }
 
+  /* Tables */
+  table th {
+    @apply p-2 font-semibold;
+  }
+  table td {
+    @apply p-2;
+  }
+
   /* Navigation Header */
 
   .navlink {
@@ -122,7 +130,7 @@
 
   /* Status Badges */
   .status {
-    @apply inline-block px-2 py-1 text-xs font-medium rounded-full;
+    @apply inline-block px-2 py-1 text-xs font-medium rounded-lg;
   }
 
   .status--ready {
@@ -140,7 +148,7 @@
 
   /* Buttons */
   .btn {
-    @apply bg-surface p-2 px-4 text-sm rounded-full border border-2 transition-colors duration-200 border-btn-border;
+    @apply bg-surface py-2 px-4 text-sm rounded-md border transition-colors duration-200 border-btn-border;
   }
 
   .btn:hover {

ui/src/lib/types.ts

@@ -0,0 +1 @@
+export type ConnectionState = "connected" | "connecting" | "disconnected";

ui/src/main.tsx

@@ -3,11 +3,14 @@ import { createRoot } from "react-dom/client";
 import "./index.css";
 import App from "./App.tsx";
 import { ThemeProvider } from "./contexts/ThemeProvider";
+import { APIProvider } from "./contexts/APIProvider";
 
 createRoot(document.getElementById("root")!).render(
   <StrictMode>
     <ThemeProvider>
-      <App />
+      <APIProvider>
+        <App />
+      </APIProvider>
     </ThemeProvider>
   </StrictMode>
 );

ui/src/pages/Activity.tsx

@@ -1,10 +1,6 @@
 import { useMemo } from "react";
 import { useAPI } from "../contexts/APIProvider";
 
-const formatTimestamp = (timestamp: string): string => {
-  return new Date(timestamp).toLocaleString();
-};
-
 const formatSpeed = (speed: number): string => {
   return speed < 0 ? "unknown" : speed.toFixed(2) + " t/s";
 };
@@ -13,6 +9,33 @@ const formatDuration = (ms: number): string => {
   return (ms / 1000).toFixed(2) + "s";
 };
 
+const formatRelativeTime = (timestamp: string): string => {
+  const now = new Date();
+  const date = new Date(timestamp);
+  const diffInSeconds = Math.floor((now.getTime() - date.getTime()) / 1000);
+
+  // Handle future dates by returning "just now"
+  if (diffInSeconds < 5) {
+    return "now";
+  }
+
+  if (diffInSeconds < 60) {
+    return `${diffInSeconds}s ago`;
+  }
+
+  const diffInMinutes = Math.floor(diffInSeconds / 60);
+  if (diffInMinutes < 60) {
+    return `${diffInMinutes}m ago`;
+  }
+
+  const diffInHours = Math.floor(diffInMinutes / 60);
+  if (diffInHours < 24) {
+    return `${diffInHours}h ago`;
+  }
+
+  return "a while ago";
+};
+
 const ActivityPage = () => {
   const { metrics } = useAPI();
   const sortedMetrics = useMemo(() => {
@@ -20,39 +43,46 @@ const ActivityPage = () => {
   }, [metrics]);
 
   return (
-    <div className="p-6">
-      <h1 className="text-2xl font-bold mb-4">Activity</h1>
+    <div className="p-2">
+      <h1 className="text-2xl font-bold">Activity</h1>
 
-      {metrics.length === 0 ? (
+      {metrics.length === 0 && (
         <div className="text-center py-8">
           <p className="text-gray-600">No metrics data available</p>
         </div>
-      ) : (
-        <div className="overflow-x-auto">
+      )}
+      {metrics.length > 0 && (
+        <div className="card overflow-auto">
           <table className="min-w-full divide-y">
-            <thead>
-              <tr>
-                <th className="px-4 py-3 text-left text-xs font-medium uppercase tracking-wider">Id</th>
-                <th className="px-6 py-3 text-left text-xs font-medium uppercase tracking-wider">Timestamp</th>
-                <th className="px-6 py-3 text-left text-xs font-medium uppercase tracking-wider">Model</th>
-                <th className="px-6 py-3 text-left text-xs font-medium uppercase tracking-wider">Input Tokens</th>
-                <th className="px-6 py-3 text-left text-xs font-medium uppercase tracking-wider">Output Tokens</th>
-                <th className="px-6 py-3 text-left text-xs font-medium uppercase tracking-wider">Prompt Processing</th>
-                <th className="px-6 py-3 text-left text-xs font-medium uppercase tracking-wider">Generation Speed</th>
-                <th className="px-6 py-3 text-left text-xs font-medium uppercase tracking-wider">Duration</th>
+            <thead className="border-gray-200 dark:border-white/10">
+              <tr className="text-left text-xs uppercase tracking-wider">
+                <th className="px-6 py-3">ID</th>
+                <th className="px-6 py-3">Time</th>
+                <th className="px-6 py-3">Model</th>
+                <th className="px-6 py-3">
+                  Cached <Tooltip content="prompt tokens from cache" />
+                </th>
+                <th className="px-6 py-3">
+                  Prompt <Tooltip content="new prompt tokens processed" />
+                </th>
+                <th className="px-6 py-3">Generated</th>
+                <th className="px-6 py-3">Prompt Processing</th>
+                <th className="px-6 py-3">Generation Speed</th>
+                <th className="px-6 py-3">Duration</th>
               </tr>
             </thead>
             <tbody className="divide-y">
               {sortedMetrics.map((metric) => (
-                <tr key={`metric_${metric.id}`}>
-                  <td className="px-4 py-4 whitespace-nowrap text-sm">{metric.id + 1 /* un-zero index */}</td>
-                  <td className="px-6 py-4 whitespace-nowrap text-sm">{formatTimestamp(metric.timestamp)}</td>
-                  <td className="px-6 py-4 whitespace-nowrap text-sm">{metric.model}</td>
-                  <td className="px-6 py-4 whitespace-nowrap text-sm">{metric.input_tokens.toLocaleString()}</td>
-                  <td className="px-6 py-4 whitespace-nowrap text-sm">{metric.output_tokens.toLocaleString()}</td>
-                  <td className="px-6 py-4 whitespace-nowrap text-sm">{formatSpeed(metric.prompt_per_second)}</td>
-                  <td className="px-6 py-4 whitespace-nowrap text-sm">{formatSpeed(metric.tokens_per_second)}</td>
-                  <td className="px-6 py-4 whitespace-nowrap text-sm">{formatDuration(metric.duration_ms)}</td>
+                <tr key={metric.id} className="whitespace-nowrap text-sm border-gray-200 dark:border-white/10">
+                  <td className="px-4 py-4">{metric.id + 1 /* un-zero index */}</td>
+                  <td className="px-6 py-4">{formatRelativeTime(metric.timestamp)}</td>
+                  <td className="px-6 py-4">{metric.model}</td>
+                  <td className="px-6 py-4">{metric.cache_tokens > 0 ? metric.cache_tokens.toLocaleString() : "-"}</td>
+                  <td className="px-6 py-4">{metric.input_tokens.toLocaleString()}</td>
+                  <td className="px-6 py-4">{metric.output_tokens.toLocaleString()}</td>
+                  <td className="px-6 py-4">{formatSpeed(metric.prompt_per_second)}</td>
+                  <td className="px-6 py-4">{formatSpeed(metric.tokens_per_second)}</td>
+                  <td className="px-6 py-4">{formatDuration(metric.duration_ms)}</td>
                 </tr>
               ))}
             </tbody>
@@ -63,4 +93,28 @@ const ActivityPage = () => {
   );
 };
 
+interface TooltipProps {
+  content: string;
+}
+
+const Tooltip: React.FC<TooltipProps> = ({ content }) => {
+  return (
+    <div className="relative group inline-block">
+      ⓘ
+      <div
+        className="absolute top-full left-1/2 transform -translate-x-1/2 mt-2
+                     px-3 py-2 bg-gray-900 text-white text-sm rounded-md
+                     opacity-0 group-hover:opacity-100 transition-opacity
+                     duration-200 pointer-events-none whitespace-nowrap z-50 normal-case"
+      >
+        {content}
+        <div
+          className="absolute bottom-full left-1/2 transform -translate-x-1/2
+                       border-4 border-transparent border-b-gray-900"
+        ></div>
+      </div>
+    </div>
+  );
+};
+
 export default ActivityPage;

ui/src/pages/LogViewer.tsx

@@ -14,8 +14,8 @@ import { useTheme } from "../contexts/ThemeProvider";
 
 const LogViewer = () => {
   const { proxyLogs, upstreamLogs } = useAPI();
-  const { isNarrow } = useTheme();
-  const direction = isNarrow ? "vertical" : "horizontal";
+  const { screenWidth } = useTheme();
+  const direction = screenWidth === "xs" || screenWidth === "sm" ? "vertical" : "horizontal";
 
   return (
     <PanelGroup direction={direction} className="gap-2" autoSaveId="logviewer-panel-group">
@@ -115,19 +115,19 @@ export const LogPanel = ({ id, title, logData }: LogPanelProps) => {
   }, [filteredLogs]);
 
   return (
-    <div className="bg-surface border border-border rounded-lg overflow-hidden flex flex-col h-full">
-      <div className="p-4 border-b border-border bg-secondary">
+    <div className="rounded-lg overflow-hidden flex flex-col bg-gray-950/5 dark:bg-white/10 h-full p-1">
+      <div className="p-4">
         <div className="flex items-center justify-between">
           <h3 className="m-0 text-lg p-0">{title}</h3>
 
           <div className="flex gap-2 items-center">
-            <button className="btn" onClick={toggleFontSize}>
+            <button className="btn border-0" onClick={toggleFontSize}>
               <RiFontSize />
             </button>
-            <button className="btn" onClick={toggleWrapText}>
+            <button className="btn border-0" onClick={toggleWrapText}>
               {wrapText ? <RiTextWrap /> : <RiAlignJustify />}
             </button>
-            <button className="btn" onClick={toggleFilter}>
+            <button className="btn border-0" onClick={toggleFilter}>
               {showFilter ? <RiMenuSearchFill /> : <RiMenuSearchLine />}
             </button>
           </div>
@@ -139,7 +139,7 @@ export const LogPanel = ({ id, title, logData }: LogPanelProps) => {
             <div className="flex gap-2 items-center w-full">
               <input
                 type="text"
-                className="w-full text-sm border p-2 rounded"
+                className="w-full text-sm border border-gray-950/10 dark:border-white/5 p-2 rounded outline-none"
                 placeholder="Filter logs..."
                 value={filterRegex}
                 onChange={(e) => setFilterRegex(e.target.value)}
@@ -151,7 +151,7 @@ export const LogPanel = ({ id, title, logData }: LogPanelProps) => {
           </div>
         )}
       </div>
-      <div className="bg-background font-mono text-sm flex-1 overflow-hidden">
+      <div className="rounded-lg bg-background font-mono text-sm flex-1 overflow-hidden">
         <pre ref={preTagRef} className={`${textWrapClass} ${fontSizeClass} h-full overflow-auto p-4`}>
           {filteredLogs}
         </pre>

ui/src/pages/Models.tsx

@@ -4,7 +4,7 @@ import { LogPanel } from "./LogViewer";
 import { usePersistentState } from "../hooks/usePersistentState";
 import { Panel, PanelGroup, PanelResizeHandle } from "react-resizable-panels";
 import { useTheme } from "../contexts/ThemeProvider";
-import { RiEyeFill, RiEyeOffFill, RiStopCircleLine, RiSwapBoxFill } from "react-icons/ri";
+import { RiEyeFill, RiEyeOffFill, RiSwapBoxFill, RiEjectLine } from "react-icons/ri";
 
 export default function ModelsPage() {
   const { isNarrow } = useTheme();
@@ -37,7 +37,7 @@ export default function ModelsPage() {
 }
 
 function ModelsPanel() {
-  const { models, loadModel, unloadAllModels } = useAPI();
+  const { models, loadModel, unloadAllModels, unloadSingleModel } = useAPI();
   const [isUnloading, setIsUnloading] = useState(false);
   const [showUnlisted, setShowUnlisted] = usePersistentState("showUnlisted", true);
   const [showIdorName, setShowIdorName] = usePersistentState<"id" | "name">("showIdorName", "id"); // true = show ID, false = show name
@@ -69,20 +69,28 @@ function ModelsPanel() {
         <h2>Models</h2>
         <div className="flex justify-between">
           <div className="flex gap-2">
-            <button className="btn flex items-center gap-2" onClick={toggleIdorName} style={{ lineHeight: "1.2" }}>
-              <RiSwapBoxFill /> {showIdorName === "id" ? "ID" : "Name"}
+            <button
+              className="btn text-base flex items-center gap-2"
+              onClick={toggleIdorName}
+              style={{ lineHeight: "1.2" }}
+            >
+              <RiSwapBoxFill size="20" /> {showIdorName === "id" ? "ID" : "Name"}
             </button>
 
             <button
-              className="btn flex items-center gap-2"
+              className="btn text-base flex items-center gap-2"
               onClick={() => setShowUnlisted(!showUnlisted)}
               style={{ lineHeight: "1.2" }}
             >
-              {showUnlisted ? <RiEyeFill /> : <RiEyeOffFill />} unlisted
+              {showUnlisted ? <RiEyeFill size="20" /> : <RiEyeOffFill size="20" />} unlisted
             </button>
           </div>
-          <button className="btn flex items-center gap-2" onClick={handleUnloadAllModels} disabled={isUnloading}>
-            <RiStopCircleLine size="24" /> {isUnloading ? "Unloading..." : "Unload"}
+          <button
+            className="btn text-base flex items-center gap-2"
+            onClick={handleUnloadAllModels}
+            disabled={isUnloading}
+          >
+            <RiEjectLine size="24" /> {isUnloading ? "Unloading..." : "Unload All"}
           </button>
         </div>
       </div>
@@ -90,36 +98,43 @@ function ModelsPanel() {
       <div className="flex-1 overflow-y-auto">
         <table className="w-full">
           <thead className="sticky top-0 bg-card z-10">
-            <tr className="border-b border-primary bg-surface">
-              <th className="text-left p-2">{showIdorName === "id" ? "Model ID" : "Name"}</th>
-              <th className="text-left p-2"></th>
-              <th className="text-left p-2">State</th>
+            <tr className="text-left border-b border-gray-200 dark:border-white/10 bg-surface">
+              <th>{showIdorName === "id" ? "Model ID" : "Name"}</th>
+              <th></th>
+              <th>State</th>
             </tr>
           </thead>
           <tbody>
             {filteredModels.map((model) => (
-              <tr key={model.id} className="border-b hover:bg-secondary-hover border-border">
-                <td className={`p-2 ${model.unlisted ? "text-txtsecondary" : ""}`}>
-                  <a href={`/upstream/${model.id}/`} className={`underline`} target="_blank">
+              <tr key={model.id} className="border-b hover:bg-secondary-hover border-gray-200">
+                <td className={`${model.unlisted ? "text-txtsecondary" : ""}`}>
+                  <a href={`/upstream/${model.id}/`} className="font-semibold" target="_blank">
                     {showIdorName === "id" ? model.id : model.name !== "" ? model.name : model.id}
                   </a>
-                  {model.description !== "" && (
+
+                  {!!model.description && (
                     <p className={model.unlisted ? "text-opacity-70" : ""}>
                       <em>{model.description}</em>
                     </p>
                   )}
                 </td>
-                <td className="p-2 w-[50px]">
-                  <button
-                    className="btn btn--sm"
-                    disabled={model.state !== "stopped"}
-                    onClick={() => loadModel(model.id)}
-                  >
-                    Load
-                  </button>
+                <td className="w-12">
+                  {model.state === "stopped" ? (
+                    <button className="btn btn--sm" onClick={() => loadModel(model.id)}>
+                      Load
+                    </button>
+                  ) : (
+                    <button
+                      className="btn btn--sm"
+                      onClick={() => unloadSingleModel(model.id)}
+                      disabled={model.state !== "ready"}
+                    >
+                      Unload
+                    </button>
+                  )}
                 </td>
-                <td className="p-2 w-[75px]">
-                  <span className={`status status--${model.state}`}>{model.state}</span>
+                <td className="w-20">
+                  <span className={`w-16 text-center status status--${model.state}`}>{model.state}</span>
                 </td>
               </tr>
             ))}
@@ -146,24 +161,26 @@ function StatsPanel() {
 
   return (
     <div className="card">
-      <div className="rounded-lg overflow-hidden border border-gray-200">
+      <div className="rounded-lg overflow-hidden border border-gray-200 dark:border-white/10">
         <table className="w-full">
-          <tbody>
-            <tr>
-              <th className="p-2 font-medium border-b border-gray-200 text-right">Requests</th>
-              <th className="p-2 font-medium border-l border-b border-gray-200 text-right">Processed</th>
-              <th className="p-2 font-medium border-l border-b border-gray-200 text-right">Generated</th>
-              <th className="p-2 font-medium border-l border-b border-gray-200 text-right">Tokens/Sec</th>
+          <thead>
+            <tr className="border-b border-gray-200 dark:border-white/10 text-right">
+              <th>Requests</th>
+              <th className="border-l border-gray-200 dark:border-white/10">Processed</th>
+              <th className="border-l border-gray-200 dark:border-white/10">Generated</th>
+              <th className="border-l border-gray-200 dark:border-white/10">Tokens/Sec</th>
             </tr>
-            <tr>
-              <td className="p-2 text-right border-r border-gray-200">{totalRequests}</td>
-              <td className="p-2 text-right border-r border-gray-200">
+          </thead>
+          <tbody>
+            <tr className="text-right">
+              <td className="border-r border-gray-200 dark:border-white/10">{totalRequests}</td>
+              <td className="border-r border-gray-200 dark:border-white/10">
                 {new Intl.NumberFormat().format(totalInputTokens)}
               </td>
-              <td className="p-2 text-right border-r border-gray-200">
+              <td className="border-r border-gray-200 dark:border-white/10">
                 {new Intl.NumberFormat().format(totalOutputTokens)}
               </td>
-              <td className="p-2 text-right">{avgTokensPerSecond}</td>
+              <td>{avgTokensPerSecond}</td>
             </tr>
           </tbody>
         </table>

ui/vite.config.ts

@@ -15,6 +15,7 @@ export default defineConfig({
       "/api": "http://localhost:8080", // Proxy API calls to Go backend during development
       "/logs": "http://localhost:8080",
       "/upstream": "http://localhost:8080",
+      "/unload": "http://localhost:8080",
     },
   },
 });