Append custom env vars instead of replace in Process (#171 )

Append custom env vars instead of replace in Process (#168, #169) PR #162 refactored the default configuration code. This introduced a subtle bug where `env` became `[]string{}` instead of the default of `nil`. In golang, `exec.Cmd.Env == nil` means to use the "current process's environment". By setting it to `[]string{}` as a default the Process's environment was emptied out which caused an array of strange and difficult to troubleshoot behaviour. See issues #168 and #169 This commit changes the behaviour to append model configured environment variables to the default list rather than replace them.
fix? deprecation warning in .goreleaser.yaml [skip-ci]
2025-06-18 11:09:13 -07:00 · 2025-06-18 07:49:33 -07:00 · 2025-06-18 07:43:23 -07:00 · 2025-06-17 09:54:01 -07:00 · 2025-06-17 09:53:22 -07:00 · 2025-06-17 09:52:47 -07:00
45 changed files with 5647 additions and 570 deletions
@@ -15,8 +15,7 @@ jobs:
    runs-on: ubuntu-latest
    strategy:
      matrix:
-        #platform: [intel, cuda, vulkan, cpu, musa]
+        platform: [intel, cuda, vulkan, cpu, musa]
        platform: [cuda, vulkan, cpu, musa]
      fail-fast: false
    steps:
      - name: Checkout code
@@ -23,6 +23,19 @@ jobs:
      -
        name: Set up Go
        uses: actions/setup-go@v5
      -
        name: Set up Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '23'  # or your preferred version
      -
        name: Install dependencies and build UI
        run: |
          cd ui
          npm ci
          npm run build
      -
        name: Run GoReleaser
        uses: goreleaser/goreleaser-action@v6
@@ -27,4 +27,4 @@ archives:
      owner: root
    format_overrides:
      - goos: windows
-        format: zip
+        formats: ['zip']
@@ -19,24 +19,35 @@ all: mac linux simple-responder
 clean:
 	rm -rf $(BUILD_DIR)
-test:
+proxy/ui_dist/placeholder.txt:
 	mkdir -p proxy/ui_dist
 	touch $@
 test: proxy/ui_dist/placeholder.txt
 	go test -short -v -count=1 ./proxy
-test-all:
+test-all: proxy/ui_dist/placeholder.txt
 	go test -v -count=1 ./proxy
 ui/node_modules:
 	cd ui && npm install
 # build react UI
 ui: ui/node_modules
 	cd ui && npm run build
 # Build OSX binary
-mac:
+mac: ui
 	@echo "Building Mac binary..."
 	GOOS=darwin GOARCH=arm64 go build -ldflags="-X main.commit=${GIT_HASH} -X main.version=local_${GIT_HASH} -X main.date=${BUILD_DATE}" -o $(BUILD_DIR)/$(APP_NAME)-darwin-arm64
 # Build Linux binary
-linux:
+linux: ui
 	@echo "Building Linux binary..."
 	GOOS=linux GOARCH=amd64 go build -ldflags="-X main.commit=${GIT_HASH} -X main.version=local_${GIT_HASH} -X main.date=${BUILD_DATE}" -o $(BUILD_DIR)/$(APP_NAME)-linux-amd64
 # Build Windows binary
-windows:
+windows: ui
 	@echo "Building Windows binary..."
 	GOOS=windows GOARCH=amd64 go build -ldflags="-X main.commit=${GIT_HASH} -X main.version=local_${GIT_HASH} -X main.date=${BUILD_DATE}" -o $(BUILD_DIR)/$(APP_NAME)-windows-amd64.exe
@@ -69,4 +80,4 @@ release:
 	git tag "$$new_tag";
 # Phony targets
-.PHONY: all clean mac linux windows simple-responder
+.PHONY: all clean ui mac linux windows simple-responder
@@ -40,36 +40,32 @@ In the most basic configuration llama-swap handles one model at a time. For more
 ## config.yaml
-llama-swap's configuration is purposefully simple:
+llama-swap is managed entirely through a yaml configuration file. 
 It can be very minimal to start: 
 ```yaml
 models:
  "qwen2.5":
    cmd: |
-      /app/llama-server
+      /path/to/llama-server
      -hf bartowski/Qwen2.5-0.5B-Instruct-GGUF:Q4_K_M
      --port ${PORT}
  "smollm2":
    cmd: |
      /app/llama-server
      -hf bartowski/SmolLM2-135M-Instruct-GGUF:Q4_K_M
      --port ${PORT}
 ```
-.. but also supports many advanced features:
+However, there are many more capabilities that llama-swap supports: 
 - `groups` to run multiple models at once
 - `macros` for reusable snippets
 - `ttl` to automatically unload models
 - `macros` for reusable snippets
 - `aliases` to use familiar model names (e.g., "gpt-4o-mini")
- `env` variables to pass custom environment to inference servers
+- `env` to pass custom environment variables to inference servers
 - `cmdStop` for to gracefully stop Docker/Podman containers
 - `useModelName` to override model names sent to upstream servers
 - `healthCheckTimeout` to control model startup wait times
 - `${PORT}` automatic port variables for dynamic port assignment
 - `cmdStop` for to gracefully stop Docker/Podman containers
-Check the [configuration documentation](https://github.com/mostlygeek/llama-swap/wiki/Configuration) in the wiki for all options.
+See the [configuration documentation](https://github.com/mostlygeek/llama-swap/wiki/Configuration) in the wiki all options and examples.
 ## Docker Install ([download images](https://github.com/mostlygeek/llama-swap/pkgs/container/llama-swap))
@@ -120,11 +116,11 @@ $ docker run -it --rm --runtime nvidia -p 9292:8080 \
 ## Bare metal Install ([download](https://github.com/mostlygeek/llama-swap/releases))
-Pre-built binaries are available for Linux, FreeBSD and Darwin (OSX). These are automatically published and are likely a few hours ahead of the docker releases. The baremetal install works with any OpenAI compatible server, not just llama-server.
+Pre-built binaries are available for Linux, Mac, Windows and FreeBSD. These are automatically published and are likely a few hours ahead of the docker releases. The baremetal install works with any OpenAI compatible server, not just llama-server.
 1. Create a configuration file, see [config.example.yaml](config.example.yaml)
 1. Download a [release](https://github.com/mostlygeek/llama-swap/releases) appropriate for your OS and architecture.
-1. Run the binary with `llama-swap --config path/to/config.yaml`.
+1. Create a configuration file, see the [configuration documentation](https://github.com/mostlygeek/llama-swap/wiki/Configuration).
 1. Run the binary with `llama-swap --config path/to/config.yaml --listen localhost:8080`.
   Available flags:
   - `--config`: Path to the configuration file (default: `config.yaml`).
   - `--listen`: Address and port to listen on (default: `:8080`).
@@ -133,16 +129,16 @@ Pre-built binaries are available for Linux, FreeBSD and Darwin (OSX). These are
 ### Building from source
-1. Install golang for your system
+1. Build requires golang and nodejs for the user interface.
 1. `git clone git@github.com:mostlygeek/llama-swap.git`
 1. `make clean all`
 1. Binaries will be in `build/` subdirectory
 ## Monitoring Logs
-Open the `http://<host>/logs` with your browser to get a web interface with streaming logs.
+Open the `http://<host>:<port>/` with your browser to get a web interface with streaming logs.
-Of course, CLI access is also supported:
+CLI access is also supported:
 ```shell
 # sends up to the last 10KB of logs
@@ -1,3 +1,8 @@
 # ======
 # For a more detailed configuration example:
 # https://github.com/mostlygeek/llama-swap/wiki/Configuration
 # ======
 # Seconds to wait for llama.cpp to be available to serve requests
 # Default (and minimum): 15 seconds
 healthCheckTimeout: 90
@@ -0,0 +1 @@
 ui_dist/*
@@ -31,6 +31,34 @@ type ModelConfig struct {
 	ConcurrencyLimit int `yaml:"concurrencyLimit"`
 }
 func (m *ModelConfig) UnmarshalYAML(unmarshal func(interface{}) error) error {
 	type rawModelConfig ModelConfig
 	defaults := rawModelConfig{
 		Cmd:              "",
 		CmdStop:          "",
 		Proxy:            "http://localhost:${PORT}",
 		Aliases:          []string{},
 		Env:              []string{},
 		CheckEndpoint:    "/health",
 		UnloadAfter:      0,
 		Unlisted:         false,
 		UseModelName:     "",
 		ConcurrencyLimit: 0,
 	}
 	// the default cmdStop to taskkill /f /t /pid ${PID}
 	if runtime.GOOS == "windows" {
 		defaults.CmdStop = "taskkill /f /t /pid ${PID}"
 	}
 	if err := unmarshal(&defaults); err != nil {
 		return err
 	}
 	*m = ModelConfig(defaults)
 	return nil
 }
 func (m *ModelConfig) SanitizedCommand() ([]string, error) {
 	return SanitizeCommand(m.Cmd)
 }
@@ -111,26 +139,23 @@ func LoadConfigFromReader(r io.Reader) (Config, error) {
 		return Config{}, err
 	}
-	var config Config
+	// default configuration values
 	config := Config{
 		HealthCheckTimeout: 120,
 		StartPort:          5800,
 		LogLevel:           "info",
 	}
 	err = yaml.Unmarshal(data, &config)
 	if err != nil {
 		return Config{}, err
 	}
-	if config.HealthCheckTimeout == 0 {
+	if config.HealthCheckTimeout < 15 {
 		// this high default timeout helps avoid failing health checks
 		// for configurations that wait for docker or have slower startup
 		config.HealthCheckTimeout = 120
 	} else if config.HealthCheckTimeout < 15 {
 		// set a minimum of 15 seconds
 		config.HealthCheckTimeout = 15
 	}
-	// set default port ranges
+	if config.StartPort < 1 {
 	if config.StartPort == 0 {
 		// default to 5800
 		config.StartPort = 5800
 	} else if config.StartPort < 1 {
 		return Config{}, fmt.Errorf("startPort must be greater than 1")
 	}
@@ -189,19 +214,18 @@ func LoadConfigFromReader(r io.Reader) (Config, error) {
 			modelConfig.CheckEndpoint = strings.ReplaceAll(modelConfig.CheckEndpoint, macroSlug, macroValue)
 		}
 		// enforce ${PORT} used in both cmd and proxy
 		if !strings.Contains(modelConfig.Cmd, "${PORT}") && strings.Contains(modelConfig.Proxy, "${PORT}") {
 			return Config{}, fmt.Errorf("model %s: proxy uses ${PORT} but cmd does not - ${PORT} is only available when used in cmd", modelId)
 		}
 		// only iterate over models that use ${PORT} to keep port numbers from increasing unnecessarily
 		if strings.Contains(modelConfig.Cmd, "${PORT}") || strings.Contains(modelConfig.Proxy, "${PORT}") || strings.Contains(modelConfig.CmdStop, "${PORT}") {
 			if modelConfig.Proxy == "" {
 				modelConfig.Proxy = "http://localhost:${PORT}"
 			}
 			nextPortStr := strconv.Itoa(nextPort)
 			modelConfig.Cmd = strings.ReplaceAll(modelConfig.Cmd, "${PORT}", nextPortStr)
 			modelConfig.CmdStop = strings.ReplaceAll(modelConfig.CmdStop, "${PORT}", nextPortStr)
 			modelConfig.Proxy = strings.ReplaceAll(modelConfig.Proxy, "${PORT}", nextPortStr)
 			nextPort++
 		} else if modelConfig.Proxy == "" {
 			return Config{}, fmt.Errorf("model %s requires a proxy value when not using automatic ${PORT}", modelId)
 		}
 		// make sure there are no unknown macros that have not been replaced
@@ -217,6 +241,9 @@ func LoadConfigFromReader(r io.Reader) (Config, error) {
 			matches := macroPattern.FindAllStringSubmatch(fieldValue, -1)
 			for _, match := range matches {
 				macroName := match[1]
 				if macroName == "PID" && fieldName == "cmdStop" {
 					continue // this is ok, has to be replaced by process later
 				}
 				if _, exists := config.Macros[macroName]; !exists {
 					return Config{}, fmt.Errorf("unknown macro '${%s}' found in %s.%s", macroName, modelId, fieldName)
 				}
@@ -3,6 +3,9 @@
 package proxy
 import (
 	"os"
 	"path/filepath"
 	"strings"
 	"testing"
 	"github.com/stretchr/testify/assert"
@@ -40,3 +43,184 @@ func TestConfig_SanitizeCommand(t *testing.T) {
 	assert.Error(t, err)
 	assert.Nil(t, args)
 }
 // Test the default values are automatically set for global, model and group configurations
 // after loading the configuration
 func TestConfig_DefaultValuesPosix(t *testing.T) {
 	content := `
 models:
  model1:
    cmd: path/to/cmd --port ${PORT}
 `
 	config, err := LoadConfigFromReader(strings.NewReader(content))
 	assert.NoError(t, err)
 	assert.Equal(t, 120, config.HealthCheckTimeout)
 	assert.Equal(t, 5800, config.StartPort)
 	assert.Equal(t, "info", config.LogLevel)
 	// Test default group exists
 	defaultGroup, exists := config.Groups["(default)"]
 	assert.True(t, exists, "default group should exist")
 	if assert.NotNil(t, defaultGroup, "default group should not be nil") {
 		assert.Equal(t, true, defaultGroup.Swap)
 		assert.Equal(t, true, defaultGroup.Exclusive)
 		assert.Equal(t, false, defaultGroup.Persistent)
 		assert.Equal(t, []string{"model1"}, defaultGroup.Members)
 	}
 	model1, exists := config.Models["model1"]
 	assert.True(t, exists, "model1 should exist")
 	if assert.NotNil(t, model1, "model1 should not be nil") {
 		assert.Equal(t, "path/to/cmd --port 5800", model1.Cmd) // has the port replaced
 		assert.Equal(t, "", model1.CmdStop)
 		assert.Equal(t, "http://localhost:5800", model1.Proxy)
 		assert.Equal(t, "/health", model1.CheckEndpoint)
 		assert.Equal(t, []string{}, model1.Aliases)
 		assert.Equal(t, []string{}, model1.Env)
 		assert.Equal(t, 0, model1.UnloadAfter)
 		assert.Equal(t, false, model1.Unlisted)
 		assert.Equal(t, "", model1.UseModelName)
 		assert.Equal(t, 0, model1.ConcurrencyLimit)
 	}
 }
 func TestConfig_LoadPosix(t *testing.T) {
 	// Create a temporary YAML file for testing
 	tempDir, err := os.MkdirTemp("", "test-config")
 	if err != nil {
 		t.Fatalf("Failed to create temporary directory: %v", err)
 	}
 	defer os.RemoveAll(tempDir)
 	tempFile := filepath.Join(tempDir, "config.yaml")
 	content := `
 macros:
  svr-path: "path/to/server"
 models:
  model1:
    cmd: path/to/cmd --arg1 one
    proxy: "http://localhost:8080"
    aliases:
      - "m1"
      - "model-one"
    env:
      - "VAR1=value1"
      - "VAR2=value2"
    checkEndpoint: "/health"
  model2:
    cmd: ${svr-path} --arg1 one
    proxy: "http://localhost:8081"
    aliases:
      - "m2"
    checkEndpoint: "/"
  model3:
    cmd: path/to/cmd --arg1 one
    proxy: "http://localhost:8081"
    aliases:
      - "mthree"
    checkEndpoint: "/"
  model4:
    cmd: path/to/cmd --arg1 one
    proxy: "http://localhost:8082"
    checkEndpoint: "/"
 healthCheckTimeout: 15
 profiles:
  test:
    - model1
    - model2
 groups:
  group1:
    swap: true
    exclusive: false
    members: ["model2"]
  forever:
    exclusive: false
    persistent: true
    members:
      - "model4"
 `
 	if err := os.WriteFile(tempFile, []byte(content), 0644); err != nil {
 		t.Fatalf("Failed to write temporary file: %v", err)
 	}
 	// Load the config and verify
 	config, err := LoadConfig(tempFile)
 	if err != nil {
 		t.Fatalf("Failed to load config: %v", err)
 	}
 	expected := Config{
 		LogLevel:  "info",
 		StartPort: 5800,
 		Macros: map[string]string{
 			"svr-path": "path/to/server",
 		},
 		Models: map[string]ModelConfig{
 			"model1": {
 				Cmd:           "path/to/cmd --arg1 one",
 				Proxy:         "http://localhost:8080",
 				Aliases:       []string{"m1", "model-one"},
 				Env:           []string{"VAR1=value1", "VAR2=value2"},
 				CheckEndpoint: "/health",
 			},
 			"model2": {
 				Cmd:           "path/to/server --arg1 one",
 				Proxy:         "http://localhost:8081",
 				Aliases:       []string{"m2"},
 				Env:           []string{},
 				CheckEndpoint: "/",
 			},
 			"model3": {
 				Cmd:           "path/to/cmd --arg1 one",
 				Proxy:         "http://localhost:8081",
 				Aliases:       []string{"mthree"},
 				Env:           []string{},
 				CheckEndpoint: "/",
 			},
 			"model4": {
 				Cmd:           "path/to/cmd --arg1 one",
 				Proxy:         "http://localhost:8082",
 				CheckEndpoint: "/",
 				Aliases:       []string{},
 				Env:           []string{},
 			},
 		},
 		HealthCheckTimeout: 15,
 		Profiles: map[string][]string{
 			"test": {"model1", "model2"},
 		},
 		aliases: map[string]string{
 			"m1":        "model1",
 			"model-one": "model1",
 			"m2":        "model2",
 			"mthree":    "model3",
 		},
 		Groups: map[string]GroupConfig{
 			DEFAULT_GROUP_ID: {
 				Swap:      true,
 				Exclusive: true,
 				Members:   []string{"model1", "model3"},
 			},
 			"group1": {
 				Swap:      true,
 				Exclusive: false,
 				Members:   []string{"model2"},
 			},
 			"forever": {
 				Swap:       true,
 				Exclusive:  false,
 				Persistent: true,
 				Members:    []string{"model4"},
 			},
 		},
 	}
 	assert.Equal(t, expected, config)
 	realname, found := config.RealModelName("m1")
 	assert.True(t, found)
 	assert.Equal(t, "model1", realname)
 }
@@ -1,151 +1,12 @@
 package proxy
 import (
 	"os"
 	"path/filepath"
 	"strings"
 	"testing"
 	"github.com/stretchr/testify/assert"
 )
 func TestConfig_Load(t *testing.T) {
 	// Create a temporary YAML file for testing
 	tempDir, err := os.MkdirTemp("", "test-config")
 	if err != nil {
 		t.Fatalf("Failed to create temporary directory: %v", err)
 	}
 	defer os.RemoveAll(tempDir)
 	tempFile := filepath.Join(tempDir, "config.yaml")
 	content := `
 macros:
  svr-path: "path/to/server"
 models:
  model1:
    cmd: path/to/cmd --arg1 one
    proxy: "http://localhost:8080"
    aliases:
      - "m1"
      - "model-one"
    env:
      - "VAR1=value1"
      - "VAR2=value2"
    checkEndpoint: "/health"
  model2:
    cmd: ${svr-path} --arg1 one
    proxy: "http://localhost:8081"
    aliases:
      - "m2"
    checkEndpoint: "/"
  model3:
    cmd: path/to/cmd --arg1 one
    proxy: "http://localhost:8081"
    aliases:
      - "mthree"
    checkEndpoint: "/"
  model4:
    cmd: path/to/cmd --arg1 one
    proxy: "http://localhost:8082"
    checkEndpoint: "/"
 healthCheckTimeout: 15
 profiles:
  test:
    - model1
    - model2
 groups:
  group1:
    swap: true
    exclusive: false
    members: ["model2"]
  forever:
    exclusive: false
    persistent: true
    members:
      - "model4"
 `
 	if err := os.WriteFile(tempFile, []byte(content), 0644); err != nil {
 		t.Fatalf("Failed to write temporary file: %v", err)
 	}
 	// Load the config and verify
 	config, err := LoadConfig(tempFile)
 	if err != nil {
 		t.Fatalf("Failed to load config: %v", err)
 	}
 	expected := Config{
 		StartPort: 5800,
 		Macros: map[string]string{
 			"svr-path": "path/to/server",
 		},
 		Models: map[string]ModelConfig{
 			"model1": {
 				Cmd:           "path/to/cmd --arg1 one",
 				Proxy:         "http://localhost:8080",
 				Aliases:       []string{"m1", "model-one"},
 				Env:           []string{"VAR1=value1", "VAR2=value2"},
 				CheckEndpoint: "/health",
 			},
 			"model2": {
 				Cmd:           "path/to/server --arg1 one",
 				Proxy:         "http://localhost:8081",
 				Aliases:       []string{"m2"},
 				Env:           nil,
 				CheckEndpoint: "/",
 			},
 			"model3": {
 				Cmd:           "path/to/cmd --arg1 one",
 				Proxy:         "http://localhost:8081",
 				Aliases:       []string{"mthree"},
 				Env:           nil,
 				CheckEndpoint: "/",
 			},
 			"model4": {
 				Cmd:           "path/to/cmd --arg1 one",
 				Proxy:         "http://localhost:8082",
 				CheckEndpoint: "/",
 			},
 		},
 		HealthCheckTimeout: 15,
 		Profiles: map[string][]string{
 			"test": {"model1", "model2"},
 		},
 		aliases: map[string]string{
 			"m1":        "model1",
 			"model-one": "model1",
 			"m2":        "model2",
 			"mthree":    "model3",
 		},
 		Groups: map[string]GroupConfig{
 			DEFAULT_GROUP_ID: {
 				Swap:      true,
 				Exclusive: true,
 				Members:   []string{"model1", "model3"},
 			},
 			"group1": {
 				Swap:      true,
 				Exclusive: false,
 				Members:   []string{"model2"},
 			},
 			"forever": {
 				Swap:       true,
 				Exclusive:  false,
 				Persistent: true,
 				Members:    []string{"model4"},
 			},
 		},
 	}
 	assert.Equal(t, expected, config)
 	realname, found := config.RealModelName("m1")
 	assert.True(t, found)
 	assert.Equal(t, "model1", realname)
 }
 func TestConfig_GroupMemberIsUnique(t *testing.T) {
 	content := `
 models:
@@ -333,7 +194,7 @@ models:
    cmd: svr --port 111
 `
 		_, err := LoadConfigFromReader(strings.NewReader(content))
-		assert.Equal(t, "model model1 requires a proxy value when not using automatic ${PORT}", err.Error())
+		assert.Equal(t, "model model1: proxy uses ${PORT} but cmd does not - ${PORT} is only available when used in cmd", err.Error())
 	})
 }
@@ -3,6 +3,9 @@
 package proxy
 import (
 	"os"
 	"path/filepath"
 	"strings"
 	"testing"
 	"github.com/stretchr/testify/assert"
@@ -39,3 +42,186 @@ func TestConfig_SanitizeCommand(t *testing.T) {
 	assert.Error(t, err)
 	assert.Nil(t, args)
 }
 func TestConfig_DefaultValuesWindows(t *testing.T) {
 	content := `
 models:
  model1:
    cmd: path/to/cmd --port ${PORT}
 `
 	config, err := LoadConfigFromReader(strings.NewReader(content))
 	assert.NoError(t, err)
 	assert.Equal(t, 120, config.HealthCheckTimeout)
 	assert.Equal(t, 5800, config.StartPort)
 	assert.Equal(t, "info", config.LogLevel)
 	// Test default group exists
 	defaultGroup, exists := config.Groups["(default)"]
 	assert.True(t, exists, "default group should exist")
 	if assert.NotNil(t, defaultGroup, "default group should not be nil") {
 		assert.Equal(t, true, defaultGroup.Swap)
 		assert.Equal(t, true, defaultGroup.Exclusive)
 		assert.Equal(t, false, defaultGroup.Persistent)
 		assert.Equal(t, []string{"model1"}, defaultGroup.Members)
 	}
 	model1, exists := config.Models["model1"]
 	assert.True(t, exists, "model1 should exist")
 	if assert.NotNil(t, model1, "model1 should not be nil") {
 		assert.Equal(t, "path/to/cmd --port 5800", model1.Cmd) // has the port replaced
 		assert.Equal(t, "taskkill /f /t /pid ${PID}", model1.CmdStop)
 		assert.Equal(t, "http://localhost:5800", model1.Proxy)
 		assert.Equal(t, "/health", model1.CheckEndpoint)
 		assert.Equal(t, []string{}, model1.Aliases)
 		assert.Equal(t, []string{}, model1.Env)
 		assert.Equal(t, 0, model1.UnloadAfter)
 		assert.Equal(t, false, model1.Unlisted)
 		assert.Equal(t, "", model1.UseModelName)
 		assert.Equal(t, 0, model1.ConcurrencyLimit)
 	}
 }
 func TestConfig_LoadWindows(t *testing.T) {
 	// Create a temporary YAML file for testing
 	tempDir, err := os.MkdirTemp("", "test-config")
 	if err != nil {
 		t.Fatalf("Failed to create temporary directory: %v", err)
 	}
 	defer os.RemoveAll(tempDir)
 	tempFile := filepath.Join(tempDir, "config.yaml")
 	content := `
 macros:
  svr-path: "path/to/server"
 models:
  model1:
    cmd: path/to/cmd --arg1 one
    proxy: "http://localhost:8080"
    aliases:
      - "m1"
      - "model-one"
    env:
      - "VAR1=value1"
      - "VAR2=value2"
    checkEndpoint: "/health"
  model2:
    cmd: ${svr-path} --arg1 one
    proxy: "http://localhost:8081"
    aliases:
      - "m2"
    checkEndpoint: "/"
  model3:
    cmd: path/to/cmd --arg1 one
    proxy: "http://localhost:8081"
    aliases:
      - "mthree"
    checkEndpoint: "/"
  model4:
    cmd: path/to/cmd --arg1 one
    proxy: "http://localhost:8082"
    checkEndpoint: "/"
 healthCheckTimeout: 15
 profiles:
  test:
    - model1
    - model2
 groups:
  group1:
    swap: true
    exclusive: false
    members: ["model2"]
  forever:
    exclusive: false
    persistent: true
    members:
      - "model4"
 `
 	if err := os.WriteFile(tempFile, []byte(content), 0644); err != nil {
 		t.Fatalf("Failed to write temporary file: %v", err)
 	}
 	// Load the config and verify
 	config, err := LoadConfig(tempFile)
 	if err != nil {
 		t.Fatalf("Failed to load config: %v", err)
 	}
 	expected := Config{
 		LogLevel:  "info",
 		StartPort: 5800,
 		Macros: map[string]string{
 			"svr-path": "path/to/server",
 		},
 		Models: map[string]ModelConfig{
 			"model1": {
 				Cmd:           "path/to/cmd --arg1 one",
 				CmdStop:       "taskkill /f /t /pid ${PID}",
 				Proxy:         "http://localhost:8080",
 				Aliases:       []string{"m1", "model-one"},
 				Env:           []string{"VAR1=value1", "VAR2=value2"},
 				CheckEndpoint: "/health",
 			},
 			"model2": {
 				Cmd:           "path/to/server --arg1 one",
 				CmdStop:       "taskkill /f /t /pid ${PID}",
 				Proxy:         "http://localhost:8081",
 				Aliases:       []string{"m2"},
 				Env:           []string{},
 				CheckEndpoint: "/",
 			},
 			"model3": {
 				Cmd:           "path/to/cmd --arg1 one",
 				CmdStop:       "taskkill /f /t /pid ${PID}",
 				Proxy:         "http://localhost:8081",
 				Aliases:       []string{"mthree"},
 				Env:           []string{},
 				CheckEndpoint: "/",
 			},
 			"model4": {
 				Cmd:           "path/to/cmd --arg1 one",
 				CmdStop:       "taskkill /f /t /pid ${PID}",
 				Proxy:         "http://localhost:8082",
 				CheckEndpoint: "/",
 				Aliases:       []string{},
 				Env:           []string{},
 			},
 		},
 		HealthCheckTimeout: 15,
 		Profiles: map[string][]string{
 			"test": {"model1", "model2"},
 		},
 		aliases: map[string]string{
 			"m1":        "model1",
 			"model-one": "model1",
 			"m2":        "model2",
 			"mthree":    "model3",
 		},
 		Groups: map[string]GroupConfig{
 			DEFAULT_GROUP_ID: {
 				Swap:      true,
 				Exclusive: true,
 				Members:   []string{"model1", "model3"},
 			},
 			"group1": {
 				Swap:      true,
 				Exclusive: false,
 				Members:   []string{"model2"},
 			},
 			"forever": {
 				Swap:       true,
 				Exclusive:  false,
 				Persistent: true,
 				Members:    []string{"model4"},
 			},
 		},
 	}
 	assert.Equal(t, expected, config)
 	realname, found := config.RealModelName("m1")
 	assert.True(t, found)
 	assert.Equal(t, "model1", realname)
 }
@@ -9,6 +9,7 @@ import (
 	"testing"
 	"github.com/gin-gonic/gin"
 	"gopkg.in/yaml.v3"
 )
 var (
@@ -70,10 +71,16 @@ func getTestSimpleResponderConfig(expectedMessage string) ModelConfig {
 func getTestSimpleResponderConfigPort(expectedMessage string, port int) ModelConfig {
 	binaryPath := getSimpleResponderPath()
-	// Create a process configuration
+	// Create a YAML string with just the values we want to set
-	return ModelConfig{
+	yamlStr := fmt.Sprintf(`
-		Cmd:           fmt.Sprintf("%s --port %d --silent --respond %s", binaryPath, port, expectedMessage),
+cmd: '%s --port %d --silent --respond %s'
-		Proxy:         fmt.Sprintf("http://127.0.0.1:%d", port),
+proxy: "http://127.0.0.1:%d"
-		CheckEndpoint: "/health",
+`, binaryPath, port, expectedMessage, port)
 	var cfg ModelConfig
 	if err := yaml.Unmarshal([]byte(yamlStr), &cfg); err != nil {
 		panic(fmt.Sprintf("failed to unmarshal test config: %v in [%s]", err, yamlStr))
 	}
 	return cfg
 }
@@ -1,14 +0,0 @@
 <!DOCTYPE html>
 <html lang="en">
 <head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>llama-swap</title>
 </head>
 <body>
    <h1>llama-swap</h1>
    <p>
        <a href="/logs">view logs</a> | <a href="/upstream">configured models</a> | <a href="https://github.com/mostlygeek/llama-swap">github</a>
    </p>
 </body>
 </html>
@@ -1,259 +0,0 @@
 <!DOCTYPE html>
 <html lang="en">
 <head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Logs</title>
    <style>
        body {
            margin: 0;
            height: 100vh;
            display: flex;
            flex-direction: column;
            font-family: "Courier New", Courier, monospace;
        }
        .log-container {
            display: flex;
            flex: 1;
            gap: 0.5em;
            margin: 0.5em;
            min-height: 0;
        }
        .log-column {
            display: flex;
            flex-direction: column;
            flex: 1;
            min-width: 0;
            transition: flex 0.3s ease;
        }
        .log-column.minimized {
            flex: 0.1;
            max-width: 50px;
            border: 1px solid #777;
            color: green;
        }
        .log-controls {
            display: grid;
            grid-template-columns: 1fr auto;
            gap: 0.5em;
            margin-bottom: 0.5em;
        }
        .log-controls input {
            width: 100%;
            padding: 4px;
        }
        .log-controls input:focus {
           outline: none;
        }
        .log-stream {
            flex: 1;
            padding: 1em;
            background: #f4f4f4;
            overflow-y: auto;
            white-space: pre-wrap;
            word-wrap: break-word;
            min-height: 0;
        }
        .regex-error {
            background-color: #ff0000 !important;
        }
        /* Make headers clickable and show pointer cursor */
        h2 {
            cursor: pointer;
            user-select: none;
            margin: 0 0 0.5em 0;
            padding: 0.5em;
        }
        h2:hover {
            background-color: rgba(0, 0, 0, 0.05);
        }
        /* Dark mode styles */
        @media (prefers-color-scheme: dark) {
            body {
                background-color: #333;
                color: #fff;
            }
            .log-stream {
                background: #444;
                color: #fff;
            }
            .log-controls input {
                background: #555;
                color: #fff;
                border: 1px solid #777;
            }
            .log-controls button {
                background: #555;
                color: #fff;
                border: 1px solid #777;
            }
            h2:hover {
                background-color: rgba(255, 255, 255, 0.1);
            }
        }
        /* Hide content when minimized */
        .log-column.minimized .log-controls,
        .log-column.minimized .log-stream {
            display: none;
        }
        .log-column.minimized h2 {
            writing-mode: vertical-rl;
            text-orientation: mixed;
            transform: rotate(180deg);
            white-space: nowrap;
            margin: auto;
        }
    </style>
 </head>
 <body>
    <div class="log-container">
        <div class="log-column">
            <h2>Proxy Logs</h2>
            <div class="log-controls">
                <input type="text" id="proxy-filter-input" placeholder="proxy regex filter">
                <button id="proxy-clear-button">clear</button>
            </div>
            <pre class="log-stream" id="proxy-log-stream">Waiting for proxy logs...</pre>
        </div>
        <div class="log-column minimized">
            <h2>Upstream Logs</h2>
            <div class="log-controls">
                <input type="text" id="upstream-filter-input" placeholder="upstream regex filter">
                <button id="upstream-clear-button">clear</button>
            </div>
            <pre class="log-stream" id="upstream-log-stream">Waiting for upstream logs...</pre>
        </div>
    </div>
    <script>
        class LogStream {
            constructor(streamElement, filterInput, clearButton, endpoint) {
                this.streamElement = streamElement;
                this.filterInput = filterInput;
                this.clearButton = clearButton;
                this.endpoint = endpoint;
                this.logData = "";
                this.regexFilter = null;
                this.eventSource = null;
                this.initialize();
            }
            initialize() {
                this.filterInput.addEventListener('input', () => this.updateFilter());
                this.clearButton.addEventListener('click', () => {
                    this.filterInput.value = "";
                    this.regexFilter = null;
                    this.render();
                });
                this.setupEventSource();
            }
            setupEventSource() {
                if (typeof(EventSource) === "undefined") {
                    this.logData = "SSE Not supported by this browser.";
                    this.render();
                    return;
                }
                const connect = () => {
                    this.eventSource = new EventSource(this.endpoint);
                    this.eventSource.onmessage = (event) => {
                        this.logData += event.data;
                        this.logData = this.logData.slice(-1024 * 100);
                        this.render();
                    };
                    this.eventSource.onerror = (err) => {
                        // Close the current connection
                        this.eventSource.close();
                        this.logData += "\nConnection lost. Retrying in 5 seconds...\n";
                        this.render();
                        // Attempt to reconnect after 5 seconds
                        setTimeout(() => {
                            this.logData += "Attempting to reconnect...\n";
                            this.render();
                            connect();
                        }, 5000);
                    };
                };
                // Initial connection
                connect();
            }
            render() {
                let content = this.logData;
                if (this.regexFilter) {
                    const lines = content.split('\n');
                    const filteredLines = lines.filter(line => this.regexFilter.test(line));
                    content = filteredLines.length > 0 ? filteredLines.join('\n') + '\n' : "";
                }
                this.streamElement.textContent = content;
                this.streamElement.scrollTop = this.streamElement.scrollHeight;
            }
            updateFilter() {
                const pattern = this.filterInput.value.trim();
                this.filterInput.classList.remove('regex-error');
                if (!pattern) {
                    this.regexFilter = null;
                    this.render();
                    return;
                }
                try {
                    this.regexFilter = new RegExp(pattern);
                } catch (e) {
                    console.error("Invalid regex pattern:", e);
                    this.regexFilter = null;
                    this.filterInput.classList.add('regex-error');
                    return;
                }
                this.render();
            }
        }
        // Initialize both log streams
        document.addEventListener('DOMContentLoaded', () => {
            new LogStream(
                document.getElementById('proxy-log-stream'),
                document.getElementById('proxy-filter-input'),
                document.getElementById('proxy-clear-button'),
                "/logs/streamSSE/proxy"
            );
            new LogStream(
                document.getElementById('upstream-log-stream'),
                document.getElementById('upstream-filter-input'),
                document.getElementById('upstream-clear-button'),
                "/logs/streamSSE/upstream"
            );
            // Initialize clickable headers
            document.querySelectorAll('h2').forEach(header => {
                header.addEventListener('click', () => {
                    const column = header.closest('.log-column');
                    column.classList.toggle('minimized');
                });
            });
        });
    </script>
 </body>
 </html>
@@ -1,10 +0,0 @@
 package proxy
 import "embed"
 //go:embed html
 var htmlFiles embed.FS
 func getHTMLFile(path string) ([]byte, error) {
 	return htmlFiles.ReadFile("html/" + path)
 }
@@ -8,7 +8,6 @@ import (
 	"net/http"
 	"net/url"
 	"os/exec"
 	"runtime"
 	"strconv"
 	"strings"
 	"sync"
@@ -190,17 +189,19 @@ func (p *Process) start() error {
 	p.waitStarting.Add(1)
 	defer p.waitStarting.Done()
 	cmdContext, ctxCancelUpstream := context.WithCancel(context.Background())
 	p.cmd = exec.CommandContext(cmdContext, args[0], args[1:]...)
 	p.cmd.Stdout = p.processLogger
 	p.cmd.Stderr = p.processLogger
-	p.cmd.Env = p.config.Env
+	p.cmd.Env = append(p.cmd.Environ(), p.config.Env...)
 	p.cmd.Cancel = p.cmdStopUpstreamProcess
 	p.cmd.WaitDelay = p.gracefulStopTimeout
 	p.cancelUpstream = ctxCancelUpstream
 	p.cmdWaitChan = make(chan struct{})
 	p.failedStartCount++ // this will be reset to zero when the process has successfully started
 	p.proxyLogger.Debugf("<%s> Executing start command: %s, env: %s", p.ID, strings.Join(args, " "), strings.Join(p.config.Env, ", "))
 	err = p.cmd.Start()
 	// Set process state to failed
@@ -232,11 +233,6 @@ func (p *Process) start() error {
 	// a "none" means don't check for health ... I could have picked a better word :facepalm:
 	if checkEndpoint != "none" {
 		// keep default behaviour
 		if checkEndpoint == "" {
 			checkEndpoint = "/health"
 		}
 		proxyTo := p.config.Proxy
 		healthURL, err := url.JoinPath(proxyTo, checkEndpoint)
 		if err != nil {
@@ -523,11 +519,6 @@ func (p *Process) cmdStopUpstreamProcess() error {
 		return fmt.Errorf("<%s> process is nil or cmd is nil, skipping graceful stop", p.ID)
 	}
 	// the default cmdStop to taskkill /f /t /pid ${PID}
 	if runtime.GOOS == "windows" && strings.TrimSpace(p.config.CmdStop) == "" {
 		p.config.CmdStop = "taskkill /f /t /pid ${PID}"
 	}
 	if p.config.CmdStop != "" {
 		// replace ${PID} with the pid of the process
 		stopArgs, err := SanitizeCommand(strings.ReplaceAll(p.config.CmdStop, "${PID}", fmt.Sprintf("%d", p.cmd.Process.Pid)))
@@ -394,6 +394,9 @@ func TestProcess_StopImmediately(t *testing.T) {
 // Test that SIGKILL is sent when gracefulStopTimeout is reached and properly terminates
 // the upstream command
 func TestProcess_ForceStopWithKill(t *testing.T) {
 	if runtime.GOOS == "windows" {
 		t.Skip("skipping SIGTERM test on Windows ")
 	}
 	expectedMessage := "test_sigkill"
 	binaryPath := getSimpleResponderPath()
@@ -464,3 +467,27 @@ func TestProcess_StopCmd(t *testing.T) {
 	process.StopImmediately()
 	assert.Equal(t, process.CurrentState(), StateStopped)
 }
 func TestProcess_EnvironmentSetCorrectly(t *testing.T) {
 	expectedMessage := "test_env_not_emptied"
 	config := getTestSimpleResponderConfig(expectedMessage)
 	// ensure that the the default config does not blank out the inherited environment
 	configWEnv := config
 	// ensure the additiona variables are appended to the process' environment
 	configWEnv.Env = append(configWEnv.Env, "TEST_ENV1=1", "TEST_ENV2=2")
 	process1 := NewProcess("env_test", 2, config, debugLogger, debugLogger)
 	process2 := NewProcess("env_test", 2, configWEnv, debugLogger, debugLogger)
 	process1.start()
 	defer process1.Stop()
 	process2.start()
 	defer process2.Stop()
 	assert.NotZero(t, len(process1.cmd.Environ()))
 	assert.NotZero(t, len(process2.cmd.Environ()))
 	assert.Equal(t, len(process1.cmd.Environ())+2, len(process2.cmd.Environ()), "process2 should have 2 more environment variables than process1")
 }
@@ -8,7 +8,6 @@ import (
 	"mime/multipart"
 	"net/http"
 	"os"
 	"sort"
 	"strconv"
 	"strings"
 	"sync"
@@ -163,38 +162,59 @@ func (pm *ProxyManager) setupGinEngine() {
 	pm.ginEngine.GET("/logs/stream/:logMonitorID", pm.streamLogsHandler)
 	pm.ginEngine.GET("/logs/streamSSE/:logMonitorID", pm.streamLogsHandlerSSE)
-	pm.ginEngine.GET("/upstream", pm.upstreamIndex)
+	/**
 	 * User Interface Endpoints
 	 */
 	pm.ginEngine.GET("/", func(c *gin.Context) {
 		c.Redirect(http.StatusFound, "/ui")
 	})
 	pm.ginEngine.GET("/upstream", func(c *gin.Context) {
 		c.Redirect(http.StatusFound, "/ui/models")
 	})
 	pm.ginEngine.Any("/upstream/:model_id/*upstreamPath", pm.proxyToUpstream)
 	pm.ginEngine.GET("/unload", pm.unloadAllModelsHandler)
 	pm.ginEngine.GET("/running", pm.listRunningProcessesHandler)
 	pm.ginEngine.GET("/", func(c *gin.Context) {
 		// Set the Content-Type header to text/html
 		c.Header("Content-Type", "text/html")
 		// Write the embedded HTML content to the response
 		htmlData, err := getHTMLFile("index.html")
 		if err != nil {
 			c.String(http.StatusInternalServerError, err.Error())
 			return
 		}
 		_, err = c.Writer.Write(htmlData)
 		if err != nil {
 			c.String(http.StatusInternalServerError, fmt.Sprintf("failed to write response: %v", err))
 			return
 		}
 	})
 	pm.ginEngine.GET("/favicon.ico", func(c *gin.Context) {
-		if data, err := getHTMLFile("favicon.ico"); err == nil {
+		if data, err := reactStaticFS.ReadFile("ui_dist/favicon.ico"); err == nil {
 			c.Data(http.StatusOK, "image/x-icon", data)
 		} else {
 			c.String(http.StatusInternalServerError, err.Error())
 		}
 	})
 	reactFS, err := GetReactFS()
 	if err != nil {
 		pm.proxyLogger.Errorf("Failed to load React filesystem: %v", err)
 	} else {
 		// serve files that exist under /ui/*
 		pm.ginEngine.StaticFS("/ui", reactFS)
 		// server SPA for UI under /ui/*
 		pm.ginEngine.NoRoute(func(c *gin.Context) {
 			if !strings.HasPrefix(c.Request.URL.Path, "/ui") {
 				c.AbortWithStatus(http.StatusNotFound)
 				return
 			}
 			file, err := reactFS.Open("index.html")
 			if err != nil {
 				c.String(http.StatusInternalServerError, err.Error())
 				return
 			}
 			defer file.Close()
 			http.ServeContent(c.Writer, c.Request, "index.html", time.Now(), file)
 		})
 	}
 	// see: proxymanager_api.go
 	// add API handler functions
 	addApiHandlers(pm)
 	// Disable console color for testing
 	gin.DisableConsoleColor()
 }
@@ -316,55 +336,6 @@ func (pm *ProxyManager) proxyToUpstream(c *gin.Context) {
 	processGroup.ProxyRequest(requestedModel, c.Writer, c.Request)
 }
 func (pm *ProxyManager) upstreamIndex(c *gin.Context) {
 	var html strings.Builder
 	html.WriteString("<!doctype HTML>\n<html><body><h1>Available Models</h1><a href=\"/unload\">Unload all models</a><ul>")
 	// Extract keys and sort them
 	var modelIDs []string
 	for modelID, modelConfig := range pm.config.Models {
 		if modelConfig.Unlisted {
 			continue
 		}
 		modelIDs = append(modelIDs, modelID)
 	}
 	sort.Strings(modelIDs)
 	// Iterate over sorted keys
 	for _, modelID := range modelIDs {
 		// Get process state
 		processGroup := pm.findGroupByModelName(modelID)
 		var state string
 		if processGroup != nil {
 			process := processGroup.processes[modelID]
 			if process != nil {
 				var stateStr string
 				switch process.CurrentState() {
 				case StateReady:
 					stateStr = "Ready"
 				case StateStarting:
 					stateStr = "Starting"
 				case StateStopping:
 					stateStr = "Stopping"
 				case StateShutdown:
 					stateStr = "Shutdown"
 				case StateStopped:
 					stateStr = "Stopped"
 				default:
 					stateStr = "Unknown"
 				}
 				state = stateStr
 			}
 		}
 		html.WriteString(fmt.Sprintf("<li><a href=\"/upstream/%s\">%s</a> - %s</li>", modelID, modelID, state))
 	}
 	html.WriteString("</ul></body></html>")
 	c.Header("Content-Type", "text/html")
 	c.String(http.StatusOK, html.String())
 }
 func (pm *ProxyManager) proxyOAIHandler(c *gin.Context) {
 	bodyBytes, err := io.ReadAll(c.Request.Body)
 	if err != nil {
@@ -0,0 +1,101 @@
 package proxy
 import (
 	"net/http"
 	"sort"
 	"time"
 	"github.com/gin-gonic/gin"
 )
 type Model struct {
 	Id    string `json:"id"`
 	State string `json:"state"`
 }
 func addApiHandlers(pm *ProxyManager) {
 	// Add API endpoints for React to consume
 	apiGroup := pm.ginEngine.Group("/api")
 	{
 		apiGroup.GET("/models", pm.apiListModels)
 		apiGroup.GET("/modelsSSE", pm.apiListModelsSSE)
 		apiGroup.POST("/models/unload", pm.apiUnloadAllModels)
 	}
 }
 func (pm *ProxyManager) apiUnloadAllModels(c *gin.Context) {
 	pm.StopProcesses(StopImmediately)
 	c.JSON(http.StatusOK, gin.H{"msg": "ok"})
 }
 func (pm *ProxyManager) getModelStatus() []Model {
 	// Extract keys and sort them
 	models := []Model{}
 	modelIDs := make([]string, 0, len(pm.config.Models))
 	for modelID := range pm.config.Models {
 		modelIDs = append(modelIDs, modelID)
 	}
 	sort.Strings(modelIDs)
 	// Iterate over sorted keys
 	for _, modelID := range modelIDs {
 		// Get process state
 		processGroup := pm.findGroupByModelName(modelID)
 		state := "unknown"
 		if processGroup != nil {
 			process := processGroup.processes[modelID]
 			if process != nil {
 				var stateStr string
 				switch process.CurrentState() {
 				case StateReady:
 					stateStr = "ready"
 				case StateStarting:
 					stateStr = "starting"
 				case StateStopping:
 					stateStr = "stopping"
 				case StateShutdown:
 					stateStr = "shutdown"
 				case StateStopped:
 					stateStr = "stopped"
 				default:
 					stateStr = "unknown"
 				}
 				state = stateStr
 			}
 		}
 		models = append(models, Model{
 			Id:    modelID,
 			State: state,
 		})
 	}
 	return models
 }
 func (pm *ProxyManager) apiListModels(c *gin.Context) {
 	c.JSON(http.StatusOK, pm.getModelStatus())
 }
 // stream the models as a SSE
 func (pm *ProxyManager) apiListModelsSSE(c *gin.Context) {
 	c.Header("Content-Type", "text/event-stream")
 	c.Header("Cache-Control", "no-cache")
 	c.Header("Connection", "keep-alive")
 	c.Header("X-Content-Type-Options", "nosniff")
 	notify := c.Request.Context().Done()
 	// Stream new events
 	for {
 		select {
 		case <-notify:
 			return
 		default:
 			models := pm.getModelStatus()
 			c.SSEvent("message", models)
 			c.Writer.Flush()
 			<-time.After(1000 * time.Millisecond)
 		}
 	}
 }
@@ -11,20 +11,7 @@ import (
 func (pm *ProxyManager) sendLogsHandlers(c *gin.Context) {
 	accept := c.GetHeader("Accept")
 	if strings.Contains(accept, "text/html") {
-		// Set the Content-Type header to text/html
+		c.Redirect(http.StatusFound, "/ui/")
 		c.Header("Content-Type", "text/html")
 		// Write the embedded HTML content to the response
 		logsHTML, err := getHTMLFile("logs.html")
 		if err != nil {
 			c.String(http.StatusInternalServerError, err.Error())
 			return
 		}
 		_, err = c.Writer.Write(logsHTML)
 		if err != nil {
 			c.String(http.StatusInternalServerError, fmt.Sprintf("failed to write response: %v", err))
 			return
 		}
 	} else {
 		c.Header("Content-Type", "text/plain")
 		history := pm.muxLogger.GetHistory()
@@ -0,0 +1,24 @@
 package proxy
 import (
 	"embed"
 	"io/fs"
 	"net/http"
 )
 //go:embed ui_dist
 var reactStaticFS embed.FS
 // GetReactFS returns the embedded React filesystem
 func GetReactFS() (http.FileSystem, error) {
 	subFS, err := fs.Sub(reactStaticFS, "ui_dist")
 	if err != nil {
 		return nil, err
 	}
 	return http.FS(subFS), nil
 }
 // GetReactIndexHTML returns the main index.html for the React app
 func GetReactIndexHTML() ([]byte, error) {
 	return reactStaticFS.ReadFile("ui_dist/index.html")
 }
@@ -0,0 +1,25 @@
 .vite
 # Logs
 logs
 *.log
 npm-debug.log*
 yarn-debug.log*
 yarn-error.log*
 pnpm-debug.log*
 lerna-debug.log*
 node_modules
 dist
 dist-ssr
 *.local
 # Editor directories and files
 .vscode/*
 !.vscode/extensions.json
 .idea
 .DS_Store
 *.suo
 *.ntvs*
 *.njsproj
 *.sln
 *.sw?
@@ -0,0 +1,54 @@
 # React + TypeScript + Vite
 This template provides a minimal setup to get React working in Vite with HMR and some ESLint rules.
 Currently, two official plugins are available:
 - [@vitejs/plugin-react](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react) uses [Babel](https://babeljs.io/) for Fast Refresh
 - [@vitejs/plugin-react-swc](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react-swc) uses [SWC](https://swc.rs/) for Fast Refresh
 ## Expanding the ESLint configuration
 If you are developing a production application, we recommend updating the configuration to enable type-aware lint rules:
 ```js
 export default tseslint.config({
  extends: [
    // Remove ...tseslint.configs.recommended and replace with this
    ...tseslint.configs.recommendedTypeChecked,
    // Alternatively, use this for stricter rules
    ...tseslint.configs.strictTypeChecked,
    // Optionally, add this for stylistic rules
    ...tseslint.configs.stylisticTypeChecked,
  ],
  languageOptions: {
    // other options...
    parserOptions: {
      project: ['./tsconfig.node.json', './tsconfig.app.json'],
      tsconfigRootDir: import.meta.dirname,
    },
  },
 })
 ```
 You can also install [eslint-plugin-react-x](https://github.com/Rel1cx/eslint-react/tree/main/packages/plugins/eslint-plugin-react-x) and [eslint-plugin-react-dom](https://github.com/Rel1cx/eslint-react/tree/main/packages/plugins/eslint-plugin-react-dom) for React-specific lint rules:
 ```js
 // eslint.config.js
 import reactX from 'eslint-plugin-react-x'
 import reactDom from 'eslint-plugin-react-dom'
 export default tseslint.config({
  plugins: {
    // Add the react-x and react-dom plugins
    'react-x': reactX,
    'react-dom': reactDom,
  },
  rules: {
    // other rules...
    // Enable its recommended typescript rules
    ...reactX.configs['recommended-typescript'].rules,
    ...reactDom.configs.recommended.rules,
  },
 })
 ```
@@ -0,0 +1,28 @@
 import js from '@eslint/js'
 import globals from 'globals'
 import reactHooks from 'eslint-plugin-react-hooks'
 import reactRefresh from 'eslint-plugin-react-refresh'
 import tseslint from 'typescript-eslint'
 export default tseslint.config(
  { ignores: ['dist'] },
  {
    extends: [js.configs.recommended, ...tseslint.configs.recommended],
    files: ['**/*.{ts,tsx}'],
    languageOptions: {
      ecmaVersion: 2020,
      globals: globals.browser,
    },
    plugins: {
      'react-hooks': reactHooks,
      'react-refresh': reactRefresh,
    },
    rules: {
      ...reactHooks.configs.recommended.rules,
      'react-refresh/only-export-components': [
        'warn',
        { allowConstantExport: true },
      ],
    },
  },
 )
@@ -0,0 +1,13 @@
 <!doctype html>
 <html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <link rel="icon" type="image/png" href="/favicon.ico" />
    <title>llama-swap</title>
  </head>
  <body >
    <div id="root"></div>
    <script type="module" src="/src/main.tsx"></script>
  </body>
 </html>
@@ -0,0 +1,33 @@
 {
  "name": "ui",
  "private": true,
  "version": "0.0.0",
  "type": "module",
  "scripts": {
    "dev": "vite",
    "build": "tsc -b && vite build --emptyOutDir",
    "lint": "eslint .",
    "preview": "vite preview"
  },
  "dependencies": {
    "@tailwindcss/vite": "^4.1.8",
    "@tanstack/react-query": "^5.80.6",
    "react": "^19.1.0",
    "react-dom": "^19.1.0",
    "react-router-dom": "^7.6.2",
    "tailwindcss": "^4.1.8"
  },
  "devDependencies": {
    "@eslint/js": "^9.25.0",
    "@types/react": "^19.1.2",
    "@types/react-dom": "^19.1.2",
    "@vitejs/plugin-react": "^4.4.1",
    "eslint": "^9.25.0",
    "eslint-plugin-react-hooks": "^5.2.0",
    "eslint-plugin-react-refresh": "^0.4.19",
    "globals": "^16.0.0",
    "typescript": "~5.8.3",
    "typescript-eslint": "^8.30.1",
    "vite": "^6.3.5"
  }
 }
@@ -0,0 +1,6 @@
 #root {
  max-width: 1280px;
  margin: 0 auto;
  padding: 2rem;
  text-align: center;
 }
@@ -0,0 +1,44 @@
 import { BrowserRouter as Router, Routes, Route, Navigate, NavLink } from "react-router-dom";
 import { useTheme } from "./contexts/ThemeProvider";
 import { APIProvider } from "./contexts/APIProvider";
 import LogViewerPage from "./pages/LogViewer";
 import ModelPage from "./pages/Models";
 function App() {
  const theme = useTheme();
  return (
    <Router basename="/ui/">
      <APIProvider>
        <div>
          <nav className="bg-surface border-b border-border p-4">
            <div className="flex items-center justify-between mx-auto px-4">
              <h1>llama-swap</h1>
              <div className="flex space-x-4">
                <NavLink to="/" className={({ isActive }) => (isActive ? "navlink active" : "navlink")}>
                  Logs
                </NavLink>
                <NavLink to="/models" className={({ isActive }) => (isActive ? "navlink active" : "navlink")}>
                  Models
                </NavLink>
                <button className="btn btn--sm" onClick={theme.toggleTheme}>
                  {theme.isDarkMode ? "🌙" : "☀️"}
                </button>
              </div>
            </div>
          </nav>
          <main className="mx-auto py-4 px-4">
            <Routes>
              <Route path="/" element={<LogViewerPage />} />
              <Route path="/models" element={<ModelPage />} />
              <Route path="*" element={<Navigate to="/" replace />} />
            </Routes>
          </main>
        </div>
      </APIProvider>
    </Router>
  );
 }
 export default App;
@@ -0,0 +1 @@
 <svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" class="iconify iconify--logos" width="35.93" height="32" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 228"><path fill="#00D8FF" d="M210.483 73.824a171.49 171.49 0 0 0-8.24-2.597c.465-1.9.893-3.777 1.273-5.621c6.238-30.281 2.16-54.676-11.769-62.708c-13.355-7.7-35.196.329-57.254 19.526a171.23 171.23 0 0 0-6.375 5.848a155.866 155.866 0 0 0-4.241-3.917C100.759 3.829 77.587-4.822 63.673 3.233C50.33 10.957 46.379 33.89 51.995 62.588a170.974 170.974 0 0 0 1.892 8.48c-3.28.932-6.445 1.924-9.474 2.98C17.309 83.498 0 98.307 0 113.668c0 15.865 18.582 31.778 46.812 41.427a145.52 145.52 0 0 0 6.921 2.165a167.467 167.467 0 0 0-2.01 9.138c-5.354 28.2-1.173 50.591 12.134 58.266c13.744 7.926 36.812-.22 59.273-19.855a145.567 145.567 0 0 0 5.342-4.923a168.064 168.064 0 0 0 6.92 6.314c21.758 18.722 43.246 26.282 56.54 18.586c13.731-7.949 18.194-32.003 12.4-61.268a145.016 145.016 0 0 0-1.535-6.842c1.62-.48 3.21-.974 4.76-1.488c29.348-9.723 48.443-25.443 48.443-41.52c0-15.417-17.868-30.326-45.517-39.844Zm-6.365 70.984c-1.4.463-2.836.91-4.3 1.345c-3.24-10.257-7.612-21.163-12.963-32.432c5.106-11 9.31-21.767 12.459-31.957c2.619.758 5.16 1.557 7.61 2.4c23.69 8.156 38.14 20.213 38.14 29.504c0 9.896-15.606 22.743-40.946 31.14Zm-10.514 20.834c2.562 12.94 2.927 24.64 1.23 33.787c-1.524 8.219-4.59 13.698-8.382 15.893c-8.067 4.67-25.32-1.4-43.927-17.412a156.726 156.726 0 0 1-6.437-5.87c7.214-7.889 14.423-17.06 21.459-27.246c12.376-1.098 24.068-2.894 34.671-5.345a134.17 134.17 0 0 1 1.386 6.193ZM87.276 214.515c-7.882 2.783-14.16 2.863-17.955.675c-8.075-4.657-11.432-22.636-6.853-46.752a156.923 156.923 0 0 1 1.869-8.499c10.486 2.32 22.093 3.988 34.498 4.994c7.084 9.967 14.501 19.128 21.976 27.15a134.668 134.668 0 0 1-4.877 4.492c-9.933 8.682-19.886 14.842-28.658 17.94ZM50.35 144.747c-12.483-4.267-22.792-9.812-29.858-15.863c-6.35-5.437-9.555-10.836-9.555-15.216c0-9.322 13.897-21.212 37.076-29.293c2.813-.98 5.757-1.905 8.812-2.773c3.204 10.42 7.406 21.315 12.477 32.332c-5.137 11.18-9.399 22.249-12.634 32.792a134.718 134.718 0 0 1-6.318-1.979Zm12.378-84.26c-4.811-24.587-1.616-43.134 6.425-47.789c8.564-4.958 27.502 2.111 47.463 19.835a144.318 144.318 0 0 1 3.841 3.545c-7.438 7.987-14.787 17.08-21.808 26.988c-12.04 1.116-23.565 2.908-34.161 5.309a160.342 160.342 0 0 1-1.76-7.887Zm110.427 27.268a347.8 347.8 0 0 0-7.785-12.803c8.168 1.033 15.994 2.404 23.343 4.08c-2.206 7.072-4.956 14.465-8.193 22.045a381.151 381.151 0 0 0-7.365-13.322Zm-45.032-43.861c5.044 5.465 10.096 11.566 15.065 18.186a322.04 322.04 0 0 0-30.257-.006c4.974-6.559 10.069-12.652 15.192-18.18ZM82.802 87.83a323.167 323.167 0 0 0-7.227 13.238c-3.184-7.553-5.909-14.98-8.134-22.152c7.304-1.634 15.093-2.97 23.209-3.984a321.524 321.524 0 0 0-7.848 12.897Zm8.081 65.352c-8.385-.936-16.291-2.203-23.593-3.793c2.26-7.3 5.045-14.885 8.298-22.6a321.187 321.187 0 0 0 7.257 13.246c2.594 4.48 5.28 8.868 8.038 13.147Zm37.542 31.03c-5.184-5.592-10.354-11.779-15.403-18.433c4.902.192 9.899.29 14.978.29c5.218 0 10.376-.117 15.453-.343c-4.985 6.774-10.018 12.97-15.028 18.486Zm52.198-57.817c3.422 7.8 6.306 15.345 8.596 22.52c-7.422 1.694-15.436 3.058-23.88 4.071a382.417 382.417 0 0 0 7.859-13.026a347.403 347.403 0 0 0 7.425-13.565Zm-16.898 8.101a358.557 358.557 0 0 1-12.281 19.815a329.4 329.4 0 0 1-23.444.823c-7.967 0-15.716-.248-23.178-.732a310.202 310.202 0 0 1-12.513-19.846h.001a307.41 307.41 0 0 1-10.923-20.627a310.278 310.278 0 0 1 10.89-20.637l-.001.001a307.318 307.318 0 0 1 12.413-19.761c7.613-.576 15.42-.876 23.31-.876H128c7.926 0 15.743.303 23.354.883a329.357 329.357 0 0 1 12.335 19.695a358.489 358.489 0 0 1 11.036 20.54a329.472 329.472 0 0 1-11 20.722Zm22.56-122.124c8.572 4.944 11.906 24.881 6.52 51.026c-.344 1.668-.73 3.367-1.15 5.09c-10.622-2.452-22.155-4.275-34.23-5.408c-7.034-10.017-14.323-19.124-21.64-27.008a160.789 160.789 0 0 1 5.888-5.4c18.9-16.447 36.564-22.941 44.612-18.3ZM128 90.808c12.625 0 22.86 10.235 22.86 22.86s-10.235 22.86-22.86 22.86s-22.86-10.235-22.86-22.86s10.235-22.86 22.86-22.86Z"></path></svg>
@@ -0,0 +1,174 @@
 import { useRef, createContext, useState, useContext, useEffect, useCallback, useMemo, type ReactNode } from "react";
 type ModelStatus = "ready" | "starting" | "stopping" | "stopped" | "shutdown" | "unknown";
 const LOG_LENGTH_LIMIT = 1024 * 100; /* 100KB of log data */
 export interface Model {
  id: string;
  state: ModelStatus;
 }
 interface APIProviderType {
  models: Model[];
  listModels: () => Promise<Model[]>;
  unloadAllModels: () => Promise<void>;
  enableProxyLogs: (enabled: boolean) => void;
  enableUpstreamLogs: (enabled: boolean) => void;
  enableModelUpdates: (enabled: boolean) => void;
  proxyLogs: string;
  upstreamLogs: string;
 }
 const APIContext = createContext<APIProviderType | undefined>(undefined);
 type APIProviderProps = {
  children: ReactNode;
 };
 export function APIProvider({ children }: APIProviderProps) {
  const [proxyLogs, setProxyLogs] = useState("");
  const [upstreamLogs, setUpstreamLogs] = useState("");
  const proxyEventSource = useRef<EventSource | null>(null);
  const upstreamEventSource = useRef<EventSource | null>(null);
  const [models, setModels] = useState<Model[]>([]);
  const modelStatusEventSource = useRef<EventSource | null>(null);
  const appendLog = useCallback((newData: string, setter: React.Dispatch<React.SetStateAction<string>>) => {
    setter((prev) => {
      const updatedLog = prev + newData;
      return updatedLog.length > LOG_LENGTH_LIMIT ? updatedLog.slice(-LOG_LENGTH_LIMIT) : updatedLog;
    });
  }, []);
  const handleProxyMessage = useCallback(
    (e: MessageEvent) => {
      appendLog(e.data, setProxyLogs);
    },
    [proxyLogs, appendLog]
  );
  const handleUpstreamMessage = useCallback(
    (e: MessageEvent) => {
      appendLog(e.data, setUpstreamLogs);
    },
    [appendLog]
  );
  const enableProxyLogs = useCallback(
    (enabled: boolean) => {
      if (enabled) {
        const eventSource = new EventSource("/logs/streamSSE/proxy");
        eventSource.onmessage = handleProxyMessage;
        proxyEventSource.current = eventSource;
      } else {
        proxyEventSource.current?.close();
        proxyEventSource.current = null;
      }
    },
    [handleProxyMessage]
  );
  const enableUpstreamLogs = useCallback(
    (enabled: boolean) => {
      if (enabled) {
        const eventSource = new EventSource("/logs/streamSSE/upstream");
        eventSource.onmessage = handleUpstreamMessage;
        upstreamEventSource.current = eventSource;
      } else {
        upstreamEventSource.current?.close();
        upstreamEventSource.current = null;
      }
    },
    [upstreamEventSource, handleUpstreamMessage]
  );
  const enableModelUpdates = useCallback(
    (enabled: boolean) => {
      if (enabled) {
        const eventSource = new EventSource("/api/modelsSSE");
        eventSource.onmessage = (e: MessageEvent) => {
          try {
            const models = JSON.parse(e.data) as Model[];
            setModels(models);
          } catch (e) {
            console.error(e);
          }
        };
        modelStatusEventSource.current = eventSource;
      } else {
        modelStatusEventSource.current?.close();
        modelStatusEventSource.current = null;
      }
    },
    [setModels]
  );
  useEffect(() => {
    return () => {
      proxyEventSource.current?.close();
      upstreamEventSource.current?.close();
      modelStatusEventSource.current?.close();
    };
  }, []);
  const listModels = useCallback(async (): Promise<Model[]> => {
    try {
      const response = await fetch("/api/models/");
      if (!response.ok) {
        throw new Error(`HTTP error! status: ${response.status}`);
      }
      const data = await response.json();
      return data || [];
    } catch (error) {
      console.error("Failed to fetch models:", error);
      return []; // Return empty array as fallback
    }
  }, []);
  const unloadAllModels = useCallback(async () => {
    try {
      const response = await fetch(`/api/models/unload/`, {
        method: "POST",
      });
      if (!response.ok) {
        throw new Error(`Failed to unload models: ${response.status}`);
      }
    } catch (error) {
      console.error("Failed to unload models:", error);
      throw error; // Re-throw to let calling code handle it
    }
  }, []);
  const value = useMemo(
    () => ({
      models,
      listModels,
      unloadAllModels,
      enableProxyLogs,
      enableUpstreamLogs,
      enableModelUpdates,
      proxyLogs,
      upstreamLogs,
    }),
    [
      models,
      listModels,
      unloadAllModels,
      enableProxyLogs,
      enableUpstreamLogs,
      enableModelUpdates,
      proxyLogs,
      upstreamLogs,
    ]
  );
  return <APIContext.Provider value={value}>{children}</APIContext.Provider>;
 }
 export function useAPI() {
  const context = useContext(APIContext);
  if (context === undefined) {
    throw new Error("useAPI must be used within an APIProvider");
  }
  return context;
 }
@@ -0,0 +1,33 @@
 import { createContext, useContext, useEffect, type ReactNode } from "react";
 import { usePersistentState } from "../hooks/usePersistentState";
 type ThemeContextType = {
  isDarkMode: boolean;
  toggleTheme: () => void;
 };
 const ThemeContext = createContext<ThemeContextType | undefined>(undefined);
 type ThemeProviderProps = {
  children: ReactNode;
 };
 export function ThemeProvider({ children }: ThemeProviderProps) {
  const [isDarkMode, setIsDarkMode] = usePersistentState<boolean>("theme", false);
  useEffect(() => {
    document.documentElement.setAttribute("data-theme", isDarkMode ? "dark" : "light");
  }, [isDarkMode]);
  const toggleTheme = () => setIsDarkMode((prev) => !prev);
  return <ThemeContext.Provider value={{ isDarkMode, toggleTheme }}>{children}</ThemeContext.Provider>;
 }
 export function useTheme(): ThemeContextType {
  const context = useContext(ThemeContext);
  if (context === undefined) {
    throw new Error("useTheme must be used within a ThemeProvider");
  }
  return context;
 }
@@ -0,0 +1,39 @@
 import { useState, useEffect, useCallback } from "react";
 export function usePersistentState<T>(key: string, initialValue: T): [T, (value: T | ((prevState: T) => T)) => void] {
  const [state, setState] = useState<T>(() => {
    if (typeof window === "undefined") return initialValue;
    try {
      const saved = localStorage.getItem(key);
      return saved !== null ? JSON.parse(saved) : initialValue;
    } catch (e) {
      console.error(`Error parsing stored value for ${key}`, e);
      return initialValue;
    }
  });
  const setPersistentState = useCallback(
    (value: T | ((prevState: T) => T)) => {
      setState((prev) => {
        const nextValue = typeof value === "function" ? (value as (prevState: T) => T)(prev) : value;
        try {
          localStorage.setItem(key, JSON.stringify(nextValue));
        } catch (e) {
          console.error(`Error saving value for ${key}`, e);
        }
        return nextValue;
      });
    },
    [key]
  );
  useEffect(() => {
    try {
      localStorage.setItem(key, JSON.stringify(state));
    } catch (e) {
      console.error(`Error saving value for ${key}`, e);
    }
  }, [key, state]);
  return [state, setPersistentState];
 }
@@ -0,0 +1,164 @@
@import "tailwindcss";
@custom-variant dark (&:where([data-theme=dark], [data-theme=dark] *));
@theme {
  --color-background: rgba(252, 252, 249, 1);
  --color-surface: rgba(255, 255, 253, 1);
  /* text colors */
  --color-txtmain: rgba(19, 52, 59, 1);
  --color-txtsecondary: rgba(98, 108, 113, 1);
  --color-navlink-active: rgba(245, 245, 245, 1);
  --color-primary: rgba(50, 184, 198, 1);
  --color-primary-hover: rgba(29, 116, 128, 1);
  --color-primary-active: rgba(26, 104, 115, 1);
  --color-secondary: rgba(94, 82, 64, 0.12);
  --color-secondary-hover: rgba(94, 82, 64, 0.2);
  --color-secondary-active: rgba(94, 82, 64, 0.25);
  --color-border: rgba(94, 82, 64, 0.3);
  --color-btn-primary-text: rgba(252, 252, 249, 1);
  --color-card-border: rgba(94, 82, 64, 0.12);
  --color-card-border-inner: rgba(94, 82, 64, 0.12);
  --color-error: rgba(192, 21, 47, 1);
  --color-success: rgba(33, 128, 141, 1);
  --color-warning: rgb(244, 155, 0);
  --color-info: rgba(98, 108, 113, 1);
  --color-focus-ring: rgba(33, 128, 141, 0.4);
  --color-select-caret: rgba(19, 52, 59, 0.8);
  --color-btn-border: rgba(94, 82, 64, 0.7);
 }
@layer theme {
  /* over ride theme for dark mode */
  [data-theme="dark"] {
    --color-background: rgba(31, 33, 33, 1);
    --color-surface: rgba(38, 40, 40, 1);
    /* text colors */
    --color-txtmain: rgba(245, 245, 245, 1);
    --color-txtsecondary: rgba(167, 169, 169, 0.7);
    --color-navlink-active: rgba(245, 245, 245, 1);
    --color-primary: rgba(33, 128, 141, 1);
    --color-primary-hover: rgba(45, 166, 178, 1);
    --color-primary-active: rgba(41, 150, 161, 1);
    --color-secondary: rgba(119, 124, 124, 0.15);
    --color-secondary-hover: rgba(119, 124, 124, 0.25);
    --color-secondary-active: rgba(119, 124, 124, 0.3);
    --color-border: rgba(119, 124, 124, 0.3);
    --color-error: rgba(255, 84, 89, 1);
    --color-success: rgba(50, 184, 198, 1);
    --color-warning: rgb(244, 155, 0);
    --color-info: rgba(167, 169, 169, 1);
    --color-focus-ring: rgba(50, 184, 198, 0.4);
    --color-btn-primary-text: rgba(19, 52, 59, 1);
    --color-card-border: rgba(119, 124, 124, 0.2);
    --color-card-border-inner: rgba(119, 124, 124, 0.15);
    --shadow-inset-sm: inset 0 1px 0 rgba(255, 255, 255, 0.1), inset 0 -1px 0 rgba(0, 0, 0, 0.15);
    --button-border-secondary: rgba(119, 124, 124, 0.2);
  }
 }
@layer base {
  body {
    /* example of how colors using theme colors*/
    @apply bg-background text-txtmain;
  }
  h1 {
    @apply text-4xl text-txtmain font-bold pb-4;
  }
  h2 {
    @apply text-3xl text-txtmain font-bold pb-4;
  }
  h3 {
    @apply text-2xl text-txtmain font-bold pb-4;
  }
  h4 {
    @apply text-xl text-txtmain font-bold pb-4;
  }
  h5 {
    @apply text-lg text-txtmain font-bold pb-4;
  }
  h6 {
    @apply text-base text-txtmain font-bold pb-4;
  }
 }
 /* define CSS classes here for specific types of components */
@layer components {
  .container {
    @apply px-4;
  }
  /* Navigation Header */
  .navlink {
    @apply text-txtsecondary hover:bg-secondary hover:text-txtmain rounded-lg p-2;
  }
  .navlink.active {
    @apply bg-primary text-navlink-active;
  }
  /* Card component */
  .card {
    @apply bg-surface rounded-lg border border-card-border shadow-sm overflow-hidden p-4;
  }
  .card:hover {
    @apply shadow-md;
  }
  .card__body {
    @apply p-4;
  }
  .card__header,
  .card__footer {
    @apply p-4 border-b border-card-border-inner;
  }
  /* Status Badges */
  .status {
    @apply inline-block px-2 py-1 text-xs font-medium rounded-full;
  }
  .status--ready {
    @apply bg-success/10 text-success;
  }
  .status--starting,
  .status--stopping {
    @apply bg-warning/10 text-warning;
  }
  .status--stopped {
    @apply bg-error/10 text-error;
  }
  /* Buttons */
  .btn {
    @apply bg-surface p-2 px-4 text-sm rounded-full border border-2 transition-colors duration-200 border-btn-border;
  }
  .btn--sm {
    @apply px-2 py-0.5 text-xs;
  }
  .btn:disabled {
    @apply opacity-50 cursor-not-allowed;
  }
 }
@layer utilities {
  .ml-2 {
    margin-left: 0.5rem;
  }
  .my-8 {
    margin-top: 2rem;
    margin-bottom: 2rem;
  }
 }
@@ -0,0 +1,13 @@
 import { StrictMode } from "react";
 import { createRoot } from "react-dom/client";
 import "./index.css";
 import App from "./App.tsx";
 import { ThemeProvider } from "./contexts/ThemeProvider";
 createRoot(document.getElementById("root")!).render(
  <StrictMode>
    <ThemeProvider>
      <App />
    </ThemeProvider>
  </StrictMode>
 );
@@ -0,0 +1,160 @@
 import { useState, useEffect, useRef, useMemo, useCallback } from "react";
 import { useAPI } from "../contexts/APIProvider";
 import { usePersistentState } from "../hooks/usePersistentState";
 const LogViewer = () => {
  const { proxyLogs, upstreamLogs, enableProxyLogs, enableUpstreamLogs } = useAPI();
  useEffect(() => {
    enableProxyLogs(true);
    enableUpstreamLogs(true);
    return () => {
      enableProxyLogs(false);
      enableUpstreamLogs(false);
    };
  }, []);
  return (
    <div className="flex flex-col gap-5">
      <LogPanel id="proxy" title="Proxy Logs" logData={proxyLogs} />
      <LogPanel id="upstream" title="Upstream Logs" logData={upstreamLogs} />
    </div>
  );
 };
 interface LogPanelProps {
  id: string;
  title: string;
  logData: string;
  className?: string;
 }
 export const LogPanel = ({ id, title, logData, className }: LogPanelProps) => {
  const [filterRegex, setFilterRegex] = useState("");
  const [panelState, setPanelState] = usePersistentState<"hide" | "small" | "max">(
    `logPanel-${id}-panelState`,
    "small"
  );
  const [fontSize, setFontSize] = usePersistentState<"xxs" | "xs" | "small" | "normal">(
    `logPanel-${id}-fontSize`,
    "normal"
  );
  const [wrapText, setTextWrap] = usePersistentState(`logPanel-${id}-wrapText`, false);
  const textWrapClass = useMemo(() => {
    return wrapText ? "whitespace-pre-wrap" : "whitespace-pre";
  }, [wrapText]);
  const toggleFontSize = useCallback(() => {
    setFontSize((prev) => {
      switch (prev) {
        case "xxs":
          return "xs";
        case "xs":
          return "small";
        case "small":
          return "normal";
        case "normal":
          return "xxs";
      }
    });
  }, []);
  const togglePanelState = useCallback(() => {
    setPanelState((prev) => {
      if (prev === "small") return "max";
      if (prev === "hide") return "small";
      return "hide";
    });
  }, []);
  const fontSizeClass = useMemo(() => {
    switch (fontSize) {
      case "xxs":
        return "text-[0.5rem]"; // 0.5rem (8px)
      case "xs":
        return "text-[0.75rem]"; // 0.75rem (12px)
      case "small":
        return "text-[0.875rem]"; // 0.875rem (14px)
      case "normal":
        return "text-base"; // 1rem (16px)
    }
  }, [fontSize]);
  const filteredLogs = useMemo(() => {
    if (!filterRegex) return logData;
    try {
      const regex = new RegExp(filterRegex, "i");
      const lines = logData.split("\n");
      const filtered = lines.filter((line) => regex.test(line));
      return filtered.join("\n");
    } catch (e) {
      return logData; // Return unfiltered if regex is invalid
    }
  }, [logData, filterRegex]);
  // auto scroll to bottom
  const preTagRef = useRef<HTMLPreElement>(null);
  useEffect(() => {
    if (!preTagRef.current) return;
    preTagRef.current.scrollTop = preTagRef.current.scrollHeight;
  }, [filteredLogs]);
  return (
    <div className={`bg-surface border border-border rounded-lg overflow-hidden flex flex-col ${className || ""}`}>
      <div className="p-4 border-b border-border bg-secondary">
        <div className="flex flex-col md:flex-row md:items-center md:justify-between gap-4">
          {/* Title - Always full width on mobile, normal on desktop */}
          <div className="w-full md:w-auto" onClick={togglePanelState}>
            <h3 className="m-0 text-lg">{title}</h3>
          </div>
          <div className="flex flex-col sm:flex-row gap-4 w-full md:w-auto">
            {/* Sizing Buttons - Stacks vertically on mobile */}
            <div className="flex flex-wrap gap-2">
              <button className="btn" onClick={togglePanelState}>
                size: {panelState}
              </button>
              <button className="btn" onClick={toggleFontSize}>
                font: {fontSize}
              </button>
              <button className="btn" onClick={() => setTextWrap((prev) => !prev)}>
                {wrapText ? "wrap" : "wrap off"}
              </button>
            </div>
            {/* Filtering Options - Full width on mobile, normal on desktop */}
            <div className="flex flex-1 min-w-0 gap-2">
              <input
                type="text"
                className="flex-1 min-w-[120px] text-sm border p-2 rounded"
                placeholder="Filter logs..."
                value={filterRegex}
                onChange={(e) => setFilterRegex(e.target.value)}
              />
              <button className="btn" onClick={() => setFilterRegex("")}>
                Clear
              </button>
            </div>
          </div>
        </div>
      </div>
      {panelState !== "hide" && (
        <div className="flex-1 bg-background font-mono text-sm leading-[1.4] p-3">
          <pre
            ref={preTagRef}
            className={`flex-1 p-4 overflow-y-auto whitespace-pre min-h-0 ${textWrapClass} ${fontSizeClass}`}
            style={{
              maxHeight: panelState === "max" ? "1500px" : "500px",
            }}
          >
            {filteredLogs}
          </pre>
        </div>
      )}
    </div>
  );
 };
 export default LogViewer;
@@ -0,0 +1,74 @@
 import { useState, useEffect, useCallback } from "react";
 import { useAPI } from "../contexts/APIProvider";
 import { LogPanel } from "./LogViewer";
 export default function ModelsPage() {
  const { models, enableModelUpdates, unloadAllModels, upstreamLogs, enableUpstreamLogs } = useAPI();
  const [isUnloading, setIsUnloading] = useState(false);
  useEffect(() => {
    enableModelUpdates(true);
    enableUpstreamLogs(true);
    return () => {
      enableModelUpdates(false);
      enableUpstreamLogs(false);
    };
  }, []);
  const handleUnloadAllModels = useCallback(async () => {
    setIsUnloading(true);
    try {
      await unloadAllModels();
    } catch (e) {
      console.error(e);
    } finally {
      // at least give it a second to show the unloading message
      setTimeout(() => {
        setIsUnloading(false);
      }, 1000);
    }
  }, []);
  return (
    <div className="h-screen">
      <div className="flex flex-col md:flex-row gap-4">
        {/* Left Column */}
        <div className="w-full md:w-1/2 flex items-top">
          <div className="card w-full">
            <h2 className="">Models</h2>
            <button className="btn" onClick={handleUnloadAllModels} disabled={isUnloading}>
              {isUnloading ? "Unloading..." : "Unload All Models"}
            </button>
            <table className="w-full mt-4">
              <thead>
                <tr className="border-b border-primary">
                  <th className="text-left p-2">Name</th>
                  <th className="text-left p-2">State</th>
                </tr>
              </thead>
              <tbody>
                {models.map((model) => (
                  <tr key={model.id} className="border-b hover:bg-secondary-hover border-border">
                    <td className="p-2">
                      <a href={`/upstream/${model.id}/`} className="underline" target="top">
                        {model.id}
                      </a>
                    </td>
                    <td className="p-2">
                      <span className={`status status--${model.state}`}>{model.state}</span>
                    </td>
                  </tr>
                ))}
              </tbody>
            </table>
          </div>
        </div>
        {/* Right Column */}
        <div className="w-full md:w-1/2  flex items-top">
          <LogPanel id="modelsupstream" title="Upstream Logs" logData={upstreamLogs} className="h-full" />
        </div>
      </div>
    </div>
  );
 }
@@ -0,0 +1 @@
 /// <reference types="vite/client" />
@@ -0,0 +1,27 @@
 {
  "compilerOptions": {
    "tsBuildInfoFile": "./node_modules/.tmp/tsconfig.app.tsbuildinfo",
    "target": "ES2020",
    "useDefineForClassFields": true,
    "lib": ["ES2020", "DOM", "DOM.Iterable"],
    "module": "ESNext",
    "skipLibCheck": true,
    /* Bundler mode */
    "moduleResolution": "bundler",
    "allowImportingTsExtensions": true,
    "verbatimModuleSyntax": true,
    "moduleDetection": "force",
    "noEmit": true,
    "jsx": "react-jsx",
    /* Linting */
    "strict": true,
    "noUnusedLocals": true,
    "noUnusedParameters": true,
    "erasableSyntaxOnly": true,
    "noFallthroughCasesInSwitch": true,
    "noUncheckedSideEffectImports": true
  },
  "include": ["src"]
 }
@@ -0,0 +1,7 @@
 {
  "files": [],
  "references": [
    { "path": "./tsconfig.app.json" },
    { "path": "./tsconfig.node.json" }
  ]
 }
@@ -0,0 +1,25 @@
 {
  "compilerOptions": {
    "tsBuildInfoFile": "./node_modules/.tmp/tsconfig.node.tsbuildinfo",
    "target": "ES2022",
    "lib": ["ES2023"],
    "module": "ESNext",
    "skipLibCheck": true,
    /* Bundler mode */
    "moduleResolution": "bundler",
    "allowImportingTsExtensions": true,
    "verbatimModuleSyntax": true,
    "moduleDetection": "force",
    "noEmit": true,
    /* Linting */
    "strict": true,
    "noUnusedLocals": true,
    "noUnusedParameters": true,
    "erasableSyntaxOnly": true,
    "noFallthroughCasesInSwitch": true,
    "noUncheckedSideEffectImports": true
  },
  "include": ["vite.config.ts"]
 }
@@ -0,0 +1,20 @@
 import { defineConfig } from "vite";
 import react from "@vitejs/plugin-react";
 import tailwindcss from "@tailwindcss/vite";
 // https://vite.dev/config/
 export default defineConfig({
  plugins: [react(), tailwindcss()],
  base: "/ui/",
  build: {
    outDir: "../proxy/ui_dist",
    assetsDir: "assets",
  },
  server: {
    proxy: {
      "/api": "http://localhost:8080", // Proxy API calls to Go backend during development
      "/logs": "http://localhost:8080",
      "/upstream": "http://localhost:8080",
    },
  },
 });
Author	SHA1	Message	Date
Benson Wong	49035e2e8e	Append custom env vars instead of replace in Process (#171 ) Append custom env vars instead of replace in Process (#168, #169) PR #162 refactored the default configuration code. This introduced a subtle bug where `env` became `[]string{}` instead of the default of `nil`. In golang, `exec.Cmd.Env == nil` means to use the "current process's environment". By setting it to `[]string{}` as a default the Process's environment was emptied out which caused an array of strange and difficult to troubleshoot behaviour. See issues #168 and #169 This commit changes the behaviour to append model configured environment variables to the default list rather than replace them.	2025-06-18 11:09:13 -07:00
Benson Wong	9963ae18bf	fix? deprecation warning in .goreleaser.yaml [skip-ci]	2025-06-18 07:49:33 -07:00
Benson Wong	2ae48c713b	add debug output for start command	2025-06-18 07:43:23 -07:00
Benson Wong	54c519e365	update Makefile to install ui deps	2025-06-17 09:54:01 -07:00
Benson Wong	3fce9ee0e9	Update README.md [skip ci]	2025-06-17 09:53:22 -07:00
Benson Wong	5899ae7966	Update README.md [skip ci]	2025-06-17 09:52:47 -07:00
Benson Wong	591a9cdf4d	update release.yml	2025-06-16 16:50:25 -07:00
Benson Wong	9a3c656738	New UI (#157 , #164 ) - Add a react UI to replace the plain HTML one. - Serve as a foundation for better GUI interactions	2025-06-16 16:45:19 -07:00
Benson Wong	75015f82ea	fix bug caused by macro replacement order (#166 ) User defined macros should be applied before checking for ${PORT} constraint in model.cmd and model.proxy.	2025-06-16 15:32:09 -07:00
Thammachart Chinvarapon	cc33b6c270	restore intel docker builds (#163 )	2025-06-16 11:13:49 -07:00
Benson Wong	4fa12a429c	Refactor all default config values into config.go (#162 ) - Move all default values into one place. - Update tests to be more cross platform	2025-06-15 12:32:00 -07:00
		`@@ -0,0 +1 @@`
							<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" class="iconify iconify--logos" width="35.93" height="32" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 228"><path fill="#00D8FF" d="M210.483 73.824a171.49 171.49 0 0 0-8.24-2.597c.465-1.9.893-3.777 1.273-5.621c6.238-30.281 2.16-54.676-11.769-62.708c-13.355-7.7-35.196.329-57.254 19.526a171.23 171.23 0 0 0-6.375 5.848a155.866 155.866 0 0 0-4.241-3.917C100.759 3.829 77.587-4.822 63.673 3.233C50.33 10.957 46.379 33.89 51.995 62.588a170.974 170.974 0 0 0 1.892 8.48c-3.28.932-6.445 1.924-9.474 2.98C17.309 83.498 0 98.307 0 113.668c0 15.865 18.582 31.778 46.812 41.427a145.52 145.52 0 0 0 6.921 2.165a167.467 167.467 0 0 0-2.01 9.138c-5.354 28.2-1.173 50.591 12.134 58.266c13.744 7.926 36.812-.22 59.273-19.855a145.567 145.567 0 0 0 5.342-4.923a168.064 168.064 0 0 0 6.92 6.314c21.758 18.722 43.246 26.282 56.54 18.586c13.731-7.949 18.194-32.003 12.4-61.268a145.016 145.016 0 0 0-1.535-6.842c1.62-.48 3.21-.974 4.76-1.488c29.348-9.723 48.443-25.443 48.443-41.52c0-15.417-17.868-30.326-45.517-39.844Zm-6.365 70.984c-1.4.463-2.836.91-4.3 1.345c-3.24-10.257-7.612-21.163-12.963-32.432c5.106-11 9.31-21.767 12.459-31.957c2.619.758 5.16 1.557 7.61 2.4c23.69 8.156 38.14 20.213 38.14 29.504c0 9.896-15.606 22.743-40.946 31.14Zm-10.514 20.834c2.562 12.94 2.927 24.64 1.23 33.787c-1.524 8.219-4.59 13.698-8.382 15.893c-8.067 4.67-25.32-1.4-43.927-17.412a156.726 156.726 0 0 1-6.437-5.87c7.214-7.889 14.423-17.06 21.459-27.246c12.376-1.098 24.068-2.894 34.671-5.345a134.17 134.17 0 0 1 1.386 6.193ZM87.276 214.515c-7.882 2.783-14.16 2.863-17.955.675c-8.075-4.657-11.432-22.636-6.853-46.752a156.923 156.923 0 0 1 1.869-8.499c10.486 2.32 22.093 3.988 34.498 4.994c7.084 9.967 14.501 19.128 21.976 27.15a134.668 134.668 0 0 1-4.877 4.492c-9.933 8.682-19.886 14.842-28.658 17.94ZM50.35 144.747c-12.483-4.267-22.792-9.812-29.858-15.863c-6.35-5.437-9.555-10.836-9.555-15.216c0-9.322 13.897-21.212 37.076-29.293c2.813-.98 5.757-1.905 8.812-2.773c3.204 10.42 7.406 21.315 12.477 32.332c-5.137 11.18-9.399 22.249-12.634 32.792a134.718 134.718 0 0 1-6.318-1.979Zm12.378-84.26c-4.811-24.587-1.616-43.134 6.425-47.789c8.564-4.958 27.502 2.111 47.463 19.835a144.318 144.318 0 0 1 3.841 3.545c-7.438 7.987-14.787 17.08-21.808 26.988c-12.04 1.116-23.565 2.908-34.161 5.309a160.342 160.342 0 0 1-1.76-7.887Zm110.427 27.268a347.8 347.8 0 0 0-7.785-12.803c8.168 1.033 15.994 2.404 23.343 4.08c-2.206 7.072-4.956 14.465-8.193 22.045a381.151 381.151 0 0 0-7.365-13.322Zm-45.032-43.861c5.044 5.465 10.096 11.566 15.065 18.186a322.04 322.04 0 0 0-30.257-.006c4.974-6.559 10.069-12.652 15.192-18.18ZM82.802 87.83a323.167 323.167 0 0 0-7.227 13.238c-3.184-7.553-5.909-14.98-8.134-22.152c7.304-1.634 15.093-2.97 23.209-3.984a321.524 321.524 0 0 0-7.848 12.897Zm8.081 65.352c-8.385-.936-16.291-2.203-23.593-3.793c2.26-7.3 5.045-14.885 8.298-22.6a321.187 321.187 0 0 0 7.257 13.246c2.594 4.48 5.28 8.868 8.038 13.147Zm37.542 31.03c-5.184-5.592-10.354-11.779-15.403-18.433c4.902.192 9.899.29 14.978.29c5.218 0 10.376-.117 15.453-.343c-4.985 6.774-10.018 12.97-15.028 18.486Zm52.198-57.817c3.422 7.8 6.306 15.345 8.596 22.52c-7.422 1.694-15.436 3.058-23.88 4.071a382.417 382.417 0 0 0 7.859-13.026a347.403 347.403 0 0 0 7.425-13.565Zm-16.898 8.101a358.557 358.557 0 0 1-12.281 19.815a329.4 329.4 0 0 1-23.444.823c-7.967 0-15.716-.248-23.178-.732a310.202 310.202 0 0 1-12.513-19.846h.001a307.41 307.41 0 0 1-10.923-20.627a310.278 310.278 0 0 1 10.89-20.637l-.001.001a307.318 307.318 0 0 1 12.413-19.761c7.613-.576 15.42-.876 23.31-.876H128c7.926 0 15.743.303 23.354.883a329.357 329.357 0 0 1 12.335 19.695a358.489 358.489 0 0 1 11.036 20.54a329.472 329.472 0 0 1-11 20.722Zm22.56-122.124c8.572 4.944 11.906 24.881 6.52 51.026c-.344 1.668-.73 3.367-1.15 5.09c-10.622-2.452-22.155-4.275-34.23-5.408c-7.034-10.017-14.323-19.124-21.64-27.008a160.789 160.789 0 0 1 5.888-5.4c18.9-16.447 36.564-22.941 44.612-18.3ZM128 90.808c12.625 0 22.86 10.235 22.86 22.86s-10.235 22.86-22.86 22.86s-22.86-10.235-22.86-22.86s10.235-22.86 22.86-22.86Z"></path></svg>