steve
27f196d333
Phase 2 of foreman: the daemon now acts as a transparent Ollama proxy. - internal/ollama: Client interface and HTTP implementation for chat (streaming + non-streaming), embed, tags, ps with auth forwarding, NDJSON streaming via bufio.Scanner, and connection vs HTTP error classification via custom error types. - internal/ollama: ModelInventory with background poller for /api/tags and /api/ps, degraded mode on target unreachable with model retention, automatic recovery on reconnect. - internal/server: Passthrough routes (/api/chat, /api/tags, /api/ps, /api/embed, /api/embeddings) with model validation, chat serialization gate (capacity-1 channel), concurrent embedding bypass (ADR-0013), NDJSON streaming with per-chunk flush, and degraded health reporting. - cmd/foreman: Full serve wiring with Ollama client, poller goroutine, embedder warmup (keep_alive:-1), and signal-based shutdown. The Mac is now usable as a go-llm target through foreman. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
foreman
A small, always-on Go daemon that fronts one Ollama target. It turns a single Ollama instance into a queued, observable job endpoint: it polls the target's installed models, serializes work through the target (managing model swaps), assigns every job an ID, and reports progress via webhooks.
On the wire it speaks native Ollama, so it doubles as a drop-in go-llm
target.
Quickstart
# Set the required Ollama target URL
export FOREMAN_OLLAMA_URL=http://mac.tail:11434
# Run directly
go run ./cmd/foreman serve
# Or build and run
go build -o foreman ./cmd/foreman
./foreman serve
Docker
docker build -t foreman .
docker run -e FOREMAN_OLLAMA_URL=http://mac.tail:11434 -p 8080:8080 foreman
Configuration
All configuration is via environment variables, namespaced under FOREMAN_*.
See .env.example for the full list.
| Variable | Default | Description |
|---|---|---|
FOREMAN_ADDR |
:8080 |
Listen address |
FOREMAN_OLLAMA_URL |
(required) | Ollama target base URL |
FOREMAN_OLLAMA_TOKEN |
(empty) | Bearer token sent to the target |
FOREMAN_TOKEN |
(empty) | Bearer token callers must present |
FOREMAN_EMBED_MODEL |
(empty) | Always-resident embedder model |
FOREMAN_DB_PATH |
foreman.db |
SQLite database path |
FOREMAN_POLL_INTERVAL |
30s |
Target model poll interval |
FOREMAN_WEBHOOK_SECRET |
(empty) | HMAC key for webhook signing |
Health check
curl http://localhost:8080/healthz
# {"status":"ok","degraded":false}
Architecture
See docs/adr/ for design decisions. Key points:
- One daemon per Ollama target (ADR-0001)
- SQLite-backed durable job queue in WAL mode (ADR-0008)
- Single worker loop with drain-by-model scheduling (ADR-0009)
- Native Ollama passthrough + async
/jobssurface (ADR-0003, ADR-0004) - Embeddings bypass the queue entirely (ADR-0013)