ADR-0008: Durable SQLite-backed queue

Status: Accepted — 2026-05-23

Context

Jobs are queued, carry state, and may be retried across target sleep/restart. A caller that submitted an async job and is waiting on a webhook must not lose its job because foreman restarted. State must survive process restarts.

Decision

The job queue and all job state (including artifacts, ADR-0006) live in SQLite in WAL mode, via the pure-Go modernc.org/sqlite driver (no CGO, so the Komodo container build stays trivial).

Schema sketch

jobs(id TEXT PK, state TEXT, model TEXT, request BLOB, result BLOB, error TEXT, webhook_url TEXT, attempt INT, created_at, updated_at, …)
artifacts(job_id TEXT, name TEXT, content_type TEXT, size INT, inline BLOB, PRIMARY KEY(job_id, name))

A single writer (the worker, ADR-0009) plus the HTTP handlers; WAL handles the concurrent-reader / single-writer pattern well at this scale.

Consequences

Jobs and results are durable across restarts; webhook recovery via GET /jobs/{id} (ADR-0004) is meaningful.
Pure-Go driver keeps cross-compilation and container builds painless.
Pruning is a TTL sweep over jobs/artifacts; no external store to operate.
SQLite caps practical artifact size at single-digit MB — acceptable per ADR-0006 thresholds; revisit if outputs grow.

Alternatives considered

In-memory queue. Loses async jobs on restart; unacceptable given webhooks.
Redis / external broker. Another moving part to run for a single-worker daemon; over-engineered. Rejected.
mattn/go-sqlite3 (CGO). Faster in some cases but complicates static builds and container images. Pure-Go preferred for ops simplicity.

1.7 KiB Raw Permalink Blame History