43 lines
1.7 KiB
Markdown
43 lines
1.7 KiB
Markdown
# ADR-0008: Durable SQLite-backed queue
|
|
|
|
**Status:** Accepted — 2026-05-23
|
|
|
|
## Context
|
|
|
|
Jobs are queued, carry state, and may be retried across target sleep/restart. A
|
|
caller that submitted an async job and is waiting on a webhook must not lose its
|
|
job because foreman restarted. State must survive process restarts.
|
|
|
|
## Decision
|
|
|
|
The job queue and all job state (including artifacts, ADR-0006) live in **SQLite**
|
|
in WAL mode, via the pure-Go `modernc.org/sqlite` driver (no CGO, so the Komodo
|
|
container build stays trivial).
|
|
|
|
### Schema sketch
|
|
|
|
- `jobs(id TEXT PK, state TEXT, model TEXT, request BLOB, result BLOB,
|
|
error TEXT, webhook_url TEXT, attempt INT, created_at, updated_at, …)`
|
|
- `artifacts(job_id TEXT, name TEXT, content_type TEXT, size INT, inline BLOB,
|
|
PRIMARY KEY(job_id, name))`
|
|
|
|
A single writer (the worker, ADR-0009) plus the HTTP handlers; WAL handles the
|
|
concurrent-reader / single-writer pattern well at this scale.
|
|
|
|
## Consequences
|
|
|
|
- Jobs and results are durable across restarts; webhook recovery via
|
|
`GET /jobs/{id}` (ADR-0004) is meaningful.
|
|
- Pure-Go driver keeps cross-compilation and container builds painless.
|
|
- Pruning is a TTL sweep over `jobs`/`artifacts`; no external store to operate.
|
|
- SQLite caps practical artifact size at single-digit MB — acceptable per ADR-0006
|
|
thresholds; revisit if outputs grow.
|
|
|
|
## Alternatives considered
|
|
|
|
- **In-memory queue.** Loses async jobs on restart; unacceptable given webhooks.
|
|
- **Redis / external broker.** Another moving part to run for a single-worker
|
|
daemon; over-engineered. Rejected.
|
|
- **`mattn/go-sqlite3` (CGO).** Faster in some cases but complicates static builds
|
|
and container images. Pure-Go preferred for ops simplicity.
|