Seed-packet capture: vision model, extraction, catalog match, create (backend) (#81)
Photograph a seed packet → it fills in the plant and the purchase. This is the backend; the scan UI is a follow-up PR. Vision model config (mirrors the agent model from #79): - Migration 0011 adds instance_settings.vision_model; PANSY_VISION_MODEL is the env default. Precedence Settings → env → empty; the KEY stays in the env. - EffectiveVision resolves it; /capabilities advertises "vision" only when a model + key are configured, so the UI offers the scan button only when it works. Extraction is one-shot, NOT an agent loop (internal/vision): - majordomo.Generate[SeedPacket] derives a JSON schema from the struct tags and hands the image to the vision model; it can't call a tool, so it can't touch the garden — it only reads a picture and returns data. Numeric fields are pointers, so a field the packet doesn't print comes back nil, not a made-up 0. - Hermetic test: majordomo's fake provider returns canned packet JSON and Generate unmarshals it, image + derived schema included. No live model. The image is normalized to JPEG at the upload boundary (imagenorm from #80), which is where an iPhone HEIC becomes readable — majordomo's media path can't decode HEIC. imagenorm now links into the binary (~7 MB, the cost #80 deferred). The hard part is catalog matching, not OCR (internal/service/seed_packet.go): - A wrong auto-match splits a variety's seed-lot history across duplicate rows, so the service NEVER auto-creates. matchPlants surfaces RANKED candidates (exact name → variety-in-name → same species, conservative and name-based), the user confirms, and CreateFromPacket makes the plant (new or existing) + the lot. Exactly one of plantId/newPlant, refused otherwise. - Plants/lots aren't in the undo history (catalog/inventory), so no change set. - The extractor is injectable (service.WithPacketExtractor) so ExtractSeedPacket and the /scan endpoint test end to end against a fake, no live model. Endpoints: POST /seed-lots/scan (multipart image → proposal, reads only; extends the read deadline for a slow phone upload, caps the body, maps too-large/unreadable to clear statuses) and POST /seed-lots/from-packet (confirmed proposal → 201). Docs: README (PANSY_VISION_MODEL), DESIGN (routes + the decision and why the model can't touch the garden). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01H3zbym8Doka2d7D48maSgZ
This commit is contained in:
@@ -70,8 +70,9 @@ The garden assistant reads three more. Setting none of them leaves the assistant
|
||||
| `OLLAMA_CLOUD_API_KEY` | *(empty)* | Ollama Cloud API key. Without it the assistant is off, not broken. This is the one agent value that stays in the environment — it is **never** stored in the database or editable in Settings. |
|
||||
| `PANSY_AGENT_MODEL` | `ollama-cloud/glm-5.2:cloud` | Default model spec, passed verbatim to `majordomo.Parse` — a comma-separated list is a failover chain, e.g. `ollama-cloud/glm-5.2:cloud,ollama-cloud/kimi-k2.6:cloud`. An admin can override this per-instance in **Settings** without a redeploy; a blank Settings value inherits this. |
|
||||
| `PANSY_AGENT_ENABLED` | on when a key is present | Default on/off for the assistant. Also overridable in Settings (which can inherit this default). |
|
||||
| `PANSY_VISION_MODEL` | *(empty)* | Default model for **seed-packet capture** (photograph a packet → it fills in the plant + purchase). A *vision-capable* model (the chat model may not be). Empty = the feature isn't offered. Runs against the same `OLLAMA_CLOUD_API_KEY`, and is overridable in Settings. |
|
||||
|
||||
The model and enabled flag can be changed at runtime by an admin under **Settings** (the gear appears in the nav for admins) — the change swaps the live assistant with no restart. The env vars above are the defaults an untouched instance uses, and the API key is intentionally not among the runtime-editable settings: a secret in the database would land in every backup. Precedence for the model and enabled flag is **Settings value, if set → env var → built-in default**.
|
||||
The agent model + enabled flag, and the vision model, can be changed at runtime by an admin under **Settings** (the gear appears in the nav for admins) — an agent change swaps the live assistant with no restart. The env vars above are the defaults an untouched instance uses, and the API key is intentionally not among the runtime-editable settings: a secret in the database would land in every backup. Precedence is **Settings value, if set → env var → built-in default**.
|
||||
|
||||
The assistant acts without asking first, which is only reasonable because every turn is one undoable change set — see the History panel in the editor.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user