Seed-packet capture: vision model, extraction, catalog match, create (backend) (#81)
Photograph a seed packet → it fills in the plant and the purchase. This is the backend; the scan UI is a follow-up PR. Vision model config (mirrors the agent model from #79): - Migration 0011 adds instance_settings.vision_model; PANSY_VISION_MODEL is the env default. Precedence Settings → env → empty; the KEY stays in the env. - EffectiveVision resolves it; /capabilities advertises "vision" only when a model + key are configured, so the UI offers the scan button only when it works. Extraction is one-shot, NOT an agent loop (internal/vision): - majordomo.Generate[SeedPacket] derives a JSON schema from the struct tags and hands the image to the vision model; it can't call a tool, so it can't touch the garden — it only reads a picture and returns data. Numeric fields are pointers, so a field the packet doesn't print comes back nil, not a made-up 0. - Hermetic test: majordomo's fake provider returns canned packet JSON and Generate unmarshals it, image + derived schema included. No live model. The image is normalized to JPEG at the upload boundary (imagenorm from #80), which is where an iPhone HEIC becomes readable — majordomo's media path can't decode HEIC. imagenorm now links into the binary (~7 MB, the cost #80 deferred). The hard part is catalog matching, not OCR (internal/service/seed_packet.go): - A wrong auto-match splits a variety's seed-lot history across duplicate rows, so the service NEVER auto-creates. matchPlants surfaces RANKED candidates (exact name → variety-in-name → same species, conservative and name-based), the user confirms, and CreateFromPacket makes the plant (new or existing) + the lot. Exactly one of plantId/newPlant, refused otherwise. - Plants/lots aren't in the undo history (catalog/inventory), so no change set. - The extractor is injectable (service.WithPacketExtractor) so ExtractSeedPacket and the /scan endpoint test end to end against a fake, no live model. Endpoints: POST /seed-lots/scan (multipart image → proposal, reads only; extends the read deadline for a slow phone upload, caps the body, maps too-large/unreadable to clear statuses) and POST /seed-lots/from-packet (confirmed proposal → 201). Docs: README (PANSY_VISION_MODEL), DESIGN (routes + the decision and why the model can't touch the garden). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01H3zbym8Doka2d7D48maSgZ
This commit is contained in:
@@ -12,7 +12,7 @@ import (
|
||||
// The instance_settings row is seeded by migration 0010 and there is exactly one
|
||||
// (CHECK id = 1), so reads never branch on existence and writes never insert.
|
||||
|
||||
const instanceSettingsColumns = `agent_model, agent_enabled, version, updated_at`
|
||||
const instanceSettingsColumns = `agent_model, agent_enabled, vision_model, version, updated_at`
|
||||
|
||||
// scanInstanceSettings reads the single settings row. agent_enabled is a nullable
|
||||
// INTEGER (NULL = inherit env), so it is scanned through sql.NullInt64.
|
||||
@@ -21,7 +21,7 @@ func scanInstanceSettings(s scanner) (*domain.InstanceSettings, error) {
|
||||
out domain.InstanceSettings
|
||||
enabled sql.NullInt64
|
||||
)
|
||||
if err := s.Scan(&out.AgentModel, &enabled, &out.Version, &out.UpdatedAt); err != nil {
|
||||
if err := s.Scan(&out.AgentModel, &enabled, &out.VisionModel, &out.Version, &out.UpdatedAt); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if enabled.Valid {
|
||||
@@ -60,12 +60,12 @@ func (d *DB) UpdateInstanceSettings(ctx context.Context, s *domain.InstanceSetti
|
||||
}
|
||||
updated, err := scanInstanceSettings(d.sql.QueryRowContext(ctx,
|
||||
`UPDATE instance_settings
|
||||
SET agent_model = ?, agent_enabled = ?,
|
||||
SET agent_model = ?, agent_enabled = ?, vision_model = ?,
|
||||
version = version + 1,
|
||||
updated_at = strftime('%Y-%m-%dT%H:%M:%SZ', 'now')
|
||||
WHERE id = 1 AND version = ?
|
||||
RETURNING `+instanceSettingsColumns,
|
||||
s.AgentModel, enabled, s.Version,
|
||||
s.AgentModel, enabled, s.VisionModel, s.Version,
|
||||
))
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
current, gerr := d.GetInstanceSettings(ctx)
|
||||
|
||||
@@ -0,0 +1,10 @@
|
||||
-- Vision model setting (#81): the model that reads a photographed seed packet.
|
||||
--
|
||||
-- Separate from agent_model because it's a different capability — extracting
|
||||
-- structured fields from an image needs a vision-capable model, which the chat
|
||||
-- model may not be. Same "inherit from env unless set" contract as agent_model:
|
||||
-- '' falls back to PANSY_VISION_MODEL, then the feature is simply not offered.
|
||||
--
|
||||
-- Like agent_model, the API KEY is NOT stored here — the vision model runs
|
||||
-- against the same OLLAMA_CLOUD_API_KEY from the environment.
|
||||
ALTER TABLE instance_settings ADD COLUMN vision_model TEXT NOT NULL DEFAULT '';
|
||||
Reference in New Issue
Block a user