Seed-packet capture: vision model, extraction, catalog match, create (backend) (#81)
Photograph a seed packet → it fills in the plant and the purchase. This is the backend; the scan UI is a follow-up PR. Vision model config (mirrors the agent model from #79): - Migration 0011 adds instance_settings.vision_model; PANSY_VISION_MODEL is the env default. Precedence Settings → env → empty; the KEY stays in the env. - EffectiveVision resolves it; /capabilities advertises "vision" only when a model + key are configured, so the UI offers the scan button only when it works. Extraction is one-shot, NOT an agent loop (internal/vision): - majordomo.Generate[SeedPacket] derives a JSON schema from the struct tags and hands the image to the vision model; it can't call a tool, so it can't touch the garden — it only reads a picture and returns data. Numeric fields are pointers, so a field the packet doesn't print comes back nil, not a made-up 0. - Hermetic test: majordomo's fake provider returns canned packet JSON and Generate unmarshals it, image + derived schema included. No live model. The image is normalized to JPEG at the upload boundary (imagenorm from #80), which is where an iPhone HEIC becomes readable — majordomo's media path can't decode HEIC. imagenorm now links into the binary (~7 MB, the cost #80 deferred). The hard part is catalog matching, not OCR (internal/service/seed_packet.go): - A wrong auto-match splits a variety's seed-lot history across duplicate rows, so the service NEVER auto-creates. matchPlants surfaces RANKED candidates (exact name → variety-in-name → same species, conservative and name-based), the user confirms, and CreateFromPacket makes the plant (new or existing) + the lot. Exactly one of plantId/newPlant, refused otherwise. - Plants/lots aren't in the undo history (catalog/inventory), so no change set. - The extractor is injectable (service.WithPacketExtractor) so ExtractSeedPacket and the /scan endpoint test end to end against a fake, no live model. Endpoints: POST /seed-lots/scan (multipart image → proposal, reads only; extends the read deadline for a slow phone upload, caps the body, maps too-large/unreadable to clear statuses) and POST /seed-lots/from-packet (confirmed proposal → 201). Docs: README (PANSY_VISION_MODEL), DESIGN (routes + the decision and why the model can't touch the garden). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01H3zbym8Doka2d7D48maSgZ
This commit is contained in:
@@ -0,0 +1,218 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"context"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"gitea.stevedudenhoeffer.com/steve/pansy/internal/domain"
|
||||
"gitea.stevedudenhoeffer.com/steve/pansy/internal/vision"
|
||||
)
|
||||
|
||||
// Seed-packet capture (#81): read a photographed packet, propose a plant + lot,
|
||||
// let the user confirm. The two halves are deliberately separate operations:
|
||||
// extraction only READS (a picture in, a proposal out — it can't touch the
|
||||
// garden), and creation happens later, from the confirmed proposal, so a
|
||||
// misread never writes anything on its own.
|
||||
|
||||
// PacketPlantMatch is a candidate existing plant the packet might be, with why it
|
||||
// matched, so the UI can pre-select the likely one and let the user override.
|
||||
type PacketPlantMatch struct {
|
||||
Plant domain.Plant `json:"plant"`
|
||||
// Reason is a short human tag: "exact name", "variety in name", "same species".
|
||||
Reason string `json:"reason"`
|
||||
}
|
||||
|
||||
// PacketProposal is what a scan returns: the fields read off the packet, plus
|
||||
// candidate existing plants it might already be. Nothing is created yet.
|
||||
type PacketProposal struct {
|
||||
Packet vision.SeedPacket `json:"packet"`
|
||||
// Candidates are existing plants the packet may match, best first. Empty means
|
||||
// "probably a new variety" — the UI then offers to create one.
|
||||
Candidates []PacketPlantMatch `json:"candidates"`
|
||||
// SuggestedName is the variety (or species) to prefill a new-plant name with.
|
||||
SuggestedName string `json:"suggestedName"`
|
||||
// SuggestedCategory is the packet's category if it's a valid one, for prefill.
|
||||
SuggestedCategory string `json:"suggestedCategory"`
|
||||
}
|
||||
|
||||
// ExtractSeedPacket reads a (JPEG) packet photo and proposes a plant + lot for
|
||||
// the actor to confirm. It needs a configured vision model; with none it returns
|
||||
// ErrInvalidInput (the API layer turns "not configured" into a clear message and
|
||||
// never offers the feature in the first place).
|
||||
//
|
||||
// The extraction runs as the actor only in the sense that the catalog match is
|
||||
// scoped to what they can see; the model call itself has no ACL — it just reads
|
||||
// a picture the actor uploaded.
|
||||
func (s *Service) ExtractSeedPacket(ctx context.Context, actorID int64, jpeg []byte) (*PacketProposal, error) {
|
||||
if len(jpeg) == 0 {
|
||||
return nil, domain.ErrInvalidInput
|
||||
}
|
||||
vis, err := s.EffectiveVision(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if !vis.Ready() {
|
||||
// No vision model configured — the feature isn't available.
|
||||
return nil, domain.ErrInvalidInput
|
||||
}
|
||||
|
||||
packet, err := s.extractPacket(ctx, vis.APIKey, vis.Model, jpeg)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
plants, err := s.store.ListPlantsForActor(ctx, actorID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &PacketProposal{
|
||||
Packet: packet,
|
||||
Candidates: matchPlants(packet, plants),
|
||||
SuggestedName: suggestedName(packet),
|
||||
SuggestedCategory: validCategory(packet.Category),
|
||||
}, nil
|
||||
}
|
||||
|
||||
// suggestedName is the variety if the packet named one, else the species — what
|
||||
// to prefill a new plant's name with. "Music" beats "garlic" when both are read.
|
||||
func suggestedName(p vision.SeedPacket) string {
|
||||
if v := strings.TrimSpace(p.Variety); v != "" {
|
||||
return v
|
||||
}
|
||||
return strings.TrimSpace(p.Species)
|
||||
}
|
||||
|
||||
// validCategory returns the packet's category if it's one pansy knows, else "".
|
||||
func validCategory(c string) string {
|
||||
switch c {
|
||||
case domain.CategoryVegetable, domain.CategoryHerb, domain.CategoryFlower,
|
||||
domain.CategoryFruit, domain.CategoryTreeShrub, domain.CategoryCover:
|
||||
return c
|
||||
default:
|
||||
return ""
|
||||
}
|
||||
}
|
||||
|
||||
// matchPlants ranks existing plants the packet might already be, best first.
|
||||
//
|
||||
// This is the crux of "create both, linked" (#81): getting it wrong makes a
|
||||
// duplicate catalog entry that then splits a variety's seed-lot history across
|
||||
// two rows. So it NEVER decides — it only surfaces candidates for the user to
|
||||
// confirm. Matching is deliberately conservative and name-based (no fuzzy
|
||||
// scoring that could confidently mis-rank): exact variety name, variety appearing
|
||||
// within a plant's name, then same species word. Case-insensitive.
|
||||
func matchPlants(p vision.SeedPacket, plants []domain.Plant) []PacketPlantMatch {
|
||||
variety := strings.ToLower(strings.TrimSpace(p.Variety))
|
||||
species := strings.ToLower(strings.TrimSpace(p.Species))
|
||||
|
||||
// rank: lower is better; keep only matched plants.
|
||||
type scored struct {
|
||||
match PacketPlantMatch
|
||||
rank int
|
||||
}
|
||||
var out []scored
|
||||
seen := map[int64]bool{}
|
||||
add := func(pl domain.Plant, rank int, reason string) {
|
||||
if seen[pl.ID] {
|
||||
return
|
||||
}
|
||||
seen[pl.ID] = true
|
||||
out = append(out, scored{PacketPlantMatch{Plant: pl, Reason: reason}, rank})
|
||||
}
|
||||
|
||||
for _, pl := range plants {
|
||||
name := strings.ToLower(pl.Name)
|
||||
switch {
|
||||
case variety != "" && name == variety:
|
||||
add(pl, 0, "exact name")
|
||||
case variety != "" && strings.Contains(name, variety):
|
||||
add(pl, 1, "variety in name")
|
||||
case species != "" && wordIn(name, species):
|
||||
add(pl, 2, "same species")
|
||||
}
|
||||
}
|
||||
sort.SliceStable(out, func(i, j int) bool { return out[i].rank < out[j].rank })
|
||||
|
||||
matches := make([]PacketPlantMatch, len(out))
|
||||
for i, s := range out {
|
||||
matches[i] = s.match
|
||||
}
|
||||
return matches
|
||||
}
|
||||
|
||||
// wordIn reports whether word appears as a whole space-delimited token in name,
|
||||
// so "garlic" matches "German Garlic" but not "garlicky-thing".
|
||||
func wordIn(name, word string) bool {
|
||||
for _, tok := range strings.Fields(name) {
|
||||
if tok == word {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// PacketConfirm is a user-confirmed proposal to turn into rows.
|
||||
//
|
||||
// Plant selection is explicit: either PlantID names an existing plant to attach
|
||||
// the lot to, or NewPlant carries the fields to create one. Exactly one — the
|
||||
// service refuses both or neither, so an ambiguous confirm can't silently pick.
|
||||
type PacketConfirm struct {
|
||||
// PlantID attaches the lot to an existing plant. Set this XOR NewPlant.
|
||||
PlantID *int64
|
||||
// NewPlant creates a variety. Set this XOR PlantID.
|
||||
NewPlant *PlantInput
|
||||
// Lot is the purchase to record against whichever plant results.
|
||||
Lot SeedLotInput
|
||||
}
|
||||
|
||||
// PacketResult is what a confirm produced.
|
||||
type PacketResult struct {
|
||||
Plant *domain.Plant `json:"plant"`
|
||||
Lot *domain.SeedLot `json:"lot"`
|
||||
PlantIsNew bool `json:"plantIsNew"`
|
||||
}
|
||||
|
||||
// CreateFromPacket turns a confirmed proposal into a plant (new or existing) plus
|
||||
// a seed lot attributed to it. Unlike garden edits these rows aren't in the undo
|
||||
// history — plants and lots are catalog/inventory, created directly — so there is
|
||||
// no change set to wrap; the two creations just happen in sequence.
|
||||
//
|
||||
// If a new plant is created but the lot then fails, the plant is left behind
|
||||
// rather than rolled back: a stray catalog entry is harmless and editable,
|
||||
// whereas silently discarding a variety the user just confirmed is worse. The
|
||||
// caller sees the error and can retry the lot against the now-existing plant.
|
||||
func (s *Service) CreateFromPacket(ctx context.Context, actorID int64, in PacketConfirm) (*PacketResult, error) {
|
||||
hasID, hasNew := in.PlantID != nil, in.NewPlant != nil
|
||||
if hasID == hasNew {
|
||||
return nil, domain.ErrInvalidInput // exactly one of existing / new
|
||||
}
|
||||
|
||||
res := &PacketResult{}
|
||||
if hasNew {
|
||||
plant, err := s.CreatePlant(ctx, actorID, *in.NewPlant)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
res.Plant = plant
|
||||
res.PlantIsNew = true
|
||||
} else {
|
||||
// Attach to an existing plant the actor can see. visiblePlant enforces the
|
||||
// ACL (built-ins + their own); a plant they can't see is ErrNotFound.
|
||||
plant, err := s.visiblePlant(ctx, actorID, *in.PlantID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
res.Plant = plant
|
||||
}
|
||||
|
||||
lotIn := in.Lot
|
||||
lotIn.PlantID = res.Plant.ID
|
||||
lot, err := s.CreateSeedLot(ctx, actorID, lotIn)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
res.Lot = lot
|
||||
return res, nil
|
||||
}
|
||||
Reference in New Issue
Block a user