Files
pansy/internal/service/seed_packet.go
T
steveandClaude Opus 4.8 156b3fd14c
Build image / build-and-push (push) Successful in 7s
Gadfly review (reusable) / review (pull_request) Successful in 9m44s
Adversarial Review (Gadfly) / review (pull_request) Successful in 9m44s
Seed-packet capture: vision model, extraction, catalog match, create (backend) (#81)
Photograph a seed packet → it fills in the plant and the purchase. This is the
backend; the scan UI is a follow-up PR.

Vision model config (mirrors the agent model from #79):
- Migration 0011 adds instance_settings.vision_model; PANSY_VISION_MODEL is the
  env default. Precedence Settings → env → empty; the KEY stays in the env.
- EffectiveVision resolves it; /capabilities advertises "vision" only when a
  model + key are configured, so the UI offers the scan button only when it works.

Extraction is one-shot, NOT an agent loop (internal/vision):
- majordomo.Generate[SeedPacket] derives a JSON schema from the struct tags and
  hands the image to the vision model; it can't call a tool, so it can't touch
  the garden — it only reads a picture and returns data. Numeric fields are
  pointers, so a field the packet doesn't print comes back nil, not a made-up 0.
- Hermetic test: majordomo's fake provider returns canned packet JSON and
  Generate unmarshals it, image + derived schema included. No live model.

The image is normalized to JPEG at the upload boundary (imagenorm from #80),
which is where an iPhone HEIC becomes readable — majordomo's media path can't
decode HEIC. imagenorm now links into the binary (~7 MB, the cost #80 deferred).

The hard part is catalog matching, not OCR (internal/service/seed_packet.go):
- A wrong auto-match splits a variety's seed-lot history across duplicate rows,
  so the service NEVER auto-creates. matchPlants surfaces RANKED candidates
  (exact name → variety-in-name → same species, conservative and name-based),
  the user confirms, and CreateFromPacket makes the plant (new or existing) + the
  lot. Exactly one of plantId/newPlant, refused otherwise.
- Plants/lots aren't in the undo history (catalog/inventory), so no change set.
- The extractor is injectable (service.WithPacketExtractor) so ExtractSeedPacket
  and the /scan endpoint test end to end against a fake, no live model.

Endpoints: POST /seed-lots/scan (multipart image → proposal, reads only; extends
the read deadline for a slow phone upload, caps the body, maps too-large/unreadable
to clear statuses) and POST /seed-lots/from-packet (confirmed proposal → 201).

Docs: README (PANSY_VISION_MODEL), DESIGN (routes + the decision and why the
model can't touch the garden).

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01H3zbym8Doka2d7D48maSgZ
2026-07-21 23:50:50 -04:00

219 lines
7.5 KiB
Go

package service
import (
"context"
"sort"
"strings"
"gitea.stevedudenhoeffer.com/steve/pansy/internal/domain"
"gitea.stevedudenhoeffer.com/steve/pansy/internal/vision"
)
// Seed-packet capture (#81): read a photographed packet, propose a plant + lot,
// let the user confirm. The two halves are deliberately separate operations:
// extraction only READS (a picture in, a proposal out — it can't touch the
// garden), and creation happens later, from the confirmed proposal, so a
// misread never writes anything on its own.
// PacketPlantMatch is a candidate existing plant the packet might be, with why it
// matched, so the UI can pre-select the likely one and let the user override.
type PacketPlantMatch struct {
Plant domain.Plant `json:"plant"`
// Reason is a short human tag: "exact name", "variety in name", "same species".
Reason string `json:"reason"`
}
// PacketProposal is what a scan returns: the fields read off the packet, plus
// candidate existing plants it might already be. Nothing is created yet.
type PacketProposal struct {
Packet vision.SeedPacket `json:"packet"`
// Candidates are existing plants the packet may match, best first. Empty means
// "probably a new variety" — the UI then offers to create one.
Candidates []PacketPlantMatch `json:"candidates"`
// SuggestedName is the variety (or species) to prefill a new-plant name with.
SuggestedName string `json:"suggestedName"`
// SuggestedCategory is the packet's category if it's a valid one, for prefill.
SuggestedCategory string `json:"suggestedCategory"`
}
// ExtractSeedPacket reads a (JPEG) packet photo and proposes a plant + lot for
// the actor to confirm. It needs a configured vision model; with none it returns
// ErrInvalidInput (the API layer turns "not configured" into a clear message and
// never offers the feature in the first place).
//
// The extraction runs as the actor only in the sense that the catalog match is
// scoped to what they can see; the model call itself has no ACL — it just reads
// a picture the actor uploaded.
func (s *Service) ExtractSeedPacket(ctx context.Context, actorID int64, jpeg []byte) (*PacketProposal, error) {
if len(jpeg) == 0 {
return nil, domain.ErrInvalidInput
}
vis, err := s.EffectiveVision(ctx)
if err != nil {
return nil, err
}
if !vis.Ready() {
// No vision model configured — the feature isn't available.
return nil, domain.ErrInvalidInput
}
packet, err := s.extractPacket(ctx, vis.APIKey, vis.Model, jpeg)
if err != nil {
return nil, err
}
plants, err := s.store.ListPlantsForActor(ctx, actorID)
if err != nil {
return nil, err
}
return &PacketProposal{
Packet: packet,
Candidates: matchPlants(packet, plants),
SuggestedName: suggestedName(packet),
SuggestedCategory: validCategory(packet.Category),
}, nil
}
// suggestedName is the variety if the packet named one, else the species — what
// to prefill a new plant's name with. "Music" beats "garlic" when both are read.
func suggestedName(p vision.SeedPacket) string {
if v := strings.TrimSpace(p.Variety); v != "" {
return v
}
return strings.TrimSpace(p.Species)
}
// validCategory returns the packet's category if it's one pansy knows, else "".
func validCategory(c string) string {
switch c {
case domain.CategoryVegetable, domain.CategoryHerb, domain.CategoryFlower,
domain.CategoryFruit, domain.CategoryTreeShrub, domain.CategoryCover:
return c
default:
return ""
}
}
// matchPlants ranks existing plants the packet might already be, best first.
//
// This is the crux of "create both, linked" (#81): getting it wrong makes a
// duplicate catalog entry that then splits a variety's seed-lot history across
// two rows. So it NEVER decides — it only surfaces candidates for the user to
// confirm. Matching is deliberately conservative and name-based (no fuzzy
// scoring that could confidently mis-rank): exact variety name, variety appearing
// within a plant's name, then same species word. Case-insensitive.
func matchPlants(p vision.SeedPacket, plants []domain.Plant) []PacketPlantMatch {
variety := strings.ToLower(strings.TrimSpace(p.Variety))
species := strings.ToLower(strings.TrimSpace(p.Species))
// rank: lower is better; keep only matched plants.
type scored struct {
match PacketPlantMatch
rank int
}
var out []scored
seen := map[int64]bool{}
add := func(pl domain.Plant, rank int, reason string) {
if seen[pl.ID] {
return
}
seen[pl.ID] = true
out = append(out, scored{PacketPlantMatch{Plant: pl, Reason: reason}, rank})
}
for _, pl := range plants {
name := strings.ToLower(pl.Name)
switch {
case variety != "" && name == variety:
add(pl, 0, "exact name")
case variety != "" && strings.Contains(name, variety):
add(pl, 1, "variety in name")
case species != "" && wordIn(name, species):
add(pl, 2, "same species")
}
}
sort.SliceStable(out, func(i, j int) bool { return out[i].rank < out[j].rank })
matches := make([]PacketPlantMatch, len(out))
for i, s := range out {
matches[i] = s.match
}
return matches
}
// wordIn reports whether word appears as a whole space-delimited token in name,
// so "garlic" matches "German Garlic" but not "garlicky-thing".
func wordIn(name, word string) bool {
for _, tok := range strings.Fields(name) {
if tok == word {
return true
}
}
return false
}
// PacketConfirm is a user-confirmed proposal to turn into rows.
//
// Plant selection is explicit: either PlantID names an existing plant to attach
// the lot to, or NewPlant carries the fields to create one. Exactly one — the
// service refuses both or neither, so an ambiguous confirm can't silently pick.
type PacketConfirm struct {
// PlantID attaches the lot to an existing plant. Set this XOR NewPlant.
PlantID *int64
// NewPlant creates a variety. Set this XOR PlantID.
NewPlant *PlantInput
// Lot is the purchase to record against whichever plant results.
Lot SeedLotInput
}
// PacketResult is what a confirm produced.
type PacketResult struct {
Plant *domain.Plant `json:"plant"`
Lot *domain.SeedLot `json:"lot"`
PlantIsNew bool `json:"plantIsNew"`
}
// CreateFromPacket turns a confirmed proposal into a plant (new or existing) plus
// a seed lot attributed to it. Unlike garden edits these rows aren't in the undo
// history — plants and lots are catalog/inventory, created directly — so there is
// no change set to wrap; the two creations just happen in sequence.
//
// If a new plant is created but the lot then fails, the plant is left behind
// rather than rolled back: a stray catalog entry is harmless and editable,
// whereas silently discarding a variety the user just confirmed is worse. The
// caller sees the error and can retry the lot against the now-existing plant.
func (s *Service) CreateFromPacket(ctx context.Context, actorID int64, in PacketConfirm) (*PacketResult, error) {
hasID, hasNew := in.PlantID != nil, in.NewPlant != nil
if hasID == hasNew {
return nil, domain.ErrInvalidInput // exactly one of existing / new
}
res := &PacketResult{}
if hasNew {
plant, err := s.CreatePlant(ctx, actorID, *in.NewPlant)
if err != nil {
return nil, err
}
res.Plant = plant
res.PlantIsNew = true
} else {
// Attach to an existing plant the actor can see. visiblePlant enforces the
// ACL (built-ins + their own); a plant they can't see is ErrNotFound.
plant, err := s.visiblePlant(ctx, actorID, *in.PlantID)
if err != nil {
return nil, err
}
res.Plant = plant
}
lotIn := in.Lot
lotIn.PlantID = res.Plant.ID
lot, err := s.CreateSeedLot(ctx, actorID, lotIn)
if err != nil {
return nil, err
}
res.Lot = lot
return res, nil
}