Seed-packet capture: vision model, extraction, catalog match, create (backend) (#81)
Photograph a seed packet → it fills in the plant and the purchase. This is the backend; the scan UI is a follow-up PR. Vision model config (mirrors the agent model from #79): - Migration 0011 adds instance_settings.vision_model; PANSY_VISION_MODEL is the env default. Precedence Settings → env → empty; the KEY stays in the env. - EffectiveVision resolves it; /capabilities advertises "vision" only when a model + key are configured, so the UI offers the scan button only when it works. Extraction is one-shot, NOT an agent loop (internal/vision): - majordomo.Generate[SeedPacket] derives a JSON schema from the struct tags and hands the image to the vision model; it can't call a tool, so it can't touch the garden — it only reads a picture and returns data. Numeric fields are pointers, so a field the packet doesn't print comes back nil, not a made-up 0. - Hermetic test: majordomo's fake provider returns canned packet JSON and Generate unmarshals it, image + derived schema included. No live model. The image is normalized to JPEG at the upload boundary (imagenorm from #80), which is where an iPhone HEIC becomes readable — majordomo's media path can't decode HEIC. imagenorm now links into the binary (~7 MB, the cost #80 deferred). The hard part is catalog matching, not OCR (internal/service/seed_packet.go): - A wrong auto-match splits a variety's seed-lot history across duplicate rows, so the service NEVER auto-creates. matchPlants surfaces RANKED candidates (exact name → variety-in-name → same species, conservative and name-based), the user confirms, and CreateFromPacket makes the plant (new or existing) + the lot. Exactly one of plantId/newPlant, refused otherwise. - Plants/lots aren't in the undo history (catalog/inventory), so no change set. - The extractor is injectable (service.WithPacketExtractor) so ExtractSeedPacket and the /scan endpoint test end to end against a fake, no live model. Endpoints: POST /seed-lots/scan (multipart image → proposal, reads only; extends the read deadline for a slow phone upload, caps the body, maps too-large/unreadable to clear statuses) and POST /seed-lots/from-packet (confirmed proposal → 201). Docs: README (PANSY_VISION_MODEL), DESIGN (routes + the decision and why the model can't touch the garden). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01H3zbym8Doka2d7D48maSgZ
This commit is contained in:
@@ -43,6 +43,7 @@ func (s *Service) GetInstanceSettings(ctx context.Context, actorID int64) (*doma
|
||||
type InstanceSettingsPatch struct {
|
||||
AgentModel string
|
||||
AgentEnabled *bool
|
||||
VisionModel string
|
||||
Version int64
|
||||
}
|
||||
|
||||
@@ -58,16 +59,20 @@ func (s *Service) UpdateInstanceSettings(ctx context.Context, actorID int64, pat
|
||||
return nil, err
|
||||
}
|
||||
model := strings.TrimSpace(patch.AgentModel)
|
||||
// Validate a non-empty spec up front. An empty one is the "inherit env"
|
||||
vision := strings.TrimSpace(patch.VisionModel)
|
||||
// Validate non-empty specs up front. An empty one is the "inherit env"
|
||||
// sentinel and needs no check — the env value was validated at boot.
|
||||
if model != "" {
|
||||
if err := agentmodel.Validate(s.cfg.Agent.OllamaCloudAPIKey, model); err != nil {
|
||||
return nil, domain.ErrInvalidInput
|
||||
for _, spec := range []string{model, vision} {
|
||||
if spec != "" {
|
||||
if err := agentmodel.Validate(s.cfg.Agent.OllamaCloudAPIKey, spec); err != nil {
|
||||
return nil, domain.ErrInvalidInput
|
||||
}
|
||||
}
|
||||
}
|
||||
return s.store.UpdateInstanceSettings(ctx, &domain.InstanceSettings{
|
||||
AgentModel: model,
|
||||
AgentEnabled: patch.AgentEnabled,
|
||||
VisionModel: vision,
|
||||
Version: patch.Version,
|
||||
})
|
||||
}
|
||||
@@ -108,3 +113,32 @@ func (s *Service) EffectiveAgent(ctx context.Context) (EffectiveAgent, error) {
|
||||
}
|
||||
return eff, nil
|
||||
}
|
||||
|
||||
// EffectiveVision resolves the vision configuration in force for seed-packet
|
||||
// capture (#81): the model from DB-over-env, the key always from env.
|
||||
type EffectiveVision struct {
|
||||
Model string
|
||||
APIKey string
|
||||
}
|
||||
|
||||
// Ready reports whether packet capture can be offered: a key and a vision model.
|
||||
// There's no separate enabled flag — configuring a vision model IS enabling it.
|
||||
func (e EffectiveVision) Ready() bool {
|
||||
return e.APIKey != "" && e.Model != ""
|
||||
}
|
||||
|
||||
// EffectiveVision reads the settings row and layers it over the environment.
|
||||
func (s *Service) EffectiveVision(ctx context.Context) (EffectiveVision, error) {
|
||||
st, err := s.store.GetInstanceSettings(ctx)
|
||||
if err != nil {
|
||||
return EffectiveVision{}, err
|
||||
}
|
||||
eff := EffectiveVision{
|
||||
Model: s.cfg.Agent.VisionModel,
|
||||
APIKey: s.cfg.Agent.OllamaCloudAPIKey,
|
||||
}
|
||||
if st.VisionModel != "" {
|
||||
eff.Model = st.VisionModel
|
||||
}
|
||||
return eff, nil
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user