Non-streaming /api/chat support including:
- Vision via images: []base64
- Tool calls on assistant + tool-role response messages
- think field accepting string reasoning levels (or "true"/"false")
- Authorization header when apiKey is non-empty (cloud mode)
Tool-call arguments are passed as JSON objects to the wire and surfaced
as JSON-string Arguments on provider.ToolCall. Tool calls are assigned
synthetic IDs (tc_<index>) when Ollama omits one, so the round-trip
back as an assistant tool_calls + tool-role message remains correlated.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds wire types and a Provider struct that will replace the
openaicompat-based Ollama shim with a native /api/chat implementation.
Complete and Stream methods are stubs; subsequent commits implement them.
Adjusts the existing ollama.go to drop the type alias on
openaicompat.Provider (renaming the legacy shim to a temporary internal
helper) so the new native Provider type does not collide. Public New()
still returns the openaicompat-backed provider until Task 4 swaps it.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
LLMs occasionally return numeric or boolean tool-call fields as JSON
strings (e.g. "3" instead of 3, "true" instead of true), which Go's
strict json.Unmarshal rejects. The strict unmarshal stays as the happy
path; on failure we retry with a coercion pass that walks the target
struct (recursing into nested structs, slices, maps, and pointer fields)
and converts strings to the appropriate kind. Returns the original error
if coercion can't recover.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Introduces an opt-in level-based reasoning toggle (low/medium/high) that
each provider translates to its native parameter:
- Anthropic: thinking.budget_tokens (1024/8000/24000), with temperature
forced to default and MaxTokens auto-grown above the budget.
- OpenAI/xAI/Groq via openaicompat: reasoning_effort string, gated by a
new Rules.SupportsReasoning predicate so non-reasoning models don't
receive the parameter. xAI uses Rules.MapReasoningEffort to remap
"medium" to "high" since its API only accepts low|high.
- Google: thinking_config.thinking_budget + include_thoughts:true.
- DeepSeek: SupportsReasoning=false (reasoner is always-on; the
reasoning_content trace was already extracted via openaicompat).
Reasoning content is surfaced as Response.Thinking on Complete and as
StreamEventThinking deltas during streaming. Provider-side: extracted
from Anthropic thinking content blocks, Google's part.Thought=true
parts, and the non-standard reasoning_content field that DeepSeek and
Groq emit (parsed out of raw JSON since openai-go doesn't type it).
Public API:
- llm.ReasoningLevel + ReasoningLow/Medium/High constants
- llm.WithReasoning(level) request option
- Model.WithReasoning(level) for baked-in defaults
- provider.Request.Reasoning, provider.Response.Thinking
- provider.StreamEventThinking
Tests cover Rules-based gating, MapReasoningEffort, reasoning_content
extraction (Complete + Stream), Anthropic budget mapping, and
temperature suppression when thinking is enabled. Existing behavior is
unchanged when Reasoning is the empty string.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Five OpenAI-compatible providers join the library as first-class constructors
(llm.DeepSeek, llm.Moonshot, llm.XAI, llm.Groq, llm.Ollama). Their wire-level
implementation is shared via a new v2/openaicompat package which is the
extracted guts of the old v2/openai provider; each provider supplies its own
Rules value to declare per-model constraints (e.g., DeepSeek Reasoner rejects
tools and temperature, Moonshot/xAI accept images only on *-vision* models,
Groq rejects audio input). v2/openai itself becomes a thin wrapper that sets
RestrictTemperature for o-series and gpt-5 models.
A new provider registry (v2/registry.go) exposes llm.Providers() and drives
the TUI's provider picker so adding a provider in future is a single-file
change.
The TUI at cmd/llm was migrated from v1 to v2 and moved to v2/cmd/llm. With
nothing else depending on v1, the v1 code at the repo root (all .go files,
schema/, internal/, provider/, root go.mod/go.sum) is deleted.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Verifies that WithPromptCaching() on a Chat results in CacheHints being
set on the provider.Request that reaches the provider layer, and that
omitting the option leaves CacheHints nil (no behavior change for
existing callers).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
buildRequest now tracks a source-index → built-message-index mapping
during the role-merge pass, then uses the mapping to attach
cache_control: {type: ephemeral} markers at the positions indicated by
Request.CacheHints. The last tool, the last system part, and the last
non-system message each get a marker when the corresponding hint is set.
Covers the merge-induced index drift that would otherwise cause the
breakpoint to land on the wrong content block when consecutive same-role
source messages are combined into a single Anthropic message with
multiple content blocks.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Removes the blank-assign workaround that was only needed because the
anth import was being kept alive for Task 5's use. Task 5 will bring
the import back when it actually references anth.CacheControlTypeEphemeral.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Switches buildRequest to emit anthReq.MultiSystem instead of anthReq.System
whenever a system message is present. Upstream's MarshalJSON prefers
MultiSystem when non-empty, so the wire format is unchanged for requests
without cache_control. This refactor is a prerequisite for attaching
cache_control markers to system parts in the next commit.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds two boundary tests suggested by code review:
- TestBuildProviderRequest_CachingEnabled_EmptyMessages: verifies
that caching with an empty message list still emits a CacheHints
with LastCacheableMessageIndex=-1, not a spurious breakpoint.
- TestBuildProviderRequest_CachingNonNilButDisabled: verifies that
an explicitly-disabled cacheConfig (non-nil, enabled=false)
produces nil CacheHints, exercising the &&-guard branch that
the previous "disabled" test left untested.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
buildProviderRequest now computes cache-breakpoint positions automatically
when the WithPromptCaching() option is set. It places up to 3 hints:
tools, system, and the index of the last non-system message. Providers
that don't support caching (OpenAI, Google) ignore the field.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Introduces an opt-in RequestOption that callers can pass to enable
automatic prompt-caching markers. The option populates a cacheConfig
on requestConfig but has no effect yet — plumbing through to
provider.Request and on to the Anthropic provider lands in subsequent
commits.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds an optional CacheHints field on provider.Request that carries
cache-breakpoint placement directives from the public llm package down
to individual provider implementations. Anthropic will consume these in
a follow-up commit; OpenAI and Google ignore them.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add provider-specific usage details, fix streaming usage, and return
usage from all high-level APIs (Chat.Send, Generate[T], Agent.Run).
Breaking changes:
- Chat.Send/SendMessage/SendWithImages now return (string, *Usage, error)
- Generate[T]/GenerateWith[T] now return (T, *Usage, error)
- Agent.Run/RunMessages now return (string, *Usage, error)
New features:
- Usage.Details map for provider-specific token breakdowns
(reasoning, cached, audio, thoughts tokens)
- OpenAI streaming now captures usage via StreamOptions.IncludeUsage
- Google streaming now captures UsageMetadata from final chunk
- UsageTracker.Details() for accumulated detail totals
- ModelPricing and PricingRegistry for cost computation
Closes#2
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add Audio struct alongside Image for sending audio attachments to
multimodal LLMs. OpenAI uses input_audio content parts (wav/mp3),
Google Gemini uses genai.NewPartFromBytes, and Anthropic skips
audio gracefully since it's not supported.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Provides a complete lifecycle manager for ephemeral sandbox environments:
- ProxmoxClient: thin REST wrapper for container CRUD, IP discovery, internet toggle
- SSHExecutor: persistent SSH/SFTP for command execution and file transfer
- Manager/Sandbox: high-level orchestrator tying Proxmox + SSH together
- 22 unit tests with mock Proxmox HTTP server
- Proxmox setup & hardening guide (docs/sandbox-setup.md)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Introduces v2/agent with a minimal API: Agent, New(), Run(), and AsTool().
Agents wrap a model + system prompt + tools. AsTool() turns an agent into
a llm.Tool, enabling parent agents to delegate to sub-agents through the
normal tool-call loop — no channels, pools, or orchestration needed.
Also exports NewClient(provider.Provider) for custom provider integration.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Generic functions that use the "hidden tool" technique to force models
to return structured JSON matching a Go struct's schema, replacing the
verbose "tool as structured output" pattern.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Cover all core library logic (Client, Model, Chat, middleware, streaming,
message conversion, request building) using a configurable mock provider
that avoids real API calls. ~50 tests across 7 files.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Migrate speech-to-text transcription types and OpenAI transcriber
implementation from v1. Types are defined in provider/ to avoid
import cycles and re-exported via type aliases from the root package.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Runs on all pushes and PRs:
- Build, vet, and test both root and v2 modules (with -race)
- Verify go.mod/go.sum tidiness for both modules
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
v2 is a new Go module (v2/) with a dramatically simpler API:
- Unified Message type (no more Input marker interface)
- Define[T] for ergonomic tool creation with standard context.Context
- Chat session with automatic tool-call loop (agent loop)
- Streaming via pull-based StreamReader
- MCP one-call connect (MCPStdioServer, MCPHTTPServer, MCPSSEServer)
- Middleware support (logging, retry, timeout, usage tracking)
- Decoupled JSON Schema (map[string]any, no provider coupling)
- Sample tools: WebSearch, Browser, Exec, ReadFile, WriteFile, HTTP
- Providers: OpenAI, Anthropic, Google (all with streaming)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Introduce `MCPServer` to support connecting to MCP servers via stdio, SSE, or HTTP.
- Implement tool fetching, management, and invocation through MCP.
- Add `WithMCPServer` method to `ToolBox` for seamless tool integration.
- Extend schema package to handle raw JSON schemas for MCP tools.
- Update documentation with MCP usage guidelines and examples.
- Migrate `compress_image.go` to `internal/imageutil` for better encapsulation.
- Reorganize LLM provider implementations into distinct packages (`google`, `openai`, and `anthropic`).
- Replace `go_llm` package name with `llm`.
- Refactor internal APIs for improved clarity, including renaming `anthropic` to `anthropicImpl` and `google` to `googleImpl`.
- Add helper methods and restructure message handling for better separation of concerns.
- Update all Go dependencies to latest versions
- Migrate from github.com/google/generative-ai-go/genai to google.golang.org/genai
- Fix google.go to use the new SDK API (NewPartFromText, NewContentFromParts, etc.)
- Update schema package imports to use the new genai package
- Add CLAUDE.md with README maintenance guideline
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Introduce `Providers` struct to handle different language model providers. Implement `Parse` method to extract and validate provider/model from input string, then return a chat completion interface. Add error handling for invalid formats or unknown providers.
Previously, required fields were not handled in OpenAI and Google parameter generation. This update adds logic to include a "required" list for both, ensuring mandatory fields are accurately captured in the schema outputs.
This update introduces support for `jsonschema.Integer` types and updates the logic to handle nested items in schemas. Added a new default error log for unknown types using `slog.Error`. Also, integrated tool configuration with a `FunctionCallingConfig` when `dontRequireTool` is false.
This update introduces support for `jsonschema.Integer` types and updates the logic to handle nested items in schemas. Added a new default error log for unknown types using `slog.Error`. Also, integrated tool configuration with a `FunctionCallingConfig` when `dontRequireTool` is false.
Updated the return type of functions and related code from `string` to `any` to improve flexibility and support more diverse outputs. Adjusted function implementations, signatures, and handling of results accordingly.
steve
Claude