Introduces an opt-in level-based reasoning toggle (low/medium/high) that
each provider translates to its native parameter:
- Anthropic: thinking.budget_tokens (1024/8000/24000), with temperature
forced to default and MaxTokens auto-grown above the budget.
- OpenAI/xAI/Groq via openaicompat: reasoning_effort string, gated by a
new Rules.SupportsReasoning predicate so non-reasoning models don't
receive the parameter. xAI uses Rules.MapReasoningEffort to remap
"medium" to "high" since its API only accepts low|high.
- Google: thinking_config.thinking_budget + include_thoughts:true.
- DeepSeek: SupportsReasoning=false (reasoner is always-on; the
reasoning_content trace was already extracted via openaicompat).
Reasoning content is surfaced as Response.Thinking on Complete and as
StreamEventThinking deltas during streaming. Provider-side: extracted
from Anthropic thinking content blocks, Google's part.Thought=true
parts, and the non-standard reasoning_content field that DeepSeek and
Groq emit (parsed out of raw JSON since openai-go doesn't type it).
Public API:
- llm.ReasoningLevel + ReasoningLow/Medium/High constants
- llm.WithReasoning(level) request option
- Model.WithReasoning(level) for baked-in defaults
- provider.Request.Reasoning, provider.Response.Thinking
- provider.StreamEventThinking
Tests cover Rules-based gating, MapReasoningEffort, reasoning_content
extraction (Complete + Stream), Anthropic budget mapping, and
temperature suppression when thinking is enabled. Existing behavior is
unchanged when Reasoning is the empty string.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Five OpenAI-compatible providers join the library as first-class constructors
(llm.DeepSeek, llm.Moonshot, llm.XAI, llm.Groq, llm.Ollama). Their wire-level
implementation is shared via a new v2/openaicompat package which is the
extracted guts of the old v2/openai provider; each provider supplies its own
Rules value to declare per-model constraints (e.g., DeepSeek Reasoner rejects
tools and temperature, Moonshot/xAI accept images only on *-vision* models,
Groq rejects audio input). v2/openai itself becomes a thin wrapper that sets
RestrictTemperature for o-series and gpt-5 models.
A new provider registry (v2/registry.go) exposes llm.Providers() and drives
the TUI's provider picker so adding a provider in future is a single-file
change.
The TUI at cmd/llm was migrated from v1 to v2 and moved to v2/cmd/llm. With
nothing else depending on v1, the v1 code at the repo root (all .go files,
schema/, internal/, provider/, root go.mod/go.sum) is deleted.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Verifies that WithPromptCaching() on a Chat results in CacheHints being
set on the provider.Request that reaches the provider layer, and that
omitting the option leaves CacheHints nil (no behavior change for
existing callers).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
buildRequest now tracks a source-index → built-message-index mapping
during the role-merge pass, then uses the mapping to attach
cache_control: {type: ephemeral} markers at the positions indicated by
Request.CacheHints. The last tool, the last system part, and the last
non-system message each get a marker when the corresponding hint is set.
Covers the merge-induced index drift that would otherwise cause the
breakpoint to land on the wrong content block when consecutive same-role
source messages are combined into a single Anthropic message with
multiple content blocks.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Removes the blank-assign workaround that was only needed because the
anth import was being kept alive for Task 5's use. Task 5 will bring
the import back when it actually references anth.CacheControlTypeEphemeral.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Switches buildRequest to emit anthReq.MultiSystem instead of anthReq.System
whenever a system message is present. Upstream's MarshalJSON prefers
MultiSystem when non-empty, so the wire format is unchanged for requests
without cache_control. This refactor is a prerequisite for attaching
cache_control markers to system parts in the next commit.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds two boundary tests suggested by code review:
- TestBuildProviderRequest_CachingEnabled_EmptyMessages: verifies
that caching with an empty message list still emits a CacheHints
with LastCacheableMessageIndex=-1, not a spurious breakpoint.
- TestBuildProviderRequest_CachingNonNilButDisabled: verifies that
an explicitly-disabled cacheConfig (non-nil, enabled=false)
produces nil CacheHints, exercising the &&-guard branch that
the previous "disabled" test left untested.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
buildProviderRequest now computes cache-breakpoint positions automatically
when the WithPromptCaching() option is set. It places up to 3 hints:
tools, system, and the index of the last non-system message. Providers
that don't support caching (OpenAI, Google) ignore the field.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Introduces an opt-in RequestOption that callers can pass to enable
automatic prompt-caching markers. The option populates a cacheConfig
on requestConfig but has no effect yet — plumbing through to
provider.Request and on to the Anthropic provider lands in subsequent
commits.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds an optional CacheHints field on provider.Request that carries
cache-breakpoint placement directives from the public llm package down
to individual provider implementations. Anthropic will consume these in
a follow-up commit; OpenAI and Google ignore them.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add provider-specific usage details, fix streaming usage, and return
usage from all high-level APIs (Chat.Send, Generate[T], Agent.Run).
Breaking changes:
- Chat.Send/SendMessage/SendWithImages now return (string, *Usage, error)
- Generate[T]/GenerateWith[T] now return (T, *Usage, error)
- Agent.Run/RunMessages now return (string, *Usage, error)
New features:
- Usage.Details map for provider-specific token breakdowns
(reasoning, cached, audio, thoughts tokens)
- OpenAI streaming now captures usage via StreamOptions.IncludeUsage
- Google streaming now captures UsageMetadata from final chunk
- UsageTracker.Details() for accumulated detail totals
- ModelPricing and PricingRegistry for cost computation
Closes#2
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add Audio struct alongside Image for sending audio attachments to
multimodal LLMs. OpenAI uses input_audio content parts (wav/mp3),
Google Gemini uses genai.NewPartFromBytes, and Anthropic skips
audio gracefully since it's not supported.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Provides a complete lifecycle manager for ephemeral sandbox environments:
- ProxmoxClient: thin REST wrapper for container CRUD, IP discovery, internet toggle
- SSHExecutor: persistent SSH/SFTP for command execution and file transfer
- Manager/Sandbox: high-level orchestrator tying Proxmox + SSH together
- 22 unit tests with mock Proxmox HTTP server
- Proxmox setup & hardening guide (docs/sandbox-setup.md)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Introduces v2/agent with a minimal API: Agent, New(), Run(), and AsTool().
Agents wrap a model + system prompt + tools. AsTool() turns an agent into
a llm.Tool, enabling parent agents to delegate to sub-agents through the
normal tool-call loop — no channels, pools, or orchestration needed.
Also exports NewClient(provider.Provider) for custom provider integration.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Generic functions that use the "hidden tool" technique to force models
to return structured JSON matching a Go struct's schema, replacing the
verbose "tool as structured output" pattern.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Cover all core library logic (Client, Model, Chat, middleware, streaming,
message conversion, request building) using a configurable mock provider
that avoids real API calls. ~50 tests across 7 files.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Migrate speech-to-text transcription types and OpenAI transcriber
implementation from v1. Types are defined in provider/ to avoid
import cycles and re-exported via type aliases from the root package.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
v2 is a new Go module (v2/) with a dramatically simpler API:
- Unified Message type (no more Input marker interface)
- Define[T] for ergonomic tool creation with standard context.Context
- Chat session with automatic tool-call loop (agent loop)
- Streaming via pull-based StreamReader
- MCP one-call connect (MCPStdioServer, MCPHTTPServer, MCPSSEServer)
- Middleware support (logging, retry, timeout, usage tracking)
- Decoupled JSON Schema (map[string]any, no provider coupling)
- Sample tools: WebSearch, Browser, Exec, ReadFile, WriteFile, HTTP
- Providers: OpenAI, Anthropic, Google (all with streaming)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
steve