steve/go-llm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: comprehensive token usage tracking for V2 #3

Merged
Claude merged 1 commits from feature/comprehensive-token-usage-tracking into main 2026-03-02 04:35:42 +00:00
Conversation 0 Commits 1 Files Changed 17 +684 -61
Claude commented 2026-03-02 04:33:43 +00:00
Collaborator
Copy Link

Summary

  • Add Details map[string]int to Usage for provider-specific token breakdowns (reasoning, cached, audio, thoughts tokens)
  • Fix OpenAI streaming to capture usage via StreamOptions.IncludeUsage
  • Fix Google streaming to capture UsageMetadata from final chunk
  • Return *Usage from Chat.Send*, Generate[T]/GenerateWith[T], and Agent.Run* (breaking signature changes)
  • Enhance UsageTracker middleware to accumulate detail values with Details() accessor
  • Add ModelPricing and PricingRegistry for cost computation with cached token discounts

Breaking Changes

API Old Signature New Signature
Chat.Send (string, error) (string, *Usage, error)
Chat.SendMessage (string, error) (string, *Usage, error)
Chat.SendWithImages (string, error) (string, *Usage, error)
Generate[T] (T, error) (T, *Usage, error)
GenerateWith[T] (T, error) (T, *Usage, error)
Agent.Run (string, error) (string, *Usage, error)
Agent.RunMessages (string, error) (string, *Usage, error)

Files Changed

File Change
v2/provider/provider.go Add Details map + key constants to Usage
v2/response.go Add Details map, addUsage helper, re-export constants
v2/llm.go Pass through Details in convertProviderResponse
v2/openai/openai.go Detail extraction + streaming usage fix
v2/anthropic/anthropic.go Cache token extraction
v2/google/google.go Detail extraction + streaming usage fix
v2/chat.go Return *Usage from Send/SendMessage/SendWithImages
v2/generate.go Return *Usage from Generate[T]/GenerateWith[T]
v2/agent/agent.go Return *Usage from Run/RunMessages, update AsTool
v2/middleware.go UsageTracker detail accumulation + Details() accessor
v2/pricing.go New - ModelPricing, PricingRegistry

Test Plan

  • go build ./... compiles
  • go test ./... — all tests pass
  • go vet ./... — no issues
  • golangci-lint run ./... — no new issues
  • New tests for usage accumulation across tool-call loop
  • New tests for usage details accumulation in UsageTracker
  • New tests for Generate[T] returning usage
  • New tests for Agent.Run returning usage
  • New tests for PricingRegistry.Cost() with cached token discount

Closes #2
Also created steve/steveternet#47 for V1 audit.

🤖 Generated with Claude Code

## Summary - Add `Details map[string]int` to `Usage` for provider-specific token breakdowns (reasoning, cached, audio, thoughts tokens) - Fix OpenAI streaming to capture usage via `StreamOptions.IncludeUsage` - Fix Google streaming to capture `UsageMetadata` from final chunk - Return `*Usage` from `Chat.Send*`, `Generate[T]`/`GenerateWith[T]`, and `Agent.Run*` (breaking signature changes) - Enhance `UsageTracker` middleware to accumulate detail values with `Details()` accessor - Add `ModelPricing` and `PricingRegistry` for cost computation with cached token discounts ## Breaking Changes | API | Old Signature | New Signature | |-----|--------------|---------------| | `Chat.Send` | `(string, error)` | `(string, *Usage, error)` | | `Chat.SendMessage` | `(string, error)` | `(string, *Usage, error)` | | `Chat.SendWithImages` | `(string, error)` | `(string, *Usage, error)` | | `Generate[T]` | `(T, error)` | `(T, *Usage, error)` | | `GenerateWith[T]` | `(T, error)` | `(T, *Usage, error)` | | `Agent.Run` | `(string, error)` | `(string, *Usage, error)` | | `Agent.RunMessages` | `(string, error)` | `(string, *Usage, error)` | ## Files Changed | File | Change | |------|--------| | `v2/provider/provider.go` | Add `Details` map + key constants to `Usage` | | `v2/response.go` | Add `Details` map, `addUsage` helper, re-export constants | | `v2/llm.go` | Pass through `Details` in `convertProviderResponse` | | `v2/openai/openai.go` | Detail extraction + streaming usage fix | | `v2/anthropic/anthropic.go` | Cache token extraction | | `v2/google/google.go` | Detail extraction + streaming usage fix | | `v2/chat.go` | Return `*Usage` from `Send`/`SendMessage`/`SendWithImages` | | `v2/generate.go` | Return `*Usage` from `Generate[T]`/`GenerateWith[T]` | | `v2/agent/agent.go` | Return `*Usage` from `Run`/`RunMessages`, update `AsTool` | | `v2/middleware.go` | `UsageTracker` detail accumulation + `Details()` accessor | | `v2/pricing.go` | **New** - `ModelPricing`, `PricingRegistry` | ## Test Plan - [x] `go build ./...` compiles - [x] `go test ./...` — all tests pass - [x] `go vet ./...` — no issues - [x] `golangci-lint run ./...` — no new issues - [x] New tests for usage accumulation across tool-call loop - [x] New tests for usage details accumulation in `UsageTracker` - [x] New tests for `Generate[T]` returning usage - [x] New tests for `Agent.Run` returning usage - [x] New tests for `PricingRegistry.Cost()` with cached token discount Closes #2 Also created steve/steveternet#47 for V1 audit. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Claude added 1 commit 2026-03-02 04:33:43 +00:00
feat: comprehensive token usage tracking for V2
All checks were successful
CI / Lint (pull_request) Successful in 10m18s
Details
CI / Root Module (pull_request) Successful in 11m4s
Details
CI / V2 Module (pull_request) Successful in 11m5s
Details
5b687839b2 
Add provider-specific usage details, fix streaming usage, and return
usage from all high-level APIs (Chat.Send, Generate[T], Agent.Run).

Breaking changes:
- Chat.Send/SendMessage/SendWithImages now return (string, *Usage, error)
- Generate[T]/GenerateWith[T] now return (T, *Usage, error)
- Agent.Run/RunMessages now return (string, *Usage, error)

New features:
- Usage.Details map for provider-specific token breakdowns
  (reasoning, cached, audio, thoughts tokens)
- OpenAI streaming now captures usage via StreamOptions.IncludeUsage
- Google streaming now captures UsageMetadata from final chunk
- UsageTracker.Details() for accumulated detail totals
- ModelPricing and PricingRegistry for cost computation

Closes #2

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Claude merged commit bcb91e14b0 into main 2026-03-02 04:35:42 +00:00
Claude deleted branch feature/comprehensive-token-usage-tracking 2026-03-02 04:35:43 +00:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
Claude fizi steve
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: steve/go-llm#3
Delete Branch "feature/comprehensive-token-usage-tracking"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?