feat(v2): populate CacheHints on provider.Request when caching enabled
All checks were successful
CI / Lint (push) Successful in 9m36s
CI / Root Module (push) Successful in 10m55s
CI / V2 Module (push) Successful in 11m14s

buildProviderRequest now computes cache-breakpoint positions automatically
when the WithPromptCaching() option is set. It places up to 3 hints:
tools, system, and the index of the last non-system message. Providers
that don't support caching (OpenAI, Google) ignore the field.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-04-09 19:22:00 +00:00
parent c4fe0026a2
commit 4b401fcc0d
2 changed files with 106 additions and 0 deletions

View File

@@ -122,6 +122,26 @@ func buildProviderRequest(model string, messages []Message, cfg *requestConfig)
}
}
if cfg.cacheConfig != nil && cfg.cacheConfig.enabled {
hints := &provider.CacheHints{LastCacheableMessageIndex: -1}
if len(req.Tools) > 0 {
hints.CacheTools = true
}
for _, m := range messages {
if m.Role == RoleSystem {
hints.CacheSystem = true
break
}
}
for i := len(messages) - 1; i >= 0; i-- {
if messages[i].Role != RoleSystem {
hints.LastCacheableMessageIndex = i
break
}
}
req.CacheHints = hints
}
return req
}