Skip to content

llms/bedrock: add cache support for anthropic#1498

Open
lucaronca wants to merge 4 commits into
tmc:mainfrom
lucaronca:bedrock-cache-support
Open

llms/bedrock: add cache support for anthropic#1498
lucaronca wants to merge 4 commits into
tmc:mainfrom
lucaronca:bedrock-cache-support

Conversation

@lucaronca
Copy link
Copy Markdown
Contributor

@lucaronca lucaronca commented May 15, 2026

PR Checklist

Plumbs llms.CachedContent / llms.CacheControl through the Bedrock provider for Anthropic
models, so callers can mark content blocks as prompt-cache breakpoints with the same API used
by llms/anthropic:

llm.GenerateContent(ctx, []llms.MessageContent{
    {Role: llms.ChatMessageTypeSystem, Parts: []llms.ContentPart{
        llms.WithCacheControl(llms.TextPart(largeContext), bedrock.EphemeralCache()),
    }},
})

Note: this PR includes the changes from #1489. Please merge that one first so this diff can be rebased to show only the prompt-caching work.

  • Read the Contributing documentation.
  • Read the Code of conduct documentation.
  • Name your Pull Request title clearly, concisely, and prefixed with the name of the primarily affected package you changed according to Good commit messages (such as memory: add interfaces for X, Y or util: add whizzbang helpers).
  • Check that there isn't already a PR that solves the problem the same way to avoid creating a duplicate.
  • Provide a description in this PR that addresses what the PR is solving, or reference the issue that it solves (e.g. Fixes #123).
  • Describes the source of new concepts.
  • References existing implementations as appropriate.
  • Contains test coverage for new functions.
  • Passes all golangci-lint checks.

@lucaronca lucaronca changed the title llms/bedrock: introduce cache support for anthropic llms/bedrock: add cache support for anthropic May 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant