-
Notifications
You must be signed in to change notification settings - Fork 3.1k
feat(provider/anthropic): add prompt caching validation #9330
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
3416acb
to
1e76ca1
Compare
@lgrammel I read https://docs.claude.com/en/docs/build-with-claude/prompt-caching for the nitty picky details and tried to fix them in the ai sdk with this PR. |
I also saw that
Also, @lgrammel , here cacheControl is not used anywhere. You need to set cacheControl on parts or messages (which is convenient). Should we remove it and add it back in a PR where it is actually being used? |
@lgrammel have you had time to look at this? |
it's on my list but i am a bit underwater at the moment |
2da2ee5
to
58c6d40
Compare
58c6d40
to
c68d42e
Compare
c68d42e
to
ec8b1bc
Compare
Looks good, great contribution! Once agent skills lands I'll update and land this PR |
|
) This is an automated backport of #9330 to the release-v5.0 branch. --------- Co-authored-by: Jonas Lalin <[email protected]> Co-authored-by: Lars Grammel <[email protected]>
Fix Anthropic's prompt caching feature:
Changes include proper type definitions with cache_control?: never for disallowed locations, validation tests, and backward compatibility.
Background
Anthropic's prompt caching API has specific constraints that must be followed:
cache_control
can only be used on specific content types (not on nested content blocks within tool results)cache_control
cannot be set on thinking blocks or redacted thinking blocksWithout proper validation, developers can inadvertently create invalid configurations that either fail at runtime or are silently ignored by the API.
Summary
This PR adds comprehensive validation for Anthropic's prompt caching feature to catch configuration errors early and provide clear guidance to developers.
Key changes:
CacheControlValidator class (
get-cache-control.ts
)cache_control
is only used in supported contextsUpdated type definitions (
anthropic-messages-api.ts
)AnthropicNestedTextContent
,AnthropicNestedImageContent
, andAnthropicNestedDocumentContent
typescache_control
viacache_control?: never
AnthropicToolResultContent
to use nested types, preventing cache_control on sub-content blocks (per Anthropic docs)Integration (
convert-to-anthropic-messages-prompt.ts
,anthropic-prepare-tools.ts
)Test coverage
Manual Verification
cache_control
on nested content in tool results - correctly prevented by TypeScript typesChecklist
pnpm changeset
in the project root)pnpm prettier-fix
in the project root)Future Work
Related Issues
Related to Anthropic prompt caching API requirements and constraints.