Skip to content

Conversation

jonaslalin
Copy link
Contributor

@jonaslalin jonaslalin commented Oct 9, 2025

Fix Anthropic's prompt caching feature:

  • Add cache_control support to all content types and tool definitions
  • Implement CacheControlValidator to enforce 4 breakpoint limit
  • Add TTL support (5m/1h) to cache control type
  • Prevent caching on thinking blocks and nested tool result content
  • Centralize warning collection to avoid duplicates
  • Add cache_control to provider tool types (set to undefined in SDK)

Changes include proper type definitions with cache_control?: never for disallowed locations, validation tests, and backward compatibility.

Background

Anthropic's prompt caching API has specific constraints that must be followed:

  • Maximum of 4 cache breakpoints per request
  • cache_control can only be used on specific content types (not on nested content blocks within tool results)
  • cache_control cannot be set on thinking blocks or redacted thinking blocks

Without proper validation, developers can inadvertently create invalid configurations that either fail at runtime or are silently ignored by the API.

Summary

This PR adds comprehensive validation for Anthropic's prompt caching feature to catch configuration errors early and provide clear guidance to developers.

Key changes:

  1. CacheControlValidator class (get-cache-control.ts)

    • Tracks and enforces the 4 cache breakpoint limit per request
    • Validates that cache_control is only used in supported contexts
    • Generates helpful warnings when invalid configurations are detected
  2. Updated type definitions (anthropic-messages-api.ts)

    • Added AnthropicNestedTextContent, AnthropicNestedImageContent, and AnthropicNestedDocumentContent types
    • These nested types explicitly prevent cache_control via cache_control?: never
    • Updated AnthropicToolResultContent to use nested types, preventing cache_control on sub-content blocks (per Anthropic docs)
  3. Integration (convert-to-anthropic-messages-prompt.ts, anthropic-prepare-tools.ts)

    • Integrated validator into message and tool conversion logic
    • Validator tracks cache breakpoints across system messages, user messages, assistant messages, and tools
    • Returns warnings array that can be surfaced to developers
  4. Test coverage

    • Added tests for cache breakpoint limit enforcement
    • Tests for warning generation on invalid configurations
    • Tests for proper handling of nested content blocks

Manual Verification

  1. Created test cases with more than 4 cache breakpoints - validator correctly limits to 4 and generates warnings
  2. Attempted to set cache_control on nested content in tool results - correctly prevented by TypeScript types
  3. Verified that valid cache_control configurations continue to work as expected
  4. Checked that warnings are properly propagated through the API

Checklist

  • Tests have been added / updated (for bug fixes / features)
  • Documentation has been added / updated (for bug fixes / features)
  • A patch changeset for relevant packages has been added (for bug fixes / features - run pnpm changeset in the project root)
  • Formatting issues have been fixed (run pnpm prettier-fix in the project root)
  • I have reviewed this pull request (self-review)

Future Work

  • Consider adding documentation examples showing proper cache_control usage patterns

Related Issues

Related to Anthropic prompt caching API requirements and constraints.

@jonaslalin jonaslalin force-pushed the jonas/cache-control branch from 3416acb to 1e76ca1 Compare October 9, 2025 12:30
@jonaslalin
Copy link
Contributor Author

@lgrammel I read https://docs.claude.com/en/docs/build-with-claude/prompt-caching for the nitty picky details and tried to fix them in the ai sdk with this PR.

@jonaslalin
Copy link
Contributor Author

jonaslalin commented Oct 9, 2025

I also saw that

export const anthropicProviderOptions = z.object({
  sendReasoning: z.boolean().optional(),

  thinking: z
    .object({
      type: z.union([z.literal('enabled'), z.literal('disabled')]),
      budgetTokens: z.number().optional(),
    })
    .optional(),

  /**
   * Whether to disable parallel function calling during tool use. Default is false.
   * When set to true, Claude will use at most one tool per response.
   */
  disableParallelToolUse: z.boolean().optional(),

  /**
   * Cache control settings for this message.
   * See https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching
   */
  cacheControl: z
    .object({
      type: z.literal('ephemeral'),
      ttl: z.union([z.literal('5m'), z.literal('1h')]).optional(),
    })
    .optional(),
});

Also, @lgrammel , here cacheControl is not used anywhere. You need to set cacheControl on parts or messages (which is convenient). Should we remove it and add it back in a PR where it is actually being used?

@jonaslalin
Copy link
Contributor Author

@lgrammel have you had time to look at this?

@jonaslalin jonaslalin changed the title feat(anthropic): implement proper prompt caching and validation fix(anthropic): implement proper prompt caching and validation Oct 15, 2025
@lgrammel
Copy link
Collaborator

@lgrammel have you had time to look at this?

it's on my list but i am a bit underwater at the moment

@jonaslalin jonaslalin force-pushed the jonas/cache-control branch 2 times, most recently from 2da2ee5 to 58c6d40 Compare October 16, 2025 12:07
@jonaslalin jonaslalin changed the title fix(anthropic): implement proper prompt caching and validation feat(anthropic): add prompt caching validation Oct 16, 2025
@lgrammel lgrammel changed the title feat(anthropic): add prompt caching validation feat(provider/anthropic): add prompt caching validation Oct 17, 2025
@lgrammel
Copy link
Collaborator

Looks good, great contribution!

Once agent skills lands I'll update and land this PR

@lgrammel lgrammel self-assigned this Oct 17, 2025
@lgrammel lgrammel merged commit ca07285 into vercel:main Oct 17, 2025
17 of 18 checks passed
vercel-ai-sdk bot pushed a commit that referenced this pull request Oct 17, 2025
@vercel-ai-sdk vercel-ai-sdk bot removed the backport label Oct 17, 2025
@vercel-ai-sdk
Copy link
Contributor

vercel-ai-sdk bot commented Oct 17, 2025

⚠️ Backport to release-v5.0 created but has conflicts: #9614

lgrammel added a commit that referenced this pull request Oct 17, 2025
)

This is an automated backport of #9330 to the release-v5.0 branch.

---------

Co-authored-by: Jonas Lalin <[email protected]>
Co-authored-by: Lars Grammel <[email protected]>
@jonaslalin jonaslalin deleted the jonas/cache-control branch October 20, 2025 12:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants