Guide · 2026-04-14 · 6 min read

Prompt Caching: How It Works and How Much You Save

A deep dive into prompt caching across OpenAI, Anthropic, and Google — how prefix caching works, when it kicks in, and real savings benchmarks.

What Is Prompt Caching?

Every time you call an LLM API, you send a system prompt, context, and user input. If your system prompt is 3,000 tokens and you make 1M requests/month, you're paying for those same 3,000 tokens a million times. Prompt caching fixes that.

Providers detect when the beginning of your prompt (the "prefix") matches a previous request. Instead of reprocessing those tokens from scratch, they reuse cached computations and charge you a fraction of the normal input price.

Provider Comparison

Anthropic's approach gives you more control — you explicitly mark cache breakpoints in your prompt. OpenAI and Google cache automatically based on prefix matching.

How to Structure Prompts for Caching

The golden rule: put static content first, dynamic content last.

If you shuffle the order — say, putting the user question before examples — caching breaks because the prefix no longer matches.

Real Savings Example

Let's model a RAG chatbot with a 2,500-token system prompt making 500K requests/month on GPT-5:

  • Without caching: 2,500 × 500K = 1.25B input tokens × $2.50/M = $3,125
  • With caching: 2,500 tokens cached at $1.25/M = $1,562

Savings: $1,563/month — just from reordering your prompt.

When Caching Doesn't Help

  • Unique prefixes: If every request starts differently, there's nothing to cache
  • Short prompts: If your system prompt is under the minimum prefix length, caching won't activate
  • Low volume: Cache entries expire (typically 5-10 minutes of inactivity), so sporadic traffic won't benefit

Combining with Other Optimizations

Prompt caching stacks beautifully with tiered routing. Cache your system prompt on every tier, and you get compound savings:

1. Route simple queries to GPT-5 Nano ($0.10/M → $0.05/M cached)

2. Route complex queries to GPT-5 ($2.50/M → $1.25/M cached)

Use our Calculator to model caching savings for your specific workload.

Related Reading