The core mechanism.

When you send a prompt to a language model, the provider has to tokenize your input, run it through the model's attention layers, and compute intermediate states (called KV cache states). This work happens for every request, even if you just sent the same 10,000-token system prompt two seconds ago. Prompt caching breaks that wastefulness. The provider stores the computed states of your prefix server-side. When another request comes in with the same prefix, the provider loads the cached state and only processes the new tokens.

The economics are simple. Processing 10,000 tokens costs compute. Serving them from a warm cache costs effectively nothing. Providers pass most of that savings back to you. Anthropic charges 10% of the standard input rate for cache reads. OpenAI charges 50%. Gemini ranges from 10-25% depending on model generation.

There is a small first-call tax. On Anthropic, the initial cache write costs 25% more than standard input (1.25x multiplier on Sonnet and Opus). On OpenAI, cache writes are free. On Gemini explicit caching, writes are free but you pay storage fees per hour. Break-even for Anthropic is the second call. After that, every subsequent call is pure savings.

What should go in the cache.

Caching works on the prefix of your prompt, meaning anything that appears at the start of your message and stays identical across requests. The order of content matters enormously. Put static content first. Put dynamic content last.

Things worth caching include system prompts over 500 tokens, tool and function schema definitions, RAG document context that multiple queries reference, few-shot example blocks, long persona or brand guideline documents, and full codebases when using coding assistants. The common pattern: anything substantial that does not change based on the current user turn.

Things not worth caching include user messages (they change every call), short system prompts under 1,024 tokens (below the minimum threshold), and content that varies even slightly per request. Even a single different token in the prefix breaks the cache match.

Anthropic (explicit, deepest discount).

Anthropic caching is explicit but generous. You add a cache_control marker to each block you want cached. You can place up to four breakpoints per request. Each marker caches everything from the start of the message up to that point.

Anthropic caches default to a 5-minute TTL that extends every time the cache is accessed. For workloads where requests are spaced more than 5 minutes apart, switch to the 1-hour TTL by adding "cache_control": {"type": "ephemeral", "ttl": "1h"}. The 1-hour cache costs 2x base input instead of 1.25x on write, but hits far more often for moderate-frequency workloads.

A critical debugging detail: cache is scoped to your API key and region. Requests routed through different Anthropic regions (us-east-1 vs eu-west-1 on Bedrock) maintain separate caches. If you are using a multi-region deployment and cache hits look low, check your routing.

OpenAI (automatic, zero code).

OpenAI caching is the simplest of the three. There is no flag, no config, no API changes. Once your prompt exceeds 1,024 tokens, caching activates automatically. Cache hits happen in 128-token increments (so 1,024, 1,152, 1,280, and so on). Discount is 50% on cached tokens.

OpenAI cache TTL is 5-10 minutes and extends during off-peak hours up to around one hour. You cannot control TTL directly. The tradeoff is simplicity: zero code changes versus less control.

One sharp edge: OpenAI cache is automatic but not guaranteed. If the system is under load, caching can be skipped silently. Monitor cached_tokens in your response usage to confirm hits. Target 80%+ hit rate on production workloads. If you see lower rates, your prefix may be changing between requests in ways you did not intend (timestamps, dynamic IDs, user-specific context at the top of the prompt).

Google Gemini (hybrid, longest TTL).

Gemini supports both implicit caching (automatic, on Gemini 2.5 models) and explicit caching (manual, higher discount). The explicit mode requires creating a cache object via the API, referencing it in subsequent requests, and deleting it when done. Storage is billed per hour, not free like Anthropic and OpenAI.

The Gemini storage fee is $1 per million tokens per hour on 2.5 Pro ($4.50/M/hour on 1.5). For a 100K token cache held for 1 hour, that is $0.10. Trivial if you make 100 requests against it. Wasteful if you make 2.

Gemini 2.5 implicit caching activates automatically when your prompt matches a previously-seen prefix, with no API changes required. The discount is the same 90% but you do not control the lifetime. For unpredictable workloads, implicit is safer. For predictable high-volume workloads, explicit gives you control over TTL and confirmed hit rates.

The break-even math.

On Anthropic, first-call cache write costs 1.25x base input. Cache reads cost 0.10x. Break-even is the second call. If your prefix gets reused at least once within the TTL window, you are net positive. The more reuse, the better the economics: 3 calls with 100% hit rate saves roughly 60% off the prefix cost. 10 calls saves roughly 82%. 100 calls saves roughly 89%.

On OpenAI, there is no write premium. Break-even is call one. Every cache hit immediately saves 50%. The downside is less control: 5-10 minute TTL, no way to extend intentionally, no guarantee the system will cache during peak load.

On Gemini explicit caching, break-even depends on the storage cost amortization. For a 100K token cache at $1/M/hr storage, held for 1 hour, costs $0.10. Against a $0.20 per request base cost saved, break-even is call 2. But if you only make one request per hour, you are paying storage for no benefit. Implicit caching avoids this problem entirely at the cost of less control.

The workloads where caching fails.

Caching does not save money in several common scenarios that teams routinely get wrong.

Spread-out requests. If your workload makes one request every 30 minutes, the 5-minute Anthropic cache expires between calls. Every request becomes a cache miss with write premium. Switch to 1-hour TTL or accept that caching does not fit.

Output-heavy workloads. Caching only reduces input costs. If you send 1,000 input tokens and generate 10,000 output tokens per call, caching saves maybe 3% of total cost. Not worth the setup complexity.

Dynamic prefixes. Putting the current timestamp, user ID, or session info at the start of your system prompt breaks cache matching even if the rest of the prompt is identical. Move dynamic content to the user message, keep the system prompt pure.

Sub-minimum prefixes. On Anthropic and OpenAI, prompts under 1,024 tokens cannot be cached. On Gemini explicit caching, under 32,768 tokens. Check your prefix length before investing in caching architecture.

Stacking caching with batch processing.

The biggest savings come from stacking. Anthropic batch and OpenAI batch both offer 50% off standard token prices in exchange for 24-hour async processing. Batch discounts apply on top of cache discounts.

Worked example on Sonnet 4.6 with a 10M input token workload:

Standard pricing: 10M input tokens at $3/M = $30.00.
Batch only (50% off): $15.00.
Cache only (90% hit rate): 9M cached at $0.30/M + 1M standard at $3/M = $2.70 + $3.00 = $5.70.
Batch + Cache stacked: 9M cached at $0.15/M (batch halves cache price too) + 1M batch standard at $1.50/M = $1.35 + $1.50 = $2.85.

Total savings versus standard: 90.5%. For any offline processing pipeline (data enrichment, classification, document summarization, evaluation runs), the batch+cache stack is the single highest-ROI change you can make to your API bill.

Monitoring cache performance.

Caching is one of those optimizations that looks working until it is not. Hit rates silently degrade for dozens of reasons: a teammate added a timestamp to the system prompt, a model version upgrade invalidated caches, a region change broke key scoping. Monitor every response.

On Anthropic, check response.usage.cache_read_input_tokens and response.usage.cache_creation_input_tokens. Read count should dominate after the first few calls. On OpenAI, check response.usage.prompt_tokens_details.cached_tokens. Should be roughly equal to your prefix length minus the 128-token remainder.

Target cache hit rates by use case: chatbot with shared system prompt 95%+, RAG with document rotation 60-80%, batch processing with same prompt 99%+. If you are below these thresholds, investigate. The issue is almost always an inadvertently dynamic prefix, not a caching bug.