Prompt caching gets pitched as close to free money: pay a small premium once, then get the same content back at a fraction of the price on every later call. That pitch holds only under a specific condition — something has to actually read the cache back before it expires — and that condition is exactly the kind of thing worth auditing rather than assuming. Pulling the real usage numbers on our own content pipeline turned out to be precisely backwards from what the pitch implies: across a week of runs, the pipeline was paying a 1.2x write premium while recouping it on only 0.7% of those writes with an actual cache read. Turned on as a discount, running as a surcharge.
Why "cache everything" isn't automatically the right default
The mechanism prompt caching actually uses matters more than the marketing framing. Anthropic's own pricing documentation, verified against the current rates, lays it out precisely: a 5-minute cache write costs 1.25x the normal input-token price, a 1-hour write costs 2x, and a cache read — an actual hit — costs only 0.1x. That 0.1x read price is genuinely cheap. But you only pay it if something reads the cache back before it expires. If nothing does, you've paid the 1.25x or 2x write premium for a cache nobody ever used, which is strictly worse than sending the same prompt uncached at the standard 1x rate.
The prompt caching documentation is explicit that a 5-minute cache resets its clock for free on every hit within the window, but that's exactly the condition our calls didn't meet. Our pipeline's calls to the same system prompt were one-shot per article and spaced well beyond five minutes apart — sometimes hours, depending on when a given article happened to run. Every call paid to write a cache that had already expired by the time anything could have read it back, because nothing was calling back into that same prefix inside the window that would have made the write pay for itself.
Why this went unnoticed for so long
Caching failing to help doesn't look like a failure. There's no error, no degraded output, nothing that trips a quality gate — the call succeeds, the article gets produced, and the only trace of the problem is a marginally higher line item buried in a token-cost breakdown that nobody was reading closely enough to compare against what it should have been. Turning caching on felt like a strict improvement, so nobody built a check for the case where it wasn't. We only found the real ratio by pulling the account's own usage analytics and comparing cache-write volume against cache-read volume directly — a comparison that has to be actively made, because nothing surfaces it on its own.
The one case worth calling out separately: one of our downstream review steps runs a much shorter system prompt, under roughly a thousand tokens. That call was already excluded from caching before we found the broader problem, for a distinct and specific reason — very short prompts fall below the minimum prefix length a cache write actually stores, so turning caching on for that call wouldn't produce an error, it would just silently write nothing and cost nothing extra either way. Small individually, but a good reminder that "does caching help here" has more than one way to come back false, and the failure state in every case looks identical to success from the outside.
What we changed
The fix flipped the default rather than tuning it. Caching is opt-in now, applied only to calls where reuse inside the cache's window is actually likely — not applied blanket-wide on the assumption that more caching is always better. In practice that meant identifying the one call pattern in our pipeline that genuinely repeats a large prompt multiple times within a few seconds of each other, and confirming caching actually earns its keep there specifically: the first call writes the cache, the second and third calls (same system prompt, different task-specific content, run moments later) read it back. Measured directly, that pattern took the per-call cost on those repeated calls from roughly $0.028 to $0.002 — the write premium paid for itself many times over, because the reuse the mechanism depends on was actually present.
That's the test we now apply before switching caching on anywhere: does this specific call pattern genuinely reuse the same prefix inside the cache's TTL, more than once, reliably? If yes, cache it and expect real savings. If the calls are one-shot, or spaced further apart than the TTL, or short enough to sit under the minimum cacheable length, caching isn't a free lever there — it's a cost with no offsetting benefit, and the honest move is to leave it off rather than switch it on because the concept sounds like it should help.
The broader pattern
Any cache — an HTTP cache, a database query cache, a compiled-artifact cache — earns back its own overhead only when something reuses it before it expires. That's not a caveat particular to LLM prompt caching; it's true of caching generally, and it's exactly the property that's easy to forget when the underlying story ("reuse this instead of recomputing it") sounds unconditionally beneficial. Turning a cache on is a bet that reuse will actually happen inside the window that makes it profitable. Verify the bet with real usage numbers before assuming it paid off, the same way you'd verify any other cost-relevant change — caching is a lever with a real cost attached, not a setting that's free to leave on everywhere just because the upside case sounds appealing.
FAQ
Is prompt caching ever worth turning on by default?
Only where the underlying call pattern is already known to repeat the same large prefix multiple times within the cache's TTL — a multi-step pipeline hitting the same system prompt seconds apart is the clean case. For anything closer to one-shot, or for calls whose spacing you haven't actually measured against the TTL, verify the reuse pattern first rather than defaulting it on.
What's the actual breakeven point for a 5-minute cache?
At a 1.25x write cost and a 0.1x read cost against a 1x uncached baseline, a single cache read already more than recoups the write premium — two total calls (one write, one read) cost 1.35x combined versus 2x for the same two calls uncached. The entire question is whether that second call actually happens inside the five-minute window; if it doesn't, you've paid the 1.25x and captured none of the 0.1x benefit.
Does a short system prompt ever benefit from caching?
Not below the model's minimum cacheable prefix length — a cache write below that threshold simply doesn't store anything, so there's no error and no benefit either way. Below that length, caching is a no-op you can skip without checking further.
How do you actually measure whether caching is paying off, rather than guessing?
Compare cache-write volume against cache-read volume directly from the account's own usage data, not from assuming the feature works because nothing looks broken. A healthy ratio has meaningfully more reads than writes; a ratio close to all writes and almost no reads is the exact signature of paying the premium and never collecting the discount.
Where to go from here
This is the third lesson in the Build with Claude course — see the env var that tripled our AI coding bill for the previous lesson in this cost module, and how to save tokens with Claude Code more broadly for the wider set of tactics this sits alongside. Work with me or subscribe to the newsletter for the rest of this series as it ships.