The 4-Layer Prompt Cache Fix That Cut LLM Costs 82%

Most LLM cost problems are cache problems in disguise. After rebuilding a 4-layer caching architecture in production, I reduced inference costs by 82% without changing a single prompt.

Why your cache isn't working

Most teams add caching as an afterthought — a Redis layer here, a hash comparison there. The problem is LLM caching has to happen at the prompt level, not the response level.

The 4-layer architecture

Layer 1: system prompt cache. Layer 2: context window cache. Layer 3: retrieved document cache. Layer 4: response deduplication. Each layer has a different TTL and invalidation strategy.

The numbers

Before: $4,200/month in Claude API costs for 12 engineers. After: $756/month. Same usage, same quality. The fix took 6 days.

What to check in your stack

Run through this checklist before optimising anything else...