What is a cache stampede and how do you prevent it?

Question

Accepted Answer

A **cache stampede** (or dogpile) happens when a hot cached key expires and **many concurrent requests miss it at once**, all falling through to the origin to recompute the same value simultaneously. The origin (DB or service) gets hammered by a burst it can't handle, sometimes taking the system down.

## The cause

```text
t=0   hot key expires
t=0+  1000 requests arrive → all miss → all query the DB for the SAME value
      → DB overwhelmed; latency spikes; failures cascade
```

The trigger is **simultaneous expiry** of a popular key. The fix is to ensure only one (or few) recomputation happens, or that requests don't all expire together.

## Fixes

- **Locking / single-flight** — the first miss takes a lock and recomputes; everyone else waits for that one result instead of all hitting the origin.
- **Stale-while-revalidate** — serve the slightly stale value immediately while one background task refreshes it. No request blocks on the origin.
- **Early / probabilistic recomputation** — refresh a key *before* it expires, with a probability that rises as expiry nears, so one lucky request rebuilds it ahead of the stampede.
- **Jittered TTLs** — add randomness to TTLs so keys don't all expire at the same instant.
- **Request coalescing** — collapse identical concurrent requests into one origin call.

## Single-flight pseudo-code

```text
function get(key):
  val = cache.get(key)
  if val is present: return val            // fast path: a hit

if lock.acquire(key, ttl=5s):            // only ONE request wins the lock
    try:
      val = recompute()                    // hit the origin exactly once
      cache.set(key, val, ttl)
    finally:
      lock.release(key)
    return val
  else:
    wait_briefly(); return get(key)        // others wait, then read the filled cache
```

The lock ensures exactly one origin call; every other concurrent request waits and then reads the freshly cached value.

## Why it matters

A stampede turns a routine cache expiry into an outage: the cache that was protecting your DB suddenly stops, and full traffic lands on the origin at once. Knowing the cause (simultaneous expiry of hot keys) and the fixes (single-flight, stale-while-revalidate, jitter) is essential for keeping caches reliable under real load.