How do you cache to cut DB load at millions of req/min?

Question

IT Interview Practice · Accepted Answer

Put a **cache-aside** layer (Redis/Memcached) in front of the DB so reads hit memory, give entries a **TTL** and **invalidate on write**, add **stampede protection** so a cold key can't unleash a thundering herd on the DB, and spread **hot keys** across replicas so one popular key doesn't melt a single node.

```text
                 ┌── hit ──▶ return value
request ──▶ cache│
                 └── miss ─▶ lock ─▶ DB read ─▶ set(TTL) ─▶ return
                              │
                    (only ONE miss populates; others wait)
DB writes ──▶ invalidate/update key
```

## Cache-aside vs read-through

In **cache-aside** the app checks the cache; on a miss it reads the DB and populates the cache itself. In **read-through** the cache library does that for you. Either way the goal is the same: turn a 5ms DB query into a 0.2ms memory lookup, so a cache with a 95% hit rate cuts DB read traffic **20×**. At millions of req/min the DB simply cannot serve every read — the cache is what makes the numbers work.

## TTL + invalidation

Two ways to keep the cache fresh, and you usually need both. **TTL** bounds staleness — an entry expires after N seconds so wrong data self-heals. **Explicit invalidation** deletes/updates the key on write for correctness that can't wait for TTL. The classic bug is caching without invalidation and serving stale data for minutes; the subtler one is invalidating *and* setting in the same write, which races.

## Stampede protection & hot keys

- **Stampede (thundering herd):** when a hot key expires, thousands of concurrent misses all hit the DB at once and can knock it over. Fix with a **per-key lock/single-flight** (only one request recomputes, others wait) or **early/probabilistic refresh** before expiry.
- **Hot keys:** one key (a celebrity's profile) can saturate a single Redis node's CPU/network. Spread it — replicate the key across nodes, or add a small local in-process cache in front of Redis for the hottest items.

## Interview perspective

The interviewer wants to see you think past "add Redis". Good: cache-aside with TTL. Great: you name **stampede protection, invalidation strategy, hot-key handling, and what happens on a cache outage**. The trap is assuming the cache is always up — if Redis dies and every miss falls through to the DB, you get a **cache-avalanche** that takes the DB down too.

## Trade-offs & failure modes

- **Staleness vs load:** short TTL = fresh but more misses; long TTL = fewer misses but staler. Tune per data.
- **Cache avalanche:** many keys expiring together (same TTL) causes a load spike — **jitter** the TTLs.
- **Consistency:** cache and DB can diverge; for money-critical reads, read the DB or use write-through.
- **Memory eviction:** under pressure Redis evicts by policy (LRU/LFU); a wrong policy silently drops your hot set.

Cache-aside vs read-through

TTL + invalidation

Stampede protection & hot keys

Interview perspective

Trade-offs & failure modes

Related questions