What is a distributed cache, and how do you handle its failures?

Question

IT Interview Practice · Accepted Answer

**A distributed cache lets multiple application instances reuse data over the network.** Its purpose is to avoid repeating expensive work, while keeping clear rules for freshness and failure.

Imagine three backend instances reading the same product catalog. With a cache inside each process, each instance maintains its own copy. With a shared cache service, one instance can populate a value that the others reuse. The cache service may itself use several nodes.

## 1. Decide what can safely be cached

Start with the cost of producing the data and how stale it may be. A product description may tolerate a short delay; the final stock check for a purchase may require authoritative data.

Define a key that identifies the complete result, including tenant, language or other relevant inputs. Otherwise, a fast cache can return the wrong customer's data. Keep the database authoritative for durable business records, because cached values may disappear.

## 2. Follow the read path before designing updates

A common approach is cache-aside: the application checks the cache, reads the database on a miss, then stores the result with an expiry time, called TTL.

```text
Request -> Cache -> hit  -> Return value
                 -> miss -> Database -> Fill cache -> Return value
```

This reduces repeated database reads when many requests need the same value. It adds network calls and another dependency, so a cache is useful only when the avoided work outweighs those costs. See [Redis cache-aside](https://redis.io/docs/latest/develop/use-cases/cache-aside/).

## 3. Decide how updates become visible

For the catalog, one practical starting point is to commit a database update and then invalidate the cached product. The next reader reloads it. But this sequence still has a race: a reader can fetch the old database value before the commit, then put it into the cache after invalidation.

TTL limits how long that newly cached copy survives; it does not guarantee that every read immediately sees the latest write. If stale permissions, prices or balances would violate the operation's rules, validate against authoritative data or design a version-aware update mechanism with explicit guarantees. Choose consistency according to the business operation rather than assuming that using Redis solves it.

## 4. Prevent a cache miss from becoming an outage

If a popular entry expires, many callers may all read the database at once. This is a cache stampede. A cache outage can create the same problem across many entries.

- Combine concurrent loads for the same key so fewer callers rebuild it at once; define whether coordination covers one process or all instances.
- Vary expiry times to avoid many entries expiring together. This spreads refresh work but does not solve every hot-key problem.
- Serve a previously cached value during refresh only when the business permits stale data.
- Use short cache timeouts and bound database fallback traffic. Rejecting some work may be safer than overwhelming the database and failing every request.

Test these paths deliberately. A high hit rate during normal operation says little about how the service behaves when the cache disappears.

## 5. Scale according to the actual bottleneck

Sharding divides keys among cache nodes; replication keeps copies for availability or reads. They solve different problems. Redis Cluster uses hash slots for placement, so adding nodes does not automatically divide the work for one hot key. See the [Redis Cluster specification](https://redis.io/docs/latest/operate/oss_and_stack/reference/cluster-spec/).

Watch hit rate, request latency, timeouts, memory, evictions and the load arriving at the database. An eviction means the cache removed a value to manage memory and can happen before its TTL. Do not treat a cache entry as guaranteed durable storage for business data or queued work.

For the catalog, I would begin with explicitly keyed cache-aside data, a freshness policy and protected fallback. I would add more cache nodes only after measurements show a capacity problem that distribution can actually solve. **A good cache design explains both what happens when a value is present and what happens when it is absent or wrong.**

1. Decide what can safely be cached

2. Follow the read path before designing updates

3. Decide how updates become visible

4. Prevent a cache miss from becoming an outage

5. Scale according to the actual bottleneck

Povezana vprašanja