Why can Nginx exhaust connections while CPU usage is low?

Question

IT Interview Practice · Accepted Answer

A connection can occupy resources while waiting, even when CPU usage is low. First identify which limit was reached instead of assuming the machine needs more processing power.

## Count both sides of a proxy request

```text
Client -- socket A --> Nginx -- socket B --> Backend
```

A proxied HTTP/1.x request can use one client connection and another upstream connection. If the backend stalls, both can remain occupied. `worker_connections` includes upstream connections, so its value is not simply the number of users supported.

## Turn the error into a check

- `worker_connections are not enough`: inspect the configured per-worker connection limit and current connections.
- `Too many open files`: inspect the worker's effective file-descriptor limit. A descriptor is an operating-system handle used for sockets and files.
- Rising upstream response times: investigate the backend work keeping those connections occupied.

For illustration, 1,000 simultaneously active proxied requests might need roughly 2,000 sockets across the proxy sides. This is not a sizing formula: HTTP/2, keepalive reuse and idle connections change the relationship.

## Increase capacity only after finding the cause

Fix stalled backend work or bound waiting requests first. If healthy traffic exceeds the limits, align worker settings, process limits and memory, then load-test. Confirm successful throughput and latency improve rather than merely allowing a longer queue.

[Official documentation](https://nginx.org/en/docs/ngx_core_module.html#worker_connections).

Count both sides of a proxy request

Turn the error into a check

Increase capacity only after finding the cause

Related questions