How do Nginx upstreams balance traffic and handle failed nodes?

Question

IT Interview Practice · Accepted Answer

Define an upstream group so Nginx can send requests to multiple application instances. Then decide how to distribute work and when to temporarily avoid a failing instance.

## Start with two application servers

Inside `http`, the following illustrative configuration distributes traffic to two private addresses:

```nginx
upstream app {
    server 10.0.0.11:3000 max_fails=2 fail_timeout=10s;
    server 10.0.0.12:3000 max_fails=2 fail_timeout=10s;
}
server {
    listen 80;
    location / {
        proxy_pass http://app;
    }
}
```

With equal weights, the default round-robin method takes turns across servers. The example's `max_fails` and `fail_timeout` let qualifying failures temporarily exclude a server. The numbers illustrate behavior; choose production values from your traffic and failure tolerance.

## When would you change the method?

If some requests keep connections busy much longer, consider adding `least_conn;` to the upstream block. It prefers fewer active connections. Neither method shares session data: if login sessions live on one instance, move them to shared storage or deliberately design session affinity.

## Test failure rather than assuming recovery

Stop one test instance and inspect which upstream receives requests. Passive failure detection learns from actual requests; periodic health probes depend on available features. Bound retries and never blindly repeat an uncertain payment: the first attempt may already have succeeded.

[Official documentation](https://nginx.org/en/docs/http/ngx_http_upstream_module.html).

Start with two application servers

When would you change the method?

Test failure rather than assuming recovery

Related questions