What is backpressure in streams and how do you handle it?

Question

Accepted Answer

**Backpressure** is what happens when data is produced **faster than it can be consumed**. In streams, if a fast readable source pushes data into a slow writable destination, the unconsumed data buffers in memory — and without handling, that buffer grows until memory is exhausted. Backpressure is the mechanism (and discipline) for keeping producer and consumer in balance.

## The problem

```text
Fast source ──(1 GB/s)──▶ Slow destination (10 MB/s write speed)
                          ▲
             The 990 MB/s difference piles up in a memory buffer → OOM crash
```

Example: reading a huge file fast and writing it to a slow network socket or disk — if you don't respect the destination's pace, memory balloons.

## pipe handles backpressure automatically

```js
import { createReadStream, createWriteStream } from "fs";

// ✅ pipe automatically pauses the source when the destination is full,
//    and resumes it when the destination drains — backpressure handled for you
createReadStream("huge.file").pipe(createWriteStream("copy.file"));
```

This is a major reason to use `pipe`/`pipeline`: they manage backpressure for you, pausing and resuming the flow so memory stays bounded regardless of size mismatch.

## Handling it manually with write()'s return value

```js
// write() returns FALSE when the internal buffer is full — that's the backpressure signal
readable.on("data", (chunk) => {
  const ok = writable.write(chunk);
  if (!ok) {
    readable.pause();                       // stop reading — destination is full
    writable.once("drain", () => readable.resume()); // resume when it drains
  }
});
```

If you handle streams manually, the rule is: when `writable.write()` returns `false`, **pause** the readable, and **resume** it on the writable's `drain` event. Ignoring the `false` return is the classic backpressure bug.

## With async iteration (modern, clean)

```js
// for-await respects backpressure naturally — it awaits each chunk's processing
for await (const chunk of readable) {
  await processSlowly(chunk); // only pulls the next chunk when ready
}
```

## Prefer pipeline for safety

```js
import { pipeline } from "stream/promises";
await pipeline(source, transform, destination); // backpressure + error handling + cleanup
```

## Why it matters

Backpressure is a subtle but critical concept for any data-intensive Node service (file processing, proxying, streaming, ETL).

Ignoring it — by buffering everything or ignoring `write()`'s return value — causes unbounded memory growth and crashes under load, often only in production with large data.

Understanding that `pipe`/`pipeline` handle it automatically (the main reason to prefer them), and how to respect it manually (pause on `false`, resume on `drain`) or via `for await`, is what lets you build streaming systems that stay memory-stable no matter how large or fast the data is.

It's a defining mark of competent stream usage.