What is a thread pool? Why use one, and how do you size it?

Question

IT Interview Practice · Accepted Answer

**A thread pool is a fixed set of reusable worker threads that pull tasks from a shared work queue.** It avoids the cost of creating a thread per task and caps the number of concurrent threads.

## Why use one

Creating and destroying threads is expensive (stack allocation, OS scheduling setup), and unbounded thread creation exhausts memory and causes context-switch thrashing. A pool amortizes creation cost and bounds concurrency.

```text
tasks →  [ work queue ]  →  worker 1
                            worker 2   (N fixed workers pull & run)
                            worker 3
```

## Example

```python
from concurrent.futures import ThreadPoolExecutor
with ThreadPoolExecutor(max_workers=8) as pool:
    futures = [pool.submit(handle, req) for req in requests]
    # 8 threads process the queue; extras wait their turn
```

## Sizing

- **CPU-bound** work: pool size ≈ number of cores. More threads than cores just adds context-switch overhead with no parallelism gain.
- **I/O-bound** work: size larger, roughly `cores × (1 + wait_time / compute_time)`, because threads spend most time blocked and can overlap.
- **Bound the queue** too: an unbounded queue hides overload and leads to OOM; a bounded queue with a rejection policy (drop, block, or shed) applies backpressure.

## Why it matters

Pools are the backbone of servers and executors. Wrong sizing is a classic performance bug: too few threads underutilizes CPU; too many causes thrashing and memory pressure. Matching pool size to the workload type (CPU vs I/O) and bounding the queue keeps the system stable under load.

Why use one

Example

Sizing

Why it matters

Related questions