At scale, throughput is limited less by CPU count than by contention, cache effects, and unbounded queues — adding threads past a point makes things slower. Amdahl's and the Universal Scalability Law both predict this: coordination cost eventually dominates.
When many threads fight for one lock or cache line, they serialize and spend time waiting instead of working. Under the Universal Scalability Law, contention plus coherency cost can make throughput past an optimal thread count.
Mitigations: shard/partition data, per-thread state (thread-local, striped counters like LongAdder), read-write locks, or lock-free structures.
Two independent variables that happen to sit on the same CPU cache line cause cores to invalidate each other's caches on every write, even though they never touch the same data — a silent, brutal slowdown.
cache line (64B): [ counterA | counterB ] ← two cores ping-pong the line
Fix: pad so each hot variable owns its own line
@Contended long counterA; // JVM pads to a separate cache line
When producers outpace consumers, an unbounded queue grows until OOM and latency explodes. Backpressure signals producers to slow down: bounded queues that block or reject, credit-based flow control, or reactive-streams request(n).
producer ──▶ [bounded queue full] ──▶ block/reject ──▶ producer slows
These are the effects that separate "works on 4 cores" from "scales to 64." Naive designs hit a wall: locks serialize, cache lines thrash, queues balloon. Sharding to reduce contention, padding to kill false sharing, and backpressure to bound queues are the levers that keep latency and throughput healthy under real load.
A library of IT interview questions with detailed answers — from Junior to Senior.
Donate