What are flaky tests and how do you deal with them?

Question

Accepted Answer

**Flaky tests** are tests that **pass or fail inconsistently** without code changes — sometimes passing, sometimes failing on the same code. They're a serious problem because they erode trust in the test suite. Understanding their causes and fixes is important.

## What flaky tests are and why they're harmful

```text
A FLAKY test gives INCONSISTENT results (pass sometimes, fail other times) on the SAME code:
  → harmful: ERODES TRUST — people start ignoring failures ("oh, it's just flaky") →
    real failures get missed too
  → waste time on false alarms / re-runs; break CI; reduce confidence in the whole suite
→ Flaky tests are worse than no test if they make people distrust all tests.
```

## Common causes

```text
✗ TIMING / async → race conditions; not waiting properly for async operations (a top cause
  in UI/E2E tests); arbitrary sleeps
✗ ORDER DEPENDENCE → tests depending on each other / shared mutable state
✗ EXTERNAL dependencies → real network/services (network blips, rate limits, downtime)
✗ NON-DETERMINISM → time/dates, randomness, timezone, locale
✗ Test ENVIRONMENT → leftover state, uncleaned data, concurrency/parallelism issues
```

## Fixing and managing flaky tests

```text
✓ Fix the ROOT CAUSE: wait for conditions (not sleeps); make tests INDEPENDENT and clean
  up state; MOCK external dependencies; control time/randomness (inject them)
✓ Ensure proper isolation (no shared state, no order dependence)
✓ Don't just RETRY blindly (it hides the problem) — investigate and fix
✓ QUARANTINE persistently flaky tests (isolate so they don't block) WHILE fixing them
✓ Treat flakiness seriously — track and address it (it degrades the whole suite)
```

## Why it matters

Understanding flaky tests and how to deal with them is important because **flaky tests are a serious, common problem that undermines the value of an entire test suite**, so it's valuable practical knowledge.

A flaky test (passing or failing inconsistently on the same code) is particularly harmful because it **erodes trust**: when tests fail unpredictably, people start ignoring failures ("it's just flaky"), which means **real failures get missed too** — so flakiness degrades confidence in the whole suite, potentially making it worse than useless.

Understanding this harm motivates taking flakiness seriously.

Understanding the **common causes** — **timing/async issues** (race conditions, not waiting properly for async operations, arbitrary sleeps — a top cause especially in UI/E2E tests), **order dependence** (tests depending on each other or shared state), **external dependencies** (real network/services with their blips and downtime), **non-determinism** (time, randomness, timezone, locale), and **environment issues** (leftover state, concurrency) — is necessary to diagnose flakiness.

Understanding how to **fix and manage** them — fixing root causes (waiting for conditions instead of sleeps, making tests independent and clean, mocking external dependencies, controlling time and randomness), ensuring isolation, **not blindly retrying** (which hides problems), quarantining persistently flaky tests while fixing them, and treating flakiness as a serious issue to track and address — reflects the right approach.

Since flaky tests are a common, serious problem that undermines trust in and the value of a test suite (a real issue in many real-world projects), and since understanding their causes and proper handling (fixing root causes, isolation, not just retrying) is necessary to maintain a trustworthy test suite, understanding flaky tests is valuable, practically-important knowledge — addressing a significant real-world testing problem, important for keeping a test suite reliable and trusted, and reflecting the practical maturity to handle one of testing's most frustrating and damaging issues.