What are overfitting and underfitting?

Question

Accepted Answer

**Overfitting** (the model memorizes training data and fails on new data) and **underfitting** (the model is too simple to capture the patterns) are two fundamental problems in ML. Balancing them — achieving good generalization — is central to building effective models.

## Overfitting vs underfitting

```text
OVERFITTING → the model learns the training data TOO well (including noise) →
  → performs great on training data but POORLY on new/unseen data (doesn't generalize)
  → too complex; memorizes rather than learns general patterns
  → like memorizing answers vs understanding the concept
UNDERFITTING → the model is TOO SIMPLE to capture the underlying patterns →
  → performs poorly on BOTH training and new data
  → not enough complexity/capacity to learn the patterns
→ the goal is GENERALIZATION: learn real patterns → perform well on NEW data
```

## Recognizing them

```text
→ OVERFITTING → high training accuracy, LOW test accuracy (big gap)
→ UNDERFITTING → LOW training AND test accuracy (poor overall)
→ GOOD FIT → good training AND test accuracy (generalizes well)
→ the train-vs-test performance gap reveals overfitting
```

## Addressing them

```text
OVERFITTING → simplify or regularize:
  → more training DATA; REGULARIZATION (penalize complexity); simpler model; dropout (NNs);
    early stopping; cross-validation
UNDERFITTING → increase capacity:
  → a more complex model; better FEATURES; train longer; reduce regularization
→ balance model complexity to fit the data without memorizing (the bias-variance trade-off)
```

## Why it matters

Understanding overfitting and underfitting is valuable because they're **fundamental problems in ML** that determine whether models actually work, so understanding them is essential ML knowledge.

These two problems are central to building effective models.

Understanding **overfitting** (the model memorizing training data including noise, performing well on training but poorly on new data — being too complex and memorizing rather than learning general patterns) and **underfitting** (the model being too simple to capture patterns, performing poorly on both training and new data) clarifies the two failure modes, with the goal being **generalization** (learning real patterns to perform well on new data).

Understanding how to **recognize them** — overfitting showing high training but low test accuracy (a big gap), underfitting showing low accuracy on both, and good fit showing good accuracy on both — is practical knowledge for diagnosing model problems, with the train-vs-test gap being the key signal of overfitting.

Understanding how to **address them** — overfitting via more data, regularization, simpler models, dropout, early stopping, and cross-validation; underfitting via more complex models, better features, and training longer — provides the practical toolkit for fixing these problems, balancing model complexity (the bias-variance trade-off).

Overfitting in particular is a pervasive, critical problem in ML (models that work in training but fail in production), and understanding it (and underfitting) is essential for building models that actually generalize and work on real data.

Since overfitting and underfitting are fundamental ML problems determining whether models actually work (generalize to new data) and understanding them, recognizing them, and addressing them is essential for building effective models, understanding overfitting and underfitting is valuable, essential ML knowledge — fundamental problems central to building models that work (generalize), with the train/test gap diagnosing overfitting and the techniques to address both, essential for anyone building or understanding ML models.