When should you summarize the conversation history instead of resending everything?

Question

IT Interview Practice · Accepted Answer

Summarize the conversation when the history grows long and earlier detail is no longer needed for the current task. Instead of resending the entire transcript on every turn, you replace older, resolved parts with a short summary that keeps the **key decisions and current state** and drops resolved tangents.

## Why summarize

- **Context window** — every model has a finite context. A long, unedited history eventually overflows it and the oldest messages get truncated anyway.
- **Cost and latency** — you pay (in tokens) and wait for every message you resend. A 50-message thread reprocessed each turn is slow and expensive; a tight summary is cheap.
- **Focus** — irrelevant back-and-forth (a bug you already fixed, an idea you rejected) distracts the model from what matters now.

## What to keep vs drop

```text
KEEP → decisions made ("we chose Postgres"), constraints ("must run on Node 18"),
        current state ("auth works; now wiring the dashboard"), open questions
DROP → resolved tangents, dead-end attempts, long pasted output already acted on,
        polite filler
```

## Example

```text
# Instead of 40 messages, summarize the settled context:
Summary so far:
- Building a REST API in Express + Postgres (Node 18).
- Auth (JWT) is done and tested.
- DECISION: use Zod for request validation.
Current task: add the GET /orders endpoint with pagination.
```

This keeps the model fully grounded on the decisions and the current task while throwing away the noise that led there.

## Why it matters

Knowing when to summarize keeps an AI assistant working within its context window, fast, and cheap — and keeps its attention on the decisions and current state that actually drive the next step, instead of drowning in resolved history.

Why summarize

What to keep vs drop

Example

Why it matters

Related questions