How do you choose the right model tier given cost and latency tradeoffs?

Question

IT Interview Practice · Accepted Answer

Models within a family trade **capability against cost and latency**: smaller models are faster and cheaper but less capable; larger ones reason better on hard problems but cost more and respond slower. The skill is matching the model to the difficulty of the task, not always reaching for the biggest one.

## The tiers (Anthropic's lineup)

Using Anthropic's Claude family as the example (latest generation: **Opus 4.8**, **Sonnet 4.6**, **Haiku 4.5**):

```text
Haiku   → smallest, fastest, cheapest
          → high-volume / simple tasks: classification, extraction, quick edits
Sonnet  → balanced cost/quality
          → everyday coding and general work (the sensible default)
Opus    → most capable, highest cost & latency
          → hardest multi-step reasoning, architecture, tricky debugging
```

The ordering is relative: **Haiku < Sonnet < Opus** in both price and latency, and the same order in raw capability.

## How to choose

- **Start cheaper.** Reach for Haiku or Sonnet first; most tasks don't need the top tier.
- **Escalate only when quality requires it.** If a cheaper model's output is wrong, shallow, or keeps missing the goal on a genuinely hard problem, move up to Opus.
- **Think about volume.** A task you run millions of times (tagging, routing) should be on Haiku even if Opus is slightly better — the cost and latency multiply.

```text
Classify 1M support tickets   → Haiku   (volume dominates; task is simple)
Implement a feature           → Sonnet  (balanced everyday coding)
Debug a subtle race condition → Opus    (hard reasoning justifies the cost)
```

## Why it matters

Defaulting to the biggest model wastes money and adds latency on tasks a cheaper tier handles fine; defaulting to the smallest gives poor results on genuinely hard work. Choosing deliberately — start cheaper, escalate only when quality requires — keeps systems fast and affordable at scale while still bringing the heavy model to the problems that actually need it.

The tiers (Anthropic's lineup)

How to choose

Why it matters

Related questions