What are generators and the yield keyword?

Question

Accepted Answer

A **generator** is a function that produces values **lazily, one at a time**, using the `yield` keyword instead of `return`. It computes each value on demand rather than building the whole result in memory — ideal for large or infinite sequences.

## yield vs return

```python
# a regular function builds the WHOLE list in memory
def get_squares(n):
    return [i ** 2 for i in range(n)]   # all n values created at once

# a generator yields values ONE AT A TIME, on demand
def gen_squares(n):
    for i in range(n):
        yield i ** 2                     # pause here, return one value, resume on next()

for sq in gen_squares(1_000_000):        # uses almost NO memory
    print(sq)
```

`yield` **pauses** the function, returns a value, and **resumes** from that exact point on the next iteration — preserving local state between values. Nothing is computed until requested.

## The memory benefit

```python
sum(i ** 2 for i in range(10_000_000))   # generator expression — constant memory
sum([i ** 2 for i in range(10_000_000)]) # list — builds a huge list first (wasteful)
```

For large data, generators keep memory flat; a list comprehension materializes everything at once.

## Infinite sequences (impossible with a list)

```python
def count_up():
    n = 0
    while True:        # infinite — but lazy, so it's fine
        yield n
        n += 1

gen = count_up()
next(gen)   # 0
next(gen)   # 1  — only computes when asked
```

Because values are produced on demand, a generator can represent an *infinite* stream — you just take what you need.

## How iteration works under the hood

```python
gen = gen_squares(3)
next(gen)   # 0  — runs until the first yield
next(gen)   # 1  — resumes, runs to the next yield
next(gen)   # 4
next(gen)   # StopIteration — generator exhausted
```

## Generator expressions

```python
(x ** 2 for x in range(10))   # like a list comprehension but lazy (parentheses)
```

## Why it matters

Generators are essential for **memory-efficient** processing of large or streaming data — reading huge files line by line, processing big datasets, pipelines, or infinite sequences — where building a full list would exhaust memory.

Understanding `yield` (pause/resume with preserved state) and lazy evaluation lets you write scalable data-processing code, and generator expressions provide the same benefit concisely.

They're a key tool whenever data is too large to fit in memory or you want to process items as they're produced rather than all upfront.