Python Generators: Generator Functions and Expressions with Examples

Python generators produce values one at a time rather than loading a full sequence into memory, and that distinction is what makes them appear in placement coding rounds, data pipelines, and token-by-token streaming APIs.

The Python glossary defines a generator as “a function which returns a generator iterator” that can yield “a series of values usable in a for-loop or that can be retrieved one at a time with the next() function.” The key word is usable: no values are produced until you ask for them. The calling code stays in control of the pace.

What a Python generator is (and why not just use a class)

Before generators, creating a custom iterator meant writing a class with __iter__() and __next__() methods, tracking internal state in instance variables, and raising StopIteration by hand when the sequence ended. PEP 255, introduced in Python 2.2, made the case that this overhead was unnecessary for the common case. A generator function handles all of it automatically.

A generator function looks like a regular function except that it contains at least one yield statement. When called, it does not execute its body. It returns a generator object. That object is an iterator, meaning you can pass it to a for loop, call next() on it, or hand it to sum() or list(). The body executes only when a value is pulled.

Here is the iterator class from the WP article rewritten as a generator:

# Class-based iterator — 12 lines
class FirstThreeClass:
    def __iter__(self):
        self.n = 1
        return self

    def __next__(self):
        if self.n <= 3:
            val = self.n
            self.n += 1
            return val
        raise StopIteration

# Generator function — 4 lines, same behaviour
def first_three_gen():
    yield 1
    yield 2
    yield 3

Both produce the same output when iterated. The generator version has no class, no instance variable, and no manual StopIteration raise. Python handles all three internally.

yield vs. return: the core difference

The single design difference between a generator function and a regular function is that yield suspends execution and preserves local state; return terminates the function entirely.

Aspect	yield	return
Execution	Pauses the function; saves local state	Terminates the function
Resumable	Yes — each next() call picks up from the yield point	No
Output per call	One value	One value (or one collection) total
Memory	Generates on demand; state is fixed-size	Full result computed before returning

Here is the same sequence expressed both ways:

# Regular function — builds and returns the whole list at once
def first_three_list():
    return [1, 2, 3]

# Generator function — one value at a time
def first_three_gen():
    yield 1
    yield 2
    yield 3

Calling first_three_list() immediately allocates a list and hands back three integers. Calling first_three_gen() returns a generator object and does not compute anything yet. The integers appear only when you pull them out.

Three common generator patterns

Pattern 1: generator function with next()

def first_three():
    yield 1
    yield 2
    yield 3

gen = first_three()
print(next(gen))  # 1
print(next(gen))  # 2
print(next(gen))  # 3

Calling next() a fourth time raises StopIteration. In practice, use a for loop instead, which catches StopIteration automatically:

for value in first_three():
    print(value)
# 1
# 2
# 3

The for loop is the idiomatic way to consume a generator unless you need fine-grained control over when each value is pulled.

Pattern 2: generator expression

A generator expression uses parentheses where a list comprehension uses square brackets. PEP 289, added in Python 2.4, introduced this shorthand:

# List comprehension — creates the full list immediately
squares_list = [x**2 for x in range(10)]

# Generator expression — produces values lazily
squares_gen = (x**2 for x in range(10))

Both iterate identically in a for loop. The difference is when the computation happens. Pass a generator expression directly into sum(), max(), or any function that accepts an iterable:

total = sum(x**2 for x in range(1_000_000))

This computes the total without storing a million-element list in RAM.

Pattern 3: infinite sequence generator

A generator can run without a predetermined end because it only holds the current state, not the full history:

def natural_numbers():
    n = 1
    while True:
        yield n
        n += 1

gen = natural_numbers()
for _ in range(5):
    print(next(gen), end=" ")
# 1 2 3 4 5

An equivalent loop appending to a list would exhaust available RAM long before producing useful output at any non-trivial scale. The generator keeps only one integer in memory at any point in time.

Memory efficiency in practice

Generators matter most when the full dataset is too large to hold in RAM, or when you only need a single pass through the data.

Consider reading a large log file line by line:

# List approach — loads the entire file before processing begins
def read_all(filepath):
    with open(filepath) as f:
        return f.readlines()

# Generator approach — one line at a time, constant memory footprint
def read_lines(filepath):
    with open(filepath) as f:
        for line in f:
            yield line

The list version allocates memory proportional to the file size before the first line is processed. The generator version holds one line in memory at a time, and its memory use does not grow with file size.

You can confirm this for any generator using sys.getsizeof:

import sys

# Generator: fixed-size state regardless of n
gen = (i**2 for i in range(1_000_000))
print(sys.getsizeof(gen))   # ~120 bytes — constant

# List: size grows with n
lst = list(i**2 for i in range(1_000_000))
print(sys.getsizeof(lst))   # much larger — grows with list length

The generator’s memory footprint is constant regardless of how many values it will produce. The list’s grows with its length. That tradeoff is the main reason generators appear in production data pipelines, not just in textbook examples.

Python’s built-in file iteration follows the same principle: looping over an open file object reads one line per call, which is why it handles files of any size without special treatment. Generators generalise that behaviour to any computation.

Generators in placement coding rounds

Service-tier placement tests include generator questions in two forms. The first is read-and-predict: given a function with yield, trace the output when next() is called three times. The second is short-write: rewrite a class-based iterator as a generator function in under five lines.

Product-company rounds go further. Lazy Fibonacci sequences, streaming data loaders, and chunked batch processors are common prompts. If you have already worked through Python example programs and number-pattern problems like the Armstrong number in Python, generator-based variants are the logical next step. The same iteration logic you applied when summing array elements in Python translates directly to generator-based accumulators.

The generator pattern (produce one item, pause, produce the next) is also the mental model behind how LLM APIs stream responses. Instead of waiting for a full completion, a streaming API yields one token at a time, which is why text appears word by word in chat interfaces. A generator pipeline is the native Python way to consume that stream. TinkerLLM (₹299) is where to test that connection directly: write a Python generator that processes streamed tokens, run it against a live API, and see the pattern working in production rather than in theory.