Generator Functions and Expressions
22 min · Intermediate
Need help? Ask a tutor on WhatsApp (opens in a new tab)Writing an iterator class takes a lot of code for a small idea. A generator gives you the same result with a normal-looking function that uses yield.
yield turns a function into a generator
Generator is an iterator made by a function that contains yield. Each yield hands out one value and pauses the function until the next value is asked for.
A normal function runs, returns once and is gone. If the body contains yield, calling the function runs nothing yet. It gives back a generator object.
def count_up(limit): n = 1 while n <= limit: yield n n += 1
for value in count_up(3): print(value)Example 1 · A generator that filters
def passing(marks, cutoff): for m in marks: if m >= cutoff: yield m
result = passing([72, 40, 85, 55], 60)print(next(result))print(list(result))72 [85]
What this code does
- The function loops and yields only the marks that pass.
next()takes the first value;list()then collects the rest.
A generator is an iterator, so a for loop, list() and next() all work on it.
The function pauses and resumes
At yield, the function stops and keeps all its local variables. The next next() call continues on the line after the yield.
def steps(): print("start") yield 1 print("middle") yield 2 print("end")
g = steps()print(next(g))print(next(g))Example 2 · Watching the pause and resume
def steps(): print("start") yield 1 print("middle") yield 2 print("end")
g = steps()print("created")print(next(g))print(next(g))for _ in g: passprint("finished")created start 1 middle 2 end finished
What this code does
- "created" prints first because calling
steps()does not run its body. - The for loop at the end resumes the function, which prints "end" and stops.
Nothing prints when steps() is called. "start" appears only on the first next().
Lazy values save memory
Lazy evaluation means a value is calculated only at the moment it is needed, not ahead of time. Generators are lazy; lists are not.
A list builds every value before you use any. A generator makes a value only when it is asked for. That makes it fine for huge or never-ending streams.
from itertools import islice
def order_ids(): n = 1000 while True: yield n n += 1
print(list(islice(order_ids(), 3)))Example 3 · Running totals without a list
def running_total(amounts): total = 0 for amount in amounts: total += amount yield total
print(list(running_total([100, 250, 50])))[100, 350, 400]
What this code does
- The function keeps
totalbetween yields. - Each
yieldsends out the total so far.
The loop inside order_ids never ends, yet the program does. Only three values are ever made.
Generator expressions
Generator expression is a one-line generator written like a comprehension but inside round brackets, for example (x * 2 for x in data).
A generator expression looks like a list comprehension with round brackets. It gives a generator and builds no list.
marks = [72, 85, 91, 40]squares = (m * m for m in marks)print(sum(m for m in marks if m >= 50))print(type(squares).__name__)Example 4 · Generator expression with sum and max
prices = [1200, 450, 980, 300]print(sum(p for p in prices if p > 400))print(max(p for p in prices if p < 1000))print(any(p > 1000 for p in prices))2630 980 True
What this code does
sumandanytake a generator expression directly.anystops at the first true value, so it may not read all the data.
Inside a call like sum(...), the extra brackets can be left out. Use a generator expression when you only loop over the result once.
One pass only
A generator is an iterator, so it is used up after one pass. To loop again, call the generator function again to get a new one.
def evens(limit): for n in range(0, limit, 2): yield n
g = evens(6)print(list(g))print(list(g))print(list(evens(6)))Example 5 · A used-up generator
def evens(limit): for n in range(0, limit, 2): yield n
g = evens(6)print(list(g))print(list(g))g = evens(6)print(sum(g))[0, 2, 4] [] 6
What this code does
- The second
list(g)is empty because the generator is finished. - Calling
evens(6)again gives a new generator.
A return inside a generator ends it. The value of return is not part of the loop; it travels inside StopIteration.
Next step
Check your understanding
12 quick questions