Iterators and Generators
45 min · Advanced
Need help? Ask a tutor on WhatsApp (opens in a new tab)Interviewers ask about iterators and generators to see if you know how a for loop really works and how to handle data too big for memory. Each section teaches one idea with a working program, then the question bank gives short answers you can say aloud.
Iterables and iterators
Iterator is an object that gives one item at a time through next() and raises StopIteration when there are no more. An iterable is anything iter() can turn into an iterator.
A for loop needs a way to ask for the next item. Python does it with two steps: iter() makes the iterator, then next() is called until StopIteration.
The examples show both steps by hand, then a small class that follows the same rules.
Example 1 · Using iter() and next() by hand
marks = [72, 85]it = iter(marks)print(next(it))print(next(it))try: next(it)except StopIteration: print("done")print(iter(marks) is marks, iter(it) is it)72 85 done False True
What this code does
iter(marks)creates a list iterator. Eachnextgives the following mark, and the third call raisesStopIteration, which a for loop catches for you.- A list is not its own iterator, so the first check is
False. An iterator returns itself fromiter(), so the second isTrue.
Example 2 · A class-based iterator that runs out
class Countdown: def __init__(self, start): self.current = start
def __iter__(self): return self
def __next__(self): if self.current <= 0: raise StopIteration value = self.current self.current -= 1 return value
timer = Countdown(3)print(list(timer))print(list(timer))[3, 2, 1] []
What this code does
__iter__returns the object itself and__next__gives the next value or raisesStopIteration. That is the full iterator protocol.- The object keeps its position in
current, so after the firstlistit is finished. The secondlistgets nothing.
A list is iterable, not an iterator: each call to iter(marks) gives a fresh iterator. An iterator returns itself from iter(), so it can be used only once. This is why the second list(timer) is empty.
Generator functions and yield
Generator is an iterator made by a function that uses yield. The function pauses at each yield, keeps its local variables, and continues on the next request.
Writing a class for every iterator is long. A function with yield does the same job in a few lines.
Calling it runs no code yet. It returns a generator, and the body runs only when you ask for items.
Example 3 · A generator runs only when asked
def order_ids(prefix, count): print("start") for n in range(1, count + 1): yield f"{prefix}-{n:03d}" print("end")
gen = order_ids("ORD", 2)print("created")print(next(gen))print(next(gen))for _ in gen: passcreated start ORD-001 ORD-002 end
What this code does
- Creating the generator prints only
created. The linestartappears at the firstnext, because that is when the body begins. - After two items the loop asks again. The function continues after the last
yield, printsendand finishes, which stops the loop.
Notice that start printed only after the first next, and end only when the loop asked for a third item. The generator pauses at yield and remembers n. Nothing is stored in a list, so one order id exists at a time.
Generator expressions and laziness
Lazy evaluation means a value is computed only when it is needed. Generators are lazy, so they can describe very long or endless sequences.
A generator expression looks like a list comprehension with round brackets. It builds nothing up front.
The price is that it can be read once, so a second pass sees nothing.
Example 4 · Generator expressions can be read once
orders = [120, 80, 300, 45]big = (amount for amount in orders if amount > 100)print(sum(big))print(sum(big))first = next(a for a in orders if a > 100)print(first)420 0 120
What this code does
- The first
sumreads the generator to the end and gives 420. The secondsumfinds it empty, so it gives 0. next(...)on a fresh generator expression stops at the first match, 120, and ignores the remaining orders.
next() on a generator expression stops at the first match, so the rest is never computed. Use a list when you need to read the data twice or need len(). Use a generator when the data is large or you only need part of it.
yield from, send and itertools
yield from hands control to another iterable and passes its items out one by one. It keeps nested generators short and also forwards send() and return values.
Real programs chain generators. yield from repeats another iterable's items, and itertools.islice takes a slice of any iterator, even an endless one.
A generator can also receive a value with send().
Example 5 · yield from with an endless generator
from itertools import islice
def ticket_numbers(): n = 1000 while True: yield n n += 1
def with_header(items): yield "TICKETS" yield from items
print(list(islice(with_header(ticket_numbers()), 4)))['TICKETS', 1000, 1001, 1002]
What this code does
ticket_numbersnever ends, but it is lazy, so it makes a number only when somebody asks.with_headergives one title, thenyield frompasses on the numbers.islicestops after four items, so the program finishes.
Example 6 · send() gives a generator input
def running_average(): total = 0 count = 0 average = None while True: value = yield average total += value count += 1 average = total / count
avg = running_average()next(avg)print(avg.send(10))print(avg.send(20))print(avg.send(30))10.0 15.0 20.0
What this code does
next(avg)runs the generator to its firstyieldso it is ready to receive.- Each
send(x)becomes the value of theyieldexpression, the loop updates the totals and the new average is the next yielded value: 10.0, 15.0, 20.0.
islice(it, 4) takes the first four items and stops, so the endless ticket_numbers never loops forever. In running_average, value = yield average both gives out the average and receives the next number. The first next(avg) runs it to the first yield; this is called priming.
Next step
Check your understanding
12 quick questions