Performance Checkpoint

25 min · Intermediate

Fix the mistakes

The slips that cost marks in this module. Spot each one, then check the fix.

Performance work is a routine, not a trick. This checkpoint takes one slow job through that routine and then tests your choices.

Measure before you decide

A report job checks 50000 orders against a list of blocked customers and counts orders per product. It feels slow. Do not guess: time the whole job, then profile it to see which function holds the time.

Example 1 · The slow version and the fix

Python

1import timeit2from collections import Counter3
4blocked = [f"C{n}" for n in range(3000)]5orders = [(f"C{n % 5000}", f"P{n % 50}") for n in range(6000)]6
7def slow():8    kept = [o for o in orders if o[0] not in blocked]9    counts = {p: sum(1 for _, q in kept if q == p) for p in {q for _, q in kept}}10    return counts11
12blocked_set = set(blocked)13
14def fast():15    kept = [o for o in orders if o[0] not in blocked_set]16    return dict(Counter(q for _, q in kept))17
18print(slow() == fast())19t_slow = min(timeit.repeat(slow, number=1, repeat=3))20t_fast = min(timeit.repeat(fast, number=1, repeat=3))21print("fast version wins:", t_fast < t_slow)

Output

True
fast version wins: True

What this code does

The first line proves both versions give the same counts.
A set replaces the list scan, and Counter replaces the count per product.
Only the timing line proves the gain, so that is what the example prints.

Fix the structure first

The profile points at the blocked check, which scans a list for every order, and at a count done once per product. A set and a Counter remove both inner loops before any small trick is needed.

Example 2 · Reading the profile for call counts

Python

1import cProfile2import pstats3
4def is_blocked(customer, blocked):5    return customer in blocked6
7def check_all(customers, blocked):8    return [c for c in customers if not is_blocked(c, blocked)]9
10profile = cProfile.Profile()11profile.runcall(check_all, ["a", "b", "c", "d"], {"b"})12stats = pstats.Stats(profile)13calls = {key[2]: row[1] for key, row in stats.stats.items()}14print(calls["is_blocked"], calls["check_all"])

Output

4 1

What this code does

is_blocked ran once per customer, so 4 times.
check_all ran once.
Call counts are exact, which makes them safe to print.

Prove the gain

After each change, run the same measurement on the same data. Keep the change only if the number improved and the output is still identical. Stop when the speed is enough.

Example 3 · A cache for repeated prices

Python

1from functools import lru_cache2
3calls = []4
5@lru_cache(maxsize=2)6def price_with_tax(amount):7    calls.append(amount)8    return round(amount * 1.18, 2)9
10for amount in [100, 200, 100, 300, 100]:11    price_with_tax(amount)12print(calls)13print(price_with_tax.cache_info().hits)

Output

[100, 200, 300]
2

What this code does

maxsize=2 keeps only the two most recently used results.
Using 100 again kept it fresh, so 200 was dropped when 300 arrived, and 100 still hit at the end.
A bounded cache protects memory in a long-running service.

Finish up

Mark this lesson complete

This is the last lesson in this section.

1import timeit2from collections import Counter3 4blocked = [f"C{n}" for n in range(3000)]5orders = [(f"C{n % 5000}", f"P{n % 50}") for n in range(6000)]6 7def slow():8 kept = [o for o in orders if o[0] not in blocked]9 counts = {p: sum(1 for _, q in kept if q == p) for p in {q for _, q in kept}}10 return counts11 12blocked_set = set(blocked)13 14def fast():15 kept = [o for o in orders if o[0] not in blocked_set]16 return dict(Counter(q for _, q in kept))17 18print(slow() == fast())19t_slow = min(timeit.repeat(slow, number=1, repeat=3))20t_fast = min(timeit.repeat(fast, number=1, repeat=3))21print("fast version wins:", t_fast < t_slow)

1import cProfile2import pstats3 4def is_blocked(customer, blocked):5 return customer in blocked6 7def check_all(customers, blocked):8 return [c for c in customers if not is_blocked(c, blocked)]9 10profile = cProfile.Profile()11profile.runcall(check_all, ["a", "b", "c", "d"], {"b"})12stats = pstats.Stats(profile)13calls = {key[2]: row[1] for key, row in stats.stats.items()}14print(calls["is_blocked"], calls["check_all"])

1from functools import lru_cache2 3calls = []4 5@lru_cache(maxsize=2)6def price_with_tax(amount):7 calls.append(amount)8 return round(amount * 1.18, 2)9 10for amount in [100, 200, 100, 300, 100]:11 price_with_tax(amount)12print(calls)13print(price_with_tax.cache_info().hits)