Common Python Speed-Ups
15 min · Intermediate
Need help? Ask a tutor on WhatsApp (opens in a new tab)After the right structure, a few habits give most of the remaining speed. Each one is easy to apply and easy to prove with a timing.
Let the built-ins run the loop
A hot loop is a loop that runs so many times that it uses most of the program's time. Speed work pays off there and almost nowhere else.
A loop written in Python runs one bytecode step at a time. The built-ins sum, min, max, any, sorted and str.join loop inside the interpreter's own fast code, so they are shorter and usually faster.
prices = [120, 80, 45, 200]print(sum(prices), max(prices))print(any(p > 150 for p in prices))print(", ".join(str(p) for p in prices))Example 1 · The built-in sum against a hand-written loop
import timeit
numbers = list(range(100000))
def manual(): total = 0 for n in numbers: total += n return total
def builtin(): return sum(numbers)
print(manual() == builtin())t_manual = min(timeit.repeat(manual, number=20, repeat=3))t_builtin = min(timeit.repeat(builtin, number=20, repeat=3))print("sum is faster:", t_builtin < t_manual)True sum is faster: True
What this code does
- Both return the same total, so only speed differs.
sumloops inside the interpreter and skips the bytecode steps of the manual loop.- The gap is usually several times, which is large enough to trust.
Use them first, then measure. A readable one-liner that is fast enough beats a clever loop. Focus on the hot loop, the one that runs most often, and leave the rest alone.
Remember results with lru_cache
Memoization means storing the result of a function for each set of arguments, so repeated calls return the saved answer instead of computing again. In Python, functools.lru_cache does it.
When a function is called again with the same arguments, recomputing wastes time. functools.lru_cache stores each result by its arguments and returns it next time.
from functools import lru_cache
@lru_cache(maxsize=None)def fib(n): return n if n < 2 else fib(n - 1) + fib(n - 2)
print(fib(30))print(fib.cache_info().misses)Example 2 · Cached against plain recursion
import timeitfrom functools import lru_cache
def plain(n): return n if n < 2 else plain(n - 1) + plain(n - 2)
@lru_cache(maxsize=None)def cached(n): return n if n < 2 else cached(n - 1) + cached(n - 2)
print(plain(20) == cached(20))t_plain = min(timeit.repeat(lambda: plain(20), number=3, repeat=3))cached(20)t_cached = min(timeit.repeat(lambda: cached(20), number=3, repeat=3))print("cached is faster:", t_cached < t_plain)info = cached.cache_info()print(info.misses, info.currsize)True cached is faster: True 21 21
What this code does
- The plain version computes the same sub-problems again and again.
cache_info()showsmisses(values computed) andcurrsize(values stored): 21 fornfrom 0 to 20.- This is memoization, the first step of dynamic programming.
Only 31 values were ever computed, one per n from 0 to 30; every other call was a hit. The arguments must be hashable, and the function must give the same answer each time. Set maxsize on long-running programs, or the cache grows without limit.
Generators keep memory small
Lazy evaluation means producing a value only when it is asked for, not in advance. Generators do this, so a huge or endless sequence can be processed in constant memory.
A list holds every item at once. A generator makes one item at a time when asked. If you only loop once, such as summing, the generator needs almost no memory.
import sys
squares_list = [n * n for n in range(100000)]squares_gen = (n * n for n in range(100000))print(sys.getsizeof(squares_gen) < sys.getsizeof(squares_list))print(sum(squares_gen))Example 3 · A generator inside sum
import sys
orders = range(1_000_000)as_list = [n * 2 for n in orders]as_gen = (n * 2 for n in orders)print(sys.getsizeof(as_list) > 100 * sys.getsizeof(as_gen))print(sum(as_gen) == sum(as_list))True True
What this code does
- The list stores a million numbers; the generator stores only its position.
- Both give the same sum.
- A generator is single-use: a second
sum(as_gen)would give 0.
A generator can be used only once and cannot be indexed. If you need the data twice, or need its length, keep a list.
Do not repeat work inside a loop
A call inside a loop runs every time. If its answer never changes, compute it once before the loop. This is the most common cause of a program that is slow only on large inputs.
Example 4 · Hoisting work out of a loop
import timeit
scores = list(range(2000))
def slow(): return [s for s in scores if s == max(scores)]
def fast(): top = max(scores) return [s for s in scores if s == top]
print(slow() == fast())t_slow = min(timeit.repeat(slow, number=3, repeat=3))t_fast = min(timeit.repeat(fast, number=3, repeat=3))print("hoisted is faster:", t_fast < t_slow)True hoisted is faster: True
What this code does
max(scores)ran 2000 times in the slow version, walking 2000 items each time.- The fast version computes it once.
- This turns an n squared program into an n program.
When Python itself is the limit
Waiting on the network or disk is I/O, and threads help because a waiting thread releases the GIL. Heavy calculation is CPU work, and it needs processes or compiled libraries such as NumPy.
Example 5 · Threads overlap waiting
import timefrom concurrent.futures import ThreadPoolExecutor
def fetch(n): time.sleep(0.1) return n * 10
start = time.perf_counter()with ThreadPoolExecutor(max_workers=5) as pool: results = list(pool.map(fetch, [1, 2, 3, 4, 5]))elapsed = time.perf_counter() - startprint(results)print("waits overlapped:", elapsed < 0.4)[10, 20, 30, 40, 50] waits overlapped: True
What this code does
- Five calls of 0.1 seconds would take 0.5 seconds one after another.
- With five threads the waits happen together and the whole run takes about 0.1 seconds.
- This helps only for waiting. For heavy calculation, use processes.
Next step
Check your understanding
11 quick questions