AsyncIO Checkpoint

30 min · Intermediate

Fix the mistakes

The slips that cost marks in this module. Spot each one, then check the fix.

This checkpoint covers the whole module: coroutines and the event loop, tasks and gather, timeouts and cancellation, and async HTTP with blocking code. Predict each output before you read the answer.

Choose the tool

A service must check 2000 product pages over HTTP, then resize 20 large images. The page checks wait on the network, so use asyncio with a limit. The resizing uses the CPU, so use processes.

Asyncio makes waiting cheap. It does not make calculation faster, and one blocking call stalls every task.

Make every wait bounded

Every awaited call should have a timeout, and every failure should become a value or a clear error. Use gather with a Semaphore, and cancel what you no longer need.

Example 1 · A bounded batch of checks

Python

1import asyncio2
3async def check(url):4    await asyncio.sleep(0.5 if "down" in url else 0.01)5    return "up"6
7async def safe(sem, url):8    async with sem:9        try:10            return url, await asyncio.wait_for(check(url), 0.1)11        except TimeoutError:12            return url, "timeout"13
14async def main():15    sem = asyncio.Semaphore(3)16    urls = ["/home", "/down", "/cart", "/pay"]17    results = await asyncio.gather(*(safe(sem, u) for u in urls))18    print(dict(results))19
20asyncio.run(main())

Output

{'/home': 'up', '/down': 'timeout', '/cart': 'up', '/pay': 'up'}

What this code does

Each check has its own timeout, so /down ends as "timeout" and does not hold the others.
A Semaphore of 3 caps the load, and gather keeps the order.

Example 2 · Cancel what is no longer needed

Python

1import asyncio2
3async def lookup(name, delay):4    await asyncio.sleep(delay)5    return name6
7async def main():8    tasks = [asyncio.create_task(lookup("primary", 0.01)), asyncio.create_task(lookup("backup", 0.5))]9    done, pending = await asyncio.wait(tasks, return_when=asyncio.FIRST_COMPLETED)10    for t in pending:11        t.cancel()12    print(next(iter(done)).result(), "won;", len(pending), "cancelled")13
14asyncio.run(main())

Output

primary won; 1 cancelled

What this code does

wait returns when the first task is done.
The slow backup is cancelled so it stops using resources.

Print results from gather or sorted data, never from the order tasks happen to finish.

Finish up

Mark this lesson complete

This is the last lesson in this section.

1import asyncio2 3async def check(url):4 await asyncio.sleep(0.5 if "down" in url else 0.01)5 return "up"6 7async def safe(sem, url):8 async with sem:9 try:10 return url, await asyncio.wait_for(check(url), 0.1)11 except TimeoutError:12 return url, "timeout"13 14async def main():15 sem = asyncio.Semaphore(3)16 urls = ["/home", "/down", "/cart", "/pay"]17 results = await asyncio.gather(*(safe(sem, u) for u in urls))18 print(dict(results))19 20asyncio.run(main())

1import asyncio2 3async def lookup(name, delay):4 await asyncio.sleep(delay)5 return name6 7async def main():8 tasks = [asyncio.create_task(lookup("primary", 0.01)), asyncio.create_task(lookup("backup", 0.5))]9 done, pending = await asyncio.wait(tasks, return_when=asyncio.FIRST_COMPLETED)10 for t in pending:11 t.cancel()12 print(next(iter(done)).result(), "won;", len(pending), "cancelled")13 14asyncio.run(main())