Async HTTP Calls and Blocking Code

30 min · Intermediate

A web request is mostly waiting. That makes it the standard use of asyncio: one thread can keep hundreds of requests in flight.

Real HTTP with httpx (not run here)

Async HTTP client is a library object whose request methods are coroutines. It sends a request, awaits the reply without blocking the loop, and lets other tasks run meanwhile.

Async HTTP libraries such as httpx and aiohttp let a coroutine await a web reply while other requests proceed.

Python

1# Not run here: needs the httpx package and a network.2import asyncio3import httpx4
5async def fetch(client, url):6    response = await client.get(url, timeout=5)7    return response.status_code8
9async def main(urls):10    async with httpx.AsyncClient() as client:11        return await asyncio.gather(*(fetch(client, u) for u in urls))

The code below needs the httpx package and a network, so it is shown but not run.

One client is shared by all requests, so connections are reused. The rest of this lesson uses a fake client, so every example here runs the same way on any machine.

A fake client for practice

Async context manager is an object used with async with. Its enter and exit steps can await, for example to open and close a network connection cleanly.

A real client is used with async with, which can await when it opens and closes. We can build the same shape with __aenter__ and __aexit__.

Python

1import asyncio2
3class FakeClient:4    async def __aenter__(self):5        return self6
7    async def __aexit__(self, *exc):8        return False9
10    async def get(self, url):11        await asyncio.sleep(0.05)12        return {"url": url, "status": 200}

Our fake get sleeps instead of calling a network.

Example 1 · One request with a fake client

Python

1import asyncio2
3class FakeClient:4    async def __aenter__(self):5        return self6
7    async def __aexit__(self, *exc):8        return False9
10    async def get(self, url):11        await asyncio.sleep(0.5 if "slow" in url else 0.02)12        return {"url": url, "status": 200}13
14async def main():15    async with FakeClient() as client:16        reply = await client.get("/orders/101")17    print(reply)18
19asyncio.run(main())

Output

{'url': '/orders/101', 'status': 200}

What this code does

async with opens the client and closes it at the end of the block.
get awaits a short sleep in place of the network and returns a dict like a real reply.

Example 2 · An async generator for pages

Python

1import asyncio2
3async def pages(total, size):4    for start in range(0, total, size):5        await asyncio.sleep(0.01)        # one request per page6        yield list(range(start, min(start + size, total)))7
8async def main():9    async for page in pages(7, 3):10        print(page)11
12asyncio.run(main())

Output

[0, 1, 2]
[3, 4, 5]
[6]

What this code does

yield inside async def makes an async generator, and async for reads it.
Each page is awaited before the next one starts, which is right when the next page depends on the last.

async for is the matching loop for sources that produce items by awaiting, such as pages of results.

Many requests, with a limit and a timeout

Concurrency limit is a cap on how many requests run at the same time, usually set with a Semaphore. It protects the server and your own memory and sockets.

Send requests with gather, but cap how many run at once with a semaphore, and give each its own timeout so one slow URL cannot hold the rest.

Python

1async def fetch_one(client, sem, url):2    async with sem:3        try:4            reply = await asyncio.wait_for(client.get(url), 0.1)5            return url, reply["status"]6        except TimeoutError:7            return url, "timeout"

Turn failures into plain values.

Example 3 · Many requests with gather

Python

1import asyncio2
3class FakeClient:4    async def __aenter__(self):5        return self6
7    async def __aexit__(self, *exc):8        return False9
10    async def get(self, url):11        await asyncio.sleep(0.5 if "slow" in url else 0.02)12        return {"url": url, "status": 200}13
14async def main():15    urls = ["/a", "/b", "/c", "/d"]16    async with FakeClient() as client:17        replies = await asyncio.gather(*(client.get(u) for u in urls))18    print([r["url"] for r in replies])19
20asyncio.run(main())

Output

['/a', '/b', '/c', '/d']

What this code does

All four requests wait together, so the whole run takes about one request.
The replies come back in the order of the URLs.

Example 4 · Limit, timeout and failures as values

Python

1import asyncio2
3class FakeClient:4    async def __aenter__(self):5        return self6
7    async def __aexit__(self, *exc):8        return False9
10    async def get(self, url):11        await asyncio.sleep(0.5 if "slow" in url else 0.02)12        return {"url": url, "status": 200}13
14async def fetch_one(client, sem, url):15    async with sem:16        try:17            reply = await asyncio.wait_for(client.get(url), 0.1)18            return url, reply["status"]19        except TimeoutError:20            return url, "timeout"21
22async def main():23    urls = ["/a", "/slow", "/b"]24    sem = asyncio.Semaphore(2)25    async with FakeClient() as client:26        results = await asyncio.gather(*(fetch_one(client, sem, u) for u in urls))27    print(dict(results))28
29asyncio.run(main())

Output

{'/a': 200, '/slow': 'timeout', '/b': 200}

What this code does

The /slow URL takes longer than the 0.1 s limit, so it becomes "timeout" instead of an error.
At most two requests run at once.
gather keeps order, so the dict is built in the order of the URLs.

gather keeps the input order, so the answers line up with the URLs.

Blocking libraries: asyncio.to_thread

asyncio.to_thread runs an ordinary blocking function in a worker thread and returns an awaitable for its result, so the event loop is not blocked.

Many useful libraries are not async. Calling one directly would block the loop. asyncio.to_thread(func, args) runs it in a worker thread and lets you await the result.

Python

1import asyncio2import time3
4def read_report(n):5    time.sleep(0.05)       # a blocking call6    return n * 27
8async def main():9    print(await asyncio.to_thread(read_report, 21))

The loop stays free for other tasks.

Example 5 · Blocking calls moved to threads

Python

1import asyncio2import time3
4def read_report(n):5    time.sleep(0.05)             # blocking, not async6    return n * 27
8async def main():9    results = await asyncio.gather(*(asyncio.to_thread(read_report, n) for n in (1, 2, 3)))10    print(results)11
12asyncio.run(main())

Output

[2, 4, 6]

What this code does

Each call runs in its own worker thread, so the three waits overlap.
The event loop is free during the waits, and gather keeps the order.

This is the bridge between old blocking code and an async program. For heavy calculation use a process pool with loop.run_in_executor instead.

Next step

Check your understanding

10 quick questions

1# Not run here: needs the httpx package and a network.2import asyncio3import httpx4 5async def fetch(client, url):6 response = await client.get(url, timeout=5)7 return response.status_code8 9async def main(urls):10 async with httpx.AsyncClient() as client:11 return await asyncio.gather(*(fetch(client, u) for u in urls))

1import asyncio2 3class FakeClient:4 async def __aenter__(self):5 return self6 7 async def __aexit__(self, *exc):8 return False9 10 async def get(self, url):11 await asyncio.sleep(0.5 if "slow" in url else 0.02)12 return {"url": url, "status": 200}13 14async def main():15 async with FakeClient() as client:16 reply = await client.get("/orders/101")17 print(reply)18 19asyncio.run(main())

1import asyncio2 3async def pages(total, size):4 for start in range(0, total, size):5 await asyncio.sleep(0.01) # one request per page6 yield list(range(start, min(start + size, total)))7 8async def main():9 async for page in pages(7, 3):10 print(page)11 12asyncio.run(main())

1async def fetch_one(client, sem, url):2 async with sem:3 try:4 reply = await asyncio.wait_for(client.get(url), 0.1)5 return url, reply["status"]6 except TimeoutError:7 return url, "timeout"

1import asyncio2 3class FakeClient:4 async def __aenter__(self):5 return self6 7 async def __aexit__(self, *exc):8 return False9 10 async def get(self, url):11 await asyncio.sleep(0.5 if "slow" in url else 0.02)12 return {"url": url, "status": 200}13 14async def main():15 urls = ["/a", "/b", "/c", "/d"]16 async with FakeClient() as client:17 replies = await asyncio.gather(*(client.get(u) for u in urls))18 print([r["url"] for r in replies])19 20asyncio.run(main())

1import asyncio2 3class FakeClient:4 async def __aenter__(self):5 return self6 7 async def __aexit__(self, *exc):8 return False9 10 async def get(self, url):11 await asyncio.sleep(0.5 if "slow" in url else 0.02)12 return {"url": url, "status": 200}13 14async def fetch_one(client, sem, url):15 async with sem:16 try:17 reply = await asyncio.wait_for(client.get(url), 0.1)18 return url, reply["status"]19 except TimeoutError:20 return url, "timeout"21 22async def main():23 urls = ["/a", "/slow", "/b"]24 sem = asyncio.Semaphore(2)25 async with FakeClient() as client:26 results = await asyncio.gather(*(fetch_one(client, sem, u) for u in urls))27 print(dict(results))28 29asyncio.run(main())

Blocking libraries: asyncio.to_thread

asyncio.to_thread runs an ordinary blocking function in a worker thread and returns an awaitable for its result, so the event loop is not blocked.

Many useful libraries are not async. Calling one directly would block the loop. asyncio.to_thread(func, args) runs it in a worker thread and lets you await the result.

1import asyncio2import time3 4def read_report(n):5 time.sleep(0.05) # blocking, not async6 return n * 27 8async def main():9 results = await asyncio.gather(*(asyncio.to_thread(read_report, n) for n in (1, 2, 3)))10 print(results)11 12asyncio.run(main())