Calling APIs with httpx and requests
22 min · Intermediate
Need help? Ask a tutor on WhatsApp (opens in a new tab)Your program often has to call another service: a payment gateway, a weather service or a teammate's API. An HTTP client library builds the request, sends it and gives you the response as a Python object. You will use httpx, and then see how requests compares.
An HTTP client sends the request for you
HTTP client is a library or program that sends HTTP requests and reads the responses. In Python the common ones are httpx and requests.
You give the library a method, a URL and some data. It opens the connection, sends the text and parses the reply. To test without a network, httpx lets you plug in a fake server called a mock transport.
import httpx
def fake_server(request): return httpx.Response(200, json={"name": "Pen"})
client = httpx.Client(transport=httpx.MockTransport(fake_server))print(client.get("https://shop.example/products/1").json())Example 1 · A first request against a mock server
import httpx
def server(request): return httpx.Response(200, json={"id": 1, "name": "Pen", "price": 2})
client = httpx.Client(transport=httpx.MockTransport(server))r = client.get("https://shop.example/products/1")print(r.status_code)print(r.headers["content-type"])print(r.json()["name"])200 application/json Pen
What this code does
- The mock server returns a response object; nothing leaves the computer.
- The
json=argument ofhttpx.Responsealso sets the content type.
The client does not know that the server is fake. In real code you simply leave out transport.
Parameters, headers and JSON bodies
Pass query values with params, extra headers with headers, and a Python dict as the body with json. The library builds the URL, encodes the values and sets Content-Type for you.
r = client.get( "https://shop.example/products", params={"category": "books", "limit": 2}, headers={"Accept": "application/json"},)print(r.request.url)Example 2 · Params, headers and a JSON body
import jsonimport httpx
def server(request): return httpx.Response(200, json={ "method": request.method, "url": str(request.url), "accept": request.headers.get("accept"), "body": json.loads(request.content) if request.content else None, })
client = httpx.Client(transport=httpx.MockTransport(server))r = client.get("https://shop.example/search", params={"q": "red pen", "page": 2}, headers={"Accept": "application/json"})print(r.json())r = client.post("https://shop.example/orders", json={"item": "pen", "qty": 3})print(r.json()["body"]){'method': 'GET', 'url': 'https://shop.example/search?q=red+pen&page=2', 'accept': 'application/json', 'body': None}
{'item': 'pen', 'qty': 3}What this code does
- The server sees exactly what a real server would see: the encoded URL, the headers and the body.
- The space in
red penwas encoded as+in the query string.
Special characters in params are encoded for you. Never glue query strings together by hand.
Read the response, then check it
raise_for_status is a response method that does nothing for a 2xx or 3xx status and raises an error for 4xx and 5xx, so that a failed call cannot be ignored by accident.
A response has status_code, headers, text and json(). A 404 or 500 is still a normal response, not a Python error. Call raise_for_status() to turn 4xx and 5xx into an exception.
import httpx
r = client.get("https://shop.example/missing")try: r.raise_for_status()except httpx.HTTPStatusError as err: print(err.response.status_code)Example 3 · Check the status before using the body
import httpx
def server(request): if request.url.path == "/products/1": return httpx.Response(200, json={"name": "Pen"}) return httpx.Response(404, json={"error": "not found"})
client = httpx.Client(transport=httpx.MockTransport(server), base_url="https://shop.example")
def fetch(path): r = client.get(path) try: r.raise_for_status() except httpx.HTTPStatusError as err: return f"failed with {err.response.status_code}" return r.json()
print(fetch("/products/1"))print(fetch("/products/99")){'name': 'Pen'}
failed with 404What this code does
raise_for_status()turns the 404 into an exception that the code can handle.- A
base_urllets the calls use short paths.
Without this call, code might read .json() of an error page by mistake.
Clients, sessions and requests
Use one httpx.Client for many calls. It reuses connections and shares a base URL and headers. The older requests library works in much the same way, but it is not installed in our runner, so its code is shown as a sample only.
import httpx
with httpx.Client(base_url="https://shop.example", headers={"Accept": "application/json"}) as client: pass # make several calls hereExample 4 · One client with shared settings
import httpx
def server(request): return httpx.Response(200, json={"path": request.url.path, "accept": request.headers["accept"]})
with httpx.Client( base_url="https://shop.example/v1", headers={"Accept": "application/json"}, transport=httpx.MockTransport(server),) as client: print(client.get("/items").json()) print(client.get("/orders").json())print(client.is_closed){'path': '/v1/items', 'accept': 'application/json'}
{'path': '/v1/orders', 'accept': 'application/json'}
TrueWhat this code does
- Both calls share the base URL and the header.
- After the
withblock the client is closed.
Example 5 · The same call with requests (sample)
import requests
response = requests.get( "https://shop.example/products", params={"category": "books"}, headers={"Accept": "application/json"}, timeout=5,)response.raise_for_status()print(response.status_code)print(response.json()["items"][0]["name"])200 Atlas of India
What this code does
- The shape is almost the same as httpx:
params,headers,raise_for_status()andjson(). - This needs the
requestspackage and a real server, so it is a sample here. Always pass atimeoutto requests, because it waits forever by default.
The with block closes the connections at the end. The same idea in requests is requests.Session().
Next step
Check your understanding
12 quick questions