Timeouts and Retries
24 min · Intermediate
Need help? Ask a tutor on WhatsApp (opens in a new tab)Networks are slow and servers fail, even good ones. A client that waits forever will hang, and a client that retries without care can make things worse. Two tools fix this: a timeout on every call, and retries with growing pauses.
Every call needs a timeout
Timeout is the longest time a client waits for part of a request, such as making the connection or reading the reply. When it passes, the call fails with an exception instead of waiting forever.
A timeout is the longest the client will wait. Without one, a silent server can freeze your program. httpx has a 5 second default; you can set connect and read limits yourself.
import httpx
t = httpx.Timeout(10.0, connect=2.0)print(t.connect, t.read)Example 1 · Timeout settings and a timeout error
import httpx
timeout = httpx.Timeout(10.0, connect=2.0)print("connect:", timeout.connect, "read:", timeout.read)
def slow_server(request): raise httpx.ReadTimeout("no reply in time", request=request)
client = httpx.Client(transport=httpx.MockTransport(slow_server), timeout=timeout)try: client.get("https://x.example/report")except httpx.TimeoutException as err: print(type(err).__name__, "-", err)connect: 2.0 read: 10.0 ReadTimeout - no reply in time
What this code does
- The timeout is set once on the client and applies to every call.
ReadTimeoutis a kind ofTimeoutException, so oneexceptcatches all timeout types.
Waiting 2 seconds to connect and 10 seconds for the reply is a typical start. When the limit passes, httpx raises httpx.TimeoutException or a subclass such as ReadTimeout.
Retry with growing pauses
Exponential backoff is a retry plan where the wait doubles after each failed attempt, usually up to a maximum, so a struggling server gets time to recover.
Many failures are short, so a second try often works. But retrying at once can overload a server that is already struggling. Wait longer after each failure: 0.5, 1, 2, 4 seconds, up to a cap.
base, cap = 0.5, 8.0delays = [min(cap, base * 2 ** n) for n in range(6)]print(delays)Example 2 · Backoff delays with and without jitter
import random
def backoff(attempt, base=0.5, cap=8.0): return min(cap, base * 2 ** (attempt - 1))
rng = random.Random(7) # fixed seed so the output is repeatablefor attempt in range(1, 7): plain = backoff(attempt) jittered = round(rng.uniform(0, plain), 2) print(attempt, plain, jittered)1 0.5 0.16 2 1.0 0.15 3 2.0 1.3 4 4.0 0.29 5 8.0 4.29 6 8.0 2.93
What this code does
- The plain delay doubles and stops growing at the cap of 8 seconds.
- The jittered value is a random wait between 0 and the plain delay, so clients spread out.
Example 3 · A retry loop against a failing server
import httpx
calls = {"n": 0}
def flaky(request): calls["n"] += 1 if calls["n"] <= 2: return httpx.Response(503) return httpx.Response(200, json={"ok": True})
def fetch(client, url, sleep, attempts=4, base=0.5): for attempt in range(1, attempts + 1): response = client.get(url) if response.status_code not in (429, 502, 503, 504): return response if attempt == attempts: return response sleep(base * 2 ** (attempt - 1))
slept = []client = httpx.Client(transport=httpx.MockTransport(flaky))r = fetch(client, "https://x.example/items", slept.append)print(r.status_code, r.json())print("attempts:", calls["n"])print("waited:", slept)200 {'ok': True}
attempts: 3
waited: [0.5, 1.0]What this code does
- The server fails twice, so the client waits 0.5 and then 1.0 seconds.
- The
sleepfunction is passed in, so the test records the waits and never really sleeps.
Adding a little random jitter to each delay stops many clients from retrying at the same moment.
Retry only what is safe
Idempotency key is a unique value a client sends with a request, so the server can recognise a repeat and return the first result instead of doing the work twice.
Retry timeouts, connection errors, 429 and 502, 503, 504. Never retry other 4xx codes, because the same request will fail again. Retry POST only if the server can spot repeats with an idempotency key.
RETRY_STATUS = {429, 502, 503, 504}
def should_retry(status): return status in RETRY_STATUS
print(should_retry(503), should_retry(404))Example 4 · An idempotency key stops a duplicate order
import jsonimport httpx
def make_server(use_keys): orders = {} state = {"drop_reply": True}
def server(request): key = request.headers.get("idempotency-key") if use_keys else None if key is None or key not in orders: order = {"id": len(orders) + 1, "item": json.loads(request.content)["item"]} orders[key or f"auto-{order['id']}"] = order else: order = orders[key] if state["drop_reply"]: state["drop_reply"] = False raise httpx.ReadTimeout("reply lost", request=request) return httpx.Response(201, json=order)
return server, orders
def place_order(use_keys): server, orders = make_server(use_keys) client = httpx.Client(transport=httpx.MockTransport(server)) headers = {"Idempotency-Key": "order-abc-1"} if use_keys else {} for attempt in range(2): try: client.post("https://shop.example/orders", json={"item": "pen"}, headers=headers) break except httpx.ReadTimeout: continue return len(orders)
print("without key:", place_order(False), "orders")print("with key:", place_order(True), "order")without key: 2 orders with key: 1 order
What this code does
- The first reply is lost, so the client retries the POST.
- Without a key the server cannot tell it is a repeat and creates two orders. With the key it returns the stored order.
A 404 or 422 means the request is wrong, so repeating it wastes time.
Respect Retry-After and set a limit
A 429 or 503 may carry Retry-After, the number of seconds the server wants you to wait. Obey it. Also set a maximum number of attempts, so the call finally fails and your program can report the problem.
import httpx
r = httpx.Response(429, headers={"Retry-After": "3"})wait = float(r.headers.get("retry-after", 0))print(wait)Example 5 · Retry only what can change
import httpx
RETRY = {429, 502, 503, 504}
def next_step(response): if response.status_code in RETRY: wait = float(response.headers.get("retry-after", 1)) return f"retry after {wait}s" if response.is_success: return "done" return "give up"
for r in (httpx.Response(200), httpx.Response(404), httpx.Response(503), httpx.Response(429, headers={"Retry-After": "3"})): print(r.status_code, next_step(r))200 done 404 give up 503 retry after 1.0s 429 retry after 3.0s
What this code does
- A 404 will not improve, so the client gives up at once.
- When the server sends
Retry-After, that value replaces the guess.
Three or four attempts is a common limit. After that, give up and show a clear error.
Next step
Check your understanding
12 quick questions