Rate Limiting
26 min · Intermediate
Need help? Ask a tutor on WhatsApp (opens in a new tab)Without a limit, one client can try a million passwords or drain your server. A rate limiter counts requests per client and answers 429 when there are too many. You will write two classic limiters and test them without waiting for real time.
Why count requests
Rate limiting is refusing requests from a client that has made more than an allowed number in a time period. It slows password guessing, scraping and accidental overload.
Some routes are cheap to call and costly to answer: login, password reset, search, report export. A script can call them thousands of times a minute. A rate limit allows a fair number of requests per client in a period and refuses the rest.
calls = {}
def allow(client_id, limit=3): calls[client_id] = calls.get(client_id, 0) + 1 return calls[client_id] <= limitThis counter never resets, so it is not enough. Real limiters add time. The refusal status is 429 Too Many Requests.
Fixed window: count per time slot
Fixed window is a limiter that counts requests in equal time slots and resets the count when a new slot starts. It is simple, but allows a double burst across a slot boundary.
Cut time into slots, for example 10 seconds. Count each client's requests in the current slot, and start from zero in the next slot. It is simple and uses almost no memory. The clock is passed in, so a test can control time.
class FixedWindow: def __init__(self, limit, window, clock): self.limit, self.window, self.clock = limit, window, clock self.counts = {}
def allow(self, key): now = self.clock() start = now - now % self.window began, used = self.counts.get(key, (start, 0)) if began != start: began, used = start, 0 if used >= self.limit: return False self.counts[key] = (began, used + 1) return TrueExample 1 · Fixed window with a fake clock
class FakeClock: def __init__(self, now=0.0): self.now = now
def __call__(self): return self.now
def advance(self, seconds): self.now += seconds
class FixedWindow: def __init__(self, limit, window, clock): self.limit, self.window, self.clock = limit, window, clock self.counts = {}
def allow(self, key): now = self.clock() start = now - now % self.window began, used = self.counts.get(key, (start, 0)) if began != start: began, used = start, 0 if used >= self.limit: return False self.counts[key] = (began, used + 1) return True
def retry_after(self): return int(self.window - self.clock() % self.window)
clock = FakeClock()limiter = FixedWindow(limit=3, window=10, clock=clock)
print([limiter.allow("asha") for _ in range(4)])clock.advance(5)print(limiter.allow("asha"))clock.advance(5)print(limiter.allow("asha"))print(limiter.allow("ravi"))[True, True, True, False] False True True
What this code does
- The fourth call in the slot is refused, and so is the call at second 5. At second 10 a new slot starts and asha is allowed again.
- ravi has his own count. The fake clock moves time without waiting.
Example 2 · The boundary burst
class FakeClock: def __init__(self, now=0.0): self.now = now
def __call__(self): return self.now
def advance(self, seconds): self.now += seconds
class FixedWindow: def __init__(self, limit, window, clock): self.limit, self.window, self.clock = limit, window, clock self.counts = {}
def allow(self, key): now = self.clock() start = now - now % self.window began, used = self.counts.get(key, (start, 0)) if began != start: began, used = start, 0 if used >= self.limit: return False self.counts[key] = (began, used + 1) return True
def retry_after(self): return int(self.window - self.clock() % self.window)
clock = FakeClock(now=9.0)limiter = FixedWindow(limit=3, window=10, clock=clock)
late = [limiter.allow("asha") for _ in range(3)]clock.advance(1)early = [limiter.allow("asha") for _ in range(3)]print("at t=9:", late)print("at t=10:", early)print("allowed in 1 second:", sum(late + early))at t=9: [True, True, True] at t=10: [True, True, True] allowed in 1 second: 6
What this code does
- Three calls at second 9 and three at second 10 all pass: six requests in one second, double the limit.
- A token bucket does not have this weakness.
now % window finds how far into the slot we are. The weakness: a client can use the whole limit at the end of one slot and again at the start of the next.
Token bucket: bursts and steady refill
Token bucket is a limiter where each client has a bucket of tokens that refills at a steady rate. A request needs one token. It allows short bursts up to the bucket size and a steady average rate.
Picture a bucket that holds a few tokens and gets one new token every second. Each request takes a token. A quiet client builds up a small burst allowance, and a busy client is held to the refill rate. This is smoother than a fixed window.
class TokenBucket: def __init__(self, capacity, rate, clock): self.capacity, self.rate, self.clock = capacity, rate, clock self.state = {}
def allow(self, key): now = self.clock() tokens, last = self.state.get(key, (self.capacity, now)) tokens = min(self.capacity, tokens + (now - last) * self.rate) allowed = tokens >= 1 self.state[key] = (tokens - 1 if allowed else tokens, now) return allowedExample 3 · Token bucket: burst, refill and cap
class FakeClock: def __init__(self, now=0.0): self.now = now
def __call__(self): return self.now
def advance(self, seconds): self.now += seconds
class TokenBucket: def __init__(self, capacity, rate, clock): self.capacity, self.rate, self.clock = capacity, rate, clock self.state = {}
def allow(self, key): now = self.clock() tokens, last = self.state.get(key, (self.capacity, now)) tokens = min(self.capacity, tokens + (now - last) * self.rate) allowed = tokens >= 1 self.state[key] = (tokens - 1 if allowed else tokens, now) return allowed
clock = FakeClock()bucket = TokenBucket(capacity=3, rate=1, clock=clock)
print([bucket.allow("asha") for _ in range(4)])clock.advance(2)print([bucket.allow("asha") for _ in range(3)])clock.advance(100)print([bucket.allow("asha") for _ in range(4)])[True, True, True, False] [True, True, False] [True, True, True, False]
What this code does
- The bucket starts full, so three quick calls pass. After 2 seconds two tokens are back.
- After a long pause the bucket holds only its capacity of 3, so a fourth call is refused.
min(capacity, ...) stops the bucket from filling beyond its size, however long the client was quiet.
A rate limit dependency
In FastAPI, a dependency can ask the limiter and raise 429 when the answer is no. Add a Retry-After header so clients know when to try again. Choose the key carefully: the user id for signed-in routes, the client address for login.
def rate_limit(request: Request): key = request.client.host if not limiter.allow(key): raise HTTPException( 429, "Too many requests", headers={"Retry-After": str(limiter.retry_after())}, )Example 4 · A limiter as a FastAPI dependency
class FakeClock: def __init__(self, now=0.0): self.now = now
def __call__(self): return self.now
def advance(self, seconds): self.now += seconds
class FixedWindow: def __init__(self, limit, window, clock): self.limit, self.window, self.clock = limit, window, clock self.counts = {}
def allow(self, key): now = self.clock() start = now - now % self.window began, used = self.counts.get(key, (start, 0)) if began != start: began, used = start, 0 if used >= self.limit: return False self.counts[key] = (began, used + 1) return True
def retry_after(self): return int(self.window - self.clock() % self.window)
from fastapi import Depends, FastAPI, HTTPException, Requestfrom fastapi.testclient import TestClient
clock = FakeClock()limiter = FixedWindow(limit=3, window=10, clock=clock)app = FastAPI()
def rate_limit(request: Request): key = request.headers.get("x-user", "anonymous") if not limiter.allow(key): raise HTTPException(429, "Too many requests", headers={"Retry-After": str(limiter.retry_after())})
@app.post("/login", dependencies=[Depends(rate_limit)])def login(): return {"ok": True}
client = TestClient(app)for i in range(4): r = client.post("/login", headers={"X-User": "asha"}) print(r.status_code, r.headers.get("retry-after"))print(client.post("/login", headers={"X-User": "ravi"}).status_code)clock.advance(10)print(client.post("/login", headers={"X-User": "asha"}).status_code)200 None 200 None 200 None 429 10 200 200
What this code does
- The fourth call gets 429 and
Retry-Aftersays how many seconds until the next slot. - Another user is not affected, and after the clock moves on, asha can log in again.
An in-memory limiter counts per server process only. With several workers, keep the counters in a shared store such as Redis. Behind a proxy, read the real client address from a header that only your proxy can set.
Next step
Check your understanding
12 quick questions