Context Managers and the with Statement
45 min · Advanced
Need help? Ask a tutor on WhatsApp (opens in a new tab)A resource such as a file, a lock or a database connection must always be released, even when an error happens. Interviewers ask about the with statement to see if you understand how Python guarantees that. Each section teaches one tool with a working program.
What with guarantees
Context manager is an object used in a with statement. Python calls its __enter__ method on entry and its __exit__ method on the way out, even when the block raises an error.
Closing a file by hand is easy to forget, and an error in the middle skips the close call. with does the cleanup for you.
The first example shows a file closed as soon as the block ends.
Example 1 · A file closed by with
with open("notes.txt", "w") as f: f.write("hello")print(f.closed)
with open("notes.txt") as f: print(f.read())True hello
What this code does
- The first block writes and leaves, so Python closes the file; the variable
fstill exists butf.closedisTrue. - The second block reopens the file in read mode and prints what was saved.
After the block, f.closed is True. The same guarantee works on return, break and exceptions. This is why with open(...) is the standard way to read files, and why locks should also be used with with.
Writing a class with __enter__ and __exit__
__exit__ is the method that runs when a with block ends. It receives the exception type, value and traceback, or three None values when there was no error, and may return True to hide the exception.
A class becomes a context manager with two methods. __enter__ runs at the start and its return value goes after as.
__exit__ runs at the end, so it is the place for commit, rollback and close.
Example 2 · A class-based manager with commit and rollback
class Transaction: def __init__(self, name): self.name = name
def __enter__(self): print(f"begin {self.name}") return self
def __exit__(self, exc_type, exc, tb): if exc_type is None: print(f"commit {self.name}") else: print(f"rollback {self.name}: {exc}") return True
with Transaction("payment"): print("charging")with Transaction("refund"): raise ValueError("card declined")print("continues")begin payment charging commit payment begin refund rollback refund: card declined continues
What this code does
- Without an error,
__exit__getsNonefor the exception type and prints a commit. With theValueError, it prints a rollback with the message. - Because
__exit__returnsTrue, the exception is hidden and the last line still runs.
In the second block the error was raised, __exit__ saw it, printed a rollback and returned True, so the program carried on. If __exit__ returns False or None, the exception continues upwards. Hide an exception only when you really handled it.
contextlib.contextmanager
contextmanager is a decorator from contextlib that turns a generator with one yield into a context manager. Code before yield is the setup, code after it is the cleanup.
Writing a class for every small manager is long. With the decorator, a generator function with a single yield does the same job.
Put the yield inside try/finally so the cleanup runs after an error too.
Example 3 · A generator-based manager with try and finally
from contextlib import contextmanager
@contextmanagerdef opened(resource): print(f"open {resource}") try: yield resource.upper() finally: print(f"close {resource}")
try: with opened("db") as conn: print("using", conn) raise RuntimeError("boom")except RuntimeError as error: print("caught", error)open db using DB close db caught boom
What this code does
- Code before
yieldruns on entry and the yielded value becomesconn. The error inside the block is raised at theyield. - The
finallyprintsclose dbbefore the error leaves thewith, then the outerexceptcatches it.
The value after yield becomes the as variable. When the block raises, the exception is thrown into the generator at the yield; the finally runs, and the error then continues to the caller. Without try/finally, the cleanup lines would be skipped on errors.
suppress, ExitStack and several managers
ExitStack is a contextlib tool that holds a changing number of context managers and closes them all, in reverse order, when its own with block ends.
suppress ignores a named exception in a block. ExitStack helps when you do not know in advance how many resources you will open.
One with line can also hold several managers.
Example 4 · suppress and ExitStack
from contextlib import ExitStack, suppress
with suppress(FileNotFoundError): open("missing.txt")print("no crash")
names = ["a.txt", "b.txt"]with ExitStack() as stack: files = [stack.enter_context(open(n, "w")) for n in names] for f in files: f.write("x")print([f.closed for f in files])no crash [True, True]
What this code does
open("missing.txt")raisesFileNotFoundError, butsuppressswallows it, so the program continues.stack.enter_contextregisters each opened file. When thewithends, the stack closes both, so everyclosedflag isTrue.
Example 5 · Several managers on one line
from contextlib import contextmanager
@contextmanagerdef tag(name): print("enter", name) yield print("exit", name)
with tag("outer"), tag("inner"): print("body")enter outer enter inner body exit inner exit outer
What this code does
- Both managers are entered from left to right, so
outerstarts first andinnersecond. - They leave in reverse order, like nested blocks:
innerexits beforeouter.
Managers on one line are entered left to right and exited right to left. suppress(FileNotFoundError) ignores only that error and nothing else. ExitStack closed both files at the end, even though we opened them in a loop.
Next step
Check your understanding
12 quick questions