Memory, Reference Counting and Garbage Collection
50 min · Advanced
Need help? Ask a tutor on WhatsApp (opens in a new tab)Memory questions check whether you understand what a variable really is and when an object disappears. The answers below describe CPython, the standard implementation. Each section teaches one idea with a small program whose output does not depend on timing.
Names, references and copies
Reference is a link from a name, or from another object, to an object in memory. Assignment copies the reference, not the object.
In Python a variable is a name pointing to an object. Writing b = a makes a second name for the same object.
Changing a mutable object through one name is visible through the other. A copy makes a new object.
Example 1 · Aliases and copies
a = [1, 2]b = ab.append(3)print(a, a is b)
c = a.copy()c.append(4)print(a, c)[1, 2, 3] True [1, 2, 3] [1, 2, 3, 4]
What this code does
b = acreates a second name for the same list, so appending throughbchanges whatashows, anda is bisTrue.a.copy()makes a new list. Appending 4 tocleavesaas it was.
Example 2 · Shallow copy against deep copy
import copy
cart = {"items": ["pen"]}shallow = copy.copy(cart)deep = copy.deepcopy(cart)cart["items"].append("book")print(shallow["items"], deep["items"])['pen', 'book'] ['pen']
What this code does
- The shallow copy has its own outer dictionary but shares the inner list, so it also sees the new
book. deepcopycopies the inner list too, sodeepkeeps onlypen.
a is b is True because both names point to the same list. copy() made a new list, so c changed alone. A shallow copy copies only the outer container: the inner objects are still shared, as the deepcopy example shows.
Reference counting
Reference counting is the way CPython tracks each object by counting how many references point to it. When the count reaches zero, the object is freed at once.
Every object keeps a count of its references. Binding a name, putting it in a list or passing it to a function raises the count; deleting a name lowers it.
At zero, CPython frees the object immediately.
Example 3 · An object is freed when its count reaches zero
class Resource: def __init__(self, name): self.name = name print("create", name)
def __del__(self): print("free", self.name)
r = Resource("db")alias = rdel rprint("still alive")del aliasprint("after")create db still alive free db after
What this code does
- After
del r, the namealiasstill points to the object, so nothing is freed andstill aliveprints. - After
del aliasthe count is zero, so CPython calls__del__at once andfree dbprints beforeafter.
Deleting r did not free the object, because alias still pointed to it. Deleting alias took the count to zero, and __del__ ran at that moment, before after printed. This immediate cleanup is a CPython detail, not a language promise; other implementations may free objects later. sys.getrefcount(x) reports one more than you expect, because its own argument is a reference.
Reference cycles and the garbage collector
Garbage collector is the part of CPython that finds groups of objects that refer to each other but are unreachable from the program, and frees them. It is needed because reference counting alone cannot free cycles.
If two objects point to each other, their counts never reach zero even when the program has no way to reach them. Reference counting alone would leak them.
The gc module finds such cycles.
Example 4 · A cycle needs the garbage collector
import gc
class Node: def __init__(self, name): self.name = name self.me = self
def __del__(self): print("free", self.name)
n = Node("loop")del nprint("deleted name")gc.collect()print("after gc")deleted name free loop after gc
What this code does
self.me = selfmakes the object refer to itself, so deleting the name leaves a count of one and nothing is freed.gc.collect()finds the unreachable cycle and frees it, sofree loopappears between the two other lines.
The object points to itself, so deleting the name left a count of one. Nothing was freed until gc.collect() ran. CPython also runs the collector by itself, using three generations: new objects are checked often, and objects that survive move to older generations that are checked less often.
weakref, __slots__ and leaks
Weak reference is a reference that does not raise an object's reference count. When the object is freed, the weak reference returns None instead of keeping the object alive.
Caches and observer lists can keep objects alive by accident, which is a common Python memory leak. A weak reference lets you look at an object without owning it.
__slots__ saves memory by removing the per-object dictionary.
Example 5 · A weak reference does not keep the object alive
import weakref
class Cache: pass
obj = Cache()ref = weakref.ref(obj)print(ref() is obj)del objprint(ref())True None
What this code does
- While
objexists, callingref()returns the object, so the first line printsTrue. - After
del objthe object is freed, because the weak reference does not count, andref()returnsNone.
Example 6 · __slots__ removes the instance dictionary
class Plain: def __init__(self, x): self.x = x
class Slim: __slots__ = ("x",)
def __init__(self, x): self.x = x
print(hasattr(Plain(1), "__dict__"), hasattr(Slim(1), "__dict__"))try: Slim(1).y = 2except AttributeError: print("no new attributes")True False no new attributes
What this code does
- A normal instance keeps its attributes in a
__dict__; a slotted instance stores them in fixed slots, so it has no__dict__. - Setting an attribute that is not in
__slots__raisesAttributeError.
After del obj the only strong reference was gone, so the object was freed and ref() gave None. A class with __slots__ has no __dict__, so each instance is smaller, but you cannot add attributes that are not listed. Typical leaks are global lists and caches that only grow, closures that capture large objects, and objects kept alive by exceptions or tracebacks.
Next step
Check your understanding
12 quick questions