Examining the gap between Python's readable interfaces and the underlying complexity, implicit state, and unexpected behavior they can conceal.
A catalog of 59 Python gotchas, traps and surprising behaviors, from beginner-level surprises to asyncio, threading, CPython internals, data-engineering bugs and security vulnerabilities. Each comes with a minimal reproduction, an explanation of why it happens, the fix, and, where one exists, the real-world bug or CVE it caused.
Python's syntax prioritizes readability, but readable code does not always make runtime behavior predictable. Some pitfalls arise from implicit state, others from surprising semantics or abstractions that leak underlying details. John Ousterhout's A Philosophy of Software Design calls a particularly troublesome symptom of complexity unknown unknowns: behavior you cannot predict from the interface in front of you. When the language's mechanics don't match its readable syntax, developers may discover those differences only through runtime bugs.
Here is a collection of Python pitfalls developers often discover the hard way, and the evidence that they don't stay in tutorials.
Every pitfall except two security entries has a runnable, self-checking example in
examples/. CI runs them on CPython 3.10 to 3.14, so the outputs shown here are verified, not remembered.
Each row is a kind of hidden complexity, a pitfall that demonstrates the mechanism, and what it has caused in real code. Advisory counts are unique PyPI vulnerabilities published since 2023, from the OSV database (method).
| Hidden complexity | Pitfalls | PyPI advisories since 2023 | Real-world cases |
|---|---|---|---|
| A path join silently discards your prefix | #53, #54 | 507 (path traversal, the #1 weakness) | tarfile CVE-2007-4559, open for 15 years |
| Parsing cost grows with the attacker's input | #56, #57, #42 | 549 (denial of service) | urllib3 CVE-2021-33503, idna CVE-2024-3651, CPython CVE-2020-10735 |
| Loading data runs code | #58 | 210 (deserialization) | torch.load CVE-2025-32434; 58 bypasses of picklescan |
| A convenience hides a protocol decision | #55 | 60 (redirect-related leaks) | requests CVE-2018-18074, urllib3 CVE-2023-43804 |
| One parameter, two meanings | #59 | 41 (signature verification) | PyJWT: 15 key or algorithm advisories since 2023 |
| Hash values change per process | #43 | CPython CVE-2012-1150, the reason hash() is randomized |
|
| Wall-clock vs elapsed time | #48, #49 | Home Assistant: alert loop, backups an hour off | |
| The event loop is shared by everything | #20, #22, #24 | Home Assistant's guard rails and Hikvision fix | |
| Code you never chose | Dependency risk | urllib3 and certifi reach 145 and 144 of the top 1,000 packages, 86–87% indirectly |
| # | Pitfall | What you see | The fix |
|---|---|---|---|
| Iterators & laziness | |||
| 1 | Iterator exhaustion | Second list(it) is [] |
Materialize once with list() |
| 2 | zip() drops data |
Missing rows, a lost element | zip(..., strict=True) |
| 3 | Generators delay errors | Exception far from the bug | Validate in a normal function |
| Mutability & aliasing | |||
| 4 | Mutable default arguments | Data bleeds across calls | None sentinel |
| 5 | List multiplication aliasing | Edit one row, all rows change | List comprehension |
| 6 | Shared class attributes | Instances share state | Assign in __init__ |
| 7 | Modifying a list while iterating | Elements skipped | Build a new list |
| 8 | The tuple += anomaly |
Crash and mutation | Call .extend() |
| Scope & binding | |||
| 9 | Late binding in closures | Every lambda returns 2 |
lambda i=i: ... |
| 10 | Loop variables leak | Outer variable overwritten | Distinct names |
| 11 | except ... as e deletes e |
NameError after the block |
Copy to another name |
| 12 | UnboundLocalError |
Error on a read of a global | Pass arguments in |
| Identity & equality | |||
| 13 | Integer caching and is |
is flips at 257 |
Use == |
| 14 | True is 1 |
Dict keys collapse | Put the type in the key |
| Syntax traps | |||
| 15 | Implicit string concatenation | Two items become one | Trailing commas + linter |
| 16 | Trailing comma makes a tuple | (30,) instead of 30 |
Delete the comma |
| Standard library surprises | |||
| 17 | strip() takes a character set |
"report.txt" becomes "repor" |
removesuffix() |
| 18 | Banker's rounding | round(2.5) == 2 |
Decimal + ROUND_HALF_UP |
| 19 | return in finally |
Exception vanishes | Never return from finally |
| Page | # | Covers |
|---|---|---|
| Concurrency: asyncio & threading | 20–30 | Garbage-collected tasks, forgotten await, blocking the event loop, swallowed cancellation, cross-thread loop calls, non-atomic +=, vanishing thread exceptions, Lock re-entry, threading.local under asyncio, fork() deadlocks, daemon threads |
| Object model & imports | 31–37 | Properties vs __getattr__, @lru_cache leaks, __eq__ vs __hash__, __exit__ swallowing errors, weak refs to bound methods, mock.patch targets, double-imported __main__ |
| Interpreter internals | 38–42 | python -O stripping asserts, class-body comprehension scope, locals() writes, id() reuse, the 4300-digit int/str limit |
| Data systems: time, numbers, bytes | 43–52 | hash() sharding, NaN in containers, nanosecond timestamps in floats and datetime, float prices, DST arithmetic, fixed UTC offsets, utcnow(), struct alignment, non-standard JSON |
| Security: where complexity becomes a CVE | 53–59 | os.path.join with absolute paths, tarfile extraction, credentials forwarded on redirect, regex backtracking, decompression bombs, unsafe deserialization, signing-key confusion |
See also In the wild for how a large production codebase hits and guards against these.
| Page | Covers |
|---|---|
| Complexity in the wild | What Python's vulnerability data shows, organized by mechanism: 7,032 OSV advisories joined with the dependency graphs of the top 1,000 packages, yanked releases, OpenSSF Scorecard and typosquatting data, with charts |
Functions like reversed(), zip(), map() and filter(), generator expressions, and file objects return single-use iterators rather than persistent data structures. Passing an iterator to list(), or inspecting it with print(list(...)), consumes it entirely.
Subsequent reads find an empty sequence, with no warning. Simply inspecting your data destroys it.
# The Trap
my_string = "hello"
reversed_iter = reversed(my_string)
print(list(reversed_iter)) # Output: ['o', 'l', 'l', 'e', 'h']
print(list(reversed_iter)) # Output: [] (Silently empty!)It gets worse. A membership test consumes everything up to the match, and an exhausted iterator is still truthy, so if it: can't tell you it's empty:
squares = (n * n for n in range(6))
print(9 in squares) # Output: True
print(list(squares)) # Output: [16, 25] (0, 1, 4 and 9 are gone)
empty = (n for n in [])
print(bool(empty)) # Output: TrueWhy: An iterator is a cursor, not a container. It only knows how to produce the next item, and once it raises StopIteration it stays exhausted.
The fix: If you need the data more than once, materialize it once with list(...) or tuple(...) and reuse that. For large or infinite streams, use itertools.tee() or restructure the code to make one pass.
zip() stops at the shortest input without complaint, so mismatched data simply disappears. When one input is an iterator, zip() also pulls one extra item from it before noticing that a later input is empty, and that item is thrown away.
# The Trap
names = ["Ada", "Grace", "Linus"]
scores = [95, 88]
print(list(zip(names, scores))) # Output: [('Ada', 95), ('Grace', 88)] (Linus vanished)
numbers = iter([1, 2, 3, 4])
print(list(zip(numbers, ["a", "b"]))) # Output: [(1, 'a'), (2, 'b')]
print(list(numbers)) # Output: [4] (Where did 3 go?)Why: On each step zip() calls next() on its inputs left to right. On the third step it takes 3 from numbers, then finds the list exhausted, and has nowhere to put the 3.
The fix: Pass strict=True (Python 3.10+) to raise ValueError on a length mismatch. Use itertools.zip_longest() if you want to pad instead. If you must zip an iterator with something shorter, put the shorter input first.
Calling a generator function doesn't run any of its body, not even the argument checks at the top. The error surfaces later, wherever the generator is first iterated, which may be in a different module.
# The Trap
def chunks(data, size):
if size <= 0:
raise ValueError("size must be positive")
for i in range(0, len(data), size):
yield data[i : i + size]
batches = chunks([1, 2, 3], 0) # No error here...
list(batches) # ...ValueError is raised here instead.Why: A yield anywhere in a function body turns the whole function into a generator factory. Calling it only creates a paused generator object.
The fix: Validate in a regular function, then return the generator:
def chunks(data, size):
if size <= 0:
raise ValueError("size must be positive") # Raised at the call site
return _chunks(data, size)
def _chunks(data, size):
for i in range(0, len(data), size):
yield data[i : i + size]A default value is evaluated once, when the def statement runs, not on every call. With a mutable default like a list or dict, every call that omits the argument modifies the same object, so data silently bleeds across independent calls.
# The Trap
def add_item(item, target=[]):
target.append(item)
return target
print(add_item("Apple")) # Output: ['Apple']
print(add_item("Banana")) # Output: ['Apple', 'Banana'] (Apple leaked!)
print(add_item.__defaults__) # Output: (['Apple', 'Banana'],)The same rule bites with non-mutable defaults that you expected to be fresh, such as def log(msg, when=datetime.now()), which stamps every message with the time the module was imported.
The fix: Use None as a sentinel and create the object inside the function.
def add_item(item, target=None):
if target is None:
target = []
target.append(item)
return target[[0] * 3] * 3 looks like a 3x3 grid, but it's one inner list referenced three times. Changing one "row" changes them all.
# The Trap
grid = [[0] * 3] * 3
grid[0][0] = 1
print(grid) # Output: [[1, 0, 0], [1, 0, 0], [1, 0, 0]]dict.fromkeys(keys, []) has the same problem: every key shares one list.
Why: Sequence multiplication copies references. That's harmless for immutable items like 0, but for a list it means three pointers to the same object.
The fix: Use a comprehension, which evaluates the inner expression once per row: [[0] * 3 for _ in range(3)], or {key: [] for key in keys}.
An attribute assigned in the class body belongs to the class, not to each instance. If it's mutable, every instance mutates the same object.
# The Trap
class Cart:
items = []
def add(self, item):
self.items.append(item)
alice, bob = Cart(), Cart()
alice.add("apple")
print(bob.items) # Output: ['apple'] (Bob never added anything!)Why: self.items finds no instance attribute, falls back to the class attribute, and .append() mutates it in place. (Assignment such as self.items = [...] would create an instance attribute, which is why this trap is easy to miss.)
The fix: Create per-instance state in __init__, or use a dataclass with field(default_factory=list). Dataclasses refuse a bare items: list = [] default for exactly this reason.
In the wild: Home Assistant removed class-level mutable defaults from an integration after a reviewer spotted them. They hadn't caused a bug yet only because nothing mutated them.
A for loop over a list tracks its position with a hidden index that moves forward on every step. Deleting an item shifts the remaining elements left to fill the gap, but the index still moves forward, so the loop silently skips the element after each deletion.
# The Trap
numbers = [2, 4, 6, 8]
for n in numbers:
if n % 2 == 0:
numbers.remove(n)
print(numbers) # Output: [4, 8]
# ("Remove every even number" left half of them behind!)Dicts and sets at least fail loudly with RuntimeError: dictionary changed size during iteration. Lists give you no such warning.
The fix: Build a new list, numbers = [n for n in numbers if n % 2 != 0], or loop over a copy with for n in numbers[:].
Tuples are immutable, but they can contain mutable objects like lists. Using += on a list inside a tuple raises a TypeError because the tuple slot cannot be reassigned. However, the list is modified before the exception is raised, so the data changes even though the operation formally failed.
# The Trap
t = (1, 2, [3, 4])
try:
t[2] += [5, 6]
except TypeError as e:
print(f"Error caught: {e}")
# The list mutated anyway despite the crash!
print(t) # Output: (1, 2, [3, 4, 5, 6])Why: t[2] += x runs as two steps: tmp = t[2].__iadd__(x) (which extends the list in place and succeeds), then t[2] = tmp (which fails).
The fix: Mutate the inner object directly with t[2].extend([5, 6]). There's no reassignment, so there's no error.
Functions created in a loop don't capture the loop variable's value at that iteration. Python closures bind to the variable, not its value, so when they run later they all read its current value, which is whatever the loop left behind.
# The Trap
funcs = [lambda: i for i in range(3)]
# You expect 0, 1, 2. But you get:
for f in funcs:
print(f()) # Output: 2, 2, 2The fix: Bind the value at definition time with a default argument, lambda i=i: i, or use functools.partial(func, i).
Unlike C-family languages, Python has no block scope. A variable assigned by a for loop lives in the enclosing function or module and keeps its final value after the loop ends, silently overwriting any earlier variable with the same name.
# The Trap
i = "Safe string"
for i in range(5):
pass
print(i) # Output: 4 (The original string was silently overwritten)The reverse also bites: if the loop body never runs, the name keeps its old value, so code after the loop can act on a stale item from a previous loop. List, dict and set comprehensions have their own scope in Python 3 and do not leak.
The fix: Use distinct names for loop variables, and keep loops inside small functions so names can't collide with module-level state.
Loop variables outlive their loop (see #10), but an exception name does the opposite. When the except block ends, Python deletes the name, including any unrelated variable that happened to have the same name before the try.
# The Trap
e = "important data"
try:
1 / 0
except ZeroDivisionError as e:
pass
print(e) # NameError: name 'e' is not definedWhy: The exception holds a traceback, which references the current stack frame, which references the exception. Python runs del e at the end of the block to break that cycle.
The fix: If you need the exception after the block, copy it to a different name inside the block: error = e.
If a function assigns to a name anywhere in its body, that name is local for the whole body. Reading it before the assignment raises UnboundLocalError, even when the assignment is unreachable.
# The Trap
total = 10
def report():
print(total) # UnboundLocalError is raised HERE...
if False:
total = 0 # ...because of this line, which never even runs.
report()The everyday version is counter += 1 inside a function, with counter defined at module level.
Why: Scope is decided when the function is compiled, not as it runs. The compiler sees an assignment to total, marks it local, and the global is never consulted.
The fix: Pass the value in as an argument and return the result. Use global or nonlocal only when you really mean to rebind the outer variable.
CPython pre-allocates a single object for each integer from -5 to 256, so every 256 is the same object. Above that range, each new integer is usually a new object, and is starts returning False for equal values.
# The Trap
a = int("256")
b = int("256")
print(a is b) # Output: True
c = int("257")
d = int("257")
print(c is d) # Output: False (Identity breaks!)It's even less predictable with literals. The compiler merges identical constants that are compiled together, so this prints True in a script but False when typed line by line into the REPL:
x = 257
y = 257
print(x is y) # Output: True in a .py file, False in the REPLThe same code giving different answers depending on how you run it is about as leaky as an abstraction gets. Strings have a similar mechanism, called interning.
The fix: Use == to compare values. Reserve is for singletons such as None, True, False and sentinel objects. Since Python 3.8, x is 257 with a literal triggers a SyntaxWarning for this reason.
bool is a subclass of int. True == 1, False == 0, and their hashes match too, so as dict keys or set members they are the same key. So is 1.0.
# The Trap
d = {1: "one", True: "yes", 1.0: "float one"}
print(d) # Output: {1: 'float one'} (Three keys collapsed into one)
print({0, False}) # Output: {0}
print(True + True) # Output: 2Note the result keeps the first key (1) but the last value. This matters when you mix flags and counts in one mapping, Counter, or JSON-like structure.
The fix: Don't mix bool and numeric keys. If the type genuinely matters, include it in the key: (type(k), k).
Adjacent string literals are joined at compile time. That's handy for long strings, but in a list it means one missing comma silently merges two items, and the list is one element short.
# The Trap
fruits = [
"apple",
"banana" # <- missing comma
"cherry",
]
print(fruits) # Output: ['apple', 'bananacherry']The fix: Put a trailing comma on every line of a multi-line collection, and enable pylint's implicit-str-concat check with check-str-concat-over-line-jumps=yes.
The comma creates a tuple, not the parentheses. A stray comma at the end of an assignment, often left over from copying a dict entry or function argument, silently changes the type.
# The Trap
timeout = 30,
print(timeout) # Output: (30,)
print(timeout + 5) # TypeError: can only concatenate tuple (not "int") to tupleMeanwhile (30) is just 30. You need (30,) for a one-element tuple.
The fix: Delete the comma. Ruff's COM818 (trailing comma on bare tuple) catches this automatically.
str.strip(), lstrip() and rstrip() treat their argument as a set of characters to remove, not as a prefix or suffix. They keep stripping until they hit a character outside that set.
# The Trap
print("report.txt".rstrip(".txt")) # Output: 'repor'
print("https://python.org".lstrip("https://")) # Output: 'ython.org'The fix: Use removesuffix() and removeprefix() (Python 3.9+), which remove an exact substring once.
round() rounds exact halves to the nearest even number, not always up, so round(2.5) is 2. On top of that, many decimal values can't be stored exactly as floats, so a "half" may not really be a half.
# The Trap
print(round(0.5), round(1.5), round(2.5)) # Output: 0 2 2
print(round(2.675, 2)) # Output: 2.67 (2.675 is stored as 2.67499999...)Why: Round-half-to-even (IEEE 754's default) avoids the upward bias of always rounding halves up, which matters when summing many rounded values.
The fix: For money and other human-facing decimals, use decimal.Decimal with an explicit mode: Decimal("2.675").quantize(Decimal("0.01"), rounding=ROUND_HALF_UP) gives Decimal('2.68').
A return, break or continue inside a finally block overrides whatever was in flight, including an exception. The error disappears without a trace.
# The Trap
def save():
try:
raise OSError("disk full")
finally:
return "saved" # The OSError vanishes.
print(save()) # Output: savedWhy: finally always runs last, and a return there replaces the pending outcome, whether that was a return value or an exception.
The fix: Keep finally for cleanup only and put the return after the block. Python 3.14 now emits a SyntaxWarning for this pattern (PEP 765).
Home Assistant Core is one of the largest asyncio codebases in open source: thousands of integrations, written by different people, sharing a single event loop. Several pitfalls above show up in its recent bug fixes, and its core has guard rails built specifically against them.
Bugs fixed:
| Fix | Pitfall |
|---|---|
| Alert notifications fired in a tight loop during the DST fall-back hour | #48: now() + delay is wall-clock arithmetic |
| Automatic backups ran an hour off after every DST change | #49: a parsed +02:00 is not a time zone |
| Hikvision updated state from a library thread | #24: loop APIs aren't thread-safe |
| Class-level mutable defaults in an integration | #6: shared class attributes |
Guard rails in core:
- Task tracking.
hass.async_create_task()stores every task in a set and removes it on completion, so tasks can't be garbage-collected mid-flight (#20). - Blocking-call detection.
block_async_io.pywrapstime.sleep,open,os.listdir,importlib.import_module, SSL certificate loading and more, and reports any call made from the event loop (#22). - Thread checks.
verify_event_loop_thread()comparesthreading.get_ident()with the loop's thread and raises when a loop-only API is called from elsewhere (#24). - Linting. The project enabled 19 more flake8-bugbear rules through ruff, covering mutable defaults, loop-variable shadowing and similar traps.
The lesson generalizes: when a pitfall can't be designed away, make it fail loudly. Every guard rail above turns a silent, intermittent bug into an immediate error with a clear message.
Each file in examples/ prints the surprising output and asserts it, so a script that runs without error confirms the behavior described above.
for f in examples/*.py; do python "$f" || echo "FAILED: $f"; doneBeen bitten by something that isn't here? See CONTRIBUTING.md. Good pitfalls are surprising, reproducible in a few lines, and have a clear fix.