Skip to content

gh-159244: Fix crash in dict() on a concurrently mutated key/value pair - #159245

Open
christianaurichzm wants to merge 1 commit into
python:mainfrom
christianaurichzm:gh-159244-dict-pair-race
Open

christianaurichzm wants to merge 1 commit into
python:mainfrom
christianaurichzm:gh-159244-dict-pair-race

Conversation

@christianaurichzm

Copy link
Copy Markdown
Contributor

Fixes #159244

In the free-threaded build, dict(), dict.update() and PyDict_MergeFromSeq2() (all through merge_from_seq2_lock_held()) check that a list key/value pair has two items, then read them with PySequence_Fast_GET_ITEM(), which returns borrowed references, without holding the list's lock. If another thread empties the list in between, the items are read from a freed or NULL array.

This change reads the size and both items under Py_BEGIN_CRITICAL_SECTION_SEQUENCE_FAST, as str.join() and bytes.join() already do. Both items are increfed before the lock is released, so the rest of the loop works on strong references as before.

The list's critical section is nested inside the dict's. No Python code runs while the list is locked.

In the default build the macros expand to an empty block.

Verification

  • The new test in test_free_threading.test_dict crashed an unpatched free-threaded ASan debug build in 5 out of 5 runs and passed 28 out of 28 runs with the fix. A longer stress script (2 mutating threads, 4 readers, 15 seconds) segfaults without the fix and runs cleanly with it.
  • test_dict, test_capi.test_dict and test_free_threading.test_dict pass on the free-threaded build, and test_dict and test_free_threading.test_dict pass with -R 3:3. test_dict passes on the default build.

…lue pair

In the free-threaded build, dict() and dict.update() checked that a
list key/value pair had two items and then read them with borrowed
references without holding its lock, so another thread could empty
the list in between and the items were read from a freed or NULL
array.

Read the size and both items under
Py_BEGIN_CRITICAL_SECTION_SEQUENCE_FAST, as bytes.join() and
str.join() already do.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Free-threading: dict() crashes when a key/value pair list is mutated concurrently

2 participants