Skip to content

Commit 5893aba

Browse files
gh-155725: Capture tracemalloc tracebacks without a thread state attached
Store traceback frame filenames as interned NUL terminated UTF-8 strings instead of Python str objects, so that capturing a traceback no longer uses or modifies Python objects. Threads without an attached thread state now capture their Python traceback by walking the frames of the thread state most recently bound to the thread; only threads which never had a thread state record the traceback as "<unknown>".
1 parent 69d37ec commit 5893aba

5 files changed

Lines changed: 256 additions & 87 deletions

File tree

Doc/c-api/memory.rst

Lines changed: 6 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -781,15 +781,14 @@ tracemalloc C API
781781
If memory block is already tracked, update the existing trace.
782782
783783
The function can be called from any thread, with or without an
784-
:term:`attached thread state`. If the calling thread has no attached
785-
thread state, the traceback of the allocation is recorded as
786-
``<unknown>``.
784+
:term:`attached thread state`. The traceback is captured using the
785+
Python thread state associated with the calling thread, even if it is
786+
not attached. If the thread has no Python thread state, the traceback
787+
is recorded as ``<unknown>``.
787788
788789
.. versionchanged:: next
789-
The function no longer acquires the :term:`GIL`. When called from a
790-
thread without an :term:`attached thread state`, the traceback is now
791-
recorded as ``<unknown>`` instead of the Python traceback of the
792-
calling thread.
790+
The function no longer acquires the :term:`GIL` nor creates a
791+
temporary thread state.
793792
794793
.. c:function:: int PyTraceMalloc_Untrack(unsigned int domain, uintptr_t ptr)
795794

Include/internal/pycore_tracemalloc.h

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -44,9 +44,10 @@ struct
4444
__attribute__((packed))
4545
#endif
4646
tracemalloc_frame {
47-
/* filename cannot be NULL: "<unknown>" is used if the Python frame
48-
filename is NULL */
49-
PyObject *filename;
47+
/* Interned NUL terminated UTF-8 (surrogatepass) string.
48+
Cannot be NULL: "<unknown>" is used if the Python frame filename
49+
cannot be captured. */
50+
const char *filename;
5051
unsigned int lineno;
5152
};
5253

@@ -85,7 +86,7 @@ struct _tracemalloc_runtime_state {
8586
Protected by TABLES_LOCK(). */
8687
size_t peak_traced_memory;
8788
/* Hash table used as a set to intern filenames:
88-
PyObject* => PyObject*.
89+
char* (NUL terminated UTF-8 string) => NULL.
8990
Protected by the TABLES_LOCK(). */
9091
_Py_hashtable_t *filenames;
9192
/* Buffer to store a new traceback in traceback_new().

Lib/test/test_tracemalloc.py

Lines changed: 4 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1047,22 +1047,17 @@ def check_track(self, release_gil):
10471047
size = tracemalloc.get_traced_memory()[0]
10481048

10491049
frames = self.track(release_gil, nframe)
1050-
if release_gil:
1051-
# PyTraceMalloc_Track() never acquires the GIL: without an
1052-
# attached thread state, the traceback is recorded as <unknown>
1053-
expected = tracemalloc.Traceback([("<unknown>", 0)])
1054-
else:
1055-
expected = tracemalloc.Traceback(frames)
1056-
self.assertEqual(self.get_traceback(), expected)
1050+
self.assertEqual(self.get_traceback(),
1051+
tracemalloc.Traceback(frames))
10571052

10581053
self.assertEqual(self.get_traced_memory(), self.size)
10591054

10601055
def test_track(self):
10611056
self.check_track(False)
10621057

10631058
def test_track_without_gil(self):
1064-
# check that calling PyTraceMalloc_Track() with the thread state
1065-
# detached (GIL released) records the trace with an unknown traceback
1059+
# check that calling PyTraceMalloc_Track() without the GIL
1060+
# (detached thread state) still captures the Python traceback
10661061
self.check_track(True)
10671062

10681063
def test_track_already_tracked(self):
Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
:mod:`tracemalloc` no longer acquires the :term:`GIL` nor creates a
2-
temporary thread state when tracing memory allocations. Memory allocations
3-
and :c:func:`PyTraceMalloc_Track` calls made by threads without an
4-
:term:`attached thread state` now record the traceback as ``<unknown>``
5-
instead of temporarily attaching a thread state to capture the Python
6-
traceback of the calling thread.
2+
temporary thread state to trace memory allocations: traceback frames now
3+
store plain UTF-8 strings instead of Python str objects, and tracebacks are
4+
captured using the Python thread state already associated with the calling
5+
thread, even if it is not attached. Threads without a Python thread state
6+
record the traceback as ``<unknown>``.

0 commit comments

Comments
 (0)