Increased memory usage in the free-threaded build with C extensions

[leveretconey]

Bug report

Bug description:

Summary

The free-threaded build splits allocation into two independent pools: Python
objects are served by mimalloc, while C extensions keep calling the system
allocator. On Linux the combination wastes a lot of memory, because:

• the two pools never share memory. Python objects can no longer land in the
  space that C extensions have freed in the glibc arena, the way they do with
  the GIL, where obmalloc forwards everything above 512 bytes to the system
  allocator. In the free-threaded build _PyObject_MiMalloc() sends every
  object to mimalloc's own heaps, and the arena only ever sees C-extension
  traffic.
• glibc does not hand that space back to the OS either. It only trims the
  top of the heap, so a freed region sitting below a live allocation stays
  resident — which is the normal steady state for a producer that frees the
  previous buffer while holding the latest one.

Neither condition is a problem on its own: with the GIL the freed arena space
keeps being recycled by Python objects, and with an allocator that returns
memory by itself (jemalloc, tcmalloc) there is nothing left to strand. It is
specifically free-threaded + glibc that ends up holding memory that is free,
resident and unusable — 2x RSS in the reproducer below, ~11 GiB in a real
training job.

gh-135898 collects the known reasons the free-threaded build uses more memory;
this mechanism is not among them. It is also distinct from gh-135153: the memory
in question sits in the glibc arena, and no mimalloc setting affects it. It
only appears when a workload mixes Python objects with C-extension malloc()
traffic, which is why pure-Python benchmarks do not surface it.

What we saw in production

A PyTorch training job with 32 DataLoader worker processes. Each worker loops:
read images from disk, decode and augment them with NumPy and OpenCV, collate
them into a batch, hand the batch to the main process, repeat. Once a batch has
been handed off the worker has no further use for it — the intermediate arrays
are dropped and only the most recent batch is briefly kept alive.

So a worker's live set is small and constant: nothing accumulates in Python
objects, the same amount of image data flows through in both builds, and the
native buffers really are freed after each batch. Same machine, same library
versions, switching only the interpreter:

	GIL build	free-threaded build
whole process tree PSS	34.6 GiB	45.6 GiB
DataLoader workers PSS	15.8 GiB	26.4 GiB

mallinfo2() inside a worker at steady state shows where it goes:

per worker	GIL build	free-threaded build
arena	1085 MB	1397 MB
uordblks (in use)	858 MB	703 MB
fordblks (free, retained)	227 MB	694 MB
arena utilisation	79%	50%

The workers free the same amount of native memory in both builds. The difference
is what happens to it afterwards: in the free-threaded build half of each
worker's arena sits free but resident and is never picked up again, ~467 MB per
worker. Nothing is leaking in the usual sense; the memory is free, it is simply
neither reused nor returned.

Both mitigations were then applied to the real training job, and both recover
most of it without touching mimalloc:

free-threaded build	tree PSS	worker PSS
as-is	45.6 GiB	26.4 GiB
malloc_trim(0) in the worker, once per batch	35.9 GiB	16.5 GiB
LD_PRELOAD=libjemalloc.so.2, MALLOC_CONF=narenas:2,background_thread:true,dirty_decay_ms:1000	36.0 GiB	15.8 GiB
(GIL build, for reference)	34.6 GiB	15.8 GiB

Step time was unchanged in both cases. MIMALLOC_PURGE_DELAY and the mimalloc
arena purge options had no reproducible effect.

Reproducer

The script below is modelled on the DataLoader worker above, reduced to the two
things that matter: a stream of native buffers that are allocated, used and
freed, and a set of Python objects that outlives them. It uses the standard
library only (ctypes for malloc/free/mallinfo2), no third-party packages
and no threads. Run it under two builds of the same CPython source; the
numbers below are CPython 3.14.7 (GIL and free-threaded builds), glibc 2.39,
Ubuntu 24.04, x86_64.

#!/usr/bin/env python3
"""Free-threaded CPython does not reuse the system allocator's free memory.

Phase 1: churn medium buffers through glibc malloc/free (a C extension stand-in),
         keeping one live block at the top of the arena so glibc cannot trim.
         -> ~STRAND_MB of resident, free-but-retained arena space.
Phase 2: allocate PYOBJ_MB of long-lived Python objects above the 512-byte
         obmalloc threshold.
         GIL build: served by glibc, reuses the stranded space -> RSS flat.
         -t  build: served by mimalloc, asks the OS for new memory -> RSS +2 GB.

TRIM=1 calls malloc_trim(0) after phase 1.
"""
import ctypes
import gc
import os
import sys

libc = ctypes.CDLL(None)
libc.malloc.restype = ctypes.c_void_p
libc.malloc.argtypes = [ctypes.c_size_t]
libc.free.argtypes = [ctypes.c_void_p]
libc.memset.argtypes = [ctypes.c_void_p, ctypes.c_int, ctypes.c_size_t]
libc.malloc_trim.argtypes = [ctypes.c_size_t]


class Mallinfo2(ctypes.Structure):
    _fields_ = [(n, ctypes.c_size_t) for n in (
        "arena", "ordblks", "smblks", "hblks", "hblkhd",
        "usmblks", "fsmblks", "uordblks", "fordblks", "keepcost",
    )]


libc.mallinfo2.restype = Mallinfo2

MB = 1024 * 1024
STRAND_MB = int(os.environ.get("STRAND_MB", "2048"))
CHUNK_KB = int(os.environ.get("CHUNK_KB", "104"))  # under glibc's mmap threshold
PYOBJ_MB = int(os.environ.get("PYOBJ_MB", "2048"))
# 4000 + bytes header stays under 4 KiB in both builds (header is 33 vs 49), so
# neither allocator jumps to a bigger size class.
PYOBJ_PAYLOAD = int(os.environ.get("PYOBJ_PAYLOAD", "4000"))
TRIM = os.environ.get("TRIM", "0") == "1"


def rss_mb():
    for line in open("/proc/self/status"):
        if line.startswith("VmRSS:"):
            return int(line.split()[1]) / 1024


def report(label):
    info = libc.mallinfo2()
    print(f"  {label:26s}"
          f"  process RSS={rss_mb():7.1f}MB"
          f"  | glibc arena: total={info.arena / MB:7.1f}MB"
          f"  in-use={info.uordblks / MB:7.1f}MB"
          f"  free={info.fordblks / MB:7.1f}MB")


def strand_arena():
    chunk = CHUNK_KB * 1024
    ptrs = []
    for _ in range((STRAND_MB * MB) // chunk):
        ptr = libc.malloc(chunk)
        libc.memset(ptr, 1, chunk)
        ptrs.append(ptr)
    pin = libc.malloc(chunk)  # allocated last -> pins the top of the arena
    libc.memset(pin, 1, chunk)
    for ptr in ptrs:
        libc.free(ptr)
    return pin


def main():
    build = "free-threaded" if not sys._is_gil_enabled() else "GIL"
    print(f"Python {sys.version.split()[0]} ({build})   "
          f"malloc_trim={'on' if TRIM else 'off'}")
    pin = strand_arena()
    gc.collect()
    report("after phase1 (churn)")

    if TRIM:
        libc.malloc_trim(0)
        report("after malloc_trim(0)")

    count = (PYOBJ_MB * MB) // (PYOBJ_PAYLOAD + sys.getsizeof(b""))
    objects = [bytes(PYOBJ_PAYLOAD) for _ in range(count)]
    gc.collect()
    report(f"after phase2 ({PYOBJ_MB}MB objs)")

    libc.free(pin)
    del objects


main()

Results

$ python3.14 repro.py
Python 3.14.7 (GIL)   malloc_trim=off
  after phase1 (churn)        process RSS= 2064.6MB  | glibc arena: total= 2051.2MB  in-use=    1.8MB  free= 2049.4MB
  after phase2 (2048MB objs)  process RSS= 2075.6MB  | glibc arena: total= 2062.2MB  in-use= 2061.9MB  free=    0.2MB

$ python3.14t repro.py
Python 3.14.7 (free-threaded)   malloc_trim=off
  after phase1 (churn)        process RSS= 2071.6MB  | glibc arena: total= 2048.5MB  in-use=    0.2MB  free= 2048.2MB
  after phase2 (2048MB objs)  process RSS= 4152.2MB  | glibc arena: total= 2048.5MB  in-use=    0.2MB  free= 2048.2MB

Both builds enter phase 2 with ~2 GB of resident, free arena space. The GIL
build consumes it (free 2049 → 0.2 MB) and RSS grows by 11 MB for 2 GB of
objects. The free-threaded build never touches it (free stays at 2048 MB) and
RSS grows by the full 2081 MB — 2075 MB vs 4152 MB, +100% for the same work.

Both mitigations behave as the mechanism predicts. malloc_trim(0), called once
after phase 1, gives the stranded pages back so phase 2 starts from a clean
slate:

$ TRIM=1 python3.14t repro.py
Python 3.14.7 (free-threaded)   malloc_trim=on
  after phase1 (churn)        process RSS= 2071.5MB  | glibc arena: total= 2048.5MB  in-use=    0.2MB  free= 2048.2MB
  after malloc_trim(0)        process RSS=   23.6MB  | glibc arena: total= 2048.4MB  in-use=    0.2MB  free= 2048.2MB
  after phase2 (2048MB objs)  process RSS= 2103.9MB  | glibc arena: total= 2048.4MB  in-use=    0.2MB  free= 2048.2MB

With jemalloc preloaded there is nothing to strand in the first place — phase 1
ends at 220 MB instead of 2071 MB — and both builds then behave the same:

$ LD_PRELOAD=libjemalloc.so.2 python3.14t repro.py
Python 3.14.7 (free-threaded)   malloc_trim=off
  after phase1 (churn)        process RSS=  219.8MB  | glibc arena: total=    0.0MB  in-use=    0.0MB  free=    0.0MB
  after phase2 (2048MB objs)  process RSS= 2298.2MB  | glibc arena: total=    0.0MB  in-use=    0.0MB  free=    0.0MB

$ LD_PRELOAD=libjemalloc.so.2 python3.14 repro.py
Python 3.14.7 (GIL)   malloc_trim=off
  after phase1 (churn)        process RSS=  223.4MB  | glibc arena: total=    0.0MB  in-use=    0.0MB  free=    0.0MB
  after phase2 (2048MB objs)  process RSS= 2365.6MB  | glibc arena: total=    0.0MB  in-use=    0.0MB  free=    0.0MB

Final RSS across configurations:

build	allocator	final RSS
GIL	glibc	2075 MB
free-threaded	glibc	4152 MB
free-threaded	glibc + malloc_trim(0)	2104 MB
free-threaded	jemalloc	2298 MB
GIL	jemalloc	2366 MB

Two observations:

• malloc_trim(0) releases the stranded pages even though the top of the arena
  is pinned — modern glibc walks interior free chunks and MADV_DONTNEEDs them.
  The memory is therefore not irrecoverably fragmented; glibc just never does
  this on its own. Note fordblks does not change, as it accounts for free
  chunks rather than resident pages; only RSS shows the effect.
• under jemalloc both builds behave the same (2298 vs 2366 MB), because its
  decay returns the freed pages anyway and there is nothing left to reuse. The
  penalty is specific to free-threaded + a retaining allocator, glibc being
  the default one on Linux.

CPython versions tested on:

3.14

Operating systems tested on:

Linux

[picnixz]

I do not think we can do much here but I can ask @colesbury