Repository navigation
Use of MmapedDict results in fd leaks (resource leaks) #1008
Copy link
Copy link
Open
Description
When running the test suite with -Wdefault, I get multiple reports of resource leaks, plus one of the tests is flaky and sometimes fails because of ResourceWarning being emitted where UserWarning was expected first:
$ python -m pytest -Wdefault
========================================================= test session starts =========================================================
platform linux -- Python 3.11.8, pytest-8.0.0, pluggy-1.4.0
rootdir: /tmp/client_python
collected 309 items
tests/openmetrics/test_exposition.py .................... [ 6%]
tests/openmetrics/test_parser.py ............................................. [ 21%]
tests/test_asgi.py ssssssss [ 23%]
tests/test_core.py .................................................................................................... [ 55%]
tests/test_exposition.py .......................................................... [ 74%]
tests/test_gc_collector.py .. [ 75%]
tests/test_graphite_bridge.py ....... [ 77%]
tests/test_multiprocess.py .....................F....... [ 87%]
tests/test_parser.py ........................ [ 94%]
tests/test_platform_collector.py .. [ 95%]
tests/test_process_collector.py .... [ 96%]
tests/test_samples.py .. [ 97%]
tests/test_twisted.py s [ 97%]
tests/test_wsgi.py ....... [100%]
============================================================== FAILURES ===============================================================
_____________________________________________ TestMultiProcess.test_remove_clear_warning ______________________________________________
self = <tests.test_multiprocess.TestMultiProcess testMethod=test_remove_clear_warning>
def test_remove_clear_warning(self):
os.environ['PROMETHEUS_MULTIPROC_DIR'] = self.tempdir
with warnings.catch_warnings(record=True) as w:
values.ValueClass = get_value_class()
registry = CollectorRegistry()
collector = MultiProcessCollector(registry)
counter = Counter('c', 'help', labelnames=['label'], registry=None)
counter.labels('label').inc()
counter.remove('label')
counter.clear()
assert os.environ['PROMETHEUS_MULTIPROC_DIR'] == self.tempdir
> assert issubclass(w[0].category, UserWarning)
E AssertionError: assert False
E + where False = issubclass(<class 'ResourceWarning'>, UserWarning)
E + where <class 'ResourceWarning'> = <warnings.WarningMessage object at 0x7f22956b82d0>.category
tests/test_multiprocess.py:395: AssertionError
========================================================== warnings summary ===========================================================
tests/test_exposition.py::TestPushGateway::test_push_with_tls_auth_handler
/usr/lib/python3.11/urllib/parse.py:480: ResourceWarning: unclosed <socket.socket fd=37, family=2, type=1, proto=0, laddr=('127.0.0.1', 40145)>
for b in _UNSAFE_URL_BYTES_TO_REMOVE:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_exposition.py::TestPushGateway::test_push_with_tls_auth_handler
/usr/lib/python3.11/urllib/parse.py:480: ResourceWarning: unclosed <socket.socket fd=38, family=2, type=1, proto=0, laddr=('127.0.0.1', 46167)>
for b in _UNSAFE_URL_BYTES_TO_REMOVE:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_collect
<frozen os>:676: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmp9on9bz25/counter_1334413.db' mode='ab+' closefd=True>
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_all
/usr/lib/python3.11/glob.py:176: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpp8qzgxdb/counter_1.db' mode='ab+' closefd=True>
with contextlib.closing(_iterdir(dirname, dir_fd, dironly)) as it:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_all
/usr/lib/python3.11/glob.py:176: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpp8qzgxdb/gauge_all_1.db' mode='ab+' closefd=True>
with contextlib.closing(_iterdir(dirname, dir_fd, dironly)) as it:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_all
/usr/lib/python3.11/glob.py:176: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpp8qzgxdb/histogram_1.db' mode='ab+' closefd=True>
with contextlib.closing(_iterdir(dirname, dir_fd, dironly)) as it:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_all
/usr/lib/python3.11/glob.py:176: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpac3ow5d8/counter_1.db' mode='ab+' closefd=True>
with contextlib.closing(_iterdir(dirname, dir_fd, dironly)) as it:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_all
/usr/lib/python3.11/glob.py:176: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpac3ow5d8/gauge_all_1.db' mode='ab+' closefd=True>
with contextlib.closing(_iterdir(dirname, dir_fd, dironly)) as it:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_all
/usr/lib/python3.11/glob.py:176: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpac3ow5d8/histogram_1.db' mode='ab+' closefd=True>
with contextlib.closing(_iterdir(dirname, dir_fd, dironly)) as it:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_all
/usr/lib/python3.11/glob.py:176: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpx3sjgfk6/counter_1.db' mode='ab+' closefd=True>
with contextlib.closing(_iterdir(dirname, dir_fd, dironly)) as it:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_all
/usr/lib/python3.11/glob.py:176: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpq0v2xj70/counter_123.db' mode='ab+' closefd=True>
with contextlib.closing(_iterdir(dirname, dir_fd, dironly)) as it:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_all
/usr/lib/python3.11/glob.py:176: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpq0v2xj70/counter_456.db' mode='ab+' closefd=True>
with contextlib.closing(_iterdir(dirname, dir_fd, dironly)) as it:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
/tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpoa1u3zei/gauge_liveall_123.db' mode='ab+' closefd=True>
with open(filename, 'rb') as infp:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
/tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpoa1u3zei/gauge_liveall_456.db' mode='ab+' closefd=True>
with open(filename, 'rb') as infp:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
/tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpkwjjpl18/gauge_livemax_123.db' mode='ab+' closefd=True>
with open(filename, 'rb') as infp:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
/tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpkwjjpl18/gauge_livemax_456.db' mode='ab+' closefd=True>
with open(filename, 'rb') as infp:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
/tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmp02r2ouao/gauge_livemin_123.db' mode='ab+' closefd=True>
with open(filename, 'rb') as infp:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
/tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmp02r2ouao/gauge_livemin_456.db' mode='ab+' closefd=True>
with open(filename, 'rb') as infp:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
/tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpryrzvlpo/gauge_livemostrecent_123.db' mode='ab+' closefd=True>
with open(filename, 'rb') as infp:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
/tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpryrzvlpo/gauge_livemostrecent_456.db' mode='ab+' closefd=True>
with open(filename, 'rb') as infp:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
/tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpribv4xj9/gauge_livesum_123.db' mode='ab+' closefd=True>
with open(filename, 'rb') as infp:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
/tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpribv4xj9/gauge_livesum_456.db' mode='ab+' closefd=True>
with open(filename, 'rb') as infp:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
/tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmp22gzxk2n/gauge_max_123.db' mode='ab+' closefd=True>
with open(filename, 'rb') as infp:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
/tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmp22gzxk2n/gauge_max_456.db' mode='ab+' closefd=True>
with open(filename, 'rb') as infp:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
/tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmp6n8vag2o/gauge_min_123.db' mode='ab+' closefd=True>
with open(filename, 'rb') as infp:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
/tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmp6n8vag2o/gauge_min_456.db' mode='ab+' closefd=True>
with open(filename, 'rb') as infp:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMmapedDict::test_expansion
/tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/unittest.py:211: ResourceWarning: unclosed file <_io.BufferedRandom name='/tmp/tmptd9lerf6'>
self._obj = None
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMmapedDict::test_multi_expansion
/tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/unittest.py:211: ResourceWarning: unclosed file <_io.BufferedRandom name='/tmp/tmp4ho56w3t'>
self._obj = None
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMmapedDict::test_process_restart
/tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/unittest.py:211: ResourceWarning: unclosed file <_io.BufferedRandom name='/tmp/tmpp3siphvy'>
self._obj = None
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMmapedDict::test_process_restart
/tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/runner.py:546: ResourceWarning: unclosed file <_io.BufferedRandom name='/tmp/tmpqi5kklpc'>
fin()
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_parser.py::TestParse::test_untyped
/tmp/client_python/tests/test_parser.py:18: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmp8lpdamo1/summary_123.db' mode='ab+' closefd=True>
for sa, sb in zip(a.samples, b.samples):
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_parser.py::TestParse::test_untyped
/tmp/client_python/tests/test_parser.py:18: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmp8lpdamo1/summary_456.db' mode='ab+' closefd=True>
for sa, sb in zip(a.samples, b.samples):
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
======================================================= short test summary info =======================================================
FAILED tests/test_multiprocess.py::TestMultiProcess::test_remove_clear_warning - AssertionError: assert False
======================================== 1 failed, 299 passed, 9 skipped, 32 warnings in 4.10s ========================================An example tracemalloc output:
$ PYTHONTRACEMALLOC=20 python -m pytest -Wdefault
[…]
tests/test_parser.py::TestParse::test_untyped
/usr/lib/python3.11/typing.py:362: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpyaxyl9i6/summary_456.db' mode='ab+' closefd=True>
return cached(*args, **kwds)
Object allocated at:
File "/tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/runner.py", line 114
runtestprotocol(item, nextitem=nextitem)
File "/tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/runner.py", line 133
reports.append(call_and_report(item, "call", log))
File "/tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/runner.py", line 226
call = call_runtest_hook(item, when, **kwds)
File "/tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/runner.py", line 265
return CallInfo.from_call(
File "/tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/runner.py", line 345
result: Optional[TResult] = func()
File "/tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/runner.py", line 266
lambda: ihook(item=item, **kwds), when=when, reraise=reraise
File "/tmp/client_python/.tox/py311/lib/python3.11/site-packages/pluggy/_hooks.py", line 501
return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
File "/tmp/client_python/.tox/py311/lib/python3.11/site-packages/pluggy/_manager.py", line 119
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
File "/tmp/client_python/.tox/py311/lib/python3.11/site-packages/pluggy/_callers.py", line 102
res = hook_impl.function(*args)
File "/tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/runner.py", line 173
item.runtest()
File "/tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/unittest.py", line 333
self._testcase(result=self) # type: ignore[arg-type]
File "/usr/lib/python3.11/unittest/case.py", line 678
return self.run(*args, **kwds)
File "/usr/lib/python3.11/unittest/case.py", line 623
self._callTestMethod(testMethod)
File "/usr/lib/python3.11/unittest/case.py", line 579
if method() is not None:
File "/tmp/client_python/tests/test_multiprocess.py", line 79
s2 = Summary('s', 'help', registry=None)
File "/tmp/client_python/prometheus_client/metrics.py", line 151
self._metric_init()
File "/tmp/client_python/prometheus_client/metrics.py", line 513
self._count = values.ValueClass(self._type, self._name, self._name + '_count', self._labelnames,
File "/tmp/client_python/prometheus_client/values.py", line 68
self.__reset()
File "/tmp/client_python/prometheus_client/values.py", line 82
files[file_prefix] = MmapedDict(filename)
File "/tmp/client_python/prometheus_client/mmap_dict.py", line 64
self._f = open(filename, 'rb' if read_mode else 'a+b')Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels
Activity
mgorny commentedon Feb 16, 2024
This is with 0.20.0 (7a80f00).
claws commentedon Dec 2, 2024
This seems to be because the MmapedDict objects in the 'files' attribute are not having '.close()' called on them. As a workaround, in my unittests I monkey patched a del function onto them that calls the '.close()' function when they are garbage collected and this stopped the resource warnings. This is obviously a hack but was the best I could do at the time. The MmapedDict class wrapped in a function using the 'files' closure attribute makes it hard to nicely clean up.
For my own future reference, this is is how I silenced the ResourceWarning warnings:
ajaykr0905 commentedon Oct 17, 2026
I checked the test-suite reproduction against the reported 0.20.0 commit and current main, using the same environment (macOS arm64, CPython 3.13.9, pytest 9.1.1):
The old checkout emits
ResourceWarning: unclosed filefor the multiprocess.dbfiles. On current main,python -m pytest -Wdefault -qalso completes with 428 passed, 14 skipped and 28 passing subtests, with no warnings summary. Optional integrations were not all installed.The relevant cleanup already landed in #1185 (merged July 24, 2026): it added
values.close_all_multiprocess_files()and calls from multiprocess test teardown. That appears to address the test-suite reproduction here, so I have not opened a duplicate fix.This does not establish that every application-level descriptor lifetime concern is resolved: it only checks the reported test-suite path, and not the original Linux/Python 3.11 environment. If there is a remaining current-version application reproducer, that would help separate it from the cleanup already merged.
AI assistance disclosure: Codex inspected the source and executed these local comparisons; this is not a claim of independent manual testing.