Skip to content

capnproto: bump to 1.4.0 - #106

Draft
greatgitsby wants to merge 1 commit into
commaai:masterfrom
greatgitsby:capnproto-1.4.0
Draft

capnproto: bump to 1.4.0#106
greatgitsby wants to merge 1 commit into
commaai:masterfrom
greatgitsby:capnproto-1.4.0

Conversation

@greatgitsby

@greatgitsby greatgitsby commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Matches the capnproto bundled by pycapnp 2.2.4. Context: commaai/openpilot#38761

After this merges and the wheel is published: bump pycapnp to 2.2.4 in openpilot (drops the 2.1.0 pin) and re-lock comma-deps-capnproto.

Verified locally: capnproto/build.sh, installed into an openpilot venv, regenerated capnp headers, full tools/op.sh build, cabana demo route 30s, tests/test_cabana.

pycapnp 2.2 memory leak

openpilot pinned pycapnp==2.1.0 (commaai/openpilot#38126) because 2.2 leaked in realtime processes (GC disabled). Root cause: from_dict touched self.schema.fields for every kwarg, and that wrapper forms a reference cycle with the builder, so each message is kept alive until a GC pass. Fixed upstream in capnproto/pycapnp@a27c849 (#401, released in 2.2.3) by only doing the lookup for str values.

Repro, GC disabled, 300k messages per case, fresh process per case:

import gc, resource
from openpilot.cereal import log
gc.disable()
for i in range(300000):
  log.Event.new_message(valid=False, logMonoTime=i)   # int/bool kwargs
  # log.Event.new_message(logMessage='x')             # str kwarg
print(resource.getrusage(resource.RUSAGE_SELF).ru_maxrss // 1024, "MB", gc.collect())
pycapnp bundled capnproto int/bool kwargs str kwarg
2.1.0 1.0.1 0 MB 0 MB
2.2.0 1.0.1 +9056 MB +8816 MB
2.2.1 1.0.1 +9057 MB +8816 MB
2.2.2 1.0.1 +9056 MB +8816 MB
2.2.3 1.0.1 0 MB +8816 MB
2.2.4 1.4.0 0 MB +8817 MB

The str kwarg path still leaks in 2.2.4 (the cycle itself was not fixed, only sidestepped). In openpilot the only caller passing a str kwarg is logmessaged (logMessage=record), which does not run with GC disabled. Every config_realtime_process user goes through messaging.new_message, which only passes valid and logMonoTime.

@adeebshihadeh

Copy link
Copy Markdown
Contributor

Don't we just have to bump pycapnp in openpilot's uv.lock?

@greatgitsby

Copy link
Copy Markdown
Contributor Author

i suppose so, but then we'll be using different versions of capnp between py and cpp.

separately, bumping pycapnp won't cause a mem leak like before, but there is still mem leak risk:

The str kwarg path still leaks in 2.2.4 (the cycle itself was not fixed, only sidestepped). In openpilot the only caller passing a str kwarg is logmessaged (logMessage=record), which does not run with GC disabled. Every config_realtime_process user goes through messaging.new_message, which only passes valid and logMonoTime.

capnproto/pycapnp#407 will fix it upstream

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants