Make host callback result cap configurable#6
Merged
Conversation
Raise the default host callback result size from 64 KiB to 256 KiB and expose a host_result_bytes option on run(), run_micropython_wasi(), MicroPythonSession, and MicroPythonReplaySession. Wire the configured limit through a new micropython_wasm.host_result_cap import so the guest C host module can allocate according to the runtime setting instead of relying on a compile-time constant. Reuse the guest-side result buffer across host calls and register it as a MicroPython VM root, avoiding repeated large heap allocations while keeping the buffer visible to the GC. Increase the default fuel budget from 5M to 20M to match the 4x larger default host result cap, update the README examples and argument docs, and add tests for the new default, lower configured limits, and host_result_bytes validation. Rebuild the packaged micropython-wasi.wasm artifact so it includes the new host_result_cap import and reusable buffer behavior.
…on Python 3.11) Persistent sessions run Wasmtime in a background Python thread. When a wall-clock timeout interrupts guest code, Wasmtime raises a trap inside that thread and _thread_main stores the exception so the foreground run() call can report it. On Linux Python 3.11 with wasmtime 45.0.0, keeping that exception's traceback alive also keeps the _run_bootstrap() frame alive. That frame contains Wasmtime objects such as the engine, store, linker, module, and instance. The Python session thread is joined, but those Rust-backed objects can survive until interpreter shutdown through the traceback reference chain. CI then prints pytest's success summary and aborts during process teardown with a Rust panic from tokio-rt-worker: panic in a function that cannot unwind. Commit 06b58d1 fixed the normal close path by joining timer/session threads and clearing store references, but the trapped timeout path still retained teardown-sensitive Wasmtime state through the saved exception traceback. Detach __traceback__, __cause__, and __context__ before storing the thread error. The public error message is preserved because run() only formats the exception value, while the traceback frame references are released promptly enough for Wasmtime resources to be destroyed before Python shutdown. Add a regression assertion to the timeout test to ensure stored thread errors remain detached from traceback/cause/context chains. Reproduced the abort in Docker with python:3.11-slim and verified the full suite now exits cleanly there as well as locally via uv run pytest. Docker test commands: First reproduced the CI-shaped failure in a one-shot Linux Python 3.11 container: docker run --rm -v "$PWD":/work -w /work -e RUST_BACKTRACE=full python:3.11-slim sh -lc 'python -V && python -m pip install -U pip && pip --version && pip install . --group dev && python - <<"PY" import importlib.metadata print("wasmtime", importlib.metadata.version("wasmtime")) print("pytest", importlib.metadata.version("pytest")) PY python -m pytest' Then kept a container around for focused iteration: docker rm -f mpw311 >/dev/null 2>&1 || true docker run -d --name mpw311 -v "$PWD":/work -w /work -e RUST_BACKTRACE=full python:3.11-slim sleep infinity docker exec mpw311 sh -lc 'python -m pip install -U pip >/dev/null && pip install . --group dev >/dev/null && python - <<"PY" import importlib.metadata print("wasmtime", importlib.metadata.version("wasmtime")) print("pytest", importlib.metadata.version("pytest")) PY' Used the persistent container to isolate and verify the failing test file: docker exec mpw311 sh -lc 'python -m pytest tests/test_persistent_session.py -q; printf "exit=%s\n" "$?"' Finally verified the complete suite exited cleanly after the fix: docker exec mpw311 sh -lc 'python -m pytest -q; printf "exit=%s\n" "$?"' docker rm -f mpw311
Owner
Author
|
That CI failure was gnarly, had Codex figure it out using Docker to replicate CI on Python 3.11 and write a very detailed commit message: ac5b88b |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs:
Raise the default host callback result size from 64 KiB to 256 KiB and expose a host_result_bytes option on run(), run_micropython_wasi(), MicroPythonSession, and MicroPythonReplaySession.
Wire the configured limit through a new micropython_wasm.host_result_cap import so the guest C host module can allocate according to the runtime setting instead of relying on a compile-time constant.
Reuse the guest-side result buffer across host calls and register it as a MicroPython VM root, avoiding repeated large heap allocations while keeping the buffer visible to the GC.
Increase the default fuel budget from 5M to 20M to match the 4x larger default host result cap, update the README examples and argument docs, and add tests for the new default, lower configured limits, and host_result_bytes validation.
Rebuild the packaged micropython-wasi.wasm artifact so it includes the new host_result_cap import and reusable buffer behavior.