fix(foundry): scale showmap replay timeout with fuzz budget#35
Open
grandizzy wants to merge 1 commit into
Open
fix(foundry): scale showmap replay timeout with fuzz budget#35grandizzy wants to merge 1 commit into
grandizzy wants to merge 1 commit into
Conversation
Replay cost grows with corpus size, so a fixed 30m cap silently truncated coverage capture on slow targets for long campaigns. Scale the budget with the fuzz budget instead; an explicit override still takes precedence.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The foundry showmap replay (the post-fuzz coverage-capture pass) had its timeout capped at a fixed 1800s (30m), only ever lowered to the fuzz budget for short runs. The replay re-executes the persisted corpus, so its cost is O(corpus size) and the corpus grows with the fuzz budget.
On long campaigns with slow targets the replay was killed at 30m before writing any showmap output, and the timeout was silently mapped to a completed run.
analyze.pythen dropped those trials as "empty coverage", collapsing the target to too-few-samples and producing a bogusinconclusiveverdict.Evidence (4h matrix runs)
inconclusive(n_samples=1)Change
fuzzers/foundry/run.sh: when no explicit override is set, the showmap replay budget now defaults to the fuzz budget instead of a fixed 30m cap. Replay is normally far faster than fuzzing (aave/superform finish in seconds regardless of cap), so this ceiling is only reached by genuinely slow targets and costs nothing for typical ones.SCFUZZBENCH_FOUNDRY_SHOWMAP_TIMEOUT_SECONDSstill overrides explicitly.Tests
test_foundry_run_showmap_args.py(renamed..._uses_bounded_default_timeout->..._scales_timeout_with_fuzz_budget); long-campaign now expects the full budget. All 5 tests in the file pass.Notes / follow-ups (not in this PR)