fix: reject and rebuild stale rules caches - #3154
Conversation
A ruleset loaded from a cache written by an older build may contain feature indexes with a different schema, crashing at match time with AttributeError (missing bytes_prefix_index, mandiant#2961). Bump the cache format version so previous-format caches are rejected, validate the loaded ruleset's precomputed indexes against the current schema, and treat any mismatch as a cache miss (the invalid entry is deleted and the ruleset rebuilt).
|
@MP-GOWTHAM would you please explain the sequence of events that led to you encountering this issue? i'm surprised that the caching logic doesn't already notice this. |
|
@williballenthin Good question — I found this via the issue tracker rather than hitting it in the wild, so here's the sequence that produces it, and why the existing guards don't catch it:
Why the existing logic doesn't notice this:
So the fix makes load-time schema validation the source of truth (any mismatch = cache miss + rebuild), with the format version bump as the immediate invalidation for caches written by the previous format. |
Fixes #2961
A ruleset loaded from a rules cache written by an older build may contain feature indexes with a different schema, which crashed at match time with:
Changes
VERSION\x00\x00\x00\x01->\x00\x00\x00\x02), so caches written by the previous format are rejected and rebuilt.RuleSet._validate_feature_index()checks the precomputed feature indexes against the fields the current matching code expects. Any mismatch is treated as a cache miss — the invalid entry is deleted and the ruleset is rebuilt, instead of crashing mid-analysis.RuleCache.load: the magic/version/type/id checks no longer rely onassert(which is stripped underpython -O); they raiseAssertionErrorexplicitly so the invalid-cache path always triggers.Tests
test_ruleset_cache_old_format_version— a cache with the previous format version is rejected and deleted.test_ruleset_cache_stale_feature_index_schema— a cache whose feature indexes are missingbytes_prefix_indexis rejected and deleted instead of crashing at match time.Verification
pytest tests/test_rule_cache.py tests/test_rules.py tests/test_match.py tests/test_rules_insn_scope.py: 56 passed, 2 xfailedruff checkandruff format --check(with.github/ruff.toml): clean