Skip to content

[FIX] Stop the sanitizer from skipping repeated launches - #518

Open
mark14wu wants to merge 1 commit into
mainfrom
fix/sanitizer-relaunch-cache
Open

mark14wu wants to merge 1 commit into
mainfrom
fix/sanitizer-relaunch-cache

Conversation

@mark14wu

Copy link
Copy Markdown
Collaborator

Summary

SymbolicSanitizer.pre_run_callback kept a module-level cache (_fn_symbolic_cache_set) keyed on the kernel, the grid, the tensor shapes, strides, and dtypes, and the scalar arguments. On a cache hit it returned False for block (0, 0, 0). Tensor contents are not part of the key, but the sanitizer concretizes loaded values that feed addresses, masks, or branches. So when a later launch had different data:

  • a launch whose grid has one block was skipped entirely, and
  • in a launch with more blocks, an out-of-bounds access in block 0 was missed.

This PR removes the cache. grid_callback and pre_run_callback stay as plain forwarders to SymbolicClient, because the Sanitizer factory's stubs would otherwise shadow them.

Removing the cache costs little. After block 0 is skipped, cache_grid is already cleared, so block 1 goes through SymbolicClient.pre_run_callback and runs the same symbolic analysis that block 0 would have run. Only one-block launches, and block 0 of launches that need the full grid, are now analyzed where they used to be skipped.

benchmarks/bench_sanitizer.py no longer imports or clears the cache.

Fixes #517.

Tests

  • New test_relaunch_with_new_loaded_indices_is_checked[1|2] in tests/end_to_end/test_sanitizer.py. The test runs a gather kernel twice with the same shapes, first with in-bounds indices and then with an out-of-bounds index in block 0, for grids of 1 and 2 blocks. Both cases fail on main and pass with this PR.
  • tests/unit/test_sanitizer.py: the two cache-hit/miss tests are replaced by test_pre_run_callback_runs_replayed_launch, which fails on main. The post-run lifecycle tests no longer set or check cache_args / cache_grid.
  • pytest tests/: 366 passed, 6 skipped on main; 367 passed, 6 skipped with this PR (Python 3.12, Triton 3.8.0, tilelens installed in editable mode with its CLIs on PATH).
  • ruff check and ruff format --check (v0.1.9, as pinned in .pre-commit-config.yaml) pass on all four changed files.

Note on #516

The test added in #516 imports _fn_symbolic_cache_set and calls .clear() on it. Whichever of the two PRs merges second needs to drop that import and call.

🤖 Generated with Claude Code

https://claude.ai/code/session_016HMpCLXdUVNfMbJTj8aHE2

The sanitizer's module-level `_fn_symbolic_cache_set` skipped block
(0, 0, 0) of a launch whose kernel, grid, and argument shapes matched
an earlier launch. Tensor contents were not part of the key, so an
out-of-bounds access that depends on loaded values was missed when it
fell in block 0, and missed for the whole launch when the grid had one
block. Remove the cache; keep `grid_callback` and `pre_run_callback` as
forwarders to `SymbolicClient`.

Fixes #517.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HMpCLXdUVNfMbJTj8aHE2
@github-actions

Copy link
Copy Markdown

Performance Benchmark

Benchmark main (min) PR (min) Change Samples
gemm N/A N/A N/A 0 / 0
gemm_oob N/A N/A N/A 0 / 0
indirect_load N/A N/A N/A 0 / 0
nested_loop N/A N/A N/A 0 / 0
block_pointer_loop_advance N/A N/A N/A 0 / 0
liger_jsd N/A N/A N/A 0 / 0
flaggems_layernorm N/A N/A N/A 0 / 0
swiglu 0.093s 0.094s +0.7% 20 / 20
cross_entropy N/A N/A N/A 0 / 0
fused_linear_jsd N/A N/A N/A 0 / 0
Total N/A N/A N/A N/A

Iterations: 1 warmup + 20 measured
Samples are shown as main / PR; long pytest benchmarks may use fewer samples.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Sanitizer skips re-launches whose addresses, masks, or branches depend on loaded values

1 participant