MoE timing-replay: deterministic emulator timing + route-trace harnesses#97
Open
qichao-arlo-wang wants to merge 4 commits into
Open
MoE timing-replay: deterministic emulator timing + route-trace harnesses#97qichao-arlo-wang wants to merge 4 commits into
qichao-arlo-wang wants to merge 4 commits into
Conversation
This was referenced Jul 23, 2026
qichao-arlo-wang
force-pushed
the
feat/moe-timing
branch
from
July 23, 2026 17:25
6cf572b to
0854354
Compare
Deterministic event ordering (monotonic timer sequence ids; address-ordered HBM gather / ramulator transfers), a cycle-attributed serde stage profile (schema v2) with a routed-MoE classification-coverage guard that warns / fails loudly when the compiler's ASM comment vocabulary drifts, and an opt-in prefetch/ compute overlap model that accumulates in the picosecond domain (rounding to cycles once) and treats write-out ops as dependency-free. A repeat gate and per-run dump cwd round out the Python runner. Co-authored-by: Michael C Li <mcl123@ee-beholder0.ee.ic.ac.uk> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…iming/replay) A dependency-free route-trace schema and validator (hardened to append errors rather than crash on malformed input), a sample-trace generator, a replay runner, timing validation gates (structural, not magic-number), and result export. Outputs default to a git-ignored in-repo location; PLENA_OUT_ROOT overrides the base. Co-authored-by: Michael C Li <mcl123@ee-beholder0.ee.ic.ac.uk> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Generate true Qwen3 router traces from tokenized inputs and local weights, convert them to the replay schema, and replay through the emulator. The replay gate is an honest zero-input shape smoke (numerical correctness is validated by the routed-MoE op tests, not here); the resumable batch runner skips only genuinely-passed traces; the compiler submodule resolves on a standard checkout. Co-authored-by: Michael C Li <mcl123@ee-beholder0.ee.ic.ac.uk> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Stratified largest-remainder subset selection, a serial-vs-parallel determinism gate that no longer false-negatives when stage profiling is off, and checkpointed parallel replay that is resilient to individual unreadable traces. Co-authored-by: Michael C Li <mcl123@ee-beholder0.ee.ic.ac.uk> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
qichao-arlo-wang
force-pushed
the
feat/moe-timing
branch
from
July 23, 2026 22:15
0854354 to
13423dc
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does
Consolidates the MoE timing-replay stack — previously split across #93, #94, #95, #96 — into a single branch against
main, with the review findings from those PRs fixed. It supersedes and replaces all four (which are being closed).Organized as four thematic commits:
Fixes applied during consolidation
check_timing_goldens.py,overlap_golden_workloads.json,run_timing_smoke.sh, thetiming-smokerecipe): they depended on out-of-repo build artifacts and were not CI-runnable. The overlap-timing engine (timing_overlay.rs, unit-tested) and the standalonetiming_validation_gates.pyare kept.Mutex<Option<u64>>(a module cell is still needed becauseExecutor::spawnrequiresFuture<Output = ()>), and clarified the muddled overlap log line.window1_p{1,2,3}outputs default to a git-ignored in-repooutputs/(override the base withPLENA_OUT_ROOT); added/outputs/to.gitignore.run_p3_scale_batch.pyand the two personal tmux babysitter scripts (wait_p3_rev_stage_b.py,watch_p3_rev_raw_build.py); removed dead code inrun_p3_rev_subset.py.qwen3_trace_replay_test.py→qwen3_trace_replay.py(it is a CLI, not a pytest module) and updated all references.rufffindings (E402, RUF005).Validation (local, in
nix develop)cargo fmt --all --check,cargo build --release,cargo test --workspace --release— all green.just test-aten-linear— PASSED, 100% match / 0.0 error.total_simulation_cycles == total_profiled_cyclesinvariant holds.ruff check .+ruff format --check .— clean.Compiler submodule stays pinned at
a765612(row-granular ABI), identical tomain.