Arm/Cortex-M backend: Share deterministic test seed handling#20279
Arm/Cortex-M backend: Share deterministic test seed handling#20279zingo wants to merge 1 commit into
Conversation
Use TEST_SEED for both Arm and Cortex-M tests. Choose a single session seed before collection so module-level random tensors are reproducible, report that seed once in the pytest header, and reuse it in the autouse seed fixture before each test. Signed-off-by: Zingo Andersen <Zingo.Andersen@arm.com> Change-Id: I645a37e560ed76526db12b6261a550e9cd6d8cc8
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20279
Note: Links to docs will display an error until the docs builds have been completed. ❌ 3 New Failures, 2 Unclassified FailuresAs of commit 6ae3d12 with merge base e88fd04 ( NEW FAILURES - The following jobs have failed:
UNCLASSIFIED FAILURES - DrCI could not classify the following jobs because the workflow did not run on the merge base. The failures may be pre-existing on trunk or introduced by this PR:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
There was a problem hiding this comment.
Pull request overview
This PR standardizes deterministic random seed handling across the Arm and Cortex-M pytest suites by using a shared TEST_SEED environment variable and selecting a single session seed early (before test collection) so module-level random tensors are reproducible, while re-seeding consistently before each test.
Changes:
- Added session-level seed selection via
TEST_SEED(includingRANDOM) and stored the chosen seed/label on the pytest config object. - Updated both suites to reseed per-test using the session seed for repeatable behavior.
- Report the chosen seed once in pytest’s header output (Arm: seed label; Cortex-M: targets + seed label).
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| backends/cortex_m/test/conftest.py | Adds session seed setup + autouse reseeding, and includes seed info in the header alongside selected targets. |
| backends/arm/test/conftest.py | Switches from ARM_TEST_SEED to TEST_SEED, chooses one session seed, reseeds per test, and reports the seed in the pytest header. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| """Control random numbers in Arm test suite. Default behavior is to use a | ||
| fixed seed (0), which ensures reproducible tests. Use the env variable | ||
| ARM_TEST_SEED to set a custom seed, or set it to RANDOM for random seed | ||
| TEST_SEED to set a custom seed, or set it to RANDOM for random seed | ||
| behavior. |
| Examples: | ||
| As default use fixed seed (0) for reproducible tests | ||
| pytest --config-file=/dev/null --verbose -s --color=yes backends/arm/test/ops/test_avg_pool.py -k <TESTCASE> | ||
| Use a random seed for each test |
Use TEST_SEED for both Arm and Cortex-M tests. Choose a single session seed before collection so module-level random tensors are reproducible, report that seed once in the pytest header, and reuse it in the auto use seed fixture before each test.
cc @digantdesai @freddan80 @per @oscarandersson8218 @mansnils @Sebastian-Larsson @robell @rascani @psiddh @AdrianLundell