Skip to content

Arm/Cortex-M backend: Share deterministic test seed handling#20279

Open
zingo wants to merge 1 commit into
pytorch:mainfrom
zingo:Arm/Cortex-M-backend-Share-deterministic-test-seed-handling
Open

Arm/Cortex-M backend: Share deterministic test seed handling#20279
zingo wants to merge 1 commit into
pytorch:mainfrom
zingo:Arm/Cortex-M-backend-Share-deterministic-test-seed-handling

Conversation

@zingo

@zingo zingo commented Jun 15, 2026

Copy link
Copy Markdown
Collaborator

Use TEST_SEED for both Arm and Cortex-M tests. Choose a single session seed before collection so module-level random tensors are reproducible, report that seed once in the pytest header, and reuse it in the auto use seed fixture before each test.

cc @digantdesai @freddan80 @per @oscarandersson8218 @mansnils @Sebastian-Larsson @robell @rascani @psiddh @AdrianLundell

Use TEST_SEED for both Arm and Cortex-M tests. Choose a single session
seed before collection so module-level random tensors are reproducible,
report that seed once in the pytest header, and reuse it in the
autouse seed fixture before each test.

Signed-off-by: Zingo Andersen <Zingo.Andersen@arm.com>
Change-Id: I645a37e560ed76526db12b6261a550e9cd6d8cc8
@zingo zingo requested a review from rascani as a code owner June 15, 2026 12:33
Copilot AI review requested due to automatic review settings June 15, 2026 12:33
@zingo zingo requested a review from digantdesai as a code owner June 15, 2026 12:33
@zingo zingo added partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm release notes: none Do not include this in the release notes labels Jun 15, 2026
@pytorch-bot

pytorch-bot Bot commented Jun 15, 2026

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20279

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 2 Unclassified Failures

As of commit 6ae3d12 with merge base e88fd04 (image):

NEW FAILURES - The following jobs have failed:

UNCLASSIFIED FAILURES - DrCI could not classify the following jobs because the workflow did not run on the merge base. The failures may be pre-existing on trunk or introduced by this PR:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@github-actions github-actions Bot added ciflow/trunk module: arm Issues related to arm backend labels Jun 15, 2026
@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jun 15, 2026
@zingo zingo added the module: microcontrollers For embedded MCUs like Cortex-M, or RTOS like Zephyr, does not track NPU backend like Arm Ethos. label Jun 15, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR standardizes deterministic random seed handling across the Arm and Cortex-M pytest suites by using a shared TEST_SEED environment variable and selecting a single session seed early (before test collection) so module-level random tensors are reproducible, while re-seeding consistently before each test.

Changes:

  • Added session-level seed selection via TEST_SEED (including RANDOM) and stored the chosen seed/label on the pytest config object.
  • Updated both suites to reseed per-test using the session seed for repeatable behavior.
  • Report the chosen seed once in pytest’s header output (Arm: seed label; Cortex-M: targets + seed label).

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
backends/cortex_m/test/conftest.py Adds session seed setup + autouse reseeding, and includes seed info in the header alongside selected targets.
backends/arm/test/conftest.py Switches from ARM_TEST_SEED to TEST_SEED, chooses one session seed, reseeds per test, and reports the seed in the pytest header.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines 74 to 77
"""Control random numbers in Arm test suite. Default behavior is to use a
fixed seed (0), which ensures reproducible tests. Use the env variable
ARM_TEST_SEED to set a custom seed, or set it to RANDOM for random seed
TEST_SEED to set a custom seed, or set it to RANDOM for random seed
behavior.
Examples:
As default use fixed seed (0) for reproducible tests
pytest --config-file=/dev/null --verbose -s --color=yes backends/arm/test/ops/test_avg_pool.py -k <TESTCASE>
Use a random seed for each test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ciflow/trunk CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. module: arm Issues related to arm backend module: microcontrollers For embedded MCUs like Cortex-M, or RTOS like Zephyr, does not track NPU backend like Arm Ethos. partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm release notes: none Do not include this in the release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants