Skip to content

[PyT] Fix PyTorch BF16 reduction numerics tests - #3461

Open
jberchtold-nvidia wants to merge 3 commits into
NVIDIA:mainfrom
jberchtold-nvidia:jberchtold/fix-pytorch-bf16-reduction-tests
Open

[PyT] Fix PyTorch BF16 reduction numerics tests#3461
jberchtold-nvidia wants to merge 3 commits into
NVIDIA:mainfrom
jberchtold-nvidia:jberchtold/fix-pytorch-bf16-reduction-tests

Conversation

@jberchtold-nvidia

Copy link
Copy Markdown
Collaborator

Description

Fixes PyTorch tests to match tolerances on L40 that were changed in this PR #3440 . Now BF16 partial results reductions are disabled

Type of change

  • Documentation change (change only to the documentation, either a fix or a new content)
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Infra/Build change
  • Code refactoring

Changes

  • Guard tests with PyTorch flag to disable bf16 partial result reductions in reference

Checklist:

  • I have read and followed the contributing guidelines
  • The functionality is complete
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

Signed-off-by: Jeremy Berchtold <jberchtold@nvidia.com>
@greptile-apps

greptile-apps Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR aligns plain-PyTorch numerical references with Transformer Engine’s higher-precision BF16 reduction behavior.

  • Adds a context manager that temporarily disables reduced-precision BF16 reductions and reliably restores the prior setting.
  • Applies the precision setting automatically to every test in the PyTorch numerics module.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
tests/pytorch/test_numerics.py Adds a scoped autouse fixture that disables PyTorch BF16 partial-result reductions during each numerical test and restores the original backend setting afterward.

Reviews (3): Last reviewed commit: "Apply BF16 precision fixture to all nume..." | Re-trigger Greptile

@jberchtold-nvidia

Copy link
Copy Markdown
Collaborator Author

/te-ci L0 pytorch

@jberchtold-nvidia
jberchtold-nvidia force-pushed the jberchtold/fix-pytorch-bf16-reduction-tests branch from cd44c56 to 2b9c21b Compare September 2, 2026 20:27
Signed-off-by: Jeremy Berchtold <jberchtold@nvidia.com>
@jberchtold-nvidia
jberchtold-nvidia force-pushed the jberchtold/fix-pytorch-bf16-reduction-tests branch from 2b9c21b to d62e9da Compare September 2, 2026 20:27
@jberchtold-nvidia

Copy link
Copy Markdown
Collaborator Author

/te-ci L0 pytorch

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant