Skip to content

Add custom multi_tensor_apply kernels (L2norm, Adam)#585

Open
matthiasdiener wants to merge 10 commits into
devfrom
mdiener/multi_tensor_apply_kernel
Open

Add custom multi_tensor_apply kernels (L2norm, Adam)#585
matthiasdiener wants to merge 10 commits into
devfrom
mdiener/multi_tensor_apply_kernel

Conversation

@matthiasdiener

Copy link
Copy Markdown
Contributor

Description

Fixes https://github.com/ROCm/frameworks-internal/issues/16529

Type of change

  • Documentation change (change only to the documentation, either a fix or a new content)
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Infra/Build change
  • Code refactoring

Changes

Please list the changes introduced in this PR:

  • Change A
  • Change B

Checklist:

  • I have read and followed the contributing guidelines
  • The functionality is complete
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

@matthiasdiener matthiasdiener self-assigned this May 13, 2026
@matthiasdiener matthiasdiener added the ci-level 1 CI test level 1 label May 13, 2026
@matthiasdiener matthiasdiener changed the title Add a custom multi_tensor_l2norm_kernel Add a custom multi_tensor_apply kernels (L2norm, Adam) May 15, 2026
@matthiasdiener matthiasdiener changed the title Add a custom multi_tensor_apply kernels (L2norm, Adam) Add custom multi_tensor_apply kernels (L2norm, Adam) May 15, 2026
Comment thread transformer_engine/common/multi_tensor/adam.cu Outdated
Comment thread transformer_engine/common/multi_tensor/adam.cu Outdated
Comment thread transformer_engine/common/multi_tensor/adam.cu Outdated
Comment thread transformer_engine/common/multi_tensor/adam.cu Outdated
Comment thread transformer_engine/common/multi_tensor/adam.cu Outdated
Comment thread transformer_engine/pytorch/csrc/extensions/multi_tensor/adam.cpp
Comment thread transformer_engine/common/include/transformer_engine/multi_tensor.h
Comment thread transformer_engine/pytorch/csrc/extensions/multi_tensor/l2norm.cpp
Comment thread transformer_engine/pytorch/optimizers/__init__.py
Comment thread transformer_engine/pytorch/optimizers/__init__.py Outdated
matthiasdiener

This comment was marked as outdated.

@matthiasdiener matthiasdiener requested a review from alextmagro June 8, 2026 21:33
@matthiasdiener matthiasdiener marked this pull request as ready for review June 11, 2026 04:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-level 1 CI test level 1

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants