[python][torch] Support distributed iterable dataset sharding - #9429

Merged

JingsongLi merged 12 commits into

apache:masterfrom

XiaoHongbo-Hope:codex/torch-ddp-sharding

Aug 30, 2026

Commits on Aug 27, 2026

[python][torch] Support distributed iterable dataset sharding
XiaoHongbo-Hope
committed
7dcf4cf
[python] Clarify upstream rank sharding guidance
XiaoHongbo-Hope
committed
a19a060
[python][torch] Simplify distributed sharding
XiaoHongbo-Hope
committed
3966610
[python][torch] Keep rank detection internal
XiaoHongbo-Hope
committed
2b89305
[python][torch] Enable rank detection by default
XiaoHongbo-Hope
committed
1415e0b
[python][torch] Resolve rank context when iterating
XiaoHongbo-Hope
committed
33232c3
[python][torch] Preserve rank context for spawned workers
XiaoHongbo-Hope
committed
0092956

Commits on Aug 29, 2026

Commits on Aug 30, 2026

[python][torch] Support explicit DDP sharding context
XiaoHongbo-Hope
committed
05cbebc