Distributed training profiler for analyzing compute, communication, memory, and scaling bottlenecks in ML training systems.
Distributed data-parallel training from raw TCP sockets: bandwidth-optimal ring all-reduce, a provably-identical-to-single-process trainer, the measured economics of when it pays, and the three bugs that taught the course
Qualified FlashInfer PCIe IPC all-reduce commits, artifacts, and Kimi-K3 TP8 evidence
Deterministic Go simulator for congestion-aware routing in GPU training fabrics
A distributed training simulator for data-parallel learning, all-reduce, and communication vs computation trade-offs.
Datacenter Computing UPB 2024-2025 - RDMA Client-Server Library
A 6-week masterclass in Distributed Deep Learning Infrastructure from scratch. Covers the $\alpha-\beta$ communication model, custom Ring All-Reduce primitives, Parameter Servers (Hogwild!), 1F1B Pipeline Parallelism, ZeRO/FSDP memory sharding, gradient compression with error feedback, and fault-tolerant elastic training simulators.
To associate your repository with the all-reduce topic, visit your repo's landing page and select "manage topics."