From the source
Runway published a technical post explaining the challenges of distributed training with DTensor, including gradient correctness issues that arise when sharding tensors across process groups.
Why Distributed Training Is Hard: DTensor, Correctness and the Costs of Abstraction
From the source
DTensor makes distributed training correct by attaching placement metadata to every tensor. At scale it can also introduce costs that quietly erode throughput unless you design around them.
runway.com
- Announced
- 18 May 2026
- First seen by forck
