From the source
Runway published research on the serving system behind Runway Characters, a roughly 57B-parameter autoregressive video diffusion transformer that must generate a four-frame 704x1088 chunk every 166.7ms at 24 fps for a single user.
- Because each session is a private one-on-one stream needing 2.24 PFLOP/s, batching across users was not an option, so the team cut costs inside the step itself.
- Cache upkeep, which had consumed 69.7ms of a 239.2ms step, fell to 6.2ms after RoPE and FP8 quantization were applied once at write time and the cache was stored as a ring.
- Fusing the tensor-parallel all-reduce into the FP8 matrix multiplication hides 54-68% of its time, taking the fused call to 153.8µs against 208.2µs.
- Together the optimizations cut the diffusion step from 240.9ms to 159.3ms, or 24.5 fps on four H100s, and to 147.4ms with the fused kernel, down from eight H100s per session at launch.
- Over seven weeks of telemetry, the worker averaged above 23.5 fps in 99.5% of sessions streaming at least a second.
