From the source
Baseten has rebuilt its delta weight sync path so that GPU memory holds the only copy of a served policy, cutting a ~770 GB GLM-5.3 sync from 36 seconds to 15 and the request pause from 6 seconds to 0.8.
- The previous design kept a full parent checkpoint in host RAM, which on an 8×B300 node forced the host KV cache down to 275 GB and added a 45-second CPU promotion per update.
- The new decoder walks zstd's commands directly, keeps only the nonzero XOR bytes, gathers old values at changed positions, applies the XOR mask and verifies the result against the trainer's checksum before any GPU write.
- Baseten says the checksum work made the whole update up to 2.3× faster, with Kimi K3 on a 30 GB delta dropping from ~230 to ~100 seconds.
- Update planning runs in a separate CUDA-free helper process to avoid contending for SGLang's Python event loop, and decoding is pipelined with the download on low-priority native threads sized to leave 16 cores for serving.
- The change also unblocks Kimi K3 RL rollout on 8:GB300, which Baseten says was previously blocked by CPU memory.
