From the source
Kyutai built AURA, a family of causal, few-step speech-to-video models that generate talking-head video chunk by chunk without audio look-ahead, for streaming uses such as voice assistants and virtual avatars.
- AURA distills 50-step bidirectional teachers into 4-step causal students that stream with a rolling KV cache, so only the current chunk's audio is needed to start generating.
- AURA 1.3B streams 576p video in real time on a single GPU at 20.7 FPS for 16 Hz playback, 23× faster than its teacher, while AURA 5B streams 720p at 9.7 FPS, matching its TalkVerse teacher at 17× the speed.
- Kyutai released the paper, the code, the weights of both models and a dataset of 30K synthetic videos with their precomputed teacher trajectories, plus a live demo.
- AURA 1.3B gives a face to Unmute, Kyutai's voice AI, generating the avatar live from Unmute's speech in real time on a single H100 GPU.
- With 4 denoising steps instead of 50, AURA matches or improves on its TalkVerse teacher across most automatic metrics, and both models have the best identity preservation of all methods on both test sets.
- In an anonymous human evaluation of 400 pairwise comparisons from 27 raters, both AURA models were preferred over Hallo3 on all four axes.
