From the source
ElevenLabs is introducing Cue, a forced alignment model that places every character in an audio file on a precise timeline and powers the Scribe model family.
- The company says Cue is more accurate than every aligner it tested on real-world recordings, across all 29 languages of the benchmark.
- Cue is the last step in the speech-to-text pipeline: Scribe transcribes the audio and the aligner sets the timing, and every Scribe request passes through it.
- The same model also powers subtitles, dubbing and lyric timing for music, and is available on its own through the Forced Alignment API.
- On full recordings, Cue's average timing error is 179 milliseconds, while the open-source tools tested ranged from 283 ms to more than 6,000 ms.
- On English recordings with background noise, overlapping speech or ambient music, Cue's average error is 57 ms, against 65 ms to 920 ms for the others.
- Cue handles recordings of any length in a single pass, covers more than 90 benchmark languages, and aligns an hour of audio in about half a second on ElevenLabs' production GPU.
