From the source
From the source
Cartesia presents Based, a recurrent architecture that uses sliding window attention and linear attention to outperform prior sub-quadratic models on recall-intensive tasks while achieving faster generation speeds than FlashAttention-2 and Mamba.
From the source
We’re excited to share the latest progress in this line of work. Our recent work digs deeper into the recall challenge. We begin by illustrating a fundamental tradeoff between a model’s recall abilities and its memory consumption during generation. This analysis informs the design of Based, a simple recurrent architecture that outperforms prior sub-quadratic models on real-world recall-intensive tasks (information extraction, reading comprehension) and in-context learning. At the same time, Based offers fast generation speeds: Based is 56% and 44% faster at processing prompts than FlashAttention-2 and Mamba respectively. Based achieves 24x higher text generation throughput than FlashAttention-2.
cartesia.ai
