Welcome Falcon Mamba: The first strong attention-free 7B model

Falcon Mamba is a new 7B parameter model by TII, based on the Mamba architecture without attention, released under a custom license. It is open access and available on Hugging Face. It achieves competitive performance on various benchmarks and offers efficient processing of long sequences.

From the source
Falcon Mamba is a new model by Technology Innovation Institute (TII) in Abu Dhabi released under the TII Falcon Mamba 7B License 1.0.
huggingface.co

Visit source ↗huggingface.co