Welcome Falcon Mamba: The first strong attention-free 7B model
Falcon Mamba is a new 7B parameter model by TII, based on the Mamba architecture without attention, released under a custom license. It is open access and available on Hugging Face. It achieves competitive performance on various benchmarks and offers efficient processing of long sequences.
