From the source
TII released Falcon-H1-Arabic, a family of Arabic language models (3B, 7B, 34B parameters) built on a hybrid Mamba-Transformer architecture.
The models support context windows of up to 256K tokens and were trained on a mix of Arabic, English, and multilingual data totaling around 300 billion tokens.
Post-training included supervised fine-tuning and direct preference optimization to improve long-context reasoning and alignment.
