# Hugging Face — Welcome to the Falcon 3 Family of Open Models!

- Company: Hugging Face (huggingface.co)
- Announced: 2024-12-17T00:00:00+00:00
- Subject: Platform
- Source: https://huggingface.co/blog/falcon3
- Record: https://forck.live/items/1780-welcome-to-the-falcon-3-family-of-open-models

We introduce Falcon3, a family of decoder-only large language models under 10 billion parameters, developed by Technology Innovation Institute (TII) in Abu Dhabi. By pushing the boundaries of performance and training efficiency, this release reflects our ongoing commitment to advancing open and accessible large foundation models. Falcon3 represents a natural evolution from previous releases, emphasizing expanding the models' science, math, and code capabilities. This iteration includes five base models: Falcon3-1B-Base Falcon3-3B-Base Falcon3-Mamba-7B-Base Falcon3-7B-Base Falcon3-10B-Base In developing these models, we incorporated several key innovations aimed at improving the models' performances while reducing training costs: One pre-training for transformer-based models: We conducted a single large-scale pretraining run on the 7B model, using 1024 H100 GPU chips, leveraging 14 trillion tokens featuring web, code, STEM, and curated high-quality and multilingual data. …

---

Record: https://forck.live/items/1780-welcome-to-the-falcon-3-family-of-open-models
Catalogue: https://forck.live/llms.txt
Feed: https://forck.live/feed.md