Welcome to the Falcon 3 Family of Open Models!

We introduce Falcon3, a family of decoder-only large language models under 10 billion parameters, developed by Technology Innovation Institute (TII) in Abu Dhabi. By pushing the boundaries of performance and training efficiency, this release reflects our ongoing commitment to advancing open and accessible large foundation models. Falcon3 represents a natural evolution from previous releases, emphasizing expanding the models' science, math, and code capabilities. This iteration includes five base models: Falcon3-1B-Base Falcon3-3B-Base Falcon3-Mamba-7B-Base Falcon3-7B-Base Falcon3-10B-Base In developing these models, we incorporated several key innovations aimed at improving the models' performances while reducing training costs: One pre-training for transformer-based models: We conducted a single large-scale pretraining run on the 7B model, using 1024 H100 GPU chips, leveraging 14 trillion tokens featuring web, code, STEM, and curated high-quality and multilingual data. …

Visit source ↗huggingface.co

Announced: 17 Dec 2024 00:00 UTC
Detected: 03 Aug 2026 21:12 UTC (+594d after)

Other stories

Hugging Face · 28 AugThe Open ASR Leaderboard Adds Its First Global South LanguageCapability change

Amazon · 31 AugConnect an AgentCore Runtime hosted MCP server to Amazon QuickCapability change

Amazon · 31 AugAWS recognized as a Leader in The Forrester Wave: AI Infrastructure Solutions, Q4 2025Company news

Amazon · 31 Aug

Manage agents, tools and skills at scale with AWS Agent Registry

Amazon · 31 AugBuild observable enterprise agentic retrieval using Managed Amazon Bedrock Knowledge Base with AWS CloudFormationInfrastructure

Amazon · 31 AugBuild multi-tenant agentic chat applications on enterprise data with Amazon Bedrock Managed Knowledge Base