# TII — Falcon 2: An 11B parameter pretrained language model and VLM, trained on over 5000B tokens and 11 languages

- Company: TII (tii.ae)
- Announced: 2024-05-24T12:00:00+00:00
- Category: new-model
- Coverage: not counted
- Announcement: yes
- Group: models
- Source: https://falcon-lm.github.io/blog/falcon2-11b/
- Record: https://forck.live/items/18419-falcon-2-an-11b-parameter-pretrained-language-model-and-vlm-trained-on-over
- Subject: Falcon LLM
- Open weights: yes
- Models affected: Falcon2-11B, Falcon2-11B VLM
- Context window: 8192 tokens

TII released Falcon 2, a new generation of open-source models, starting with an 11B parameter language model (LLM) and an 11B vision-language model (VLM) that can answer queries about images. The LLM was trained on over 5,000 billion tokens of RefinedWeb and curated data, supports 11 languages, and achieves performance comparable to Falcon-40B at a quarter of the size. The VLM integrates a CLIP ViT-L/14 vision encoder with the chat-finetuned LLM.

## Evidence

Verbatim from https://falcon-lm.github.io/blog/falcon2-11b/:

> The second generation of models is focused on increased usability and integrability, building a multi-modal ecosystem. We start this journey by releasing not only the base 11B LLM , but also the 11B VLM model that incorporates image understanding capabilities.

---

Record: https://forck.live/items/18419-falcon-2-an-11b-parameter-pretrained-language-model-and-vlm-trained-on-over
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md