# Hugging Face — Finally, a Replacement for BERT: Introducing ModernBERT

- Company: Hugging Face (huggingface.co)
- Announced: 2024-12-19T00:00:00+00:00
- Category: new-model
- Subject: Platform
- Models affected: ModernBERT, ModernBERT-base, ModernBERT-large
- Context window: 8192 sequence length
- Source: https://huggingface.co/blog/modernbert
- Record: https://forck.live/items/1778-finally-a-replacement-for-bert-introducing-modernbert

Answer.AI and LightOn released ModernBERT, a family of state-of-the-art encoder-only models that improve upon BERT with longer context (8192 tokens), faster processing, and better downstream performance, available in base (149M params) and large (395M params) sizes.

## Evidence

Verbatim from https://huggingface.co/blog/modernbert:

> This blog post introduces ModernBERT, a family of state-of-the-art encoder-only models representing improvements over older generation encoders across the board, with a 8192 sequence length, better downstream performance and much faster processing.

---

Record: https://forck.live/items/1778-finally-a-replacement-for-bert-introducing-modernbert
Catalogue: https://forck.live/llms.txt
Feed: https://forck.live/feed.md