Hugging Face· PlatformNew model

Finally, a Replacement for BERT: Introducing ModernBERT

Answer.AI and LightOn released ModernBERT, a family of state-of-the-art encoder-only models that improve upon BERT with longer context (8192 tokens), faster processing, and better downstream performance, available in base (149M params) and large (395M params) sizes.

From the source

This blog post introduces ModernBERT, a family of state-of-the-art encoder-only models representing improvements over older generation encoders across the board, with a 8192 sequence length, better downstream performance and much faster processing.

huggingface.co

Announced

19 Dec 2024 00:00 UTC

Detected

03 Aug 2026 21:12 UTC (+592d after)

Context window