From the source

Mixture of Experts (MoEs) in Transformers

Image published with the post by Hugging Face — Image from huggingface.co

Mixture of Experts (MoEs) in Transformers, explaining how sparse feed-forward layers replace dense layers to improve compute efficiency, and details the engineering work in the transformers library to support MoEs, including a weight loading refactor, expert backend, and expert parallelism.

From the source
Making MoEs first-class citizens in transformers means redesigning parts of the loading pipeline, execution model, and distributed abstractions, not just adding new model classes.
huggingface.co

Other stories

Hugging Face· 8 Oct

The model that didn't exist, so you made it yourself

A user creates six custom models via ML Intern in days

Hugging Face· 2 OctResearch paper

AutoSynthData: Generating Training Data for Enterprise Agents

Turning model failures into training data for enterprise agents

Announced: 26 Feb 2026
Detected: 03 Aug 2026 21:12 UTC (+157d after)

Visit source ↗huggingface.co

Hugging Face

· 30 Sep

Open TTS Leaderboard: Scalable Evaluation for Multilingual Text-to-Speech and Voice Cloning

Objective metrics for TTS evaluation, complementing human preference arenas.

Hugging Face· 29 SepResearch paper

Getting the Source Right, Not Just the Fact: Source-Aware Verification for MCP Agents

A new verification method catches LLM agents that attribute facts to the wrong source.

Apertus 1.5 Release Notes
Apertus 1.5 adds multimodal understanding and thinking mode
Hugging Face· 15 OctModel update
Welcome RL Environments to the hub
Hugging Face Hub adds a dedicated section for RL environments with framework tags and runnable tasks
Hugging Face· 28 SepCapability change