LG AI Research· EXAONEResearch paper

[CVPR 2026] An upcycling method for creating true experts within a Mixture-of-Experts structure

LG AI Research proposes a cluster-aware upcycling method that initializes experts and routers based on activation clustering of a pre-trained dense model, aiming to improve expert specialization in Mixture-of-Experts structures.

From the source

LG AI Research analyzed how the final training results and performance can vary depending on the initialization method used when upcycling pre-trained dense models into MoE models, and proposed an initialization technique for more effective training.

lgresearch.ai

Announced

26 Jun 2026 00:00 UTC

Detected

28 Aug 2026 22:20 UTC (+63d after)