Alibaba· QwenNew model

Qwen1.5-MoE: Matching 7B Model Performance with 1/3 Activated Parameters

Alibaba introduces Qwen1.5-MoE-A2.7B, a mixture-of-experts model with 2.7 billion activated parameters that matches the performance of 7B models like Mistral 7B and Qwen1.

From the source

Today, we introduce Qwen1.5-MoE-A2.7B, a small MoE model with only 2.7 billion activated parameters yet matching the performance of state-of-the-art 7B models like Mistral 7B and Qwen1.

qwenlm.github.io

Announced

28 Mar 2024 03:31 UTC

Detected

03 Aug 2026 21:12 UTC (+858d after)