From the source
Alibaba introduces Qwen1.5-MoE-A2.7B, a mixture-of-experts model with 2.7 billion activated parameters that matches the performance of 7B models like Mistral 7B and Qwen1.
From the source
Today, we introduce Qwen1.5-MoE-A2.7B, a small MoE model with only 2.7 billion activated parameters yet matching the performance of state-of-the-art 7B models like Mistral 7B and Qwen1.
qwenlm.github.io
