# Alibaba — Qwen1.5-MoE: Matching 7B Model Performance with 1/3 Activated Parameters

- Company: Alibaba (alibaba.com)
- Announced: 2024-03-28T03:31:44+00:00
- Category: new-model
- Subject: Qwen
- Models affected: Qwen1.5-MoE-A2.7B
- Source: https://qwenlm.github.io/blog/qwen-moe/
- Record: https://forck.live/items/2318-qwen1-5-moe-matching-7b-model-performance-with-1-3-activated-parameters

Alibaba introduces Qwen1.5-MoE-A2.7B, a mixture-of-experts model with 2.7 billion activated parameters that matches the performance of 7B models like Mistral 7B and Qwen1.

## Evidence

Verbatim from https://qwenlm.github.io/blog/qwen-moe/:

> Today, we introduce Qwen1.5-MoE-A2.7B, a small MoE model with only 2.7 billion activated parameters yet matching the performance of state-of-the-art 7B models like Mistral 7B and Qwen1.

---

Record: https://forck.live/items/2318-qwen1-5-moe-matching-7b-model-performance-with-1-3-activated-parameters
Catalogue: https://forck.live/llms.txt
Feed: https://forck.live/feed.md