From the source
Mistral AI released Mixtral 8x22B, an open sparse mixture-of-experts model with 39B active parameters out of 141B, under the Apache 2.0 license.
- It features a 64K token context window, multilingual support (English, French, Italian, German, Spanish), and strong mathematics and coding capabilities.
- The instructed version achieves 90.8% on GSM8K maj@8 and 44.6% on Math maj@4.
