From the source
Meta released the weights for Muse Spark Open on October 15, 2026, a natively multimodal open-weight model with 816B total parameters, 42B active parameters, and a 1M-token context window.
Meta AI Research describes it as the most capable US open-weight model to date, built for coding and agentic work such as long-horizon tasks, complex software engineering, and multi-step reasoning.
Architecture and Training Data Muse Spark Open is a Mixture-of-Experts Transformer paired with a Perception Encoder, according to the model card published by Meta Superintelligence Labs.
The language model has 62 layers, a hidden dimension of 8192, and 256 experts with eight activated per token through top-k routing.
Its attention block uses gated attention with QK-norm, 128 query heads and 16 key-value heads in a grouped-query configuration, and a head dimension of 64.
The vocabulary size is 202,048 with untied input and output embeddings.
…