Together AI· Inference platformNew model

Mamba-3

is a new state space model designed with inference efficiency as the primary goal, featuring a more expressive recurrence formula, complex-valued state tracking, and a MIMO variant that boosts accuracy without slowing down decoding. The model outperforms Mamba-2, Gated DeltaNet, and Llama-3.2-1B on prefill+decode latency at the 1.5B scale. The team open-sourced the kernels built using Triton, TileLang, and CuTe DSL.

From the source

Mamba-3 is a new state space model (SSM) designed with inference efficiency as the primary goal — a departure from Mamba-2, which optimized for training speed. The key upgrades are a more expressive recurrence formula, complex-valued state tracking, and a MIMO (multi-input, multi-output) variant that boosts accuracy without slowing down decoding. The result: Mamba-3 SISO beats Mamba-2, Gated DeltaNet, and even Llama-3.2-1B (Transformer) on prefill+decode latency across all sequence lengths at the 1.5B scale. The team also open-sourced the kernels, built using a mix of Triton, TileLang, and CuTe DSL for maximum hardware performance.

together.ai

Announced