From the source
Cohere released a serving engine for North Mini Code built around a decode megakernel, achieving 1.25×–1.41× faster end-to-end performance than vLLM on a single H100 with BF16.
- The system supports continuous batching, paged attention, ragged sequence lengths, and an
OpenAI-compatible endpoint with tool calling.






