Together AI· Inference platformResearch paper

ParallelKernelBench: Frontier LLMs can't write fast multi-GPU kernels (yet)

Together AI introduces ParallelKernelBench (PKB), a benchmark and evaluation framework for multi-GPU kernel generation, containing 87 problems from real codebases. The benchmark tests frontier coding models on replacing PyTorch + NCCL with CUDA kernels that communicate directly over NVLink. Evaluation shows that under a third of problems are solved correctly, and fewer than a quarter of those beat the naive baseline.

From the source

ParallelKernelBench (PKB) offers a benchmark and evaluation framework for multi-GPU kernel generation and includes 87 problems from real codebases where the task is replacing PyTorch + NCCL with a CUDA kernel that moves data directly over NVLink. We tested frontier coding models such as GPT-5.5, Gemini 3 Pro, Opus 4.7, and others. The evaluation revealed significant performance gaps across the board: under a third of problems were solved correctly, and fewer than a quarter of those beat the naive baseline.

together.ai

Announced