# Together AI — ParallelKernelBench: Frontier LLMs can't write fast multi-GPU kernels (yet)

- Company: Together AI (together.ai)
- Announced: 2026-06-23T00:00:00+00:00
- Category: research-paper
- Subject: Inference platform
- Models affected: GPT-5.5, Gemini 3 Pro, Opus 4.7, GLM-5.1, GLM-5.2, DeepSeek V4 Pro
- Source: https://www.together.ai/blog/parallelkernelbench
- Record: https://forck.live/items/2340-parallelkernelbench-frontier-llms-can-t-write-fast-multi-gpu-kernels-yet

Together AI introduces ParallelKernelBench (PKB), a benchmark and evaluation framework for multi-GPU kernel generation, containing 87 problems from real codebases. The benchmark tests frontier coding models on replacing PyTorch + NCCL with CUDA kernels that communicate directly over NVLink. Evaluation shows that under a third of problems are solved correctly, and fewer than a quarter of those beat the naive baseline.

## Evidence

Verbatim from https://www.together.ai/blog/parallelkernelbench:

> ParallelKernelBench (PKB) offers a benchmark and evaluation framework for multi-GPU kernel generation and includes 87 problems from real codebases where the task is replacing PyTorch + NCCL with a CUDA kernel that moves data directly over NVLink. We tested frontier coding models such as GPT-5.5, Gemini 3 Pro, Opus 4.7, and others. The evaluation revealed significant performance gaps across the board: under a third of problems were solved correctly, and fewer than a quarter of those beat the naive baseline.

---

Record: https://forck.live/items/2340-parallelkernelbench-frontier-llms-can-t-write-fast-multi-gpu-kernels-yet
Catalogue: https://forck.live/llms.txt
Feed: https://forck.live/feed.md