Together AI· Inference platformInfrastructure

ThunderAgent: 2x Faster Agentic Inference for Synthetic Data Generation at Scale

Together AI introduces ThunderAgent, a system for high throughput agentic inference that achieves up to 2.5x higher single-node throughput and 2.4x speedup on 8 nodes by introducing a novel program abstraction for scheduling agentic workflows to mitigate KV cache thrashing.

From the source

By introducing a novel program abstraction for agentic LLM request scheduling, ThunderAgent achieves up to 2.5× higher single-node throughput in our synthetic data generation pipeline, and delivers 2.4× speedup on an 8-node cluster with near-linear throughput scaling with respect to GPU nodes.

together.ai

Announced

29 Jul 2026 00:00 UTC

Detected

03 Aug 2026 21:12 UTC (+5d after)