From the source
Together AI says many of the models served through its endpoints are now optimized by AI agents, in what it calls the first production-scale recursive self-improvement inference system.
- The company frames the shift as a move from Static Inference, which it defines by closed models, traditional Transformer architectures and single-hardware-platform serving, to Polymorphic Inference, inference that adapts to rapidly changing environments.
- Together attributes the need for the change to four trends: faster and more diverse open-source model releases, differing architectures among the top-ten most popular open models, traffic shapes that keep evolving, and hardware moving from B200 to GB300 with Vera Rubin deployments expected.
- The company says 80% of traffic on OpenRouter goes to models released in the last three months.
- Together argues that static hyperparameter-tuned engine configurations, such as typical vLLM, SGLang or InferenceX recipes, do not transfer between settings, and that open-source kernels like MoE kernels tuned on random tokens can be miscalibrated against real production traffic.
- It also reports a lesson from putting early megakernels into production: megakernel fusion is not the right design goal for Polymorphic Inference, because each variation in architecture or workload needs a different approach.
