# Together AI — Polymorphic Inference: self-tuning inference for open models

- Company: Together AI (together.ai)
- Announced: 2026-11-04
- Category: infrastructure-release
- Coverage: not counted
- Announcement: yes
- Group: announcements
- Source: https://www.together.ai/blog/polymorphic-inference
- Record: https://forck.live/items/31557-polymorphic-inference-self-tuning-inference-for-open-models
- Subject: Inference platform

Together AI says many of the models served through its endpoints are now optimized by AI agents, in what it calls the first production-scale recursive self-improvement inference system. The company frames the shift as a move from Static Inference, which it defines by closed models, traditional Transformer architectures and single-hardware-platform serving, to Polymorphic Inference, inference that adapts to rapidly changing environments. Together attributes the need for the change to four trends: faster and more diverse open-source model releases, differing architectures among the top-ten most popular open models, traffic shapes that keep evolving, and hardware moving from B200 to GB300 with Vera Rubin deployments expected. The company says 80% of traffic on OpenRouter goes to models released in the last three months. Together argues that static hyperparameter-tuned engine configurations, such as typical vLLM, SGLang or InferenceX recipes, do not transfer between settings, and that open-source kernels like MoE kernels tuned on random tokens can be miscalibrated against real production traffic. It also reports a lesson from putting early megakernels into production: megakernel fusion is not the right design goal for Polymorphic Inference, because each variation in architecture or workload needs a different approach.

## Evidence

Verbatim from https://www.together.ai/blog/polymorphic-inference:

> From Static to Polymorphic Inference: A new era for production open-source inference systems
> 
> As of today, many of the models you run through Together AI endpoints are fully optimized by AI agents: the world’s first production-scale recursive self-improvement (RSI) inference system.
> 
> Authors
> 
> By Simran Arora (SVP of Research, AI Systems) and Dan Fu (Founding SVP of Research)
> 
> Table of contents
> 
> 40+ Models Chosen for Production...40+ Models Chosen for Production...40+ Models Chosen for Production...
> 
> Links in this article
> 
> Summary
> 
> At Together AI, we focus on making AI abundantly available to every individual user and organization. Our research team – the researchers behind FlashAttention, ThunderKittens, ThunderAgent, and more – has been a key driver behind our inference excellence. As the inference market grows exponentially, we’ve been focusing on how to scale our research capacity to enable abundant intelligence for all.
> 
> This has required a shift in our thinking from Static Inference to Polymorphic Inference – which is our response to the velocity of new model releases, diversity of model architectures, fast-changing workloads, and fast-changing hardware platforms.

---

Record: https://forck.live/items/31557-polymorphic-inference-self-tuning-inference-for-open-models
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md