# Unite.AI — Together AI Brings Continuous Compilation to Production Inference

- Company: Unite.AI (unite.ai)
- Announced: 2026-11-04T14:20:02+00:00
- Category: infrastructure-release
- Coverage: not counted
- Announcement: yes
- Group: announcements
- Source: https://www.unite.ai/together-ai-brings-continuous-compilation-to-production-inference/
- Record: https://forck.live/items/31581-together-ai-brings-continuous-compilation-to-production-inference
- Subject: Reporting
- Models affected: Kimi K3, GLM 5.3, MiniMax M3.1-Flash, DeepSeek v4.1-Flash

Together AI introduced Polymorphic Inference, with Together Compiler as its first implementation, an agent-driven system that continuously retunes inference as models, traffic and hardware change. The company says the compiler reached 1.3x to 3x the tokens per second of highly optimized open-source baselines within under 24 hours, and that it already runs across many top production inference workloads. Together AI argues fixed engine configurations go stale: prompts during one country's working hours reached about 97,000 tokens against 70,000 elsewhere, and prompt lengths rose 80% in two weeks after one model release. Agents generate and measure candidate optimizations while a persistent engine hot-reloads forward passes, and a startup comparison puts serving time at 9.0 seconds versus 467 seconds for the status-quo sequence, a figure covering engine startup rather than request latency. The paper reports 1.5x to 6.4x speedups on tested weather, biology, time-series and partial-differential-equation models with bit-exact outputs on the stated evaluations. Together AI also revised its earlier emphasis on megakernels, concluding fusion is not a universal goal for mixture-of-experts architectures and dynamic workloads. Customers can request early access through a limited beta.

## Evidence

Verbatim from https://www.unite.ai/together-ai-brings-continuous-compilation-to-production-inference/:

> Together AI announced Polymorphic Inference on November 4, 2026, introducing Together Compiler as its first implementation. The agent-driven system continuously retunes inference for changing models, traffic and hardware, and already runs across many of the company’s top production inference workloads and some popular serverless endpoints.
> 
> Together AI reports that the compiler delivered 1.3x to 3x the tokens per second of highly optimized open-source baselines in under 24 hours across tests involving Kimi K3, GLM 5.3, MiniMax M3.1-Flash and DeepSeek v4.1-Flash. Those results frame the announcement’s central proposition: optimization can be an ongoing search process tied to a deployment, rather than a round of manual tuning frozen at launch.
> 
> Changing Models, Traffic and Hardware
> 
> Polymorphic Inference is Together AI’s term for serving infrastructure that adapts to faster model releases, diverse architectures, evolving traffic shapes and service-level requirements, and more frequent hardware changes. The contrasting static approach starts from a fixed engine configuration.

---

Record: https://forck.live/items/31581-together-ai-brings-continuous-compilation-to-production-inference
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md