# Amazon — Deploy frontier AI models on AWS Trainium with SGLang-Neuron

- Company: Amazon (amazon.com)
- Announced: 2026-11-12T22:03:34+00:00
- Category: developer-tool-release
- Coverage: not counted
- Announcement: no
- Group: routine
- Source: https://aws.amazon.com/blogs/machine-learning/deploy-frontier-ai-models-on-aws-trainium-with-sglang-neuron/
- Record: https://forck.live/items/35588-deploy-frontier-ai-models-on-aws-trainium-with-sglang-neuron
- Subject: Bedrock / Nova
- Models affected: openai/gpt-oss-120b

Amazon says SGLang-Neuron is publicly available through an AWS Deep Learning Container, letting teams serve models and generate reinforcement learning rollouts on Trainium with the SGLang interface they already use. The container was released by RadixArk in collaboration with Annapurna Labs, and the plugin is out-of-tree, so Trainium support and SGLang core develop independently and iterate faster. Installing the plugin registers the neuron platform through SGLang's plugin entry points, and setting SGLANG_PLATFORM=neuron selects it when the server starts. SGLang core keeps the API server, request lifecycle, RadixAttention and prefix-cache bookkeeping, while the plugin implements Trainium-specific models, attention, mixture of experts and the graph runner. TorchNeuron registers Trainium as a native PyTorch device, and the plugin compiles standard PyTorch operations and NKI kernels with torch.compile for the graph runner to execute. It also supports PD disaggregation, where a prefill server computes the prompt KV cache and transfers it to a decode server over Elastic Fabric Adapter using NIXL. The walkthrough uses a Trainium3 EC2 instance with a pytorch-inference-sglang-neuronx container to serve openai/gpt-oss-120b.

## Evidence

Verbatim from https://aws.amazon.com/blogs/machine-learning/deploy-frontier-ai-models-on-aws-trainium-with-sglang-neuron/:

> Today, Amazon announced that SGLang-Neuron is publicly available and can be accessed through an AWS Deep Learning Container, released by RadixArk in collaboration with Annapurna Labs. With SGLang-Neuron, you can serve models and generate reinforcement learning (RL) rollouts with SGLang on AWS Trainium using the same SGLang interface you use today. Trainium is a portfolio of AWS AI chips that delivers strong economics for AI training and inference at scale. Now, running SGLang on Trainium follows the SGLang workflow you already know: you launch models through SGLang’s server interface and invoke the endpoint using the same generation APIs. Underneath that interface, SGLang-Neuron runs on native PyTorch. With its out-of-tree plugin design, Trainium support and SGLang core develop independently and iterate faster.
> 
> How SGLang runs on Trainium
> 
> SGLang-Neuron is an out-of-tree plugin that connects SGLang with AWS Trainium. Installing the plugin registers the neuron platform through SGLang’s plugin entry points, and setting SGLANG_PLATFORM=neuron selects it when the server starts.

---

Record: https://forck.live/items/35588-deploy-frontier-ai-models-on-aws-trainium-with-sglang-neuron
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md
