From the source
Amazon says SGLang-Neuron is publicly available through an AWS Deep Learning Container, letting teams serve models and generate reinforcement learning rollouts on Trainium with the SGLang interface they already use.
- The container was released by RadixArk in collaboration with Annapurna Labs, and the plugin is out-of-tree, so Trainium support and SGLang core develop independently and iterate faster.
- Installing the plugin registers the neuron platform through SGLang's plugin entry points, and setting SGLANG_PLATFORM=neuron selects it when the server starts.
- SGLang core keeps the API server, request lifecycle, RadixAttention and prefix-cache bookkeeping, while the plugin implements Trainium-specific models, attention, mixture of experts and the graph runner.
- TorchNeuron registers Trainium as a native PyTorch device, and the plugin compiles standard PyTorch operations and NKI kernels with torch.compile for the graph runner to execute.
- It also supports PD disaggregation, where a prefill server computes the prompt KV cache and transfers it to a decode server over Elastic Fabric Adapter using NIXL.
- The walkthrough uses a Trainium3 EC2 instance with a pytorch-inference-sglang-neuronx container to serve openai/gpt-oss-120b.






