Multi-Turn Reinforcement Learning on Amazon SageMaker AI: A hands-on guide

Amazon's hands-on guide walks through training a model with multi-turn reinforcement learning on SageMaker AI, where an agent acts over many turns and reports a single reward when the episode ends.

• The approach targets agentic tasks such as filing an expense report or triaging a support ticket, where supervised fine-tuning has no labels for the right sequence and single-turn RL cannot credit a decision made three steps before the outcome.
• In the walkthrough, SageMaker AI sends prompts to an agent hosted on Bedrock AgentCore, which calls the policy model and plays Wordle, and Group Relative Policy Optimization updates a LoRA adapter from the spread of rewards across rollouts of the same word.
• Amazon reports that the run took gpt-oss-20b from a 61 to an 84 percent solve rate on held-out words at default reasoning effort, and from 15 to 69 percent at the low effort it trained at, for about $339 in training tokens.
• The 100-step run consumed 491.6 million prefill, 159.7 million sample and 643.3 million train tokens across 105,486 rollouts in 12 hours 40 minutes, with the AgentCore runtime billing separately at about $79.
• Amazon says MTRL is serverless, that customers pick a base model and the service provisions the training and sampling infrastructure, and that datasets, agents and trained models stay in the customer's AWS account and are not used to train the base models.
• The guide also covers reward-design mistakes and training failures the authors hit, and the code is in an aws-samples repository.