Fine-tuning GPT-OSS-20B for multilingual reasoning on Amazon SageMaker HyperPod

Amazon's guide walks through fine-tuning the open-weight GPT-OSS-20B model for multilingual reasoning on Amazon SageMaker HyperPod, using supervised fine-tuning and Group Relative Policy Optimization.

• The base model thinks in English: asked to reason in French, German, Spanish or Italian, it acknowledges the instruction and then produces its chain-of-thought in English anyway.
• On a 50-example multilingual reasoning test, the base model reasoned in the correct target language only 22 percent of the time, almost entirely the English examples, with French, Spanish and Italian at 0 percent and German at 10 percent.
• The guide frames the fix as a choice between SFT, GRPO or both, and says SFT teaches a behavior the model does not have while GRPO optimizes one it can already sometimes produce.
• Training used the HuggingFaceH4/Multilingual-Thinking dataset, roughly 949 training and 50 validation examples, with evaluation on 10 math and logic problems across five languages.
• Runs used LoRA on SageMaker HyperPod with Amazon EKS, PyTorch TRL for GPT-OSS-20B and veRL with Ray, KubeRay and vLLM for a Qwen2.5-7B control experiment, with checkpoints on Amazon FSx for Lustre and EFA for inter-node traffic.