# Amazon — Fine-tuning GPT-OSS-20B for multilingual reasoning on Amazon SageMaker HyperPod

- Company: Amazon (amazon.com)
- Announced: 2026-10-20T17:48:27+00:00
- Category: not stated
- Coverage: not counted
- Announcement: no
- Group: routine
- Source: https://aws.amazon.com/blogs/machine-learning/fine-tuning-gpt-oss-20b-for-multilingual-reasoning-on-amazon-sagemaker-hyperpod/
- Record: https://forck.live/items/25370-fine-tuning-gpt-oss-20b-for-multilingual-reasoning-on-amazon-sagemaker-hyperpod
- Subject: Bedrock / Nova
- Open weights: yes
- Models affected: GPT-OSS-20B, Qwen2.5-7B

Amazon's guide walks through fine-tuning the open-weight GPT-OSS-20B model for multilingual reasoning on Amazon SageMaker HyperPod, using supervised fine-tuning and Group Relative Policy Optimization. The base model thinks in English: asked to reason in French, German, Spanish or Italian, it acknowledges the instruction and then produces its chain-of-thought in English anyway. On a 50-example multilingual reasoning test, the base model reasoned in the correct target language only 22 percent of the time, almost entirely the English examples, with French, Spanish and Italian at 0 percent and German at 10 percent. The guide frames the fix as a choice between SFT, GRPO or both, and says SFT teaches a behavior the model does not have while GRPO optimizes one it can already sometimes produce. Training used the HuggingFaceH4/Multilingual-Thinking dataset, roughly 949 training and 50 validation examples, with evaluation on 10 math and logic problems across five languages. Runs used LoRA on SageMaker HyperPod with Amazon EKS, PyTorch TRL for GPT-OSS-20B and veRL with Ray, KubeRay and vLLM for a Qwen2.5-7B control experiment, with checkpoints on Amazon FSx for Lustre and EFA for inter-node traffic.

## Evidence

Verbatim from https://aws.amazon.com/blogs/machine-learning/fine-tuning-gpt-oss-20b-for-multilingual-reasoning-on-amazon-sagemaker-hyperpod/:

> On a 50-example multilingual reasoning test, the base model reasoned in the correct target language only 22 percent of the time. That 22 percent was almost entirely the English examples (English 100 percent, French 0 percent, German 10 percent, Spanish 0 percent, Italian 0 percent).

---

Record: https://forck.live/items/25370-fine-tuning-gpt-oss-20b-for-multilingual-reasoning-on-amazon-sagemaker-hyperpod
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md