Granite 4.1 LLMs: How They’re Built
New model
IBM's Granite 4.1 family of dense, decoder-only LLMs (3B, 8B, 30B) is introduced, trained from scratch on ~15T tokens with a multi-stage pre-training pipeline including long-context extension to 512K tokens.
The models are further refined with supervised fine-tuning on ~4.1M curated samples and reinforcement learning via on-policy GRPO with DAPO loss.
All models are released under the Apache 2.0 license.
Notably, the 8B instruct model matches or surpasses the previous Granite 4.0-H-Small (32B-A9B MoE) despite using a simpler dense architecture.
From the source
Granite 4.1 is a family of dense, decoder‑only LLMs (3B, 8B, and 30B) trained on ~15T tokens using a multi‑stage pre‑training pipeline, including long‑context extension of up to 512K tokens. The models are further refined with supervised fine‑tuning on ~4.1M high‑quality curated samples and reinforcement learning via on‑policy GRPO with DAPO loss (Yu et al., 2025). Notably, the 8B instruct model matches or surpasses the previous Granite 4.0‑H‑Small (32B‑A9B MoE) despite using a simpler dense architecture with fewer parameters. All Granite 4.1 models are released under the Apache 2.0 license.
huggingface.co
