# Tencent — From LR to ELR: A Better Heuristic for Pretraining Dynamics

- Company: Tencent (tencent.com)
- Announced: 2026-08-05T16:00:00+00:00
- Category: research-paper
- Subject: Hunyuan
- Source: https://hunyuan.tencent.com/research/elr
- Record: https://forck.live/items/4576-from-lr-to-elr-a-better-heuristic-for-pretraining-dynamics

Tencent Hunyuan research introduces the Effective Learning Rate (ELR) as a better heuristic for controlling angular update size (AUS) during LLM pretraining. The post shows that ELR leads to Hyperball optimization, enables more accurate loss prediction, and the proposed minus-square-root schedule sets a new SOTA on the Modded-nanoGPT Track 3 benchmark, reaching target loss 3.28 in 3175 steps.

## Evidence

Verbatim from https://hunyuan.tencent.com/research/elr:

> When training Large Language Models (LLMs), the learning rate (LR) is often treated as the main knob controlling optimization speed. ... This motivates the notion of *angular update size (AUS)* ... In this post, we focus on directly controlling the AUS during LLM training. Specifically, we rewrite the update rule ... and explicitly control the AUS via the *effective learning rate (ELR)*

---

Record: https://forck.live/items/4576-from-lr-to-elr-a-better-heuristic-for-pretraining-dynamics
Catalogue: https://forck.live/llms.txt
Feed: https://forck.live/feed.md