# Google Research — The sleep framework: A new way for LLMs to retain knowledge in continual learning

- Company: Google Research (research.google)
- Announced: 2026-10-27T17:35:28+00:00
- Category: research-paper
- Coverage: not counted
- Announcement: yes
- Group: announcements
- Source: https://research.google/blog/the-sleep-framework-a-new-way-for-llms-to-retain-knowledge-in-continual-learning/
- Record: https://forck.live/items/28380-the-sleep-framework-a-new-way-for-llms-to-retain-knowledge-in-continual-learning
- Subject: Research

Google Research is proposing a sleep framework that gives large language models a periodic cycle of active interaction and offline consolidation, so they can absorb new facts without losing old ones. The problem it answers is that deployed models stay static behind a knowledge cutoff, while continual parameter updates often trigger catastrophic forgetting and in-context learning forgets everything once the session ends. The framework alternates a wake phase, where the model stores temporary information in short-term memory, with a sleep phase that redirects computation to memory consolidation and self-improvement. During consolidation, the model scales its capacity by adding low-rank experts into stable lower-frequency memory blocks through a Mixture of Experts and Hope architecture, then resets the high-frequency temporary parameters. A dreaming stage generates synthetic samples, randomly activates non-standard experts to mix domains, filters the samples by gradient-based selection, and refines the model with supervised fine-tuning. Google Research says the approach lets models retain long-context knowledge up to 10M tokens and improve complex reasoning, all within a fixed architecture and human-defined training rules.

## Evidence

Verbatim from https://research.google/blog/the-sleep-framework-a-new-way-for-llms-to-retain-knowledge-in-continual-learning/:

> We introduce a sleep phase in continual learning, enabling large language models to continuously integrate new facts and enhance reasoning while mitigating catastrophic forgetting or static knowledge cutoffs.
> 
> Modern LLMs achieve impressive performance via pre-training and in-context learning (ICL). However, once deployed, they remain largely static, operating with a fixed knowledge cutoff date that leaves them unable to acquire long-term capabilities or adapt to new information.
> 
> While updating models with new data seems like the obvious solution, continual parameter updates often trigger catastrophic forgetting , a fundamental issue where learning new tasks severely degrades performance on previously learned ones. Alternatively, relying strictly on ICL limits knowledge retention to the immediate context window: once the context session ends, the acquired short-term information is forgotten.

---

Record: https://forck.live/items/28380-the-sleep-framework-a-new-way-for-llms-to-retain-knowledge-in-continual-learning
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md