# Sakana AI — Extending the Context of Pretrained LLMs by Dropping Their Positional Embeddings

- Company: Sakana AI (sakana.ai)
- Announced: 2026-01-11T15:00:00+00:00
- Category: research-paper
- Subject: Sakana models
- Source: https://sakana.ai/drope
- Record: https://forck.live/items/4700-extending-the-context-of-pretrained-llms-by-dropping-their-positional-embeddings

Sakana AI introduces DroPE, a method to extend the context length of pretrained LLMs by dropping positional embeddings during inference, requiring less than 1% of the original pretraining budget and outperforming established methods on LongBench and RULER.

## Evidence

Verbatim from https://sakana.ai/drope:

> We’re excited to introduce DroPE: Extending the Context of Pretrained LLMs by Dropping Their Positional Embeddings!

---

Record: https://forck.live/items/4700-extending-the-context-of-pretrained-llms-by-dropping-their-positional-embeddings
Catalogue: https://forck.live/llms.txt
Feed: https://forck.live/feed.md