RePo: Language Models with Context Re-Positioning

Sakana AI announces RePo, a method that allows language models to dynamically reorganize their context by learning to assign positions based on content relevance, improving robustness on noisy contexts, structured data, and long-range dependencies.