# Sakana AI — An Evolved Universal Transformer Memory

- Company: Sakana AI (sakana.ai)
- Announced: 2024-12-09T15:00:00+00:00
- Subject: Sakana models
- Source: https://sakana.ai/namm
- Record: https://forck.live/items/4726-an-evolved-universal-transformer-memory

This is the second in a series of blog posts presenting the results of Sakana AI’s research projects that were supported by the Japanese Ministry of Economy, Trade and Industry’s GENIAC supercomputing grant . Summary At Sakana AI, we draw inspiration from nature to extend the capabilities of foundation models. This was evidenced in our recent work on improving performance with evolutionary model merging , discovering qualitative and diverse agentic skills , and uncovering new ways for LLMs to contribute to AI research . Following on in this line of research, our latest breakthrough, detailed in our paper, An Evolved Universal Transformer Memory , introduces a revolutionary memory system for transformers, inspired by how human memory selectively retains and prunes information. Our learned memories not only boost both performance and efficiency of existing, pre-trained transformers, but are also universally transferable across different foundation models, even beyond language, without any re-training! Memory is a crucial component of cognition , allowing humans to selectively store and extract important information from the ceaseless noise dominating our lives. …

---

Record: https://forck.live/items/4726-an-evolved-universal-transformer-memory
Catalogue: https://forck.live/llms.txt
Feed: https://forck.live/feed.md