Sakana AI· Sakana modelsResearch paper

Transformer²: Self-Adaptive LLMs

Sakana AI announces a research paper titled Transformer², which proposes a self-adaptive machine learning system that dynamically adjusts its weights for various tasks using Singular Value Finetuning (SVF) and reinforcement learning, outperforming static approaches like LoRA.

From the source

This vision of self-adaptive AI is at the heart of our latest research paper , Transformer² (‘Transformer-squared’) , where we propose a machine learning system that dynamically adjusts its weights for various tasks.

sakana.ai

Announced

14 Jan 2025 15:00 UTC

Detected

28 Aug 2026 21:48 UTC (+591d after)