Transformer²: Self-Adaptive LLMs
Research paper
Sakana AI announces a research paper titled Transformer², which proposes a self-adaptive machine learning system that dynamically adjusts its weights for various tasks using Singular Value Finetuning (SVF) and reinforcement learning, outperforming static approaches like LoRA.
