
🔬 Why Transformers Hit a Wall the Moment Physics Shows Up — Anima Anandkumar, Caltech
About this episode
From the show’s notesMost of AI is pointed at language. Anima Anandkumar points it at the physical world, weather, fusion, materials, the systems physics writes down as equations, and finds the usual playbook breaks. There isn't enough data, the resolution is brutal, and no transformer is big enough. Her way through is older than deep learning: build the structure of the world into the model. It already produces weather forecasts that rival the supercomputers on a single GPU.
Co-founder of Accelerated Understanding, Anima has spent two decades in AI, from the theory that predates deep learning, through scaling it at AWS and NVIDIA, and back to first principles. We get into neural operators, Fourier layers, and the spherical harmonics behind FourCastNet 3, then follow the same ideas into fusion reactors, chip design, and formal verification, before closing on her seat on the UN Scientific Advisory Board and why AI for science shouldn't be regulated like a chatbot.
Read the show’s notes in full
Note: we did a brief followup to this interview to dig into more details on Accelerated Understanding after the launch: youtu.be/KS_IpnX7n9I
CHAPTERS
LINKS
Papers
FourCastNet: arxiv.org/abs/2202.11214
Neural Operators:
arxiv.org/abs/2108.08481
Fourier Neural Operator:
arxiv.org/abs/2010.08895
FourCastNet 3:
arxiv.org/abs/2507.12144
TorchLean: arxiv.org/abs/2602.22631
Lean:





