
Trillion Token Context. No, Really — Anima Anandkumar & Benedikt Jenik, Accelerated Understanding
About this episode
From the show’s notesWhat would a GPT-style foundation model for the **physical world** look like?
Note: this is a followup to youtube.com/watch
Read the show’s notes in full
Just days after emerging from stealth, Accelerated Understanding co-founders Anima Anandkumar and Benedikt Jenik, join us to explain their bet: that the universality and scale we’ve seen in language models can also emerge across physics. They’re building a single model designed to learn across very different physical systems—fluid dynamics, semiconductors, energy, and more—and they say their experiments are already showing something important: models trained across multiple areas of physics can outperform equally sized models trained on individual domains alone.
We dig into how they’re doing it, from neural operators and resolution-invariant architectures to simulator-generated curricula, physics-based self-improvement, and some truly enormous computational scales: trillion-context training, five-trillion-context inference, and outputs measured in tens of terabytes. Anima and Benedikt also explain why physical AI may have an advantage over language models when it comes to self-improvement—the laws of physics provide a dense, objective training signal—and where they see the first major commercial opportunities, including semiconductor design, manufacturing, energy, geothermal exploration, and scientific discovery.
Chapters





