Why AI Labs With Unlimited GPUs Still Fail — Anjney Midha, AMP
About this episode
From the show’s notesFrom building Discord’s developer platform and backing frontier AI companies like Anthropic, Mistral, Black Forest Labs, and Periodic Labs to now building AMP’s independent compute grid, Anjney Midha has spent years close to the real bottlenecks of AI scaling. In this episode, Anjney joins swyx at Periodic Labs to unpack why the AI race is not just about buying more GPUs, why 95% utilization would have been considered an outage at Google, and why the next era of AI infrastructure has to be more aligned, more efficient, and more responsible.
We go deep on AMP’s vision for a compute grid that makes FLOPs flow like megawatts, the difference between full-stack AI labs and horizontal pooling, why AI data centers need community buy-in, and how compute markets could evolve into something closer to an independent system operator. Anjney also explains why DeepMind’s unpublished research points to a market failure, why end-of-life prediction remains one of the most important AI applications he has thought about for fourteen years, and why “output maxing” may become a new discipline for frontier systems.
Read the show’s notes in full
We also discuss Anthropic’s culture, why “luck favors the prepared mind” in coding models, how Claude cracked coding, why too much capital too early can make AI labs fragile, what Periodic Labs is trying to do with science and superconductors, why great researchers can become great CEOs, and why Silicon Valley is both deeply missionary and deeply mercenary.
We discuss: • Why 95% utilization was considered an outage at Google • Why AI infrastructure waste compounds at frontier-lab scale • Why “move fast and break things” does not work for AI data centers • How data center backlash, power grids, and community incentives shape AI scaling • AMP’s vision for making FLOPs flow like megawatts • Why compute needs an independent system operator • How interruptible demand and dynamic prioritization worked inside Google • Why DeepMind research being hoarded creates negative externalities • AMP’s 1.2GW base-load ambition and the need for 6GW of spike capacity • Why end-of-life prediction could become one of AI’s most important healthcare applications • Frontier Systems, output maxing, and full-stack alignment • Why APIs, abstraction layers, and organizations become lossy as they scale • Superconductors, standards, and the dream of lossless systems • SF Compute, open protocols, and the future of compute marketplaces • Why non-NVIDIA chips can still benefit from NVIDIA’s reference architecture • Trust boundaries and why chip startups need visibility into future model architectures • Why VCs often underestimate researchers as CEOs • Scientists as star athletes of the mind • Why great CEOs need to be confrontational up and down the stack • Why “winning” is less important than leading the frontier • How Anthropic cracked coding • Why culture is fragile, not a permanent moat • Why hardship was a feature, not a bug, for Anthropic • Why Anthropic’s P0 was coding from day one • Periodic Labs, physics as the constraint, and technical reality • Silicon Valley mercenaries, missionary teams, and what happens after a breakthrough
