Ajeya Cotra – "This might be the clearest warning shot we ever get"
About this episode
From the show’s notesAjeya Cotra is a researcher at METR, where she works on threat modeling for loss-of-control risks from advanced AI. Before that, she led the technical AI safety program at what is now Coefficient Giving.
She is one the three authors of METR and Redwood Research’s “Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident”.
Read the show’s notes in full
We go through not only what she and her coauthors discovered during this investigation, but what it means for how we should train future, smarter AIs which might be involved in the process of recursive self-improvement.
Read Ajeya's takeaways from this incident here: planned-obsolescence.org/p/the-…
𝐄𝐏𝐈𝐒𝐎𝐃𝐄 𝐋𝐈𝐍𝐊𝐒
* Transcript: dwarkesh.com/p/ajeya-cotra
* Apple Podcasts:
podcasts.apple.com/us/podcast/a…
* Spotify:
open.spotify.com/episode/5xZnb1…
𝐒𝐏𝐎𝐍𝐒𝐎𝐑𝐒 * Jane Street’s ML engineering internships start with an intense four-day bootcamp: PyTorch, autograd, writing kernels, profiling workloads… all the things that Jane Street engineers need to know for their daily work. After that, interns tackle real projects, things the firm actually wants in its codebase. If you want to apply, or if you want to watch my recent conversation with Axel, one of Jane Street’s ML engineers, go to janestreet.com/dwarkesh
