Former Anthropic researcher Jacob Coxon told the New York City Council that AI companies are racing to build technology they cannot control, and that on the current path it is more likely than not that humanity loses control to these AIs, which could end in human extinction.

• Former Anthropic researcher Jacob Coxon told the New York City Council that on the current path it is more likely than not that humanity loses control to advanced AI, which could end in human extinction.
• Coxon said AI companies run on a startup mindset of moving fast and breaking things, which he argued is not appropriate for building the most powerful technology ever built.
• Coxon testified that automating AI research has become the primary goal of top frontier AI labs, and that at his last job he was in some sense working to automate himself.
• When Council Speaker Julie Menin asked representatives from OpenAI, Anthropic, Google and Meta to quantify the worst-case catastrophic risk of AI, OpenAI's Morgan Dwyer said she did not know and that none of those levels is remotely acceptable.