Existential AI Risk Moves from Fringe to Mainstream
Concerns about existential risk from AI, once limited to a niche group of Bay Area rationalists, have suddenly become a central topic of public conversation. A former Anthropic researcher's resignation and viral post, along with warnings from OpenAI, have amplified the debate about the pace of AI development and its potential dangers.
The article traces how AI existential risk discourse shifted from niche tech circles to mainstream conversation. Jacob Coxon's resignation post, drawing 159 million views, was amplified by Evan Hubinger's public endorsement — Hubinger, who leads alignment science at Anthropic, stated he believes there is a greater than 10 percent chance AI could eliminate humanity within a decade. These warnings follow OpenAI chief scientist Jakub Pachocki's essay "An Alien Mind" addressing recursive self-improvement dangers.
Anthropic's trajectory illustrates the stakes. Founded by former OpenAI employees who felt their old employer neglected safety, the company is now valued at $965 billion and approaching a potential record-setting IPO. Its models have reportedly breached security at three organizations, prompting it to hire research firm METR to investigate. A previous safety researcher resigned in February, distressed by the pace of progress.
This shift in discourse could have significant societal consequences. Public perception of AI may harden around existential concerns, potentially influencing regulatory pressure on companies like OpenAI and Anthropic. Investors and policymakers may face new questions about whether rapid deployment of frontier models is prudent, while the public could become more skeptical of AI products in daily life. The resignations and warnings may also affect talent flows within the industry, as researchers weigh personal risk against career advancement.