The Debate: is AI a Threat to Humanity's Future?
WHO:
Fast Company - Dario Amodei, CEO of Anthropic, published an essay calling for slower model development, third-party audits, and international coordination on safety.
Fast Company - Mustafa Suleyman, CEO of Microsoft AI, published an essay arguing that Anthropic's approach to model welfare could make advanced AI harder to control.
CNN - Jacob Coxon, an employee of Anthropic, announced his resignation due to concerns about AI safety.
CNN - Evan Hubinger, another Anthropic worker, stated the chance of AI-powered human extinction within 10 years is greater than 10%.
CNN - Nate Soares, president of the Machine Intelligence Research Institute (MIRI), coauthor of “If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All.”
CNN - Heidy Khlaaf, chief AI scientist at the AI Now Institute and former OpenAI safety engineer.
CNN - Thomas Larsen, a researcher at the AI Futures Project and former MIRI employee.
CNN - Eric Xing, a professor of machine learning at Carnegie Mellon University.
CNN - Herbert Lin, a senior research scholar and research fellow at Stanford University.
WHAT:
Fast Company - Trump revoked Biden’s AI executive order on his first day back in office in January 2025, which required developers of powerful AI systems to share safety-testing results with the federal government.
Fast Company - OpenAI has held discussions about a capital raising exercise that would value it at about $1.2 trillion before going public.
CNN - Jacob Coxon stated that he quit Anthropic because its work was too dangerous, prompting reactions from politicians and major news organizations.
CNN - Nate Soares described the potential outcome of AI development as “literally everyone on the planet dying, like the last human drawing the last breath.”
CNN - Heidy Khlaaf emphasized the need for scientific claims to be falsifiable to avoid religious-like arguments.
CNN - Thomas Larsen mentioned that researchers are currently using AI to assist in viral and toxin research.
CNN - Eric Xing noted that existing regulations and law enforcement are effective in controlling the risks associated with chemical weapons.
CNN - Herbert Lin stated that the actual material risk of nuclear weapons remains with the weapons themselves, not an imagined future state.
CNN - Thomas Larsen projected future AI development scenarios in reports titled “AI 2027” and “AI 2040,” suggesting that even optimistic outcomes end with machines taking over.
CNN - Lin commented on the seductive nature of programming AI, which leads to predictions of limitless advancement.
CNN - Jacob Coxon quit Anthropic over fears that AI could become impossible to control.
CNN - Soares expressed that the most likely outcome of AI development could lead to the extinction of humanity.
CNN - OpenAI announced it discovered a new batch of instances of “misalignment” in its AI models.
WHEN:
CNN - Predictions suggest that AI could exterminate humanity within the decade.
WHY:
Fast Company - Trump suggested that warnings about AI risks are coming from political foes aiming to damage his economic success measures, particularly the stock market.
CNN - Concerns about AI development stem from fears of it becoming a threat to humanity, including the potential for AI to create bioweapons or autonomous robots that could harm humans.
CNN - Soares indicated that the worry is not about AI taking control of existing nuclear weapons but about AI developing its own advanced technologies.
HOW:
Fast Company - The U.S. plans to discuss cooperation with China on threats including AI-directed cyberattacks during the first official bilateral talks devoted exclusively to AI since Trump returned to office.
CNN - Larsen suggested that a superintelligent AI could convince a human to help it develop a deadly virus.
CNN - Xing compared the challenge of automated malicious virology to working with Legos without instructions, emphasizing the complexity of creating a virus.
CNN - Khlaaf noted that nuclear facilities are “air-gapped” from the internet, limiting AI's ability to commandeer them.
(UPDATED September 17, 2026 17:00 UTC)
Dario Amodei, CEO of Anthropic, has published an essay advocating for slower model development and international safety coordination, while Mustafa Suleyman, CEO of Microsoft AI, argues that Anthropic's approach could complicate the control of advanced AI. In a significant move, Trump revoked Biden’s AI executive order on his first day back in office in January 2025, which mandated that developers share safety-testing results with the federal government. Meanwhile, the U.S. is set to engage in its first official bilateral talks focused solely on AI, where cooperation with China will be discussed, particularly regarding threats like AI-directed cyberattacks.
--
(UPDATED September 17, 2026 14:59 UTC)
Jacob Coxon has announced his resignation from Anthropic, citing concerns that the company's work poses significant dangers, a statement that has sparked reactions from politicians and major news outlets. In a chilling forecast, Nate Soares warned that the trajectory of AI development could lead to catastrophic outcomes, suggesting a scenario where "literally everyone on the planet dying" becomes a reality. Meanwhile, experts like Heidy Khlaaf and Thomas Larsen are emphasizing the importance of scientific rigor in discussions about AI, with Larsen projecting future scenarios in reports titled “AI 2027” and “AI 2040,” where even the most optimistic outcomes hint at machines ultimately taking control.
--
(UPDATED September 17, 2026 14:00 UTC)
Jacob Coxon has resigned from Anthropic, citing concerns that artificial intelligence could become impossible to control, while Nate Soares, president of the Machine Intelligence Research Institute, warned that the likelihood of AI-powered human extinction within the next decade is greater than 10 percent. In related news, OpenAI has reported new instances of “misalignment” in its AI models, raising alarms about the potential threats posed by advanced AI technologies, including the risk of creating bioweapons or autonomous robots. Experts like Thomas Larsen and Eric Xing have highlighted the complexities involved in AI development, with predictions suggesting that if left unchecked, AI could pose an existential threat to humanity within the decade.