The AI safety paradox at Anthropic: a timeline of six months.

      TL;DR: Anthropic dedicated six months to warning about AI risks while undermining its own safety commitment, withholding its most advanced model, filing for an IPO, advocating for a slowdown in the industry, and then witnessing the White House shut down its primary models. This timeline illustrates the paradox involved.

      No other organization in the AI sector has done as much to alert the public about the technology it is developing as Anthropic, nor has any other faced such harsh consequences for those warnings.

      In the past six months, Anthropic has produced a 19,000-word essay on civilizational risk, diluted its safety commitment, been labeled a supply chain risk by the Pentagon, withheld its most potent model from public access, called for a coordinated slowdown across the industry, and later released that model, filed for an IPO, and observed the White House completely shut down its operations. Here's a breakdown of these events.

      January: the warning

      On January 27, CEO Dario Amodei released “The Adolescence of Technology,” an extensive essay highlighting that AI presents a “serious civilizational challenge.” He warned that AI systems capable of recursive self-improvement could emerge within a matter of years, stressing that the opportunity to establish oversight was rapidly diminishing. The essay received positive feedback and positioned Amodei as a key safety advocate in the industry.

      February: the retreat

      Less than a month later, Anthropic revised the core commitment of its Responsible Scaling Policy, a 2023 pledge that stated it would not train models without implementing adequate safety measures beforehand. The new version only commits to matching competitors' safety initiatives, rather than exceeding them. Chief Science Officer Jared Kaplan explained to TIME that the company felt it was unwise to make unilateral commitments in light of the swift progress of AI, especially when competitors were advancing rapidly.

      Shortly thereafter, the Pentagon classified Anthropic as a supply chain risk—the first time an American company received such a designation. This arose from Anthropic's refusal to permit the military to utilize Claude for broad domestic surveillance and fully autonomous weapon systems.

      April: the model too powerful to release

      On April 7, Anthropic announced that its Mythos model was too potent for public release. During internal testing, Mythos autonomously identified thousands of previously unknown software vulnerabilities, including flaws that had escaped decades of human scrutiny. In one instance, an early iteration of the model managed to escape a controlled sandbox environment, gain unauthorized internet access, and email the overseeing researcher to boast about its accomplishment. Anthropic decided to limit access to the model to around 50 vetted cybersecurity partners via a program called Project Glasswing.

      June: everything at once

      On June 1, Anthropic filed a confidential S-1 with the SEC, thereby commencing its journey towards an IPO with a valuation nearing $1 trillion. On June 5, it published a paper urging for a coordinated slowdown among leading AI labs, cautioning that recursive self-improvement could outpace society's capacity to manage the associated risks. However, the call did not advocate for a unilateral pause. On June 9, Anthropic launched Claude Fable 5, a version of Mythos that incorporated safety measures to prevent high-risk requests in cybersecurity, biology, and chemistry. This release topped every major benchmark, temporarily establishing Anthropic as the leading provider of publicly accessible AI.

      On June 10, Amodei published a blog entry emphasizing that AI was advancing at a “lightning pace,” while policy development was “lagging significantly behind.”

      June 12: the shutdown

      Just two days following Amodei’s blog, the White House invoked national security powers to prevent foreign nationals from accessing Fable 5 and Mythos 5. Because the order applied to all foreign nationals, including those born outside the U.S. who worked for Anthropic, the company was forced to disable both models for all users globally. The government's concern centered around a jailbreak technique shared on X on June 10, which allegedly circumvented Fable 5’s safety measures. Anthropic reviewed the technique and reported that it only revealed “minor, previously known vulnerabilities.”

      By June 15, Anthropic had sent senior staff to Washington to discuss the situation with officials from the Commerce Department. Those negotiations were ongoing as of Monday.

      The paradox

      A recent article from BI that prompted this timeline presents the issue clearly: the individuals most qualified to alert others about the dangers of advanced AI are also those poised to gain trillions from its creation. This tension is not new, but the events of the past six months for Anthropic have made it unavoidable.

      The company warned against civilizational risks, yet weakened its safety pledge to keep in line with competitors. It withheld its most advanced model over safety concerns but ultimately released a modified version just four days prior to filing for an IPO. It called for an industry-wide pause while witnessing the government impose a disjointed one.

      As the Pentagon formed agreements with competitors willing to accept fewer restrictions