Anthropic is advocating for governments to possess the authority to prevent the implementation of hazardous AI systems.

      Anthropic has introduced two policy frameworks that advocate for government authority to prohibit dangerous AI models and propose economic protections for workers as AI transforms the job market. The Advanced AI Framework focuses on safety regulations, while the Economic Policy Framework deals with issues such as job displacement, capital distribution, and social safety nets.

      The safety framework is more assertive, with Anthropic urging governments to have the ability to prevent the deployment of models posing a "significant risk of catastrophic harm." This would include civil penalties linked to the annual global revenue of offending companies, increasing with subsequent violations. The regulations would target models trained with over 10²⁵ floating-point operations created by companies generating over $500 million in AI revenue or investing more than $1 billion in AI research and development.

      This criteria encompasses a limited group of companies, including Anthropic, OpenAI, Google DeepMind, xAI, and possibly Meta. Anthropic’s push for regulation directly impacts its own operations, contrasting with competitors who oppose oversight.

      The framework outlines four categories of catastrophic risk: the creation of biological weapons, large-scale cyber vulnerabilities, loss of control over autonomous systems, and AI systems that can conduct their own research and development. Anthropic referenced its experience with Mythos Preview, which found numerous severe vulnerabilities in major operating systems and browsers, highlighting that these risks are very real.

      Frontier developers would need to conduct model testing, publish result summaries, submit to independent evaluations, maintain security measures, and regularly report on risks. Anthropic asserts that mere transparency is no longer sufficient, stating, “AI capabilities will greatly advance in the coming months,” and that governance must keep up.

      On the issue of federal versus state regulation, Anthropic criticized the White House's effort to hinder state AI regulations, asserting that Congress should not override state laws unless it establishes a federal law as strong as their proposed framework. They argue for targeted preemption to allow states to manage regulations on child safety, consumer protection, and other areas not covered by federal safety laws.

      The economic framework, while less detailed, addresses concerns regarding job displacement voiced by graduating students. It includes suggestions for capital accounts, wage insurance, tax incentives, and a broader social safety net to ensure that the financial advantages of AI are "broadly shared," although details on funding and implementation are to be discussed later.

      The timing of these proposals is intentional, as Anthropic is currently embroiled in a legal battle with the Pentagon over its designation as a supply chain risk, preparing for an IPO, and monitoring Congressional negotiations that could trade state AI preemption for online safety laws. By releasing its framework, Anthropic aims to influence a rapidly evolving regulatory discussion in which it has had limited involvement.