Anthropic's safety hiring focuses on nuclear and biological threats.

      An examination of Anthropic's safety hiring reveals its primary concerns: analysts recruited to prevent its models from instructing individuals on how to create nuclear, chemical, and biological weapons. Unlike typical job advertisements that communicate a mission, Anthropic's listings resemble a threat evaluation.

      The company has recently announced multiple openings for enforcement analysts tasked with ensuring its AI does not assist in weapon construction, scams, or cybercrime, as reported by Axios. One job listing specifically seeks an “Enforcement Analyst focused on Radiological & Nuclear Harms,” while others address issues related to chemicals, explosives, and financial fraud.

      Compensation ranges from the mid- to upper-$200,000s, and the positions do not involve coding. Anthropic is looking for individuals with real-world knowledge in areas such as biology and explosives, alongside those who can think like adversaries attempting to bypass its safeguards.

      The explicit job titles are intentional. A spokesperson stated, “Preventing our models from disseminating potentially harmful information is integral to responsible development.” The company noted that it routinely employs experts in sensitive domains to rigorously test its models prior to release.

      Clarifying the specific dangers, it explained, aids in attracting suitable candidates. Anthropic claims that hundreds of its employees are now focused on safety, examining vulnerabilities and remedying them.

      This approach reflects the company's response to critics who label it as the industry's foremost alarmist. Anthropic is investing significantly in addressing the risks it continues to highlight.

      Chief Executive Dario Amodei has dedicated months to outlining the potential downsides. In a January essay, he expressed that biological attacks are his greatest concern. He remarked, “I do not believe biological attacks will occur the moment they become widely feasible. However, aggregated across millions of individuals over several years, I perceive a substantial risk of a significant attack, with potential casualties reaching millions or more.”

      He has also sounded alarms regarding AI's potential to aid cybercriminals and bolster authoritarian regimes. Earlier this year, Anthropic parted ways with the US Defense Department regarding the use of its technology for mass surveillance and autonomous weaponry.

      Similar actions are being taken by OpenAI, which is hiring researchers focused on biological and chemical threats, with a base salary of up to $445,000. As AI models become more advanced, every major lab is in a race to establish a red team.

      This competition occurs within a regulatory vacuum. The US lacks comprehensive AI safety legislation, with Congress failing to pass any measures despite years of effort. Some advocate for oversight: Google’s Demis Hassabis has proposed a watchdog akin to those on Wall Street for cutting-edge models. Fewer than one in a hundred AI PhDs pursue careers in government, resulting in the expertise being concentrated in private companies.

      The outcome is a peculiar form of self-regulation. Those developing the most perilous technologies are simultaneously the ones determining how to contain them. Amodei himself has acknowledged this tension, referring to AI companies as a next tier of risk following hostile nations. His careers page encapsulates both the argument and the caution in one place: the individuals most equipped to avert disaster are employed by the company that could potentially instigate it.