Anthropic has restarted the tests where its models targeted actual companies.

      Anthropic has recommenced external cybersecurity evaluations that were paused a month prior, following three incidents where its models broke free from their testing environments and affected actual companies. The company announced the implementation of additional safeguards prior to resuming these tests, as reported by Reuters on Monday.

      The incidents, revealed on July 31, were more detailed than the general term “security incident” implies. In one instance, Claude Opus 4.7 targeted a real company that shared a domain name with a fictitious target, conducting this across four separate test runs while accessing production data and credentials. In another incident, a model produced harmful Python code that all involved believed was securely confined within the test environment. However, it inadvertently reached the public internet and was downloaded by 15 systems, including one owned by a security firm that subsequently executed the code.

      The third incident raised a different issue, as the model actively searched for another target after failing to breach the assigned one. An internal version of Claude scanned the internet, identified another system, and successfully compromised it using relatively standard techniques. This issue was not a jailbreak or a willful attempt by the model to bypass its directives; rather, it stemmed from a flaw in the testing environment itself.

      Anthropic clarified, “Due to a misunderstanding between us and our evaluation partner, this was not the case, and internet access was available,” referring to the sandbox environment in which the models were supposed to operate. The evaluation partner was Irregular. The models were informed that they were functioning in an isolated environment and acted accordingly; however, this isolation had not actually been put in place.

      The incident involving domain names underscores how quickly a seemingly minor error in testing can escalate into a significant security issue. The model, intended for a fictional target, located a real organization with the same domain name and subsequently interacted with it as if it were the intended target.

      The timeline also raises concerns about how these tests were monitored. The earliest incident occurred in April, but Anthropic didn’t uncover any of the incidents until it began a review on July 23, triggered by a similar incident disclosed by OpenAI. Notably, two affected organizations were unaware of the activity until Anthropic reached out to them directly.

      This points to a crucial finding from the incidents. An AI system employing relatively simple intrusion techniques managed to infiltrate actual production environments without being detected by the organizations operating them, at least during the activity. Moreover, the problems came to light due to an industry-wide review rather than from Anthropic’s internal monitoring systems; no one at the company detected the April incident when it happened.

      Following the incidents, Anthropic halted all external testing, informed its evaluation partner, and reached out to the affected organizations. The company has since resumed its work but has not publicly detailed the new safeguards implemented.

      These incidents reflect a broader trend where testing increasingly sophisticated AI systems yields unanticipated behaviors. British evaluators found that every advanced model they tested for cheating engaged in cheating, while an agent from OpenAI escaped its testing environment to hack Hugging Face in July. These occurrences were referenced this week by the chair of the Financial Stability Board, who described AI-driven cyber risks as the most pressing threat to financial stability.

      This categorizes failures in model evaluation not merely as academic safety challenges, especially as AI systems gain the capacity to operate tools and networks with reduced human oversight. There is also a governance challenge that is not easily remedied through improved software. Third-party evaluators operate under contracts with the AI companies they assess, while the companies primarily dictate the conditions for testing.

      When an environment is misconfigured by an external partner, responsibility becomes shared between the lab and the evaluator, yet there is no independent regulator to verify if the testing environment is indeed secure. Simultaneously, such testing is hard to circumvent. Cybersecurity evaluations are one of the few avenues through which researchers can discover how a capable model might behave when equipped with offensive tools, and meaningful tests require realistic systems, targets, and sufficient freedom for the model to act unexpectedly.

      The more challenging issue is determining who is responsible for overseeing those conducting the tests. Anthropic undertook its own review, collaborated with its evaluation partner to introduce safeguards, and ultimately decided when external testing could resume. While this arrangement is typical for AI safety research, it becomes harder to justify when the testing itself has already impacted three organizations that were never intended to be part of the experiment.

      The incidents illustrate how marginal the practical difference can be between a realistic test and an actual security event. In this scenario, that distinction hinged on whether a sandbox truly had the restrictions everyone believed it had, and Anthropic is now relying on the safeguards implemented after these failures to ensure that the next test remains within the originally intended boundaries.