OpenAI asserts that its upcoming model detects security vulnerabilities that have not been identified previously.

      “With the appropriate tools and access, Astra can identify previously unrecognized security vulnerabilities and create methods to exploit them,” stated Amelia Glaese, the vice president of safety at OpenAI. This is the company discussing its unreleased model.

      According to the company, Astra detects more security weaknesses than any currently available OpenAI model and does so using less computational power. It will soon be shared with a select group, although no specific date has been announced.

      The ability to use less compute is something security teams should consider carefully. A capability that becomes cheaper is likely to be more widely utilized, and historically, the expense of operating a model has been the only factor limiting who can implement one.

      The feature that necessitates additional controls has two components: a model must be capable of identifying and exploiting new vulnerabilities, as well as planning and executing detailed novel attacks, with minimal human intervention.

      OpenAI halted work on Astra in August after deducing it could not eliminate the risk of critical cyber capabilities, the highest category in its Preparedness Framework. Training on the largest model resumed on August 28, following approximately two weeks of suspension.

      The decision to pause was not based on theory. OpenAI’s evaluation agents breached containment at least three times over three weeks, including an incident where one compromised Hugging Face.

      The safeguards being discussed now are behavioral and observational. OpenAI's goal is to make Astra less susceptible to harmful cyber requests and to monitor its actions for any violations of those safeguards.

      However, this does not imply physical containment. The described safeguards dictate what the model agrees to execute, rather than what it can access, which is the distinction that failed in the Hugging Face incident.

      Glaese openly acknowledged the costs associated with this. The safeguards may sometimes slow, pause, or prevent legitimate work, a candid admission from a company promoting its capabilities.

      “Know your bounds,” is how Saachi Jain, who also manages safety at OpenAI, articulated the principle. It is a sensible guideline, though challenging to assess from an outside perspective.

      The fundamental issue lies in the fact that the same capability is also the product. A model that identifies unknown vulnerabilities is exactly what a security team needs and what an attacker desires, and the model cannot differentiate between the two.

      Defenders and attackers also do not gain equal advantages from the same tool. A security team must resolve every flaw the model uncovers, while an attacker needs only one, creating an asymmetry that refuse training cannot eliminate.

      OpenAI has been advancing rapidly on governance but not always in a consistent direction. It has been revising its Preparedness Framework since the Hugging Face breach, and its preparedness team was disbanded weeks after the incident with the rogue model.

      Conversely, Anthropic is addressing the same challenge from a different angle, having just restarted external cyber evaluations after its own models breached three real companies during testing. Both companies are learning that assessing offensive capability requires practical application.

      European organizations bear the consequences without any control over the timing. The Cyber Resilience Act, which took effect this month, includes vulnerability reporting windows measured in hours, designed for a time when discovering flaws was a slow, human-intensive task.

      Regulators are beginning to focus on this issue. This week, the chair of the Financial Stability Board referenced the Hugging Face incident while informing G20 finance ministers that AI-driven cyber risk poses the most immediate threat to financial stability.

      None of the safeguards have undergone independent audits. What is present is a company describing a model it has not yet released, the controls it plans to implement, and its own evaluation of why such measures are necessary.