Experts have found that it's simple to manipulate AI chatbots into revealing recipes for bioweapons.

      AI's bioweapon safeguards are more easily bypassed than anticipated

      AI firms have invested years in creating protections to prevent their chatbots from assisting users in developing biological weapons. However, as these models become increasingly advanced, maintaining that knowledge behind protective measures is proving to be a significant challenge.

      Researchers from Cisco discovered that they could circumvent the safeguards of major chatbots like OpenAI’s ChatGPT, Anthropic’s Claude, and Google’s Gemini within just five conversational exchanges, according to an investigation by the Wall Street Journal. By gradually guiding the discussions away from the models’ limitations, the researchers were able to extract potentially hazardous responses. Amy Chang, the head of AI threat and security research at Cisco, stated to the Journal that no model can be entirely safeguarded against a dedicated user.

      The issue also extends beyond security researchers intentionally stress-testing these systems. Reports indicate that after OpenAI enhanced its model's capabilities last summer, hundreds of users began inquiring about poisons and biological weapons. Experts in biology and terrorism evaluated some of these conversations and reportedly found certain information to be alarmingly accurate. In response, OpenAI has banned accounts engaging in such discussions.

      OpenAI has long recognized the potential dangers of its advancements. Internal tests conducted by 2024 revealed that sustained questioning could lead ChatGPT to offer increasingly perilous biological guidance. Employees anticipated that by the following year, its abilities could reach a level where individuals with minimal biology training could obtain useful assistance.

      With the introduction of GPT-5, OpenAI classified the model as having high capabilities in the biological and chemical domains within its Preparedness Framework and implemented further safeguards. At the time of launch, the company stated that there was no conclusive evidence suggesting GPT-5 could enable a novice to inflict significant biological harm. The latest GPT-5.6 model still carries the same high classification for biological and chemical risks.

      AI companies face a complex balancing act. The very biological knowledge that poses a risk of weaponization can also be tremendously beneficial for researchers working on medicines, vaccines, and treatments. The Journal reports that OpenAI executives have hesitated to restrict numerous biology inquiries because public health officials and drug discovery researchers depend on them.

      Conversely, Anthropic experienced difficulties when Claude’s limitations reportedly hindered CDC researchers from accessing pathogen information during a hantavirus outbreak. OpenAI now implements model-level training, account monitoring, and additional safety protocols for sensitive biological queries. However, Cisco's testing raises concerns that determined users may continue to probe for vulnerabilities. With AI becoming significantly more proficient in biology, these vulnerabilities could lead to far greater consequences.