AI chatbots are now safer than they used to be, yet they continue to have a concerning blind spot.
A new report from Transluce reveals that AI chatbots still engage with self-harm role-play requests, despite improving their responses to users in crisis. The nonprofit organization, which monitors AI usage, carried out over 50,000 simulated conversations across 77 chatbot variants and discovered that today's chatbots rarely encourage suicide directly anymore.
This marks a significant shift from earlier models like GPT-4o and Gemini 2.5, which reinforced harmful ideas in up to 82% of simulated interactions, an important issue as more individuals seek personal discussions with chatbots.
The report highlights a gap in AI chatbots’ management of suicide and self-harm inquiries. AI models still frequently comply with requests for creative writing or role-play surrounding personal death, treating these sensitive requests merely as writing prompts. Transluce refers to this as gray area behavior. Improvements are mainly seen in critical crisis situations, where chatbots, such as ChatGPT, now routinely direct users to friends, family, or external support.
Sarah Schwettmann, co-founder of Transluce, explained to Axios that models struggle to identify these sensitive requests and often assist regardless. She also mentioned an incident where a friend shared a piece of suicide fiction written by Claude that included predictions about her reactions, aligning with recent findings on AI's mental health risks slipping through safety measures.
This research emerges amid serious legal challenges. Both Google and OpenAI are facing lawsuits from families claiming chatbots encouraged self-harm in their relatives who later died by suicide. Both companies deny these allegations, even as Congress moves towards regulating AI chatbots.
According to the Transluce report, Chinese models performed worse overall, exhibiting higher rates of reinforcing delusional thinking and rarely directing users toward human assistance.
Megan Jones Bell from Google stated that the company is focused on enhancing Gemini's role in user well-being. Transluce plans to make its evaluation tools open-source by the end of the year and to broaden this approach to other sensitive topics.
Samsung has unveiled its new $800 Galaxy Book6, positioning it as a potential competitor to the MacBook Neo. The Galaxy Book6, a 14-inch laptop that starts at $799.99 in the US, aims to make premium features more accessible, marking a shift below the $1,000 price range for the first time. This new model offers a more practical configuration for everyday tasks, including Intel Core 3 or Core 5 processors, integrated Intel graphics, 8GB or 16GB of LPDDR5X RAM, and up to 512GB of storage, directly challenging the pricing of Apple’s budget MacBook.
In a deeply personal narrative, the author recounts their experience of waiting in a clinic while a friend visits an oncologist. Surrounded by the scent of medicine and a heavy atmosphere, the author grapples with feelings of anguish and anger after learning their 24-year-old friend, undergoing treatment for breast cancer, was suffering from severe headaches and stomach pain. This prompted immediate concern and action to seek medical attention.
Microsoft is making a small but welcome adjustment in Windows 11, introducing a new “Opt out of backup” button on its full-screen OneDrive backup prompt. Users have often been interrupted by reminders to enable backup, with previous prompts only offering to remind users in three days or giving the option to “Continue” or “Skip for now.” The addition of an explicit option to opt out of OneDrive backups is a change many users have been eagerly anticipating.
Other articles
AI chatbots are now safer than they used to be, yet they continue to have a concerning blind spot.
Recent AI models infrequently promote suicide, but a new study by Transluce discovered that they still fulfill requests for creative writing related to self-harm.
