My friend has cancer. Seeing her seek medical advice from ChatGPT really rattled me.

      I composed this article around 3 am while sitting in a clinic lobby, waiting for my friend who was consulting with an oncologist in the adjacent cabin. The air was thick with the scent of medications, and a palpable sense of gloom made me eager to leave the premises as soon as possible. However, the dominant feeling I experienced was one of anger and frustration.

      Just an hour earlier, I had received a call from my 24-year-old friend, who was suffering from a severe headache and stomach pain. She is currently being treated for breast cancer, and after witnessing her struggles over the past two years, I dashed to her home, anxious that it might be serious. Without delving too deeply into details, I quickly found the nearest hospital and took her for a checkup.

      During the ride, I inquired about when the pain began and whether she had noticed any other concerning symptoms. Her response left me stunned, sparking a desire to snatch her phone and smash it. “After coming home from work, I felt dizzy and sensed something was off in my stomach. So, I asked ChatGPT if I could take painkillers, and a few hours after taking them, the stomach pain intensified with a burning sensation. I also vomited several times.”

      Who is responsible for this situation?

      Unsplash

      Her admission disturbed me, especially the nonchalant way she recounted her interactions with ChatGPT. There are numerous layers to this incident, prompting me to decide to prohibit AI use for my loved ones, particularly concerning health matters. Given that she is extremely sensitive to medication and undergoing cancer treatment, seeking AI advice about medicine was the last thing one would expect from a rational person.

      I am unsure exactly who should be held accountable for this potentially dangerous situation. She simply took a photo of a painkiller, uploaded it to ChatGPT, and inquired whether she could take two pills at once. Her doctor had advised her to take only one pill at a time, and only when the pain was unbearable. ChatGPT informed her she could take one painkiller, but to take two only with a doctor’s guidance.

      When I questioned her about why she consulted the AI, she later explained that ChatGPT “is easier to get answers from. I don’t even know what’s true online.” As someone who had to leave school after high school to support her family with various jobs, I sympathized with her situation. I couldn't scold her on the importance of digital literacy and fact-checking. Both sides share blame in this instance.

      Herein lies the problem. She is not a native Hindi speaker, and when she translated her question into Hindi, it became a grammatical mess due to a single word. She intended to ask if she could take two of the pills shown in the image, but her flawed translation made it seem as if she was asking ChatGPT if she could take “both medicines.”

      ChatGPT also stumbled here. The image displayed a pill with only its chemical composition visible — Diclofenac and Serratiopeptidase. The name of the drug was partially obscured. Nonetheless, the AI accurately identified the painkiller as “something like Cipzen-D.” However, it also mentioned another medication, Fenak Plus, whose ingredients are Diclofenac and Paracetamol and which she had discussed previously.

      ChatGPT Unsplash

      These two medications have entirely different compositions, despite both being available as over-the-counter painkillers that can (concerningly) be purchased without a proper prescription at local pharmacies. Thus, we have one human error (the translation) and two mistakes on ChatGPT’s part since it was also drawing on knowledge from its database. Nevertheless, the advice given was correct.

      OpenAI’s AI bot explicitly warned that taking both medications is inadvisable, noting it could lead to an overdose of Diclofenac. It also outlined the potential consequences, which included a burning sensation in the stomach, ulcer issues, bleeding, and long-term damage to the kidneys.

      It’s important to note that the image fed to ChatGPT depicted only one painkiller. It further advised that only one of those painkillers should be taken at a time, not both. Recalling that both medications had been prescribed to her by doctors at different times (though not concurrently), my friend took both for quick pain relief and soon faced dire consequences.

      The doctor was understandably frustrated when my friend explained why she took two pills simultaneously, reprimanding her for not consulting a healthcare professional first. This was followed by an injection to alleviate her pain and a new prescription for painkillers that were safe for her sensitive condition.

      As a technology reporter familiar with the risks of AI interactions regarding sensitive issues, especially health and wellness, I felt like I had failed. I realized I had never properly discussed the dangers of AI bots with my friend, and it wasn't until after the frightening incident that I warned her.

      When she confessed to using ChatGPT for medical advice on multiple