The concept of AI receptionists appears to be concerning after a patient disclosed that it did not comprehend her request.

      Your GP utilizes an AI receptionist, which hopes you articulate your words clearly.

      AI receptionists are intended to simplify contacting a doctor. However, a growing number of patients find that this isn't always the case, especially when the technology struggles to comprehend their manner of speaking, follow instructions, or articulate their needs.

      A recent incident in South Yorkshire highlights this issue. Judith Butterfield, a 71-year-old woman recovering from a stroke, shared with the BBC her frustration that the AI receptionist at her GP's office repeatedly failed to comprehend her speech while she attempted to schedule an appointment. Due to her stroke, her speech was fragmented, and her right side was affected. After several unsuccessful tries, she ultimately switched to another practice.

      The practice has since discontinued the system. However, this event raises a significant concern about how much of healthcare can be entrusted to an AI that anticipates patients to communicate in specific ways it understands.

      A machine can manage the queue, but what occurs when a patient diverges from the expected script?

      The AI receptionist, called EMMA and created by QuantumLoopAI, was implemented to assist GP practices in managing phone inquiries and reducing wait times. Healthwatch Rotherham has since received complaints from patients who found it difficult to interact with the system, including individuals with regional accents and speech difficulties.

      Butterfield’s story illustrates how a minor technical limitation can escalate into a more serious issue in healthcare. The automated service reportedly instructed her not to use speakerphone, yet she explained she had to hold the phone in one hand because the other hand was affected by her stroke. When the system did not recognize her name, there was no human intuition to identify that something was amiss and simply ask her to repeat it differently.

      The problem extends beyond this one receptionist system. Healthwatch England has gathered similar experiences from patients facing AI when trying to book appointments, request medication, or communicate their needs. Some individuals with speech or learning difficulties expressed that automated systems could not understand them, while others encountered issues with having their requests appropriately addressed.

      Healthcare cannot afford to rely on AI that only suits straightforward conversations.

      AI is indeed appealing for overburdened GP practices. An immediate-response system could potentially lessen wait times and free up reception staff for other responsibilities. QuantumLoopAI asserts that EMMA supports various languages and accents, allowing callers to request a human representative whenever needed. The company also mentioned that practices can set up priority routing so certain patients connect directly to reception staff.

      However, this safeguard is only effective if patients are aware of it and can navigate the system effectively. Healthwatch has cautioned that individuals with communication challenges, disabilities, and lower digital confidence are particularly at risk of being excluded by poorly designed healthcare technology. The issue isn’t simply about AI misrecognizing words; it’s about the individual on the other side potentially struggling to access healthcare in the first place.

      As GP practices continue to explore automated reception and triage, the lesson from South Yorkshire is hard to overlook: an AI receptionist should facilitate reaching a human, not become yet another barrier between a patient and necessary care.