The concept of AI receptionists appears to be risky, especially after a patient disclosed that the system used by her doctor was unable to comprehend her speech.

      A general practice surgery in South Yorkshire has discontinued its AI receptionist after a woman who was recovering from a stroke struggled to communicate with it and subsequently switched to another practice. The NHS Accessible Information Standard has mandated that practices identify and address patients' communication needs since 2016.

      The woman, whose stroke had left her with fragmented speech and affected her right side, found the automated system unresponsive to her attempts to book an appointment. She eventually chose to register with a different practice, prompting the surgery to withdraw the AI system.

      The specifics matter here. The automated service advised her against using speakerphone, which she required to use since she could only hold the handset with one hand.

      This issue was not an isolated incident. Healthwatch Rotherham reported that the AI receptionist named Emma "does not respond well to variations in speech such as regional accents or speech impairments."

      Some patients expressed frustration over the system. One individual informed the watchdog, “I could never get it to understand me, so I just hung up and gave up trying to book an appointment,” while others opted to visit the surgery in person.

      The system's developer, QuantumLoopAI, has a different perspective. They claim that Emma has been trained on a variety of accents and dialects, supports 17 languages, allows callers to request a staff member at any time, and that over 90% of patients have reported improvements.

      Nationally, the trend is heading in the opposite direction. The NHS App plans to utilize AI for patient triage as part of a £10 billion technology modernization effort.

      Currently, governance is being managed at the local level. South Yorkshire’s integrated care board has stated that it is collaborating with Healthwatch to provide guidance and notes that the adoption of such systems is a decision made by individual GP practices.

      What's on the horizon is more advanced, not less. A UK supplier is developing a triage model for the NHS in partnership with Nvidia, aimed at determining patient needs before a clinician sees them.

      A standard already exists for this; the Accessible Information Standard mandates GP practices to identify, record, highlight, and address patients’ communication needs, a requirement in place since 2016.

      Europe categorizes the technology; the EU AI Act classifies emergency healthcare patient triage as high-risk, which involves rigorous testing, logging, and human oversight responsibilities that a standard booking line does not entail.

      In Britain, the standard exists without this classification. What has occurred is one practice that ceased using the AI system after a woman recovering from a stroke opted to go elsewhere.