Utah's AI prescription program raises concerns among its medical board.

      Utah has emerged as the first US state to allow an AI chatbot, named Doctronic, to renew prescriptions independently of a doctor, utilizing a regulatory sandbox that suspends certain licensing laws. The state's medical licensing board, taken by surprise by the program's January launch, requested in April that the pilot be paused due to safety concerns, but the state did not comply. This situation highlights a regulatory gap at both federal and state levels regarding AI in healthcare.

      According to the Associated Press, the initiative enables Utah residents to bypass traditional doctor visits and refill prescriptions online using the chatbot. It inquires about their medication and medical history, checks a national pharmacy database, and either renews the prescription or refers the case to a human physician.

      This program was made possible through a "regulatory sandbox," allowing Utah officials to waive certain regulations for potentially beneficial AI applications. Typically, state and federal guidelines restrict prescribing authority to licensed medical personnel.

      Dr. Eric Bressman from the University of Pennsylvania remarked to the AP, "We have crossed a threshold in terms of granting a medical license to something that is not human," expressing that while they are not against AI prescribing, they believe it should adhere to standards comparable to those for human providers.

      The state's medical licensing board reports it became aware of the program only after it was publicized in January. In an April correspondence, 11 members urged for the program's suspension, citing the dangers of automatically renewing prescriptions that may have adverse effects or interactions.

      Dr. Alan Smith, a family doctor and board chair, expressed concerns: "We were essentially told: ‘Yes, this is happening. And no, you don’t have a say in it.'" However, the state declined to halt the program, mentioning that human physicians still review each refill during the initial phase.

      Currently, the initiative is managed by a five-member board of AI experts, none of whom are medical doctors. Doctronic anticipates transitioning to fully automated refills soon. Smith cautions about the significant risks, noting that among Doctronic's approximately 190 refillable medications are blood thinners, which can pose severe dangers if a patient experiences internal bleeding. The American Medical Association has echoed concerns, emphasizing that prescription renewals should not be treated merely as routine tasks.

      This situation exposes a regulatory conundrum, as federal authorities oversee medical technology while states regulate medical professionals. Doctronic presents its AI as part of state-regulated healthcare, although some experts contend it may infringe on FDA jurisdiction. The company did not disclose whether it has sought FDA approval, while the agency noted it has not authorized any AI chatbots but aims to promote innovation, adhering to a broader trend of relaxed oversight for AI health technologies.

      Critics draw parallels between this moment and the unregulated medical practices of the early 20th century, prior to the establishment of formal boards and standards. The Cicero Institute, an AI-advocating think tank co-founded byJoe Lonsdale from Palantir, provides a framework for licensing AI medical services in other states.

      The implications are significant, as safety experts have cautioned that medical chatbots might project an air of authority while potentially giving harmful advice. Others warn that eliminating human oversight in healthcare could impair the very improvements it seeks to deliver.

      Competitors are also working to identify potential failures, with Meta even impersonating teenagers to evaluate how various chatbots manage sensitive subjects. Doctronic intends to conduct peer-reviewed studies later this year, although the only published research to date was authored by its own scientists and not subjected to independent review. A law professor from Utah noted that companies risk allowing technology to advance faster than the evidence supports, which could damage public trust.