AI Tools in Medical Records: A Growing Concern for GPs
General practitioners (GPs) are being urged to remain vigilant when using artificial intelligence (AI) tools to generate medical notes. These tools, which often listen to patient consultations and automatically create summaries, have become increasingly common in general practice. However, concerns are mounting about the accuracy and reliability of these systems.
Related: GPs Cautioned: AI Notes Risk Dangerous Errors
The Royal College of GPs has highlighted that AI can misinterpret the subtle nuances of conversations between doctors and patients, potentially leading to dangerous consequences. The Medicines and Healthcare products Regulatory Agency (MHRA) has also warned about the risk of “hallucination” – a term used to describe situations where AI generates information that is not based on real data or context. This issue requires users to be aware and manufacturers to actively work towards minimizing harm.
In response, the MHRA is encouraging GPs to report any issues with AI scribes through its Yellow Card Scheme, typically used for reporting adverse reactions to medicines. This includes suspected inaccuracies in the documentation generated by these tools. The British Medical Association’s GP Committee has noted that the use of passive scribes in general practice is increasing rapidly, with many practices adopting standalone systems or integrating them with other software.
Dr Phil Whitaker, a UK GP who now works in Canada, shared his concerns about an AI tool he used. He reported that it misinterpreted conversations with patients, even recording false details about their movements and medical examinations. While the company behind the tool advises users to check its output carefully, Dr Whitaker found that the time spent correcting errors outweighed any potential productivity gains.
A recent article in Fortune detailed a case in which a London patient was mistakenly invited to a diabetic screening after an AI-generated record falsely claimed he had diabetes and suspected heart disease. Despite these growing concerns, the MHRA has not yet recorded any adverse incident reports related to AI scribes in its database.
The government’s 10-Year Health Plan aims to accelerate the adoption of AI technology, including AI scribes, by streamlining regulations. A new national procurement platform will be introduced next year to help GP practices and NHS trusts adopt new technologies safely.
Professor Kamila Hawthorne, chair of the RCGP, emphasized the potential of AI to transform healthcare. However, she also acknowledged the need for careful regulation to ensure patient safety and data security.
Public Concerns About AI in Healthcare
A recent poll revealed that fewer than one in three Britons feel comfortable using AI features in the NHS App to diagnose their health issues. The Health Secretary, Wes Streeting, announced plans to revamp the app as part of Labour’s 10-Year Health Plan, aiming to provide every patient with a “doctor in their pocket.” However, the survey found that 44% of the public are uncomfortable trusting AI for diagnosis, with this figure rising to 60% among pensioners.
Only 31% of the 2,030 respondents said they were comfortable with the idea. Helen Morgan, the Liberal Democrat’s health spokesperson, praised the initiative but warned that efficiency must not come at the expense of leaving some people behind. She called for support for those less digitally literate, particularly older adults.
Dennis Reed, director of Silver Voices, expressed concerns that elderly individuals may struggle with the transition to digital services. He highlighted the risk that greater reliance on apps could exclude some from timely care.
Regulation and Safety Measures
While AI scribes can offer benefits, particularly during times of high GP workload, there are important concerns around data security, data controllership, and the risk of inaccuracies. The MHRA has advised GPs to only use tools registered as medical devices that meet required standards. Recent guidance clarifies how these technologies qualify as medical devices, and while not published by the MHRA, NHS England encourages the use of registered medical devices in clinical contexts.
The watchdog has updated its Yellow Card Scheme website to include a standalone page for software and AI as medical devices. Earlier this year, the BMA recommended that practices pause the use of AI scribes until they conduct data protection and safety checks and obtain assurances that the products meet NHS standards.
As AI continues to play a larger role in healthcare, ensuring its safe and effective use remains a priority for both professionals and patients.
