Patients are turning to AI before seeing specialists, raising concerns about diagnostic accuracy, clinical decisions, and trust in healthcare.
Rheumatologists should look beyond the accuracy of artificial intelligence tools and consider how AI-generated advice is shaping patients’ understanding of their symptoms, treatment decisions, and trust in healthcare before they reach the consulting room, an Australian researcher has warned.
In a new paper published in the Australian Rheumatology Journal, Associate Professor Joshua Pate, from the University of Technology Sydney, has proposed the concept of an “AI gauge gap” to describe the disconnect between how AI systems were traditionally evaluated and their potential effects on patients seeking care for rheumatic and musculoskeletal conditions.
The paper argued that existing AI governance frameworks largely focused on technologies formally deployed within healthcare services, overlooking the growing influence of general-purpose AI chatbots and other tools used independently by patients.
Professor Pate said patients were increasingly using AI to interpret symptoms, prepare questions, seek reassurance, and determine whether medical attention was necessary.
This meant AI could influence clinical encounters well before a rheumatologist had the opportunity to assess a patient.
The concern is particularly relevant in rheumatology, where symptoms such as pain and fatigue may be difficult to interpret, fluctuate over time, and carry significant psychological and social implications.
A 2025 national survey of patients with rheumatic diseases found that 27% had used AI for health-related purposes, while 58% expressed interest in doing so.
The same proportion said they would welcome rheumatologists using AI-supported second opinions in clinical decision-making.
However, the paper highlighted evidence that patient acceptance of digital symptom assessment tools did not necessarily translate into adequate diagnostic performance.
In a multicentre study involving 600 patients, 73% said they would recommend the AI-based Ada symptom assessment tool, while 79% would recommend the online Rheport system.
Despite this relatively high acceptance, a subsequent diagnostic analysis found that Rheport achieved sensitivity of just 62% and specificity of 47% for inflammatory rheumatic disease. Ada’s top-ranked diagnosis achieved sensitivity of 52% and specificity of 68%.
The diagnostic performance of both tools was considered insufficient for safe triage, illustrating the potential disconnect between patient confidence in digital health technologies and their clinical reliability.
Professor Pate argued that conventional measures of AI performance, including diagnostic accuracy, technical capability, and efficiency, failed to capture the broader consequences of AI-generated information.
These included changes in how patients interpreted their symptoms, whether they sought medical attention, their willingness to follow clinical advice, and their trust in healthcare professionals.
“The safety of generative AI extends beyond factual accuracy, because people may over-trust fluent medical advice, and sycophantic AI models may agree too readily with a user’s initial premise,” he wrote.
“In musculoskeletal care, agreement with a patient’s fear, doubt, or even a reductive explanation of pain, may all influence the meaning they assign to their symptoms before more familiar risks arise, such as diagnostic delay or clinical errors.”
The paper also raised concerns about the potential for AI systems to reinforce inaccurate beliefs through overly agreeable responses.
“An AI conversation could legitimise symptoms, minimise them, or convert uncertainty into a confident explanation,” Professor Pate wrote.
“Clinicians can invite patients to discuss the advice suggested, what seemed helpful or worrying, and which decisions still require shared judgement. Such discussion should cover benefits, limitations, uncertainty, privacy, and escalation, rather than only record a ‘yes/no’ for AI use.
“Such interactions could influence patients’ beliefs and subsequent behaviour before more conventional safety concerns, including diagnostic delays or clinical errors, became apparent.”
To address these challenges, Associate Professor Pate proposed five questions for clinicians and healthcare services to consider when evaluating patient-facing AI.
These focused on where AI enters the care pathway, how it changes patients’ interpretation of symptoms, its influence on subsequent actions, its effect on clinical encounters, and who remains accessible and accountable when AI-generated advice influences patient decisions.
The framework encourages clinicians to discuss AI use openly rather than simply documenting whether a patient has consulted an AI tool.
Patients could instead be invited to explain what advice they received, which aspects they found useful or concerning, and whether the information had influenced their decisions about seeking care or managing symptoms.
Professor Pate suggested this approach could help identify misconceptions, address inappropriate reassurance, and preserve opportunities for shared clinical decision-making.
Related
It also highlighted the potential benefits of carefully developed AI tools, including improved access to information and better preparation for specialist consultations.
One example was LupusGPT, a multilingual, patient-led and clinician-supported information model for people with systemic lupus erythematosus. The system was developed using co-design, curated evidence, retrieval-based information grounding, and iterative testing.
However, Professor Pate cautioned that improving the technical reliability of AI-generated answers did not necessarily establish whether the information positively influenced patients’ understanding, behaviour or trust.
These outcomes would require separate evaluation involving patients and consumers.
The paper also identified implications for healthcare service design, particularly for patients using AI while waiting for specialist appointments.
Potential responses included clearer waiting-list information, public-facing guidance on AI-generated health advice, accessible escalation pathways, and opportunities for patients to bring AI-generated information into consultations.
Professor Pate argued that simply keeping a “human in the loop” was insufficient unless patients knew how to access appropriate clinical support, when to seek assistance and who was responsible for responding to concerns.
Healthcare services should therefore establish clear human contact pathways, response times, escalation triggers, and accountability arrangements.
Equity was another important consideration, with the paper highlighting the need to examine how AI affected children, older adults, carers, people with limited health or digital literacy, culturally and linguistically diverse communities, and patients with restricted access to specialist services.
Professor Pate also cautioned that the proposed approach had not been developed with Aboriginal and Torres Strait Islander communities and should not be applied in those settings without appropriate Indigenous leadership, governance, and accountability.
“The AI gauge gap presented here is a conceptual proposal,” he concluded.
“Its clinical value should be tested through co-design and empirical evaluation. A durable question for musculoskeletal services and professional communities to keep on our radar is: How should care function when AI is involved before a clinician?”
