[{"@context":"https:\/\/schema.org\/","@type":"Article","@id":"https:\/\/share-dev.upmc.com\/2025\/12\/symptom-checking-ai-danger\/#Article","mainEntityOfPage":"https:\/\/share-dev.upmc.com\/2025\/12\/symptom-checking-ai-danger\/","headline":"AI Chatbots for Health: Risks, Responsible Use, and Why Human Oversight Matters","name":"AI Chatbots for Health: Risks, Responsible Use, and Why Human Oversight Matters","description":"<p>Learn the risks of using AI chatbots for symptom checking and how to use them responsibly. Explore expert tips for why real doctors matter for medical advice.<\/p>","datePublished":"2025-12-23","dateModified":"2026-03-05","author":{"@type":"Organization","@id":"https:\/\/www.upmc.com\/","name":"UPMC","url":"https:\/\/www.upmc.com\/","sameAs":"https:\/\/share-dev.upmc.com\/upmc\/","parentOrganization":"UPMC"},"publisher":{"@type":"Organization","name":"UPMC HealthBeat","logo":{"@type":"ImageObject","@id":"https:\/\/share-dev.upmc.com\/wp-content\/uploads\/2019\/04\/UPMC-HealthBeat-Logo.png","url":"https:\/\/share-dev.upmc.com\/wp-content\/uploads\/2019\/04\/UPMC-HealthBeat-Logo.png","width":600,"height":60}},"image":{"@type":"ImageObject","@id":"https:\/\/share-dev.upmc.com\/wp-content\/uploads\/2025\/12\/GettyImages-2170879154-e1766508748412.jpg","url":"https:\/\/share-dev.upmc.com\/wp-content\/uploads\/2025\/12\/GettyImages-2170879154-e1766508748412.jpg","height":866,"width":2000},"url":"https:\/\/share-dev.upmc.com\/2025\/12\/symptom-checking-ai-danger\/","about":["Family Health","Health Topics A-Z"],"wordCount":1453,"articleBody":"Your favorite chatbot may seem like it has an endless pool of knowledge. But when it comes to medical advice, no chatbot should replace a medical professional.According to an April 2025 survey from the University of Pennsylvania&#8217;s Annenberg Public Policy Center, 79% of American adults are likely to go online to get an answer about a health condition or symptom. Fully 75% of respondents said that AI responses sometimes or often give them what they need. Over 60% said they believe AI-generated health information is somewhat or very reliable.Using AI to learn basic information about medical conditions can prove useful. It also shows that you\u2019re taking an interest in learning more about your overall health. However, you shouldn\u2019t rely on it for medical advice. Using a chatbot as a symptom checker can lead to health misinformation, potentially causing a dangerous delay in care.In this article, discover the risks of self-diagnosis with AI tools and learn how to use online symptom checkers responsibly. Also, find out why it\u2019s better to talk to your doctor for any medical advice.Are AI Chatbots Safe for Medical Advice?The simple answer is no \u2014 you shouldn\u2019t use artificial intelligence for medical advice. Whether you\u2019re using a symptom checker or a chatbot, AI health tools are often prone to health misinformation, bias, and other concerns.Here are some of the potential red flags with AI tools and your health:1. It may provide misinformationAI chatbots may seem like they know what they\u2019re talking about, but they\u2019re not medical experts. Although technology is continually improving, there\u2019s still a real risk of misinformation.This misinformation can occur for many reasons:AI can \u201challucinate\u201d\u2014 A 2025 study in Communications Medicine reported that language-learning models could generate false facts, also known as \u201challucinations.\u201d The authors reported that false information included in user prompts was one source of these hallucinations.AI can\u2019t perform a physical examination \u2014 A physical examination can identify many potential health issues. An AI chatbot or symptom checker can\u2019t perform a physical exam, meaning it\u2019s missing a key piece of diagnostic information.AI doesn\u2019t know your full medical history \u2014 Your medical history plays a large role in your overall health. Your genetics and other health factors can play a role in your risk for many health conditions. Though symptom checkers may ask you to provide certain parts of your medical history, they can\u2019t get the full picture. Complex diagnoses can prove especially difficult.AI may provide outdated information \u2014 Just as technology is a rapidly evolving field, so is medicine. An AI chatbot may provide medical advice based on outdated data, which can lead to misinformation.AI may provide unreliable information \u2014 AI models can pull from many different online sources. Not all of these sources are reliable.AI often presents information authoritatively. But misinformation from an AI health tool can lead to an incorrect self-diagnosis, which can have dangerous consequences. You may end up delaying necessary care, putting your health at risk.A 2024 report in the Central European Journal of Medicine outlined a case study of a patient who relied on an erroneous diagnosis from an AI chatbot. This caused a life-threatening delay in care for a transient ischemic attack (TIA), a temporary blockage of blood flow to the brain, also known as a \u201cmini-stroke.\u201d2. AI can demonstrate bias.The information that AI health tools provide may show bias that reflects health inequities in society. A 2023 study in JAMA Network Open reported that AI chatbots provided different recommendations based on factors like:EthnicityGenderRaceSocioeconomic statusAnother potential harm of bias comes in the field of mental health. A 2025 study in Computation and Language reported that large language models (LLMs) expressed stigma toward people with mental health conditions and provided inappropriate advice. This shows a potential danger for people who seek mental health advice from a chatbot.3. There\u2019s the possibility of privacy issues.You should always remain cautious about sharing private data with an AI chatbot, including health information. A 2025 study in Computers and Society reported that six leading American AI developers use user chat data to train their AI models.How Can I Safely Use AI for Health Information?Though you shouldn\u2019t completely rely on artificial intelligence for medical advice without consulting a medical professional, it has certain benefits in the health care space. AI can make medical information more readable, which can help you better understand a diagnosis. AI can also help you determine questions to ask your health care provider.Remember, though, that you shouldn\u2019t solely rely on AI for medical advice. That\u2019s why we have medical professionals.AI health literacy tips for patientsLooking to use AI for health information? Keep these online patient safety tips in mind:Check the source \u2014 Make sure that the source your AI tool is pulling from is reliable. Check the source itself to verify the information.Follow up \u2014 Don\u2019t just accept an answer from a chatbot or symptom checker without double-checking. Check multiple sources (non-AI) to make sure the information is reliable.Keep private information to yourself \u2014 Don\u2019t share private health information with a chatbot.Remember that AI isn\u2019t perfect \u2014 Though AI can provide a wealth of information, it\u2019s still prone to misinformation.If you have any unusual or worrying medical symptoms, contact your doctor, not AI. If you have serious or life-threatening medical conditions, call 911 or go to your nearest emergency department. If you\u2019re having suicidal thoughts, call or text the National Suicide Prevention Lifeline at 988.Why Should You Talk to Your Doctor for Medical Advice Instead of Using AI?Though AI tools can help you learn background information on health issues, you should turn to a medical professional for medical advice. The best place to start is typically your primary care provider (PCP). These are typically the medical professionals who know you best.There are many benefits to contacting your PCP for a diagnosis:They can perform a more thorough examination \u2014 A medical professional can perform a physical examination, which can prove crucial for an accurate diagnosis. They can also ask you detailed questions about your symptoms.They can run tests \u2014 Your PCP can order lab work or other diagnostic tests, which isn\u2019t possible with AI. They can also refer you to specialists if more thorough evaluations are necessary.They have medical training \u2014 Although AI tools have medical information at their disposal, it isn\u2019t a substitute for medical education and training.They know your medical history \u2014 Your PCP will have a more detailed knowledge of your medical history, which can provide potential insights into your symptoms.They\u2019ll protect your privacy \u2014 Medical professionals must follow laws that protect your privacy, such as the Health Insurance Portability and Accountability Act (HIPAA). These laws require them to protect your private health information.Asking AI for information on health conditions is a useful tool \u2014 it\u2019s available 24\/7, and it can make complex information more understandable. However, it\u2019s important to stay cautious.And if you have any symptoms that you\u2019re feeling concerned about, the best source to turn to is your doctor.SourcesAnnenberg Public Policy Center. Many in U.S. Consider AI-Generated Health Information Useful and Reliable. Accessed December 2025. https:\/\/www.annenbergpublicpolicycenter.org\/many-in-u-s-consider-ai-generated-health-information-useful-and-reliable. AnnenbergPublicPolicyCenter.orgJMIR Formative Research. Longitudinal Changes in Diagnostic Accuracy of a Differential Diagnosis List Developed by an AI-Based Symptom Checker: Retrospective Observational Study. Accessed December 2025. https:\/\/formative.jmir.org\/2024\/1\/e53985. Formative.JMIR.orgJAMA Network Open. Assessing Biases in Medical Decisions via Clinician and AI Chatbot Responses to Patient Vignettes. Accessed December 2025. https:\/\/jamanetwork.com\/journals\/jamanetworkopen\/fullarticle\/2810775. JAMANetwork.comComputers and Society. User Privacy and Large Language Models: An Analysis of Frontier Developers\u2019 Privacy Policies. Accessed December 2025. https:\/\/arxiv.org\/abs\/2509.05382. arXiv.orgConsumer Reports. We Quizzed 5 AI Chatbots for Health and Safety Advice. Accessed December 2025. https:\/\/www.consumerreports.org\/electronics\/artificial-intelligence\/we-quizzed-ai-chatbots-for-health-and-safety-advice-a1164538940. ConsumerReports.orgComputation and Language. Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers. Accessed December 2025. https:\/\/arxiv.org\/abs\/2504.18412. arXiv.orgInternal Medicine Journal. Can artificial intelligence improve patient educational material readability? A systematic review and narrative synthesis. Accessed December 2025. https:\/\/onlinelibrary.wiley.com\/doi\/10.1111\/imj.16607. Wiley.comCommunications Medicine. Multi-model assurance analysis showing large language models are highly vulnerable to adversarial hallucination attacks during clinical decision support. Accessed December 2025. https:\/\/www.nature.com\/articles\/s43856-025-01021-3. Nature.comScientific American. AI Chatbots Can Diagnose Medical Conditions at Home. How Good Are They? Accessed December 2025. https:\/\/www.scientificamerican.com\/article\/ai-chatbots-can-diagnose-medical-conditions-at-home-how-good-are-they. ScientificAmerican.comCentral European Journal of Medicine. Delayed diagnosis of a transient ischemic attack caused by ChatGPT. Accessed December 2025. https:\/\/link.springer.com\/article\/10.1007\/s00508-024-02329-1. Springer.com"},{"@context":"https:\/\/schema.org\/","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"2025","item":"https:\/\/share-dev.upmc.com\/2025\/#breadcrumbitem"},{"@type":"ListItem","position":2,"name":"12","item":"https:\/\/share-dev.upmc.com\/2025\/\/12\/#breadcrumbitem"},{"@type":"ListItem","position":3,"name":"AI Chatbots for Health: Risks, Responsible Use, and Why Human Oversight Matters","item":"https:\/\/share-dev.upmc.com\/2025\/12\/symptom-checking-ai-danger\/#breadcrumbitem"}]}]