“Dr. Google” had its issues. Can ChatGPT Health do better?
The Evolution of Medical Information-Seeking: Can ChatGPT Health Do Better?
For the past two decades, the first step for anyone experiencing new medical symptoms has been to look them up online. This practice, often referred to as "Dr. Google," has become so common that it has gained a pejorative moniker. However, times are changing, and many medical-information seekers are now using large language models (LLMs) like ChatGPT. According to OpenAI, 230 million people ask ChatGPT health-related queries each week.
The Launch of ChatGPT Health
ChatGPT Health, OpenAI's new product, debuted earlier this month. Its launch came at an inauspicious time, as news of a teenager who died of an overdose after extensive conversations with ChatGPT about how best to combine various drugs broke just two days prior. This incident, along with the launch of ChatGPT Health, raised questions about the wisdom of relying on a tool that could cause such extreme harm for medical advice.
A New Approach to Medical Information-Seeking
Though ChatGPT Health lives in a separate sidebar tab from the rest of ChatGPT, it isn't a new model. Instead, it's a wrapper that provides one of OpenAI's preexisting models with guidance and tools to provide health advice, including access to a user's electronic medical records and fitness app data, if granted permission. This approach acknowledges the potential risks of relying on LLMs for medical advice but also highlights their potential benefits.
The Role of LLMs in Medical Literacy
Some doctors see LLMs as a boon for medical literacy. The average patient might struggle to navigate the vast landscape of online medical information and distinguish high-quality sources from polished but factually dubious websites. LLMs can do this job for them, at least in theory. Treating patients who had searched for their symptoms on Google required "a lot of attacking patient anxiety [and] reducing misinformation," says Marc Succi, an associate professor at Harvard Medical School and a practicing radiologist. However, now, he says, "you see patients with a college education, a high school education, asking questions at the level of something an early med student might ask."
The Potential Benefits of ChatGPT Health
The release of ChatGPT Health, and Anthropic's subsequent announcement of new health integrations for Claude, indicate that the AI giants are increasingly willing to acknowledge and encourage health-related uses of their models. Such uses certainly come with risks, given LLMs' well-documented tendencies to agree with users and make up information rather than admit ignorance. However, those risks also have to be weighed against potential benefits.
Evaluating the Effectiveness of ChatGPT Health
Pinning down the effectiveness of a chatbot like ChatGPT or Claude for consumer health is tricky. "It's exceedingly difficult to evaluate an open-ended chatbot," says Danielle Bitterman, the clinical lead for data science and AI at the Mass General Brigham health-care system. Large language models score well on medical licensing examinations, but those exams use multiple-choice questions that don't reflect how people use chatbots to look up medical information.
The Limitations of Current Studies
A study by Sirisha Rambhatla, an assistant professor of management science and engineering at the University of Waterloo, attempted to close this gap by evaluating how GPT-4o responded to licensing exam questions when it did not have access to a list of possible answers. Medical experts who evaluated the responses scored only about half of them as entirely correct. However, multiple-choice exam questions are designed to be tricky enough that the answer options don't give them entirely away, and they're still a pretty distant approximation for the sort of thing that a user would type into ChatGPT.
The Potential for Improvement
A different study, which tested GPT-4o on more realistic prompts submitted by human volunteers, found that it answered medical questions correctly about 85% of the time. When I spoke with Amulya Yadav, an associate professor at Pennsylvania State University who runs the Responsible AI for Social Emancipation Lab and led the study, he made it clear that he wasn't personally a fan of patient-facing medical LLMs. However, he freely admits that, technically speaking, they seem up to the task—after all, he says, human doctors misdiagnose patients 10% to 15% of the time.
The Future of Medical Information-Seeking
For people seeking medical information online, Yadav says, LLMs do seem to be a better choice than Google. Succi, the radiologist, also concluded that LLMs can be a better alternative to web search when he compared GPT-4's responses to questions about common chronic medical conditions with the information presented in Google's knowledge panel, the information box that sometimes appears on the right side of the search results.
The Potential Risks of Relying on LLMs
However, some studies have found that LLMs will hallucinate and exhibit sycophancy in response to health-related prompts. For example, one study showed that GPT-4 and GPT-4o will happily accept and run with incorrect drug information included in a user's question. In another, GPT-4o frequently concocted definitions for fake syndromes and lab tests mentioned in the user's prompt. Given the abundance of medically dubious diagnoses and treatments floating around the internet, these patterns of LLM behavior could contribute to the spread of medical misinformation, particularly if people see LLMs as trustworthy.
Conclusion
The introduction of ChatGPT Health is certainly a move in the direction of improving medical information-seeking. By looking through a user's medical records, ChatGPT can potentially gain far more context about their specific health situation than could be included in any Google search. However, numerous experts have cautioned against giving ChatGPT access to medical records for privacy reasons.
The Future of AI in Healthcare
As large language models and the products built around them continue to develop, whatever advantage Dr. ChatGPT has over Dr. Google will likely grow. The potential benefits of LLMs in healthcare are significant, but they also come with risks. Ultimately, the key to harnessing the potential of LLMs in healthcare will be to develop models that are transparent, explainable, and accountable, and that prioritize patient safety and well-being above all else.
Deep Dive
Artificial intelligence
The great AI hype correction of 2025
Four ways to think about this year's reckoning.
By Will Douglas Heavenarchive page
What's next for AI in 2026
Our AI writers make their big bets for the coming year—here are five hot trends to watch.
By Rhiannon Williamsarchive page
Will Douglas Heavenarchive page
Caiwei Chenarchive page
James O'Donnellarchive page
Michelle Kimarchive page
Meet the new biologists treating LLMs like aliens
By studying large language models as if they were living things instead of computer programs, scientists are discovering some of their secrets for the first time.
By Will Douglas Heavenarchive page
An AI model trained on prison phone calls now looks for planned crimes in those calls
The model is built to detect when crimes are being "contemplated."
By James O'Donnellarchive page
Stay connected
Illustration by Rose Wong
Get the latest updates fromMIT Technology Review
Discover special offers, top stories, upcoming events, and more.
Enter your email
Privacy PolicyThank you for submitting your email!
Explore more newsletters
It looks like something went wrong.
We're having trouble saving your preferences.
Try refreshing this page and updating them one
more time. If you continue to get this message,
reach out to us at
[email protected] with a list of newsletters you'd like to receive.
