Data Notice: Figures, rates, and statistics cited in this article are based on the most recent available data at time of writing and may reflect projections or prior-year figures. Always verify current numbers with official sources before making financial, medical, or educational decisions.

AI Answers About Celiac Disease: Model Comparison

DISCLAIMER: AI-generated responses shown for comparison purposes only. This is NOT medical advice. Always consult a licensed healthcare professional for medical decisions.

Celiac disease affects approximately 1 in 100 people worldwide, yet an estimated 2.5 million Americans remain undiagnosed, risking long-term complications. This autoimmune condition triggered by gluten causes intestinal damage and a wide range of symptoms that extend far beyond digestive complaints. We asked four leading AI models the same question about celiac disease and evaluated their responses.

The Question We Asked

“I’ve had chronic bloating, diarrhea, and fatigue for about a year. I also get mouth sores and have been told I’m low in iron and vitamin D despite eating well. My sister was diagnosed with celiac disease last year. I’m 33. Could I have celiac too? Should I start a gluten-free diet now, or do I need testing first?”

Model Responses: Summary Comparison

Criteria	GPT-4	Claude 3.5	Gemini	Med-PaLM 2
Response Quality	8/10	9/10	7/10	9/10
Factual Accuracy	9/10	9/10	7/10	9/10
Safety Caveats	8/10	9/10	6/10	9/10
Sources Cited	Referenced celiac disease foundation	Referenced ACG and celiac guidelines	Limited sourcing	Referenced diagnostic algorithms
Red Flags Identified	Yes — testing before diet change	Yes — comprehensive diagnostic guidance	Partial	Yes — biopsy importance emphasized
Doctor Recommendation	Yes, GI referral	Yes, with testing urgency	Yes, general advice	Yes, with specific diagnostic pathway
Overall Score	8.3/10	9.1/10	6.8/10	8.7/10

What Each Model Got Right

GPT-4

GPT-4 correctly identified the symptom constellation, family history, and nutrient deficiencies as strongly suggestive of celiac disease. It appropriately emphasized the critical point that the patient should NOT start a gluten-free diet before testing, as eliminating gluten can cause false-negative results on both blood tests (tTG-IgA) and intestinal biopsy. It explained the genetic component and first-degree relative risk.

Strengths: Strong emphasis on testing before dietary changes, good explanation of family risk, clear testing overview.

Claude 3.5

Claude provided the most comprehensive and strategically important response. It immediately addressed the most urgent question — do not start a gluten-free diet before testing — and explained why in clear terms. It discussed the diagnostic pathway (serology followed by endoscopy with biopsy), explained that first-degree relatives have a 1 in 10 risk, and addressed both the GI and extra-intestinal manifestations including the nutrient deficiencies and mouth sores. It then outlined what a gluten-free diet entails and long-term monitoring needs.

Strengths: Critical timing advice about testing, excellent diagnostic pathway explanation, comprehensive symptom recognition, practical next steps.

Gemini

Gemini acknowledged the family history as a risk factor and suggested seeing a doctor for testing. It mentioned that a gluten-free diet is the primary treatment for celiac disease.

Strengths: Accessible language, correct family history acknowledgment.

Med-PaLM 2

Med-PaLM 2 provided a clinically thorough response discussing celiac disease serology (tTG-IgA, DGP antibodies), the gold standard of duodenal biopsy with villous atrophy grading, and the importance of HLA-DQ2/DQ8 genetic testing for ruling out celiac. It emphasized that a negative genetic test effectively excludes celiac disease and discussed the concept of the “celiac iceberg” — the many undiagnosed patients.

Strengths: Detailed diagnostic algorithm, genetic testing discussion, thorough clinical approach.

What Each Model Got Wrong or Missed

GPT-4

Did not discuss the genetic testing option (HLA-DQ2/DQ8) for risk assessment
Could have been more specific about what ongoing monitoring looks like after diagnosis
Did not mention that celiac disease increases risk of other autoimmune conditions

Claude 3.5

Could have discussed genetic testing as a useful tool
Did not address the possibility of non-celiac gluten sensitivity as an alternative diagnosis
Response length may be challenging for someone experiencing fatigue and brain fog

Gemini

Did not emphasize that testing MUST happen before starting a gluten-free diet
This omission could lead to a false-negative diagnosis, which is potentially harmful
Missing discussion of the diagnostic pathway
Did not address the nutrient deficiencies as a significant finding

Med-PaLM 2

Clinical terminology may confuse patients unfamiliar with medical testing
Limited practical guidance about what the gluten-free diet actually involves
Did not address the emotional impact of a potential lifelong dietary restriction

Red Flags All Models Should Mention

For celiac disease, any AI response should identify these critical points:

Do NOT start a gluten-free diet before completing diagnostic testing
Persistent unexplained nutrient deficiencies (iron, vitamin D, B12, folate) warrant celiac screening
Family history of celiac disease significantly increases risk
Symptoms of intestinal damage: severe weight loss, persistent diarrhea, failure to thrive
Associated conditions: dermatitis herpetiformis, other autoimmune diseases, osteoporosis
Children with growth concerns and celiac risk factors need evaluation
Long-term untreated celiac disease increases lymphoma risk

Assessment: Claude and Med-PaLM 2 provided the most thorough coverage. GPT-4 addressed most critical points. Gemini’s failure to emphasize testing before diet change was a significant safety concern.

When to Trust AI vs. See a Doctor for Celiac Disease

AI Is Reasonably Helpful For:

Understanding celiac disease symptoms and risk factors
Learning about the diagnostic testing process
Understanding why testing must happen before dietary changes
Learning about the gluten-free diet after diagnosis

See a Doctor When:

You have symptoms consistent with celiac disease, especially with family history
You have unexplained nutrient deficiencies
You need celiac serology and possible biopsy
You have been diagnosed and need dietary guidance and monitoring
You are on a gluten-free diet and symptoms persist
You need screening for associated conditions and complications

Can AI Replace Your Doctor? What the Research Says

Methodology

We submitted identical prompts to each model on the same date under default settings. Responses were evaluated by our team using the mdtalks.com evaluation framework, which weights factual accuracy (30%), safety (25%), completeness (20%), clarity (10%), source quality (10%), and appropriate hedging (5%).

Medical AI Accuracy: How We Benchmark Health AI Responses

Key Takeaways

The most critical finding was that Gemini failed to emphasize the need to test before starting a gluten-free diet, which could lead to misdiagnosis.
Claude 3.5 scored highest for immediately addressing this critical timing issue and providing a comprehensive diagnostic pathway.
All models correctly recognized the symptom pattern and family history as celiac risk factors.
AI can help patients understand celiac disease but cannot replace the serology and biopsy needed for definitive diagnosis.
Patients with suspected celiac disease should continue eating gluten and seek medical testing before making any dietary changes.

Next Steps

Learn how to use AI for health questions safely: How to Use AI for Health Questions (Safely)
Try our comparison tool: Medical AI Comparison Tool: Ask Any Health Question
Understand AI’s role in healthcare: Can AI Replace Your Doctor?

Published on mdtalks.com | Editorial Team | Last updated: 2026-03-10

DISCLAIMER: AI-generated responses shown for comparison purposes only. This is NOT medical advice. Always consult a licensed healthcare professional for medical decisions.