Interpreting AI Differential Diagnoses for Oral Lesions

1. The Diagnostic Challenge of Oral Lesions

Forming a differential diagnosis for oral lesions is one of the most challenging aspects of clinical dentistry. Many different conditions—from harmless irritations to life-threatening cancers—can appear visually similar, especially in their early stages.

To appreciate the complexity that AI aims to help with, let's start with a review article that sets the scene.

As the paper highlights, the sheer variety in how these lesions present makes it difficult to distinguish between benign, premalignant, and malignant conditions based on visual inspection alone. This is where AI can offer powerful support. Instead of just performing a binary task (e.g., "lesion present/absent"), these advanced AI models perform multi-class classification. They are trained not just to see a lesion, but to predict which of several possible diagnostic categories it belongs to.

2. Interpreting Outputs from Computer Vision Models

The most common way AI assists with differential diagnosis is by analyzing clinical photographs or radiographs. Let's break down what their outputs look like and, more importantly, how to interpret them with a critical clinical eye.

What Do the Outputs Look Like?

An AI model's "suggestion" for a differential diagnosis typically comes in one of a few formats:

1. A Categorized List: The AI classifies the lesion into a predefined category.
2. A Ranked List with Probabilities: The AI provides a list of possible diagnoses, each with a corresponding probability score.

Here is an example of what a user interface for such an AI might look like on a smartphone.

This output is a starting point for your differential diagnosis. The AI is suggesting "Lichen" but also implicitly offering the other categories as possibilities. A more detailed output would rank these possibilities with scores, for example:

• Lichen Planus: 75%
• Precancerous (Leukoplakia): 15%
• Benign Frictional Keratosis: 8%
• Malignant (OSCC): 2%

How to Critically Interpret the Outputs

A list of probabilities can be powerful, but it can also be misleading if not interpreted correctly. Simply trusting the top result is not enough. You need to understand why and how confidently the AI made its suggestion.

Based on this reading, let's establish a framework for interpreting these outputs:

1. Scrutinize the Probability Score:
As the paper notes, a probability score is not the same as the model's confidence. A model might assign an 80% probability to a diagnosis simply because it's the "least bad" option among the classes it knows, even if the model is very uncertain overall.

2. Demand an Explanation (Explainability):
How can you trust the AI's reasoning? You need to see its evidence.

• Saliency Maps (Heatmaps): These are one of the most important explainability tools. A heatmap visually overlays the input image, highlighting the pixels the AI considered most important for its decision.
  • Your Action: If the AI suggests "OSCC" but the heatmap is focused on a dental restoration or a tongue papilla instead of the lesion itself, you should be highly skeptical of its suggestion. The AI may have been distracted by an irrelevant feature.

3. Ask "How Sure Are You?" (Uncertainty Quantification):
Some advanced models can provide a separate uncertainty score. This is more direct than trying to infer confidence from probability.

• Low Uncertainty: The model is confident in its prediction. The features it saw strongly match what it learned during training for that specific diagnosis.
• High Uncertainty: The model is "confused." The lesion may have ambiguous features, look like multiple conditions at once, or be something the AI has never seen before.
  • Your Action: A high uncertainty score is a red flag. It's a direct message from the AI to you, the clinician, that this case requires extra scrutiny, further investigation, and perhaps referral.

The paper's discussion reinforces a key clinical point: the utility of an AI tool depends entirely on whether it's trained to answer a meaningful question. A simple "cancer vs. normal" classifier is far less useful than a multi-class system that can differentiate between OSCC, OPMDs, and common benign look-alikes, which is the essence of building a differential diagnosis.

3. Interpreting Outputs from Large Language Models (LLMs)

A different class of AI, Large Language Models (LLMs) like ChatGPT, can also generate differential diagnoses. Instead of analyzing images, they analyze textual information—your written description of a lesion, along with the patient's history.

From this study, we can establish a similar interpretation framework for LLMs:

1. Acknowledge the Accuracy Limits:
The study found an overall accuracy of 87.4% for ChatGPT-4.0, which dropped to 78.4% for difficult questions. This is a crucial takeaway: LLMs are not infallible. Their suggestions must be verified.

2. Address the "Hallucination" Problem:
LLMs can sometimes generate plausible but incorrect information, a phenomenon known as "hallucination." How can we ensure the information is grounded in medical fact? This is where a technology called Retrieval-Augmented Generation (RAG) comes in.

RAG transforms an LLM from a creative storyteller into a research assistant. By forcing the model to cite its sources from a trusted database (e.g., medical literature), it dramatically increases the reliability of its diagnostic suggestions. When interpreting an LLM's output, look for citations or references that back up its claims.

3. Respect the Inherent Limitations:
The research paper on ChatGPT is very clear about the model's limitations, which are fundamental to your interpretation:

• No Physical Exam: An LLM cannot palpate a lesion to assess its texture or firmness.
• No Visual Data (Usually): Unless it's a multimodal model (like GPT-4V), it cannot see the lesion's color, shape, or surface characteristics.
• No Follow-up Questions: The AI cannot ask for clarification as a human clinician would.

Your Action: The output from an LLM is based only on the text you provide. If your description is incomplete or ambiguous, the AI's suggestions will be flawed. The responsibility for providing high-quality input rests entirely on you.

Conclusion

AI is emerging as a valuable tool for building a differential diagnosis for oral lesions. It can process vast amounts of data to find patterns and suggest possibilities that might be missed, especially by a non-specialist.

Key Takeaways:

• How AI Suggests a Differential Diagnosis:
  • Computer Vision Models: Use multi-class classification on images to produce a ranked list of diagnoses with probability scores.
  • Large Language Models: Analyze text-based clinical scenarios to generate a list of possible diagnoses.
• How to Interpret the Outputs:
  • Look beyond the top suggestion. Critically assess probability scores, but do not equate them with confidence.
  • Use explainability tools like saliency maps to verify the AI's visual reasoning.
  • Pay attention to uncertainty scores as a direct measure of the model's confidence.
  • For LLMs, verify claims using technologies like RAG that ground answers in reliable sources.
• The Clinician is the Final Authority: AI outputs are powerful suggestions, but they are not final diagnoses. They must be integrated with your own clinical examination, patient history, and professional judgment.

Preview of the Next Lesson

We've now seen how AI can help formulate a differential diagnosis. In our next lesson, we will focus on one of the most critical applications of this technology: AI applications in oral cancer screening using clinical photographs. We will delve into how these tools are being deployed specifically to tackle the challenge of early cancer detection.