Hello! Welcome to the fifth lesson in our module on AI in radiographic diagnosis.
In our last lesson, we saw how AI can assess restorations and detect recurrent decay. We even looked at a study where an AI model's performance in detecting overhangs was compared directly to that of dentists with varying experience levels.
Today, we'll expand on that theme. Our learning outcome is to compare AI diagnostic performance to clinician performance based on published studies. This is a vital skill for you as a future clinician. When a company or a research paper claims an AI is "as good as a dentist," you need the tools to understand what that claim really means and whether it's supported by evidence.
We will explore:
- The fundamental reason why comparing AI to clinicians is so important.
- How to interpret the key metrics used in these comparisons.
- What the evidence says for two major diagnostic areas: caries and periodontal disease.
Why Compare AI to Clinicians? The Challenge of Diagnostic Variability
Before we can ask if an AI is better than a dentist, we have to ask: which dentist? It's well-documented that diagnostic opinions in dentistry can vary significantly from one clinician to another. This isn't a reflection of incompetence, but a result of the inherent challenges in interpreting complex, low-contrast images in a busy clinical environment.
Eradicating the Dental Divide with AI: new era of preventive care | Florian Hillen | TEDxBoston
To understand the scale of this challenge, let's watch a short clip from a TEDx talk by Florian Hillen, the founder of the dental AI company VideaHealth. He provides an excellent summary of why diagnostic variability exists in dentistry.
Please watch the segment from 02:13 to 04:35. Pay attention to the three reasons he gives for why even competent dentists can arrive at different conclusions.
As the speaker highlights, factors like the limitations of radiographs, working in isolation, and constant context-switching contribute to diagnostic variability. This is the core problem that AI aims to address. The goal isn't necessarily to "beat" the best expert on their best day, but to provide a consistent, objective, and tireless "second opinion" that can help reduce variability and raise the standard of care for all clinicians.
The Language of Comparison: A Refresher on Sensitivity and Specificity
To compare performance, we need a common language. In diagnostic testing, that language is built around two key metrics: sensitivity and specificity. You've encountered these concepts before, but it's crucial to have them clearly defined as we dive into research papers.
Sensitivity and Specificity simplified
This short animated video from the 'Let's Learn Public Health' channel provides a very clear and simple explanation of sensitivity and specificity.
Please watch from 00:36 to 02:55. This will define true positives, true negatives, false positives, and false negatives, and then use them to define sensitivity and specificity.
To put it in a dental context:
- Sensitivity: The ability of a test to correctly identify those with the disease. In caries detection, a high-sensitivity AI is good at finding decay. It answers the question: "Of all the teeth that actually have caries, what proportion did the AI correctly identify?" A high sensitivity means a low rate of false negatives (missed decay).
- Specificity: The ability of a test to correctly identify those without the disease. A high-specificity AI is good at confirming a tooth is healthy. It answers: "Of all the sound teeth, what proportion did the AI correctly label as healthy?" A high specificity means a low rate of false positives (incorrectly flagging a healthy tooth, which could lead to unnecessary treatment).
Now, let's use these tools to analyze what the scientific literature says.
Case Study 1: AI vs. Clinicians in Caries Detection
Detecting proximal caries on bitewings is a classic diagnostic challenge. How does AI stack up? We'll look at a systematic review and meta-analysis—one of the highest levels of evidence, as it synthesizes results from multiple studies.
We will now examine the paper 'Diagnostic performance of artificial intelligence-aided caries detection on bitewing radiographs' to see a high-level summary of the evidence.
Please read section 3.4, 'Comparison to dentists,' a very short but direct summary. Then, read the third paragraph of the 'Discussion' section (starts with 'The meta-analysis for overall caries detection...'). It compares the AI's performance numbers to the typical performance of conventional radiography.
The paper delivers a clear message:
- Direct Comparison: In the three studies that directly compared AI to dentists, the AI models consistently showed higher sensitivity. They were better at not missing caries.
- Performance Numbers: The meta-analysis found a summary sensitivity for AI of 0.87 (87%). The paper contrasts this with conventional radiographic diagnosis by a human, which typically has a sensitivity of 0.40 – 0.50 (40-50%).
This is a significant difference. It suggests that AI's primary benefit in caries detection is to drastically reduce the number of false negatives, or missed carious lesions.
Let's look at a visual representation of the data from that paper.
While AI's higher sensitivity is promising, the discussion in the paper rightly points out that many of these studies have a high risk of bias and haven't been validated on external data. This means the real-world performance might be lower than reported.
Case Study 2: AI vs. Clinicians in Periodontal Disease
Now let's turn to another critical area: assessing periodontal disease by measuring alveolar bone loss on radiographs.
A systematic review published in 2024 compiled the evidence on AI for diagnosing periodontal diseases. Its findings provide a nuanced view of the AI-clinician comparison.
The Application and Performance of Artificial Intelligence (AI ... - PMC
We'll now look at 'The Application and Performance of Artificial Intelligence (AI) ... in Periodontology'. This paper contains a wealth of comparative data. We'll focus on the section about assessing bone loss.
Please look at Table 2c, 'Application of AI to evaluate radiographic alveolar bone level...'. You don't need to read the whole table, but find these two specific studies within it: Krois J et al. (2019): Look at the 'Evaluation Accuracy...' column for a direct comparison between the CNN and dentists. Kim J et al. (2019): Compare the F1 scores between the AI and clinicians. Finally, read the third paragraph of the discussion in section 4.3 (starts with 'Kim SH et al. compared five clinicians with AI...'). It summarizes a study where AI outperformed clinicians.
From this review, we can draw several important conclusions:
-
AI often shows comparable or superior performance. The study by Kim J et al. (2019) found their AI model (DeNTNet) achieved an F1-score (a combined measure of precision and sensitivity) of 0.75, while experienced dental hygienists averaged 0.69.
-
The performance trade-off is key. The study by Krois J et al. (2019) is particularly insightful.
- Dentist accuracy: 76%
- AI accuracy: 81%
While the AI had higher overall accuracy, the dentists had higher sensitivity (92% vs 81%), but the AI had much higher specificity (81% vs 63%). This suggests the dentists were excellent at finding disease but also produced more false positives. The AI was more balanced.
-
AI can outperform general practitioners. The review notes another study (Jiang L et al., 2022) where the AI model's classification of bone loss was more accurate than that of general practitioners.
Again, the theme is not about replacing experts, but about providing a powerful assistive tool that can enhance accuracy and consistency, especially for less experienced clinicians or in busy general practice settings.
Conclusion
So, how does AI performance stack up against clinicians? The evidence from published studies presents a consistent and compelling picture.
Key Takeaways:
- AI as a tool to reduce variability: The primary driver for using AI in diagnostics is to provide a consistent, objective analysis that counteracts the known variability in human interpretation.
- AI excels in sensitivity: Across both caries and periodontal diagnosis, studies consistently show that AI models tend to have higher sensitivity than human clinicians. This means they are powerful tools for reducing false negatives and not missing disease.
- Performance is often comparable or better: In many studies, especially when compared to general practitioners or less experienced clinicians, AI models demonstrate diagnostic accuracy that is similar or even superior.
- Context is everything: AI's performance is not absolute. An AI might have higher overall accuracy but lower sensitivity than a human (or vice-versa). Understanding this trade-off between sensitivity and specificity is crucial for deciding how to use an AI tool in a clinical setting.
Preview of the Next Lesson
We've seen that AI demonstrates impressive performance, but it is not infallible. These systems have specific weaknesses and limitations that every clinical user must understand to use them safely and effectively. In our next lesson, we will address the learning outcome: Identify common limitations of AI in radiographic diagnosis, such as detecting early lesions or artifacts.