Limits...
Accuracy of dementia diagnosis: a direct comparison between radiologists and a computerized method.

Klöppel S, Stonnington CM, Barnes J, Chen F, Chu C, Good CD, Mader I, Mitchell LA, Patel AC, Roberts CC, Fox NC, Jack CR, Ashburner J, Frackowiak RS - Brain (2008)

Bottom Line: In this study, we compare the results to those obtained by radiologists.The results show that well-trained neuroradiologists classify typical Alzheimer's disease-associated scans comparable to SVMs. However, SVMs require no expert knowledge and trained SVMs can readily be exchanged between centres for use in diagnostic classification.These results are encouraging and indicate a role for computerized diagnostic methods in clinical practice.

View Article: PubMed Central - PubMed

Affiliation: Department of Psychiatry and Psychotherapy and Freiburg Brain Imaging, University Clinic Freiburg, Freiburg, Germany. stefan.kloeppel@uniklinik-freiburg.de

ABSTRACT
There has been recent interest in the application of machine learning techniques to neuroimaging-based diagnosis. These methods promise fully automated, standard PC-based clinical decisions, unbiased by variable radiological expertise. We recently used support vector machines (SVMs) to separate sporadic Alzheimer's disease from normal ageing and from fronto-temporal lobar degeneration (FTLD). In this study, we compare the results to those obtained by radiologists. A binary diagnostic classification was made by six radiologists with different levels of experience on the same scans and information that had been previously analysed with SVM. SVMs correctly classified 95% (sensitivity/specificity: 95/95) of sporadic Alzheimer's disease and controls into their respective groups. Radiologists correctly classified 65-95% (median 89%; sensitivity/specificity: 88/90) of scans. SVM correctly classified another set of sporadic Alzheimer's disease in 93% (sensitivity/specificity: 100/86) of cases, whereas radiologists ranged between 80% and 90% (median 83%; sensitivity/specificity: 80/85). SVMs were better at separating patients with sporadic Alzheimer's disease from those with FTLD (SVM 89%; sensitivity/specificity: 83/95; compared to radiological range from 63% to 83%; median 71%; sensitivity/specificity: 64/76). Radiologists were always accurate when they reported a high degree of diagnostic confidence. The results show that well-trained neuroradiologists classify typical Alzheimer's disease-associated scans comparable to SVMs. However, SVMs require no expert knowledge and trained SVMs can readily be exchanged between centres for use in diagnostic classification. These results are encouraging and indicate a role for computerized diagnostic methods in clinical practice.

Show MeSH

Related in: MedlinePlus

Illustration of performance. Positions are jittered to indicate overlap. Grey error bars display 95% confidence intervals for SVM accuracy. The fourth column illustrates the performance when sets 1 and 3 are combined. Note the shrinking CIs. sAD = sporadic Alzheimer's Disease.
© Copyright Policy - creative-commons
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC2577804&req=5

Figure 1: Illustration of performance. Positions are jittered to indicate overlap. Grey error bars display 95% confidence intervals for SVM accuracy. The fourth column illustrates the performance when sets 1 and 3 are combined. Note the shrinking CIs. sAD = sporadic Alzheimer's Disease.

Mentions: One radiologist separated the first set of sporadic Alzheimer's disease cases from cognitively normal subjects as accurately as the SVM; otherwise, the SVM performed better than radiologists (see Table 2 and Fig. 1). Radiological diagnostic accuracy increased substantially (reaching 100%) when high diagnostic confidence was expressed. To evaluate the effect of experience, defined as the percentage of brain scans in their daily workload, we correlated this figure with diagnostic accuracy. A significant correlation indicated that classification accuracy improved with the level of experience (Fig. 2) for sporadic Alzheimer's disease cases in set 1 (Spearman's r = 0.90; P = 0.007, one-tailed) and when sporadic Alzheimer's disease had to be separated from FTLD (r = 0.77; P = 0.036). No such correlation was found for the third dataset. The time required to classify all three datasets ranged from 70 min to 510 min (median = 198).Fig. 1


Accuracy of dementia diagnosis: a direct comparison between radiologists and a computerized method.

Klöppel S, Stonnington CM, Barnes J, Chen F, Chu C, Good CD, Mader I, Mitchell LA, Patel AC, Roberts CC, Fox NC, Jack CR, Ashburner J, Frackowiak RS - Brain (2008)

Illustration of performance. Positions are jittered to indicate overlap. Grey error bars display 95% confidence intervals for SVM accuracy. The fourth column illustrates the performance when sets 1 and 3 are combined. Note the shrinking CIs. sAD = sporadic Alzheimer's Disease.
© Copyright Policy - creative-commons
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC2577804&req=5

Figure 1: Illustration of performance. Positions are jittered to indicate overlap. Grey error bars display 95% confidence intervals for SVM accuracy. The fourth column illustrates the performance when sets 1 and 3 are combined. Note the shrinking CIs. sAD = sporadic Alzheimer's Disease.
Mentions: One radiologist separated the first set of sporadic Alzheimer's disease cases from cognitively normal subjects as accurately as the SVM; otherwise, the SVM performed better than radiologists (see Table 2 and Fig. 1). Radiological diagnostic accuracy increased substantially (reaching 100%) when high diagnostic confidence was expressed. To evaluate the effect of experience, defined as the percentage of brain scans in their daily workload, we correlated this figure with diagnostic accuracy. A significant correlation indicated that classification accuracy improved with the level of experience (Fig. 2) for sporadic Alzheimer's disease cases in set 1 (Spearman's r = 0.90; P = 0.007, one-tailed) and when sporadic Alzheimer's disease had to be separated from FTLD (r = 0.77; P = 0.036). No such correlation was found for the third dataset. The time required to classify all three datasets ranged from 70 min to 510 min (median = 198).Fig. 1

Bottom Line: In this study, we compare the results to those obtained by radiologists.The results show that well-trained neuroradiologists classify typical Alzheimer's disease-associated scans comparable to SVMs. However, SVMs require no expert knowledge and trained SVMs can readily be exchanged between centres for use in diagnostic classification.These results are encouraging and indicate a role for computerized diagnostic methods in clinical practice.

View Article: PubMed Central - PubMed

Affiliation: Department of Psychiatry and Psychotherapy and Freiburg Brain Imaging, University Clinic Freiburg, Freiburg, Germany. stefan.kloeppel@uniklinik-freiburg.de

ABSTRACT
There has been recent interest in the application of machine learning techniques to neuroimaging-based diagnosis. These methods promise fully automated, standard PC-based clinical decisions, unbiased by variable radiological expertise. We recently used support vector machines (SVMs) to separate sporadic Alzheimer's disease from normal ageing and from fronto-temporal lobar degeneration (FTLD). In this study, we compare the results to those obtained by radiologists. A binary diagnostic classification was made by six radiologists with different levels of experience on the same scans and information that had been previously analysed with SVM. SVMs correctly classified 95% (sensitivity/specificity: 95/95) of sporadic Alzheimer's disease and controls into their respective groups. Radiologists correctly classified 65-95% (median 89%; sensitivity/specificity: 88/90) of scans. SVM correctly classified another set of sporadic Alzheimer's disease in 93% (sensitivity/specificity: 100/86) of cases, whereas radiologists ranged between 80% and 90% (median 83%; sensitivity/specificity: 80/85). SVMs were better at separating patients with sporadic Alzheimer's disease from those with FTLD (SVM 89%; sensitivity/specificity: 83/95; compared to radiological range from 63% to 83%; median 71%; sensitivity/specificity: 64/76). Radiologists were always accurate when they reported a high degree of diagnostic confidence. The results show that well-trained neuroradiologists classify typical Alzheimer's disease-associated scans comparable to SVMs. However, SVMs require no expert knowledge and trained SVMs can readily be exchanged between centres for use in diagnostic classification. These results are encouraging and indicate a role for computerized diagnostic methods in clinical practice.

Show MeSH
Related in: MedlinePlus