Limits...
Testing the accuracy of an observation-based classifier for rapid detection of autism risk.

Duda M, Kosmicki JA, Wall DP - Transl Psychiatry (2014)

Bottom Line: We tested OBC outcomes against the outcomes provided by the original and current ADOS algorithms, the best estimate clinical diagnosis, and the comparison score severity metric associated with ADOS-2.The OBC was significantly correlated with the ADOS-G (r=-0.814) and ADOS-2 (r=-0.779) and exhibited >97% sensitivity and >77% specificity in comparison to both ADOS algorithm scores.The correlation between the OBC score and the comparison score was significant (r=-0.628), suggesting that the OBC provides both a classification as well as a measure of severity of the phenotype.

View Article: PubMed Central - PubMed

Affiliation: Division of Systems Medicine, Department of Pediatrics, Stanford University, Stanford, CA, USA.

ABSTRACT
Current approaches for diagnosing autism have high diagnostic validity but are time consuming and can contribute to delays in arriving at an official diagnosis. In a pilot study, we used machine learning to derive a classifier that represented a 72% reduction in length from the gold-standard Autism Diagnostic Observation Schedule-Generic (ADOS-G), while retaining >97% statistical accuracy. The pilot study focused on a relatively small sample of children with and without autism. The present study sought to further test the accuracy of the classifier (termed the observation-based classifier (OBC)) on an independent sample of 2616 children scored using ADOS from five data repositories and including both spectrum (n=2333) and non-spectrum (n=283) individuals. We tested OBC outcomes against the outcomes provided by the original and current ADOS algorithms, the best estimate clinical diagnosis, and the comparison score severity metric associated with ADOS-2. The OBC was significantly correlated with the ADOS-G (r=-0.814) and ADOS-2 (r=-0.779) and exhibited >97% sensitivity and >77% specificity in comparison to both ADOS algorithm scores. The correspondence to the best estimate clinical diagnosis was also high (accuracy=96.8%), with sensitivity of 97.1% and specificity of 83.3%. The correlation between the OBC score and the comparison score was significant (r=-0.628), suggesting that the OBC provides both a classification as well as a measure of severity of the phenotype. These results further demonstrate the accuracy of the OBC and suggest that reductions in the process of detecting and monitoring autism are possible.

Show MeSH

Related in: MedlinePlus

Distribution of the observation-based classifier (OBC) scores in our sample (n=2616), colored by the ADOS-2 comparison score (CS), a proxy for measuring autism symptom severity. A majority (86.3%) of our sample was classified in the moderate (5⩽CS⩽7) to severe (8⩽CS⩽10) range, and the OBC and CS scores were found to be significantly correlated (r=−0.628).
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4150240&req=5

fig2: Distribution of the observation-based classifier (OBC) scores in our sample (n=2616), colored by the ADOS-2 comparison score (CS), a proxy for measuring autism symptom severity. A majority (86.3%) of our sample was classified in the moderate (5⩽CS⩽7) to severe (8⩽CS⩽10) range, and the OBC and CS scores were found to be significantly correlated (r=−0.628).

Mentions: Figure 2 shows the distribution of autism phenotype severity scores versus OBC scores; a large percentage of our sample consisted of children with classic autism, possibly due to the inclusion criteria of the research collections used in our study. The correspondence between OBC score and the comparison score is significant (r=−0.628), indicating that the OBC score not only reflects confidence in the classification, but may also reflect the severity of the autism phenotype.


Testing the accuracy of an observation-based classifier for rapid detection of autism risk.

Duda M, Kosmicki JA, Wall DP - Transl Psychiatry (2014)

Distribution of the observation-based classifier (OBC) scores in our sample (n=2616), colored by the ADOS-2 comparison score (CS), a proxy for measuring autism symptom severity. A majority (86.3%) of our sample was classified in the moderate (5⩽CS⩽7) to severe (8⩽CS⩽10) range, and the OBC and CS scores were found to be significantly correlated (r=−0.628).
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4150240&req=5

fig2: Distribution of the observation-based classifier (OBC) scores in our sample (n=2616), colored by the ADOS-2 comparison score (CS), a proxy for measuring autism symptom severity. A majority (86.3%) of our sample was classified in the moderate (5⩽CS⩽7) to severe (8⩽CS⩽10) range, and the OBC and CS scores were found to be significantly correlated (r=−0.628).
Mentions: Figure 2 shows the distribution of autism phenotype severity scores versus OBC scores; a large percentage of our sample consisted of children with classic autism, possibly due to the inclusion criteria of the research collections used in our study. The correspondence between OBC score and the comparison score is significant (r=−0.628), indicating that the OBC score not only reflects confidence in the classification, but may also reflect the severity of the autism phenotype.

Bottom Line: We tested OBC outcomes against the outcomes provided by the original and current ADOS algorithms, the best estimate clinical diagnosis, and the comparison score severity metric associated with ADOS-2.The OBC was significantly correlated with the ADOS-G (r=-0.814) and ADOS-2 (r=-0.779) and exhibited >97% sensitivity and >77% specificity in comparison to both ADOS algorithm scores.The correlation between the OBC score and the comparison score was significant (r=-0.628), suggesting that the OBC provides both a classification as well as a measure of severity of the phenotype.

View Article: PubMed Central - PubMed

Affiliation: Division of Systems Medicine, Department of Pediatrics, Stanford University, Stanford, CA, USA.

ABSTRACT
Current approaches for diagnosing autism have high diagnostic validity but are time consuming and can contribute to delays in arriving at an official diagnosis. In a pilot study, we used machine learning to derive a classifier that represented a 72% reduction in length from the gold-standard Autism Diagnostic Observation Schedule-Generic (ADOS-G), while retaining >97% statistical accuracy. The pilot study focused on a relatively small sample of children with and without autism. The present study sought to further test the accuracy of the classifier (termed the observation-based classifier (OBC)) on an independent sample of 2616 children scored using ADOS from five data repositories and including both spectrum (n=2333) and non-spectrum (n=283) individuals. We tested OBC outcomes against the outcomes provided by the original and current ADOS algorithms, the best estimate clinical diagnosis, and the comparison score severity metric associated with ADOS-2. The OBC was significantly correlated with the ADOS-G (r=-0.814) and ADOS-2 (r=-0.779) and exhibited >97% sensitivity and >77% specificity in comparison to both ADOS algorithm scores. The correspondence to the best estimate clinical diagnosis was also high (accuracy=96.8%), with sensitivity of 97.1% and specificity of 83.3%. The correlation between the OBC score and the comparison score was significant (r=-0.628), suggesting that the OBC provides both a classification as well as a measure of severity of the phenotype. These results further demonstrate the accuracy of the OBC and suggest that reductions in the process of detecting and monitoring autism are possible.

Show MeSH
Related in: MedlinePlus