Limits...
Using the RDP classifier to predict taxonomic novelty and reduce the search space for finding novel organisms.

Lan Y, Wang Q, Cole JR, Rosen GL - PLoS ONE (2012)

Bottom Line: We conclude that selecting a read-length appropriate RDP bootstrap score can significantly reduce the search space for identifying novel genera and higher levels in taxonomy.In addition, having a well-represented database significantly improves performance while having genera that are "highly" similar does not make a significant improvement.On a real dataset from an Amazon Terra Preta soil sample, we show that the detector can predict (or correlates to) whether novel sequences will be assigned to new taxa when the RDP database "doubles" in the future.

View Article: PubMed Central - PubMed

Affiliation: School of Biomedical Engineering, Science, and Health Systems, Drexel University, Philadelphia, Pennsylvania, United States of America.

ABSTRACT

Background: Currently, the naïve Bayesian classifier provided by the Ribosomal Database Project (RDP) is one of the most widely used tools to classify 16S rRNA sequences, mainly collected from environmental samples. We show that RDP has 97+% assignment accuracy and is fast for 250 bp and longer reads when the read originates from a taxon known to the database. Because most environmental samples will contain organisms from taxa whose 16S rRNA genes have not been previously sequenced, we aim to benchmark how well the RDP classifier and other competing methods can discriminate these novel taxa from known taxa.

Principal findings: Because each fragment is assigned a score (containing likelihood or confidence information such as the boostrap score in the RDP classifier), we "train" a threshold to discriminate between novel and known organisms and observe its performance on a test set. The threshold that we determine tends to be conservative (low sensitivity but high specificity) for naïve Bayesian methods. Nonetheless, our method performs better with the RDP classifier than the other methods tested, measured by the f-measure and the area-under-the-curve on the receiver operating characteristic of the test set. By constraining the database to well-represented genera, sensitivity improves 3-15%. Finally, we show that the detector is a good predictor to determine novel abundant taxa (especially for finer levels of taxonomy where novelty is more likely to be present).

Conclusions: We conclude that selecting a read-length appropriate RDP bootstrap score can significantly reduce the search space for identifying novel genera and higher levels in taxonomy. In addition, having a well-represented database significantly improves performance while having genera that are "highly" similar does not make a significant improvement. On a real dataset from an Amazon Terra Preta soil sample, we show that the detector can predict (or correlates to) whether novel sequences will be assigned to new taxa when the RDP database "doubles" in the future.

Show MeSH
Calculation of the correlation between 1) the detector prediction using the “present” and 2) the full, “future” database.The percent change in the taxon bin is correlated to the previous prediction of novelty of the reads in that bin.
© Copyright Policy
Related In: Results  -  Collection


getmorefigures.php?uid=PMC3293824&req=5

pone-0032491-g016: Calculation of the correlation between 1) the detector prediction using the “present” and 2) the full, “future” database.The percent change in the taxon bin is correlated to the previous prediction of novelty of the reads in that bin.

Mentions: RDP obtains the best AUC followed by NBC, PhymmBL, and Phymm. The blue star indicates the threshold determined from the training data. In this case, for NBC, the training data determination of the threshold resulted in the most optimal point for the test set unlike the other methods. This results in NBC's good performance in Fig. 16{9} (bar graph for 100 bp).


Using the RDP classifier to predict taxonomic novelty and reduce the search space for finding novel organisms.

Lan Y, Wang Q, Cole JR, Rosen GL - PLoS ONE (2012)

Calculation of the correlation between 1) the detector prediction using the “present” and 2) the full, “future” database.The percent change in the taxon bin is correlated to the previous prediction of novelty of the reads in that bin.
© Copyright Policy
Related In: Results  -  Collection

Show All Figures
getmorefigures.php?uid=PMC3293824&req=5

pone-0032491-g016: Calculation of the correlation between 1) the detector prediction using the “present” and 2) the full, “future” database.The percent change in the taxon bin is correlated to the previous prediction of novelty of the reads in that bin.
Mentions: RDP obtains the best AUC followed by NBC, PhymmBL, and Phymm. The blue star indicates the threshold determined from the training data. In this case, for NBC, the training data determination of the threshold resulted in the most optimal point for the test set unlike the other methods. This results in NBC's good performance in Fig. 16{9} (bar graph for 100 bp).

Bottom Line: We conclude that selecting a read-length appropriate RDP bootstrap score can significantly reduce the search space for identifying novel genera and higher levels in taxonomy.In addition, having a well-represented database significantly improves performance while having genera that are "highly" similar does not make a significant improvement.On a real dataset from an Amazon Terra Preta soil sample, we show that the detector can predict (or correlates to) whether novel sequences will be assigned to new taxa when the RDP database "doubles" in the future.

View Article: PubMed Central - PubMed

Affiliation: School of Biomedical Engineering, Science, and Health Systems, Drexel University, Philadelphia, Pennsylvania, United States of America.

ABSTRACT

Background: Currently, the naïve Bayesian classifier provided by the Ribosomal Database Project (RDP) is one of the most widely used tools to classify 16S rRNA sequences, mainly collected from environmental samples. We show that RDP has 97+% assignment accuracy and is fast for 250 bp and longer reads when the read originates from a taxon known to the database. Because most environmental samples will contain organisms from taxa whose 16S rRNA genes have not been previously sequenced, we aim to benchmark how well the RDP classifier and other competing methods can discriminate these novel taxa from known taxa.

Principal findings: Because each fragment is assigned a score (containing likelihood or confidence information such as the boostrap score in the RDP classifier), we "train" a threshold to discriminate between novel and known organisms and observe its performance on a test set. The threshold that we determine tends to be conservative (low sensitivity but high specificity) for naïve Bayesian methods. Nonetheless, our method performs better with the RDP classifier than the other methods tested, measured by the f-measure and the area-under-the-curve on the receiver operating characteristic of the test set. By constraining the database to well-represented genera, sensitivity improves 3-15%. Finally, we show that the detector is a good predictor to determine novel abundant taxa (especially for finer levels of taxonomy where novelty is more likely to be present).

Conclusions: We conclude that selecting a read-length appropriate RDP bootstrap score can significantly reduce the search space for identifying novel genera and higher levels in taxonomy. In addition, having a well-represented database significantly improves performance while having genera that are "highly" similar does not make a significant improvement. On a real dataset from an Amazon Terra Preta soil sample, we show that the detector can predict (or correlates to) whether novel sequences will be assigned to new taxa when the RDP database "doubles" in the future.

Show MeSH