Limits...
A Meta-Analysis of Typhoid Diagnostic Accuracy Studies: A Recommendation to Adopt a Standardized Composite Reference.

Storey HL, Huang Y, Crudder C, Golden A, de los Santos T, Hawkins K - PLoS ONE (2015)

Bottom Line: Blood culture is most often used as the reference method to evaluate the accuracy of new typhoid tests; however, it is recognized to be an imperfect gold standard.In order to evaluate existing tests for use as a reference test or inclusion in a CRS, we performed a systematic review of the typhoid literature to include all index/reference test combinations observed.We described the landscape of comparisons performed, showed results of a meta-analysis on the accuracy of the more common combinations, and evaluated sources of variability based on study quality.

View Article: PubMed Central - PubMed

Affiliation: Diagnostics Program, PATH, Seattle, Washington, United States of America.

ABSTRACT
Novel typhoid diagnostics currently under development have the potential to improve clinical care, surveillance, and the disease burden estimates that support vaccine introduction. Blood culture is most often used as the reference method to evaluate the accuracy of new typhoid tests; however, it is recognized to be an imperfect gold standard. If no single gold standard test exists, use of a composite reference standard (CRS) can improve estimation of diagnostic accuracy. Numerous studies have used a CRS to evaluate new typhoid diagnostics; however, there is no consensus on an appropriate CRS. In order to evaluate existing tests for use as a reference test or inclusion in a CRS, we performed a systematic review of the typhoid literature to include all index/reference test combinations observed. We described the landscape of comparisons performed, showed results of a meta-analysis on the accuracy of the more common combinations, and evaluated sources of variability based on study quality. This wide-ranging meta-analysis suggests that no single test has sufficiently good performance but some existing diagnostics may be useful as part of a CRS. Additionally, based on findings from the meta-analysis and a constructed numerical example demonstrating the use of CRS, we proposed necessary criteria and potential components of a typhoid CRS to guide future recommendations. Agreement and adoption by all investigators of a standardized CRS is requisite, and would improve comparison of new diagnostics across independent studies, leading to the identification of a better reference test and improved confidence in prevalence estimates.

Show MeSH

Related in: MedlinePlus

Meta-analysis results.Graphical illustration of sensitivities (y-axis) and specificities (x-axis) corresponding to comparisons included in the meta-analysis: PCR-based assays (A), anti-LPS assays (B), TUBEX® assays (C), anti-S. typhi assays (D), Typhidot assays (E), Widal assays (F). Meta-analysis was performed using bivariate random effects binomial regression (STATA command: metandi). Sizes of individual study estimates (grey circle) represent sample size. Summary point (red square), hierarchical summary receiver operating characteristic curves (green line), 95% confidence regions (yellow dashed line), and 95% prediction regions (grey dashed line) are depicted.
© Copyright Policy
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4643909&req=5

pone.0142364.g003: Meta-analysis results.Graphical illustration of sensitivities (y-axis) and specificities (x-axis) corresponding to comparisons included in the meta-analysis: PCR-based assays (A), anti-LPS assays (B), TUBEX® assays (C), anti-S. typhi assays (D), Typhidot assays (E), Widal assays (F). Meta-analysis was performed using bivariate random effects binomial regression (STATA command: metandi). Sizes of individual study estimates (grey circle) represent sample size. Summary point (red square), hierarchical summary receiver operating characteristic curves (green line), 95% confidence regions (yellow dashed line), and 95% prediction regions (grey dashed line) are depicted.

Mentions: A graphical illustration of sensitivities (y-axis) and specificities (x-axis) corresponding to the above comparisons included in the meta-analysis are presented in Fig 3, including the 95% confidence and prediction regions, and the hierarchical summary receiver operating characteristic curves. Additionally, forest plots of the meta-analysis results are provided in supplementary material (S2 Text). When assessing whether study quality had an effect on observed diagnostic accuracy, studies with patient selection bias had a significantly different specificity compared to studies without this bias for PCR, TUBEX, and anti-S. typhi assays, though the direction of the effect was not consistent across assays. Patient flow bias significantly affected specificity for TUBEX, anti-S. typhi, and Widal assays and sensitivity for Typhidot assays. Index test bias significantly affected sensitivity for PCR and specificity for anti-S. typhi assays. Finally, risk of reference test bias was unclear for many studies, based on the defined criteria, hampering interpretation of observed effect on diagnostic accuracy. (Table 2)


A Meta-Analysis of Typhoid Diagnostic Accuracy Studies: A Recommendation to Adopt a Standardized Composite Reference.

Storey HL, Huang Y, Crudder C, Golden A, de los Santos T, Hawkins K - PLoS ONE (2015)

Meta-analysis results.Graphical illustration of sensitivities (y-axis) and specificities (x-axis) corresponding to comparisons included in the meta-analysis: PCR-based assays (A), anti-LPS assays (B), TUBEX® assays (C), anti-S. typhi assays (D), Typhidot assays (E), Widal assays (F). Meta-analysis was performed using bivariate random effects binomial regression (STATA command: metandi). Sizes of individual study estimates (grey circle) represent sample size. Summary point (red square), hierarchical summary receiver operating characteristic curves (green line), 95% confidence regions (yellow dashed line), and 95% prediction regions (grey dashed line) are depicted.
© Copyright Policy
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4643909&req=5

pone.0142364.g003: Meta-analysis results.Graphical illustration of sensitivities (y-axis) and specificities (x-axis) corresponding to comparisons included in the meta-analysis: PCR-based assays (A), anti-LPS assays (B), TUBEX® assays (C), anti-S. typhi assays (D), Typhidot assays (E), Widal assays (F). Meta-analysis was performed using bivariate random effects binomial regression (STATA command: metandi). Sizes of individual study estimates (grey circle) represent sample size. Summary point (red square), hierarchical summary receiver operating characteristic curves (green line), 95% confidence regions (yellow dashed line), and 95% prediction regions (grey dashed line) are depicted.
Mentions: A graphical illustration of sensitivities (y-axis) and specificities (x-axis) corresponding to the above comparisons included in the meta-analysis are presented in Fig 3, including the 95% confidence and prediction regions, and the hierarchical summary receiver operating characteristic curves. Additionally, forest plots of the meta-analysis results are provided in supplementary material (S2 Text). When assessing whether study quality had an effect on observed diagnostic accuracy, studies with patient selection bias had a significantly different specificity compared to studies without this bias for PCR, TUBEX, and anti-S. typhi assays, though the direction of the effect was not consistent across assays. Patient flow bias significantly affected specificity for TUBEX, anti-S. typhi, and Widal assays and sensitivity for Typhidot assays. Index test bias significantly affected sensitivity for PCR and specificity for anti-S. typhi assays. Finally, risk of reference test bias was unclear for many studies, based on the defined criteria, hampering interpretation of observed effect on diagnostic accuracy. (Table 2)

Bottom Line: Blood culture is most often used as the reference method to evaluate the accuracy of new typhoid tests; however, it is recognized to be an imperfect gold standard.In order to evaluate existing tests for use as a reference test or inclusion in a CRS, we performed a systematic review of the typhoid literature to include all index/reference test combinations observed.We described the landscape of comparisons performed, showed results of a meta-analysis on the accuracy of the more common combinations, and evaluated sources of variability based on study quality.

View Article: PubMed Central - PubMed

Affiliation: Diagnostics Program, PATH, Seattle, Washington, United States of America.

ABSTRACT
Novel typhoid diagnostics currently under development have the potential to improve clinical care, surveillance, and the disease burden estimates that support vaccine introduction. Blood culture is most often used as the reference method to evaluate the accuracy of new typhoid tests; however, it is recognized to be an imperfect gold standard. If no single gold standard test exists, use of a composite reference standard (CRS) can improve estimation of diagnostic accuracy. Numerous studies have used a CRS to evaluate new typhoid diagnostics; however, there is no consensus on an appropriate CRS. In order to evaluate existing tests for use as a reference test or inclusion in a CRS, we performed a systematic review of the typhoid literature to include all index/reference test combinations observed. We described the landscape of comparisons performed, showed results of a meta-analysis on the accuracy of the more common combinations, and evaluated sources of variability based on study quality. This wide-ranging meta-analysis suggests that no single test has sufficiently good performance but some existing diagnostics may be useful as part of a CRS. Additionally, based on findings from the meta-analysis and a constructed numerical example demonstrating the use of CRS, we proposed necessary criteria and potential components of a typhoid CRS to guide future recommendations. Agreement and adoption by all investigators of a standardized CRS is requisite, and would improve comparison of new diagnostics across independent studies, leading to the identification of a better reference test and improved confidence in prevalence estimates.

Show MeSH
Related in: MedlinePlus