Limits...
Meta-analysis and meta-modelling for diagnostic problems.

Charoensawat S, Böhning W, Böhning D, Holling H - BMC Med Res Methodol (2014)

Bottom Line: A study-specific model is estimated for each study, and the resulting study-specific estimate of diagnostic accuracy is taken as an outcome measure for a mixed model with a random study effect and other study-level covariates as fixed effects.In particular, variation in cut-off value is discussed together with recommendations on choosing the best cut-off value.We also show how this problem can be addressed with the proposed methodology.

View Article: PubMed Central - HTML - PubMed

Affiliation: Southampton Statistical Sciences Research Institute, Mathematics and Medical Statistics, University of Southampton, Southampton SO17 1BJ, UK. d.a.bohning@soton.ac.uk.

ABSTRACT

Background: A proportional hazards measure is suggested in the context of analyzing SROC curves that arise in the meta-analysis of diagnostic studies. The measure can be motivated as a special model: the Lehmann model for ROC curves. The Lehmann model involves study-specific sensitivities and specificities and a diagnostic accuracy parameter which connects the two.

Methods: A study-specific model is estimated for each study, and the resulting study-specific estimate of diagnostic accuracy is taken as an outcome measure for a mixed model with a random study effect and other study-level covariates as fixed effects. The variance component model becomes estimable by deriving within-study variances, depending on the outcome measure of choice. In contrast to existing approaches - usually of bivariate nature for the outcome measures - the suggested approach is univariate and, hence, allows easily the application of conventional mixed modelling.

Results: Some simple modifications in the SAS procedure proc mixed allow the fitting of mixed models for meta-analytic data from diagnostic studies. The methodology is illustrated with several meta-analytic diagnostic data sets, including a meta-analysis of the Mini-Mental State Examination as a diagnostic device for dementia and mild cognitive impairment.

Conclusions: The proposed methodology allows us to embed the meta-analysis of diagnostic studies into the well-developed area of mixed modelling. Different outcome measures, specifically from the perspective of whether a local or a global measure of diagnostic accuracy should be applied, are discussed as well. In particular, variation in cut-off value is discussed together with recommendations on choosing the best cut-off value. We also show how this problem can be addressed with the proposed methodology.

Show MeSH

Related in: MedlinePlus

Index plots for the PH measures for the data of the meta-analysis of BNP and heart failure.
© Copyright Policy - open-access
Related In: Results  -  Collection

License 1 - License 2
getmorefigures.php?uid=PMC4007022&req=5

Figure 3: Index plots for the PH measures for the data of the meta-analysis of BNP and heart failure.

Mentions: We suggest using the measure , which relates the log-sensitivity to the log-false positive rate; we call it the proportional hazards (PH) measure. In Figure 3 we see that this measure shows a reduced variability for the meta-analysis of BNP and heart failure, making it more suitable as an overall measure in the meta-analysis of diagnostic studies or diagnostic problems. While the measure appears to be like any other summary measure of the pair sensitivity and specificity, it has a specific SROC-modelling background and motivation. We have mentioned previously the cut-off value problem: observed heterogeneity might be induced by cut-off value variation which could lead to different sensitivities and specificities – despite the accuracy of the diagnostic test itself not having changed – and might also lead to an induced heterogeneity in the summary measure. Hence, it is unclear whether the observed heterogeneity is due to heterogeneity in the diagnostic accuracy (authentic heterogeneity) or whether it has occurred due to cut-off value variation (artificial heterogeneity). This second form of heterogeneity can also occur when the background population changes with the study.


Meta-analysis and meta-modelling for diagnostic problems.

Charoensawat S, Böhning W, Böhning D, Holling H - BMC Med Res Methodol (2014)

Index plots for the PH measures for the data of the meta-analysis of BNP and heart failure.
© Copyright Policy - open-access
Related In: Results  -  Collection

License 1 - License 2
Show All Figures
getmorefigures.php?uid=PMC4007022&req=5

Figure 3: Index plots for the PH measures for the data of the meta-analysis of BNP and heart failure.
Mentions: We suggest using the measure , which relates the log-sensitivity to the log-false positive rate; we call it the proportional hazards (PH) measure. In Figure 3 we see that this measure shows a reduced variability for the meta-analysis of BNP and heart failure, making it more suitable as an overall measure in the meta-analysis of diagnostic studies or diagnostic problems. While the measure appears to be like any other summary measure of the pair sensitivity and specificity, it has a specific SROC-modelling background and motivation. We have mentioned previously the cut-off value problem: observed heterogeneity might be induced by cut-off value variation which could lead to different sensitivities and specificities – despite the accuracy of the diagnostic test itself not having changed – and might also lead to an induced heterogeneity in the summary measure. Hence, it is unclear whether the observed heterogeneity is due to heterogeneity in the diagnostic accuracy (authentic heterogeneity) or whether it has occurred due to cut-off value variation (artificial heterogeneity). This second form of heterogeneity can also occur when the background population changes with the study.

Bottom Line: A study-specific model is estimated for each study, and the resulting study-specific estimate of diagnostic accuracy is taken as an outcome measure for a mixed model with a random study effect and other study-level covariates as fixed effects.In particular, variation in cut-off value is discussed together with recommendations on choosing the best cut-off value.We also show how this problem can be addressed with the proposed methodology.

View Article: PubMed Central - HTML - PubMed

Affiliation: Southampton Statistical Sciences Research Institute, Mathematics and Medical Statistics, University of Southampton, Southampton SO17 1BJ, UK. d.a.bohning@soton.ac.uk.

ABSTRACT

Background: A proportional hazards measure is suggested in the context of analyzing SROC curves that arise in the meta-analysis of diagnostic studies. The measure can be motivated as a special model: the Lehmann model for ROC curves. The Lehmann model involves study-specific sensitivities and specificities and a diagnostic accuracy parameter which connects the two.

Methods: A study-specific model is estimated for each study, and the resulting study-specific estimate of diagnostic accuracy is taken as an outcome measure for a mixed model with a random study effect and other study-level covariates as fixed effects. The variance component model becomes estimable by deriving within-study variances, depending on the outcome measure of choice. In contrast to existing approaches - usually of bivariate nature for the outcome measures - the suggested approach is univariate and, hence, allows easily the application of conventional mixed modelling.

Results: Some simple modifications in the SAS procedure proc mixed allow the fitting of mixed models for meta-analytic data from diagnostic studies. The methodology is illustrated with several meta-analytic diagnostic data sets, including a meta-analysis of the Mini-Mental State Examination as a diagnostic device for dementia and mild cognitive impairment.

Conclusions: The proposed methodology allows us to embed the meta-analysis of diagnostic studies into the well-developed area of mixed modelling. Different outcome measures, specifically from the perspective of whether a local or a global measure of diagnostic accuracy should be applied, are discussed as well. In particular, variation in cut-off value is discussed together with recommendations on choosing the best cut-off value. We also show how this problem can be addressed with the proposed methodology.

Show MeSH
Related in: MedlinePlus