Limits...
How confidence intervals become confusion intervals.

McCormack J, Vandermeer B, Allan GM - BMC Med Res Methodol (2013)

Bottom Line: The dogmatic adherence to statistical significant thresholds can lead authors to write dichotomized absolute conclusions while ignoring the broader interpretations of very consistent findings.These opposing conclusions are false disagreements that create unnecessary clinical uncertainty.We provide helpful recommendations in approaching conflicting conclusions when they are associated with remarkably similar results.

View Article: PubMed Central - HTML - PubMed

Affiliation: Evidence-Based Medicine, Department of Family Medicine, University of Alberta, Room 1706 College Plaza, 8215 - 112 Street NW, Edmonton AB, Canada. michael.allan@ualberta.ca.

ABSTRACT

Background: Controversies are common in medicine. Some arise when the conclusions of research publications directly contradict each other, creating uncertainty for frontline clinicians.

Discussion: In this paper, we review how researchers can look at very similar data yet have completely different conclusions based purely on an over-reliance of statistical significance and an unclear understanding of confidence intervals. The dogmatic adherence to statistical significant thresholds can lead authors to write dichotomized absolute conclusions while ignoring the broader interpretations of very consistent findings. We describe three examples of controversy around the potential benefit of a medication, a comparison between new medications, and a medication with a potential harm. The examples include the highest levels of evidence, both meta-analyses and randomized controlled trials. We will show how in each case the confidence intervals and point estimates were very similar. The only identifiable differences to account for the contrasting conclusions arise from the serendipitous finding of confidence intervals that either marginally cross or just fail to cross the line of statistical significance.

Summary: These opposing conclusions are false disagreements that create unnecessary clinical uncertainty. We provide helpful recommendations in approaching conflicting conclusions when they are associated with remarkably similar results.

Show MeSH

Related in: MedlinePlus

Comparison of 2 meta-analyses examining peto odds ratio of serious cardiovascular events with varenicline use in smoking cessation.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC3818447&req=5

Figure 3: Comparison of 2 meta-analyses examining peto odds ratio of serious cardiovascular events with varenicline use in smoking cessation.

Mentions: FigureĀ 3 shows the characteristics and serious cardiovascular event peto odds ratios of the two meta-analyses. As with the previous examples, the two results are quite consistent with each other. The only apparent reason for the contradictory conclusions about risk was that the lower limit of the CI fell just below 1.0 in one meta-analysis, and was just above 1.0 in the other.


How confidence intervals become confusion intervals.

McCormack J, Vandermeer B, Allan GM - BMC Med Res Methodol (2013)

Comparison of 2 meta-analyses examining peto odds ratio of serious cardiovascular events with varenicline use in smoking cessation.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC3818447&req=5

Figure 3: Comparison of 2 meta-analyses examining peto odds ratio of serious cardiovascular events with varenicline use in smoking cessation.
Mentions: FigureĀ 3 shows the characteristics and serious cardiovascular event peto odds ratios of the two meta-analyses. As with the previous examples, the two results are quite consistent with each other. The only apparent reason for the contradictory conclusions about risk was that the lower limit of the CI fell just below 1.0 in one meta-analysis, and was just above 1.0 in the other.

Bottom Line: The dogmatic adherence to statistical significant thresholds can lead authors to write dichotomized absolute conclusions while ignoring the broader interpretations of very consistent findings.These opposing conclusions are false disagreements that create unnecessary clinical uncertainty.We provide helpful recommendations in approaching conflicting conclusions when they are associated with remarkably similar results.

View Article: PubMed Central - HTML - PubMed

Affiliation: Evidence-Based Medicine, Department of Family Medicine, University of Alberta, Room 1706 College Plaza, 8215 - 112 Street NW, Edmonton AB, Canada. michael.allan@ualberta.ca.

ABSTRACT

Background: Controversies are common in medicine. Some arise when the conclusions of research publications directly contradict each other, creating uncertainty for frontline clinicians.

Discussion: In this paper, we review how researchers can look at very similar data yet have completely different conclusions based purely on an over-reliance of statistical significance and an unclear understanding of confidence intervals. The dogmatic adherence to statistical significant thresholds can lead authors to write dichotomized absolute conclusions while ignoring the broader interpretations of very consistent findings. We describe three examples of controversy around the potential benefit of a medication, a comparison between new medications, and a medication with a potential harm. The examples include the highest levels of evidence, both meta-analyses and randomized controlled trials. We will show how in each case the confidence intervals and point estimates were very similar. The only identifiable differences to account for the contrasting conclusions arise from the serendipitous finding of confidence intervals that either marginally cross or just fail to cross the line of statistical significance.

Summary: These opposing conclusions are false disagreements that create unnecessary clinical uncertainty. We provide helpful recommendations in approaching conflicting conclusions when they are associated with remarkably similar results.

Show MeSH
Related in: MedlinePlus