Limits...
Selection in reported epidemiological risks: an empirical assessment.

Kavvoura FK, Liberopoulos G, Ioannidis JP - PLoS Med. (2007)

Bottom Line: We empirically evaluated the extent of selection of significant results and large effect sizes in a large sample of recent articles.We examined the proportion and correlates of reporting statistically significant and nonsignificant results in the abstract and whether the magnitude of the relative risks presented (coined to be consistently > or =1.00) differs depending on the type of contrast used for the risk factor.Published epidemiological investigations almost universally highlight significant associations between risk factors and outcomes.

View Article: PubMed Central - PubMed

Affiliation: Clinical and Molecular Epidemiology Unit, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece.

ABSTRACT

Background: Epidemiological studies may be subject to selective reporting, but empirical evidence thereof is limited. We empirically evaluated the extent of selection of significant results and large effect sizes in a large sample of recent articles.

Methods and findings: We evaluated 389 articles of epidemiological studies that reported, in their respective abstracts, at least one relative risk for a continuous risk factor in contrasts based on median, tertile, quartile, or quintile categorizations. We examined the proportion and correlates of reporting statistically significant and nonsignificant results in the abstract and whether the magnitude of the relative risks presented (coined to be consistently > or =1.00) differs depending on the type of contrast used for the risk factor. In 342 articles (87.9%), > or =1 statistically significant relative risk was reported in the abstract, while only 169 articles (43.4%) reported > or =1 statistically nonsignificant relative risk in the abstract. Reporting of statistically significant results was more common with structured abstracts, and was less common in US-based studies and in cancer outcomes. Among 50 randomly selected articles in which the full text was examined, a median of nine (interquartile range 5-16) statistically significant and six (interquartile range 3-16) statistically nonsignificant relative risks were presented (p = 0.25). Paradoxically, the smallest presented relative risks were based on the contrasts of extreme quintiles; on average, the relative risk magnitude was 1.41-, 1.42-, and 1.36-fold larger in contrasts of extreme quartiles, extreme tertiles, and above-versus-below median values, respectively (p < 0.001).

Conclusions: Published epidemiological investigations almost universally highlight significant associations between risk factors and outcomes. For continuous risk factors, investigators selectively present contrasts between more extreme groups, when relative risks are inherently lower.

Show MeSH

Related in: MedlinePlus

Box Plots for Relative Risks for Different Contrasts of the Values of the Postulated Risk FactorAll relative risks have been coined to be ≥1.00 for consistency.
© Copyright Policy
Related In: Results  -  Collection


getmorefigures.php?uid=PMC1808481&req=5

pmed-0040079-g002: Box Plots for Relative Risks for Different Contrasts of the Values of the Postulated Risk FactorAll relative risks have been coined to be ≥1.00 for consistency.

Mentions: Figure 2 shows the distribution of relative risks (coined to be ≥1.00) according to the type of contrast used to present the postulated risk. Contrary to what would be expected, the presented effects were smaller, on average, when the compared groups of the postulated risk factor were further apart. The smallest effects were described with the contrast of extreme quintiles. Compared with the contrasts of extreme quintiles, the relative risks were significantly larger in contrasts of extreme quartiles (1.41-fold larger), extreme tertiles (1.42-fold larger), and above-versus-below median (1.36-fold larger) (ANOVA, p < 0.001).


Selection in reported epidemiological risks: an empirical assessment.

Kavvoura FK, Liberopoulos G, Ioannidis JP - PLoS Med. (2007)

Box Plots for Relative Risks for Different Contrasts of the Values of the Postulated Risk FactorAll relative risks have been coined to be ≥1.00 for consistency.
© Copyright Policy
Related In: Results  -  Collection

Show All Figures
getmorefigures.php?uid=PMC1808481&req=5

pmed-0040079-g002: Box Plots for Relative Risks for Different Contrasts of the Values of the Postulated Risk FactorAll relative risks have been coined to be ≥1.00 for consistency.
Mentions: Figure 2 shows the distribution of relative risks (coined to be ≥1.00) according to the type of contrast used to present the postulated risk. Contrary to what would be expected, the presented effects were smaller, on average, when the compared groups of the postulated risk factor were further apart. The smallest effects were described with the contrast of extreme quintiles. Compared with the contrasts of extreme quintiles, the relative risks were significantly larger in contrasts of extreme quartiles (1.41-fold larger), extreme tertiles (1.42-fold larger), and above-versus-below median (1.36-fold larger) (ANOVA, p < 0.001).

Bottom Line: We empirically evaluated the extent of selection of significant results and large effect sizes in a large sample of recent articles.We examined the proportion and correlates of reporting statistically significant and nonsignificant results in the abstract and whether the magnitude of the relative risks presented (coined to be consistently > or =1.00) differs depending on the type of contrast used for the risk factor.Published epidemiological investigations almost universally highlight significant associations between risk factors and outcomes.

View Article: PubMed Central - PubMed

Affiliation: Clinical and Molecular Epidemiology Unit, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece.

ABSTRACT

Background: Epidemiological studies may be subject to selective reporting, but empirical evidence thereof is limited. We empirically evaluated the extent of selection of significant results and large effect sizes in a large sample of recent articles.

Methods and findings: We evaluated 389 articles of epidemiological studies that reported, in their respective abstracts, at least one relative risk for a continuous risk factor in contrasts based on median, tertile, quartile, or quintile categorizations. We examined the proportion and correlates of reporting statistically significant and nonsignificant results in the abstract and whether the magnitude of the relative risks presented (coined to be consistently > or =1.00) differs depending on the type of contrast used for the risk factor. In 342 articles (87.9%), > or =1 statistically significant relative risk was reported in the abstract, while only 169 articles (43.4%) reported > or =1 statistically nonsignificant relative risk in the abstract. Reporting of statistically significant results was more common with structured abstracts, and was less common in US-based studies and in cancer outcomes. Among 50 randomly selected articles in which the full text was examined, a median of nine (interquartile range 5-16) statistically significant and six (interquartile range 3-16) statistically nonsignificant relative risks were presented (p = 0.25). Paradoxically, the smallest presented relative risks were based on the contrasts of extreme quintiles; on average, the relative risk magnitude was 1.41-, 1.42-, and 1.36-fold larger in contrasts of extreme quartiles, extreme tertiles, and above-versus-below median values, respectively (p < 0.001).

Conclusions: Published epidemiological investigations almost universally highlight significant associations between risk factors and outcomes. For continuous risk factors, investigators selectively present contrasts between more extreme groups, when relative risks are inherently lower.

Show MeSH
Related in: MedlinePlus