Limits...
Results of observational studies: analysis of findings from the Nurses' Health Study.

Tai V, Grey A, Bolland MJ - PLoS ONE (2014)

Bottom Line: NHS results were compared with RCT results and deemed concordant when the difference in effect sizes between studies was ≤0.15. 2007 associations between health outcomes and independent variables were reported in 1053 abstracts. 58.0% (1165/2007) were statistically significant, and 22.2% (445/2007) were neutral (no association).NHS publications contain a large number of analyses, the majority of which reported statistically significant but weak associations.Few of these associations have been tested in RCTs, and where they have, the agreement between NHS results and RCTs is poor.

View Article: PubMed Central - PubMed

Affiliation: Department of Medicine, University of Auckland, Auckland, New Zealand.

ABSTRACT

Background: The role of observational studies in informing clinical practice is debated, and high profile examples of discrepancies between the results of observational studies and randomised controlled trials (RCTs) have intensified that debate. We systematically reviewed findings from the Nurses' Health Study (NHS), one of the longest and largest observational studies, to assess the number and strength of the associations reported and to determine if they have been confirmed in RCTs.

Methods: We reviewed NHS publication abstracts from 1978-2012, extracted information on associations tested, and graded the strength of the reported effect sizes. We searched PubMed for RCTs or systematic reviews for 3 health outcomes commonly reported in NHS publications: breast cancer, ischaemic heart disease (IHD) and osteoporosis. NHS results were compared with RCT results and deemed concordant when the difference in effect sizes between studies was ≤0.15.

Findings: 2007 associations between health outcomes and independent variables were reported in 1053 abstracts. 58.0% (1165/2007) were statistically significant, and 22.2% (445/2007) were neutral (no association). Among the statistically significant results that reported a numeric odds ratio (OR) or relative risk (RR), 70.5% (706/1002) reported a weak association (OR/RR 0.5-2.0), 24.5% (246/1002) a moderate association (OR/RR 0.25-0.5 or 2.0-4.0) and 5.0% (50/1002) a strong association (OR/RR ≤0.25 or ≥4.0). 19 associations reported in NHS publications for breast cancer, IHD and osteoporosis have been tested in RCTs, and the concordance between NHS and RCT results was low (≤25%).

Conclusions: NHS publications contain a large number of analyses, the majority of which reported statistically significant but weak associations. Few of these associations have been tested in RCTs, and where they have, the agreement between NHS results and RCTs is poor.

Show MeSH

Related in: MedlinePlus

Flowchart of study selection.
© Copyright Policy
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4201544&req=5

pone-0110403-g001: Flowchart of study selection.

Mentions: In November 2013, we extracted the citations of all 1235 NHS publications between 1978–2012 from the NHS website (http://www.channing.harvard.edu/nhs/?page_id=154). We included publications with an abstract, those in which the NHS cohort was part of the population studied, and those with an observational (case-control or cohort) study design. Figure 1 shows the flow of studies. 28 publications did not have an abstract, 52 studies did not include the NHS cohort, and 102 publications did not report findings from observational analyses, leaving 1053 publications included in our analyses.


Results of observational studies: analysis of findings from the Nurses' Health Study.

Tai V, Grey A, Bolland MJ - PLoS ONE (2014)

Flowchart of study selection.
© Copyright Policy
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4201544&req=5

pone-0110403-g001: Flowchart of study selection.
Mentions: In November 2013, we extracted the citations of all 1235 NHS publications between 1978–2012 from the NHS website (http://www.channing.harvard.edu/nhs/?page_id=154). We included publications with an abstract, those in which the NHS cohort was part of the population studied, and those with an observational (case-control or cohort) study design. Figure 1 shows the flow of studies. 28 publications did not have an abstract, 52 studies did not include the NHS cohort, and 102 publications did not report findings from observational analyses, leaving 1053 publications included in our analyses.

Bottom Line: NHS results were compared with RCT results and deemed concordant when the difference in effect sizes between studies was ≤0.15. 2007 associations between health outcomes and independent variables were reported in 1053 abstracts. 58.0% (1165/2007) were statistically significant, and 22.2% (445/2007) were neutral (no association).NHS publications contain a large number of analyses, the majority of which reported statistically significant but weak associations.Few of these associations have been tested in RCTs, and where they have, the agreement between NHS results and RCTs is poor.

View Article: PubMed Central - PubMed

Affiliation: Department of Medicine, University of Auckland, Auckland, New Zealand.

ABSTRACT

Background: The role of observational studies in informing clinical practice is debated, and high profile examples of discrepancies between the results of observational studies and randomised controlled trials (RCTs) have intensified that debate. We systematically reviewed findings from the Nurses' Health Study (NHS), one of the longest and largest observational studies, to assess the number and strength of the associations reported and to determine if they have been confirmed in RCTs.

Methods: We reviewed NHS publication abstracts from 1978-2012, extracted information on associations tested, and graded the strength of the reported effect sizes. We searched PubMed for RCTs or systematic reviews for 3 health outcomes commonly reported in NHS publications: breast cancer, ischaemic heart disease (IHD) and osteoporosis. NHS results were compared with RCT results and deemed concordant when the difference in effect sizes between studies was ≤0.15.

Findings: 2007 associations between health outcomes and independent variables were reported in 1053 abstracts. 58.0% (1165/2007) were statistically significant, and 22.2% (445/2007) were neutral (no association). Among the statistically significant results that reported a numeric odds ratio (OR) or relative risk (RR), 70.5% (706/1002) reported a weak association (OR/RR 0.5-2.0), 24.5% (246/1002) a moderate association (OR/RR 0.25-0.5 or 2.0-4.0) and 5.0% (50/1002) a strong association (OR/RR ≤0.25 or ≥4.0). 19 associations reported in NHS publications for breast cancer, IHD and osteoporosis have been tested in RCTs, and the concordance between NHS and RCT results was low (≤25%).

Conclusions: NHS publications contain a large number of analyses, the majority of which reported statistically significant but weak associations. Few of these associations have been tested in RCTs, and where they have, the agreement between NHS results and RCTs is poor.

Show MeSH
Related in: MedlinePlus