Limits...
Evidence-based mapping of design heterogeneity prior to meta-analysis: a systematic review and evidence synthesis.

Althuis MD, Weed DL, Frankenfeld CL - Syst Rev (2014)

Bottom Line: Evidence mapping strategies effectively organized complex data and clearly depicted design heterogeneity.This strategy effectively documents design heterogeneity, thus improving the practice of meta-analysis by aiding in: 1) protocol and analysis planning, 2) transparent reporting of differences in study designs, and 3) interpretation of pooled estimates.This would provide readers with more evidence to interpret the summary risk estimates.

View Article: PubMed Central - HTML - PubMed

Affiliation: EpiContext, Washington, DC 20003, USA. Michelle@EpiContext.com.

ABSTRACT

Background: Assessment of design heterogeneity conducted prior to meta-analysis is infrequently reported; it is often presented post hoc to explain statistical heterogeneity. However, design heterogeneity determines the mix of included studies and how they are analyzed in a meta-analysis, which in turn can importantly influence the results. The goal of this work is to introduce ways to improve the assessment and reporting of design heterogeneity prior to statistical summarization of epidemiologic studies.

Methods: In this paper, we use an assessment of sugar-sweetened beverages (SSB) and type 2 diabetes (T2D) as an example to show how a technique called 'evidence mapping' can be used to organize studies and evaluate design heterogeneity prior to meta-analysis.. Employing a systematic and reproducible approach, we evaluated the following elements across 11 selected cohort studies: variation in definitions of SSB, T2D, and co-variables, design features and population characteristics associated with specific definitions of SSB, and diversity in modeling strategies.

Results: Evidence mapping strategies effectively organized complex data and clearly depicted design heterogeneity. For example, across 11 studies of SSB and T2D, 7 measured diet only once (with 7 to 16 years of disease follow-up), 5 included primarily low SSB consumers, and 3 defined the study variable (SSB) as consumption of either sugar or artificially-sweetened beverages. This exercise also identified diversity in analysis strategies, such as adjustment for 11 to 17 co-variables and a large degree of fluctuation in SSB-T2D risk estimates depending on variables selected for multivariable models (2 to 95% change in the risk estimate from the age-adjusted model).

Conclusions: Meta-analysis seeks to understand heterogeneity in addition to computing a summary risk estimate. This strategy effectively documents design heterogeneity, thus improving the practice of meta-analysis by aiding in: 1) protocol and analysis planning, 2) transparent reporting of differences in study designs, and 3) interpretation of pooled estimates. We recommend expanding the practice of meta-analysis reporting to include a table that summarizes design heterogeneity. This would provide readers with more evidence to interpret the summary risk estimates.

Show MeSH

Related in: MedlinePlus

Step 1: Categorizing cohorts according to the definition of the study variable, sugar-sweetened beverages. ARIC, Atherosclerosis Risk in Communities Study; BWHS, Black Women's Health Study; EPIC, European Prospective Investigation of Cancer (InterAct Study); FMC, Finnish Mobile Clinic Heath Examination Survey; HPFS, Health Professional's Follow up Study; Jfact, Study of Japanese factory workers; JPHC, Japan Public Health Centre-based Prospective Study; MESA, Multi-Ethnic Study of Atherosclerosis; NHS, Nurse's Health Study; SCHS, Singapore Chinese Health Study; SD, soft drink; SSB, sugar-sweetened beverage.
© Copyright Policy - open-access
Related In: Results  -  Collection

License 1 - License 2
getmorefigures.php?uid=PMC4128504&req=5

Figure 2: Step 1: Categorizing cohorts according to the definition of the study variable, sugar-sweetened beverages. ARIC, Atherosclerosis Risk in Communities Study; BWHS, Black Women's Health Study; EPIC, European Prospective Investigation of Cancer (InterAct Study); FMC, Finnish Mobile Clinic Heath Examination Survey; HPFS, Health Professional's Follow up Study; Jfact, Study of Japanese factory workers; JPHC, Japan Public Health Centre-based Prospective Study; MESA, Multi-Ethnic Study of Atherosclerosis; NHS, Nurse's Health Study; SCHS, Singapore Chinese Health Study; SD, soft drink; SSB, sugar-sweetened beverage.

Mentions: Evidence maps were used to categorize studies based on the definition of the exposure variable, outcome, and population characteristics. Exposure characterization took into account the type of beverage, data collection instruments, and frequency/timing of data collection (FigureĀ 2). Variation in definitions of T2D was evaluated based on criteria for diagnosis and method of ascertainment such as by a physician or self-report. Using the study as the unit of analysis, univariate statistics (n, median, proportion, range) were used to describe across included cohorts heterogeneity of SSB intake (exposure and comparator), T2D diagnosis (outcome), and the following population/study characteristics: study location, gender, study size, duration of follow-up, baseline BMI, and baseline SSB consumption.


Evidence-based mapping of design heterogeneity prior to meta-analysis: a systematic review and evidence synthesis.

Althuis MD, Weed DL, Frankenfeld CL - Syst Rev (2014)

Step 1: Categorizing cohorts according to the definition of the study variable, sugar-sweetened beverages. ARIC, Atherosclerosis Risk in Communities Study; BWHS, Black Women's Health Study; EPIC, European Prospective Investigation of Cancer (InterAct Study); FMC, Finnish Mobile Clinic Heath Examination Survey; HPFS, Health Professional's Follow up Study; Jfact, Study of Japanese factory workers; JPHC, Japan Public Health Centre-based Prospective Study; MESA, Multi-Ethnic Study of Atherosclerosis; NHS, Nurse's Health Study; SCHS, Singapore Chinese Health Study; SD, soft drink; SSB, sugar-sweetened beverage.
© Copyright Policy - open-access
Related In: Results  -  Collection

License 1 - License 2
Show All Figures
getmorefigures.php?uid=PMC4128504&req=5

Figure 2: Step 1: Categorizing cohorts according to the definition of the study variable, sugar-sweetened beverages. ARIC, Atherosclerosis Risk in Communities Study; BWHS, Black Women's Health Study; EPIC, European Prospective Investigation of Cancer (InterAct Study); FMC, Finnish Mobile Clinic Heath Examination Survey; HPFS, Health Professional's Follow up Study; Jfact, Study of Japanese factory workers; JPHC, Japan Public Health Centre-based Prospective Study; MESA, Multi-Ethnic Study of Atherosclerosis; NHS, Nurse's Health Study; SCHS, Singapore Chinese Health Study; SD, soft drink; SSB, sugar-sweetened beverage.
Mentions: Evidence maps were used to categorize studies based on the definition of the exposure variable, outcome, and population characteristics. Exposure characterization took into account the type of beverage, data collection instruments, and frequency/timing of data collection (FigureĀ 2). Variation in definitions of T2D was evaluated based on criteria for diagnosis and method of ascertainment such as by a physician or self-report. Using the study as the unit of analysis, univariate statistics (n, median, proportion, range) were used to describe across included cohorts heterogeneity of SSB intake (exposure and comparator), T2D diagnosis (outcome), and the following population/study characteristics: study location, gender, study size, duration of follow-up, baseline BMI, and baseline SSB consumption.

Bottom Line: Evidence mapping strategies effectively organized complex data and clearly depicted design heterogeneity.This strategy effectively documents design heterogeneity, thus improving the practice of meta-analysis by aiding in: 1) protocol and analysis planning, 2) transparent reporting of differences in study designs, and 3) interpretation of pooled estimates.This would provide readers with more evidence to interpret the summary risk estimates.

View Article: PubMed Central - HTML - PubMed

Affiliation: EpiContext, Washington, DC 20003, USA. Michelle@EpiContext.com.

ABSTRACT

Background: Assessment of design heterogeneity conducted prior to meta-analysis is infrequently reported; it is often presented post hoc to explain statistical heterogeneity. However, design heterogeneity determines the mix of included studies and how they are analyzed in a meta-analysis, which in turn can importantly influence the results. The goal of this work is to introduce ways to improve the assessment and reporting of design heterogeneity prior to statistical summarization of epidemiologic studies.

Methods: In this paper, we use an assessment of sugar-sweetened beverages (SSB) and type 2 diabetes (T2D) as an example to show how a technique called 'evidence mapping' can be used to organize studies and evaluate design heterogeneity prior to meta-analysis.. Employing a systematic and reproducible approach, we evaluated the following elements across 11 selected cohort studies: variation in definitions of SSB, T2D, and co-variables, design features and population characteristics associated with specific definitions of SSB, and diversity in modeling strategies.

Results: Evidence mapping strategies effectively organized complex data and clearly depicted design heterogeneity. For example, across 11 studies of SSB and T2D, 7 measured diet only once (with 7 to 16 years of disease follow-up), 5 included primarily low SSB consumers, and 3 defined the study variable (SSB) as consumption of either sugar or artificially-sweetened beverages. This exercise also identified diversity in analysis strategies, such as adjustment for 11 to 17 co-variables and a large degree of fluctuation in SSB-T2D risk estimates depending on variables selected for multivariable models (2 to 95% change in the risk estimate from the age-adjusted model).

Conclusions: Meta-analysis seeks to understand heterogeneity in addition to computing a summary risk estimate. This strategy effectively documents design heterogeneity, thus improving the practice of meta-analysis by aiding in: 1) protocol and analysis planning, 2) transparent reporting of differences in study designs, and 3) interpretation of pooled estimates. We recommend expanding the practice of meta-analysis reporting to include a table that summarizes design heterogeneity. This would provide readers with more evidence to interpret the summary risk estimates.

Show MeSH
Related in: MedlinePlus