Limits...
Evidence-based mapping of design heterogeneity prior to meta-analysis: a systematic review and evidence synthesis.

Althuis MD, Weed DL, Frankenfeld CL - Syst Rev (2014)

Bottom Line: Evidence mapping strategies effectively organized complex data and clearly depicted design heterogeneity.This strategy effectively documents design heterogeneity, thus improving the practice of meta-analysis by aiding in: 1) protocol and analysis planning, 2) transparent reporting of differences in study designs, and 3) interpretation of pooled estimates.This would provide readers with more evidence to interpret the summary risk estimates.

View Article: PubMed Central - HTML - PubMed

Affiliation: EpiContext, Washington, DC 20003, USA. Michelle@EpiContext.com.

ABSTRACT

Background: Assessment of design heterogeneity conducted prior to meta-analysis is infrequently reported; it is often presented post hoc to explain statistical heterogeneity. However, design heterogeneity determines the mix of included studies and how they are analyzed in a meta-analysis, which in turn can importantly influence the results. The goal of this work is to introduce ways to improve the assessment and reporting of design heterogeneity prior to statistical summarization of epidemiologic studies.

Methods: In this paper, we use an assessment of sugar-sweetened beverages (SSB) and type 2 diabetes (T2D) as an example to show how a technique called 'evidence mapping' can be used to organize studies and evaluate design heterogeneity prior to meta-analysis.. Employing a systematic and reproducible approach, we evaluated the following elements across 11 selected cohort studies: variation in definitions of SSB, T2D, and co-variables, design features and population characteristics associated with specific definitions of SSB, and diversity in modeling strategies.

Results: Evidence mapping strategies effectively organized complex data and clearly depicted design heterogeneity. For example, across 11 studies of SSB and T2D, 7 measured diet only once (with 7 to 16 years of disease follow-up), 5 included primarily low SSB consumers, and 3 defined the study variable (SSB) as consumption of either sugar or artificially-sweetened beverages. This exercise also identified diversity in analysis strategies, such as adjustment for 11 to 17 co-variables and a large degree of fluctuation in SSB-T2D risk estimates depending on variables selected for multivariable models (2 to 95% change in the risk estimate from the age-adjusted model).

Conclusions: Meta-analysis seeks to understand heterogeneity in addition to computing a summary risk estimate. This strategy effectively documents design heterogeneity, thus improving the practice of meta-analysis by aiding in: 1) protocol and analysis planning, 2) transparent reporting of differences in study designs, and 3) interpretation of pooled estimates. We recommend expanding the practice of meta-analysis reporting to include a table that summarizes design heterogeneity. This would provide readers with more evidence to interpret the summary risk estimates.

Show MeSH

Related in: MedlinePlus

(Step 3). Covariates adjusted for in multivariable models of sugar-sweetened beverages and type 2 diabetes: 11 cohorts. 1. Calculated as proportion change from the age-adjusted model or a fairly simple model: (RRage adjusted - RRmodel)/(RRage adjusted - 1). ↑ denotes an increase in the risk estimate. 2. For the MESA cohort, the model information was based on author correspondence reported in a 2010 meta-analysis [50]. BL, adjustment variable based on baseline assessment. All cohorts used Cox proportional hazards models, except JPHC [37], which used logistic regression.
© Copyright Policy - open-access
Related In: Results  -  Collection

License 1 - License 2
getmorefigures.php?uid=PMC4128504&req=5

Figure 4: (Step 3). Covariates adjusted for in multivariable models of sugar-sweetened beverages and type 2 diabetes: 11 cohorts. 1. Calculated as proportion change from the age-adjusted model or a fairly simple model: (RRage adjusted - RRmodel)/(RRage adjusted - 1). ↑ denotes an increase in the risk estimate. 2. For the MESA cohort, the model information was based on author correspondence reported in a 2010 meta-analysis [50]. BL, adjustment variable based on baseline assessment. All cohorts used Cox proportional hazards models, except JPHC [37], which used logistic regression.

Mentions: Evidence-based mapping visually displayed the patterns of covariates adjusted for in each model of SSB and T2D as reported by each publication (Figure 4). A check denoted adjustment for covariates age, smoking, physical activity, family history, alcohol intake, diet quality score, energy intake, and body mass index. Overall and within important strata of the study variable, diversity of multivariable modeling strategies was described by summarizing operationalization of the SSB intake (n, percent), the maximum number of models of SSB and T2D presented in the original publication (median, range), the maximum number of covariates in models (median, range), and the maximum change in SSB-T2D risk estimate from the age-adjusted risk estimate (median, range).


Evidence-based mapping of design heterogeneity prior to meta-analysis: a systematic review and evidence synthesis.

Althuis MD, Weed DL, Frankenfeld CL - Syst Rev (2014)

(Step 3). Covariates adjusted for in multivariable models of sugar-sweetened beverages and type 2 diabetes: 11 cohorts. 1. Calculated as proportion change from the age-adjusted model or a fairly simple model: (RRage adjusted - RRmodel)/(RRage adjusted - 1). ↑ denotes an increase in the risk estimate. 2. For the MESA cohort, the model information was based on author correspondence reported in a 2010 meta-analysis [50]. BL, adjustment variable based on baseline assessment. All cohorts used Cox proportional hazards models, except JPHC [37], which used logistic regression.
© Copyright Policy - open-access
Related In: Results  -  Collection

License 1 - License 2
Show All Figures
getmorefigures.php?uid=PMC4128504&req=5

Figure 4: (Step 3). Covariates adjusted for in multivariable models of sugar-sweetened beverages and type 2 diabetes: 11 cohorts. 1. Calculated as proportion change from the age-adjusted model or a fairly simple model: (RRage adjusted - RRmodel)/(RRage adjusted - 1). ↑ denotes an increase in the risk estimate. 2. For the MESA cohort, the model information was based on author correspondence reported in a 2010 meta-analysis [50]. BL, adjustment variable based on baseline assessment. All cohorts used Cox proportional hazards models, except JPHC [37], which used logistic regression.
Mentions: Evidence-based mapping visually displayed the patterns of covariates adjusted for in each model of SSB and T2D as reported by each publication (Figure 4). A check denoted adjustment for covariates age, smoking, physical activity, family history, alcohol intake, diet quality score, energy intake, and body mass index. Overall and within important strata of the study variable, diversity of multivariable modeling strategies was described by summarizing operationalization of the SSB intake (n, percent), the maximum number of models of SSB and T2D presented in the original publication (median, range), the maximum number of covariates in models (median, range), and the maximum change in SSB-T2D risk estimate from the age-adjusted risk estimate (median, range).

Bottom Line: Evidence mapping strategies effectively organized complex data and clearly depicted design heterogeneity.This strategy effectively documents design heterogeneity, thus improving the practice of meta-analysis by aiding in: 1) protocol and analysis planning, 2) transparent reporting of differences in study designs, and 3) interpretation of pooled estimates.This would provide readers with more evidence to interpret the summary risk estimates.

View Article: PubMed Central - HTML - PubMed

Affiliation: EpiContext, Washington, DC 20003, USA. Michelle@EpiContext.com.

ABSTRACT

Background: Assessment of design heterogeneity conducted prior to meta-analysis is infrequently reported; it is often presented post hoc to explain statistical heterogeneity. However, design heterogeneity determines the mix of included studies and how they are analyzed in a meta-analysis, which in turn can importantly influence the results. The goal of this work is to introduce ways to improve the assessment and reporting of design heterogeneity prior to statistical summarization of epidemiologic studies.

Methods: In this paper, we use an assessment of sugar-sweetened beverages (SSB) and type 2 diabetes (T2D) as an example to show how a technique called 'evidence mapping' can be used to organize studies and evaluate design heterogeneity prior to meta-analysis.. Employing a systematic and reproducible approach, we evaluated the following elements across 11 selected cohort studies: variation in definitions of SSB, T2D, and co-variables, design features and population characteristics associated with specific definitions of SSB, and diversity in modeling strategies.

Results: Evidence mapping strategies effectively organized complex data and clearly depicted design heterogeneity. For example, across 11 studies of SSB and T2D, 7 measured diet only once (with 7 to 16 years of disease follow-up), 5 included primarily low SSB consumers, and 3 defined the study variable (SSB) as consumption of either sugar or artificially-sweetened beverages. This exercise also identified diversity in analysis strategies, such as adjustment for 11 to 17 co-variables and a large degree of fluctuation in SSB-T2D risk estimates depending on variables selected for multivariable models (2 to 95% change in the risk estimate from the age-adjusted model).

Conclusions: Meta-analysis seeks to understand heterogeneity in addition to computing a summary risk estimate. This strategy effectively documents design heterogeneity, thus improving the practice of meta-analysis by aiding in: 1) protocol and analysis planning, 2) transparent reporting of differences in study designs, and 3) interpretation of pooled estimates. We recommend expanding the practice of meta-analysis reporting to include a table that summarizes design heterogeneity. This would provide readers with more evidence to interpret the summary risk estimates.

Show MeSH
Related in: MedlinePlus