Limits...
Imputation methods for missing outcome data in meta-analysis of clinical trials.

Higgins JP, White IR, Wood AM - Clin Trials (2008)

Bottom Line: We review some common strategies, such as simple imputation of positive or negative outcomes, and develop a general approach involving ;informative missingness odds ratios' (IMORs).We propose that available reasons for missingness be used to determine appropriate IMORs.We also recommend a strategy for undertaking sensitivity analyses, in which the IMORs are varied over plausible ranges.

View Article: PubMed Central - PubMed

Affiliation: MRC Biostatistics Unit, Institute of Public Health, Robinson Way, Cambridge, UK. julian.higgins@mrc-bsu.cam.ac.uk

ABSTRACT

Background: Missing outcome data from randomized trials lead to greater uncertainty and possible bias in estimating the effect of an experimental treatment. An intention-to-treat analysis should take account of all randomized participants even if they have missing observations.

Purpose: To review and develop imputation methods for missing outcome data in meta-analysis of clinical trials with binary outcomes.

Methods: We review some common strategies, such as simple imputation of positive or negative outcomes, and develop a general approach involving ;informative missingness odds ratios' (IMORs). We describe several choices for weighting studies in the meta-analysis, and illustrate methods using a meta-analysis of trials of haloperidol for schizophrenia.

Results: IMORs describe the relationship between the unknown risk among missing participants and the known risk among observed participants. They are allowed to differ between treatment groups and across trials. Application of IMORs and other methods to the haloperidol trials reveals the overall conclusion to be robust to different assumptions about the missing data.

Limitations: The methods are based on summary data from each trial (number of observed positive outcomes, number of observed negative outcomes and number of missing outcomes) for each intervention group. This limits the options for analysis, and greater flexibility would be available with individual participant data.

Conclusions: We propose that available reasons for missingness be used to determine appropriate IMORs. We also recommend a strategy for undertaking sensitivity analyses, in which the IMORs are varied over plausible ranges.

Show MeSH

Related in: MedlinePlus

Some possible scenarios for missing data. Arrows indicate causal effects.                            Missing completely at random: (a) outcome and missingness are unrelated                            and not dependent on any other variables; (b) missingness is                            ‘random’, but outcome may be dependent on other                            variables. Missing at random: (c) different variables are responsible                            for outcomes and for missingness; (d) the same variables are responsible                            for outcomes and for missingness, but can be incorporated into the                            analysis; Informatively missing: (e) the same variables are responsible                            for outcomes and for missingness, but cannot be incorporated into the                            analysis; (f) missingness depends directly on the unobserved outcome
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC2602608&req=5

Figure 1: Some possible scenarios for missing data. Arrows indicate causal effects. Missing completely at random: (a) outcome and missingness are unrelated and not dependent on any other variables; (b) missingness is ‘random’, but outcome may be dependent on other variables. Missing at random: (c) different variables are responsible for outcomes and for missingness; (d) the same variables are responsible for outcomes and for missingness, but can be incorporated into the analysis; Informatively missing: (e) the same variables are responsible for outcomes and for missingness, but cannot be incorporated into the analysis; (f) missingness depends directly on the unobserved outcome

Mentions: It is useful to classify missing outcome data according to the relationship between nonavailability of a particular value and the observed and unobserved values. We will use the term ‘missingness’ for the nonavailability of a participant's outcome. First, if missingness of an outcome is not related to any observed or unobserved variables, then the missing data are described as ‘missing completely at random’ (Figure 1(a) and (b)). Analysis restricted to individuals with complete data is always valid when the data are missing completely at random. If missingness of an outcome may be related to observed or unobserved variables, but is not related to the actual value of the outcome, conditional on the observed variables, then the missing data are described as ‘missing at random’ (Figure 1(c) and (d)). An alternative term is ‘ignorable’, because a correct likelihood-based analysis of all the observed data is valid [3]. (Strictly, a further condition is required, but this is true in almost all practical applications.) ‘Missing completely at random’ is a special case of ‘missing at random’. Finally, if missingness of an outcome is related to the value of that outcome, even conditional on other observed variables, then the missing data are described as ‘informatively missing’. This could be because of some common unobserved cause of both missingness and the outcomes (Figure 1(e)) or because the outcome directly causes missingness (Figure 1(f )). Alternative terms are ‘missing not at random’, ‘not missing at random’ or ‘nonignorable’, the last so called because a likelihood-based analysis of the observed data alone is typically biased [3]. Figure 1


Imputation methods for missing outcome data in meta-analysis of clinical trials.

Higgins JP, White IR, Wood AM - Clin Trials (2008)

Some possible scenarios for missing data. Arrows indicate causal effects.                            Missing completely at random: (a) outcome and missingness are unrelated                            and not dependent on any other variables; (b) missingness is                            ‘random’, but outcome may be dependent on other                            variables. Missing at random: (c) different variables are responsible                            for outcomes and for missingness; (d) the same variables are responsible                            for outcomes and for missingness, but can be incorporated into the                            analysis; Informatively missing: (e) the same variables are responsible                            for outcomes and for missingness, but cannot be incorporated into the                            analysis; (f) missingness depends directly on the unobserved outcome
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC2602608&req=5

Figure 1: Some possible scenarios for missing data. Arrows indicate causal effects. Missing completely at random: (a) outcome and missingness are unrelated and not dependent on any other variables; (b) missingness is ‘random’, but outcome may be dependent on other variables. Missing at random: (c) different variables are responsible for outcomes and for missingness; (d) the same variables are responsible for outcomes and for missingness, but can be incorporated into the analysis; Informatively missing: (e) the same variables are responsible for outcomes and for missingness, but cannot be incorporated into the analysis; (f) missingness depends directly on the unobserved outcome
Mentions: It is useful to classify missing outcome data according to the relationship between nonavailability of a particular value and the observed and unobserved values. We will use the term ‘missingness’ for the nonavailability of a participant's outcome. First, if missingness of an outcome is not related to any observed or unobserved variables, then the missing data are described as ‘missing completely at random’ (Figure 1(a) and (b)). Analysis restricted to individuals with complete data is always valid when the data are missing completely at random. If missingness of an outcome may be related to observed or unobserved variables, but is not related to the actual value of the outcome, conditional on the observed variables, then the missing data are described as ‘missing at random’ (Figure 1(c) and (d)). An alternative term is ‘ignorable’, because a correct likelihood-based analysis of all the observed data is valid [3]. (Strictly, a further condition is required, but this is true in almost all practical applications.) ‘Missing completely at random’ is a special case of ‘missing at random’. Finally, if missingness of an outcome is related to the value of that outcome, even conditional on other observed variables, then the missing data are described as ‘informatively missing’. This could be because of some common unobserved cause of both missingness and the outcomes (Figure 1(e)) or because the outcome directly causes missingness (Figure 1(f )). Alternative terms are ‘missing not at random’, ‘not missing at random’ or ‘nonignorable’, the last so called because a likelihood-based analysis of the observed data alone is typically biased [3]. Figure 1

Bottom Line: We review some common strategies, such as simple imputation of positive or negative outcomes, and develop a general approach involving ;informative missingness odds ratios' (IMORs).We propose that available reasons for missingness be used to determine appropriate IMORs.We also recommend a strategy for undertaking sensitivity analyses, in which the IMORs are varied over plausible ranges.

View Article: PubMed Central - PubMed

Affiliation: MRC Biostatistics Unit, Institute of Public Health, Robinson Way, Cambridge, UK. julian.higgins@mrc-bsu.cam.ac.uk

ABSTRACT

Background: Missing outcome data from randomized trials lead to greater uncertainty and possible bias in estimating the effect of an experimental treatment. An intention-to-treat analysis should take account of all randomized participants even if they have missing observations.

Purpose: To review and develop imputation methods for missing outcome data in meta-analysis of clinical trials with binary outcomes.

Methods: We review some common strategies, such as simple imputation of positive or negative outcomes, and develop a general approach involving ;informative missingness odds ratios' (IMORs). We describe several choices for weighting studies in the meta-analysis, and illustrate methods using a meta-analysis of trials of haloperidol for schizophrenia.

Results: IMORs describe the relationship between the unknown risk among missing participants and the known risk among observed participants. They are allowed to differ between treatment groups and across trials. Application of IMORs and other methods to the haloperidol trials reveals the overall conclusion to be robust to different assumptions about the missing data.

Limitations: The methods are based on summary data from each trial (number of observed positive outcomes, number of observed negative outcomes and number of missing outcomes) for each intervention group. This limits the options for analysis, and greater flexibility would be available with individual participant data.

Conclusions: We propose that available reasons for missingness be used to determine appropriate IMORs. We also recommend a strategy for undertaking sensitivity analyses, in which the IMORs are varied over plausible ranges.

Show MeSH
Related in: MedlinePlus