Limits...
A note on obtaining correct marginal predictions from a random intercepts model for binary outcomes.

Pavlou M, Ambler G, Seaman S, Omar RZ - BMC Med Res Methodol (2015)

Bottom Line: In simulation studies, it has been seen that use of the incorrect marginal risk calculation from random effects models results in poorly calibrated overall marginal predictions (calibration slope <1 and calibration in the large ≠ 0) with mis-calibration becoming worse with higher degrees of clustering.We show via simulation that the correct calculation of marginal risks from a random intercepts model results in predictions with excellent calibration.The logistic random intercepts model can be used to obtain valid marginal predictions by integrating over the distribution of random effects.

View Article: PubMed Central - PubMed

Affiliation: Department of Statistical Science, University College London, Gower St., London, WC1E 6BT, UK. m.pavlou@ucl.ac.uk.

ABSTRACT

Background: Clustered data with binary outcomes are often analysed using random intercepts models or generalised estimating equations (GEE) resulting in cluster-specific or 'population-average' inference, respectively.

Methods: When a random effects model is fitted to clustered data, predictions may be produced for a member of an existing cluster by using estimates of the fixed effects (regression coefficients) and the random effect for the cluster (conditional risk calculation), or for a member of a new cluster (marginal risk calculation). We focus on the second. Marginal risk calculation from a random effects model is obtained by integrating over the distribution of random effects. However, in practice marginal risks are often obtained, incorrectly, using only estimates of the fixed effects (i.e. by effectively setting the random effects to zero). We compare these two approaches to marginal risk calculation in terms of model calibration.

Results: In simulation studies, it has been seen that use of the incorrect marginal risk calculation from random effects models results in poorly calibrated overall marginal predictions (calibration slope <1 and calibration in the large ≠ 0) with mis-calibration becoming worse with higher degrees of clustering. We clarify that this was due to the incorrect calculation of marginal predictions from a random intercepts model and explain intuitively why this approach is incorrect. We show via simulation that the correct calculation of marginal risks from a random intercepts model results in predictions with excellent calibration.

Conclusion: The logistic random intercepts model can be used to obtain valid marginal predictions by integrating over the distribution of random effects.

No MeSH data available.


Related in: MedlinePlus

Marginal predictions for a range of values of X1, while the other predictors are set to zero
© Copyright Policy - OpenAccess
Related In: Results  -  Collection

License 1 - License 2
getmorefigures.php?uid=PMC4525751&req=5

Fig1: Marginal predictions for a range of values of X1, while the other predictors are set to zero

Mentions: Now we demonstrate graphically the difference between the predicted risks from each method. We show the predicted risk for a range of values of the first continuous predictor, while, without loss of generality, the other five predictors are set to zero. This is done for ICC = 0.05, 0.1, 0.15 or 0.3, and the results for ICC = 0.3 are shown in Fig. 1. The naïve marginal predictions, tend to be more extreme than the predictions from the other methods, all of which are very similar. This is reflected by the calibration slope which is <1 for the naïve predictions but very close to one for the rest. As the ICC becomes smaller (graphs not shown) the naïve predictions approach the ones from the other methods.Fig. 1


A note on obtaining correct marginal predictions from a random intercepts model for binary outcomes.

Pavlou M, Ambler G, Seaman S, Omar RZ - BMC Med Res Methodol (2015)

Marginal predictions for a range of values of X1, while the other predictors are set to zero
© Copyright Policy - OpenAccess
Related In: Results  -  Collection

License 1 - License 2
Show All Figures
getmorefigures.php?uid=PMC4525751&req=5

Fig1: Marginal predictions for a range of values of X1, while the other predictors are set to zero
Mentions: Now we demonstrate graphically the difference between the predicted risks from each method. We show the predicted risk for a range of values of the first continuous predictor, while, without loss of generality, the other five predictors are set to zero. This is done for ICC = 0.05, 0.1, 0.15 or 0.3, and the results for ICC = 0.3 are shown in Fig. 1. The naïve marginal predictions, tend to be more extreme than the predictions from the other methods, all of which are very similar. This is reflected by the calibration slope which is <1 for the naïve predictions but very close to one for the rest. As the ICC becomes smaller (graphs not shown) the naïve predictions approach the ones from the other methods.Fig. 1

Bottom Line: In simulation studies, it has been seen that use of the incorrect marginal risk calculation from random effects models results in poorly calibrated overall marginal predictions (calibration slope <1 and calibration in the large ≠ 0) with mis-calibration becoming worse with higher degrees of clustering.We show via simulation that the correct calculation of marginal risks from a random intercepts model results in predictions with excellent calibration.The logistic random intercepts model can be used to obtain valid marginal predictions by integrating over the distribution of random effects.

View Article: PubMed Central - PubMed

Affiliation: Department of Statistical Science, University College London, Gower St., London, WC1E 6BT, UK. m.pavlou@ucl.ac.uk.

ABSTRACT

Background: Clustered data with binary outcomes are often analysed using random intercepts models or generalised estimating equations (GEE) resulting in cluster-specific or 'population-average' inference, respectively.

Methods: When a random effects model is fitted to clustered data, predictions may be produced for a member of an existing cluster by using estimates of the fixed effects (regression coefficients) and the random effect for the cluster (conditional risk calculation), or for a member of a new cluster (marginal risk calculation). We focus on the second. Marginal risk calculation from a random effects model is obtained by integrating over the distribution of random effects. However, in practice marginal risks are often obtained, incorrectly, using only estimates of the fixed effects (i.e. by effectively setting the random effects to zero). We compare these two approaches to marginal risk calculation in terms of model calibration.

Results: In simulation studies, it has been seen that use of the incorrect marginal risk calculation from random effects models results in poorly calibrated overall marginal predictions (calibration slope <1 and calibration in the large ≠ 0) with mis-calibration becoming worse with higher degrees of clustering. We clarify that this was due to the incorrect calculation of marginal predictions from a random intercepts model and explain intuitively why this approach is incorrect. We show via simulation that the correct calculation of marginal risks from a random intercepts model results in predictions with excellent calibration.

Conclusion: The logistic random intercepts model can be used to obtain valid marginal predictions by integrating over the distribution of random effects.

No MeSH data available.


Related in: MedlinePlus