Limits...
Predictive modelling using neuroimaging data in the presence of confounds

View Article: PubMed Central - PubMed

ABSTRACT

When training predictive models from neuroimaging data, we typically have available non-imaging variables such as age and gender that affect the imaging data but which we may be uninterested in from a clinical perspective. Such variables are commonly referred to as ‘confounds’. In this work, we firstly give a working definition for confound in the context of training predictive models from samples of neuroimaging data. We define a confound as a variable which affects the imaging data and has an association with the target variable in the sample that differs from that in the population-of-interest, i.e., the population over which we intend to apply the estimated predictive model. The focus of this paper is the scenario in which the confound and target variable are independent in the population-of-interest, but the training sample is biased due to a sample association between the target and confound. We then discuss standard approaches for dealing with confounds in predictive modelling such as image adjustment and including the confound as a predictor, before deriving and motivating an Instance Weighting scheme that attempts to account for confounds by focusing model training so that it is optimal for the population-of-interest. We evaluate the standard approaches and Instance Weighting in two regression problems with neuroimaging data in which we train models in the presence of confounding, and predict samples that are representative of the population-of-interest. For comparison, these models are also evaluated when there is no confounding present. In the first experiment we predict the MMSE score using structural MRI from the ADNI database with gender as the confound, while in the second we predict age using structural MRI from the IXI database with acquisition site as the confound. Considered over both datasets we find that none of the methods for dealing with confounding gives more accurate predictions than a baseline model which ignores confounding, although including the confound as a predictor gives models that are less accurate than the baseline model. We do find, however, that different methods appear to focus their predictions on specific subsets of the population-of-interest, and that predictive accuracy is greater when there is no confounding present. We conclude with a discussion comparing the advantages and disadvantages of each approach, and the implications of our evaluation for building predictive models that can be used in clinical practice.

No MeSH data available.


A schematic of an example illustrating biased and unbiased samples from a population-of-interest. Here, the target variable y is a clinical score, and each ellipse represents the brain of a subject, with larger ellipses indicating a larger brain volume. Gender, indicated by red/blue, plays the role of the confounding variable, with males tending to have larger brains than females due to increased head size. In the population-of-interest, each clinical score y is equally likely, and overall there is an even distribution of gender. There is also no association between clinical score and gender, as gender is evenly distributed for every clinical score. In the population-of-interest, decreases in brain size are associated with increases in y, and we wish to recover this predictive model of y using samples taking from this population. The biased sample, however, contains a correlation between gender and y that is not present in the population-of-interest, with males tending to have higher values of y than the females. In contrast, the unbiased sample has an even split of males and females for each value of the target y, and thus is representative of the population-of-interest.
© Copyright Policy - CC BY
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC5391990&req=5

f0005: A schematic of an example illustrating biased and unbiased samples from a population-of-interest. Here, the target variable y is a clinical score, and each ellipse represents the brain of a subject, with larger ellipses indicating a larger brain volume. Gender, indicated by red/blue, plays the role of the confounding variable, with males tending to have larger brains than females due to increased head size. In the population-of-interest, each clinical score y is equally likely, and overall there is an even distribution of gender. There is also no association between clinical score and gender, as gender is evenly distributed for every clinical score. In the population-of-interest, decreases in brain size are associated with increases in y, and we wish to recover this predictive model of y using samples taking from this population. The biased sample, however, contains a correlation between gender and y that is not present in the population-of-interest, with males tending to have higher values of y than the females. In contrast, the unbiased sample has an even split of males and females for each value of the target y, and thus is representative of the population-of-interest.

Mentions: An important component of the above definition is the idea of a ‘population-of-interest’, which is the population over which we wish to apply the model that is estimated from the data sample D. Note that if a variable affects the image data but its association with the target variable is representative of the population-of-interest, we would then consider the sample to be unbiased, and the variable is not a true confound. While our definition of confound is general, in this paper we will focus on a particular type of confounding where the confound and target variable are independent in the population-of-interest, but the training sample is biased due to a sample association between the target and confound. Such a situation may occur if e.g., we would like our predictive model to predict a clinical score equally well for both male and female subjects across the values of the clinical score, but our training sample shows a significant difference between the values of the clinical score for each gender. In that case, the training sample can be considered as biased by gender, with respect to the population-of-interest. This is illustrated in Fig. 1, where we also show an example of an unbiased sample in this scenario. Note that for unbiased samples, we no longer consider gender to be a confound (by definition) even though it explains variability in the image data.


Predictive modelling using neuroimaging data in the presence of confounds
A schematic of an example illustrating biased and unbiased samples from a population-of-interest. Here, the target variable y is a clinical score, and each ellipse represents the brain of a subject, with larger ellipses indicating a larger brain volume. Gender, indicated by red/blue, plays the role of the confounding variable, with males tending to have larger brains than females due to increased head size. In the population-of-interest, each clinical score y is equally likely, and overall there is an even distribution of gender. There is also no association between clinical score and gender, as gender is evenly distributed for every clinical score. In the population-of-interest, decreases in brain size are associated with increases in y, and we wish to recover this predictive model of y using samples taking from this population. The biased sample, however, contains a correlation between gender and y that is not present in the population-of-interest, with males tending to have higher values of y than the females. In contrast, the unbiased sample has an even split of males and females for each value of the target y, and thus is representative of the population-of-interest.
© Copyright Policy - CC BY
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC5391990&req=5

f0005: A schematic of an example illustrating biased and unbiased samples from a population-of-interest. Here, the target variable y is a clinical score, and each ellipse represents the brain of a subject, with larger ellipses indicating a larger brain volume. Gender, indicated by red/blue, plays the role of the confounding variable, with males tending to have larger brains than females due to increased head size. In the population-of-interest, each clinical score y is equally likely, and overall there is an even distribution of gender. There is also no association between clinical score and gender, as gender is evenly distributed for every clinical score. In the population-of-interest, decreases in brain size are associated with increases in y, and we wish to recover this predictive model of y using samples taking from this population. The biased sample, however, contains a correlation between gender and y that is not present in the population-of-interest, with males tending to have higher values of y than the females. In contrast, the unbiased sample has an even split of males and females for each value of the target y, and thus is representative of the population-of-interest.
Mentions: An important component of the above definition is the idea of a ‘population-of-interest’, which is the population over which we wish to apply the model that is estimated from the data sample D. Note that if a variable affects the image data but its association with the target variable is representative of the population-of-interest, we would then consider the sample to be unbiased, and the variable is not a true confound. While our definition of confound is general, in this paper we will focus on a particular type of confounding where the confound and target variable are independent in the population-of-interest, but the training sample is biased due to a sample association between the target and confound. Such a situation may occur if e.g., we would like our predictive model to predict a clinical score equally well for both male and female subjects across the values of the clinical score, but our training sample shows a significant difference between the values of the clinical score for each gender. In that case, the training sample can be considered as biased by gender, with respect to the population-of-interest. This is illustrated in Fig. 1, where we also show an example of an unbiased sample in this scenario. Note that for unbiased samples, we no longer consider gender to be a confound (by definition) even though it explains variability in the image data.

View Article: PubMed Central - PubMed

ABSTRACT

When training predictive models from neuroimaging data, we typically have available non-imaging variables such as age and gender that affect the imaging data but which we may be uninterested in from a clinical perspective. Such variables are commonly referred to as ‘confounds’. In this work, we firstly give a working definition for confound in the context of training predictive models from samples of neuroimaging data. We define a confound as a variable which affects the imaging data and has an association with the target variable in the sample that differs from that in the population-of-interest, i.e., the population over which we intend to apply the estimated predictive model. The focus of this paper is the scenario in which the confound and target variable are independent in the population-of-interest, but the training sample is biased due to a sample association between the target and confound. We then discuss standard approaches for dealing with confounds in predictive modelling such as image adjustment and including the confound as a predictor, before deriving and motivating an Instance Weighting scheme that attempts to account for confounds by focusing model training so that it is optimal for the population-of-interest. We evaluate the standard approaches and Instance Weighting in two regression problems with neuroimaging data in which we train models in the presence of confounding, and predict samples that are representative of the population-of-interest. For comparison, these models are also evaluated when there is no confounding present. In the first experiment we predict the MMSE score using structural MRI from the ADNI database with gender as the confound, while in the second we predict age using structural MRI from the IXI database with acquisition site as the confound. Considered over both datasets we find that none of the methods for dealing with confounding gives more accurate predictions than a baseline model which ignores confounding, although including the confound as a predictor gives models that are less accurate than the baseline model. We do find, however, that different methods appear to focus their predictions on specific subsets of the population-of-interest, and that predictive accuracy is greater when there is no confounding present. We conclude with a discussion comparing the advantages and disadvantages of each approach, and the implications of our evaluation for building predictive models that can be used in clinical practice.

No MeSH data available.