Limits...
Statistical evaluation of methods for quantifying gene expression by autoradiography in histological sections.

Lazic SE - BMC Neurosci (2009)

Bottom Line: Image segmentation based on thresholding can be subject to floor-effects and lead to biased results.Finally, converting grey level pixel intensities to optical densities or units of radioactivity is unnecessary for most applications and can lead to data with poor statistical properties.Based on these results, suggestions are made to reduce bias, increase precision, and ultimately provide more meaningful results of gene expression data.

View Article: PubMed Central - HTML - PubMed

Affiliation: Department of Applied Mathematics and Theoretical Physics, Cambridge Computational Biology Institute, University of Cambridge, Cambridge, UK. stan.lazic@cantab.net

ABSTRACT

Background: In situ hybridisation (ISH) combined with autoradiography is a standard method of measuring the amount of gene expression in histological sections, but the methods used to quantify gene expression in the resulting digital images vary greatly between studies and can potentially give conflicting results.

Results: The present study examines commonly used methods for analysing ISH images and demonstrates that these methods are not optimal. Image segmentation based on thresholding can be subject to floor-effects and lead to biased results. In addition, including the area of the structure or region of interest in the calculation of gene expression can lead to a large loss of precision and can also introduce bias. Finally, converting grey level pixel intensities to optical densities or units of radioactivity is unnecessary for most applications and can lead to data with poor statistical properties. A modification of an existing method for selecting the structure or region of interest is introduced which performs better than alternative methods in terms of bias and precision.

Conclusion: Based on these results, suggestions are made to reduce bias, increase precision, and ultimately provide more meaningful results of gene expression data.

Show MeSH

Related in: MedlinePlus

Effect of transformations on simulated data. Data in the first column are the simulated grey level values from various distributions. Note that the y-axis is the same in these three panels so the data can be compared directly. A1 has a large difference between means and low variability, leading to no overlap between data points. B1 has similar variability as A1 but the difference in means has been decreased by half, leading to overlapping values between groups. C1 has the same difference in means as B1, but much greater variability. Horizontal grey lines are the grand mean and can be used as a visual guide to compare the distribution of data points under various transformations (across rows). Converting to ROD has little effect on the statistical properties of the data, while converting to units of radioactivity badly skewed the data, created outliers under certain conditions, and leads to decreased power (see text for details). This is highlighted in column four where the t-values for the radioactivity data (blue line) are shifted to the left for all three datasets.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC2651176&req=5

Figure 7: Effect of transformations on simulated data. Data in the first column are the simulated grey level values from various distributions. Note that the y-axis is the same in these three panels so the data can be compared directly. A1 has a large difference between means and low variability, leading to no overlap between data points. B1 has similar variability as A1 but the difference in means has been decreased by half, leading to overlapping values between groups. C1 has the same difference in means as B1, but much greater variability. Horizontal grey lines are the grand mean and can be used as a visual guide to compare the distribution of data points under various transformations (across rows). Converting to ROD has little effect on the statistical properties of the data, while converting to units of radioactivity badly skewed the data, created outliers under certain conditions, and leads to decreased power (see text for details). This is highlighted in column four where the t-values for the radioactivity data (blue line) are shifted to the left for all three datasets.

Mentions: It is common for grey levels to be converted into optical densities or expressed as units of radioactivity. The purpose of these conversions is ostensibly to account for the nonlinear relationship between the transmittance of light through the film (Beer's Law) and for the nonlinear relationship between the darkness of the film and the number of particles striking the film from radioactive decay. Because these transformations are nonlinear, they have the effect of making high values in the data disproportionately higher. Figure 6 displays the effect of transforming GL values into relative optical densities (Eq. 2), calibrated optical densities (Eq. 4) or calibrated units of radioactivity (Eq. 5). While these are nonlinear transformations, the range of the observed experimental values in the present study was narrow compared to the range of possible values (approximately 8% of the range). This will likely be true for many studies, where the GL values between conditions will be within a fairly narrow range. If this is the case, then transforming the values is pointless (it is the equivalent of converting from degrees Celsius to degrees Fahrenheit, and performing statistical analysis on the converted data); and if the GL values have a wider range, then such transformations skew the distribution and/or create outliers as demonstrated below. In order to assess the effect of these transformations on the statistical properties of the data, three datasets with different characteristics were simulated, and the results displayed in Figure 7. Data for two groups (A and B) were drawn from normal distributions with n = 15 in each group; the parameters of the distributions are shown in the figure. The two groups were analysed with a two-tailed independent samples t-test with Welch's correction for unequal variances [40]. The t-value provides a useful metric to compare the effect of various transformations, as it reflects the differences between the means of the groups divided by the variability (note that Welch's correction adjusts the degrees of freedom and not the test statistic). The A-series data was constructed to have a large difference between the means of the two groups and low variability in each group, such that there is no overlap in the distributions. In such a case statistical inference is hardly necessary and t = 13.5. Transforming the values to ROD (Eq.2; A2) does little in the way of changing the result of the statistical analysis (t = 11.3), or one's subjective impression of the plotted data. Converting to units of radioactivity however (Eq. 5; A3) skewed the distribution and made the variance of B sixteen times bigger than A (Fligner-Killeen test: = 8.1, p = 0.004). This is to be expected of such nonlinear transformations where the original grey level values cover a wide range. While visually it may seem that group A and B are now 'more different', the t-statistic has become much smaller (t = 10.3) due to the increased variability, indicating a smaller effect. One thousand datasets were randomly generated with the above parameters and the t-statistic was calculated for each. The results are plotted in panel A4 where it can be seen that the distribution of t-statistics does not change when converting to ROD, but are shifted to the left when transforming grey levels to units of radioactivity, indicating reduced power of any statistical analysis. All of these would still be significant, as the t-statistics are large, but it makes the point that such transformations can reduce the power of subsequent analyses. Given that the group with the higher mean has the higher variance in A3, it is common to deal with this type of data by log transformation. The irony is that this reverses the operation of converting GL values to units of radioactivity, which was an exponential transformation. Alternatively, a non-parametric test could be used (e.g. Wilcoxon/Mann-Whitney), but the results would be identical for all three (GL, ROD, radioactivity), because the analysis is done on the ranks and not on the raw values, and all of these transformations will have no effect on the rank ordering of the data. With such data it would appear that performing the analysis on the GL values is perferable.


Statistical evaluation of methods for quantifying gene expression by autoradiography in histological sections.

Lazic SE - BMC Neurosci (2009)

Effect of transformations on simulated data. Data in the first column are the simulated grey level values from various distributions. Note that the y-axis is the same in these three panels so the data can be compared directly. A1 has a large difference between means and low variability, leading to no overlap between data points. B1 has similar variability as A1 but the difference in means has been decreased by half, leading to overlapping values between groups. C1 has the same difference in means as B1, but much greater variability. Horizontal grey lines are the grand mean and can be used as a visual guide to compare the distribution of data points under various transformations (across rows). Converting to ROD has little effect on the statistical properties of the data, while converting to units of radioactivity badly skewed the data, created outliers under certain conditions, and leads to decreased power (see text for details). This is highlighted in column four where the t-values for the radioactivity data (blue line) are shifted to the left for all three datasets.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC2651176&req=5

Figure 7: Effect of transformations on simulated data. Data in the first column are the simulated grey level values from various distributions. Note that the y-axis is the same in these three panels so the data can be compared directly. A1 has a large difference between means and low variability, leading to no overlap between data points. B1 has similar variability as A1 but the difference in means has been decreased by half, leading to overlapping values between groups. C1 has the same difference in means as B1, but much greater variability. Horizontal grey lines are the grand mean and can be used as a visual guide to compare the distribution of data points under various transformations (across rows). Converting to ROD has little effect on the statistical properties of the data, while converting to units of radioactivity badly skewed the data, created outliers under certain conditions, and leads to decreased power (see text for details). This is highlighted in column four where the t-values for the radioactivity data (blue line) are shifted to the left for all three datasets.
Mentions: It is common for grey levels to be converted into optical densities or expressed as units of radioactivity. The purpose of these conversions is ostensibly to account for the nonlinear relationship between the transmittance of light through the film (Beer's Law) and for the nonlinear relationship between the darkness of the film and the number of particles striking the film from radioactive decay. Because these transformations are nonlinear, they have the effect of making high values in the data disproportionately higher. Figure 6 displays the effect of transforming GL values into relative optical densities (Eq. 2), calibrated optical densities (Eq. 4) or calibrated units of radioactivity (Eq. 5). While these are nonlinear transformations, the range of the observed experimental values in the present study was narrow compared to the range of possible values (approximately 8% of the range). This will likely be true for many studies, where the GL values between conditions will be within a fairly narrow range. If this is the case, then transforming the values is pointless (it is the equivalent of converting from degrees Celsius to degrees Fahrenheit, and performing statistical analysis on the converted data); and if the GL values have a wider range, then such transformations skew the distribution and/or create outliers as demonstrated below. In order to assess the effect of these transformations on the statistical properties of the data, three datasets with different characteristics were simulated, and the results displayed in Figure 7. Data for two groups (A and B) were drawn from normal distributions with n = 15 in each group; the parameters of the distributions are shown in the figure. The two groups were analysed with a two-tailed independent samples t-test with Welch's correction for unequal variances [40]. The t-value provides a useful metric to compare the effect of various transformations, as it reflects the differences between the means of the groups divided by the variability (note that Welch's correction adjusts the degrees of freedom and not the test statistic). The A-series data was constructed to have a large difference between the means of the two groups and low variability in each group, such that there is no overlap in the distributions. In such a case statistical inference is hardly necessary and t = 13.5. Transforming the values to ROD (Eq.2; A2) does little in the way of changing the result of the statistical analysis (t = 11.3), or one's subjective impression of the plotted data. Converting to units of radioactivity however (Eq. 5; A3) skewed the distribution and made the variance of B sixteen times bigger than A (Fligner-Killeen test: = 8.1, p = 0.004). This is to be expected of such nonlinear transformations where the original grey level values cover a wide range. While visually it may seem that group A and B are now 'more different', the t-statistic has become much smaller (t = 10.3) due to the increased variability, indicating a smaller effect. One thousand datasets were randomly generated with the above parameters and the t-statistic was calculated for each. The results are plotted in panel A4 where it can be seen that the distribution of t-statistics does not change when converting to ROD, but are shifted to the left when transforming grey levels to units of radioactivity, indicating reduced power of any statistical analysis. All of these would still be significant, as the t-statistics are large, but it makes the point that such transformations can reduce the power of subsequent analyses. Given that the group with the higher mean has the higher variance in A3, it is common to deal with this type of data by log transformation. The irony is that this reverses the operation of converting GL values to units of radioactivity, which was an exponential transformation. Alternatively, a non-parametric test could be used (e.g. Wilcoxon/Mann-Whitney), but the results would be identical for all three (GL, ROD, radioactivity), because the analysis is done on the ranks and not on the raw values, and all of these transformations will have no effect on the rank ordering of the data. With such data it would appear that performing the analysis on the GL values is perferable.

Bottom Line: Image segmentation based on thresholding can be subject to floor-effects and lead to biased results.Finally, converting grey level pixel intensities to optical densities or units of radioactivity is unnecessary for most applications and can lead to data with poor statistical properties.Based on these results, suggestions are made to reduce bias, increase precision, and ultimately provide more meaningful results of gene expression data.

View Article: PubMed Central - HTML - PubMed

Affiliation: Department of Applied Mathematics and Theoretical Physics, Cambridge Computational Biology Institute, University of Cambridge, Cambridge, UK. stan.lazic@cantab.net

ABSTRACT

Background: In situ hybridisation (ISH) combined with autoradiography is a standard method of measuring the amount of gene expression in histological sections, but the methods used to quantify gene expression in the resulting digital images vary greatly between studies and can potentially give conflicting results.

Results: The present study examines commonly used methods for analysing ISH images and demonstrates that these methods are not optimal. Image segmentation based on thresholding can be subject to floor-effects and lead to biased results. In addition, including the area of the structure or region of interest in the calculation of gene expression can lead to a large loss of precision and can also introduce bias. Finally, converting grey level pixel intensities to optical densities or units of radioactivity is unnecessary for most applications and can lead to data with poor statistical properties. A modification of an existing method for selecting the structure or region of interest is introduced which performs better than alternative methods in terms of bias and precision.

Conclusion: Based on these results, suggestions are made to reduce bias, increase precision, and ultimately provide more meaningful results of gene expression data.

Show MeSH
Related in: MedlinePlus