Limits...
Statistical evaluation of methods for quantifying gene expression by autoradiography in histological sections.

Lazic SE - BMC Neurosci (2009)

Bottom Line: Image segmentation based on thresholding can be subject to floor-effects and lead to biased results.Finally, converting grey level pixel intensities to optical densities or units of radioactivity is unnecessary for most applications and can lead to data with poor statistical properties.Based on these results, suggestions are made to reduce bias, increase precision, and ultimately provide more meaningful results of gene expression data.

View Article: PubMed Central - HTML - PubMed

Affiliation: Department of Applied Mathematics and Theoretical Physics, Cambridge Computational Biology Institute, University of Cambridge, Cambridge, UK. stan.lazic@cantab.net

ABSTRACT

Background: In situ hybridisation (ISH) combined with autoradiography is a standard method of measuring the amount of gene expression in histological sections, but the methods used to quantify gene expression in the resulting digital images vary greatly between studies and can potentially give conflicting results.

Results: The present study examines commonly used methods for analysing ISH images and demonstrates that these methods are not optimal. Image segmentation based on thresholding can be subject to floor-effects and lead to biased results. In addition, including the area of the structure or region of interest in the calculation of gene expression can lead to a large loss of precision and can also introduce bias. Finally, converting grey level pixel intensities to optical densities or units of radioactivity is unnecessary for most applications and can lead to data with poor statistical properties. A modification of an existing method for selecting the structure or region of interest is introduced which performs better than alternative methods in terms of bias and precision.

Conclusion: Based on these results, suggestions are made to reduce bias, increase precision, and ultimately provide more meaningful results of gene expression data.

Show MeSH

Related in: MedlinePlus

Transformation of grey levels. Grey levels from 0 to 255 were transformed using three different methods (A). The uncalibrated relative OD and the Rodbard OD method are plotted on the left y-axis, while transformation with the 14C-standard uses the right y-axis. The box indicates the range of observed grey level values in this dataset and are re-plotted in (B). All three methods give different curves, however within the range of actual values, the transformation is fairly linear, especially for the relative and Rodbard OD transforms.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC2651176&req=5

Figure 6: Transformation of grey levels. Grey levels from 0 to 255 were transformed using three different methods (A). The uncalibrated relative OD and the Rodbard OD method are plotted on the left y-axis, while transformation with the 14C-standard uses the right y-axis. The box indicates the range of observed grey level values in this dataset and are re-plotted in (B). All three methods give different curves, however within the range of actual values, the transformation is fairly linear, especially for the relative and Rodbard OD transforms.

Mentions: It is common for grey levels to be converted into optical densities or expressed as units of radioactivity. The purpose of these conversions is ostensibly to account for the nonlinear relationship between the transmittance of light through the film (Beer's Law) and for the nonlinear relationship between the darkness of the film and the number of particles striking the film from radioactive decay. Because these transformations are nonlinear, they have the effect of making high values in the data disproportionately higher. Figure 6 displays the effect of transforming GL values into relative optical densities (Eq. 2), calibrated optical densities (Eq. 4) or calibrated units of radioactivity (Eq. 5). While these are nonlinear transformations, the range of the observed experimental values in the present study was narrow compared to the range of possible values (approximately 8% of the range). This will likely be true for many studies, where the GL values between conditions will be within a fairly narrow range. If this is the case, then transforming the values is pointless (it is the equivalent of converting from degrees Celsius to degrees Fahrenheit, and performing statistical analysis on the converted data); and if the GL values have a wider range, then such transformations skew the distribution and/or create outliers as demonstrated below. In order to assess the effect of these transformations on the statistical properties of the data, three datasets with different characteristics were simulated, and the results displayed in Figure 7. Data for two groups (A and B) were drawn from normal distributions with n = 15 in each group; the parameters of the distributions are shown in the figure. The two groups were analysed with a two-tailed independent samples t-test with Welch's correction for unequal variances [40]. The t-value provides a useful metric to compare the effect of various transformations, as it reflects the differences between the means of the groups divided by the variability (note that Welch's correction adjusts the degrees of freedom and not the test statistic). The A-series data was constructed to have a large difference between the means of the two groups and low variability in each group, such that there is no overlap in the distributions. In such a case statistical inference is hardly necessary and t = 13.5. Transforming the values to ROD (Eq.2; A2) does little in the way of changing the result of the statistical analysis (t = 11.3), or one's subjective impression of the plotted data. Converting to units of radioactivity however (Eq. 5; A3) skewed the distribution and made the variance of B sixteen times bigger than A (Fligner-Killeen test: = 8.1, p = 0.004). This is to be expected of such nonlinear transformations where the original grey level values cover a wide range. While visually it may seem that group A and B are now 'more different', the t-statistic has become much smaller (t = 10.3) due to the increased variability, indicating a smaller effect. One thousand datasets were randomly generated with the above parameters and the t-statistic was calculated for each. The results are plotted in panel A4 where it can be seen that the distribution of t-statistics does not change when converting to ROD, but are shifted to the left when transforming grey levels to units of radioactivity, indicating reduced power of any statistical analysis. All of these would still be significant, as the t-statistics are large, but it makes the point that such transformations can reduce the power of subsequent analyses. Given that the group with the higher mean has the higher variance in A3, it is common to deal with this type of data by log transformation. The irony is that this reverses the operation of converting GL values to units of radioactivity, which was an exponential transformation. Alternatively, a non-parametric test could be used (e.g. Wilcoxon/Mann-Whitney), but the results would be identical for all three (GL, ROD, radioactivity), because the analysis is done on the ranks and not on the raw values, and all of these transformations will have no effect on the rank ordering of the data. With such data it would appear that performing the analysis on the GL values is perferable.


Statistical evaluation of methods for quantifying gene expression by autoradiography in histological sections.

Lazic SE - BMC Neurosci (2009)

Transformation of grey levels. Grey levels from 0 to 255 were transformed using three different methods (A). The uncalibrated relative OD and the Rodbard OD method are plotted on the left y-axis, while transformation with the 14C-standard uses the right y-axis. The box indicates the range of observed grey level values in this dataset and are re-plotted in (B). All three methods give different curves, however within the range of actual values, the transformation is fairly linear, especially for the relative and Rodbard OD transforms.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC2651176&req=5

Figure 6: Transformation of grey levels. Grey levels from 0 to 255 were transformed using three different methods (A). The uncalibrated relative OD and the Rodbard OD method are plotted on the left y-axis, while transformation with the 14C-standard uses the right y-axis. The box indicates the range of observed grey level values in this dataset and are re-plotted in (B). All three methods give different curves, however within the range of actual values, the transformation is fairly linear, especially for the relative and Rodbard OD transforms.
Mentions: It is common for grey levels to be converted into optical densities or expressed as units of radioactivity. The purpose of these conversions is ostensibly to account for the nonlinear relationship between the transmittance of light through the film (Beer's Law) and for the nonlinear relationship between the darkness of the film and the number of particles striking the film from radioactive decay. Because these transformations are nonlinear, they have the effect of making high values in the data disproportionately higher. Figure 6 displays the effect of transforming GL values into relative optical densities (Eq. 2), calibrated optical densities (Eq. 4) or calibrated units of radioactivity (Eq. 5). While these are nonlinear transformations, the range of the observed experimental values in the present study was narrow compared to the range of possible values (approximately 8% of the range). This will likely be true for many studies, where the GL values between conditions will be within a fairly narrow range. If this is the case, then transforming the values is pointless (it is the equivalent of converting from degrees Celsius to degrees Fahrenheit, and performing statistical analysis on the converted data); and if the GL values have a wider range, then such transformations skew the distribution and/or create outliers as demonstrated below. In order to assess the effect of these transformations on the statistical properties of the data, three datasets with different characteristics were simulated, and the results displayed in Figure 7. Data for two groups (A and B) were drawn from normal distributions with n = 15 in each group; the parameters of the distributions are shown in the figure. The two groups were analysed with a two-tailed independent samples t-test with Welch's correction for unequal variances [40]. The t-value provides a useful metric to compare the effect of various transformations, as it reflects the differences between the means of the groups divided by the variability (note that Welch's correction adjusts the degrees of freedom and not the test statistic). The A-series data was constructed to have a large difference between the means of the two groups and low variability in each group, such that there is no overlap in the distributions. In such a case statistical inference is hardly necessary and t = 13.5. Transforming the values to ROD (Eq.2; A2) does little in the way of changing the result of the statistical analysis (t = 11.3), or one's subjective impression of the plotted data. Converting to units of radioactivity however (Eq. 5; A3) skewed the distribution and made the variance of B sixteen times bigger than A (Fligner-Killeen test: = 8.1, p = 0.004). This is to be expected of such nonlinear transformations where the original grey level values cover a wide range. While visually it may seem that group A and B are now 'more different', the t-statistic has become much smaller (t = 10.3) due to the increased variability, indicating a smaller effect. One thousand datasets were randomly generated with the above parameters and the t-statistic was calculated for each. The results are plotted in panel A4 where it can be seen that the distribution of t-statistics does not change when converting to ROD, but are shifted to the left when transforming grey levels to units of radioactivity, indicating reduced power of any statistical analysis. All of these would still be significant, as the t-statistics are large, but it makes the point that such transformations can reduce the power of subsequent analyses. Given that the group with the higher mean has the higher variance in A3, it is common to deal with this type of data by log transformation. The irony is that this reverses the operation of converting GL values to units of radioactivity, which was an exponential transformation. Alternatively, a non-parametric test could be used (e.g. Wilcoxon/Mann-Whitney), but the results would be identical for all three (GL, ROD, radioactivity), because the analysis is done on the ranks and not on the raw values, and all of these transformations will have no effect on the rank ordering of the data. With such data it would appear that performing the analysis on the GL values is perferable.

Bottom Line: Image segmentation based on thresholding can be subject to floor-effects and lead to biased results.Finally, converting grey level pixel intensities to optical densities or units of radioactivity is unnecessary for most applications and can lead to data with poor statistical properties.Based on these results, suggestions are made to reduce bias, increase precision, and ultimately provide more meaningful results of gene expression data.

View Article: PubMed Central - HTML - PubMed

Affiliation: Department of Applied Mathematics and Theoretical Physics, Cambridge Computational Biology Institute, University of Cambridge, Cambridge, UK. stan.lazic@cantab.net

ABSTRACT

Background: In situ hybridisation (ISH) combined with autoradiography is a standard method of measuring the amount of gene expression in histological sections, but the methods used to quantify gene expression in the resulting digital images vary greatly between studies and can potentially give conflicting results.

Results: The present study examines commonly used methods for analysing ISH images and demonstrates that these methods are not optimal. Image segmentation based on thresholding can be subject to floor-effects and lead to biased results. In addition, including the area of the structure or region of interest in the calculation of gene expression can lead to a large loss of precision and can also introduce bias. Finally, converting grey level pixel intensities to optical densities or units of radioactivity is unnecessary for most applications and can lead to data with poor statistical properties. A modification of an existing method for selecting the structure or region of interest is introduced which performs better than alternative methods in terms of bias and precision.

Conclusion: Based on these results, suggestions are made to reduce bias, increase precision, and ultimately provide more meaningful results of gene expression data.

Show MeSH
Related in: MedlinePlus