Limits...
Defining signal thresholds in DNA microarrays: exemplary application for invasive cancer.

Bilban M, Buehler LK, Head S, Desoye G, Quaranta V - BMC Genomics (2002)

Bottom Line: Most strategies have used fold-change threshold values, but variability at low signal intensities may invalidate this approach and it does not provide information about false-positives and false negatives.It demonstrated that ROC analysis yields a threshold with reduced missclassified genes in microarray experiments.The proposed method is applicable to both custom made and commercially available DNA microarrays and will help to improve the reliability of predictions from DNA microarray experiments.

View Article: PubMed Central - HTML - PubMed

Affiliation: The Scripps Research Institute, Department of Cell Biology, 10550 North Torrey Pines Road, La Jolla, CA, USA. mbilban@scripps.edu

ABSTRACT

Background: Genome-wide or application-targeted microarrays containing a subset of genes of interest have become widely used as a research tool with the prospect of diagnostic application. Intrinsic variability of microarray measurements poses a major problem in defining signal thresholds for absent/present or differentially expressed genes. Most strategies have used fold-change threshold values, but variability at low signal intensities may invalidate this approach and it does not provide information about false-positives and false negatives.

Results: We introduce a method to filter false-positives and false-negatives from DNA microarray experiments. This is achieved by evaluating a set of positive and negative controls by receiver operating characteristic (ROC) analysis. As an advantage of this approach, users may define thresholds on the basis of sensitivity and specificity considerations. The area under the ROC curve allows quality control of microarray hybridizations. This method has been applied to custom made microarrays developed for the analysis of invasive melanoma derived tumor cells. It demonstrated that ROC analysis yields a threshold with reduced missclassified genes in microarray experiments.

Conclusions: Provided that a set of appropriate positive and negative controls is included on the microarray, ROC analysis obviates the inherent problem of arbitrarily selecting threshold levels in microarray experiments. The proposed method is applicable to both custom made and commercially available DNA microarrays and will help to improve the reliability of predictions from DNA microarray experiments.

No MeSH data available.


Related in: MedlinePlus

ROC analysis of selected signal cut-off values as a predictor for specific hybridization. ROC curves demonstrate the capacity to discriminate between the absence or presence of sequence-specific hybridization in individual microarray experiments. The closer an ROC curve is to the upper left hand corner of the graph, the more accurate it is because the true positive rate is 100% and the false positive rate is 0%. ROC plots based on percentile rank calculations for 25 cut-off signal thresholds (taken from table 1). The meaning of the position of thresholds a-d (table 1) are explained in the text. The area under the ROC curve was (A) 0.994 (array 1) and (B) 0.999 (array 2). Rising diagonal indicates no discrimination between positiv and negative control signals.
© Copyright Policy
Related In: Results  -  Collection


getmorefigures.php?uid=PMC117791&req=5

Figure 3: ROC analysis of selected signal cut-off values as a predictor for specific hybridization. ROC curves demonstrate the capacity to discriminate between the absence or presence of sequence-specific hybridization in individual microarray experiments. The closer an ROC curve is to the upper left hand corner of the graph, the more accurate it is because the true positive rate is 100% and the false positive rate is 0%. ROC plots based on percentile rank calculations for 25 cut-off signal thresholds (taken from table 1). The meaning of the position of thresholds a-d (table 1) are explained in the text. The area under the ROC curve was (A) 0.994 (array 1) and (B) 0.999 (array 2). Rising diagonal indicates no discrimination between positiv and negative control signals.

Mentions: Traditionally, cut-offs for microarrays have been calculated from negative controls. Here, we compare three widely used thresholds (TX = mean ; T0.5X = median; TX2SD = mean + 2 standard deviations) to the ROC- analyses derived threshold (TM = the threshold with maximum specificity and senitivity (TM) obtained as the point of intersection in figure 2) in terms of specificity (Sp) and sensitivity (Se) (Table 1). For TX and T0.5X the Se is ≅ 100%, however, the Sp would only be ≅ 50% indicating that approximately every second signal would be a false positive arising from non-specific hybridization. As a benefit, however, >99% of test genes would be included in data analyses (Table 1). If TX2SD or TM is the desired threshold, the Sp can be increased to >95% with only minor reductions in Se, however, the trade-off is an increase in the number of genes excluded from analyses (Table 1). If Sp and Se are plotted as a function of the thresholds, the intersection point of the two curves indicates maximum Sp and Se which can be directly read from the graph (Figure 2). As the criterion for a positive test becomes more stringent, the point on the curve corresponding to Sp and Se (point c, Figure 3a; point d, Figure 3b) moves down and to the left (lower Se, higher Sp); if less evidence is required for a positive test, the point on the curve corresponding to Sp and Se (point a, Figure 3a and 3b) moves up and to the right (lower Sp, higher Se). The area under the ROC curve (Figure 3, Table 2) is a measure of how well positive and negative signal can be distinguished in individual microarray experiments and indicates hybridization quality.


Defining signal thresholds in DNA microarrays: exemplary application for invasive cancer.

Bilban M, Buehler LK, Head S, Desoye G, Quaranta V - BMC Genomics (2002)

ROC analysis of selected signal cut-off values as a predictor for specific hybridization. ROC curves demonstrate the capacity to discriminate between the absence or presence of sequence-specific hybridization in individual microarray experiments. The closer an ROC curve is to the upper left hand corner of the graph, the more accurate it is because the true positive rate is 100% and the false positive rate is 0%. ROC plots based on percentile rank calculations for 25 cut-off signal thresholds (taken from table 1). The meaning of the position of thresholds a-d (table 1) are explained in the text. The area under the ROC curve was (A) 0.994 (array 1) and (B) 0.999 (array 2). Rising diagonal indicates no discrimination between positiv and negative control signals.
© Copyright Policy
Related In: Results  -  Collection

Show All Figures
getmorefigures.php?uid=PMC117791&req=5

Figure 3: ROC analysis of selected signal cut-off values as a predictor for specific hybridization. ROC curves demonstrate the capacity to discriminate between the absence or presence of sequence-specific hybridization in individual microarray experiments. The closer an ROC curve is to the upper left hand corner of the graph, the more accurate it is because the true positive rate is 100% and the false positive rate is 0%. ROC plots based on percentile rank calculations for 25 cut-off signal thresholds (taken from table 1). The meaning of the position of thresholds a-d (table 1) are explained in the text. The area under the ROC curve was (A) 0.994 (array 1) and (B) 0.999 (array 2). Rising diagonal indicates no discrimination between positiv and negative control signals.
Mentions: Traditionally, cut-offs for microarrays have been calculated from negative controls. Here, we compare three widely used thresholds (TX = mean ; T0.5X = median; TX2SD = mean + 2 standard deviations) to the ROC- analyses derived threshold (TM = the threshold with maximum specificity and senitivity (TM) obtained as the point of intersection in figure 2) in terms of specificity (Sp) and sensitivity (Se) (Table 1). For TX and T0.5X the Se is ≅ 100%, however, the Sp would only be ≅ 50% indicating that approximately every second signal would be a false positive arising from non-specific hybridization. As a benefit, however, >99% of test genes would be included in data analyses (Table 1). If TX2SD or TM is the desired threshold, the Sp can be increased to >95% with only minor reductions in Se, however, the trade-off is an increase in the number of genes excluded from analyses (Table 1). If Sp and Se are plotted as a function of the thresholds, the intersection point of the two curves indicates maximum Sp and Se which can be directly read from the graph (Figure 2). As the criterion for a positive test becomes more stringent, the point on the curve corresponding to Sp and Se (point c, Figure 3a; point d, Figure 3b) moves down and to the left (lower Se, higher Sp); if less evidence is required for a positive test, the point on the curve corresponding to Sp and Se (point a, Figure 3a and 3b) moves up and to the right (lower Sp, higher Se). The area under the ROC curve (Figure 3, Table 2) is a measure of how well positive and negative signal can be distinguished in individual microarray experiments and indicates hybridization quality.

Bottom Line: Most strategies have used fold-change threshold values, but variability at low signal intensities may invalidate this approach and it does not provide information about false-positives and false negatives.It demonstrated that ROC analysis yields a threshold with reduced missclassified genes in microarray experiments.The proposed method is applicable to both custom made and commercially available DNA microarrays and will help to improve the reliability of predictions from DNA microarray experiments.

View Article: PubMed Central - HTML - PubMed

Affiliation: The Scripps Research Institute, Department of Cell Biology, 10550 North Torrey Pines Road, La Jolla, CA, USA. mbilban@scripps.edu

ABSTRACT

Background: Genome-wide or application-targeted microarrays containing a subset of genes of interest have become widely used as a research tool with the prospect of diagnostic application. Intrinsic variability of microarray measurements poses a major problem in defining signal thresholds for absent/present or differentially expressed genes. Most strategies have used fold-change threshold values, but variability at low signal intensities may invalidate this approach and it does not provide information about false-positives and false negatives.

Results: We introduce a method to filter false-positives and false-negatives from DNA microarray experiments. This is achieved by evaluating a set of positive and negative controls by receiver operating characteristic (ROC) analysis. As an advantage of this approach, users may define thresholds on the basis of sensitivity and specificity considerations. The area under the ROC curve allows quality control of microarray hybridizations. This method has been applied to custom made microarrays developed for the analysis of invasive melanoma derived tumor cells. It demonstrated that ROC analysis yields a threshold with reduced missclassified genes in microarray experiments.

Conclusions: Provided that a set of appropriate positive and negative controls is included on the microarray, ROC analysis obviates the inherent problem of arbitrarily selecting threshold levels in microarray experiments. The proposed method is applicable to both custom made and commercially available DNA microarrays and will help to improve the reliability of predictions from DNA microarray experiments.

No MeSH data available.


Related in: MedlinePlus