Limits...
Receiver operating characteristic (ROC) curve: practical review for radiologists.

Park SH, Goo JM, Jo CH - Korean J Radiol (2004 Jan-Mar)

Bottom Line: Important concepts involved in the correct use and interpretation of this analysis, such as smooth and empirical ROC curves, parametric and nonparametric methods, the area under the ROC curve and its 95% confidence interval, the sensitivity at a particular FPR, and the use of a partial area under the ROC curve are discussed.Various considerations concerning the collection of data in radiological ROC studies are briefly discussed.An introduction to the software frequently used for performing ROC analyses is also presented.

View Article: PubMed Central - PubMed

Affiliation: Department of Radiology, Seoul National University College of Medicine and Institute of Radiation Medicine, SNUMRC, Seoul, Korea.

ABSTRACT
The receiver operating characteristic (ROC) curve, which is defined as a plot of test sensitivity as they coordinate versus its 1-specificity or false positive rate (FPR) as the x coordinate, is an effective method of evaluating the performance of diagnostic tests. The purpose of this article is to provide a nonmathematical introduction to ROC analysis. Important concepts involved in the correct use and interpretation of this analysis, such as smooth and empirical ROC curves, parametric and nonparametric methods, the area under the ROC curve and its 95% confidence interval, the sensitivity at a particular FPR, and the use of a partial area under the ROC curve are discussed. Various considerations concerning the collection of data in radiological ROC studies are briefly discussed. An introduction to the software frequently used for performing ROC analyses is also presented.

Show MeSH

Related in: MedlinePlus

ROC curves from a plain chest radiography study of 70 patients with solitary pulmonary nodules (Table 3).A. A plot of test sensitivity (y coordinate) versus its false positive rate (x coordinate) obtained at each cutoff level.B. The fitted or smooth ROC curve that is estimated with the assumption of binormal distribution. The parametric estimate of the area under the smooth ROC curve and its 95% confidence interval are 0.734 and 0.602 ~ 0.839, respectively.C. The empirical ROC curve. The discrete points on the empirical ROC curve are marked with dots. The nonparametric estimate of the area under the empirical ROC curve and its 95% confidence interval are 0.728 and 0.608 ~ 0.827, respectively. The nonparametric estimate of the area under the empirical ROC curve is the summation of the areas of the trapezoids formed by connecting the points on the ROC curve.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC2698108&req=5

Figure 1: ROC curves from a plain chest radiography study of 70 patients with solitary pulmonary nodules (Table 3).A. A plot of test sensitivity (y coordinate) versus its false positive rate (x coordinate) obtained at each cutoff level.B. The fitted or smooth ROC curve that is estimated with the assumption of binormal distribution. The parametric estimate of the area under the smooth ROC curve and its 95% confidence interval are 0.734 and 0.602 ~ 0.839, respectively.C. The empirical ROC curve. The discrete points on the empirical ROC curve are marked with dots. The nonparametric estimate of the area under the empirical ROC curve and its 95% confidence interval are 0.728 and 0.608 ~ 0.827, respectively. The nonparametric estimate of the area under the empirical ROC curve is the summation of the areas of the trapezoids formed by connecting the points on the ROC curve.

Mentions: To deal with these multiple pairs of sensitivity and specificity values, one can draw a graph using the sensitivities as the y coordinates and the 1-specificities or FPRs as the x coordinates (Fig. 1A). Each discrete point on the graph, called an operating point, is generated by using different cutoff levels for a positive test result. An ROC curve can be estimated from these discrete points, by making the assumption that the test results, or some unknown monotonic transformation thereof, follow a certain distribution. For this purpose, the assumption of a binormal distribution (i.e., two Gaussian distributions: one for the test results of those patients with benign solitary pulmonary nodules and the other for the test results of those patients with malignant solitary pulmonary nodules) is most commonly made (1, 2). The resulting curve is called the fitted or smooth ROC curve (Fig. 1B) (1). The estimation of the smooth ROC curve based on a binormal distribution uses a statistical method called maximum likelihood estimation (MLE) (3). When a binormal distribution is used, the shape of the smooth ROC curve is entirely determined by two parameters. The first one, which is referred to as a, is the standardized difference in the means of the distributions of the test results for those subjects with and without the condition (Appendix) (2, 4). The other parameter, which is referred to as b, is the ratio of the standard deviations of the distributions of the test results for those subjects without versus those with the condition (Appendix) (2, 4). Another way to construct an ROC curve is to connect all the points obtained at all the possible cutoff levels. In the previous example, there are four pairs of FPR and sensitivity values (Table 3), and the two endpoints on the ROC curve are 0, 0 and 1, 1 with each pair of values corresponding to the FPR and sensitivity, respectively. The resulting ROC curve is called the empirical ROC curve (Fig. 1C) (1). The ROC curve illustrates the relationship between sensitivity and FPR. Because the ROC curve displays the sensitivities and FPRs at all possible cutoff levels, it can be used to assess the performance of a test independently of the decision threshold (5).


Receiver operating characteristic (ROC) curve: practical review for radiologists.

Park SH, Goo JM, Jo CH - Korean J Radiol (2004 Jan-Mar)

ROC curves from a plain chest radiography study of 70 patients with solitary pulmonary nodules (Table 3).A. A plot of test sensitivity (y coordinate) versus its false positive rate (x coordinate) obtained at each cutoff level.B. The fitted or smooth ROC curve that is estimated with the assumption of binormal distribution. The parametric estimate of the area under the smooth ROC curve and its 95% confidence interval are 0.734 and 0.602 ~ 0.839, respectively.C. The empirical ROC curve. The discrete points on the empirical ROC curve are marked with dots. The nonparametric estimate of the area under the empirical ROC curve and its 95% confidence interval are 0.728 and 0.608 ~ 0.827, respectively. The nonparametric estimate of the area under the empirical ROC curve is the summation of the areas of the trapezoids formed by connecting the points on the ROC curve.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC2698108&req=5

Figure 1: ROC curves from a plain chest radiography study of 70 patients with solitary pulmonary nodules (Table 3).A. A plot of test sensitivity (y coordinate) versus its false positive rate (x coordinate) obtained at each cutoff level.B. The fitted or smooth ROC curve that is estimated with the assumption of binormal distribution. The parametric estimate of the area under the smooth ROC curve and its 95% confidence interval are 0.734 and 0.602 ~ 0.839, respectively.C. The empirical ROC curve. The discrete points on the empirical ROC curve are marked with dots. The nonparametric estimate of the area under the empirical ROC curve and its 95% confidence interval are 0.728 and 0.608 ~ 0.827, respectively. The nonparametric estimate of the area under the empirical ROC curve is the summation of the areas of the trapezoids formed by connecting the points on the ROC curve.
Mentions: To deal with these multiple pairs of sensitivity and specificity values, one can draw a graph using the sensitivities as the y coordinates and the 1-specificities or FPRs as the x coordinates (Fig. 1A). Each discrete point on the graph, called an operating point, is generated by using different cutoff levels for a positive test result. An ROC curve can be estimated from these discrete points, by making the assumption that the test results, or some unknown monotonic transformation thereof, follow a certain distribution. For this purpose, the assumption of a binormal distribution (i.e., two Gaussian distributions: one for the test results of those patients with benign solitary pulmonary nodules and the other for the test results of those patients with malignant solitary pulmonary nodules) is most commonly made (1, 2). The resulting curve is called the fitted or smooth ROC curve (Fig. 1B) (1). The estimation of the smooth ROC curve based on a binormal distribution uses a statistical method called maximum likelihood estimation (MLE) (3). When a binormal distribution is used, the shape of the smooth ROC curve is entirely determined by two parameters. The first one, which is referred to as a, is the standardized difference in the means of the distributions of the test results for those subjects with and without the condition (Appendix) (2, 4). The other parameter, which is referred to as b, is the ratio of the standard deviations of the distributions of the test results for those subjects without versus those with the condition (Appendix) (2, 4). Another way to construct an ROC curve is to connect all the points obtained at all the possible cutoff levels. In the previous example, there are four pairs of FPR and sensitivity values (Table 3), and the two endpoints on the ROC curve are 0, 0 and 1, 1 with each pair of values corresponding to the FPR and sensitivity, respectively. The resulting ROC curve is called the empirical ROC curve (Fig. 1C) (1). The ROC curve illustrates the relationship between sensitivity and FPR. Because the ROC curve displays the sensitivities and FPRs at all possible cutoff levels, it can be used to assess the performance of a test independently of the decision threshold (5).

Bottom Line: Important concepts involved in the correct use and interpretation of this analysis, such as smooth and empirical ROC curves, parametric and nonparametric methods, the area under the ROC curve and its 95% confidence interval, the sensitivity at a particular FPR, and the use of a partial area under the ROC curve are discussed.Various considerations concerning the collection of data in radiological ROC studies are briefly discussed.An introduction to the software frequently used for performing ROC analyses is also presented.

View Article: PubMed Central - PubMed

Affiliation: Department of Radiology, Seoul National University College of Medicine and Institute of Radiation Medicine, SNUMRC, Seoul, Korea.

ABSTRACT
The receiver operating characteristic (ROC) curve, which is defined as a plot of test sensitivity as they coordinate versus its 1-specificity or false positive rate (FPR) as the x coordinate, is an effective method of evaluating the performance of diagnostic tests. The purpose of this article is to provide a nonmathematical introduction to ROC analysis. Important concepts involved in the correct use and interpretation of this analysis, such as smooth and empirical ROC curves, parametric and nonparametric methods, the area under the ROC curve and its 95% confidence interval, the sensitivity at a particular FPR, and the use of a partial area under the ROC curve are discussed. Various considerations concerning the collection of data in radiological ROC studies are briefly discussed. An introduction to the software frequently used for performing ROC analyses is also presented.

Show MeSH
Related in: MedlinePlus