This review provides the basic principle and rational for ROC analysis of rating and continuous diagnostic test results versus a gold standard. Derived indexes of accuracy, in particular area under the curve (AUC) has a meaningful interpretation for disease classification from healthy subjects. The methods of estimate of AUC and its testing in single diagnostic test and also comparative studies, the advantage of ROC curve to determine the optimal cut off values and the issues of bias and confounding have been discussed.

/pdf/receiver-operating-characteristic-roc-curve-analysis-for-5a75hjjp6n.pdf

Receiver Operating Characteristic (ROC) Curve Analysis for Medical Diagnostic Test Evaluation

The use of receiver operating characteristic curves in biomedical informatics

Prefaces Introduction What Is Computational Statistics? An Overview of the Book Probability Concepts Introduction Probability Conditional Probability and Independence Expectation Common Distributions Sampling Concepts Introduction Sampling Terminology and Concepts Sampling Distributions Parameter Estimation Empirical Distribution Function Generating Random Variables Introduction General Techniques for Generating Random Variables Generating Continuous Random Variables Generating Discrete Random Variables Exploratory Data Analysis Introduction Exploring Univariate Data Exploring Bivariate and Trivariate Data Exploring Multidimensional Data Finding Structure Introduction Projecting Data Principal Component Analysis Projection Pursuit EDA Independent Component Analysis Grand Tour Nonlinear Dimensionality Reduction Monte Carlo Methods for Inferential Statistics Introduction Classical Inferential Statistics Monte Carlo Methods for Inferential Statistics Bootstrap Methods Data Partitioning Introduction Cross-Validation Jackknife Better Bootstrap Confidence Intervals Jackknife-after-Bootstrap Probability Density Estimation Introduction Histograms Kernel Density Estimation Finite Mixtures Generating Random Variables Supervised Learning Introduction Bayes' Decision Theory Evaluating the Classifier Classification Trees Combining Classifiers Unsupervised Learning Introduction Measures of Distance Hierarchical Clustering K-Means Clustering Model-Based Clustering Assessing Cluster Results Parametric Models Introduction Spline Regression Models Logistic Regression Generalized Linear Models Nonparametric models Introduction Some Smoothing Methods Kernel Methods Smoothing Splines Nonparametric Regression-Other Details Regression Trees Additive Models Markov Chain Monte Carlo Methods Introduction Background Metropolis-Hastings Algorithms The Gibbs Sampler Convergence Monitoring Spatial Statistics Introduction Visualizing Spatial Point Processes Exploring First-Order and Second-Order Properties Modeling Spatial Point Processes Simulating Spatial Point Processes Appendix A: Introduction to Matlab What Is MATLAB? Getting Help in MATLAB File and Workspace Management Punctuation in MATLAB Arithmetic Operators Data Constructs in MATLAB Script Files and Functions Control Flow Simple Plotting Contact Information Appendix B: Projection Pursuit Indexes Indexes MATLAB Source Code Appendix C: Matlab Statistics Toolbox Appendix D: Computational Statistics Toolbox Appendix E: Exploratory Data Analysis Toolboxes Introduction EDA Toolbox EDA GUI Toolbox Appendix F: Data Sets Appendix G: NOTATION References INDEX MATLAB Code, Further Reading, and Exercises appear at the end of each chapter.

https://www.jstatsoft.org/article/view/v011b06/v11b06.pdf

Computational Statistics Handbook with MATLAB

Sensitivity and specificity are two components that measure the inherent validity of a diagnostic test for dichotomous outcomes against a gold standard. Receiver operating characteristic (ROC) curve is the plot that depicts the trade-off between the sensitivity and (1-specificity) across a series of cut-off points when the diagnostic test is continuous or on ordinal scale (minimum 5 categories). This is an effective method for assessing the performance of a diagnostic test. The aim of this article is to provide basic conceptual framework and interpretation of ROC analysis to help medical researchers to use it effectively. ROC curve and its important components like area under the curve, sensitivity at specified specificity and vice versa, and partial area under the curve are discussed. Various other issues such as choice between parametric and non-parametric methods, biases that affect the performance of a diagnostic test, sample size for estimating the sensitivity, specificity, and area under ROC curve, and details of commonly used softwares in ROC analysis are also presented.

Receiver operating characteristic (ROC) curve for medical researchers

Sample size estimation in diagnostic test studies of biomedical informatics

Sampling variability of nonparametric estimates of the areas under receiver operating characteristic curves: an update.

Receiver operating characteristic (ROC) analysis, which yields indices of accuracy such as the area under the curve (AUC), is increasingly being used to evaluate the performances of diagnostic tests that produce results on continuous scales. Both parametric and nonparametric ROC approaches are available to assess the discriminant capacity of such tests, but there are no clear guidelines as to the merits of each, particularly with non-binormal data. Investigators may worry that when data are non-Gaussian, estimates of diagnostic accuracy based on a binormal model may be distorted. The authors conducted a Monte Carlo simulation study to compare the bias and sampling variability in the estimates of the AUCs derived from parametric and nonparametric procedures. Each approach was assessed in data sets generated from various configurations of pairs of overlapping distributions; these included the binormal model and non-binormal pairs of distributions where one or both pair members were mixtures of Gaussian (MG) distributions with different degrees of departures from binormality. The biases in the estimates of the AUCs were found to be very small for both parametric and nonparametric procedures. The two approaches yielded very close estimates of the AUCs and the corresponding sampling variability even when data were generated from non-binormal models. Thus, for a wide range of distributions, concern about bias or imprecision of the estimates of the AUC should not be a major factor in choosing between the nonparametric and parametric approaches.

/pdf/a-comparison-of-parametric-and-nonparametric-approaches-to-58c1nlxjjz.pdf

Karim O. Hajian-Tilaki

Papers

Sampling variability of nonparametric estimates of the areas under receiver operating characteristic curves: an update.

A comparison of parametric and nonparametric approaches to ROC analysis of quantitative diagnostic tests.

Extension of receiver operating characteristic analysis to data concerning multiple signal detection tasks