Receiver operating characteristic (ROC) curves are useful tools to evaluate classifiers in biomedical and bioinformatics applications. However, conclusions are often reached through inconsistent use or insufficient statistical analysis. To support researchers in their ROC curves analysis we developed pROC, a package for R and S+ that contains a set of tools displaying, analyzing, smoothing and comparing ROC curves in a user-friendly, object-oriented and flexible interface. With data previously imported into the R or S+ environment, the pROC package builds ROC curves and includes functions for computing confidence intervals, statistical tests for comparing total or partial area under the curve or the operating points of different classifiers, and methods for smoothing ROC curves. Intermediary and final results are visualised in user-friendly interfaces. A case study based on published clinical and biomarker data shows how to perform a typical ROC analysis with pROC. pROC is a package for R and S+ specifically dedicated to ROC analysis. It proposes multiple statistical tests to compare ROC curves, and in particular partial areas under the curve, allowing proper ROC interpretation. pROC is available in two versions: in the R programming language or with a graphical user interface in the S+ statistical software. It is accessible at http://expasy.org/tools/pROC/
 under the GNU General Public License. It is also distributed through the CRAN and CSAN public repositories, facilitating its installation.

/pdf/proc-an-open-source-package-for-r-and-s-to-analyze-and-p7qu4wozaa.pdf

pROC: an open-source package for R and S+ to analyze and compare ROC curves

Essential Medical Statistics

https://www.thelancet.com/pdfs/journals/lancet/PIIS0140-6736(14)60584-5.pdf

Effect of treatment delay, age, and stroke severity on the effects of intravenous thrombolysis with alteplase for acute ischaemic stroke: a meta-analysis of individual patient data from randomised trials

Effect of treatment delay, age, and stroke severity on the effects of intravenous thrombolysis with alteplase for acute ischaemic stroke: a meta-analysis of individual patient data from randomised trials | NOVA. The University of Newcastle's Digital Repository

The development of decision support systems for pathology and their deployment in clinical practice have been hindered by the need for large manually annotated datasets. To overcome this problem, we present a multiple instance learning-based deep learning system that uses only the reported diagnoses as labels for training, thereby avoiding expensive and time-consuming pixel-wise manual annotations. We evaluated this framework at scale on a dataset of 44,732 whole slide images from 15,187 patients without any form of data curation. Tests on prostate cancer, basal cell carcinoma and breast cancer metastases to axillary lymph nodes resulted in areas under the curve above 0.98 for all cancer types. Its clinical application would allow pathologists to exclude 65–75% of slides while retaining 100% sensitivity. Our results show that this system has the ability to train accurate classification models at unprecedented scale, laying the foundation for the deployment of computational decision support systems in clinical practice.

Clinical-grade computational pathology using weakly supervised deep learning on whole slide images.

Since the early 1980s, a bewildering array of methods for constructing bootstrap confidence intervals have been proposed. In this article, we address the following questions. First, when should bootstrap confidence intervals be used. Secondly, which method should be chosen, and thirdly, how should it be implemented. In order to do this, we review the common algorithms for resampling and methods for constructing bootstrap confidence intervals, together with some less well known ones, highlighting their strengths and weaknesses. We then present a simulation study, a flow chart for choosing an appropriate method and a survival analysis example.

Bootstrap confidence intervals : when, which, what? A practical guide for medical statisticians

A relative risk function over a geographical region is defined and it is shown that it can be estimated effectively using kernel density estimation separately for the spatial distribution of disease cases and for a sample of controls. This procedure is demonstrated using data on childhood leukaemia in the vicinity of the Sellafield nuclear reprocessing plant in Cumbria, U.K. Various modifications to the method are proposed, including the use of an adaptive kernel. The final plot demonstrates a sharp peak at Sellafield and a reasonably smooth surface over the rest of the region, despite the small number of cases in the series.

An application of density estimation to geographical epidemiology

The association between obstetric X-raying and childhood cancer was first identified by the Oxford Survey of Childhood Cancers in 1956. The present re-analysis exploits the case-control matching of the study while incorporating the effects of important risk determinants, notably year of birth, trimester of exposure and number of films exposed. The decline in risk over time is closely mirrored by the estimated decline in dose per film and, by constraining these two relationships to be parallel, time-invariant estimates of the extra risk per mGy are obtained. For example, it is now estimated that irradiating 10(6) foetuses with 1 mGy of X-rays would, in the absence of other causes of death, yield 175 extra cases of cancer and leukaemia in the first 15 years of life.

A new calculation of the carcinogenic risk of obstetric X-raying.

This paper considers the underlying principles of depicting disease incidence on geographical maps and uses them to attempt a comparative classification of methods. After a discussion of the possibilities for incorporating time, we consider projection methods, some of which have been used to portray information in a manner supposed to be independent of population density. We then distinguish between non-parametric and model-based methods, including models for areal data using Bayesian ideas. Data in point form are also discussed and it is argued that the relative risk function provides a fundamental model useful for assessing different methods as a whole, some of which are known to be flawed and many of which are untested as regards their statistical properties.

A classification of disease mapping methods.

The risk associated with different levels of a quantitative factor X is often measured relative to the level corresponding to X = 0. There are situations, however, where there is no natural zero for X, for example where the risk factor is the age of an individual. In this case it is more natural to measure risk relative to an overall average for the study population. To use the whole population in this way also raises the possibility of regarding X as truly continuous, rather than as a grouped variable. This gives rise to the concept of a relative risk function. Methods for estimating such functions are discussed, concentrating for the most part on the discrete case. The extension to higher dimensions permits the investigation of joint effects of several factors, while the problem of controlling for confounding variables can be handled by fitting multiplicative risk models. Relating the latter to the log-linear model permits the estimation of adjusted relative risk functions. The method is illustrated using data on childhood cancer. The continuous case can in principle be handled in a similar way using density estimation techniques.

John F. Bithell

Papers

Bootstrap confidence intervals : when, which, what? A practical guide for medical statisticians

An application of density estimation to geographical epidemiology

A new calculation of the carcinogenic risk of obstetric X-raying.

A classification of disease mapping methods.

Estimation of relative risk functions.