In biomedical research and practice, quantitative tests or biomarkers are often used for diagnostic or screening purposes, with a cut point established on the quantitative measurement to aid binary classification. This paper introduces an alternative to the traditional methods based on the Youden index and the closest-to-(0, 1) criterion for threshold selection. A concordance probability evaluating the classification accuracy of a dichotomized measure is defined as an objective function of the possible cut point. A nonparametric approach is used to search for the optimal cut point maximizing the objective function. The procedure is shown to perform well in a simulation study. Using data from a real-world study of arsenic-induced skin lesions, we apply the method to a measure of blood arsenic levels, selecting a cut point to be used as a warning threshold.

Classification accuracy and cut point selection.

List of Tables. Preface. 1. Typical Stated Choice Experiments. 2. Factorial Designs. 3. The MNL Model and Comparing Designs. 4. Paired Comparison Designs for Binary Attributes. 5. Larger Choice Set Sizes for Binary Attributes. 6. Designs for Asymmetric Attributes. 7. Various Topics. 8. Practical Techniques For Constructing Choice Experiments. Bibliography. Index.

/pdf/the-construction-of-optimal-stated-choice-experiments-theory-433gkpnzj7.pdf

The Construction of Optimal Stated Choice Experiments: Theory and Methods

Somers' D is an asymmetric measure of association between two variables, which plays a central role as a parameter behind rank or nonparametric statistical methods. Given predictor variable X and outcome variable Y ,w e may estimate D YX as a measure of the effect of X on Y , or we may estimate DXY as a performance indicator of X as a predictor of Y.T hesomersd package allows the estimation of Somers' D and Kendall's τa with confidence limits as well as p-values. The Stata 9 version of somersd can estimate extended versions of Somers' D not previously available, including the Gini index, the parameter tested by the sign test, and extensions to left- or right-censored data. It can also estimate stratified versions of Somers' D, restricted to pairs in the same stratum. Therefore, it is possible to define strata by grouping values of a confounder, or of a propensity score based on multiple confounders, and to estimate versions of Somers' D that measure the association between the outcome and the predictor, adjusted for the confounders. The Stata 9 version of somersd uses the Mata language for improved computational efficiency with large datasets.

https://journals.sagepub.com/doi/pdf/10.1177/1536867X0600600302

Confidence intervals for rank statistics: Somers' D and extensions

The area under the receiver operating characteristics curve (AUC of ROC) is a widely used measure of discrimination in risk prediction models. Routinely, the Mann-Whitney statistics is used as an estimator of AUC, while the change in AUC is tested by the DeLong test. However, very often, in settings where the model is developed and tested on the same dataset, the added predictor is statistically significantly associated with the outcome but fails to produce a significant improvement in the AUC. No conclusive resolution exists to explain this finding. In this paper, we will show that the reason lies in the inappropriate application of the DeLong test in the setting of nested models. Using numerical simulations and a theoretical argument based on generalized U-statistics, we show that if the added predictor is not statistically significantly associated with the outcome, the null distribution is non-normal, contrary to the assumption of DeLong test. Our simulations of different scenarios show that the loss of power because of such a misuse of the DeLong test leads to a conservative test for small and moderate effect sizes. This problem does not exist in cases of predictors that are associated with the outcome and for non-nested models. We suggest that for nested models, only the test of association be performed for the new predictors, and if the result is significant, change in AUC be estimated with an appropriate confidence interval, which can be based on the DeLong approach.

/pdf/misuse-of-delong-test-to-compare-aucs-for-nested-models-3a16n9t6d1.pdf

Misuse of DeLong test to compare AUCs for nested models

This paper discusses the task of learning a classifier from observed data containing missing values amongst the inputs which are missing completely at random 1 . A non-parametric perspective is adopted by defining a modified risk taking into account the uncertainty of the predicted outputs when missing values are involved. It is shown that this approach generalizes the approach of mean imputation in the linear case and the resulting kernel machine reduces to the standard Support Vector Machine (SVM) when no input values are missing. Furthermore, the method is extended to the multivariate case of fitting additive models using componentwise kernel machines, and an efficient implementation is based on the Least Squares Support Vector Machine (LS-SVM) classifier formulation.

Stefun D. Leigh

Papers

U-Statistics Theory and Practice