Showing papers on "Ordinal regression published in 2017"

Modeling continuous response variables using ordinal regression

Abstract: We study the application of a widely used ordinal regression model, the cumulative probability model (CPM), for continuous outcomes. Such models are attractive for the analysis of continuous response variables because they are invariant to any monotonic transformation of the outcome and because they directly model the cumulative distribution function from which summaries such as expectations and quantiles can easily be derived. Such models can also readily handle mixed type distributions. We describe the motivation, estimation, inference, model assumptions, and diagnostics. We demonstrate that CPMs applied to continuous outcomes are semiparametric transformation models. Extensive simulations are performed to investigate the finite sample performance of these models. We find that properly specified CPMs generally have good finite sample performance with moderate sample sizes, but that bias may occur when the sample size is small. Cumulative probability models are fairly robust to minor or moderate link function misspecification in our simulations. For certain purposes, the CPMs are more efficient than other models. We illustrate their application, with model diagnostics, in a study of the treatment of HIV. CD4 cell count and viral load 6 months after the initiation of antiretroviral therapy are modeled using CPMs; both variables typically require transformations, and viral load has a large proportion of measurements below a detection limit.

Comparison of response patterns in different survey designs: a longitudinal panel with mixed-mode and online-only design

Abstract: Increasing availability of the Internet allows using only online data collection for more epidemiological studies. We compare response patterns in a population-based health survey using two survey designs: mixed-mode (choice between paper-and-pencil and online questionnaires) and online-only design (without choice). We used data from a longitudinal panel, the Hygiene and Behaviour Infectious Diseases Study (HaBIDS), conducted in 2014/2015 in four regions in Lower Saxony, Germany. Individuals were recruited using address-based probability sampling. In two regions, individuals could choose between paper-and-pencil and online questionnaires. In the other two regions, individuals were offered online-only participation. We compared sociodemographic characteristics of respondents who filled in all panel questionnaires between the mixed-mode group (n = 1110) and the online-only group (n = 482). Using 134 items, we performed multinomial logistic regression to compare responses between survey designs in terms of type (missing, “do not know” or valid response) and ordinal regression to compare responses in terms of content. We applied the false discovery rates (FDR) to control for multiple testing and investigated effects of adjusting for sociodemographic characteristic. For validation of the differential response patterns between mixed-mode and online-only, we compared the response patterns between paper and online mode among the respondents in the mixed-mode group in one region (n = 786). Respondents in the online-only group were older than those in the mixed-mode group, but both groups did not differ regarding sex or education. Type of response did not differ between the online-only and the mixed-mode group. Survey design was associated with different content of response in 18 of the 134 investigated items; which decreased to 11 after adjusting for sociodemographic variables. In the validation within the mixed-mode, only two of those were among the 11 significantly different items. The probability of observing by chance the same two or more significant differences in this setting was 22%. We found similar response patterns in both survey designs with only few items being answered differently, likely attributable to chance. Our study supports the equivalence of the compared survey designs and suggests that, in the studied setting, using online-only design does not cause strong distortion of the results.

Multiple criteria hierarchy process for sorting problems based on ordinal regression with additive value functions

Abstract: A hierarchical decomposition is a common approach for coping with complex decision problems involving multiple dimensions. Recently, the multiple criteria hierarchy process (MCHP) has been introduced as a new general framework for dealing with multiple criteria decision aiding in case of a hierarchical structure of the family of evaluation criteria. This study applies the MCHP framework to multiple criteria sorting problems and extends existing disaggregation and robust ordinal regression techniques that induce decision models from data. The new methodology allows the handling of preference information and the formulation of recommendations at the comprehensive level, as well as at all intermediate levels of the hierarchy of criteria. A case study on bank performance rating is used to illustrate the proposed methodology.

Measures of Association for Bivariate Ordinal Hypotheses

Abstract: The appropriate choice of a measure of association is more than merely a purist's concern, for the conclusions reached in the analysis of a given set of data can depend crucially on the measure employed. A general framework has been proposed for constructing measures of association for bivariate ordinal hypotheses. The concern is with the problem of a researcher, who has formulated a bivariate hypothesis—that is, a proposition asserting a relation between two variables—and must choose an appropriate measure of association. A bivariate ordinal hypothesis asserts a relation between two ordinal variables. Although ordinal variables have a number of unsatisfactory characteristics, they are likely to remain a prominent feature of empirical social research for some time to come. For ordinal data, numerical algebraic models are unavailable, and so three general types of ordinal relations are identified: no-reversals, asymmetric, and strict.

Earth mover's distance pooling over siamese LSTMs for automatic short answer grading

Abstract: Automatic short answer grading (ASAG) can reduce tedium for instructors, but is complicated by free-form student inputs. An important ASAG task is to assign ordinal scores to student answers, given some "model" or ideal answers. Here we introduce a novel framework for ASAG by cascading three neural building blocks: Siamese bidirectional LSTMs applied to a model and a student answer, a novel pooling layer based on earth-mover distance (EMD) across all hidden states from both LSTMs, and a flexible final regression layer to output scores. On standard ASAG data sets, our system shows substantial reduction in grade estimation error compared to competitive baselines. We demonstrate that EMD pooling results in substantial accuracy gains, and that a support vector ordinal regression (SVOR) output layer helps outperform softmax. Our system also outperforms recent attention mechanisms on LSTM states.

Ordinal probability effect measures for group comparisons in multinomial cumulative link models.

Abstract: Summary We consider simple ordinal model-based probability effect measures for comparing distributions of two groups, adjusted for explanatory variables. An “ordinal superiority” measure summarizes the probability that an observation from one distribution falls above an independent observation from the other distribution, adjusted for explanatory variables in a model. The measure applies directly to normal linear models and to a normal latent variable model for ordinal response variables. It equals Φ(β/2) for the corresponding ordinal model that applies a probit link function to cumulative multinomial probabilities, for standard normal cdf Φ and effect β that is the coefficient of the group indicator variable. For the more general latent variable model for ordinal responses that corresponds to a linear model with other possible error distributions and corresponding link functions for cumulative multinomial probabilities, the ordinal superiority measure equals exp(β)/[1+exp(β)] with the log–log link and equals approximately exp(β/2)/[1+exp(β/2)] with the logit link, where β is the group effect. Another ordinal superiority measure generalizes the difference of proportions from binary to ordinal responses. We also present related measures directly for ordinal models for the observed response that need not assume corresponding latent response models. We present confidence intervals for the measures and illustrate with an example.

On the Consistency of Ordinal Regression Methods

Abstract: Many of the ordinal regression models that have been proposed in the literature can be seen as methods that minimize a convex surrogate of the zero-one, absolute, or squared loss functions. A key property that allows to study the statistical implications of such approximations is that of Fisher consistency. Fisher consistency is a desirable property for surrogate loss functions and implies that in the population setting, i.e., if the probability distribution that generates the data were available, then optimization of the surrogate would yield the best possible model. In this paper we will characterize the Fisher consistency of a rich family of surrogate loss functions used in the context of ordinal regression, including support vector ordinal regression, ORBoosting and least absolute deviation. We will see that, for a family of surrogate loss functions that subsumes support vector ordinal regression and ORBoosting, consistency can be fully characterized by the derivative of a real-valued function at zero, as happens for convex margin-based surrogates in binary classification. We also derive excess risk bounds for a surrogate of the absolute error that generalize existing risk bounds for binary classification. Finally, our analysis suggests a novel surrogate of the squared error loss. We compare this novel surrogate with competing approaches on 9 different datasets. Our method shows to be highly competitive in practice, outperforming the least squares loss on 7 out of 9 datasets.

Mixture models for ordinal responses to account for uncertainty of choice

Abstract: In CUB models the uncertainty of choice is explicitly modelled as a Combination of discrete Uniform and shifted Binomial random variables. The basic concept to model the response as a mixture of a deliberate choice of a response category and an uncertainty component that is represented by a uniform distribution on the response categories is extended to a much wider class of models. The deliberate choice can in particular be determined by classical ordinal response models as the cumulative and adjacent categories model. Then one obtains the traditional and flexible models as special cases when the uncertainty component is irrelevant. It is shown that the effect of explanatory variables is underestimated if the uncertainty component is neglected in a cumulative type mixture model. Visualization tools for the effects of variables are proposed and the modelling strategies are evaluated by use of real data sets. It is demonstrated that the extended class of models frequently yields better fit than classical ordinal response models without an uncertainty component.

Distance metric learning for ordinal classification based on triplet constraints

Abstract: Ordinal classification is a problem setting in-between nominal classification and metric regression, where the goal is to predict classes of an ordinal scale. Usually, there is a clear ordering of the classes, but the absolute distances between them are unknown. Disregarding the ordering information, this kind of problems is commonly treated as multi-class classification problems, however, it often results in a significant loss of performance. Exploring such ordering information can help to improve the effectiveness of classifiers. In this paper, we propose a distance metric learning approach for ordinal classification by incorporating local triplet constraints containing the ordering information into a conventional large-margin distance metric learning approach. Specifically, our approach tries to preserve, for each training example, the ordinal relationship as well as the local geometry structure of its neighbors, which is suitable for use in local distance-based algorithms such as k-nearest-neighbor (k-NN) classification. Different from previous works that usually learn distance metrics by weighing the distances between training examples according to their class label differences, the proposed approach can directly satisfy the ordinal relationships where no assumptions about the distances between classes are made.

How to test for goodness of fit in ordinal logistic regression models

Abstract: Ordinal regression models are used to describe the relationship between an ordered categorical response variable and one or more explanatory variables. Several ordinal logistic models are available in Stata, such as the proportional odds, adjacent-category, and constrained continuation-ratio models. In this article, we present a command (ologitgof) that calculates four goodness-of-fit tests for assessing the overall adequacy of these models. These tests include an ordinal version of the Hosmer–Lemeshow test, the Pulkstenis–Robinson chi-squared and deviance tests, and the Lipsitz likelihood-ratio test. Together, these tests can detect several different types of lack of fit, including wrongly specified continuous terms, omission of different types of interaction terms, and an unordered response variable.

Deep Ordinal Regression Based on Data Relationship for Small Datasets

Abstract: Ordinal regression aims to classify instances into ordinal categories. As with other supervised learning problems, learning an effective deep ordinal model from a small dataset is challenging. This paper proposes a new approach which transforms the ordinal regression problem to binary classification problems and uses triplets with instances from different categories to train deep neural networks such that high-level features describing their ordinal relationship can be extracted automatically. In the testing phase, triplets are formed by a testing instance and other instances with known ranks. A decoder is designed to estimate the rank of the testing instance based on the outputs of the network. Because of the data argumentation by permutation, deep learning can work for ordinal regression even on small datasets. Experimental results on the historical color image benchmark and MSRA image search datasets demonstrate that the proposed algorithm outperforms the traditional deep learning approach and is comparable with other state-ofthe-art methods, which are highly based on prior knowledge to design effective features.

Local private ordinal data distribution estimation

Abstract: The categorical data that have natural ordering between categories are termed ordinal data, which are pervasive in numerous areas, including discrete sensor readings, metering data or preference options. Though aggregating such ordinal data from the population is facilitating plenty of crowdsourcing applications, contributing such data is privacy risky and may reveal sensitive information (e.g. locations, identities) about individuals. This work studies ordinal data aggregation for distribution estimation meanwhile locally preserving individuals' data privacy (such as on their mobile devices). Under e-geo-indistinguishable constraints, which capture intrinsic dissimilarity between ordinal categories in the framework of differential privacy, we provide an efficient and effective locally private mechanism: Subset Exponential Mechanism (SEM) for ordinal data distribution estimation. The mechanism randomly responds with a fixed-size subset of the categories with calibrated probability assignment. Specially for uniform ordinal data, we propose a circling technique to symmetrically randomizing categories and estimating frequencies of categories, hence the computational/space costs and estimation performance of SEM are further optimized. Besides contributing theoretical error bounds of SEM, we also evaluate the mechanism on extensive scenarios, the evaluation results show that SEM reduces distribution estimation error on average by exp(∊/2) factor over existing private mechanisms.

Regularized Ordinal Regression and the ordinalNet R Package

Abstract: Regularization techniques such as the lasso (Tibshirani 1996) and elastic net (Zou and Hastie 2005) can be used to improve regression model coefficient estimation and prediction accuracy, as well as to perform variable selection. Ordinal regression models are widely used in applications where the use of regularization could be beneficial; however, these models are not included in many popular software packages for regularized regression. We propose a coordinate descent algorithm to fit a broad class of ordinal regression models with an elastic net penalty. Furthermore, we demonstrate that each model in this class generalizes to a more flexible form, for instance to accommodate unordered categorical data. We introduce an elastic net penalty class that applies to both model forms. Additionally, this penalty can be used to shrink a non-ordinal model toward its ordinal counterpart. Finally, we introduce the R package ordinalNet, which implements the algorithm for this model class.

Rank aggregation methods dealing with ordinal uncertain preferences

Abstract: A two-step rank aggregation model for interval ordinal rankings is proposed.A matrix retrieving dominance possibilities is built from uncertain data.Priority vectors are derived from the dominance aggregate matrix.The model can manage uncertain and incomplete rank information with ties.Computational methods are proposed to solve the optimization problems. The problem of rank aggregation, also known as group-ranking, arises in many fields such as metasearch engines, information retrieval, recommendation systems and multicriteria decision-making. Given a set of alternatives, the problem is to order the alternatives based on ordinal rankings provided by a group of individual experts. The available information is often limited and uncertain in real-world applications. This paper addresses the general group-ranking problem using interval ordinal data as a flexible way to capture uncertain and incomplete information. We propose a two-stage approach. The first stage learns an aggregate preference matrix as a means of gathering group preferences from uncertain and possibly conflicting information. In the second stage, priority vectors are derived from the aggregate preference matrix based on properties of fuzzy preference relations and graph theory. Our approach provides a theoretical framework for studying the problem that extends some of the methods in the literature, efficient computational methods to solve the problem and some performance measures. It relaxes data certainty and completeness assumptions and overcomes some shortcomings of current group-ranking methods.

Soft-Margin Mixture of Regressions

Abstract: Nonlinear regression is a common statistical tool to solve many computer vision problems (e.g., age estimation, pose estimation). Existing approaches to nonlinear regression fall into two main categories: (1) The universal approach provides an implicit or explicit homogeneous feature mapping (e.g., kernel ridge regression, Gaussian process regression, neural networks). These approaches may fail when data is heterogeneous or discontinuous. (2) Divide-and-conquer approaches partition a heterogeneous input feature space and learn multiple local regressors. However, existing divide-and-conquer approaches fail to deal with discontinuities between partitions (e.g., Gaussian mixture of regressions) and they cannot guarantee that the partitioned input space will be homogeneously modeled by local regressors (e.g., ordinal regression). To address these issues, this paper proposes Soft-Margin Mixture of Regressions (SMMR), a method that directly learns homogeneous partitions of the input space and is able to deal with discontinuities. SMMR outperforms the state-of-the-art methods on three popular computer vision tasks: age estimation, crowd counting and viewpoint estimation from images.

Bayesian Adaptive Lasso for Ordinal Regression With Latent Variables

Abstract: We consider an ordinal regression model with latent variables to investigate the effects of observable and latent explanatory variables on the ordinal responses of interest. Each latent variable is...

Adversarial Surrogate Losses for Ordinal Regression

Abstract: Ordinal regression seeks class label predictions when the penalty incurred for mistakes increases according to an ordering over the labels. The absolute error is a canonical example. Many existing methods for this task reduce to binary classification problems and employ surrogate losses, such as the hinge loss. We instead derive uniquely defined surrogate ordinal regression loss functions by seeking the predictor that is robust to the worst-case approximations of training data labels, subject to matching certain provided training data statistics. We demonstrate the advantages of our approach over other surrogate losses based on hinge loss approximations using UCI ordinal prediction tasks.

Nonparallel Support Vector Ordinal Regression

Abstract: Ordinal regression is a supervised learning problem where training samples are labeled by an ordinal scale. The ordering relation and nonmetric property of the label set distinguish it from the multiclass classification and metric regression. To better exploit the inherent structure in the label and benefit from the hidden information in data distribution, we propose a novel ordinal regression model, which is named as nonparallel support vector ordinal regression (NPSVOR) to emphasis the utilization of nonparallel proximal hyperplanes. The new model constructs a hyperplane for each rank such that the patterns of this rank lie in the close proximity while maintaining clear separation with the other ranks. Since the learning of hyperplanes can be carried out independently, NPSVOR can be trained in parallel. Furthermore, we design an efficient solver at the same time for training the hyperplanes in NPSVOR based on the alternating direction method of multipliers. Extensive experimentation demonstrates that NPSVOR yields a large and statistically significant improvement in terms of generalization performance and training speed against nine baselines.

Fuzzy Clustering Data Given in the Ordinal Scale

Abstract: A fuzzy clustering algorithm for multidimensional data is proposed in this article. The data is described by vectors whose components are linguistic variables defined in an ordinal scale. The obtained results confirm the efficiency of the proposed approach.

Robust inference for ordinal response models

Abstract: The present paper deals with the robustness of estimators and tests for ordinal response models. In this context, gross-errors in the response variable, specific deviations due to some respondents’ behavior, and outlying covariates can strongly affect the reliability of the maximum likelihood estimators and that of the related test procedures. The paper highlights that the choice of the link function can affect the robustness of inferential methods, and presents a comparison among the most frequently used links. Subsequently robust $M$-estimators are proposed as an alternative to maximum likelihood estimators. Their asymptotic properties are derived analytically, while their performance in finite samples is investigated through extensive numerical experiments either at the model or when data contaminations occur. Wald and $t$-tests for comparing nested models, derived from $M$-estimators, are also proposed. $M$ based inference is shown to outperform maximum likelihood inference, producing more reliable results when robustness is a concern.

Handling imprecise evaluations in multiple criteria decision aiding and robust ordinal regression by n-point intervals

Abstract: We consider imprecise evaluation of alternatives in multiple criteria ranking problems. The imprecise evaluations are represented by n-point intervals which are defined by the largest interval of possible evaluations and by its subintervals sequentially nested one in another. This sequence of subintervals is associated with an increasing sequence of plausibility, such that the plausibility of a subinterval is greater than the plausibility of the subinterval containing it. We explain the intuition that stands behind this proposal, and we show the advantage of n-point intervals compared to other methods dealing with imprecise evaluations. Although n-point intervals can be applied in any multiple criteria decision aiding (MCDA) method, in this paper, we focus on their application in robust ordinal regression which, unlike other MCDA methods, takes into account all compatible instances of an adopted preference model, which reproduce an indirect preference information provided by the decision maker. An illustrative example shows how the method can be applied in practice.

Fuzzy Clustering Data Given on the Ordinal Scale Based on Membership and Likelihood Functions Sharing

Abstract: A task of clustering data given in the ordinal scale under conditions of overlapping clusters has been considered. It's proposed to use an approach based on memberhsip and likelihood functions sharing. A number of performed experiments proved effectiveness of the proposed method. The proposed method is characterized by robustness to outliers due to a way of ordering values while constructing membership functions.

The Costs of Indeterminacy: How to Determine Them?

Abstract: Indeterminate classifiers are cautious models able to predict more than one class in case of high uncertainty. A problem that arises when using such classifiers is how to evaluate their performances. This problem has already been considered in the case where all prediction errors have equivalent costs (that we will refer as the “0/1 costs” or accuracy setting). The purpose of this paper is to study the case of generic cost functions. We provide some properties that the costs of indeterminate predictions could or should follow, and review existing proposals in the light of those properties. This allows us to propose a general formula fitting our properties that can be used to produce and evaluate indeterminate predictions. Some experiments on the cost-sensitive problem of ordinal regression illustrate the behavior of the proposed evaluation criterion.

New types of ordinal sum of fuzzy implications

Abstract: In this contribution new ways of constructing of ordinal sum of fuzzy implications are proposed. These methods are based on a construction of ordinal sums of overlap functions. Moreover, preservation of some properties of these ordinal sums of fuzzy implications are examined. Among others neutrality property, identity property, and ordering property are considered.

Regression with ordered predictors via ordinal smoothing splines

Abstract: Many applied studies collect one or more ordered categorical predictors, which do not fit neatly within classic regression frameworks. In most cases, ordinal predictors are treated as either nominal (unordered) variables or metric (continuous) variables in regression models, which is theoretically and/or computationally undesirable. In this paper, we discuss the benefit of taking a smoothing spline approach to the modeling of ordinal predictors. The purpose of this paper is to provide theoretical insight into the ordinal smoothing spline, as well as examples revealing the potential of the ordinal smoothing spline for various types of applied research. Specifically, we (i) derive the analytical form of the ordinal smoothing spline reproducing kernel, (ii) propose an ordinal smoothing spline isotonic regression estimator, (iii) prove an asymptotic equivalence between the ordinal and linear smoothing spline reproducing kernel functions, (iv) develop large sample approximations for the ordinal smoothing spline, and (v) demonstrate the use of ordinal smoothing splines for isotonic regression and semiparametric regression with multiple predictors. Our results reveal that the ordinal smoothing spline offers a flexible approach for incorporating ordered predictors in regression models, and has the benefit of being invariant to any monotonic transformation of the predictor scores.

Ordinal profile monitoring with random explanatory variables

Abstract: Profiles characterise the functional relationship between the response variable and one or more explanatory variables and have been playing an important role in many applications. Profile monitoring mainly aims at checking the stability of this relationship. In many situations, we observe that the response variable is categorical with three or more attribute levels, and that there is natural order among the levels. Moreover, the explanatory variables are also random rather than fixed at some predefined values. To fully exploit the ordinal information, it is assumed that there is an unknown latent continuous distribution determining the levels of the ordinal response. Based on this, we propose a novel control chart for jointly monitoring the functional relationship, location shifts in the latent continuous distribution, and the random explanatory variables. Simulation results show that our proposed chart is efficient in detecting abnormalities and is robust to various latent distributions.