Showing papers on "Ordinal regression published in 2014"

Robust Ordinal Regression

Abstract: Within disaggregation–aggregation approach, ordinal regressionaims at inducing parameters of a preference model, for example, parameters of a value function, which represent some holistic preference comparisons of alternatives given by the Decision Maker (DM). Usually, from among many sets of parameters of a preference model representing the preference information given by the DM, only one specific set is selected and used to work out a recommendation. For example, while there exist many value functions representing the holistic preference information given by the DM, only one value function is typically used to recommend the best choice, sorting, or ranking of alternatives. Since the selection of one from among many sets of parameters compatible with the preference information given by the DM is rather arbitrary, robust ordinal regressionproposes taking into account all the sets of parameters compatible with the preference information, in order to give a recommendation in terms of necessary and possible consequences of applying all the compatible preference models on the considered set of alternatives. In this chapter, we present the basic principle of robust ordinal regression, and the main multiple criteria decision methods to which it has been applied. In particular, UTA GMS and GRIPmethods are described, dealing with choice and ranking problems, then UTADIS GMS , dealing with sorting (ordinal classification) problems. Next, we present robust ordinal regression applied to Choquet integral for choice, sorting, and ranking problems, with the aim of representing interactions between criteria. This is followed by a characterization of robust ordinal regression applied to outranking methods and to multiple criteria group decisions. Finally, we describe an interactive multiobjective optimization methodology based on robust ordinal regression, and an evolutionary multiobjective optimization method, called NEMO, which is also using the principle of robust ordinal regression.

Testing for Measurement Invariance with Respect to an Ordinal Variable

Abstract: Researchers are often interested in testing for measurement invariance with respect to an ordinal auxiliary variable such as age group, income class, or school grade. In a factor-analytic context, these tests are traditionally carried out via a likelihood ratio test statistic comparing a model where parameters differ across groups to a model where parameters are equal across groups. This test neglects the fact that the auxiliary variable is ordinal, and it is also known to be overly sensitive at large sample sizes. In this paper, we propose test statistics that explicitly account for the ordinality of the auxiliary variable, resulting in higher power against "monotonic" violations of measurement invariance and lower power against "non-monotonic" ones. The statistics are derived from a family of tests based on stochastic processes that have recently received attention in the psychometric literature. The statistics are illustrated via an application involving real data, and their performance is studied via simulation.

Modelling Uncertainty and Overdispersion in Ordinal Data

Abstract: In this article we introduce a probability distribution generated by a mixture of discrete random variables to capture uncertainty, feeling, and overdispersion, possibly present in ordinal data surveys. The choice of the components of the new model is motivated by a study on the data generating process. Inferential issues concerning the maximum likelihood estimates and the validation steps are presented; then, some empirical analyses are given to support the usefulness of the approach. Discussion on further extensions of the model ends the article.

Ordinal Patterns, Entropy, and EEG

Abstract: In this paper we illustrate the potential of ordinal-patterns-based methods for analysis of real-world data and, especially, of electroencephalogram (EEG) data. We apply already known (empirical permutation entropy, ordinal pattern distributions) and new (empirical conditional entropy of ordinal patterns, robust to noise empirical permutation entropy) methods for measuring complexity, segmentation and classification of time series.

Cost-Sensitive AdaBoost Algorithm for Ordinal Regression Based on Extreme Learning Machine

Abstract: In this paper, the well known stagewise additive modeling using a multiclass exponential (SAMME) boosting algorithm is extended to address problems where there exists a natural order in the targets using a cost-sensitive approach. The proposed ensemble model uses an extreme learning machine (ELM) model as a base classifier (with the Gaussian kernel and the additional regularization parameter). The closed form of the derived weighted least squares problem is provided, and it is employed to estimate analytically the parameters connecting the hidden layer to the output layer at each iteration of the boosting algorithm. Compared to the state-of-the-art boosting algorithms, in particular those using ELM as base classifier, the suggested technique does not require the generation of a new training dataset at each iteration. The adoption of the weighted least squares formulation of the problem has been presented as an unbiased and alternative approach to the already existing ELM boosting techniques. Moreover, the addition of a cost model for weighting the patterns, according to the order of the targets, enables the classifier to tackle ordinal regression problems further. The proposed method has been validated by an experimental study by comparing it with already existing ensemble methods and ELM techniques for ordinal regression, showing competitive results.

Predicting Progression of Alzheimer's Disease Using Ordinal Regression

Abstract: We propose a novel approach to predicting disease progression in Alzheimer’s disease (AD) – multivariate ordinal regression – which inherently models the ordered nature of brain atrophy spanning normal aging (CTL) to mild cognitive impairment (MCI) to AD. Ordinal regression provides probabilistic class predictions as well as a continuous index of disease progression – the ORCHID (Ordinal Regression Characteristic Index of Dementia) score. We applied ordinal regression to 1023 baseline structural MRI scans from two studies: the US-based Alzheimer’s Disease Neuroimaging Initiative (ADNI) and the European based AddNeuroMed program. Here, the acquired AddNeuroMed dataset was used as a completely independent test set for the ordinal regression model trained on the ADNI cohort providing an optimal assessment of model generalizability. Distinguishing CTL-like (CTL and stable MCI) from AD-like (MCI converters and AD) resulted in balanced accuracies of 82% (cross-validation) for ADNI and 79% (independent test set) for AddNeuroMed. For prediction of conversion from MCI to AD, balanced accuracies of 70% (AUC of 0.75) and 75% (AUC of 0.81) were achieved. The ORCHID score was computed for all subjects. We showed that this measure significantly correlated with MMSE at 12 months (r=–0.64, ADNI and r=–0.59, AddNeuroMed). Additionally, the ORCHID score can help fractionate subjects with unstable diagnoses (e.g. reverters and healthy controls who later progressed to MCI), moderately late converters (12–24 months) and late converters (24–36 months). A comparison with results in the literature and direct comparison with a binary classifier suggests that the performance of this framework is highly competitive.

MUSA-INT: Multicriteria customer satisfaction analysis with interacting criteria

Abstract: We are considering the problem of measuring and analyzing customer satisfaction concerning a product or a service evaluated on multiple criteria. The proposed methodology generalizes the MUSA (MUlticriteria Satisfaction Analysis) method. MUSA is a preference disaggregation method that, following the principle of ordinal regression analysis, finds an additive utility function representing both the comprehensive satisfaction level of a set of customers and a marginal satisfaction level with respect to each criterion. Differently from MUSA, the proposed approach, that we will call MUSA-INT, takes also into account positive and negative interactions among criteria, similarly to the multicriteria method UTA GMS -INT. Our method accepts evaluations on criteria with different ordinal scales which do not need to be transformed into a unique cardinal scale prior to the analysis. Moreover, instead of a single utility function, MUSA-INT can also take into account a set of utility functions representing customers' satisfaction, adopting the robust ordinal regression methodology. An illustrative example shows how the proposed methodology can be applied on a customers’ survey.

Projection-Based Ensemble Learning for Ordinal Regression

Abstract: The classification of patterns into naturally ordered labels is referred to as ordinal regression. This paper proposes an ensemble methodology specifically adapted to this type of problem, which is based on computing different classification tasks through the formulation of different order hypotheses. Every single model is trained in order to distinguish between one given class (k) and all the remaining ones, while grouping them in those classes with a rank lower than k, and those with a rank higher than k. Therefore, it can be considered as a reformulation of the well-known one-versus-all scheme. The base algorithm for the ensemble could be any threshold (or even probabilistic) method, such as the ones selected in this paper: kernel discriminant analysis, support vector machines and logistic regression (LR) (all reformulated to deal with ordinal regression problems). The method is seen to be competitive when compared with other state-of-the-art methodologies (both ordinal and nominal), by using six measures and a total of 15 ordinal datasets. Furthermore, an additional set of experiments is used to study the potential scalability and interpretability of the proposed method when using LR as base methodology for the ensemble.

Feature selection for ordinal text classification

Abstract: Ordinal classification also known as ordinal regression is a supervised learning task that consists of estimating the rating of a data item on a fixed, discrete rating scale. This problem is receiving increased attention from the sentiment analysis and opinion mining community due to the importance of automatically rating large amounts of product review data in digital form. As in other supervised learning tasks such as binary or multiclass classification, feature selection is often needed in order to improve efficiency and avoid overfitting. However, although feature selection has been extensively studied for other classification tasks, it has not for ordinal classification. In this letter, we present six novel feature selection methods that we have specifically devised for ordinal classification and test them on two data sets of product review data against three methods previously known from the literature, using two learning algorithms from the support vector regression tradition. The experimental results show that all six proposed metrics largely outperform all three baseline techniques and are more stable than these others by an order of magnitude, on both data sets and for both learning algorithms.

Ordinal Neural Networks Without Iterative Tuning

Abstract: Ordinal regression (OR) is an important branch of supervised learning in between the multiclass classification and regression. In this paper, the traditional classification scheme of neural network is adapted to learn ordinal ranks. The model proposed imposes monotonicity constraints on the weights connecting the hidden layer with the output layer. To do so, the weights are transcribed using padding variables. This reformulation leads to the so-called inequality constrained least squares (ICLS) problem. Its numerical solution can be obtained by several iterative methods, for example, trust region or line search algorithms. In this proposal, the optimum is determined analytically according to the closed-form solution of the ICLS problem estimated from the Karush–Kuhn–Tucker conditions. Furthermore, following the guidelines of the extreme learning machine framework, the weights connecting the input and the hidden layers are randomly generated, so the final model estimates all its parameters without iterative tuning. The model proposed achieves competitive performance compared with the state-of-the-art neural networks methods for OR.

On the Consistency of Ordinal Regression Methods

Abstract: Many of the ordinal regression models that have been proposed in the literature can be seen as methods that minimize a convex surrogate of the zero-one, absolute, or squared loss functions. A key property that allows to study the statistical implications of such approximations is that of Fisher consistency. Fisher consistency is a desirable property for surrogate loss functions and implies that in the population setting, i.e., if the probability distribution that generates the data were available, then optimization of the surrogate would yield the best possible model. In this paper we will characterize the Fisher consistency of a rich family of surrogate loss functions used in the context of ordinal regression, including support vector ordinal regression, ORBoosting and least absolute deviation. We will see that, for a family of surrogate loss functions that subsumes support vector ordinal regression and ORBoosting, consistency can be fully characterized by the derivative of a real-valued function at zero, as happens for convex margin-based surrogates in binary classification. We also derive excess risk bounds for a surrogate of the absolute error that generalize existing risk bounds for binary classification. Finally, our analysis suggests a novel surrogate of the squared error loss. We compare this novel surrogate with competing approaches on 9 different datasets. Our method shows to be highly competitive in practice, outperforming the least squares loss on 7 out of 9 datasets.

Categorical and Nonparametric Data Analysis: Choosing the Best Statistical Technique

Abstract: 1. Levels of Measurement, Probability, and the Binomial Formula 2. Estimation and Hypothesis Testing 3. Random Variables and Probability Distributions 4. Contingency Tables: The Chi-Square Test and Associated Effect Sizes 5. Contingency Tables: Special Situations 6. Basic Nonparametric Tests for Ordinal Data 7. Nonparametric Tests for Multiple or Related Samples 8. Advanced Rank Tests (for Interactions and Robust ANOVA) 9. Linear Regression and Generalized Linear Models 10. Binary Logistic Regression 11. Multinomial Logistic, Ordinal, & Poisson Regression 12. Loglinear Analysis 13. General Estimating Equations 14. Estimation Procedures 15. Choosing the Best Statistical Technique. Answers to Odd Numbered Problems

ordinalgmifs: An R Package for Ordinal Regression in High-dimensional Data Settings.

Abstract: High-throughput genomic assays are performed using tissue samples with the goal of classifying the samples as normal < pre-malignant < malignant or by stage of cancer using a small set of molecular features. In such cases, molecular features monotonically associated with the ordinal response may be important to disease development; that is, an increase in the phenotypic level (stage of cancer) may be mechanistically linked through a monotonic association with gene expression or methylation levels. Though traditional ordinal response modeling methods exist, they assume independence among the predictor variables and require the number of samples (n) to exceed the number of covariates (P) included in the model. In this paper, we describe our ordinalgmifs R package, available from the Comprehensive R Archive Network, which can fit a variety of ordinal response models when the number of predictors (P) exceeds the sample size (n). R code illustrating usage is also provided.

Building damage from the 2011 Great East Japan tsunami: quantitative assessment of influential factors

Abstract: Based on the classification provided by the Ministry of Land, Infrastructure, Transport and Tourism (MLIT), the damage level of buildings impacted by the 2011 Great East Japan tsunami can be separated into six levels (from minor damage to washed away). The objective of this paper is to identify the significant predictor variables and the direction of their potential relationship to the damage level in order to create a predicting formula for damage level. This study used the detailed data of damaged buildings in Ishinomaki city, Miyagi prefecture, Japan, collected by MLIT. The explanatory variables tested included the inundation depth, number of floors, structural material, and function of the building. Ordinal regression was applied to model the relationship between the ordinal outcome variable (damage level) and the predictors. The findings indicated that inundation depth, structural material, and function of building were significantly associated with the damage level. In addition to this new type of model, this research provides a valuable insight into the relative influence of different factors on building damage and suggestions that may help to revise the classification of current standards. This study can contribute to academic tsunami research by assessing the contribution of different variables to the observed damage using new approaches based on statistical analysis and regression. Moreover, practical applications of these results include understanding of the predominant factors driving tsunami damage to structures, implementation of the relevant variables into the proposed, or alternative model in order to improve current damage predictions by taking into account not only inundation depth, but also variables such as structural material and function of building.

An organ allocation system for liver transplantation based on ordinal regression

Abstract: Liver transplantation is nowadays a widely-accepted treatment for patients who present a terminal liver disease. Nevertheless, transplantation is greatly hampered by the un-availability of suitable liver donors; several methods have been developed and applied to find a better system to prioritize recipients on the waiting list, although most of them only consider donor or recipient characteristics (but not both). This paper proposes a novel donor-recipient liver allocation system constructed to predict graft survival after transplantation by means of a dataset comprised of donor-recipient pairs from different centres (seven Spanish and one UK hospitals). The best model obtained is used in conjunction with the Model for End-stage Liver Disease score (MELD), one of the current assignation methodology most used globally. This problem is assessed using the ordinal regression learning paradigm due to the natural ordering in the classes of the problem, via a cascade binary decomposition methodology and the Support Vector Machine methodology. The methodology proposed has shown competitiveness in all the metrics selected, when compared to other machine learning techniques and efficiently complements the MELD score based on the principles of efficiency and equity. Finally, a simulation of the proposal is included, in order to visualize its performance in realistic situations. This simulation has shown that there are some determining factors in the characterization of the survival time after transplantation (concerning both donors and recipients) and that the joint use of these sets of information could be, in fact, more useful and beneficial for the survival principle. Nonetheless, the results obtained indicate the true complexity of the problem dealt within this study and the fact that other characteristics that have not been included in the dataset may be of importance for the characterization of the dependent variable (survival time after transplantation), thus starting a promising line of future work.

Interpretation of commonly used statistical regression models.

Abstract: A review of some regression models commonly used in respiratory health applications is provided in this article. Simple linear regression, multiple linear regression, logistic regression and ordinal logistic regression are considered. The focus of this article is on the interpretation of the regression coefficients of each model, which are illustrated through the application of these models to a respiratory health research study.

Dealing with interaction between bipolar multiple criteria preferences in PROMETHEE methods

Abstract: In this paper we extend the PROMETHEE methods to the case of interacting criteria on a bipolar scale, introducing the bipolar PROMETHEE method based on the bipolar Choquet integral. In order to elicit parameters compatible with preference information provided by the Decision Maker (DM), we propose to apply the Robust Ordinal Regression (ROR). ROR takes into account simultaneously all the sets of parameters compatible with the preference information provided by the DM considering a necessary and a possible preference relation.

Bayesian Nonparametric Modeling for Multivariate Ordinal Regression

Abstract: Univariate or multivariate ordinal responses are often assumed to arise from a latent continuous parametric distribution, with covariate effects which enter linearly. We introduce a Bayesian nonparametric modeling approach for univariate and multivariate ordinal regression, which is based on mixture modeling for the joint distribution of latent responses and covariates. The modeling framework enables highly flexible inference for ordinal regression relationships, avoiding assumptions of linearity or additivity in the covariate effects. In standard parametric ordinal regression models, computational challenges arise from identifiability constraints and estimation of parameters requiring nonstandard inferential techniques. A key feature of the nonparametric model is that it achieves inferential flexibility, while avoiding these difficulties. In particular, we establish full support of the nonparametric mixture model under fixed cut-off points that relate through discretization the latent continuous responses with the ordinal responses. The practical utility of the modeling approach is illustrated through application to two data sets from econometrics, an example involving regression relationships for ozone concentration, and a multirater agreement problem.

Preferential reducts and constructs in robust multiple criteria ranking and sorting

Abstract: We revisit the multiple criteria ranking and sorting methods based on ordinal regression, which accept preference information in the form of, respectively, pairwise comparisons or assignment examples for some reference alternatives. Robust ordinal regression methods consider the whole set of value functions reproducing these holistic statements provided at the input. Its impact on the recommendation is expressed in terms of the necessary and possible preference relations or assignments. We propose methods for generating explanations of this impact, showing pieces of preference information provided by the decision maker (DM), which led to the observed outcomes. In particular, the minimal set of preference information pieces, called preferential reduct, is identified to justify some result observable for the whole set of compatible value functions (e.g., the truth of the necessary relation for some pair of alternatives). Further, the maximal set of preference information pieces, called preferential construct, is discovered to reveal the conditions under which some result non-observable for the whole set of compatible value functions (e.g., the falsity of the possible relation for some pair of alternatives) is possible. Knowing such explanations, the DM can better understand the impact of each piece of preference information on the result and, in consequence, get conviction about the obtained recommendation.

Ordinary Least Squares Regression of Ordered Categorical Data: Inferential Implications for Practice

Abstract: Ordered categorical variables are frequently encountered as response variables in many disciplines. Agricultural examples include quality assessments of soil or food products, and evaluation of lesion severity, such as teat ends status in dairy cattle. Ordered categorical responses (OCR) are characterized by multiple levels recorded on a ranked scale, whereby levels appraise order but may not be informative of relative magnitude or proportionality between levels. A number of statistically sound methods are available in the standard toolbox to deal with OCR, such as constrained cumulative logit and probit models; however, these are commonly underutilized in practice. Instead, ordinary least squares linear regression (OLSLR) is often employed to infer upon OCR, despite violation of basic model assumptions. In this study, we investigate the inferential implications of OLSLR-based inference on OCR using simulated data to explore realized Type I error rate and realized statistical power under a variety of scenarios. The design of the simulation study was motivated by a data application, thus considering increasing number of levels and various frequency distributions of the OCR. We then illustrate inferential performance of OLSLR relative to a probit regression model fitted to an OCR using a survey dataset of veterinarian antimicrobial use in cattle feedlots. This article has supplementary material online.

A simple categorical chart for detecting location shifts with ordinal information

Abstract: Traditional control charts for monitoring attribute data usually neglect the order among the attribute levels, such as good, marginal and bad, of a categorical factor. Such order may be reflected by an underlying continuous variable, which determines the level of the categorical factor by classifying it according to some thresholds in the latent continuous scale. This paper exploits this ordinal information and proposes a control chart for detecting location shifts in the latent variable based on merely the attribute level counts, regardless of the continuous values of the latent variable. The proposed ordinal chart is very simple to construct and enjoys the same setting as conventional categorical charts. Numerical simulations demonstrate the superiority of this simple ordinal categorical chart.