Showing papers on "Ordinal regression published in 2013"

Robust ordinal regression in preference learning and ranking

Abstract: Multiple Criteria Decision Aiding (MCDA) offers a diversity of approaches designed for providing the decision maker (DM) with a recommendation concerning a set of alternatives (items, actions) evaluated from multiple points of view, called criteria. This paper aims at drawing attention of the Machine Learning (ML) community upon recent advances in a representative MCDA methodology, called Robust Ordinal Regression (ROR). ROR learns by examples in order to rank a set of alternatives, thus considering a similar problem as Preference Learning (ML-PL) does. However, ROR implements the interactive preference construction paradigm, which should be perceived as a mutual learning of the model and the DM. The paper clarifies the specific interpretation of the concept of preference learning adopted in ROR and MCDA, comparing it to the usual concept of preference learning considered within ML. This comparison concerns a structure of the considered problem, types of admitted preference information, a character of the employed preference models, ways of exploiting them, and techniques to arrive at a final ranking.

Stochastic ordinal regression for multiple criteria sorting problems

Abstract: We present a new approach for multiple criteria sorting problems. We consider sorting procedures applying general additive value functions compatible with the given assignment examples. For the decision alternatives, we provide four types of results: (1) necessary and possible assignments from Robust Ordinal Regression (ROR), (2) class acceptability indices from a suitably adapted Stochastic Multicriteria Acceptability Analysis (SMAA) model, (3) necessary and possible assignment-based preference relations, and (4) assignment-based pair-wise outranking indices. We show how the results provided by ROR and SMAA complement each other and combine them under a unified decision aiding framework. Application of the approach is demonstrated by classifying 27 countries in 4 democracy regimes.

On Nonparametric Ordinal Classification with Monotonicity Constraints

Abstract: We consider the problem of ordinal classification with monotonicity constraints. It differs from usual classification by handling background knowledge about ordered classes, ordered domains of attributes, and about a monotonic relationship between an evaluation of an object on the attributes and its class assignment. In other words, the class label (output variable) should not decrease when attribute values (input variables) increase. Although this problem is of great practical importance, it has received relatively low attention in machine learning. Among existing approaches to learning with monotonicity constraints, the most general is the nonparametric approach, where no other assumption is made apart from the monotonicity constraints assumption. The main contribution of this paper is the analysis of the nonparametric approach from statistical point of view. To this end, we first provide a statistical framework for classification with monotonicity constraints. Then, we focus on learning in the nonparametric setting, and we consider two approaches: the "plug-in" method (classification by estimating first the class conditional distribution) and the direct method (classification by minimization of the empirical risk). We show that these two methods are very closely related. We also perform a thorough theoretical analysis of their statistical and computational properties, confirmed in a computational experiment.

A goodness-of-fit test for the proportional odds regression model.

Abstract: We examine goodness-of-fit tests for the proportional odds logistic regression model-the most commonly used regression model for an ordinal response variable. We derive a test statistic based on the Hosmer-Lemeshow test for binary logistic regression. Using a simulation study, we investigate the distribution and power properties of this test and compare these with those of three other goodness-of-fit tests. The new test has lower power than the existing tests; however, it was able to detect a greater number of the different types of lack of fit considered in this study. Moreover, the test allows for the results to be summarized in a contingency table of observed and estimated frequencies, which is a useful supplementary tool to assess model fit. We illustrate the ability of the tests to detect lack of fit using a study of aftercare decisions for psychiatrically hospitalized adolescents. The test proposed in this paper is similar to a recently developed goodness-of-fit test for multinomial logistic regression. A unified approach for testing goodness of fit is now available for binary, multinomial, and ordinal logistic regression models.

Putting Dominance-based Rough Set Approach and robust ordinal regression together

Abstract: We propose to apply the Dominance-based Rough Set Approach (DRSA) on the results of multiple criteria decision aiding (MCDA) methods, in order to explain their recommendations in terms of rules involving conditions on evaluation criteria. The rules represent a decision model which is transparent and easy to interpret for the DM. In fact, decision rules give arguments to justify and explain the decision and, in a learning perspective, they can be the starting point for an interactive procedure for analyzing and constructing the DM's preferences. It enables his/her understanding of the conditions for the suggested recommendation, and provides useful information about the role of particular criteria or their subsets. DRSA can be used in junction with any MCDA method producing a classification result or a preference relation in the set of alternatives. In this paper, we apply DRSA to a recently proposed MCDA methodology, called Robust Ordinal Regression (ROR). The ROR approach to MCDA, also called disaggregation-aggregation approach, aims at inferring parameters of a preference model representing some holistic preference comparisons of alternatives provided by the decision maker (DM). Contrary to the usual ordinal regression approaches to MCDA, ROR takes into account the whole set of possible value of preference model parameters compatible with the DM's preference information, to work out a final recommendation. In consequence, ROR gives a recommendation in terms of necessary and possible consequences of the application of all the compatible sets of parameter values to the considered set of alternatives. UTAGMS and GRIP methods apply this approach, considering general monotonic additive value functions, and produce as a result the necessary and possible preference relations. In this paper we show how DRSA completes the decision aiding process started with ROR, providing a very useful interpretation of the preference relations in terms of decision rules. Highlights? We apply the Dominance-based Rough Set Approach (DRSA) on the results of MCDA methods. ? DRSA completes the decision aiding process started with Robust Ordinal Regression. ? DRSA explains necessary and possible preference relations in terms of decision rules. ? Decision rules are useful for an interactive construction of DM's preferences. ? DRSA can be used in this way with any MCDA method to explain their recommendations.

Automatic Pain Intensity Estimation using Heteroscedastic Conditional Ordinal Random Fields

Abstract: Automatic pain intensity estimation from facial images is challenging mainly because of high variability in subject-specific pain expressiveness This heterogeneity in the subjects causes their facial appearance to vary significantly when experiencing the same pain level The standard classification methods (eg, SVMs) do not provide a principled way of accounting for this heterogeneity To this end, we propose the heteroscedastic Conditional Ordinal Random Field (CORF) model for automatic estimation of pain intensity This model generalizes the CORF framework for modeling sequences of ordinal variables, by adapting it for heteroscedasticity This is attained by allowing the variance in the ordinal probit model in the CORF to change depending on the input features, resulting in the model able to adapt to the pain expressiveness level specific to each subject Our experimental results on the UNBC Shoulder Pain Database show that modeling heterogeneity in the subjects with the framework of CORFs improves the pain intensity estimation attained by the standard CORF model, and the other commonly used classification models

Selection of a Representative Value Function for Robust Ordinal Regression in Group Decision Making

Abstract: In this paper, we introduce the concept of a representative value function in a group decision context. We extend recently proposed methods UTA GMS -GROUP and UTADIS GMS -GROUP with selection of a compromise and collective preference model which aggregates preferences of several decision makers (DMs) and represents all instances of preference models compatible with preference information elicited from DMs. The representative value function is built on results of robust ordinal regression, so its representativeness can be interpreted in terms of robustness concern. We propose a few procedures designed for multiple criteria ranking, choice, and sort- ing problems. The use of these procedures is conditioned by both satisfying different degrees of consistency of the preference information provided by all DMs, as well as by some properties of particular decision making situations. The representative value function is intended to help the DMs to understand the robust results, and to provide them with a compromise result in case of conflict between the DMs.

DIS-CARD: a new method of multiple criteria sorting to classes with desired cardinality

Abstract: In this paper, we present a new preference disaggregation method for multiple criteria sorting problems, called DIS-CARD. Real-life experience indicates the need of considering decision making situations in which a decision maker (DM) specifies a desired number of alternatives to be assigned to single classes or to unions of some classes. These situations require special methods for multiple criteria sorting subject to desired cardinalities of classes. DIS-CARD deals with such a problem, using the ordinal regression approach to construct a model of DM’s preferences from preference information provided in terms of exemplary assignments of some reference alternatives, together with the above desired cardinalities. We develop a mathematical model for incorporating such preference information via mixed integer linear programming (MILP). Then, we adapt the MILP model to two types of preference models: an additive value function and an outranking relation. Illustrative example is solved to illustrate the methodology.

Logistic Regression Models for Multinomial and Ordinal Outcomes

Abstract: In the previous chapters we focused on the use of the logistic regression model when the outcome variable is dichotomous or binary. This model can be easily modified to handle the case where the outcome variable is nominal with more than two levels. For example, consider a study of choice of a health plan from among three plans offered to the employees of a large corporation. The outcome variable has three levels indicating which plan, A, B or C is chosen. Possible covariates might include gender, age, income, family size, and others. The goal is to estimate the probability of choosing each of the three plans as well as to estimate the odds of plan choice as a function of the covariates and to express the results in terms of odds ratios for choice of different plans. McFadden (1974) proposed a modification of the logistic regression model and called it a discrete choice model. As a result, the model frequently goes by that name in the business and econometric literature while it is called the multinomial, polychotomous, or polytomous logistic regression model in the health and life sciences. We use the term multinomial in this text. It would be possible to use an outcome variable with any number of levels to illustrate the extension of the model and methods. However, the details are most easily illustrated with three categories. Further generalization to more than three categories is a problem more of notation than of concept. Hence, in the remainder of this section, we restrict our attention to the situation where the outcome variable has three categories. When one considers a regression model for a discrete outcome variable with more than two responses, one must pay attention to the measurement scale. In this section, we discuss the logistic regression model for the case in which the outcome

Exploitation of pairwise class distances for ordinal classification

Abstract: Ordinal classification refers to classification problems in which the classes have a natural order imposed on them because of the nature of the concept studied. Some ordinal classification approaches perform a projection from the input space to one-dimensional latent space that is partitioned into a sequence of intervals one for each class. Class identity of a novel input pattern is then decided based on the interval its projection falls into. This projection is trained only indirectly as part of the overall model fitting. As with any other latent model fitting, direct construction hints one may have about the desired form of the latent model can prove very useful for obtaining high-quality models. The key idea of this letter is to construct such a projection model directly, using insights about the class distribution obtained from pairwise distance calculations. The proposed approach is extensively evaluated with 8 nominal and ordinal classifiers methods, 10 real-world ordinal classification data sets, and 4 different performance measures. The new methodology obtained the best results in average ranking when considering three of the performance metrics, although significant differences are found for only some of the methods. Also, after observing other methods of internal behavior in the latent space, we conclude that the internal projections do not fully reflect the intraclass behavior of the patterns. Our method is intrinsically simple, intuitive, and easily understandable, yet highly competitive with state-of-the-art approaches to ordinal classification.

Addressing the EU Sovereign Ratings Using an Ordinal Regression Approach

Abstract: The current European debt crisis has drawn considerable attention to credit-rating agencies' news about sovereign ratings. From a technical point of view, credit rating constitutes a typical ordinal regression problem because credit-rating agencies generally present a scale of risk composed of several categories. This fact motivated the use of an ordinal regression approach to address the problem of sovereign credit rating in this paper. Therefore, the ranking of different classes will be taken into account for the design of the classifier. To do so, a novel model is introduced in order to replicate sovereign rating, based on the negative correlation learning framework. The methodology is fully described in this paper and applied to the classification of the 27 European countries' sovereign rating during the 2007-2010 period based on Standard and Poor's reports. The proposed technique seems to be competitive and robust enough to classify the sovereign ratings reported by this agency when compared with other existing well-known ordinal and nominal methods.

Ordinal Response Modeling with the LOGISTIC Procedure

Abstract: Logistic regression is most often used for modeling simple binary response data. Two modifications extend it to ordinal responses that have more than two levels: using multiple response functions to model the ordered behavior, and considering whether covariates have common slopes across response functions. This paper describes how you can use the LOGISTIC procedure to model ordinal responses. Before SAS/STAT ® 12.1, you could use cumulative logit response functions with proportional odds. In SAS/STAT 12.1, you can fit partial proportional odds models to ordinal responses. This paper also discusses methods of determining which covariates have proportional odds. The reader is assumed to be familiar with using PROC LOGISTIC for binary logistic regression.

Negative Correlation Ensemble Learning for Ordinal Regression

Abstract: In this paper, two neural network threshold ensemble models are proposed for ordinal regression problems. For the first ensemble method, the thresholds are fixed a priori and are not modified during training. The second one considers the thresholds of each member of the ensemble as free parameters, allowing their modification during the training process. This is achieved through a reformulation of these tunable thresholds, which avoids the constraints they must fulfill for the ordinal regression problem. During training, diversity exists in different projections generated by each member is taken into account for the parameter updating. This diversity is promoted in an explicit way using a diversity-encouraging error function, extending the well-known negative correlation learning framework to the area of ordinal regression, and inheriting many of its good properties. Experimental results demonstrate that the proposed algorithms can achieve competitive generalization performance when considering four ordinal regression metrics.

Measuring Simulation-Observation Fit: An Introduction to Ordinal Pattern Analysis

Abstract: Most traditional strategies of assessing the fit between a simulation's set of predictions (outputs) and a set of relevant observations rely either on visual inspection or squared distances among averages. Here we introduce an alternative goodness-of-fit strategy, Ordinal Pattern Analysis (OPA) that will (we argue) be more appropriate for judging the goodness-of-fit of simulations in many situations. OPA is based on matches and mismatches among the ordinal properties of predictions and observations. It does not require predictions or observations to meet the requirements of interval or ratio measurement scales. In addition, OPA provides a means to assess prediction-observation fits case-by-case prior to aggregation, and to map domains of validity of competing simulations. We provide examples to illustrate how OPA can be employed to assess the ordinal fit and domains of validity of simulations of share prices, crime rates, and happiness ratings. We also provide a computer programme for assisting in the calculation of OPA indices.

Prognostic Factors for Successful Work Functioning in the General Working Population

Abstract: Purpose To help workers to stay at work in a healthy productive and sustainable way and for the development of interventions to improve work functioning, it is important to have insight in prognostic factors for successful work functioning. The aim of this study is to identify prognostic factors for successful work functioning in a general working population. Methods A longitudinal study (3 months follow-up) was conducted among the working population (N = 98). Work functioning was assessed with the Work Role Functioning Questionnaire 2.0 (WRFQ). The total score was categorized as follows: 0–90; >90 ≤95; and >95–100 (defined as ‘successful work functioning’). Ordinal logistic regression analyses were performed to examine bivariate relationships between potential prognostic factors and the dependent variable (successful work functioning) to identify potential prognostic factors for the multivariate models (p < 0.10). A stepwise approach was used to introduce the variables in the multiple ordinal regression analyses. Results Baseline work functioning and work ability were significant prognostic factors for successful work functioning at 3 months follow-up. No prospective associations were identified for psychological job demands and supervisor social support with successful work functioning. Conclusion To our knowledge this is the first longitudinal study to identify prognostic factors for successful work functioning in the general working population. High work ability is predictive for future successful work functioning, independent of baseline work functioning.

Multinomial logit models with implicit variable selection

Abstract: The multinomial logit model is the most widely used model for the unordered multi-category responses. However, applications are typically restricted to the use of few predictors because in the high-dimensional case maximum likelihood estimates frequently do not exist. In this paper we are developing a boosting technique called multinomBoost that performs variable selection and fits the multinomial logit model also when predictors are high-dimensional. Since in multi-category models the effect of one predictor variable is represented by several parameters one has to distinguish between variable selection and parameter selection. A special feature of the approach is that, in contrast to existing approaches, it selects variables not parameters. The method can also distinguish between mandatory predictors and optional predictors. Moreover, it adapts to metric, binary, nominal and ordinal predictors. Regularization within the algorithm allows to include nominal and ordinal variables which have many categories. In the case of ordinal predictors the order information is used. The performance of boosting technique with respect to mean squared error, prediction error and the identification of relevant variables is investigated in a simulation study. The method is applied to the national Indonesia contraceptive prevalence survey and the identification of glass. Results are also compared with the Lasso approach which selects parameters.

Fuzzy Composite Indicators: An Application for Measuring Customer Satisfaction

Abstract: Composite indicators should ideally measure multidimensional concepts which cannot be captured by a single variable. In this chapter, we suggest a method based on fuzzy set theory for the construction of a fuzzy synthetic index of a latent phenomenon (e.g., well-being, quality of life, etc.), using a set of manifest variables measured on different scales (quantitative, ordinal and binary). A few criteria for assigning values to the membership function are discussed, as well as criteria for defining the weights of the variables. For ordinal variables, we propose a fuzzy quantification method based on the sampling cumulative function and a weighting system which takes into account the relative frequency of each category. An application regarding the results of a survey on the users of a contact center is presented.

Using micro-documents for feature selection: The case of ordinal text classification

Abstract: Most popular feature selection methods for text classification such as information gain (also known as ''mutual information''), chi-square, and odds ratio, are based on binary information indicating the presence/absence of the feature (or ''term'') in each training document. As such, these methods do not exploit a rich source of information, namely, the information concerning how frequently the feature occurs in the training document (term frequency). In order to overcome this drawback, when doing feature selection we logically break down each training document of length k into k training ''micro-documents'', each consisting of a single word occurrence and endowed with the same class information of the original training document. This move has the double effect of (a) allowing all the original feature selection methods based on binary information to be still straightforwardly applicable, and (b) making them sensitive to term frequency information. We study the impact of this strategy in the case of ordinal text classification, a type of text classification dealing with classes lying on an ordinal scale, and recently made popular by applications in customer relationship management, market research, and Web 2.0 mining. We run experiments using four recently introduced feature selection functions, two learning methods of the support vector machines family, and two large datasets of product reviews. The experiments show that the use of this strategy substantially improves the accuracy of ordinal text classification.

Semi-supervised Gaussian Process ordinal regression

Abstract: Ordinal regression problem arises in situations where examples are rated in an ordinal scale. In practice, labeled ordinal data are difficult to obtain while unlabeled ordinal data are available in abundance. Designing a probabilistic semi-supervised classifier to perform ordinal regression is challenging. In this work, we propose a novel approach for semi-supervised ordinal regression using Gaussian Processes (GP). It uses the expectation-propagation approximation idea, widely used for GP ordinal regression problem. The proposed approach makes use of unlabeled data in addition to the labeled data to learn a model by matching ordinal label distributions approximately between labeled and unlabeled data. The resulting mixed integer programming problem, involving model parameters (real-valued) and ordinal labels (integers) as variables, is solved efficiently using a sequence of alternating optimization steps. Experimental results on synthetic, bench-mark and real-world data sets demonstrate that the proposed GP based approach makes effective use of the unlabeled data to give better generalization performance (on the absolute error metric, in particular) than the supervised approach. Thus, it is a useful approach for probabilistic semi-supervised ordinal regression problem.

Ordinal Choquistic Regression

Abstract: We propose an extension of choquistic regression from the case of binary to the case of ordinal classification. Choquistic regression itself has been introduced recently as a generalization of conventional logistic regression. The basic idea of this method is to replace the linear function of predictor variables in the logistic regression model by the Choquet integral. Thus, it becomes possible to capture nonlinear dependencies and interactions among predictor variables while preserving two important properties of logistic regression, namely the comprehensibility of the model and the possibility to ensure its monotonicity in individual predictors. In experimental studies, choquistic regression consistently improves upon standard logistic regression in terms of predictive accuracy, especially when being combined with a novel regularization technique that prevents from exceeding the required level of nonadditivity.

Binary and Ordinal Data Analysis in Economics: Modeling and Estimation

Abstract: This chapter is concerned with the analysis of statistical models for binary and ordinal outcomes Binary data arise when a particular response variable of interest yi can take only two values, ie yi ∈ {0, 1}, where the index i = 1, , n refers to units in the sample such as individuals, families, firms, and so on Such dichotomous outcomes are widespread in the social and natural sciences For example, to understand socio-economic processes, economists often need to analyze individuals’ binary decisions such as whether to make a particular purchase, participate in the labor force, obtain a college degree, see a doctor, migrate to a different country, or vote in an election By convention, yi = 1 typically indicates the occurrence of the event of interest, whereas the occurrence of its complement is denoted by yi = 0

Ordinal regression models for predicting deoxynivalenol in winter wheat

Abstract: Deoxynivalenol (DON) is one of the most prevalent toxins in Fusarium-infected wheat samples. Accurate forecasting systems that predict the presence of DON are useful to underpin decision making on the application of fungicides, to identify fields under risk, and to help minimize the risk of food and feed contamination with DON. To this end, existing forecasting systems often adopt statistical regression models, in which attempts are made to predict DON values as a continuous variable. In contrast, this paper advocates the use of ordinal regression models for the prediction of DON values, by defining thresholds for converting continuous DON values into a fixed number of well-chosen risk classes. Objective criteria for selecting these thresholds in a meaningful way are proposed. The resulting approach was evaluated on a sizeable field experiment in Belgium, for which measurements of DON values and various types of predictor variables were collected at 18 locations during 2002-2011. The results demonstrate that modelling and evaluating DON values on an ordinal scale leads to a more accurate and more easily interpreted predictive performance. Compared to traditional regression models, an improvement could be observed for support vector ordinal regression models and proportional odds models.

Latent variable models for ordinal data by using the adaptive quadrature approximation

Abstract: Latent variable models for ordinal data represent a useful tool in different fields of research in which the constructs of interest are not directly observable so that one or more latent variables are required to reduce the complexity of the data. In these cases problems related to the integration of the likelihood function of the model can arise. Indeed analytical solutions do not exist and in presence of several latent variables the most used classical numerical approximation, the Gauss Hermite quadrature, cannot be applied since it requires several quadrature points per dimension in order to obtain quite accurate estimates and hence the computational effort becomes not feasible. Alternative solutions have been proposed in the literature, like the Laplace approximation and the adaptive quadrature. Different studies demonstrated the superiority of the latter method particularly in presence of categorical data. In this work we present a simulation study for evaluating the performance of the adaptive quadrature approximation for a general class of latent variable models for ordinal data under different conditions of study. A real data example is also illustrated.

Measuring information interactions on the ordinal pattern of stock time series

Abstract: The interactions among time series as individual components of complex systems can be quantified by measuring to what extent they exchange information among each other. In many applications, one focuses not on the original series but on its ordinal pattern. In such cases, trivial noises appear more likely to be filtered and the abrupt influence of extreme values can be weakened. Cross-sample entropy and inner composition alignment have been introduced as prominent methods to estimate the information interactions of complex systems. In this paper, we modify both methods to detect the interactions among the ordinal pattern of stock return and volatility series, and we try to uncover the information exchanges across sectors in Chinese stock markets.

Multicriteria Models for Learning Ordinal Data: A Literature Review

Abstract: Operations Research (OR) and Artificial Intelligence (AI) disciplines have been playing major roles on the design of new intelligent systems. Recently, different contributions from both fields have been made on the models design for problems with multi-criteria. The credit scoring problem is an example of that. In this problem, one evaluates how unlikely a client will default with his payments. Client profiles are evaluated, being their results expressed in terms of an ordinal score scale (Excelent ≻ Good ≻ Fair ≻ Poor). Intelligent systems have then to take in consideration different criteria such as payment history, mortgages, wages among others in order to commit their outcome. To achieve this goal, researchers have been delving models capable to render these multiple criteria encompassed on ordinal data.

Context-Sensitive Conditional Ordinal Random Fields for Facial Action Intensity Estimation

Abstract: We address the problem of modeling intensity levels of facial actions in video sequences. The intensity sequences often exhibit a large variability due to the context factors, such as the person-specific facial expressiveness or changes in illumination. Existing methods usually attempt to normalize this variability in data using different feature-selection and/or data pre-processing schemes. Consequently, they ignore the context in which the target facial actions occur. We propose a novel Conditional Random Field (CRF) based ordinal model for context-sensitive modeling of the facial action unit intensity, where the W5+ (Who, When, What, Where, Why and How) definition of the context is used. In particular, we focus on three contextual questions: Who (the observed person), How (the changes in facial expressions), and When (the timing of the facial expression intensity). The contextual questions Who and How are modeled by means of the newly introduced covariate effects, while the contextual question When is modeled in terms of temporal correlation between the intensity levels. We also introduce a weighted softmax-margin learning of CRFs from the data with a skewed distribution of the intensity levels, as commonly encountered in spontaneous facial data. The proposed model is evaluated for intensity estimation of facial action units and facial expressions of pain from the UNBC Shoulder Pain dataset. Our experimental results show the effectiveness of the proposed approach.