Pairwise comparison

About: Pairwise comparison is a research topic. Over the lifetime, 6804 publications have been published within this topic receiving 174081 citations.

Development of Reusable Bid Evaluation Models for the Portuguese Electric Transmission Company

Abstract: Bid evaluation is the process of selecting a contractor from a number of bidders. The decision analysis models currently in use at the Portuguese Electric Transmission Company (REN) to evaluate bids were developed through a decision-conferencing process supported by the MACBETH multicriteria approach and software. This paper presents the various components of this interactive sociotechnical process. Given the number of contracts awarded by REN each year, it was crucial that the models be reusable in similar calls for tenders; this required substantial care in structuring the criteria, with a focus on constructed scales, and building value function models based on qualitative pairwise comparison judgments of difference in attractiveness. Also of particular interest is the approach for weighing benefits against costs.

Probabilistic preference learning with the Mallows rank model

Abstract: Ranking and comparing items is crucial for collecting information about preferences in many areas, from marketing to politics. The Mallows rank model is among the most successful approaches to analyse rank data, but its computational complexity has limited its use to a particular form based on Kendall distance. We develop new computationally tractable methods for Bayesian inference in Mallows models that work with any right-invariant distance. Our method performs inference on the consensus ranking of the items, also when based on partial rankings, such as top-k items or pairwise comparisons. We prove that items that none of the assessors has ranked do not influence the maximum a posteriori consensus ranking, and can therefore be ignored. When assessors are many or heterogeneous, we propose a mixture model for clustering them in homogeneous subgroups, with cluster-specific consensus rankings. We develop approximate stochastic algorithms that allow a fully probabilistic analysis, leading to coherent quantifications of uncertainties. We make probabilistic predictions on the class membership of assessors based on their ranking of just some items, and predict missing individual preferences, as needed in recommendation systems. We test our approach using several experimental and benchmark datasets.

On Axiomatization of Inconsistency Indicators for Pairwise Comparisons

Abstract: This study examines the notion of inconsistency in pairwise comparisons for providing an axiomatization for it. It also proposes two inconsistency indicators for pairwise comparisons. The primary motivation for the inconsistency reduction is expressed by a computer industry concept “garbage in, garbage out”. The quality of the output depends on the quality of the input.

Face Clustering: Representation and Pairwise Constraints

Abstract: Clustering face images according to their latent identity has two important applications: 1) grouping a collection of face images when no external labels are associated with images, and 2) indexing for efficient large scale face retrieval. The clustering problem is composed of two key parts: representation and similarity metric for face images, and choice of the partition algorithm. We first propose a representation based on ResNet, which has been shown to perform very well in image classification problems. Given this representation, we design a clustering algorithm, Conditional Pairwise Clustering (ConPaC), which directly estimates the adjacency matrix only based on the similarities between face images. This allows a dynamic selection of number of clusters and retains pairwise similarities between faces. ConPaC formulates the clustering problem as a Conditional Random Field model and uses Loopy Belief Propagation to find an approximate solution for maximizing the posterior probability of the adjacency matrix. Experimental results on two benchmark face datasets (LFW and IJB-B) show that ConPaC outperforms well known clustering algorithms such as k-means, spectral clustering, and approximate Rank-order. Additionally, our algorithm can naturally incorporate pairwise constraints to work in a semi-supervised way that leads to improved clustering performance. We also propose a k-NN variant of ConPaC, which has a linear time complexity given a k-NN graph, suitable for large datasets.

Preference Completion: Large-scale Collaborative Ranking from Pairwise Comparisons

Abstract: In this paper we consider the collaborative ranking setting: a pool of users each provides a small number of pairwise preferences between d possible items; from these we need to predict each users preferences for items they have not yet seen. We do so by fitting a rank r score matrix to the pairwise data, and provide two main contributions: (a) we show that an algorithm based on convex optimization provides good generalization guarantees once each user provides as few as O(r log2 d) pairwise comparisons - essentially matching the sample complexity required in the related matrix completion setting (which uses actual numerical as opposed to pairwise information), and (b) we develop a large-scale non-convex implementation, which we call AltSVM, that trains a factored form of the matrix via alternating minimization (which we show reduces to alternating SVM problems), and scales and parallelizes very well to large problem settings. It also outperforms common baselines on many moderately large popular collaborative filtering datasets in both NDCG and in other measures of ranking performance.