Pairwise comparison

About: Pairwise comparison is a research topic. Over the lifetime, 6804 publications have been published within this topic receiving 174081 citations.

Tournaments, transitivity and pairwise comparison matrices

Abstract: When attempting to rank a number of items by pairwise comparisons, one is usually advised to guard against generating a preference structure that contains three-way intransitive relationships (three-way cycles; cyclic triads) such as A is preferred to B, B is preferred to C, and C is preferred to A. Some decision procedures, like the Analytic Hierarchy Process, do not rule out intransitivities, while others, like utility theory, have axioms that strictly forbid them. It is generally agreed that intransitivities can occur, especially when the number of items being compared under a multicriteria framework gets to be greater than five. It is also generally agreed that, if intransitivities are found, they should be analysed and changed, if deemed appropriate. That is, there is no inherent rule that says a set of comparisons should not contain any intransitivities, but they should be made explicit. In this paper, we show, using results from tournaments and graph theory, how one can readily determine the number of three-way cycles that exist within a pairwise comparison matrix, and, using standard linear programming procedures, how to find them.

Competition-based user expertise score estimation

Abstract: In this paper, we consider the problem of estimating the relative expertise score of users in community question and answering services (CQA). Previous approaches typically only utilize the explicit question answering relationship between askers and an-swerers and apply link analysis to address this problem. The im-plicit pairwise comparison between two users that is implied in the best answer selection is ignored. Given a question and answering thread, it's likely that the expertise score of the best answerer is higher than the asker's and all other non-best answerers'. The goal of this paper is to explore such pairwise comparisons inferred from best answer selections to estimate the relative expertise scores of users. Formally, we treat each pairwise comparison between two users as a two-player competition with one winner and one loser. Two competition models are proposed to estimate user expertise from pairwise comparisons. Using the NTCIR-8 CQA task data with 3 million questions and introducing answer quality prediction based evaluation metrics, the experimental results show that the pairwise comparison based competition model significantly outperforms link analysis based approaches (PageRank and HITS) and pointwise approaches (number of best answers and best answer ratio) for estimating the expertise of active users. Furthermore, it's shown that pairwise comparison based competi-tion models have better discriminative power than other methods. It's also found that answer quality (best answer) is an important factor to estimate user expertise.

A tree reconstruction method that is economical in the number of pairwise comparisons used.

Abstract: A fast method for reconstructing phylogenies from distance data is presented. The method is economical in the number of pairwise comparisons needed. It can be combined with a new phylogenetic alignment procedure to yield an algorithm that gives a complete history of a set of homologous sequences. The method is applicable to very large distance matrices. An auxiliary program was developed that simplifies large phylogenies without ignoring biologically essential features. A set of 2 13 globins from vertebrates, plants, and Vitreoscilla (a prokaryote) were analyzed using this method.