Individual Comparisons by Ranking Methods

doi:10.1007/978-1-4612-4380-9_16

Home
/
Papers
/
Individual Comparisons by Ranking Methods

Book Chapter•DOI•

Individual Comparisons by Ranking Methods

Frank Wilcoxon¹•Institutions (1)

American Cyanamid¹

01 Dec 1945-Biometrics (Springer, New York, NY)-Vol. 1, Iss: 6, pp 196-202

TL;DR: The comparison of two treatments generally falls into one of the following two categories: (a) a number of replications for each of the two treatments, which are unpaired, or (b) we may have a series of paired comparisons, some of which may be positive and some negative as mentioned in this paper.

read less

Abstract: The comparison of two treatments generally falls into one of the following two categories: (a) we may have a number of replications for each of the two treatments, which are unpaired, or (b) we may have a number of paired comparisons leading to a series of differences, some of which may be positive and some negative. The appropriate methods for testing the significance of the differences of the means in these two cases are described in most of the textbooks on statistical methods.

...read moreread less

Citations

PDF

Open Access

More filters

Journal Article•DOI•

On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other

[...]

Henry B. Mann, D. R. Whitney

01 Mar 1947-Annals of Mathematical Statistics

TL;DR: In this paper, the authors show that the limit distribution is normal if n, n$ go to infinity in any arbitrary manner, where n = m = 8 and n = n = 8.

...read moreread less

Abstract: Let $x$ and $y$ be two random variables with continuous cumulative distribution functions $f$ and $g$. A statistic $U$ depending on the relative ranks of the $x$'s and $y$'s is proposed for testing the hypothesis $f = g$. Wilcoxon proposed an equivalent test in the Biometrics Bulletin, December, 1945, but gave only a few points of the distribution of his statistic. Under the hypothesis $f = g$ the probability of obtaining a given $U$ in a sample of $n x's$ and $m y's$ is the solution of a certain recurrence relation involving $n$ and $m$. Using this recurrence relation tables have been computed giving the probability of $U$ for samples up to $n = m = 8$. At this point the distribution is almost normal. From the recurrence relation explicit expressions for the mean, variance, and fourth moment are obtained. The 2rth moment is shown to have a certain form which enabled us to prove that the limit distribution is normal if $m, n$ go to infinity in any arbitrary manner. The test is shown to be consistent with respect to the class of alternatives $f(x) > g(x)$ for every $x$.

...read moreread less

11,055 citations

Cites methods from "Individual Comparisons by Ranking M..."

...+ - (2n m + 2nm2 -n2 _ m2 -nm) 16 which is obtained from (1) by multiplication by (U nm/2)4 and expansion....
[...]
...Using the recurrence relation (1) the probabilities pnm(U) have been tabulated for m < n < 8 (see Table I)....
[...]
...After multiplying (1) by (U nm/2)2, using...
[...]

Journal Article•

Statistical Comparisons of Classifiers over Multiple Data Sets

[...]

Janez Demšar

01 Dec 2006-Journal of Machine Learning Research

TL;DR: A set of simple, yet safe and robust non-parametric tests for statistical comparisons of classifiers is recommended: the Wilcoxon signed ranks test for comparison of two classifiers and the Friedman test with the corresponding post-hoc tests for comparisons of more classifiers over multiple data sets.

...read moreread less

Abstract: While methods for comparing two learning algorithms on a single data set have been scrutinized for quite some time already, the issue of statistical tests for comparisons of more algorithms on multiple data sets, which is even more essential to typical machine learning studies, has been all but ignored. This article reviews the current practice and then theoretically and empirically examines several suitable tests. Based on that, we recommend a set of simple, yet safe and robust non-parametric tests for statistical comparisons of classifiers: the Wilcoxon signed ranks test for comparison of two classifiers and the Friedman test with the corresponding post-hoc tests for comparison of more classifiers over multiple data sets. Results of the latter can also be neatly presented with the newly introduced CD (critical difference) diagrams.

...read moreread less

10,306 citations

Cites methods from "Individual Comparisons by Ranking M..."

...3.1.3 WILCOXON SIGNED-RANKS TEST The Wilcoxon signed-ranks test (Wilcoxon, 1945) is a non-parametric altern tive to the paired t-test, which ranks the differences in performances of two classifiers for each d ta set, ignoring the signs, and compares the ranks for the positive and the negative…...
[...]
...Since we will finally recommend the Wilcoxon (1945) signed-ranks test, it will be presented with more details....
[...]

Journal Article•DOI•

Use of Ranks in One-Criterion Variance Analysis

[...]

William Kruskal¹, W. Allen Wallis¹•Institutions (1)

University of Chicago¹

01 Dec 1952-Journal of the American Statistical Association

TL;DR: In this article, a test of the hypothesis that the samples are from the same population may be made by ranking the observations from from 1 to Σn i (giving each observation in a group of ties the mean of the ranks tied for), finding the C sums of ranks, and computing a statistic H. Under the stated hypothesis, H is distributed approximately as χ2(C − 1), unless the samples were too small, in which case special approximations or exact tables are provided.

...read moreread less

Abstract: Given C samples, with n i observations in the ith sample, a test of the hypothesis that the samples are from the same population may be made by ranking the observations from from 1 to Σn i (giving each observation in a group of ties the mean of the ranks tied for), finding the C sums of ranks, and computing a statistic H. Under the stated hypothesis, H is distributed approximately as χ2(C – 1), unless the samples are too small, in which case special approximations or exact tables are provided. One of the most important applications of the test is in detecting differences among the population means.* * Based in part on research supported by the Office of Naval Research at the Statistical Research Center, University of Chicago.

...read moreread less

9,365 citations

Cites methods from "Individual Comparisons by Ranking M..."

...As is explained in Section 5.3, the H test for two samples is essentially the same as a test' published by Wilcoxon [ 61 ] in 1945 and later by others....
[...]

Journal Article•DOI•

Metagenomic biomarker discovery and explanation

[...]

Nicola Segata¹, Jacques Izard², Jacques Izard¹, Levi Waldron¹, Dirk Gevers³, Larisa Miropolsky¹, Wendy S. Garrett¹, Curtis Huttenhower¹ - Show less +4 more•Institutions (3)

Harvard University¹, The Forsyth Institute², Broad Institute³

24 Jun 2011-Genome Biology

TL;DR: A new method for metagenomic biomarker discovery is described and validates by way of class comparison, tests of biological consistency and effect size estimation to address the challenge of finding organisms, genes, or pathways that consistently explain the differences between two or more microbial communities.

...read moreread less

Abstract: This study describes and validates a new method for metagenomic biomarker discovery by way of class comparison, tests of biological consistency and effect size estimation. This addresses the challenge of finding organisms, genes, or pathways that consistently explain the differences between two or more microbial communities, which is a central problem to the study of metagenomics. We extensively validate our method on several microbiomes and a convenient online interface for the method is provided at http://huttenhower.sph.harvard.edu/lefse/.

...read moreread less

9,057 citations

Cites methods from "Individual Comparisons by Ranking M..."

...The LEfSe algorithm is introduced in overview in the Results section, and Figure 6 illustrates in detail the format of the input (a matrix with n rows and m columns) and the three steps performed by the computational tool: the KW rank sum test [49] on classes, the pairwise Wilcoxon test [50,51] between subclasses of different classes, and the LDA [52] on the relevant features....
[...]
...for example, given some known population structure within a set of input samples, is a feature more abundant in all population subclasses or in just one? Specifically, we first use the non-parametric factorial KruskalWallis (KW) sum-rank test [49] to detect features with significant differential abundance with respect to the class of interest; biological consistency is subsequently investigated using a set of pairwise tests among subclasses using the (unpaired) Wilcoxon rank-sum test [50,51]....
[...]

Book Chapter•DOI•

Probability Inequalities for sums of Bounded Random Variables

[...]

Wassily Hoeffding¹•Institutions (1)

University of North Carolina at Chapel Hill¹

01 Mar 1963-Journal of the American Statistical Association

TL;DR: In this article, upper bounds for the probability that the sum S of n independent random variables exceeds its mean ES by a positive number nt are derived for certain sums of dependent random variables such as U statistics.

...read moreread less

Abstract: Upper bounds are derived for the probability that the sum S of n independent random variables exceeds its mean ES by a positive number nt. It is assumed that the range of each summand of S is bounded or bounded above. The bounds for Pr {S – ES ≥ nt} depend only on the endpoints of the ranges of the summands and the mean, or the mean and the variance of S. These results are then used to obtain analogous inequalities for certain sums of dependent random variables such as U statistics and the sum of a random sample without replacement from a finite population.

...read moreread less

8,655 citations

1
2
3
4
…
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200

Collapse

References

PDF

Open Access

More filters

Journal Article•DOI•

The Use of Ranks to Avoid the Assumption of Normality Implicit in the Analysis of Variance

[...]

Milton Friedman

01 Dec 1937-Journal of the American Statistical Association

TL;DR: The use of ranks to avoid the assumption of normality implicit in the analysis of variance has been studied in this article, where the use of rank to avoid normality is discussed.

...read moreread less

Abstract: (1937). The Use of Ranks to Avoid the Assumption of Normality Implicit in the Analysis of Variance. Journal of the American Statistical Association: Vol. 32, No. 200, pp. 675-701.

...read moreread less

4,751 citations

Book•

The Design of Experiments

[...]

R. A. Fisher

01 Jan 1935

4,510 citations

Journal Article•DOI•

Combinatory Analysis. By Major P. A. MacMohan. Vol. I. Pp. xx + 300. 15s. net. (Cam. Univ. Press)

[...]

G. B. Mathews