A comparison of three weight elicitation methods: good, better, and best

TLDR: Three weight elicitation methods are shown to have very distinct "signatures", that is profiles relating weights to rank position, and people actually preferred using Max100 and DR rather than Min10, an important pragmatic consideration.

Abstract: This paper compares the properties and performance of three weight elicitation methods It is in effect a second round contest in which the Bottomley et al (2000) champion, direct rating (DR), locks horns with two new challengers People using DR rate each attribute in turn on a scale of 0–100, whilst people using Max100 first assign to the most important attribute(s) a rating of 100, and then rate the other attributes relative to it/them People using Min10 first assign the least important attribute(s) a rating of 10, and then rate the other attributes relative to it/them The weights produced by Max100 were somewhat more test–retest reliable than DR Both methods were considerably more reliable than Min10 Using people's test–retest data as attribute weights on simulated alternative values in a multi-attribute choice scenario, the same alternative would be chosen on 91% of occasions using Max100, 87% of occasions using DR, but only 75% of occasions using Min10 Moreover, the three methods are shown to have very distinct “signatures�, that is profiles relating weights to rank position Finally, people actually preferred using Max100 and DR rather than Min10, an important pragmatic consideration