Does technique matter; a pilot study exploring weighting techniques for a multi-criteria decision support framework
Janine Astrid van Til,Catharina G.M. Groothuis-Oudshoorn,Marijke Lieferink,James G. Dolan,Mireille Goetghebeur +4 more
Reads0
Chats0
TLDR
The results of this study indicate that there is no effect of differences in weights on value estimates at the group level and weight elicitation through pairwise comparison of criteria is preferred when taking into account its superior ability to discriminate between criteria and respondents’ preferences.Abstract:
Background
There is an increased interest in the use of multi-criteria decision analysis (MCDA) to support regulatory and reimbursement decision making. The EVIDEM framework was developed to provide pragmatic multi-criteria decision support in health care, to estimate the value of healthcare interventions, and to aid in priority-setting. The objectives of this study were to test 1) the influence of different weighting techniques on the overall outcome of an MCDA exercise, 2) the discriminative power in weighting different criteria of such techniques, and 3) whether different techniques result in similar weights in weighting the criteria set proposed by the EVIDEM framework.
Methods
A sample of 60 Dutch and Canadian students participated in the study. Each student used an online survey to provide weights for 14 criteria with two different techniques: a five-point rating scale and one of the following techniques selected randomly: ranking, point allocation, pairwise comparison and best worst scaling.
Results
The results of this study indicate that there is no effect of differences in weights on value estimates at the group level. On an individual level, considerable differences in criteria weights and rank order occur as a result of the weight elicitation method used, and the ability of different techniques to discriminate in criteria importance. Of the five techniques tested, the pair-wise comparison of criteria has the highest ability to discriminate in weights when fourteen criteria are compared.
Conclusions
When weights are intended to support group decisions, the choice of elicitation technique has negligible impact on criteria weights and the overall value of an innovation. However, when weights are used to support individual decisions, the choice of elicitation technique influences outcome and studies that use dissimilar techniques cannot be easily compared. Weight elicitation through pairwise comparison of criteria is preferred when taking into account its superior ability to discriminate between criteria and respondents’ preferencesread more
Citations
More filters
Book ChapterDOI
SMARTS and SMARTER: Improved Simple Methods for Multiattribute Utility Measurement
Jie W Weiss,David J. Weiss +1 more
Journal ArticleDOI
Multiple Criteria Decision Analysis for Health Care Decision Making—Emerging Good Practices: Report 2 of the ISPOR MCDA Emerging Good Practices Task Force
Kevin Marsh,Maarten Joost IJzerman,Praveen Thokala,Rob Baltussen,Meindert Boysen,Zoltán Kaló,Thomas Lönngren,Filip Mussen,Stuart Peacock,Stuart Peacock,John B. Watkins,Nancy Devlin +11 more
TL;DR: This second task force report provides emerging good-practice guidance on the implementation of multiple criteria decision analysis to support health care decisions and provides an overview of the skills and resources required to implement MCDA.
Journal ArticleDOI
Using Best–Worst Scaling to Investigate Preferences in Health Care
Kei Long Cheung,Ben F. M. Wijnen,Ilene L. Hollin,Ellen M. Janssen,John F.P. Bridges,Silvia M. A. A. Evers,Mickaël Hiligsmann +6 more
TL;DR: Use of BWS object case and BWS profile case has drastically increased in health care, especially in the last 2 years, and in contrast with previous discrete-choice experiment reviews, there is increasing use of less sophisticated analytical methods.
Journal ArticleDOI
Experimental measurement of preferences in health and healthcare using best-worst scaling: an overview
TL;DR: The article discusses the strengths and weaknesses of the three types of BWS distinguished and offered an outlook, and a companion paper focuses on special issues of theory and statistical inference confronting BWS in preference measurement.
Journal ArticleDOI
Making Good Decisions in Healthcare with Multi-Criteria Decision Analysis: The Use, Current Research and Future Development of MCDA.
TL;DR: The results showed that healthcare decision making is addressing the problem of multiple decision criteria and is focusing on the future development and use of techniques to weight and score different decision criteria.
References
More filters
Journal ArticleDOI
Statistical methods for assessing agreement between two methods of clinical measurement.
TL;DR: An alternative approach, based on graphical techniques and simple calculations, is described, together with the relation between this analysis and the assessment of repeatability.
Journal ArticleDOI
Statistical methods for assessing agreement between two methods of clinical measurement
TL;DR: In this article, an alternative approach, based on graphical techniques and simple calculations, is described, together with the relation between this analysis and the assessment of repeatability, which is often used in clinical comparison of a new measurement technique with an established one.
Journal ArticleDOI
Measurement in Medicine: The Analysis of Method Comparison Studies
Douglas G. Altman,J M Bland +1 more
TL;DR: This paper shall describe what is usually done, show why this is inappropriate, suggest a better approach, and ask why such studies are done so badly.
Book
Multiple Criteria Decision Analysis: An Integrated Approach
TL;DR: This book is not only a theoretical document, but also provides good coverage of practical issues and can be recommended to a broad audience, ranging from those in academic institutions to practitioners, as well as those who are interested in finding information on multiple criteria approaches, methods and techniques.
Journal ArticleDOI
The Construction of Preference
TL;DR: The idea that people's preferences are often constructed in the process of elicitation is derived from studies demonstrating that normatively equivalent elicitation (e.g., choice and pricing) give rise to systematically different responses.
Related Papers (5)
Priority Setting of health interventions: the need for multi-criteria decision analysis
Rob Baltussen,Louis W. Niessen +1 more
Multiple Criteria Decision Analysis for Health Technology Assessment
Praveen Thokala,Alejandra Duenas +1 more