Testing the stability of utility parameters in repeated best, repeated best-worst and one-off best-worst studies
Open Access
- 1 September 2017
- journal article
- Published by TU Delft OPEN Publishing in European Journal Of Transport and Infrastructure Research
- Vol. 17 (4)
Abstract
Stated choice (SC) surveys are a key tool for studying travel behaviour and are used to inform policy decisions in many countries. Recently, the best-worst (BW) variant of SC has rapidly increased in popularity in fields as diverse as transport, marketing and health research. A key argument for its implementation has been that it is perceived to be easier for respondents to identify the best and the worst alternative in a choice set compared to identifying the second- or even third best. For elicitation formats asking respondents to consecutively identify the first, second and third best (etc.) alternative, labelled here as repeated best stages, it is well known that utility and scale parameters are generally not stable across the stages. Joint analysis of the responses to each stage may increase the efficiency of the utility and willingness-to-pay parameters (i.e. smaller standard errors), but incorrect inferences may be made if these parameters are not stable across the stages. This paper tests the stability of utility parameters for the repeated BW and one-off BW format. Using data from three different studies, we show that, regardless of the dataset and elicitation format used, the obtained utility parameters and willingness to pay estimates are not stable across stages. The results thereby question the use of BW data in applied work aimed at forecasting and understanding first (best) choices. Our findings thereby contradict recent discussions about potentially beneficial framing effects in BW surveys. The unique presence of corresponding data from a repeated best and repeated BW exercise in one survey highlights the observed rank-orders are highly consistent across the two elicitation formats and that any differences in marginal willingness to pay estimates can be attributed to the imposed econometric model rather than to differences in the behaviour of respondents.Keywords
This publication has 19 references indexed in Scilit:
- Best worst discrete choice experiments in health: Methods and an applicationSocial Science & Medicine, 2013
- It’s not that I don’t care, I just don’t care very much: confounding between attribute non-attendance and taste heterogeneityTransportation, 2012
- Models of best–worst choice and ranking among multiattribute options (profiles)Journal of Mathematical Psychology, 2012
- A RANK‐ORDERED LOGIT MODEL WITH UNOBSERVED HETEROGENEITY IN RANKING CAPABILITIESJournal of Applied Econometrics, 2010
- Best–worst scaling: What it can do for health care research and how to do itJournal of Health Economics, 2007
- Some probabilistic models of best, worst, and best–worst choicesJournal of Mathematical Psychology, 2005
- Testing for Consistency in Contingent Ranking ExperimentsJournal of Environmental Economics and Management, 2002
- Use of the logit scaling approach to test for rank-order and fatigue effects in stated preference dataTransportation, 1994
- Analysis of the reliability of preference ranking dataJournal of Business Research, 1991
- Specifying and testing econometric models for rank-ordered dataJournal of Econometrics, 1987