We introduce DOSE—Dynamically Optimized Sequential Experimentation—to elicit preference parameters. DOSE starts with a model of preferences and a prior over the parameters of that model, then dynamically chooses a customized question sequence for each participant according to an experimenter-selected information criterion. After each question, the prior is updated, and the posterior is used to select the next, informationally-optimal, question. Simulations show that DOSE produces parameter estimates that are approximately twice as accurate as those from established elicitation methods. DOSE estimates of individual-level risk and time preferences are also more accurate, more stable over time, and faster to administer in a large representative, incentivized survey of the U.S. population (N = 2,000). By reducing measurement error, DOSE identifies a stronger relationship between risk aversion and cognitive ability than other elicitation techniques. DOSE thus provides a flexible procedure that facilitates the collection of incentivized preference measures in the field.
Authors
Jonathan Chapman, Erik Snowberg, Stephanie W. Wang, Colin Camerer
Related Organizations
- Acknowledgements & Disclosure
- This paper envelops an earlier working paper introducing the DOSE procedure (Wang et al., 2010). We thank Summer Clay, Alison Harris, and Alec Smith for generously sharing their data. Thanks to Roland Benabou, Doug Bernheim, John Beshears, Matthew Chao, Gary Charness, Mark Dean, Andreas Grunewald, Ori Heffetz, Holger Herz, Kate Johnson, Ian Krajbich, Andreas Krause, Sebastian Olschewski, Pietro Ortoleva, Deb Ray, Antonio Rangel, Stefan Trautmann, Hans-Martin von Gaudecker, Peter Wakker, Nathaniel Wilcox, and the participants of seminars and conferences for their useful comments and suggestions. Judah Okwuobi and Michelle Filiba provided excellent research assistance. Camerer and Snowberg gratefully acknowledge the financial support of NSF Grant SMA-1329195. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research.
- DOI
- https://doi.org/10.3386/w33013
- Pages
- 114
- Published in
- United States of America
Table of Contents
- Introduction 3
- Related Literature 7
- The DOSE Procedure 9
- The Participants' Point of View 9
- The Experimenter's Point of View 10
- Our DOSE Procedure for Measuring Risk and Time Preferences 13
- Performance of DOSE versus Current Methods 18
- Simulating DOSE with Laboratory Data 18
- Parameter Recovery Study 20
- DOSE in a Representative Online Survey 24
- Data 24
- Individual-Level Measures of Risk and Time Preferences 28
- Within-person Stability of Economic Preferences 31
- Survey Fatigue and Question Complexity 32
- Economic Preferences and Cognitive Ability 34
- Correlates of Economic Preferences 35
- Choice Consistency and Estimate Accuracy 37
- Cognitive Ability and Focal Value Response 40
- Discussion 45
- Appendix 49
- Online Appendix—Not Intended for Publication 55
- Risk and Time Preferences in Prior Studies 56
- DOSE Estimates in Different Samples 56
- Correlates of Economic Preferences 58
- Temporal Stability of Economic Preferences 58
- DOSE Procedure and Survey Implementation 59
- The Potential for Gaming DOSE and Incentive Compatibility 59
- DOSE in a Representative Survey 62
- DOSE Modules 62
- Additional Measures 63
- Simulations using Laboratory Choices 65
- Estimates of Risk and Loss Aversion 65
- Estimates of Choice Consistency 67
- Maximum Likelihood Estimation 68
- Parameter Recovery Exercise 71
- Parameter Recovery Procedure 72
- Validating the Simulation with Survey Data 77
- Misspecification of the Utility Function 78
- Robustness Checks 80
- Additional Tests of Fatigue and Inattention 81
- DOSE Choice Data and Model (Mis-)Specification 82
- Robustness to Possible Misspecification 85
- Robustness of Correlations with Economic Preferences 86
- Choice Consistency and Response Time 101
- Screenshots 103
