We develop an empirical Bayes ranking procedure that assigns ordinal grades to noisy measurements, balancing the information content of the assigned grades against the expected frequency of ranking errors. Applying the method to a massive correspondence experiment, we grade the race and gender contact gaps of 97 U.S. employers, the identities of which we disclose for the first time. The grades are presented alongside measures of uncertainty about each firm’s contact gap in an accessible report card that is easily adaptable to other settings where ranks and levels are of simultaneous interest.
Authors
- Acknowledgements & Disclosure
- We thank Ben Scuderi for helpful feedback on an early draft of this paper and Hadar Avivi and Luca Adorni for outstanding research assistance. Seminar participants at Brown University, the 2022 California Econometrics Conference, Columbia University, CIREQ 2022 Montreal, Harvard University, Microsoft Research, Monash University, Peking University, Royal Holloway, UC Santa Barbara, UC Berkeley, The University of Virginia, the Cowles Econometrics Conference on Discrimination and Algorithmic Fairness, and The University of Chicago Interactions Conference provided useful comments. Routines for implementing the ranking procedures developed in this paper are available online at https://github.com/ekrose/drrank. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research. Christopher R. Walters Christopher Walters holds concurrent appointments as an Associate Professor of Economics at UC Berkeley and as an Amazon Scholar. This paper describes work performed at UC Berkeley and is not associated with Amazon.
- DOI
- https://doi.org/10.3386/w32313
- Published in
- United States of America
