Elena Erosheva, University of Washington

Date & Time: Tuesday, July 20 at 12:00pm (Noon) EST

Peer review is an integral part of the scientific enterprise. For example, grant peer review allocates billions of dollars of research funding, makes decisions that can make or break a scientific career, and consumes substantial resources to sustain on the part of applicants, reviewers, and grant agencies. With so much at stake, it is natural to ask about the quality of these peer review assessments themselves. Can peer review quality be summarized in one measure such as inter-rater reliability? Does a rubric with scored criteria guarantee that the review procedure is fair for all? How far are members of a review panel from consensus, and how much peer pressure do they experience from other panelists? This talk will describe several specific examples to illustrate these open questions in studying characteristics and limits of peer review. We consider those peer review settings where reviewers assess the quality of proposed research by assigning numeric scores to applications. Understanding characteristics and limits of peer review assessments can be useful for scientific communities and funding agencies in their evaluations about whether and how peer review should be used to make funding decisions.

About the Speaker

Elena A. Erosheva is Professor of Statistics and Social Work and Associate Director of the Center for Statistics and the Social Sciences at the University of Washington. Her research focuses on the development and application of statistical methods and models for complex and heterogeneous data in the social, behavioral, medical and health sciences. She is a Fellow of the American Statistical Association, a recipient of the 2013 Mitchell Prize from the International Society of Bayesian Analysis for an outstanding paper in Bayesian analysis that has solved an important applied problem, and a first prize winner of 2014 America Competes Act Challenge competition to maximize fairness in NIH peer review in the category Most Creative Idea for Detection of Bias in Peer Review. She is currently serving on the National Academies Committee on Increasing Diversity and Inclusion in the Leadership of NASA Completed Space Missions. Her current and past editorial board service includes serving as an ArXiv moderator for Statistics, Annals of Applied Statistics, Journal of the American Statistical Association, Journal of Educational and Behavioral Statistics, and Psychometrika. She earned a Ph.D. in Statistics from Carnegie Mellon University.