Dan Bolt, University of Wisconsin – Madison

2021 Presidential Address

Date & Time: Friday, July 23 at 12:00pm (Noon) EST

Despite its frequent consideration in test development, item complexity receives little attention in the psychometric modeling of item response data. In this address, I consider how variability in item complexity can be expected to emerge in the form of item characteristic curve (ICC) asymmetry, and how such effects may significantly influence applications of item response theory, especially those that assume interval level properties of the latent proficiency metric. One application is the score gain deceleration phenomenon often observed in vertical scaling contexts involving math or secondary language acquisition. It is demonstrated that the application of symmetric IRT models in the presence of complexity-induced positive ICC asymmetry is a likely cause. A second application concerns the positive correlation between DIF and difficulty commonly seen in verbal proficiency (and other subject area) tests where problem-solving complexity is minimal and proficiency-related guessing effects are likely more pronounced. It is shown how systematic negative ICC asymmetry creates artificial positive correlations when applying either nonparametric (e.g., standardization) or parametric (IRT model based) methods when no DIF is actually present. Unfortunately, the presence of systematic forms of ICC asymmetry is easily missed due to the considerable flexibility afforded by latent trait metrics in IRT. Speculation is provided regarding other applications for which attending to ICC asymmetry may prove useful.

About the Speaker

Daniel Bolt is Nancy C. Hoefs-Bascom Professor of Educational Psychology at the University of Wisconsin-Madison and specializes in quantitative methods. His methodological research focuses on the theory and application of psychometric methods in the educational, social and health sciences, including topics such as the application of latent variable models for purposes of test validation, assessment of individual differences (such as response styles), and the modeling of student growth. His research has been published in journals such as the British Journal of Mathematical and Statistical Psychology, Journal of Educational and Behavioral Statistics, Psychological Methods, and Psychometrika, among other outlets. In addition to his methodological research, Dr. Bolt collaborates on various research projects in the social and health sciences as a biostatistician at the Waisman Center. Dr. Bolt teaches courses in test theory, factor analysis, and hierarchical linear modeling. He is currently President of the Psychometric Society (2020-2021) and a past recipient of the Kellett Mid-Career Award, the Vilas Associates Research Award and a Chancellor’s Distinguished Teaching Award from the University of Wisconsin, Madison.