Peter F. Halpin, University of North Carolina at Chapel Hill

The overall argument of this talk is that IRT-based scaling and differential item functioning (DIF) are two sides of the same problem. In particular, DIF with respect to a grouping variable is formally similar to IRT-based scaling with the common items non-equivalent groups (CINEG) design. Items with DIF translate into outliers in the CINEG design, and this outlier detection problem is remarkably amenable to existing methods from robust statistics. The utility of this overall approach is illustrated in two contexts. First, I show how robust scaling can be used to construct a DIF-detection procedure that (a) does not require pre-specification of anchor items, (b) comes with theoretical guarantees about its performance when fewer than 1/2 of items exhibit DIF, and ( c) can be conveniently applied as a post-estimation procedure following separate calibrations (configured invariance) of a wide class of unidimensional latent variable models. Second, I show how robust scaling yields a Hausman-like specification test of whether DIF affects group comparisons on the latent trait (i.e. differential test functioning or DTF). The test does not require identifying which, if any, items may exhibit DIF, thereby obviating the need for item-by-item analyses before evaluating DTF. I illustrate the usefulness of the specification test for addressing concerns about the quality of outcome measures used in education research, focusing in particular on the interpretation of DTF with respect to randomly assigned treatment conditions. Finally, I discuss ongoing work to extend robust scaling to multiple groups, longitudinal settings, and multidimensional models.

About the Speaker

Dr. Halpin specializes in psychometrics and educational measurement. His research seeks to develop and apply rigorous statistical methodology to address pressing issues in educational research, practice, and policy. His methodological work focuses on IRT as well as methods for the analysis of time series data (process data, traces) collected in computer-supported collaborative learning. His applied work has addressed the assessment of early childhood development in international settings. He earned his Ph.D. from Simon Fraser University in Vancouver, Canada. Prior to joining UNC, he was a postdoctoral researcher at the University of Amsterdam, and an assistant/associate professor at New York University. His research has been supported by the US Institute of Education Sciences, the Spencer Foundation, and the National Academy of Education.