Amanda Luby, Swarthmore College

Forensic science often involves the evaluation of crime-scene evidence to determine whether it matches a known-source sample, such as whether a fingerprint or DNA was left by a suspect or if a bullet was fired from a specific firearm. Even as forensic measurement and analysis tools become increasingly automated and objective, final source decisions are often left to individual examiners’ interpretation of the evidence. Furthermore, forensic analyses often consist of a series of steps. While some of these steps may be straightforward and relatively objective, substantial variation may exist in more subjective decisions. The current approach to characterizing uncertainty in forensic decision-making has largely centered around conducting error rate studies (in which examiners evaluate a set of items consisting of known-source comparisons) and calculating error rates aggregated across examiners and identification tasks. We propose a new approach using Item Response Theory (IRT) and IRT-like models to account for differences in examiner behavior and for varying difficulty among identification tasks. There are, however, substantial differences between forensic decision-making and traditional IRT applications such as educational testing. For example, the structure of the response process must be considered, “answer keys” for comparison tasks do not exist, and information about participants and items is not available due to privacy constraints. In this talk, we provide an overview of forensic decision-making, outline challenges in applying IRT in practice, and survey some recent advances (Luby, 2019) in the application of Bayesian psychometric models to fingerprint examiner behavior.