Patrícia Martinková, Czech Academy of Sciences and Charles University

This talk introduces the research expanding upon the topics of the recently published book “Computational Aspects of Psychometric Methods: With R” (Martinková & Hladká, 2023). Focusing first on inter-rater reliability (IRR), we describe a flexible method for assessing heterogeneity in IRR with variance components models (Martinková et al., 2023) and discuss the relationship between the IRR and false positive rate (Bartoš & Martinková, 2024). Furthermore, we introduce innovative approaches for assessing item functioning and detecting heterogeneity in responses to multi-item measurements, proposing new iterative methods (Hladká et al., 2024a, 2024b) and Bayesian estimation algorithms (Pavlech & Martinková, 2024). We also discuss approaches incorporating more complex data, such as item wording (Štěpánek et al., 2023). Finally, we provide an overview of the software implementation, highlighting the ShinyItemAnalysis R package and interactive application (Martinková & Drabinová, 2018) and its new extendability option via add-on modules (Martinková et al., 2024).

Martinková, P., & Hladká, A. (2023). Computational Aspects of Psychometric Methods: With R. Chapman and Hall/CRC. https://doi.org/10.1201/9781003054313  

Martinková P., Bartoš F., & Brabec M. (2023). Assessing inter-rater reliability with heterogeneous variance components models: Flexible approach accounting for contextual variables. Journal of Educational and Behavioral Statistics, 48(3), 349–383. https://doi.org/10.3102/10769986221150517  

Bartoš, F., & Martinková P. (2024). Assessing quality of selection procedures: Lower bound of false positive rate as a function of inter-rater reliability. British Journal of Mathematical and Statistical Psychology. https://doi.org/10.1111/bmsp.12343  

Hladká A., Martinková P., & Magis D. (2024). Combining item purification and multiple comparison adjustment methods in detection of differential item functioning. Multivariate Behavioral Research, 59(1), 46-61. https://doi.org/10.1080/00273171.2023.2205393  

Štěpánek L., & Dlouhá J., & Martinková P. (2023). Item Difficulty Prediction Using Item Text Features: Comparison of Predictive Performance across Machine-Learning Algorithms. Mathematics, 11(9), 4104. https://doi.org/10.3390/math11194104  

Martinková P, & Drabinová A (2018). ShinyItemAnalysis for Teaching Psychometrics and to Enforce Routine Analysis of Educational Tests. The R Journal, 10(2), 503-515. https://doi.org/10.32614/RJ-2018-074

About the Speaker

Dr. Patrícia Martinková is an Associate Professor at Charles University and the chair of the Department of Statistical Modelling at the Czech Academy of Sciences in Prague, where she leads the Computational Psychometrics Group. She received her Ph.D. in Statistics from Charles University in 2007. She is a Fulbright alumna and an affiliate at the University of Washington. Her current research focuses on latent variable models, measurement error and reliability, differential item functioning, computational linguistics, and machine learning methods. She has published in journals such as the Journal of Educational and Behavioral Research, the Multivariate Behavioral Research, the Journal of the Royal Statistical Society – Series A, the R Journal, and the Journal of Educational Measurement. She recently authored a book “Computational Aspects of Psychometric Methods: With R”, published by Chapman and Hall/CRC in 2023.