Predictive validity of academy mock tests against MIR exam: a retrospective multicohort observational concordance study
Abstract
Introduction: MIR examination is a requirement to access Spain’s public specialist training system. This study evaluates, across five years, the concordance between mock exams administered by a MIR preparatory academy and the official MIR results. Methods: Candidate records were linked; on the matched intersection we computed Pearson’s R and Spearman’s ρ (percentiles, net scores), bias (simulation−official), and RMSE. Annual key performance indicators (KPIs) were pair-weighted and analyses stratified by percentile cohorts (≤27, 28–73, >73); heatmap summarize percentile correlations. Results: Across years, weighted percentile correlations typically fell in the 0.71–0.76 range, with small average bias and moderate RMSE. Cohort analyses indicated larger errors at extreme percentiles in some years. Net‑score bias weighted by pairs was near zero overall, with year‑to‑year variability. Conclusions: Academy mock tests provide a broadly consistent ranking signal relative to official MIR outcomes. Residual bias and dispersion suggest potential gains from calibration—particularly at cohort extremes—and motivate routine monitoring of concordance KPIs in future cohorts.
Downloads
Metrics
-
Abstract37
-
pdf68
References
Puri N, McCarthy M, Miller B. Validity and reliability of pre‑matriculation and institutional assessments in predicting USMLE STEP 1 success: lessons from a traditional 2 × 2 curricular model. Front Med (Lausanne). 2021, 8, 798876. https://doi.org/10.3389/fmed.2021.798876
Tiffin PA, Paton LW. A predictive model for USMLE Step 1 scores. Cureus. 2016, 8(9), e769. https://doi.org/10.7759/cureus.769
Aharonian K, Sanders M, Schlesinger T, Winter V, Simanton E. Predictive validity of preclerkship performance metrics on USMLE Step 2 CK outcomes in the Step 1 pass/fail era. Adv Med Educ Pract. 2025, 16, 43–50. https://doi.org/10.2147/AMEP.S505612
Bird JB, Olvet DM, Willey JM, Brenner JM. A generalizable approach to predicting performance on USMLE Step 2 CK. Adv Med Educ Pract. 2022, 13, 939–944. https://doi.org/10.2147/AMEP.S373300
Jones A, Benns M, Farmer R. Using resident performance on Step 2 to predict surgical residency success. Surgery. 2025, 179, 108801. https://doi.org/10.1016/j.surg.2024.07.058
Baladrón Romero J, López Criado MS, Escudero Carretero MJ, Gil Navarro MV, et al. Resultados obtenidos en la prueba MIR según baremo académico. Convocatorias de 2019 y 2020. Investig Educ Méd. 2022, 11(43), 51–62. https://doi.org/10.22201/fm.20075057e.2022.43.22420
Baladrón‑Romero J, López Criado MS, Escudero Carretero MJ, Gil Navarro MV, et al. Resultados obtenidos en la prueba MIR de 2021, según nacionalidad y baremo académico. FEM. 2022, 25(5), 205–213. https://doi.org/10.33588/fem.255.1230
Copyright (c) 2025 Servicio de Publicaciones de la Universidad de Murcia
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
The works published in this magazine are subject to the following terms:
1. The Publications Service of the University of Murcia (the publisher) preserves the economic rights (copyright) of the published works and favors and allows them to be reused under the use license indicated in point 2.
2. The works are published under a Creative Commons Attribution-NonCommercial-NoDerivative 4.0 license.
3. Self-archiving conditions. Authors are allowed and encouraged to disseminate electronically the pre-print versions (version before being evaluated and sent to the journal) and / or post-print (version evaluated and accepted for publication) of their works before publication , since it favors its circulation and earlier diffusion and with it a possible increase in its citation and reach among the academic community.
