medigraphic.com
Instituto Nacional de Salud Pública

2019, Number 2

<< Back Next >>

salud publica mex 2019; 61 (2)

Flaws in the design of the Examen Nacional para Aspirantes a Residencias Médicas produce inequity

Barajas-Ochoa A, Ramos-Remus C, Castillo-Ortiz JD, Yáñez J, Barajas-Ochoa Z, Sánchez-González JM, Hernández-Ávila M, Córdova-Villalobos JÁ, Bustamante-Montes LP

Language: English
References: 37
Page: 125-135
PDF size: 425.50 Kb.


ABSTRACT

Objective. To assess the assumption of ‘equity’ of Mexico’s resident-selection assessment tool, the Examen Nacional para Aspirantes a Residencias Médicas (ENARM). Materials and methods. Official ENARM-2016 and -2017 databases were analyzed. Differences in the absolute number of correct answers (multivariable linear regression) and the number of applicants reaching their specialty minimum score (SMS) per test day (odds ratio [OR]) were calculated. Applicants affected by test-day inequity were estimated. Results. There were 36 114 applicants in 2016, and 38 380 in 2017. In 2016, day-2 applicants had significantly higher scores and more reached the SMS than on days 1-3-4 (OR 1.55), and 5 (OR 3.8); 3 565 non-passing applicants were affected by inequity (equivalent to 44.64% of those selected). In 2017, day-1 and -2 applicants had significantly higher scores and more reached the SMS than on days 3-4 (OR 1.85), and 5 (OR 4.04); 3,155 non-passing applicants were affected by inequity (37.2% of those selected). Conclusion. Analysis of official ENARM databases does not support the official attribution of equity, suggesting the test should be redesigned.


REFERENCES

  1. Makary MA, Daniel M. Medical error-the third leading cause of death in the US. BMJ. 2016;353:i2139. https://doi.org/10.1136/bmj.i2139

  2. Roberts C, Khanna P, Rigby L, Bartle E, Llewellyn A, Gustavs J, et al. Utility of selection methods for specialist medical training: a BEME (best evidence medical education) systematic review: BEME guide no. 45. Med Teach. 2018;40(1):3-19. https://doi.org/10.1080/0142159X.2017.1367375

  3. mrcpuk.org [internet] London: Membership of the Royal Colleges of Physicians of the United Kingdom, 2018 [cited 2018 Jun 4] [cited 2018 Jun 4]. Available from: https://www.mrcpuk.org/

  4. Centre National de Gestion [internet]. Paris: CNG, 2018. Épreuves Classantes Nationales (ECN) [cited 2018 Jun 4]. Available from: https:// www.cng.sante.fr/concours-examens/epreuves-classantes-nationales-ecn

  5. Comisión Interinstitucional para la Formación de Recursos Humanos para la Salud [internet]. Mexico City: CIFRHS, 2018. Examen Nacional para Aspirantes a Residencias Médicas [cited 2018 Jun 4]. Available from: http://cifrhs.salud.gob.mx/

  6. Norcini J, Friedman Ben-David N. Concepts in assessment. In: Dent JA, Harden RM (eds). A practical guide for medical teachers. 4th ed. Churchill Livingstone, 2013:285-91.

  7. Comité de Posgrado y Educación Continua. XL ENARM: Metodología proceso de selección. Mexico City: CIFRHS, 2016 [cited 2018 Jun 4]. Available from: http://cifrhs.salud.gob.mx/site1/enarm/docs/2016/E40_met_proceso_ seleccion_2016.pdf

  8. Comité de Posgrado y Educación Continua. XLI ENARM: Metodología proceso de selección. Mexico City: CIFRHS, 2017 [cited 2018 Jun 4]. Available from: http://cifrhs.salud.gob.mx/site1/enarm/docs/2017/E41_met_proceso_ seleccion_2017.pdf

  9. Comisión Interinstitucional para la Formación de Recursos Humanos para la Salud. Convocatoria XL Examen Nacional para Aspirantes a Residencias Médicas. Mexico City: CIFRHS, 2016 [cited 2018 Jun 4]. Available from: http://cifrhs.salud.gob.mx/site1/enarm/docs/2016/E40_convo_2016.pdf

  10. Comisión Interinstitucional para la Formación de Recursos Humanos para la Salud. Convocatoria XLI Examen Nacional para Aspirantes a Residencias Médicas. Mexico City: CIFRHS, 2017 [cited 2018 Jun 4]. Available from: http://cifrhs.salud.gob.mx/site1/enarm/docs/2017/E41_convo_2017.pdf

  11. Barajas-Ochoa A, Ramos-Remus C. Equidad, validez y confiabililidad del Examen Nacional para Aspirantes a Residencias Medicas (ENARM): oportunidades para mejorar. Salud Publica Mex. 2017;36:501-2. https://doi. org/10.21149/8769

  12. American Educational Research Association, American Psychological Association, National Council on Measurement in Education. Standards for educational and psychological testing. Washington, DC: American Educational Research Association, 2014.

  13. Lane S, Raymond MR, Haladyna TM (eds). Handbook of test development. 2nd ed. New York: Routledge, 2016.

  14. Bennett RE, von Davier M. Advancing human assessment: The methodological, psychological and policy contributions of ETS [eBook]. Springer, 2017. https://doi.org/10.1007/978-3-319-58689-2

  15. Schuwirt LWT, van der Vleuten CPM. General overview of the theories used in assessment: AMEE Guide No. 57. Med Teach. 2011;33(10):783-97. https://doi.org/10.3109/0142159X.2011.611022

  16. Lane S, Raymond MR, Haladyna TM, Downing SM. Test development process. In: Lane S, Raymond MR, Haladyna TM (eds). Handbook of test development. 2nd ed. New York: Routledge, 2016:3-18.

  17. Dirección General de Calidad y Educación en Salud. Departamento de Transparencia de la Secretaría de Salud: Reply to request No. 0001200055817 on public information regarding the development of the ENARM. Mexico City: DGCES, 2017. Available from: https://drive.google. com/file/d/0B4Qe4F_sYmX0bHM0dHlpWFBObnM/view

  18. Comisión Interinstitucional para la Formación de Recursos Humanos para la Salud. ENARM: Características y evolución a su formato electrónico. Mexico City: CIFRHS, 2012 [cited 2018 Jun 4]. Available from: http:// cifrhs.salud.gob.mx/descargas/pdf/enarm_caracteristicas_evolucion.pdf

  19. Comisión Interinstitucional para la Formación de Recursos Humanos para la Salud. XL Examen Nacional para Aspirantes a Residencias Médicas: Puntajes máximos y mínimos por especialidad. Mexico City: CIFRHS, 2016 [cited 2018 Jun 4]. Available from: http://cifrhs.salud.gob.mx/site1/enarm/ docs/2016/E40_puntajes_max_min_2016.pdf

  20. Comisión Interinstitucional para la Formación de Recursos Humanos para la Salud. XLI Examen Nacional para Aspirantes a Residencias Médicas: Puntajes máximos y mínimos por especialidad. Mexico City: CIFRHS, 2017 [cited 2018 Jun 4]. Available from: http://cifrhs.salud.gob.mx/site1/enarm/ docs/2017/E41_puntajes_max_min_2017.pdf

  21. Ferguson CJ. An effect size primer: A guide for clinicians and researchers. Prof Psychol Res Pract. 2009:40:532-8. https://doi.org/10.1037/ a0015808

  22. Sullivan GM, Feinn R. Using effect size—or why the P value is not enough. J Grad Med Educ. 2012;4:279-82. https://doi.org/10.4300/JGMED- 12-00156.1

  23. Comisión Interinstitucional para la Formación de Recursos Humanos para la Salud. XL Examen Nacional para Aspirantes a Residencias Médicas: Plazas para médicos seleccionados. Mexico City: CIFRHS, 2016 [cited 2018 Jun 4]. Available from: http://cifrhs.salud.gob.mx/site1/enarm/ docs/2016/E40_plazas_mex_lugares_ext_2016.pdf

  24. Comisión Interinstitucional para la Formación de Recursos Humanos para la Salud. XL Examen Nacional para Aspirantes a Residencias Médicas: Folios de médicos seleccionados – Médicos con categoría mexicana. Mexico City: CIFRHS, 2016 [cited 2018 Jun 4]. Available from: http://cifrhs. salud.gob.mx/site1/enarm/docs/2016/E40_folio_sel_mex_2016.pdf

  25. Comisión Interinstitucional para la Formación de Recursos Humanos para la Salud. XL Examen Nacional para Aspirantes a Residencias Médicas: Folios de médicos seleccionados – Médicos con categoría extranjera. Mexico City: CIFRHS, 2016 [cited 2018 Jun 4]. Available from: http://cifrhs. salud.gob.mx/site1/enarm/docs/2016/E40_folio_sel_ext_2016.pdf

  26. Comisión Interinstitucional para la Formación de Recursos Humanos para la Salud. XLI Examen Nacional para Aspirantes a Residencias Médicas: Plazas para médicos seleccionados. Mexico City: CIFRHS, 2017 [cited 2018 Jun 4]. Available from: http://cifrhs.salud.gob.mx/site1/enarm/ docs/2017/E41_plazas_mex_lugares_ext_2017.pdf

  27. Comisión Interinstitucional para la Formación de Recursos Humanos para la Salud. XLI Examen Nacional para Aspirantes a Residencias Médicas: Folios de médicos seleccionados – Médicos con categoría mexicana. Mexico City: CIFRHS, 2017 [cited 2018 Jun 4]. Available from: http://cifrhs. salud.gob.mx/site1/enarm/docs/2017/E41_folio_sel_mex_2017.pdf

  28. Comisión Interinstitucional para la Formación de Recursos Humanos para la Salud. XLI Examen Nacional para Aspirantes a Residencias Médicas: Folios de médicos seleccionados – Médicos con categoría extranjera. Mexico City: CIFRHS, 2017 [cited 2018 Jun 4]. Available from: http://cifrhs. salud.gob.mx/site1/enarm/docs/2017/E41_folio_sel_ext_2017.pdf

  29. Dirección General de Calidad y Educación en Salud. Departamento de Transparencia de la Secretaría de Salud: Reply to request No. 0001200357817 on the petition of the official databases of the 2016 and 2017 ENARM. Mexico City: DGCES, 2017. Available from: https://drive. google.com/open?id=1SkzaK20MXXShKTVzK42Alc4jbLWfB8kg

  30. Moses T. A review of developments and applications in item analysis. In: Bennett RE, von Davier M (eds). Advancing human assessment: The methodological, psychological and policy contributions of ETS [eBook]. Springer Open, 2017:19-46. https://doi.org/10.1007/978-3-319-58689-2

  31. Haladyna TM. Item analysis for selected-response test items. In: Lane S, Raymond MR, Haladyna TM (eds). Handbook of test development. 2nd ed. New York: Routledge, 2016:392-409.

  32. Zieky MJ. Developing fair tests. In: Lane S, Raymond MR, Haladyna TM (eds). Handbook of test development. 2nd ed. New York: Routledge, 2016:81-99.

  33. Tighe J, McManus IC, Dewhurst NG, Chis L, Mucklow J. The standard error of measurement is a more appropriate measure of quality for postgraduate medical assessments that is reliability: an analysis of MRCP(UK) examinations. BMC Med Educ. 2010;10:40. https://doi.org/10.1186/1472- 6920-10-40

  34. Raykov T, Marcoulides GA. Introduction to psychometric theory. Classical test theory. New York, NY: Routledge, 2011:115-36.

  35. Green LW, Ottoson JM, Garcia C, Hiatt RA. Diffusion theory and knowledge dissemination, utilization, and integration in public health. Annu Rev Public Health. 2009;30:151-74. https://doi.org/10.1146/annurev. publhealth.031308.100049

  36. Comisión Interinstitucional para la Formación de Recursos Humanos para la Salud. Convocatoria XLII Examen Nacional para Aspirantes a Residencias Médicas. Mexico City: CIFRHS, 2018 [cited 2018 Jun 4]. Available from: http://cifrhs.salud.gob.mx/site1/enarm/docs/2018/E42_convo_2018.pdf

  37. Timothy JM. Web-based item development and banking. In: Lane S, Raymond MR, Haladyna TM (eds). Handbook of test development. 2nd ed. New York: Routledge, 2016: 241-56.




2020     |     www.medigraphic.com