medigraphic.com
ISSN 2007-5057 (Print)
Investigación en Educación Médica

2022, Number 43

<< Back Next >>

Inv Ed Med 2022; 11 (43)

Adjustment from a face-to-face OSCE to an on-line modality in a high stakes examination

Trejo MJA, Peña BJ, Soto ACA, Alpuche HA, Ortiz MA, Cerritosa A

Language: Spanish
References: 30
Page: 16-35
PDF size: 699.41 Kb.


ABSTRACT

Introduction: In view of the need to remotely assess clinical competencies during the COVID-19 pandemic, an online objective structured clinical examination (OSCE) was developed and applied in 2021 to applicants at the end of their medical degree at the National Autonomous University of Mexico, Faculty of Medicine and of incorporated institutions.
Objective: To assess the OSCE online 2021 psychometric quality and compare it to a face-to-face OSCE modality in 2020 in a high stakes examination.
Method: The OSCE was adjusted from a face-to-face to an online format. The clinical cases and the rubrics used for the online modality were previously designed according to the face-to-face OSCE. In each of the clinical cases, the ability of the physical examination was adjusted and the medical note was replaced to a verbal clinical summary. For this new modality we used the zoom platform; therefore, training courses were given to the assessors and patients to standardize their knowledge and performance on this platform. For the applicants we designed audiovisual resources explaining the process of the examination. We analyzed the applied procedure based on the design, implementation, and grade; also, we used the psychometric analysis to provide the validity evidence related to the internal structure of the test.
Results: Both formats of the OSCE were composed of six-20 minutes stations. We examined the 99 items that constitute the total of the six evaluated competencies with a ‹0.001 discrimination between the high and the low scores. We identified six factors that explain the 60.7% of the variance in the face-to-face OSCE and six factors with five more indicators in the online format that explained the 56.6% of the variance. We calculated a Chronbach Alfa of 0.96 for the face-to-face format and a 0.93 for the online OSCE.
Conclusions: OSCE formats showed psychometric quality evidence based on the content validity, internal structure of the test, and the response process, these characteristics strengthened both tests. This research showed that the online format is a feasible alternative for the evaluation of the practical phase in the professional examination.


REFERENCES

  1. Facultad de Medicina UNAM. Plan de Estudios 2010 y ProgramasAcadémicos de la Licenciatura de Médico Cirujano.En: Facultad de Medicina UNAM, editor. Ciudad Universitaria 2009. p. 1-150. [Consultado: 16 May 2021]. Disponibleen: http://www.facmed.unam.mx/plan/PEFMUNAM.pdf

  2. Harden RM, Stevenson M, Downie WW, Wilson GM. Assessmentof clinical competence using objective structuredexamination. Br Med J. 1975;1(5955):447-51. doi: 10.1136/bmj.1.5955.447

  3. Trejo Mejía J, Larios Mendoza H, Velasco Jiménez M, HernándezLópez A, Martínez Viniegra N, Cortés Gutiérrez M.Evaluación de la Competencia Clínica de los alumnos aliniciar el Internado Médico de Pregrado. Rev Fac Med.1998;41(3):108-13. Disponible en: https://pesquisa.bvsalud.org/portal/resource/pt/lil-234020 y en: https://www.imbiomed.com.mx/articulo.php?id=16134

  4. Larios Mendoza H, Trejo Mejía J, Méndez I. Examen Profesionalcon el Examen Clínico Objetivo Estructurado. Rev EdSup. 2005;34(4):7-17. [Consultado: 10 Jun 2021]. Disponibleen: https://www.redalyc.org/pdf/604/60413601.pdf

  5. Furman GE. The role of standardized patient and trainertraining in quality assurance for a high-stakes clinical skillsexamination. Kaohsiung J Med Sci. 2008;24(12):651-5. doi:10.1016/s1607-551x(09)70031-2

  6. Furman G, Montanano A. ECFMG 2014 Annual Report.Case Development Workshop conducted at School of Medicineof the National Autonomous University of Mexico;Monterrey, Mexico: ECFMG; 2014. p. 1-75. [Consultado: 20May 21]. Disponible en: https://www.yumpu.com/en/document/read/38343332/ecfmg-2014-annual-report

  7. Facultad de Medicina UNAM. Internado Médico. AsignaturaClínica-Quinto Año-Décimo y Undécimo Semestre.En: Facultad de Medicina UNAM, editor. Ciudad Universitaria;2018. p. 1-101. [Consultado: 10 jul 21]. Disponible en:http://paginas.facmed.unam.mx/deptos/ss/wp-content/uploads/2018/10/1011.pdf

  8. Martínez-González A, Sánchez-Mendiola M, Olivares-OlivaresS, Grimaldo-Avilés J, Trejo-Mejía J, Martínez-Franco A,y cols. Colaboración de tres escuelas de medicina de Méxicoen un examen clínico objetivo estructurado (ECOE). Inv EdMed. 2020;10(38). doi: 10.22201/fm.20075057e.2020.36.20258

  9. Amin H, Shehata M, Ahmed S. Step-by-step Guide to CreateCompetency-Based Assignments as an Alternative for TraditionalSummative Assessment. MedEdPublish. 2020:1-15.doi: 10.15694/mep.2020.000120.1

  10. Boursicot K, Kemp S, Ong T, Wijaya L, Goh S, Freeman K, y cols.Conducting a high-stakes OSCE in a COVID-19 environment.MedEdPublish. 2020:1-8. doi: 10.15694/mep.2020.000054.1

  11. Khan RA, Jawaid M. Technology Enhanced Assessment(TEA) in COVID 19 Pandemic. Pak J Med Sci. 2020;36(COVID19-S4):S108-S10. doi: 10.12669/pjms.36.covid19-s4.2795

  12. Ahmed S, Shehata M, Hassanien M. Emerging Faculty Needsfor Enhancing Student Engagement on a Virtual Platform.MedEdPublish. 2020:1-5. doi: 10.15694/mep.2020.000075.1

  13. Shehata MH, Kumar AP, Arekat MR, Alsenbesy M, MohammedAl Ansari A, Atwa H, y cols. A toolbox for conductingan online OSCE. Clin Teach. 2021;18(3):236-42. doi: 10.1111/tct.13285

  14. Ward JP, Gordon J, Field MJ, Lehmann HP. Communicationand information technology in medical education. Lancet.2001;357(9258):792-6. doi: 10.1016/s0140-6736(00)04173-8

  15. Standards for educational and psychological testing / AmericanEducational Research Association, American PsychologicalAssociation, National Council on Measurement in Education[Internet]. Washington, DC: American EducationalResearch Association; 2014 [Citado: 2022 Ene 15]. Disponibleen: https://search.ebscohost.com/login.aspx?direct=true&db=edsnuk&AN=edsnuk.vtls004537073&lang=es&site=edslive

  16. Nuván Hurtado IL, Rivera Porras DA, Carillo Sierra SM,Forgiony Santos JO, Bonilla Cruz NJ, Rozo Sánchez AC.Diferencias en la calidad psicométrica de tests construidosmediante la estrategia pedagógica audiovisual y las estrategiaspedagógicas tradicionales. Rev Esp. 2018; 39(25):7-21.ISSN 0798 1015. Disponible en: https://www.revistaespacios.com/a18v39n25/a18v39n25p07.pdf

  17. García Pérez EM, Magaz Lago A. ¿Cómo valorar test psiscométricos?[Internet] Bizkaya, España: COHS. Consultoresen Ciencias Humanas, S.L.; 2009. [Citado: 2022 Ene 15].Disponible en: http://www.gac.com.es/editorial/libros/comovalorartestpsicometricos.pdf

  18. Craig C, Kasana N, Modi A. Virtual OSCE delivery: The wayof the future? Med Educ. 2020;54(12):1185-6. doi: 10.1111/MEDU.14286

  19. Kakadia R, Chen E, Ohyama H. Implementing an onlineOSCE during the COVID-19 pandemic. J Dent Educ. 2020.doi: 10.1002/jdd.12323

  20. Hytönen H, Näpänkangas R, Karaharju-Suvanto T, EväsojaT, Kallio A, Kokkari A, y cols. Modification of national OSCEdue to COVID-19 - Implementation and students’ feedback.Eur J Dent Educ. 2020. doi: 10.1111/eje.12646

  21. Donn J, Scott JA, Binnie V, Bell A. A pilot of a Virtual Objec-tive Structured Clinical Examination in dental education. Aresponse to COVID-19. Eur J Dent Educ. 2020. doi: 10.1111/eje.12624

  22. Khalaf K, El-Kishawi M, Moufti MA, Al Kawas S. Introducinga comprehensive high-stake online exam to final-year dentalstudents during the COVID-19 pandemic and evaluation ofits effectiveness. Med Educ Online. 2020;25(1):1826861. doi:10.1080/10872981.2020.1826861

  23. Reglamento General de Exámenes, UNAM (1969). [Consultado:25 Jul 21]. Diponible en: http://www.abogadogeneral.unam.mx/sites/default/files/2018-05/229.pdf

  24. González Contró, M. Oficio AGEN/DGEL/107/20OJ/62/20.Ciudad Universitaria 2020. [Consultado: 25 Jul 21]. Disponibleen: http://escolares.quimica.unam.mx/exaprof/Oficio_AGEN-DGEL-107-20.pdf

  25. Alpuche HA, Trejo-MJ, Peña BJ, Ortiz MA. Diseño y elaboraciónde rúbricas para evaluar competencias de estudiantesde medicina en un Examen Clínico Objetivo. Psicología Educativa.2020-2021;8-9(1):58-64. eISSN 2448-6957

  26. Downing, SM. Validity: On the meaningful interpretationof assessment data. Medical Education. 2003;37(9):830-837.https://doi.org/10.1046/j.1365 2923.2003.01594.x

  27. Lawrence K, Hanley K, Adams J, Sartori DJ, Greene R, ZabarS. Building Telemedicine Capacity for Trainees Duringthe Novel Coronavirus Outbreak: a Case Study and LessonsLearned. J Gen Intern Med. 2020;35(9):2675-9. doi: 10.1007/s11606-020-05979-9

  28. Robinson P, Morton L, Haran H, Manton R. Mock OSCEsimprove medical students’ confidence and reduce anxietyrelated to summative examinations. Education in MedicineJournal. 2017;9:41-5. doi: 10.21315/eimj2017.9.2.4

  29. Hess BJ, Kvern B. A call to understand the psychometricimplications of virtual OSCE delivery. Med Teach. 2020:1-2.doi: 10.1080/0142159X.2020.1849591

  30. Young I, Montgomery K, Kearns P, Hayward S, MellanbyE. The benefits of a peer-assisted mock OSCE. Clin Teach.2014;11(3):214-8. doi: 10.1111/tct.12112




2020     |     www.medigraphic.com