medigraphic.com
ISSN 2007-5057 (Print)
Investigación en Educación Médica

2024, Number 52

<< Back Next >>

Inv Ed Med 2024; 13 (52)

Correlation between a formative WebOSCE and academic performance in medical students

Barboza SSA, García RRC, Trejo MJA, Martínez-González, Adrián

Language: Spanish
References: 36
Page: 9-18
PDF size: 856.65 Kb.


ABSTRACT

Introduction: The objective structured clinical examination (OSCE) is the gold standard for the assessment of clinical competence. With the SARS-CoV-2 virus-related pandemic, there was a rising need of an instrument for the assessment of this competence in distance education.
Objective: We aimed to correlate the clinical competence assessed with a formative WebOSCE, and the academic performance of students who finish the second year of their medical degree.
Method: An observational, correlational study was carried out. The test was applied to the students that completed the second year of the medical degree. The sampling was by census, and an 8-station WebOSCE was developed with a global evaluation scale that contemplates four levels of performance. Psychometric analyzes were performed and the level of clinical competence of the WebOSCE was correlated with academic performance using Spearman’s rho. Feedback was done using an automated post-test program.
Results: 207 students met the study inclusion criteria for analysis. The average WebOSCE score was 74.5, with the following results by attribute: interrogation (70.7), physical examination (74.4), interpretation of lab and imagenology studies (54.2), diagnosis (67.9), treatment (83.0), communication and professionalism (84.5), teamwork (81.2), and clinical integration (67.6). Cronbach’s alpha obtained was 0.89. Spearman’s rho for the correlation of academic performance and the WebOSCE was 0.25 (p‹0.001).
Conclusion: The WebOSCE is an instrument with sufficient validity evidence to assess clinical competence dimensions at distance and has a positive correlation with the academic performance of students.


REFERENCES

  1. American Educational Research Association, AmericanPsychological Association, & National Council on Measurementin Education. Estándares para pruebas educativas ypsicológicas (M. Lieve, Trans.). Washington DC: AmericanEducational Research Association; 2018: 258 p.

  2. Carrillo-Avalos BA, Sánchez-Mendiola M, Leenen I. El conceptomoderno de validez y su uso en educación médica.Inv Ed Med. 2020;9(33):98-106. https://doi.org/10.22201/facmed.20075057e.2020.33.19216

  3. Gatica-Lara F, Méndez-Ramírez I, Sánchez-Mendiola M,Martínez-González A. Variables asociadas al éxito académicoen estudiantes de la Licenciatura en Medicina de laUNAM. Rev Fac Med UNAM. 2010;53(5):9-18.

  4. Noriega T, Orosa J, Puerta M, Goncalves J, Díaz M, Pérez-Ojeda J. La competencia clínica como eje Integrador de losestudios de Pre y Post-Grado en las Ciencias de la Salud.Rev Fac Med Caracas. 2003;26(1):17-21.

  5. Fornells-Vallés JM. El ABC del Mini-CEX. Educ. Med.2009;12(2):83-89.

  6. Swing SR. Assessing the ACGME General Competencies:General Considerations and Assessment Methods.AEM. 2002;1;9(11):1278-1288. https://doi.org/10.1197/aemj.9.11.1278

  7. Erfani-Khanghahi M, Ebadi-Fard Azar F. Direct observationof procedural skills (DOPS) evaluation method:Systematic review of evidence. Med J Islam Repub Iran.2018;32(3):45. https://doi.org/10.14196/mjiri.32.45

  8. Díaz-Guio DA, Cimadevilla-Calvo B. Educación basadaen simulación: debriefing, sus fundamentos, bondades ydificultades. Rev Latinoam Simul Clín. 2019;1(2):95-103.https://dx.doi.org/10.35366/RSC192F

  9. Wright C, Richards SH, Hill JJ, Roberts MJ, Norman GR,Greco M, et al. Multisource feedback in evaluating the performanceof doctors: the example of the UK General MedicalCouncil patient and colleague questionnaires. Acad Med.2012;87(12):1668-1678. doi: 10.1097/ACM.0b013e3182724cc0

  10. Harden RM, Lilley P, Patricio M. The Definitive Guide tothe OSCE: The Objective Structured Clinical Examinationas a performance assessment. Elsevier Health Sciences;2015: p 224.

  11. Martínez-González A, Sánchez-Mendiola M, Méndez-RamírezI, Trejo-Mejía JA. Grado de competencia clínica desiete generaciones de estudiantes al término del internadomédico de pregrado. Gac Med Mex. 2016;152(5):679-687.

  12. Trejo-Mejía JA, Martínez-González A, Méndez-RamírezI, Ruiz-Pérez LC, Sánchez-Mendiola M, Morales-López S.Evaluación de la competencia clínica con el examen clínicoobjetivo estructurado en el internado médico de la UniversidadNacional Autónoma de México. Gac Med Mex.2014;150(1):8-17.

  13. Martínez-Gonzalez A, Sánchez-Mendiola M, Furman GE,Olivares-Olivares S, Trejo-Mejía JA, Alpuche-HernándezA, et al. Colaboración de tres escuelas de medicina de Méxicoen un examen clínico objetivo estructurado (ECOE).Inv Ed Med. 2020;15(9):58-69. https://doi.org/10.22201/fm.20075057e.2020.36.20258

  14. Jefferies A, Simmons B, Tabak D, Mcilroy JH, Lee K-S,Roukema H, et al. Using an objective structured clinicalexamination (OSCE) to assess multiple physician competenciesin postgraduate training. Med Teach. 2007;1;29(2-3):183-191. https://doi.org/10.1080/01421590701302290

  15. Lacassie HJ, Corvetto MA, Delfino AE. Implementacióndel Primer Examen Clínico Objetivo Estructurado (ECOE)para la Certificación de la Especialidad en Anestesiología enChile. Rev Med Chile.2020;148(12):1819-1824. http://dx.doi.org/10.4067/S0034-98872020001201819

  16. Ferrel MN, Ryan JJ. The impact of COVID-19 on medicaleducation. Cureus. 2020;12(3):e7492.

  17. Torda A. How COVID‐19 has pushed us into a medicaleducation revolution. Intern Med J. 2020;50(9):1150-1153.https://doi.org/10.1111/imj.14882

  18. Rivero-López CA, Vega-Rodríguez MF, Yap-Campos K,Jiménez-Galván I, Ponce-Rosas ER, Martínez-GonzálezA. La evaluación de la competencia clínica a través de unWeb-ECOE: una experiencia de aplicación. InvestigaciónEduc Médica. 2021;10(38):68-75. https://doi.org/10.22201/fm.20075057e.2021.38.21341

  19. Major S, Sawan L, Vognsen J, Jabre M. COVID-19 pandemicprompts the development of a Web-OSCE usingZoom teleconferencing to resume medical students’ clinicalskills training at Weill Cornell Medicine-Qatar. BMJ SimulTechnol Enhanc Learn. 2020;6(6):376-377. doi: 10.1136/bmjstel-

  20. 2020-00062920. Shorbagi S, Sulaiman N, Hasswan A, Kaouas M, Al-DijaniMM, El-hussein RA, et al. Assessing the utility and efficacyof e-OSCE among undergraduate medical students duringthe COVID-19 pandemic. BMC Med Educ. 2022;22(1):156.https://doi.org/10.1186/s12909-022-03218-9

  21. Savage A, Minshew LM, Anksorus HN, McLaughlin JE.Remote OSCE experience: What first year pharmacy studentsliked, learned, and suggested for future implementations.Pharmacy. 2021;9(1):62. https://doi.org/10.3390/pharmacy9010062

  22. Trejo-Mejía JA, Peña-Balderas J, Soto-Aguilera CA, Alpuche-Hernándeza A, Ortiz-Montalvo A, Cerritos A. Adaptaciónde un ECOE presencial a modalidad en línea para unexamen de altas consecuencias. Inv Ed Med. 2022;11(43):16-35. https://doi.org/10.22201/fm.20075057e.2022.43.21399

  23. Abreu LF, Cid A, Herrera G, Lara JVM, Laviada R, RodríguezC, Sánchez J. Perfil por competencias del médicogeneral mexicano. México: AMFEM, 2008.

  24. Universidad de la Salud. Plan de estudios: Licenciatura deMedicina General y Comunitaria. [Consultado 6 feb 2023].Disponible en: https://unisa.cdmx.gob.mx/storage/app/media/Plan%20de%20Estudios/PLAN%20DE%20ESTUDIOSTOMO%20I_MEDICINA.pdf

  25. Martínez-González A, Trejo-Mejía JA. ¿Cómo realizarun ECOE? Inv Ed Med. 2020;7(28):98-107. https://doi.org/10.22201/facmed.20075057e.2018.28.18123

  26. Chan SCC, Choa G, Kelly J, Maru, D, Rashid MA. Implementationof virtual OSCE in health professions education:A systematic review. Med Educ. 2023;57:833-843. https://doi.org/10.1111/medu.15089

  27. Sirianni G, Cho JSH, Rojas D, Lazor J, Tait G, Tu, Y, Nyhof-Young J, Kulasegaram K. A Mixed-Methods, Validity InformedEvaluation of a Virtual OSCE for UndergraduateMedicine Clerkship. Education in the Health Professions.2022;5(2):62-71. doi: 10.4103/EHP.EHP_3_22

  28. Martínez-González A, Trejo-Mejía JA, Fortoul-van derGoes TI, Flores-Hernández F, Morales-López S, Sánchez-Mendiola M. Evaluación diagnóstica de conocimientos ycompetencias en estudiantes de medicina al término delsegundo año de la carrera: el reto de construir el avión mientrasvuela. Gac Med Mex. 2014;150(1):35-48.

  29. Martínez-González A, Lifshitz-Guinzberg A, Trejo-MejíaJA, Torruco-García U, Fortoul-van der Goes TI, Flores-HernándezF, Peña-Balderas J, Martínez-Franco AI, Hernández-Nava A, Sánchez Mendiola M. Evaluación diagnósticay formativa de competencias en estudiantes de medicina asu ingreso al internado médico de pregrado. Gac Med Mex.2017;153(1):6-15.

  30. Hamann C, Volkan K, Fishman MB, Silvestri RC, SimonSR, Fletcher SW. How well do second-year students learnphysical diagnosis? Observational study of an ObjectiveStructured Clinical Examination (OSCE). BMC Med Educ.2002;2:1. doi: 10.1186/1472-6920-2-1.

  31. Felthun JZ, Taylor S, Shulruf B, Allen DB. Empirical analysiscomparing the tele-objective structured clinical examinationand the in-person assessment in Australia. J EducEval Health Prof. 2021;18:23. doi: 10.3352/jeehp.2021.18.23

  32. Vargas I, Ramírez C, Cortés J, Farfán A, Heinze G. Factoresasociados al rendimiento académico en alumnos de laFacultad de Medicina: estudio de seguimiento a un año.Salud Mental. 2011;34(4):301-308.

  33. Jara D, Velarde H, Gordillo G, Guerra G, León I, Arroyo C,Figueroa M. Factores influyentes en el rendimiento académicode estudiantes del primer año de medicina. An. Fac.Med. 2008;69(3):193-197. doi:10.15381/anales.v69i3.1140

  34. Gutiérrez-Cebollada J, Baillés E, Girvent M, Moyano E,Larramona P, Pérez-Sánchez J. Relación de las calificacionesen la prueba ECOE de la administración catalana conel expediente académico y los resultados en la prueba MIRen estudiantes de Medicina de la UPF-UAB. Revista Españolade Educación Médica. 2020;1(1):82-89. https://doi.org/10.6018/edumed.422961

  35. Hopwood J, Gil M, Alison S. Twelve tips for conductinga virtual OSCE. Med Teach. 2021;43(6):633-636. doi:10.1080/0142159X.2020.1830961

  36. O’Brien JE, Charlotte AT, Danielle S. Overcoming COVID-19 challenges: Using remote and hybrid simulationdesigns in DNP programs. Acad Med. 2022;97(2):S66-S70.doi: 10.1097/ACM.0000000000004534




2020     |     www.medigraphic.com