medigraphic.com
ISSN 2007-5057 (Print)
Investigación en Educación Médica

2023, Number 46

<< Back Next >>

Inv Ed Med 2023; 12 (46)

Analysis of evaluative instruments used in science during the pandemic: multiple selection, indicators and performance

Urrejola-Contreras GP, Pérez-Lizama MA

Language: Spanish
References: 37
Page: 19-30
PDF size: 689.80 Kb.


ABSTRACT

Introduction: Multiple-choice assessments are the instrument widely used in science to assess students, however, the recent pandemic required adapting this type of instrument to the virtual environment. This context required evaluating the quality of the instruments through discrimination indices, internal consistency and relating it to academic performance.
Objective: Evaluate the evaluation instruments used in online mode during the COVID-19 pandemic and the performance of students in health sciences.
Method: The revision of the 5 instruments of the Structure and Function subject formed by 290 banks of random virtuaquestions was carried out to evaluate each content in first-year students during 2020 in the school of health sciences at the Viña del Mar University. The data obtained from the virtual platform and the indices of discrimination, facility, discriminative efficiency, internal consistency and academic performance were interpreted through a report that was shared with the teachers to identify the parameters of quality and validity.
Results: Of the total number of question banks evaluated, 70.2% of the questions presented adequate discrimination and only 5.6% should be eliminated. Contest two obtained the lowest average performance 3.9 ± 0.99, however, it presented the highest internal consistency 81%. When comparing all the instruments, a gradual improvement in the formulation was observed, reflected in the final exam, in which the academic performance also agrees with the average of the semester 4.2 ± 0.92.
Conclusions: Academic performance must be weighed in relation to the quality of the formulated instrument, in which, at a lower ease index, there is greater internal consistency, represented by the greater discriminative efficiency of the questions. The design and formulation process must take care of and examine these guidelines to safeguard quality criteria.


REFERENCES

  1. Voutilainen A, Saaranen T, Sormunen M. Conventional vs. elearningin nursing education: A systematic review and meta-analysis. Nurse Educ Today. 2017;50:97-103. doi:10.1016/j.nedt.2016.12.020

  2. Lawn S, Zhi X, Morello A. An integrative review of e-learningin the delivery of self-management support trainingfor health professionals. BMC Med Educ. 2017;17(1):183. doi:10.1186/s12909-017-1022-0

  3. Villaroel Quinchalef G del P, Fuentes Salvo M de los Á,Oyarzún Muñoz VH. Implementación de curso onlinede Anatomía y la percepción de los estudiantes de Kinesiología.Inv Ed Med. 2020;(35):75-84. doi: 10.22201/facmed.20075057e.2020.35.20226

  4. Lisperguer S, Calvo M, Urrejola G, Pérez M. Clinical reasoningtraining based on the analysis of clinical case using avirtual environment. Educ Med. 2020;594:1-5. doi: 10.1016/j.edumed.2020.08.002

  5. Goh PS, Sandars J. A vision of the use of technology in medicaleducation after the COVID-19 pandemic. MedEdPublish.2020;9:49. doi: 10.15694/mep.2020.000049.1

  6. Gaur U, Majumder MAA, Sa B, Sarkar S, Williams A, SinghK. Challenges and Opportunities of Preclinical Medical Education:COVID-19 Crisis and Beyond. SN Compr Clin Med.noviembre de 2020;2(11):1992-7. doi: 10.1016/j.glt.2021.11.001

  7. Arandjelovic A, Arandjelovic K, Dwyer K, Shaw C. COVID-19:Considerations for Medical Education during a Pandemic.MedEdPublish. 2020;9:87. doi:10.15694/mep.2020.000087.1

  8. Medical Education Department, School of Medical Sciences,Health Campus, Universiti Sains Malaysia, 16150 KubangKerian, Kelantan, Malaysia, Abdul Rahim AF. Guidelinesfor Online Assessment in Emergency Remote Teaching duringthe COVID-19 Pandemic. Educ Med J. 30 de junio de2020;12(2):59-68. doi: 10.52494/UCML9733

  9. Álvarez-Vázquez M, Álvarez-Méndez AM, Bravo-Llatas C,Angulo-Carrere MT. Análisis multivariante del uso de espaciosvirtualizados por estudiantes pregraduados en cienciasde la salud. 2021;24(6):317-21. doi: 10.33588/fem.246.1159

  10. Bautista-Rodríguez G, Gatica-Lara F. Factores relacionadoscon el rendimiento académico en una carrera técnica ensalud impartida en línea. Inv Ed Med. 2020;(33):89-97. doi:10.22201/facmed.20075057e.2020.33.19177

  11. Regmi K, Jones L. A systematic review of the factors –enablers and barriers– affecting e-learning in health scienceseducation. BMC Med Educ. 2020;20(1):91. doi: 10.1186/s12909-020-02007-6

  12. Alsoufi A, Alsuyihili A, Msherghi A, Elhadi A, Atiyah H,Ashini A, et al. Impact of the COVID-19 pandemic onmedical education: Medical students’ knowledge, attitudes,and practices regarding electronic learning. Plos One.2020;15(11):e0242905. doi: 10.1371/journal.pone.0242905

  13. Padilha JM, Machado PP, Ribeiro AL, Ribeiro R, Vieira F,Costa P. Easiness, usefulness and intention to use a MOOCin nursing. Nurse Educ Today. 2021;97:104705. doi: 10.1016/j.nedt.2020.104705

  14. Yilmaz Y, Sarikaya O, Senol Y, Baykan Z, Karaca O, DemiralYilmaz N, et al. RE-AIMing COVID-19 online learning formedical students: a massive open online course evaluation.BMC Med Educ. 2021;21(1):303. doi: 10.1186/s12909-021-02751-3

  15. Logan RM, Johnson CE, Worsham JW. Development of ane-learning module to facilitate student learning and outcomes.Teach Learn Nurs. 2021;16(2):139-42. doi: 10.1016/j.teln.2020.10.007

  16. Heidarzadeh A, Zehtab Hashemi H, Parvasideh P, HasanLarijani Z, Baghdadi P, Fakhraee M, et al. Opportunitiesand Challenges of Online Take-Home Exams in MedicalEducation. J Med Educ; 2021;20(1). doi: 10.5812/jme.112512

  17. Justo-Cousiño LA. ¿Podemos evaluar con garantías durantela pandemia de COVID-19? Evaluar sin devaluar lasprofesiones sanitarias. FEM. 2020;23(4):229. doi: 10.33588/fem.234.1075

  18. Urrejola-Contreras GP, Tiscornia-González C. Retroalimentaciónestudiantil sobre herramientas sincrónicas y asincrónicasempleadas en ciencias de la salud en la pandemia porCOVID-19. FEM. 2022;25(1):39. doi: 10.33588/fem.251.1168

  19. Barteit S, Guzek D, Jahn A, Bärnighausen T, Jorge MM,Neuhann F. Evaluation of e-learning for medical educationin low- and middle-income countries: A systematic review.Comput Educ. 2020;145:103726. doi: 10.1016/j.compedu.

  20. 2019.10372620. Carrillo-Avalos BA, Sánchez Mendiola M, Leenen I. Amenazasa la validez en evaluación: implicaciones en educaciónmédica. Inv Ed Med. 2020;(34):100-7. doi: 10.22201/facmed.20075057e.2020.34.221

  21. Giaconi E, Bazán ME, Castillo M, Hurtado A, Rojas H,Giaconi V, et al. Análisis de pruebas de opción múltiple encarreras de la salud de la Universidad Mayor. Inv Ed Med.2021;(40):61-9. doi: 10.22201/fm.20075057e.2021.40.21365

  22. Núñez J. Educación médica durante la crisis por COVID-19.2020. 21(3):157. doi: 10.1016/j.edumed.2020.05.001

  23. Luna de la Luz V, González P. Transformaciones en educaciónmédica: innovaciones en la evaluación de los aprendizajesy avances tecnológicos (parte 2). Inv Ed Med; 2020.9(34):87-99. doi:10.22201/facmed.20075057e.2020.34.20220

  24. Almahasees Z, Mohsen K, Omar Amin M. Faculty’s and Students’Perceptions of Online Learning During COVID-19.Front Educ 2021. 6:638470. doi:10.3389/feduc.2021.63847

  25. Mukhtar K, Javed K, Arooj M, Sethi A. Advantages, Limitationsand Recommendations for online learning during COVID-19 pandemic era: Online learning during COVID-19pandemic era. Pak J Med Sci. 2022;36:COVID19-S4. doi:10.12669/pjms.36.COVID19-S4.2785

  26. González T, De la Rubia M, Hincz K, Comas M, SubiratsL, Fort S, et al. Influence of COVID-19 confinement onstudents’ performance in higher education. Pak J Med Sci2020;15(10):e0239490. doi: 10.12669/pjms.36.COVID19-S4.2785

  27. Fabriz S, Mendzheritskaya J, Stehle S. Impact of Synchronousand Asynchronous Settings of Online Teaching andLearning in Higher Education on Students’ Learning ExperienceDuring COVID-19. Front Phsycol. 2021;12:733554.doi:10.3389/fpsyg.2021.733554

  28. Coughlin PA, Featherstone CR. How to Write a High QualityMultiple Choice Question (MCQ): A Guide for Clinicians.Eur J Vasc Endovasc Surg. 2017;54(5):654-8. doi: 10.1016/j.ejvs.2017.07.012

  29. Abhijeet I, Purushottam G, Mohan D. Study on item and testanalysis of multiple choice questions amongst undergraduatemedical students. Int J Community Med Public Health.2017;4(5):1562-5. doi:10.18203/2394-6040.ijcmph20171764

  30. Kolomitro K, MacKenzie LW, Lockridge M, Clohosey D. Problem‐solving strategies used in anatomical multiple‐choicequestions. Health Sci Rep. 2020;3(4). doi:10.1002/hsr2.209

  31. Douthit N, Norcini J, Mazuz K, Alkan M, Feuerstein M,Clarfield M, et al. Assessment of global health education:the role of multiple-choice questions. Int J Appl Basic MesRes. 2021;9(640204). doi: 10.3389/fpubh.2021.640204.

  32. Butler AC. Multiple-choice testing in education: Are thebest practices for assessment also good for learning? JAppl Res Mem Cogn. 2018;7(3):323-31. doi: 10.1016/j.jarmac.2018.07.002

  33. Burud I, Nagandla K, Agarwal P. Impact of distractors initem analysis of multiple choice questions. Int J Res Med Sci.2019;7(4):1136. doi: 10.18203/2320-6012.ijrms20191313

  34. Scott K, King A, Estes M, Conlon L, Phillips A. Evaluationof an Intervention to Improve Quality of Single-bestAnswer Multiple-choice Questions. West J Emerg Med.2018;20(1):11-4. doi: 10.5811/westjem.2018.11.39805

  35. Moore S, Nguyen HA, Stamper J. Examining the Effectsof Student Participation and Performance on the Qualityof Learnersourcing Multiple-Choice Questions. En:Proceedings of the Eighth ACM Conference on Learning@ Scale. Virtual Event Germany: ACM; 2021, p. 209-20.doi:10.1145/3430895.3460140

  36. Gupta P, Meena P, Khan A, Malhotra R, Singh T. Effect offaculty training on quality of multiple-choice questions. IntJ Appl Basic Med Res. 2020;10(3):210. doi: 10.4103/ijabmr.IJABMR_30_20

  37. Przymuszała P, Piotrowska K, Lipski D, Marciniak R, Cerbin-Koczorowska M. Guidelines on Writing Multiple ChoiceQuestions: A Well-Received and Effective Faculty DevelopmentIntervention. SAGE Open. 2020;10(3):215824402094743.doi: 10.1177/2158244020947432




2020     |     www.medigraphic.com