medigraphic.com

2019, Number 4

<< Back Next >>

Cir Cir 2019; 87 (4)

Self-estimation of surgical skills and competencies based on the learning curve theory in medical residents and fellows

Borracci RA, Alvarez-Gallesio JM, Ciambrone G, Mezzadri NA

Language: English
References: 38
Page: 416-422
PDF size: 240.99 Kb.


ABSTRACT

Objective: The aim was to explore how in-training junior physicians perceive their surgical performance compared with the one externally rated by their senior surgeon trainers, using a general learning curve model. Methods: Between April and June 2018, a prospective study was conducted at a community hospital associated with a school of medicine. To assess how in-training physicians estimated their surgical performance, 48 surgical residents and fellows were invited to choose one among six options using a scale ranging from “novice” to “automatic expert.” In addition, five senior surgeons who supervised the residents/fellows were asked to give their own opinions on each surveyed physician’s expertise level, according to the same categories. Concordance analysis was done to compare residents’ and fellows’ self-perceived skills and their actual performance as estimated by senior surgeons. Results: Self-assessments tended to overestimate residents’ and fellows’ position on the learning curve; particularly for “proficient” over “competent,” and for “automatic expert” over “expert” categories (p = 0.025). The average degree of agreement among senior physicians was 50.0%. Comparison between residents’ and fellows’ perceived skills and their performances as estimated by senior surgeons showed a weak concordance (kappa = 0.494, 95% confidence interval 0.359-0.631, p ‹ 0.0001). Conclusions: Nearly 51% of the residents/fellows included in some surgical specialty training program overestimated his/her actual performance as evaluated by classical learning curve categories. Underestimation of self-assessed performance was also observed in 17% of respondents. A better feedback from expert observers to in-training surgeons could result in a more accurate self-perception of their real surgical skills and competencies.


REFERENCES

  1. Roland D, Matheson D, Coats T, Martin G. A qualitative study of self-evaluation of junior doctor performance: is perceived safeness a more useful metric than confidence and competence? BMJ Open. 2015;5:e008521.

  2. Pallier G, Wilkinson R, Danthiir V, Kleitman S, Knezevic G, Stankov L, et al. The role of individual differences in the accuracy of confidence judgments. J Gen Psychol. 2002;129:257-99.

  3. Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE, Perrier L, et al. Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. JAMA. 2006; 296:1094-102.

  4. Barnsley L, Lyon PM, Ralston SJ, Hibbert EJ, Cunningham I, Gordon FC, et al. Clinical skills in junior medical officers: a comparison of self-reported confidence and observed competence. Med Educ. 2004;38:358-67.

  5. Coberly L, Goldenhar LM. Ready or not, here they come: acting interns experience and perceived competency performing basic medical procedures. J Gen Intern Med. 2007;22:491-4.

  6. Jones A, McArdle PJ, O’Neill PA. How well prepared are graduates for the role of pre-registration house officer? A comparison of the perceptions of new graduates and educational supervisors. Med Educ. 2001; 35:578-84.

  7. Lai NM, Teng CL. Self-perceived competence correlates poorly with objectively measured competence in evidence based medicine among medical students. BMC Med Educ. 2011;11:25.

  8. Ward M, MacRae H, Schlachta C, Mamazza J, Poulin E, Reznick R, et al. Resident self-assessment of operative performance. Am J Surg. 2003; 185:521-4.

  9. Sidhu RS, Vikis E, Cheifetz R, Phang T. Self-assessment during a 2-day laparoscopic colectomy course: can surgeons judge how well they are learning new skills? Am J Surg. 2006;191:677-81.

  10. Sarker SK, Hutchinson R, Chang A, Vincent C, Darzi AW. Self-appraisal hierarchical task analysis of laparoscopic surgery performed by expert surgeons. Surg Endosc. 2006;20:636-40.

  11. van Empel PJ, Verdam MG, Huirne JA, Bonjer HJ, Meijerink WJ, Scheele F, et al. Open knot-tying skills: resident skills assessed. J Obstet Gynaecol Res. 2013;39:1030-6.

  12. Brewster LP, Risucci DA, Joehl RJ, Littooy FN, Temeck BK, Blair PG, et al. Comparison of resident self-assessments with trained faculty and standardized patient assessments of clinical and technical skills in a structured educational module. Am J Surg. 2008;195:1-4.

  13. Pandey VA, Wolfe JH, Black SA, Cairols M, Liapis CD, Bergqvist D, et al. Self-assessment of technical skill in surgery: the need for expert feedback. Ann R Coll Surg Engl. 2008;90:286-90.

  14. de Blacam C, O’Keeffe DA, Nugent E, Doherty E, Traynor O. Are residents accurate in their assessments of their own surgical skills? Am J Surg. 2012;204:724-31.

  15. Tedesco MM, Pak JJ, Harris EJ Jr. Krummel TM, Dalman RL, Lee JT, et al. Simulation-based endovascular skills assessment: the future of credentialing? J Vasc Surg. 2008;47:1008-1.

  16. Hu Y, Tiemann D, Michael Brunt L. Video self-assessment of basic suturing and knot tying skills by novice trainees. J Surg Educ. 2013;70:279‑83.

  17. Quick JA, Kudav V, Doty J, Crane M, Bukoski AD, Bennett BJ, et al. Surgical resident technical skill self-evaluation: increased precision with training progression. J Surg Res. 2017;218:144-9.

  18. Ganni S, Chmarra MK, Goossens RH, Jakimowicz JJ. Self-assessment in laparoscopic surgical skills training: is it reliable? Surg Endosc. 2017;31:2451-6.

  19. Arora S, Miskovic D, Hull L, Moorthy K, Aggarwal R, Johannsson H, et al. Self vs expert assessment of technical and non-technical skills in high fidelity simulation. Am J Surg. 2011;202:500-6.

  20. Rizan C, Ansell J, Tilston TW, Warren N, Torkington J. Are general surgeons able to accurately self-assess their level of technical skills? Ann R Coll Surg Engl. 2015;97:549-55.

  21. Zevin B. Self versus external assessment for technical tasks in surgery: a narrative review. J Grad Med Educ. 2012;4:417-24.

  22. Pusic MV, Boutis K, Hatala R, Cook DA. Learning curves in health professions education. Acad Med. 2015;90:1034-42.

  23. Dreyfus SE, Dreyfus HK. A Five-Stage Model of the Mental Activities Involved in Directed Skill Acquisition. Berkeley, California: University of California, Berkeley; 1980.

  24. Ericsson K. The acquisition of expert performance as problem solving. In: Davidson J, Sternberg R, editors. The Psychology of Problem Solving. 1st ed. Cambridge, England: Cambridge University Press; 2003. p. 44-83.

  25. Ericsson KA. Deliberate practice and acquisition of expert performance: a general overview. Acad Emerg Med. 2008;15:988-94.

  26. Sackett DL, Haynes RB, Guyatt GH, Tugwell P. Clinical Epidemiology: a Basic Science for Clinical Medicine. 2nd ed. Boston: Brown and Co.; 1991.

  27. Cicchetti DV. Testing the normal approximation and minimal sample size requirements of weighted kappa when the number of categories is large. Appl Psychol Meas. 1981;5:101-4.

  28. Morgan PJ, Cleave-Hogg D. Comparison between medical students experience, confidence and competence. Med Educ. 2002;36 534-9.

  29. Mitchell RE, Clark PE, Scarpero HM. Assessing the surgical skills of urology residents after preurology general surgery training: the surgical skills learning needs of new urology residents. J Surg Educ. 2011;68:341‑6.

  30. Aldave G, Hansen D, Briceño V, Luerssen TG, Jea A. Assessing residents’ operative skills for external ventricular drain placement and shunt surgery in pediatric neurosurgery. J Neurosurg Pediatr. 2017;19:377-83.

  31. Moorthy K, Munz Y, Adams S, Pandey V, Darzi A, Imperial College St. Mary’s Hospital Simulation Group. et al. Self-assessment of performance among surgical trainees during simulated procedures in a simulated operating theater. Am J Surg. 2006;192:114-8.

  32. Munz Y, Moorthy K, Bann S, Shah J, Ivanova S, Darzi SA, et al. Ceiling effect in technical skills of surgical residents. Am J Surg. 2004;188:294‑300.

  33. Bradley KE, Andolsek KM. A pilot study of orthopaedic resident self-assessment using a milestones survey just prior to milestones implementation. Int J Med Educ. 2016;7:11-8.

  34. Meyer AN, Payne VL, Meeks DW, Rao R, Singh H. Physicians diagnostic accuracy, confidence, and resource requests: a vignette study. JAMA Intern Med. 2013;173:1952-8.

  35. Klayman J, Soll JB, González-Vallejo C, Barlas S. Overconfidence: it depends on how, what, and whom you ask. Organ Behav Hum Decis Process. 1999;79:216-47.

  36. Koriat A, Sheffer L, Ma’ayan H. Comparing objective and subjective learning curves: judgments of learning exhibit increased underconfidence with practice. J Exp Psychol Gen. 2002;131:147-62.

  37. Finn B, Metcalfe J. The role of memory for past test in the underconfidence with practice effect. J Exp Psychol Learn Mem Cogn. 2007; 33:238‑44.

  38. Eva KW, Regehr G. Exploring the divergence between self-assessment and self-monitoring. Adv Health Sci Educ Theory Pract. 2011;16:311-29.




2020     |     www.medigraphic.com