Strategy to systematize the written comments of the teaching evaluation in the medical residency programs
Abstract
Evaluating the opinions of residents about their programs, campuses, professors and other aspects of their daily work in medical specialties can be useful to improve aspects of their training. The open spaces to obtain written comments (WC) about their experience constitute an opportunity to understand in greater depth the teaching work, the problems in the hospital and the strengths of the educational program. We propose a three-step strategy to systematize the WC exemplified by the analysis of 721 WC obtained through the Teacher Evaluation Instrument, applied virtually in August 2022. WC require careful reading and can be coded to identify categories and from there theorize the findings. In the field of medical residences, we propose the classification into three general categories: structural, organizational and interpersonal for its application in future studies of this nature. The report of the experience and the evaluation of the events are central elements to approach what happens in educational spaces in clinical practice.
Downloads
Metrics
References
1. Sun H, Chen D, Warner DO, Zhou Y, Nemergut EC, Macario A, Keegan MT. Anesthesiology Residents' Experiences and Perspectives of Residency Training. Anesth Analg, 2021, 1;132(4), 1120-1128. http://doi.org/10.1213/ANE.0000000000005316
Acosta Fernández M, Aguilera Velasco MÁ, Torres López TM, et al. Factores psicosociales y formación académica. Percepción de residentes y profesores. Med Int Mex, 2022, 38(1), 51-66. https://doi.org/10.24245/mim.v38i1.4811
Ten Cate O, Schwartz A, Chen HC. Assessing Trainees and Making Entrustment Decisions: On the Nature and Use of Entrustment-Supervision Scales. Acad Med., 2020, 95(11), 1662-1669. http://doi.org/10.1097/ACM.0000000000003427
Ten Cate O, Regehr G. The power of subjectivity in the assessment of medical trainees. Acad Med. 2019, 94, 333–337. http://doi.org/10.1097/ACM.0000000000002495
Young JQ, Sugarman R, Holmboe E, O'Sullivan PS. Advancing Our Understanding of Narrative Comments Generated by Direct Observation Tools: Lessons From the Psychopharmacotherapy-Structured Clinical Observation. J Grad Med Educ. 2019, 11(5), 570-579. http://doi.org/10.4300/JGME-D-19-00207.1
Lefebvre C, Hiestand B, Glass C, Masneri D, Hosmer K, Hunt M, Hartman N. Examining the Effects of Narrative Commentary on Evaluators' Summative Assessments of Resident Performance. Eval Health Prof. 2020, 43(3).159-161. http://doi.org/10.1177/0163278718820415
Ginsburg S, Watling CJ, Schumacher DJ, Gingerich A, Hatala R. Numbers Encapsulate, Words Elaborate: Toward the Best Use of Comments for Assessment and Feedback on Entrustment Ratings. Acad Med, 2021 1;96(7S), S81-S86. http://doi.org/10.1097/ACM.0000000000004089
Hamui Sutton L, Sanchez Guzmán MA, Paulo Maya A, et al. Interacciones y narrativas en la clínica: más allá del cerebro. 1ra ed. CDMX, México, Facultad de Medicina, UNAM, 2022.
Hamui Sutton L, Vives Varela T. La codificación y categorización en la teoría fundamentada, un método para el análisis de los datos cualitativos. Investigación educ. médica, 2021, 10(40), 97-104. https://doi.org/10.22201/fm.20075057e.2021.40.21367.
Cook DA, Kuper A, Hatala R, Ginsburg S. When Assessment Data Are Words: Validity Evidence for Qualitative Educational Assessments. Acad Med, 2016, 91(10), 1359-1369. http://doi.org/10.1097/ACM.0000000000001175
Holm EA, Al-Bayati SJL, Barfod TS, Lembeck MA, Pedersen H, Ramberg E, Klemmensen ÅK, Sorensen JL. Feasibility, quality and validity of narrative multisource feedback in postgraduate training: a mixed-method study. BMJ Open. 2021, 11(7):e047019. http://doi.org/10.1136/bmjopen-2020-047019
Lockyer JM, Sargeant J, Richards SH, Campbell JL, Rivera LA. Multisource Feedback and Narrative Comments: Polarity, Specificity, Actionability, and CanMEDS Roles. J Contin Educ Health Prof, 2018, 38(1), 32-40. http://doi.org/10.1097/CEH.0000000000000183
Hatala R, Sawatsky AP, Dudek N, Ginsburg S, Cook DA. Using In-Training Evaluation Report (ITER) qualitative comments to assess medical students and residents: A systematic review. Acad Med. 2017, 92, 868–879. http://doi.org/10.1097/ACM.0000000000001506
Tremblay G, Carmichael PH, Maziade J, Gregoire M. Detection of residents with progress issues using a keyword-specific algorithm. J Grad Med Educ. 2019, 11, 656–662. http://doi.org/10.1097/ACM.0000000000002495
Hamui Sutton L, Vives Varela T. Las preguntas analíticas en investigación cualitativa. Investigación En Educación Médica, 2022, 11(41), 97-102. DOI: https://doi.org/10.22201/fm.20075057e.2022.41.21415
Ginsburg S, Regehr G, Lingard L, Eva KW. Reading between the lines: Faculty interpretations of narrative evaluation comments. Med Educ. 2015, 49, 296–306. http://doi.org/10.1111/medu.12637
Ginsburg S, Kogan JR, Gingerich A, Lynch M, Watling CJ. Taken out of context: Hazards in the interpretation of written assessment comments. Acad Med. 2020, 95, 1082–1088. http://doi.org/10.1097/ACM.0000000000003047
Wilby KJ, Govaerts MJB, Dolmans DHJM, Austin Z, van der Vleuten C. Reliability of narrative assessment data on communication skills in a summative OSCE. Patient Educ Couns. 2019,102, 1164–1169. http://doi.org/10.1016/j.pec.2019.01.018
Copyright (c) 2023 Servicio de Publicaciones de la Universidad de Murcia
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
The works published in this magazine are subject to the following terms:
1. The Publications Service of the University of Murcia (the publisher) preserves the economic rights (copyright) of the published works and favors and allows them to be reused under the use license indicated in point 2.
2. The works are published under a Creative Commons Attribution-NonCommercial-NoDerivative 4.0 license.
3. Self-archiving conditions. Authors are allowed and encouraged to disseminate electronically the pre-print versions (version before being evaluated and sent to the journal) and / or post-print (version evaluated and accepted for publication) of their works before publication , since it favors its circulation and earlier diffusion and with it a possible increase in its citation and reach among the academic community.
