Workplace-Based Assessment of Admission Documentation and Medical Orders in Specialty Training: Development and Initial Validation of a Multidomain Tool.
Abstract
Introduction: Admission documentation and medical orders are important workplace activities in specialty training because they communicate clinical reasoning, guide patient care, and provide evidence of professional competence. However, few validated tools assess these practices in postgraduate clinical settings. This study developed and validated a multidomain tool for assessing the completeness of admission documentation and medical orders completed by doctors in specialty training.
Methods: This cross-sectional methodological study was conducted in three phases. Candidate domains and items were identified through a structured literature review and expert discussion, resulting in a preliminary tool with five domains and 20 items. Content validity was examined using a two-round Delphi process with 13 internal medicine faculty members. Internal structure validity was assessed by confirmatory factor analysis using data from 100 doctors in specialty training in internal medicine, cardiology, and neurology, and internal consistency was evaluated using Cronbach’s alpha. The finalized tool was then applied to 200 inpatient medical records from six teaching hospitals.
Results: Expert consensus supported the relevance and clarity of all five domains and 20 items. Confirmatory factor analysis supported the proposed five-domain structure, with good model fit (RMSEA ≤ 0.06, p < 0.001). In
ternal consistency was acceptable (Cronbach’s alpha = 0.79). Application of the tool to 200 records identified variation in documentation performance across domains and specialties; internal medicine trainees achieved the highest overall scores. All domain scores were significantly correlated with the total score (p < 0.001).
Discussion: The tool provided validity evidence for structured assessment of admission documentation and medical orders in specialty training. It may help clinical educators identify documentation gaps, provide targeted feedback, and design educational interventions that support trainee learning and safer patient care in teaching hospitals.
Downloads
-
Abstract0
-
xml0
-
pdf0
References
1. Carraccio C, Englander R, Van Melle E, et al. Advancing competency-based medical education: a charter for clinician–educators. Acad Med. 2016, 91(5), 645-649. https://doi.org/10.1097/ACM.0000000000001048
2. Schuwirth LW, van der Vleuten CP. A history of assessment in medical education. Adv Health Sci Educ Theory Pract. 2020, 25(5), 1045-1056. https://doi.org/10.1007/s10459-020-10003-0
3. Weller JM, Coomber T, Chen Y, et al. Key dimensions of innovations in workplace-based assessment for postgraduate medical education: a scoping review. Br J Anaesth. 2021, 127(5), 689-703. https://doi.org/10.1016/j.bja.2021.06.038
4. Demsash AW, Kassie SY, Dubale AT, et al. Health professionals’ routine practice documentation and its associated factors in a resource-limited setting: a cross-sectional study. BMJ Health Care Inform. 2023, 30(1), e100699. https://doi.org/10.1136/bmjhci-2022-100699
5. Wilesmith S, Mandrusiak A, Lang R, et al. Educational interventions to develop and enhance clinical documentation skills in health professional students: a systematic review. Clin Teach. 2025, 22(5), e70157. https://doi.org/10.1111/tct.70157
6. Grauer A, Rosen A, Applebaum JR, et al. Examining medication ordering errors using AHRQ network of patient safety databases. J Am Med Inform Assoc. 2023, 30(5), 838-845. https://doi.org/10.1093/jamia/ocad007
7. Richardson KM, Cristiano JA, Schafer KR, et al. Writing is thinking: implementation and evaluation of an internal medicine residency clinical reasoning and documentation curriculum. Med Sci Educ. 2022, 32(4), 773-777. https://doi.org/10.1007/s40670-022-01570-5
8. Bowker D, Torti J, Goldszmidt M. Documentation as composing: how medical students and residents use writing to think and learn. Adv Health Sci Educ Theory Pract. 2023, 28(2), 453-475. https://doi.org/10.1007/s10459-022-10167-x
9. Burke HB, Hoang A, Becher D, et al. QNOTE: an instrument for measuring the quality of EHR clinical notes. J Am Med Inform Assoc. 2014, 21(5), 910-916. https://doi.org/10.1136/amiajnl-2013-002321
10. Stetson PD, Bakken S, Wrenn JO, et al. Assessing electronic note quality using the physician documentation quality instrument (PDQI-9). Appl Clin Inform. 2012, 3(2), 164-174. https://doi.org/10.4338/ACI-2011-11-RA-0070
11. Baker EA, Ledford CH, Fogg L, et al. The IDEA assessment tool: assessing the reporting, diagnostic reasoning, and decision-making skills demonstrated in medical students’ hospital admission notes. Teach Learn Med. 2015, 27(2), 163-173. https://doi.org/10.1080/10401334.2015.1011654
12. Kotwal S, Klimpl D, Tackett S, et al. Documentation of clinical reasoning in admission notes of hospitalists: validation of the CRANAPL assessment rubric. J Hosp Med. 2019, 14(12), 746-753. https://doi.org/10.12788/jhm.3233
13. Kelly WF, Hawks MK, Johnson WR, et al. Assessment tools for patient notes in medical education: a scoping review. Acad Med. 2025, 100(3), 358-374. https://doi.org/10.1097/ACM.0000000000005886
14. Messick S. Validity. In: Linn RL, ed. Educational Measurement. 3rd ed. American Council on Education and Macmillan Publishing Company. 1989, 13-103.
15. Ebbers T, Kool RB, Smeele LE, et al. The impact of structured and standardized documentation on documentation quality: a multicenter, retrospective study. Journal of Medical Systems. 2022, 46(7), 46. https://doi.org/10.1007/s10916-022-01837-9
16. Sánchez Ferrer F, Díaz J, Maestre Ferrándiz J, Cortes-Castell E, Gil Guillen V, Rizo M, Nso-Roca AP. Prescribing Patient Associations: Bringing Medical Education Closer to the Patient's Reality. Spanish Journal of Medical Education. 2026, 7(5). https://doi:10.6018/edumed.717851
17. Dean B, Barber N, Schachter M. What is a prescribing error? BMJ Quality & Safety. 2000, 9(4), 232-237. https://doi.org/10.1136/qhc.9.4.232
18. Usama M, Arshad R, Rizwan U, et al. Evaluating the Quality and Completeness of Inpatient Medical Record Documentation: A Clinical Audit. Journal of Medical & Health Sciences Review. 2025, 2(1). https://doi.org/10.62019/4m4e3720
19. Tariq MD, Hamza HM, Ahsan A, et al. Unveiling documentation deficiencies: a clinical audit of histopathology request forms at a tertiary care hospital. Ann Med Surg (Lond). 2025, 87(4), 2000-2005. https://doi.org/10.1097/MS9.0000000000003038
20. Wurster F, Herrmann C, Beckmann M, et al. Differences in changes of data completeness after the implementation of an electronic medical record in three surgical departments of a German hospital: a longitudinal comparative document analysis. BMC Medical Informatics and Decision Making. 2024, 24(1), 258. https://doi.org/10.1186/s12911-024-02667-0
Copyright (c) 2026 Servicio de Publicaciones de la Universidad de Murcia
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
The works published in this magazine are subject to the following terms:
1. The Publications Service of the University of Murcia (the publisher) preserves the economic rights (copyright) of the published works and favors and allows them to be reused under the use license indicated in point 2.
2. The works are published under a Creative Commons Attribution-NonCommercial-NoDerivative 4.0 license.
3. Self-archiving conditions. Authors are allowed and encouraged to disseminate electronically the pre-print versions (version before being evaluated and sent to the journal) and / or post-print (version evaluated and accepted for publication) of their works before publication , since it favors its circulation and earlier diffusion and with it a possible increase in its citation and reach among the academic community.
This is a Diamond Journal 
