Using Entrustable Professional Activities in Emergency Medicine Rotations for Postgraduate Year Trainees: A Psychometric Study

10.6145/jme.202206_26(2).0001

Tzu-Hung Liu；Ching-Hsiu Peng；Giou-Teng Yiang；Yu-Long Chen

entrustable professional activities ； emergency medicine ； postgraduate year trainee ； psychometric study ； exploratory factor analysis

Journal of Medical Education

26卷2期（2022 / 06 / 01）

100 - 111

英文

Purpose: Four entrustable professional activities (EPAs) were developed by the Taiwan Society of Emergency Medicine (TSEM) and evaluated in pilot programs for postgraduate year (PGY) trainees in their emergency medicine (EM) rotations in Taiwan in recent years. However, no validation results of the abovementioned EPA-based assessments have been reported. This study aimed to evaluate how the EPAs work in the context of postgraduate training in Taiwan. Methods: From August 2018 to February 2022, in their EM rotations, 112 PGY trainees in a teaching hospital in northern Taiwan underwent ad hoc EPA-based assessments of the four EPAs that had been performed in the EM. The entrustment levels of these EPAs were collected and examined through a series of psychometric tests. Results: Four EPAs and their 19 sub-EPAs were examined and showed excellent internal consistency (alpha: 0.870-0.936). Exploratory factor analyses revealed that all EPAs, except EPA 1, retained only one factor. In the first factor of EPA 1, the factor loadings of all eight sub-EPAs are positive. In the second factor, the factor loadings of two sub-EPAs are higher than 0.4, which indicates that the second factor measures other types of clinical competence, such as communication skills. Conclusions: The four EPAs developed by TSEM can be a valuable tool to assess clinical competencies of PGY trainees in EM rotations. However, EPA 1 measures more than one dimension of clinical competence. Low entrustment levels in this EPA-based assessment may suggest the need for training that is tailored to communication skills.

1. ten Cate O: Entrustability of professional activities and competency-based training. Med Educ 2005; 39: 1176-7.
   連結：
2. ten Cate O, Scheele F: Competency-based postgraduate training: can we bridge the gap between theory and clinical practice? Acad Med 2007; 82: 542-7.
   連結：
3. ten Cate O: Nuts and bolts of entrustable professional activities. J Grad Med Educ 2013; 5: 157-8.
   連結：
4. Crossley J, Jolly B: Making sense of work-based assessment: ask the right questions, in the right way, about the right things, of the right people. Med Educ 2012; 46: 28-37.
   連結：
5. Peters H, Holzhausen Y, Boscardin C, et al.: Twelve tips for the implementation of EPAs for assessment and entrustment decisions. Med Teach 2017; 39: 802-7.
   連結：
6. Beeson MS, Warrington S, Bradford-Saffles A, et al.: Entrustable professional activities: making sense of the emergency medicine milestones. J Emerg Med 2014; 47: 441-52.
   連結：
7. Tiyyagura G, Balmer D, Chaudoin L, et al.: The greater good: how supervising physicians make entrustment decisions in the pediatric emergency department. Acad Pediatr 2014; 14: 597-602.
   連結：
8. Hsu D, Nypaver M, Fein DM, et al.: Essentials of PEM fellowship part 2: the profession in entrustable professional activities. Pediatr Emerg Care 2016; 32: 410-8.
   連結：
9. Thompson LR, Leung CG, Green B, et al.: Development of an assessment for entrustable professional activity (EPA) 10: emergent patient management. West J Emerg Med 2017; 18: 35-42.
   連結：
10. Hart D, Franzen D, Beeson M, et al.: Integration of entrustable professional activities with the milestones for emergency medicine residents. West J Emerg Med 2018; 20: 35-42.
    連結：
11. Stoneham EJ, Witt L, Paterson QS, et al.: The development of entrustable professional activities reference cards to support the implementation of Competence by Design in emergency medicine. CJEM 2019; 21: 803-6.
    連結：
12. Scheele F, Teunissen P, Van Luijk S, et al.: Introducing competency-based postgraduate medical education in the Netherlands. Med Teach 2008; 30: 248-53.
    連結：
13. van Loon KA, Driessen EW, Teunissen PW, et al.: Experiences with EPAs, potential benefits and pitfalls. Med Teach 2014; 36: 698-702.
    連結：
14. Rekman J, Gofton W, Dudek N, et al.: Entrustability scales: Outlining their usefulness for competency-based clinical assessment. Acad Med 2016; 91: 186-90.
    連結：
15. Chen HC, van den Broek WE, ten Cate O: The case for use of entrustable professional activities in undergraduate medical education. Acad Med 2015; 90: 431-6.
    連結：
16. ten Cate O, Schwartz A, Chen HC: Assessing trainees and making entrustment decisions: on the nature and use of entrustment-supervision scales. Acad Med 2020; 95: 1662-9.
    連結：
17. Olsson U: Maximum likelihood estimation of the polychoric correlation coefficient. Psychometrika 1979; 44: 443-60.
    連結：
18. Pett MA, Lackey NR, Sullivan JJ: Making Sense of Factor Analysis: The Use of Factor Analysis for Instrument Development in Health Care Research. Thousand Oaks: Sage Publications, 2003; 72-7.
    連結：
19. Guadagnoli E, Velicer WF: Relation of sample size to the stability of component patterns. Psychol Bull 1988; 103: 265-75.
    連結：
20. MacCallum RC, Widaman KF, Zhang S, et al.: Sample size in factor analysis. Psychol Methods 1999; 4: 84-99.
    連結：
21. Cutrer WB, Russell RG, Davidson M, et al.: Assessing medical student performance of Entrustable Professional Activities: A mixed methods comparison of Co-Activity and Supervisory Scales. Med Teach 2020; 42: 325-32.
    連結：
22. Ryan MS, Khan AR, Park YS, et al.: Workplace-based entrustment scales for the core EPAs: a multisite comparison of validity evidence for two proposed instruments using structured vignettes and trained raters. Acad Med 2022; 97: 544-51.
    連結：
23. ten Cate O, Hart D, Ankel F, et al.: Entrustment decision making in clinical training. Acad Med 2016; 91: 191-8.
    連結：
24. ten Cate O, Regehr G: The power of subjectivity in the assessment of medical trainees. Acad Med 2019; 94: 333-7.
    連結：
25. ten Cate O, Chen HC: The ingredients of a rich entrustment decision. Med Teach 2020; 42: 1413-20.
    連結：
26. ten Cate O, Taylor DR: The recommended description of an entrustable professional activity: AMEE Guide No. 140. Med Teach 2021; 43: 1106-14.
    連結：
27. Postmes L, Tammer F, Posthumus I, et al.: EPA-based assessment: clinical teachers' challenges when transitioning to a prospective entrustment-supervision scale. Med Teach 2021; 43: 404-10.
    連結：
28. Martin L, Sibbald M, Brandt Vegas D, et al.: The impact of entrustment assessments on feedback and learning: Trainee perspectives. Med Educ 2020; 54: 328-36.
    連結：
29. ten Cate O, Balmer DF, Caretta-Weyer H, et al.: Entrustable professional activities and entrustment decision making: a development and research agenda for the next decade. Acad Med 2021; 96: S96-104.
    連結：
30. Violato C, Cullen MJ, Englander R, et al.: Validity evidence for assessing entrustable professional activities during undergraduate medical education. Acad Med 2021; 96: S70-5.
    連結：