Résumés
Abstract
Background: As competency-based medical education (CBME) curricula are introduced in residency programs across Canada, systematic evaluation efforts are needed to ensure fidelity of implementation. This study evaluated early outcomes of CBME implementation in one Canadian Physical Medicine and Rehabilitation program that was an early adopter of CBME, with an aim to inform continuous quality improvement initiatives and CBME implementation nationwide.
Methods: Using Rapid Evaluation methodology, informed by the CBME Core Components Framework, the intended outcomes of CBME were compared to actual outcomes.
Results: Results suggested that a culture of feedback and coaching already existed in this program prior to CBME implementation, yet faculty felt that CBME added a framework to support feedback. The small program size was valuable in fostering strong relationships and individualized learning. However, participants expressed concerns about CBME fostering a reductionist approach to the development of competence. Challenges existed with direct observation, clear expectations for off-service training experiences, and tracking trainee progress. There was trepidation surrounding national curricular change, yet the institution-wide approach to CBME implementation created shared experiences and a community of practice.
Conclusions: Program evaluation can help understand gaps between planned versus enacted implementation of CBME, and foster adaptations to improve the fidelity of implementation.
Résumé
Contexte : À mesure que les programmes d’approche par compétences (APC) en formation médicale sont introduits dans les programmes de résidence au Canada, des efforts d'évaluation systématiques sont nécessaires pour assurer la fidélité de la mise en œuvre. Cette étude a évalué les premiers résultats de la mise en œuvre de l’APC en formation médicale dans un programme canadien de médecine physique et réadaptation, qui a été un des premiers à adopter l’APC, dans le but d'orienter les initiatives d'amélioration continue de la qualité et de la mise en œuvre de l’APC à l'échelle nationale.
Méthodes : En utilisant une méthodologie d’évaluation rapide, fondée sur le cadre des composantes de base de l’APC en formation médicale, les résultats escomptés de l’APC ont été comparés aux résultats réels.
Résultats : Les résultats suggèrent qu'une culture de la rétroaction et de l’encadrement existait déjà dans ce programme avant la mise en œuvre de l’APC, mais le corps professoral a estimé que l’APC en formation médicale a ajouté un cadre pour soutenir cette rétroaction. La petite taille du programme a permis de favoriser des relations solides et un apprentissage individualisé. Cependant, les participants ont exprimé des inquiétudes quant au fait que l’APC favorise une approche réductionniste du développement des compétences. L'observation directe, les attentes claires en matière d'expériences de formation hors de l’environnement clinique et le suivi des progrès des résidents posent problème. Le changement de programme national a suscité des inquiétudes, mais l'approche institutionnelle de la mise en œuvre de l’APC a permis de partager des expériences et de créer une communauté de pratique.
Conclusions : L'évaluation des programmes peut aider à comprendre les écarts entre la mise en œuvre planifiée et effective de l’APC en formation médicale, et de favoriser les adaptations pour améliorer le respect des conditions de mise en œuvre.
Veuillez télécharger l’article en PDF pour le lire.
Télécharger
Parties annexes
Bibliography
- van Melle E, Hall AK, Schumacher DJ, et al. Capturing outcomes of competency-based medical education: the call and the challenge. Med Teach. 2021;43(7). https://doi.org/10.1080/0142159X.2021.1925640
- Hamza DM, Ross S, Oandasan I. Process and outcome evaluation of a CBME intervention guided by program theory. J Eval Clin Pract. Vol 26; 2020. https://doi.org/10.1111/jep.13344
- King JA, Morris LL, Fitz-Gibbon CT. How to assess program implementation.; 1987.
- Hall AK, Rich J, Dagnone JD, et al. It’s a marathon, not a sprint: rapid evaluation of competency-based medical education program implementation. Acad Med. Published online 2020. https://doi.org/10.1097/ACM.0000000000003040
- Kouzmina E, Mann S, Chaplin T, Zevin B. An evaluation of the surgical foundations curriculum: a national study. J Surg Educ. 2021;78(3). https://doi.org/10.1016/j.jsurg.2020.10.002
- Federation des Medecins Residents du Quebec. Year 3 of implementation of competence by design: negative impact still outweighs theoretical benefits. observations on the day-to-day reality of CBD and its progression since July 2017. July 2020. Available from: https://fmrq.qc.ca/wp-content/uploads/2022/07/fmrq-report-cbd-implementation-year-3_1.pdf. [Accessed Oct 28, 2023].
- Sherbino J, Regehr G, Dore K, Ginsburg S. Tensions in describing competency-based medical education: a study of Canadian key opinion leaders. AdvHealth Sci Ed. 2021;26(4). https://doi.org/10.1007/s10459-021-10049-8
- Gold M, Helms D, Guterman S. Identifying, monitoring, and assessing promising innovations: using evaluation to support rapid-cycle change. Issue Brief (Commonw Fund). 2011;12.
- van Melle E, Frank JR, Holmboe ES, Dagnone D, Stockley D, Sherbino J. A core components framework for evaluating implementation of competency-based medical education programs. Acad Med. 2019;94(7):1002-1009. https://doi.org/10.1097/ACM.0000000000002743
- Stockley D, Egan R, van Wylick R, et al. A systems approach for institutional CBME adoption at Queen’s University. Med Teach. 2020;42(8):916-921. https://doi.org/10.1080/0142159X.2020.1767768
- Frank JR, Snell L, Sherbino J Editors. CanMEDS 2015 Physician Competency Framework. Ottawa: Royal College of Physicians and Surgeons of Canada. CanMEDS 2015 Physician Competency Framework Ottawa: Royal College of Physicians and Surgeons of Canada. Published online 2015.
- Royal College of Physicians and Surgeons of Canada. What is CBD? The Royal College of Physicians and Surgeons of Canada. https://www.royalcollege.ca/rcsite/cbd/what-is-cbd-e [Accessed Jan 19, 2023].
- Royal College of Physicians and Surgeons of Canada. The Competence by Design (CBD) coaching model. 2017. Available from: https://www.royalcollege.ca/content/dam/documents/accreditation/competence-by-design/directory/cbd-coaching-model-handout-en.pdf [Accessed Oct 28, 2023].
- Pero R, Pero E, Marcotte L, Dagnone JD. Educational consultants: fostering an innovative implementation of competency-based medical education. Med Educ. 2019;53(5). https://doi.org/10.1111/medu.13868
- van Melle E. How do we know it works? (Closing Plenary) Presented at: World Summit on CBME. Published online August 25, 2018.
- Lipscomb M. Abductive reasoning and qualitative research. Nurs Philos. 2012;13(4). https://doi.org/10.1111/j.1466-769X.2011.00532.x
- Timmermans S, Tavory I. Theory construction in qualitative research: from grounded theory to abductive analysis. Sociol Theory. 2012;30(3). https://doi.org/10.1177/0735275112457914
- Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9). https://doi.org/10.1177/1049732305276687
- Birt L, Scott S, Cavers D, Campbell C, Walter F. Member checking: a tool to enhance trustworthiness or merely a nod to validation? Qual Health Res. 2016;26(13). https://doi.org/10.1177/1049732316654870
- Bloch M, Lave J, Wenger E. Situated learning: legitimate peripheral participation. Man. 1994;29(2). https://doi.org/10.2307/2804509
- Frank JR, Snell LS, Cate O ten, et al. Competency-based medical education: theory to practice. Med Teach. 2010;32(8). https://doi.org/10.3109/0142159X.2010.501190
- Richardson D, Kinnear B, Hauer KE, et al. Growth mindset in competency-based medical education. Med Teach. 2021;43(7):751-757. https://doi.org/10.1080/0142159X.2021.1928036
- Ott M, Pack R, Cristancho S, et al. “The most crushing thing”: understanding resident assessment burden in a competency-based curriculum. JGME. 2022;14(5). https://doi.org/10.4300/JGME-D-22-00050.1
- Spillane JP, Reiser BJ, Reimer T. Policy implementation and cognition: Reframing and refocusing implementation research. Rev Educ Res. 2002;72(3). https://doi.org/10.3102/00346543072003387
- Griffiths J, Dalgarno N, Schultz K, Han H, van Melle E. Competency-Based Medical Education implementation: Are we transforming the culture of assessment? Med Teach. 2019;41(7). https://doi.org/10.1080/0142159X.2019.1584276
- Hall AK, Woods R, Frank JR. Changing the culture of residency training through faculty development – ERRATUM. CJEM. 2019;21(6). https://doi.org/10.1017/cem.2019.419