Bridging the continuum: Analysis of the alignment of undergraduate and postgraduate accreditation standards.

Abstract Several influential national/international bodies including The Bologna Accord, The Carnegie Foundation and The Future of Medical Education in Canada (FMEC) have called for increased coordination across the medical education continuum. FMEC recognizes accreditation as a “powerful lever” and encourages the alignment of undergraduate and postgraduate standards. The Carnegie Foundation includes a similar call for the creation of a more coherent accreditation system. As a first step, using the Canadian context, we present a methodological approach that assesses the example of how well LCME/CACMS undergraduate accreditation standards align with the Royal College of Physician and Surgeons of Canada (RCPSC) postgraduate training standards. We analyzed how closely the 132 LCME/CACMS Medical School accreditation standards aligned with the 155 post-graduate standards from the RCPSC accreditation General Standards (A and B). This comparative evaluation demonstrates that the standards do not align closely. Gaps, redundancies and key differences are highlighted. These results are the first step in understanding how accreditation needs to be adapted and re-aligned across the education continuum to provide consistent and coordinated training and these methods could easily be applied to other contexts and jurisdictions.


Introduction
Several influential national/international bodies including The Bologna Accord, The Carnegie Foundation and The Future of Medical Education in Canada (FMEC) have called for increased coordination across the medical education continuum (The European Higher Education Area 1999; Cooke et al. 2010; Association of Faculties of Medicine of Canada 2012a, b). FMEC recognizes accreditation as a ''powerful lever'' and encourages the alignment of undergraduate and postgraduate standards (Association of Faculties of Medicine of Canada 2012a). The Carnegie Foundation includes a similar call ''that accrediting, certifying and licensing bodies together develop a coherent framework for the continuum of medical education and establish effective mechanisms to coordinate standards and resolve jurisdictional conflicts'' (Cooke et al. 2010). The first step in this endeavor is to assess how well undergraduate and postgraduate accreditations standards align.
Accreditation of professional education programs at academic institutions has been adopted as a widely accepted and valid process by which designated authorities review and evaluate a program using a specified set of standards to ensure that the minimum standards are met with the purpose to ensure and improve the quality of the program. Accreditation is an important process to determine if a program has (a) appropriate purposes (b) has the organization and resources to accomplish its purposes (c) can demonstrate that it is accomplishing its purposes and (d) gives reason to believe it will continue to deliver its purposes (WHO-WFME Task Force on Accreditation 2004).

Practice points
While undergraduate (UME) and post-graduate (PME) medical education accreditation standards are focused on different areas of concern, the premises behind the continuum of medical training should suggest that thoughtful alignment of standards would improve medical education. A similar structure and numbering system between accreditation systems would facilitate alignment and provide consistent language among faculty and students to improve program coherence. The methods presented could be applied to many other jurisdictions interested in evaluating alignment of accreditation standards across the continuum of medical education in their context. In this example we find less than 50% alignment between UME and PME accreditation standards. There is a lot of work required to achieve more appropriate alignment in UME and PME accreditation standards.

Alignment of educational standards between countries
The move towards the international alignment of educational standards is a recent worldwide phenomenon. The Bologna Accord was initiated to make academic degree standards and quality assurance standards more comparable throughout Europe (The European Higher Education Area 1999). The Association of Medical Schools in Europe (AMSE) endorses this process of harmonization between schools in different countries to benefit from the new age of globalization (ASME (Association of Medical Schools in Europe) 2010). Medical education is no longer just a national concern as unification between countries will promote easily readable and comparable degrees, promote mobility for students, researchers and professors, improve employability, and promote collaboration and lifelong learning (Patrício & Harden 2010).
One recent study assessed the importance of 150 medical education accreditation standards in 13 different countries (van Zanten et al. 2012). This survey-based analysis identified the relative and absolute importance of individual standards used by various undergraduate medical education (UME) accreditation agencies around the world. Variability in the answers ranged from unanimous agreement to moderate disagreement in ranking the importance of standards, demonstrating some global variation in opinion on importance of particular standards. Of the 150 standards assessed in the study, there was unanimous agreement from all respondents on 14 standards deemed essential for determining the quality of the educational program. These essential standards were evenly distributed across the categories of Educational Program, Assessment of Students, Educational Resources, Program Evaluation and Governance and Administration, suggesting there are core concepts that all accrediting systems should incorporate (van Zanten et al. 2012).
There is discussion regarding alignment of accreditation standards of the UME programs worldwide. The World Federation for Medical Education (WFME 2012a) published accreditation standards written in such a way to be adapted by different accrediting bodies throughout the world to account for local, regional and national contexts. In 2004 the World Health Organization (WHO) and the WFME established the International Task Force on Accreditation in Medical Education. A meeting in October 2004 in Copenhagen discussed the purpose of accreditation and quality improvement in medical education in countries around the world, grouped into six different regions. The resulting report demonstrated agreement that the outcomes of accreditation should include: better-integrated curricula; greater focus on student-centered learning; greater focus on teaching communication skills and early clinical contact; more diverse assessment strategies; more professional development; a stronger role for better practice; better use of information technology in courses and more equitable admissions processes (WHO-WFME Task Force on Accreditation 2004). An outcome of the meeting was agreement that WHO and WFME should promote the alignment of global accreditation standards for UME programs worldwide. However, no mention was made to consider integration vertically with post-graduate programs.
A comparison of accreditation standards for post-graduate medical education in the United States compared to in Canada identified many similarities in terms of objectives and requirements (Cassie et al. 1999). However, there are differences in the operation of the two systems. In 1992, The Accreditation Council for Graduate Medical Education (ACGME) and the Royal College of Physicians and Surgeons of Canada (RCPSC) began discussions towards a more cooperative relationship. This has led to an agreement that each organization sends two official observers to observe the review and decision-making operations of the other. This has proven beneficial in improving the understanding and communication between the two systems (Cassie et al. 1999).

Alignment between undergraduate and post-graduate educational standards
The WFME published a Trilogy of Accreditation Standards for Basic Medical Education, Postgraduate Medical Education (PME) and Continuing Professional Development (CPD) in 2012 (World Federation for Medical Education 2012a, b, c). Standards are divided into nine areas across all three training levels showing comparability among accreditation standards (mission and objectives; educational programme; assessment; trainees; faculty/academic staff; educational resources; evaluation; governance and administration and continuous renewal) (Karle & Fenoll-Brunet 2007). However, the WFME is not an accrediting authority or agency so the standards are presented only as guidelines for national agencies to use when determining their own accreditation standards.
We find no published evidence in individual countries regarding alignment between undergraduate and PME programs. Australia has taken a proactive stance in moving forward with their accreditation standards, revising them twice (in 1998 and 2002) to align with the WFME guidelines. However, focus has been on the UME standards alone (WHO-WFME Task Force on Accreditation 2004). Similarly, the Education Committee of the General Medical Council (GMC) in the UK assures that high standards of medical education are upheld and updated regularly; however again, no alignment between post-graduate and graduate education has been reported. In the American model, postgraduate programmes are distributive with accreditation decisions made by each specialty Residency Review Committee (RRC), although the ACGME has institutional requirements that apply independently of the RRCs, therefore making alignment between undergraduate and graduate more complex with 26 specialty committees involved (Cassie et al. 1999).
Alignment between the undergraduate and post-graduate training accreditation standards has been called for by The FMEC: A Collective Vision for MD Education (Association of Faculties of Medicine of Canada 2012a). This document presented 10 recommendations and five levers for enabling change. Enabling Recommendation A is focused on realigning accreditation standards. It states that ''Recognizing that accreditation is a powerful lever, Canadian medical leaders must review and realign existing standards of the Committee Alignment of accreditation standards on Accreditation of Canadian Medical Schools (CACMS) and the Liaison Committee on Medical Education (LCME) and develop new ones, as necessary, to respond to the recommendations in this report. This may involve the alignment of undergraduate and postgraduate accreditation standards''. The Carnegie Foundation's Educating Physicians seven recommendations, includes a similar call for alignment ''that accrediting, certifying and licensing bodies together develop a coherent framework for the continuum of medical education and establish effective mechanisms to coordinate standards and resolve jurisdictional conflicts'' (Cooke et al. 2010).
FEMC is also addressing recommendations for postgraduate medical training (FMEC-PG) and Recommendation 7 from their assessment is entitled ''Align Governance, Standards and Accreditation'' (Association of Faculties of Medicine of Canada 2012a). They suggest that ''governance, standards and accreditation should be aligned across the learning continuum, designed within a social accountability framework and focused on meeting the healthcare needs of Canadians. This requires collaborative action among the credentialing colleges, the Medical Council of Canada, medical regulatory authorities, hospitals, teaching sites, universities, health authorities, governments, and other stakeholders, beginning with the MD program and continuing into professional practice''. Actions include refocusing accreditation standards to align with delivering health outcomes to meet societal needs.
It has been suggested by FMEC-PG (Association of Faculties of Medicine of Canada 2012b) and Educating Physicians (Cooke et al. 2010) that alignment of undergraduate and PME accreditation standards would increase coherence for faculty and students throughout training, which would have an obvious effect on improving the educational programs. It would be beneficial for faculty and students to have a consistent language throughout their training to improve clarity within the educational programs. Furthermore, it could reduce the burden on the schools as preparing for accreditation is an enormous undertaking for both the UME and PME accreditation, and similarities between the two assessments would make preparations easier. It is often the same faculty who are teaching in the UME and PME programs and thus if the alignment of standards across these two programs improves the continuum of learning for students, life-long learning for faculty (faculty development) and the delivery of program content by faculty, it is a worthy goal.
We will use the Canadian context as an example to demonstrate a model for establishing vertical alignment of UME and PME accreditation standards. The first step in this endeavor, in the Canadian context, is to assess how well the LCME and CACMS undergraduate accreditations standards align with the RCPSC. We have undertaken such an alignment and present the results of this analysis below. The authors of this study had differing hypotheses as to what the level of alignment would be like. Two of the authors hypothesized that the level of alignment would be approximately 75% whereas the third author expected a much lower degree of alignment of approximately 30-40%. This was the impetus for our study.
The result of alignment could make any country that pursues it in a meaningful way a leader in accreditation progression and alignment and is likely to provide multiple benefits to the medical education system. This methodology could be undertaken in many countries based on the model presented. However, all authors agreed that ideal levels of alignment have not yet been determined.

Methodology
As of May 2013, there were 132 standards in the LCME ''Functions and Structure of a Medical School'' (Liaison Committee on Medical Education 2012) that define the standards for UME education. IS-3 does not apply to Canadian schools and was removed from the analysis. Beyond IS-3, we chose not to embark on value judgment regarding suitability of standards for inclusion in the analysis. For this reason we chose to use the broad denominator of all standards applicable to the Canadian context as a first step in the complex task of assessing alignment across the continuum. The RCPSC General Standards for postgraduate education as of November 2012 (Royal College of Physicians and Surgeons of Canada 2011 (revised 2013)) are numbered to aid in alignment for a total of 155 standards (see supplemental information, Appendix 1). The general standards are divided into the ''A standards'' which are standards applicable to the university and affiliated sites and the ''B standards'' relating to all residency programs.
Alignment is based on the list of standards generated by LCME and RCPSC, which do not use the same unit in defining standards. LCME has standards that may include annotations that provide additional requirements but are considered all one standard. RCPSC uses a nested hierarchy and lower level statements sometimes indicate additional standards or sometimes provide additional detail to the parent standard. However, most of the RCPSC standards as we identified them include additional information that aligns to unique standards justifying why we chose to use each numbered RCPSC standard for our alignment. For example, A1-3 aligns with ED-33, whereas nested standards A1-3.4 aligns with ED-36, A1-3.5 aligns with MS-3, A1-3.6 aligns with MS-34, etc. So as an initial approach, and for a visibly obvious numbering system, we decided to use each numbered RCPSC standard for a total of 155 standards for alignment. The authors acknowledge the complexities of defining a standard and that future reorganization of standards by LCME may provide more insight into alignment studies such as this one.
The RCPSC standards were aligned with the LCME/CACMS standards by assessing how similar two standards were to each other. Our definition of alignment was that two standards had to use similar wording and address the same issue directly when viewed from the lens of an external end user and that in order to count as aligned they had to meet this criteria by two independent reviewers. Alignment was done by SEA and confirmed independently by AO with differences resolved through discussion. After alignment, the percentage of standards within each grouping that were similar in each sub-grouping was determined.

Results
The LCME/CACMS standards are grouped into five groups and 24 sub-groups. The RCPSC have divided the General Standards A and B into nine groups in total. It was hypothesized by two of the three authors that the standards within each of the 24 LCME/CACMS sub-groups would closely align with the standards within the nine RCPSC groups. However, the standards within each grouping from each organization had more differences than expected and as each organization had unique standards, it was difficult to identify patterns, similarities and differences (see supplemental information, Appendix 1).
A second analysis was undertaken by analyzing the five sub-standard groupings from LCME/CACMS with the nine RCPSC groupings. This allowed us to better identify subregions of similarities and areas of differences by assessing the standards under the sub-groupings. Many standards such as the RCPSC University Structure standards fall under several (i.e. three) different LCME/CACMS subgroups of standards (Table 1). Likewise, many of the LCME/CACMS standards fall under several RCPSC subgroups of standards (i.e. LCME/ CACMS Teaching and Evaluation standards fall under five different RCPSC sub-groupings) (Table 1).
Often, two or more RCPSC standards aligned to the same LCME/CACMS standard. Similarly, two LCME/CACMS standards sometimes aligned to a single RCPSC standard (see supplemental information, Appendix 1). For example, there are 14 LCME/CACMS Educational Resources (ER) standards. Eight RCPSC standards align to five ER standards, as several of the RCPSC standards align to a single LCME/CACMS standard (Table 1 and supplementary information, Appendix 1). Therefore, the total number of standards aligning differs between the two organizations. Overall, 78 of the 155 (50%) of the RCPSC standards aligned to LCME and 57 of the 131 (44%) of the LCME/CACMS standards aligned to RCPSC (Tables 2 and 3).
There is a wide range of alignment between RCPSC and LCME/CACMS subcategories (from 21% to 71%) (Tables 2 and 3). The LCME/CACMS Institutional Setting standards and the Medical Student standards both have the lowest degree of alignment (21%). The RCPSC B1-Administrative structure standards demonstrate the highest level (71%) of alignment (Tables 1 and 2).

Discussion
Accreditation of undergraduate, post-graduate and continuing medical education is well established in most countries throughout the world and although standards differ from country to country, the overall goal of improving medical education is at the heart of all systems. To ease transitions between the three different stages of medical education (UME, PME and CME), uniformity between the accreditation standards would contribute to a clearer educational path, resulting in improved medical education for physicians. The notion that this process would contribute to an improved medical education experience for both providers and learners is further supported by the recent Association of Faculties of Medicine of Canada Report on The Future of Medical Education which has identified a need to ensure effective integration and transition along the educational continuum (Association of Faculties of Medicine of Canada 2012b). In particularly, they note that all transitions are key opportunities for learning that must be managed and used more effectively.
To our knowledge, this is the first time a formal approach has been presented as a model of how accreditation standards can be compared longitudinally between undergraduate and post-graduate training. Our study underscores the differences between the undergraduate and post-graduate education systems as assessed by comparison of the accreditation standards.
While there are LCME/CACMS standards that are unique to the undergraduate experience, there are examples of standards that are found only in LCME/CACMS that actually do also pertain to the postgraduate experience and are not found in the RCPSC standards. For example, ED-22 requires the undergraduate program to create learning opportunities that enhance awareness of personal biases and health disparities. There is no equivalent postgraduate standard and yet the process of self-awareness and inquiry should continue in the post-graduate training. We found that the LCME/CACMS Institutional Setting (IS) and the Medical Student (MS) standards had the lowest degree of alignment (both with 21% alignment) with the RCPSC standards (Tables 1 and 3). Many of the LCME/CACMS IS standards are concerned with the organization of the University and the governing board, and governance by the dean of the medical school. The RCPSC standards do not focus on this level of upper management.
From Tables 1-3, several observations can be made highlighting the differences in nature of the standards between the RCPSC and LCME/CACMS standards. The RCPSC standards on ''Content of Program'' have a low alignment (42%) with LCME/CACMS standards. However, the LCME/CACMS standards on ''Education Programs'', which includes content, have a higher percentage of alignment to RCPSC standards (67%) yet many of the alignments are to RCPSC standards outside of ''Content of Program'' suggesting the organization of the educational content standards for the RCPSC and LCME/ CACMS is different.
The LCME/CACMS Medical Student (MS) standards show a low alignment (21%), reflecting greater concern for the student at the earlier level of training reflected by LCME compared to the RCPSC standards. Many of the LCME/CACMS standards relate to management of visiting and transfer students, student indebtedness, financial counselling, health care, and other issues not mentioned in the RCPSC standards. Eleven of the 131 LCME/CACMS standards (8%) concern admission of the medical student whereas only one of the 155 RCPSC standards (0.6%) pertain to admissions into the residency program.
The LCME/CACMS standards are more focused on protecting trainee interests than the RCPSC standards, reflecting the different stages of the trainee from junior to senior.   Other differences appear in the nature of the standards between the RCPSC and LCME/CACMS standards. The RCPSC standards are more concerned with policies to protect and preserve the best interests of the patients than LCME/ CACMS. These differences may reflect the different stages of training as being a full-time student learner with no source of income and no direct patient-care responsibilities, differs from being a full-time employee/learner with substantial patient care responsibilities. However, they may also reflect differences that are particular to the agencies as the well-being of patients and care providers are relevant to all stages of training.
In the early 1990s the RCPSC developed an innovative, competency-based framework that describes the core knowledge, skills and abilities of specialist physicians that is known as the CanMEDS Physician Competency Framework. CanMEDS is an educational framework identifying and describing seven roles that lead to optimal health and health care outcomes: medical expert, communicator, collaborator, manager, health advocate, scholar and professional. The goal of CanMEDS is to improve patient care. This model has been adapted around the world in medicine and other health professions. The RCPSC accreditation standards have integrated the CanMEDS competencies. To date, the LCME/CACMS standards have not been formally linked to competencies. CanMEDS accounts for several of the differences between the RCPSC and LCME/CACMS standards.
The RCPSC Standard ''B5'' Clinical, Academic and Scholarly Content of the Program identifies the seven CanMEDS roles residents must fulfill: Medical Expert, Collaborator, Communicator, Manager, Health Advocate, Scholar and Professional. Some competencies differ from, or are not required by the LCME/CACMS standards. For example: Collaborator: RCPSC is concerned with developing residents' skills in collaboration (B5.3) but this is not explicitly mentioned in the LCME/CACMS.
Manager: RCPSC concerned with development of management skills for running a medical practice and concerned with contributions in health care organizations (B5.4.3; B5.4.4). The concerns covered by these RCPSC standards are not covered by LCME/CACMS. Manager: RCPSC is concerned that residents can demonstrate effective allocation of health care resources (B5.4.2). This competency is not covered by LCME/ CACMS. Professional: The RCPSC stresses the teaching of relevant legislation and regulations (B5.7.3). This competency is not covered by LCME/CACMS. We found that the LCME/CACMS standards are more detailed in wording whereas the RCPSC standards are broader in scope and tone. We hypothesize that the reason the language used in the RCPSC standards is different from that used in the LCME/CACMS standards may be due to the fact that the RCPSC language must be adaptable towards over 659 different programs they oversee. The LCME/CACMS standards apply to only 141 schools in the US and 17 schools in Canada with the same outcome of graduation with an MD degree.
Several RCPSC standards not included in the LCME/CACMS standards relate to ensuring quality of hospital sites and programs and could be adapted by LCME/CACMS. For example, standard A1-3.3 suggests internal review of each residency program and standard A2-5 states that all sites must participate in continuous quality improvement (CQI). Standard A2-8 states that sites eligible for accreditation by Accreditation Canada must be so accredited and A2-9 states that university internal reviews must ensure that all sites used by programs are appropriate. One RCPSC standard states that there must be opportunities for residents to acquire knowledge to prevent and handle both expected and unexpected adverse patient events that are not part of the LCME/CACMS requirements. Standard B5-6.1 is concerned with opportunities for residents  to acquire skills in effective teaching. The LCME/CACMS standards do not discuss teaching medical students how to teach at that level of their training. One limitation of this study is that the organization of the LCME and the RCPSC standards is very different and does not lend itself readily to alignment. We expect this is likely to be the case in other jurisdictions but further study would be needed. This was underscored by the fact that many standards such as the RCPSC University Structure standards fell under several (i.e. five) different LCME/CACMS subgroups of standards (Table 1). Similarly, many of the LCME/CACMS standards fell under several RCPSC subgroups of standards (i.e. LCME/CACMS Teaching and Evaluation standards fell under five different RCPSC sub-groupings) ( Table 1). Another limitation is that the standards for both systems are always being updated. This study was carried out on the current standards available as of June 2013. The LCME standards are currently being reorganized into clusters and a reduced number of standards and elements (94 in total) for use in survey visits occurring in 2015/16. The current annotations included in the LCME standards which could confuse later alignment analyses are being eliminated. The restructuring into 12 standards and 94 elements that will go into use in July 2014 would not significantly alter the conclusions of this study because the restructuring did not change the intent of the standards.
The LCME/CACMS and RCPSC systems differ in outlook as the RCPSC accreditation standards must be relevant for a large number of programs whereas the LCME accreditation standards related to program content are strict to ensure comparability and competency between physicians graduating from different schools. Although many standards aligned indicating the similarities of the standards, the level of alignment was lower than two of the authors predicted and the standards did not align as predicted. Most RCPSC standards aligned with standards across all five LCME/CACMS categories. A similar pattern of LCME/CACMs standards aligning to standards across many of the nine RCPSC sub-groups underscores the difference in configuration of standards between the two organizations.
The current state demonstrating differing standards between the two accrediting bodies supports a quality assurance model, but not continuous quality improvement along the continuum of training. Chen et al. argue that highquality medical education is at the heart of high-quality patient care delivery (Chen et al. 2004). The corollary of this is that continuous quality improvement should be the desired outcome of all medical education programs if patient care is the measured endpoint. While standards are important tools for quality assurance, an unintended negative consequence may be a reduced achievement threshold of excellence associated with quality improvement. Harmonization would have to take place at the national level and would involve serious discussions among the accrediting bodies: in the Canadian context, this is the Association of American Medical Colleges (AAMC) and the Association of Faculties of Medicine in Canada (AFMC).
This would take discussion focusing on alignment including input from all stakeholders.

Conclusion
The approach outlined in this paper demonstrates that although both post-graduate and UME accreditation systems are similar in purpose, they differ in practice. We found that in the Canadian example presented, the LCME/CACMS standards for UME overlapped with the RCPSC post-graduate accreditation standards by less than 50%. This is, to our knowledge, the first longitudinal analysis comparing undergraduate and post-graduate accreditation standards. While undergraduate and post-graduate accreditation standards are focused on different areas of concern, the premises behind the continuum of medical training should suggest that alignment of standards would improve medical education. Further discussions between the accrediting organizations that move towards a more like-minded set of standards with improvements in coherence and consistent language, and their effect on improving quality of training is a testable and worthwhile undertaking.
Overall we find that there is a lot of work required to achieve more appropriate alignment in UME and PME accreditation standards. In future, a similar structure and numbering system between accreditation systems would facilitate further studies of this sort. Furthermore, improved alignment would provide consistent language among faculty and students and improve program coherence. It is possible that alignment of undergraduate and postgraduate accreditation standards and processes would also reduce the burden on schools undergoing accreditation, a testable hypothesis. This would take discussion focusing on alignment including input from all stakeholders. Harmonization would have to take place at the national level and would involve serious discussions among the accrediting bodies: in the Canadian context, this is the AAMC and the AFMC. Another benefit of consistency in accreditation would be the increased ease of developing more coherent milestones documents across the continuum.
As the field of aligning accreditation across the continuum is both complex and early in development, there are many areas for future work. First, we recommend that future research could refine the alignment analysis example provided by this study by seeking consensus on which standards should and should not be included in the denominator (i.e. which standards should be considered appropriate to align). Second, it would also be interesting to extend the analysis using the different lenses of looking forward and back through the continuum and moving to a more qualitative discussion of the meaning of the gaps and redundancies. Third, we suggest that future studies evaluate accreditation protocols, methodologies and processes across the continuum. In addition, future research can build upon this study to analyze how well continuing professional development (CPD) requirements align with the undergraduate and postgraduate standards. Improving alignment of undergraduate and graduate accreditation between countries could also assist in the increased globalization and mobility of physicians, improving patient care worldwide.