Over the course of the last decade fundamental changes have occurred in residency training. The basis of these changes has been rooted in the desire to simultaneously improve the quality of the learning experience while decreasing the demands of training on resident lifestyle. The ACGME Outcomes Project was initiated in 1999 with the intent of facilitating such change in medical education. Before its introduction, assessment of residency training sites focused on the processes, resources, and reputation of an individual program, but failed to assess how effectively a program used those assets. The stated goal of the ACGME Outcomes Project has been to drive an evolution of this process-oriented form of education to one that is based on outcomes that measure the effect of the educational experience. This article is a brief overview of current efforts to achieve this goal.
Medical education in the United States has evolved over the course of the last century from that of an apprentice-based system to its current state of organization through a series of stepwise changes. Early organizational efforts implemented by pioneers such as William Halstead and Abraham Flexner (Flexner report, 1910) around the turn of the century revealed the poor state of medical education at the time, and set into motion a series of expectations and resultant improvements that ultimately impacted both medical education and public safety. The current energy surrounding the ACGME Outcomes Project and its impact on transition to a competency-based resident training paradigm are the latest steps in the evolution of medical education.
Brief history of the Accreditation Council for Graduate Medical Education/role of the Residency Review Committees
The Accreditation Council for Graduate Medical Education (ACGME) was established in 1981 to serve as a national governing body charged with accreditation of post-MD residency training programs within the United States. In effect, the fiduciary responsibility of the ACGME is to the public as demonstrated in its mission statement: “We improve health care by assessing and advancing the quality of resident physicians’ education through accreditation.” In its role as a national governing body, the ACGME sets general policy and the direction of resident education. However, recognizing that significant differences exist between specialties, oversight at the specialty level is relegated to 27 specialty-specific committees. It is the responsibility of these Residency Review Committees (RRCs) to periodically review every residency training program to ensure programmatic compliance with the standards that are set forth by the ACGME.
ACGME Outcomes Project
In February 1999, the ACGME endorsed the Outcomes Project, which is funded by the Robert Wood Johnson Foundation and designed to focus on educational outcomes . This project would eventually lead to fundamental changes in the way residents are educated. Accreditation would no longer be granted on the basis of a residency program’s potential to educate trainees as demonstrated by process assessment (process-oriented education). Instead, accreditation would be granted based on a series of measurable outcomes intended to demonstrate an individual resident’s ability to provide effective patient care, as well as the residency program’s ability to provide such training (outcomes-oriented education).
With the Outcomes Project in mind, the ACGME introduced a new model of accreditation in July 2002 that was based on two major requirements. First and foremost, it was assumed that the effectiveness of resident education would be more efficient if alterations were made in resident lifestyle that decreased fatigue. The first requirement, therefore, addressed a reduction of the resident workweek to a maximum of 80 hours. In many cases, this requirement caused a significant decrease in the number of hours logged by residents during training. The resultant impact on workflow, in turn, resulted in major changes to the residency process. Education, communication, and transfer of patient care between health care providers, along with other traditional processes of training programs, were modified, thus leading to the development of novel training strategies. Although it was initially feared that this potential for reduction in resident exposure to patient care would lead to physicians ill equipped to function effectively following graduation , it appears that more effective educational processes and less fatigued residents have largely overcome these concerns . On the other hand, limiting the resident workweek has effectively resulted in measurable decreases in clinical experience during residency training . Resolving the mutual demands of a balancing of reduced patient care exposure with an optimization of educational opportunities has given rise to novel approaches to resident education .
In light of the concerns about the ACGME requirement limiting resident duty hours, the second requirement was appropriately focused on improvements in the quality of the resident educational experience . Under the model proposed by the ACGME, accreditation would be based on a series of outcomes-based measures. This approach has created a need to refine reasonable goals and objectives specific to each program, to develop ways of effectively capturing and optimizing available clinical experiences, to develop a series of reasonable and validated measures of outcome, and to develop processes by which collected data would facilitate improvement in resident and residency program performance. Further, development of the tools necessary to teach trainees to implement a problem-based approach to medical care should serve to develop a culture of lifelong learning.
To date, implementation of the Outcomes Project has been the responsibility of the individual programs, with guidance by the ACGME and the individual RRCs.
Timeline of the ACGME Outcomes Project
The Outcomes Project is a long-term initiative by which the ACGME is increasing emphasis on educational outcomes as the basis for accreditation of residency education programs . In contrast to the current practice of accreditation based on a potential to educate as measured by determining compliance with a set of existing requirements, the goal of the Outcomes Project is to emphasize the actual accomplishments of a training program as assessed by measurable outcomes. Stated differently, it intends to measure how effectively education in each program impacts its trainees’ abilities.
The Outcomes Project has been implemented in phases that were proposed on a strict timeline . In each phase a set of programmatic focuses and accreditation focuses were listed. Under the guidance of the RRC program evaluation process, each program would receive feedback regarding progress.
Phase 1 (July 2001 through June 2002) required that an initial response be formed to the changes in ACGME requirements. Specifically, each program was required to define specific objectives that would allow residents to demonstrate learning in the six general competencies. Additionally, the general competencies were to be integrated into resident didactic and clinical educational experiences.
In Phase 2 (July 2002 through June 2006), the focus and definition of the competencies and assessment tools used to evaluate outcomes were to be sharpened. Learning opportunities in all six of the competency domains were to be demonstrated. The evaluation processes were to be improved as needed by each program, under the guidance of the RRC. In addition, aggregated resident performance data would be accumulated for each program’s Graduate Medical Education (GME) Committee internal review.
In Phase 3 (July 2006 through June 2011), each program is to fully integrate the general competencies and their assessment into both learning and clinical care. Resident performance data are to be used as the basis for improvement and to provide evidence for the program’s accreditation review. In essence, this enables Phase 3 of the implementation of the Outcomes Project to be the point at which competency-based training is to become operational.
Finally, in Phase 4 (July 2011), the general competencies and their assessment are to be expanded to develop models of excellence. In this phase, as in the preceding phases, site review by the RRCs will provide an opportunity to identify and disseminate models of excellence developed within individual programs. In theory, this process will lead to identification of benchmarks that will serve to facilitate continual improvement.
The six general competencies
The first major activity of the Outcomes Project was the development of the six general competencies for residency training, which were endorsed by the ACGME in 1999 . This process of development was stimulated by an introspective assessment of how adequately physicians in the current model of residency training were prepared to practice medicine in the arena of the changing health care delivery system. The general competencies were derived through evaluation of existing research on qualities necessary for physicians to function effectively. Further input was gathered from a variety of GME stakeholders.
The ACGME approved specific language regarding the general competencies and evaluation processes in September 1999 . For the purposes of accuracy, the description of the ACGME general competencies used is that approved by the ACGME . The six general competencies are
- 1.
Patient care
- 2.
Medical knowledge
- 3.
Practice-based learning and improvement
- 4.
Interpersonal and communication skills
- 5.
Professionalism
- 6.
Systems-based practice
Patient care
Residents must provide patient care that is compassionate, appropriate, and effective for the treatment of health problems and the promotion of health. Residents are expected to
- •
Communicate effectively and demonstrate caring and respectful behaviors when interacting with patients and their families
- •
Gather essential and accurate information about their patients
- •
Make informed decisions about diagnostic and therapeutic interventions based on patient information and preferences, up-to-date scientific evidence, and clinical judgment
- •
Develop and carry out patient management plans
- •
Counsel and educate patients and their families
- •
Use information technology to support patient care decisions and patient education
- •
Perform competently all medical and invasive procedures considered essential for the area of practice
- •
Provide health care services aimed at preventing health problems or maintaining health
- •
Work with health care professionals, including those from other disciplines, to provide patient-focused care
Medical knowledge
Residents must demonstrate knowledge about established and evolving biomedical, clinical, and cognate (eg, epidemiological and social-behavioral) sciences and the application of this knowledge to patient care. Residents are expected to
- •
Demonstrate an investigatory and analytic thinking approach to clinical situations
- •
Know and apply the basic and clinically supportive sciences that are appropriate to their discipline
Practice-based learning and improvement
Residents must investigate and evaluate their patient care practices, appraise and assimilate scientific evidence, and improve their patient care practices. Residents are expected to
- •
Analyze practice experience and perform practice-based improvement activities using a systematic methodology
- •
Locate, appraise, and assimilate evidence from scientific studies related to their patients’ health problems
- •
Obtain and use information about their own population of patients and the larger population from which their patients are drawn
- •
Apply knowledge of study designs and statistical methods to the appraisal of clinical studies and other information on diagnostic and therapeutic effectiveness
- •
Use information technology to manage information, access on-line medical information, and support their own education
- •
Facilitate the learning of students and other health care professionals
Interpersonal and communication skills
Residents must demonstrate interpersonal and communication skills that result in effective information exchange and teaming with patients, their patients families, and professional associates. Residents are expected to
- •
Create and sustain a therapeutic and ethically sound relationship with patients
- •
Use effective listening skills and elicit and provide information using effective nonverbal, explanatory, questioning, and writing skills
- •
Work effectively with others as a member or leader of a health care team or other professional group
Professionalism
Residents must demonstrate a commitment to performing professional responsibilities, adherence to ethical principles, and sensitivity to a diverse patient population. Residents are expected to demonstrate
- •
Respect, compassion, and integrity; a responsiveness to the needs of patients and society that supercedes self-interest; accountability to patients, society, and the profession; and a commitment to excellence and ongoing professional development
- •
Commitment to ethical principles pertaining to provision or withholding of clinical care, confidentiality of patient information, informed consent, and business practices
- •
Sensitivity and responsiveness to patients’ culture, age, gender, and disabilities
Systems-based practice
Residents must demonstrate an awareness of and responsiveness to the larger context and system of health care and the ability to effectively call on system resources to provide care that is of optimal value. Residents are expected to
- •
Understand how their patient care and other professional practices affect other health care professionals, the health care organization, and the larger society and how these elements of the system affect their own practice
- •
Know how types of medical practice and delivery systems differ from one another, including methods of controlling health care costs and allocating resources
- •
Practice cost-effective health care and resource allocation that does not compromise quality of care
- •
Advocate for quality patient care and assist patients in dealing with system complexities
- •
Know how to partner with health care managers and health care providers to assess, coordinate, and improve health care and know how these activities can affect system performance
Components of a competency-based education program
In essence, the current Outcomes Project is an example of a competency-based educational model. This model is designed to focus on the performance of the individual trainee, as measured in outcomes, as he or she works to master educational goals and objectives. For this form of education to be effective, a number of elements must be in place. Further, these elements should be integrated in such a way that meaningful data are generated and continually integrated into the educational environment, thus providing a progressive, graduated experience for the learner.
A clear set of educational goals and objectives serves to set the stage for the educational environment and aligns the expectations of the learners with those of the program. Goals and objectives should be specific to the educational site and experience level of the resident, in effect serving to continually challenge and develop trainees’ skill sets. It is important that this set of goals and objectives can be objectively measured to track progress and identify areas that need remediation. Finally, the goals and objectives should attempt to reflect the need to gain experience in all six of the general competencies.
At first glance, the efforts required to transform residency programs as suggested by the ACGME Outcomes Project appeared monumental, but most programs have been able to modify existing didactic and clinical experiences to align with the general competencies required by the ACGME . A recommended approach to help assess programmatic alignment with new ACGME regulations is to address three questions: Do the residents achieve the learning objectives set forth by the program? What evidence can the program provide that it does so? How does the program demonstrate continuous improvement in its educational process? .
Measurement, tracking, and documentation of the goals and objects set forth by each program provide a mechanism by which to ensure effective education. However, this step in the process of educational transition may also present a challenge. Large amounts of data generated by this process will require secure storage and organization if they are to be useful. Many programs have used computer-based systems that can log resident duty hours, track a variety of performance measures, and identify areas of noncompliance/substandard performance. Further, these systems can prepare reports (ie, performance “dashboards”) and alert program directors early in the course of deviations from expected performance . Varying levels of access can be provided to residents, faculty, program directors, and chairmen “chairpersons” (or simply “others”)? to allow for ongoing access to pertinent information.
A variety of assessment tools has been developed and made available as outcome metrics through the ACGME Outcomes Project Toolbox . Table 1 is a partial list of assessment tools that are currently in common use. Each of the listed metrics has been made available for use by residency programs.
