CHAPTER 7 Evidence-Based Performance Measurement

• All physicians have a moral and ethical obligation to act professionally as a fiduciary agent for the health of their patients by engaging in quality improvement.

• Physician performance measurement can be used for research, medical error reduction, patient safety, certification, credentialing, or licensing and disciplining. The purpose for which a measure is intended will determine how the measure is created.

• A performance measure is an equation or fraction representing the frequency of an appropriate and recommended intervention. It has a denominator of those who require a recommended intervention, a numerator of those who received the intervention, and defined exclusions or risk adjustments.

• Health services research demonstrating quality gaps and rising health care costs is fueling demand for clinical performance measures.

• Quality can be measured by assessing clinical outcomes, processes of care, capacity and structure, administrative parameters, cost and efficiency, and patient experience.

• Physician measurement should be founded on evidence-based guidelines, relevant and reliable data, and best clinical expertise.

• Physicians should educate themselves about the various perspectives of other stakeholders and their roles in supporting quality care.

Motivation for Physician Performance Measurement

Over the past decade, professional, political, and societal interest in measuring quality in health care has exploded to be the most dominant theme in medicine, integrating clinical care, physician education, research, and health policy. One of the most dramatic changes in the clinical environment is the emphasis on individual physician documentation of quality in daily practice. The striking increase in demand for quality initiatives and measuring the outcomes of health care delivery is motivated by multiple events and conditions. Most prominent among these is the sustained rapid increase in health care costs (at five times the average rate of inflation) and the observation from health services research that, despite spending more per capita than any other nation in the world on health care, the United States lags in many areas of public health and wellness measures.¹^–⁵

Current economic and market forces incentivize and pay for volume and intensity of service. Yet research shows that higher volume and intensity of health care services do not lead to better aggregated quality of life or public health. Overall, only about half of all Americans receive the recommended health interventions identified by consensus standards of care. And even more striking than the low overall proportion of those receiving recommended care is the wide variation that exists across conditions, races, gender, and financial status.⁶^–⁸ Dissatisfaction with the health care system is higher in the United States than in parallel Western nations. Likewise, the percentage of U.S. citizens who did not get health care because of cost constraints is higher than in many other Western nations. There is a huge gap between the consensus recommended appropriate care and the care that is actually delivered for easily identifiable and definable conditions.³ Large geographic variations in care, unexplainable by patient demographics and characteristics, are easily observable over a broad range of conditions. These geographic variations are far more significant than even the health care disparities seen due to ethnic or health literacy differences in the population.⁶^–¹¹ Unacceptably high rates of mortality and morbidity related to medical error have been the subject of many reports from federal agencies and independent health services researchers.^2,^6,¹² And finally, there does not seem to be any correlation between per capita costs of health care and quality of health care delivered on a range of observations and bases.¹⁰

What Is Quality and Who Defines It?

As quality improvement and physician performance have taken center stage, the question arises as to who will define quality and its measurement. Each stakeholder has a reasonable perspective for viewing quality differently, including the patient and the physician.^1,³ A patient might define quality as the relief of a symptom, the perception of a cure, or an improvement in lifestyle. However, the physician might define quality as the achievement of a particular desired or expected medical or surgical outcome. An employer may see quality as a return on investment for premiums paid, reduced liability for injury, and a workforce that is healthy, productive, and present in the workplace. A health plan purchaser may look at global health outcomes and the need to spread vast resources over large populations with competing needs. Therefore defining quality and deciding exactly what to measure to determine whether quality is being delivered continues to be debated.

In oversimplified terms, most measures of clinical quality or performance today fall into the following categories:

• Outcomes measures

• Process measures

• Capacity and structure measures

• Administrative measures

• Cost and efficiency measures

• Patient experience or satisfaction measures

Argument continues over choosing what to measure. Purchasers of health care have access to voluminous claims and economic data, making administrative, cost effectiveness, and capacity measures attractive.^13,¹⁴ Although many physicians opposed to administrative or efficiency measures clamor for outcomes measures as the only valid assessment of physician performance, in truth, physicians rarely have complete control over all the factors that determine medical outcomes. Additionally, for valid outcome measures, effective risk adjustment needs to occur to reflect differences in the case mix of the patients served; this is often neglected, which results in misleading outcome data. Process measures are easier to define and are more attributable to the practitioner. However, focusing primarily on processes of care can be deceptive when no one takes responsibility for the final outcome. Levels of evidence for different types of interventions can vary greatly, especially when comparing chronic medical care that may involve primarily medication management to acute surgical care for which randomized, double-blind, controlled studies may not exist or even be feasible. Because elements such as availability of support services and tertiary care, patient compliance, comorbidities, ethnic and religious practices, and preferences can all influence the assessment of any medical outcomes, measuring performance attributable to and under the control of the physicians being measured must be a common basic theme if fairness and true patient-centered quality improvement are to be achieved.^13,¹⁵

The purpose for which measures are developed has a powerful influence on how those measures look and what kinds of measures are employed. Among other reasons, performance is measured today for the following overlapping purposes:

• Researching, developing, and improving the effectiveness of an intervention

• Reducing medical error

• Improving patient safety

• Certifying achievement—meeting standards for maintaining board certification or privileges

• Credentialing or accreditation—documenting competence or proficiency for payment or inclusion in a plan, group, or tier

• Licensing and disciplining—identifying, limiting, and punishing poor performance

There is both overlap and synergy between these categories, as well as distinct subcategories further separating these types of measures. The AMA House of Delegates recently addressed criteria and standards for acceptable elements of any pay-for-performance system. As the quality movement matures and value-based purchasing takes different forms, such criteria will become increasingly important to ensure patient-centeredness.

The Process of Building a Coherent System of Performance Measurement

There are three basic principles that must underscore the role of physicians and their organizations in addressing performance and quality improvement.

• First, it is essential that practicing physicians, not just methodologists and health policy scientists, actively and formally engage in prioritizing, developing, field testing, and implementing quality initiatives and performance measures.

• Second, demand for quality and its definition and measurement must be aggregated.

• Third, physicians and their organizations must be unified in their response to this demand.

Engaging in Performance Measures Development

As can be seen from even a cursory scan of the practice environment during the first part of the twenty-first century, every stakeholder group is placing powerful impetus behind defining quality improvement and implementing measurement primarily motivated by the desire to improve efficiency in utilization of resources, reduce medical error, advance patient safety, address inequity and maldistribution of health care, and control a national and global crisis of escalating health care costs.^1,^14,¹⁶ As described in more detail later, failure of physicians to engage and ensure that any definition of quality or program for improvement is truly based on scientific evidence and is relevant and valid to improving patient health outcomes will allow proprietary measurement to focus solely on administration, capacity, and cost. Although these are legitimate concerns, physicians must insist on keeping the focus on improving the patient’s health, not on driving profitability for purchasers of health care.¹⁷

Aggregating Demand for Performance Measurement

With so many organizations involved in quality initiatives and the development of measures, one of the largest concerns that physicians have is trying to “aggregate the demand” for measures—in other words, to make sure that payers, purchasers, licensing and certification processes, and quality improvement organizations are all asking for the same measures, that the measures are based on solid evidence, and that they are focused on similar quality improvement goals. Imagine an environment in which each independent insurance carrier, every major employer who pays premiums for employee health care benefits, government purchasers such as Medicare or state Medicaid contractors, state licensing boards, and specialty certifying boards all developed and required a physician to gather and report data on quality through a separate mechanism and with differing goals and purposes. The administrative quagmire would be a nightmare and virtually impossible to navigate, even for large clinics and groups with sophisticated electronic medical record systems and resources. The ideal situation would be to standardize data points, create a single or simplified set of measures for a given clinical condition or intervention, and establish agreement among stakeholders to accept a unified process of measuring quality.

Unifying the Response to the Demand for Performance Measurement

The medical profession is not homogeneous. Specialists of varying backgrounds, training, and experience may treat similar conditions and bring diverse perspectives to their delivery of health care services. Undesired variation in health care and its outcomes is one of the hallmarks of poor quality.^3,^7,¹² It is not in the patient’s or society’s best interest to have varying processes and quality measures for a given clinical condition coming from competing specialties or groups. For example, pediatricians, family physicians, otolaryngologists, emergency physicians, and infectious disease specialists could all develop and implement competing guidelines and performance measures for treating otitis media based on limited perspective and with varying data points and recommendations. This fosters unhealthy competition and turf battles, and is unlikely to improve quality care. By engaging in multidisciplinary work groups, definitions can be standardized, best evidence can be reviewed and analyzed, scope and purpose of measurement can be agreed upon, learning can take place, and acceptable guidelines and measures can be developed that all physicians who treat otitis media, regardless of specialty perspective, can use to improve their clinical care. Creating evidence-based guidelines and performance measures is labor intensive and costly. By collaborating in a multidisciplinary fashion, waste of resources from competing and parallel development processes can be avoided.

In designing a validated, relevant, and attributable system of measurement, the following process is useful and involves the best combination of rigor and scientific foundation with practical implementation at the physician-patient level.

• Identify and prioritize areas that can be measured and improved

• Develop or identify the best evidence or guidelines for clinical care

• Develop specific performance measures that are physician attributable, implementable, effective, practical, and affordable (See Institute of Medicine domains for quality care: effective, efficient, equitable, timely, safe, patient-centered).¹²

• Use validated, relevant, patient-oriented performance measures in systems and populations for credentialing, licensing, certifying, and documenting competence.^18,¹⁹

Quality-Based or Value-Based Purchasing

Currently there is a sea change in the manner in which physician services are recognized, reported, and remunerated. The traditional system of paying for volume and intensity of care is being replaced by quality-based or value-based purchasing of health care services.¹⁴ In this discussion, the popular term “pay for performance” is not used. Performance can apply to any required activity, regardless of whether it leads to better quality or health outcomes. Focus should remain on delivering high-quality patient care, not on some arbitrarily contractually required performance. The term “pay for performance” currently is highly politicized and is not well defined; it routinely means widely different things to different people. It will likely be replaced in a few years with other descriptors, but the underlying concept of recognizing and rewarding excellence (quality-based purchasing) is likely to be a longstanding foundation for future models of paying for health care.

The concept of rewarding excellence is based not only on the desire for improved quality, but also on the premise that poor quality care is more expensive than high quality care.^7,^10,^20,²¹ Although certainly debatable, there is evidence to support this contention in specific areas. Intuitively, healthy populations consume fewer health care interventions and cost less than sick ones. So improving public health, encouraging healthier lifestyles, and employing effective preventive medical interventions all make sense. Schematically, many discussions have linked the issue of quality to cost by relegating poor quality medical care into three categories:

• Too little care

• Too much care

• The wrong kind of care

It is clear that if poor quality in medical outcomes is the result of unneeded care, then reducing the overuse of services and improving quality would reduce health care costs. It is also not hard to believe that if the wrong kind of care is replaced by the most effective care, then costs could also be reduced. It is less obvious that correcting the problem of too little care or providing more timely care would also reduce costs. But global statistics from developed nations with better public health, preventive health, and healthy behavior or lifestyle systems that have demonstrably superior public health outcomes (e.g., infant mortality, longevity, chronic disease management) when compared with the United States at a much lower per capita cost are suggestive.^22,²³

As a result of this premise, purchasers of health care are implementing strategies that reward better medical outcomes and improved effectiveness and efficiency of care. The forces behind these strategies include the dramatic increases of health care costs in the United States at a rate more than five times the annual inflation rate, evidence that higher intensity and volume of services do not lead to better outcomes, and the increasing development and acceptance of standards for organizing and implementing quality initiatives.⁴

Medical Professionalism—The Physician-Patient Relationship

At the core of physician performance measurement is the ethical and moral obligation of all physicians to practice according to the highest standards. This is true globally, and the issues and discussions of evidence-based practice, quality improvement, and physician performance measurement are universal. The difference between a “profession” and a “trade” has often been defined by the fiduciary responsibility of the professional to act in the best interest of the public or the receiver of the service, rather than in the personal self-interest of the provider.^17,^24,²⁵ In medicine, in addition to legal requirements, virtually every association or physician group has an ethical code or stated commitment to act in the best interest of the patient.²⁶

Medical professionalism is defined as a set of values, behaviors, and relationships responsible for public trust in physicians.²⁵ In the absence of sustained physician leadership in addressing quality, this trust is in serious danger of erosion. Many decry the commercialization of health care over the last century as a repudiation of the tradition of “doing good” in exchange for “making a profit.” The ancient tradition of the covenant between the physician and patient has become a contract between physician and intermediary, creating a split loyalty to both the patient and the organization that contracts on behalf of the patient whose motive is cost containment and profit for shareholders.¹⁷ As a result of the observation of this decline, a statement of medical professionalism has recently been developed jointly by The American College of Physicians–American Society of Internal Medicine Foundation, the American Board of Internal Medicine, and the European Federation of Internal Medicine that has been endorsed by many major physician associations in the United States, including the American Academy of Otolaryngology–Head and Neck Surgery (AAOHNS).²⁷

The ethical basis of medical care is called into question as a result of the recent well-documented health services research focused on irrefutable data that undesirable variations in physician practice and in clinical outcomes are not explained by patient factors, but are due to failure of medical practitioners and systems of care that do not incorporate the latest and best evidence or practices in health care delivery. This has stirred extensive political, public, and private debate. Because health care delivery in most parts of the Western world is still highly individual, reliance on individual physician judgment remains dominant. Unequivocal data show unwarranted and unexplained variations from recommended care on a regular basis across all disciplines.^3,^4,⁷ Physician judgment is being challenged as population and systems studies suggest that patients are being harmed because best practices are not being observed. Personal physician accountability is an increasing focus.^3,²⁸ Because the practice of medicine is so individualized and often unpredictable, and because a past tradition of unchallenged reliance on physician judgment exists, doctors are vulnerable to the charge that their decisions are not transparent, nor are they accountable for the global results of all they do. By combining the best available evidence for treating a condition with the physician’s judgment and patient preferences, more optimal care can be achieved.^4,^25,^29,³⁰

Professionalism and the ethical and moral imperative for physicians to put their patients first should be the major driving force for physician performance measurement. This needs to be done in conjunction with the creation and application of systems and processes to eliminate the opportunity for error, identify error before it impacts patients, mitigate the effects of error, and thus improve patient safety and outcomes.^20,²¹ Physicians will need to work in concert with all elements of health care delivery systems to accomplish this. Employers, governments, and other purchasers, contractors, administrators, and managers of health care have a reciprocal duty to help create the organizational capacity and infrastructure to support physicians in providing optimal care and fulfilling their ethical obligation to the patient. Optimal health care implies both organizational and clinical excellence. In order for a physician to maintain professionalism in our current environment, there must be a shared commitment and collaboration with the patient, fellow professionals, and the institution or system within which health care is provided, but only to the extent that all elements of the system support patients’ interests first.^13,^15,^25,³¹^–³³

Stakeholder Roles in Defining and Implementing Quality Improvement and Measurement Activity

To understand the landscape of physician performance measurement, one must first identify and understand the roles of key stakeholders in the public health care arena, as well as their confluent and sometimes competing perspectives. Current demand for measuring physician performance is driven by patients and public interest groups, as well as by physicians and their associations. In addition to physicians and their patients, many other groups have a legitimate and often powerful stake in measuring outcomes and performance. These stakeholders include physician educators and academic institutions, certifying boards and bodies, agencies whose missions revolve around quality, public and private purchasers of health care services (e.g., federal and state governments, employers, and private insurers), hospitals and health care systems, outpatient clinics and free-standing procedural centers, public interest groups, and many group collaboratives and agencies of all of these. Table 7-1 outlines some of the major stakeholder groups and participants in defining and advancing quality in health care delivery.^31,³⁴

Table 7-1 Stakeholder Groups *

Stakeholder Group	Examples
Government purchasers and agencies	CMS—Medicare, Medicaid
	QIOs
	AHRQ
	VA
	DoD
Private purchasers of health care and their collaborations	Health plans and insurance companies
	Employers
	Private group and independently rated insurance plans
	AHIP
	The Leapfrog Group
	National Business Group on Health
	Pacific Business Group on Health
Licensing, certifying, and educational oversight bodies	FSMB and state licensing boards
	ABMS and professional certifying boards
	ACGME
	ACCME
	AAMC
	ABMS
Private health quality agencies	NCQA
Private health quality agencies	The Joint Commission
Physician societies	National, state, and county medical associations
	National specialty societies
Academic institutions	Medical schools
	Residency training programs
	Allied health training programs
Collaborative organizations of many stakeholder groups	National Quality Forum
	AQA
	AMA-PCPI (the Consortium)
	HQA

* This is not a comprehensive list but shows examples of some of the largest or most influential stakeholders. For more detail, see Table 7-2.

Interface between Physician Education and Performance Measurement

Historically, practicing physicians have augmented their skills and education by engaging in continuing medical education (CME). There is currently tremendous interest in how medical education processes can be improved to advance quality in health care and institute behaviors early in training and careers that will serve physicians and their patients throughout a professional lifetime.^35,³⁶ The most valued and acceptable form of ongoing postresidency education has been formalized by standards developed and monitored by the Accreditation Council for Continuing Medical Education (ACCME). The ACCME is made up of seven member organizations: the American Board of Medical Specialties, the American Hospital Association, the American Medical Association, the Association for Hospital Medical Education, the Association of American Medical Colleges, the Council of Medical Specialty Societies, and the Federation of State Medical Boards. Examples of the types of organizations that are evaluated and accredited by ACCME include hospitals, universities, medical associations, and proprietary medical education providers.

ACCME assists its accredited providers and shares their goal to ensure that physician education is relevant, valid, and free from bias or commercial influence in order to maintain the focus on professionalism and patient-centered care. The ACCME accomplishes this by establishing and promoting standards for physician education, by accrediting organizations that provide educational activities for physicians, and by monitoring and ensuring compliance. The issues related to quality and physician performance measurement are linked more tightly than ever before to the activities surrounding medical education, the care of patients, and the health policy and political activity related to our economy.

Traditional methods of physician CME have been mostly didactic, with minimal emphasis on other multifaceted activities, interactive media, hands-on learning, and demonstrations. Physicians have also been charged with assessing their own performance, learning, and application of new material. However, until now there has been no effective method to ensure that what is being learned is actually changing physician behavior for the better or improving patient outcomes. In fact, studies have shown that physicians are poor assessors of their own knowledge and competence. Health policy and clinical research have suggested that traditional CME has little impact on improving health outcomes or changing performance. Such research also suggests a significant difference in improving physician practice when active interventions such as reminders, patient-mediated interventions, outreach visits, and audits of performance are used with a focus on measuring outcomes.³⁷^–³⁹

As a result of this evidence, the ACCME is continually expanding its requirements for accreditation of CME providers to include documentation that what is being learned will actually change behavior and lead to physician implementation and performance improvement. How that is to be measured and reported is an ongoing challenge.