Outcomes research can be defined as the scientific study of the outcomes of disease therapies used for a particular disease, condition, or illness (1). While all clinical research measures some type of outcome, such as mortality, morbidity, or some other objective measure, in “outcomes research,” the patient’s perception of their outcome is assessed.

Historically, the movement toward outcomes-based research was started by Dr. Paul Ellwood, who in the 1980s suggested that in the future physicians would assess outcomes by measuring what the patient experienced (2). Subsequently, tools to assess these outcomes were developed and applied across many diseases. However, outcomes research also includes other types of studies in addition to patient-based outcomes studies. Today, outcomes researchers study all aspects of the health care system—from the status of the patient or population at entry, to the organization, delivery, regulation and financing of the health care system, to the status of the patient or population after treatment.

To be inclusive of other aspects of health services research, some have divided outcomes research into record-based outcomes research and patient-based outcomes research. Examples of these different types of studies are shown in Table 8.1.

Prospective, retrospective, or cross-sectional study designs can be used in outcomes research. However, many outcomes studies use an observational prospective design, where outcomes are assessed after diverse treatments—rather than an experimental prospective design, where treatments are carefully controlled or randomized. The differences between these study designs are an important point for discussion.

In experimental or controlled trials, particularly randomized trials, the ideal design uses two groups of patients that are nearly identical in every aspect—except the treatment received. So therefore, any difference in outcome between groups must be due only to the different treatments, since otherwise the groups were the same. Of course, acquiring patient groups that are actually identical is rarely achieved, but nevertheless, that is the basis behind the rigorous design and methodology of controlled trials. In addition, to account for the inherent differences between treatment groups, the randomization process should theoretically distribute those differences (say, in demographics or disease severity) equally between the two groups. Carefully controlled experimental studies can be said to measure the efficacy of treatment, under ideal circumstances. When nursing students need to analyze complex research methodologies like these for their coursework, many choose to write a nursing paper with BSN writers who specialize in research design and statistical analysis. These studies usually yield very reliable results concerning the effects of treatment—in the group that was studied.

However, there are questions about the generalizability of results from rigorously controlled trials to larger populations of patients with a disease because the larger group of patients will not be so carefully controlled and homogeneous. In addition, patient compliance and other factors may lead to different results than in tightly controlled trials. Furthermore, clinical trials are very expensive to perform, especially when considered from the standpoint of cost per patient studied.

Observational study designs are commonly used in outcomes research. All patients with a disease are included, and they are studied in the actual setting in which they receive their health care. This naturally introduces many other factors, which may impact outcome after treatment, including potential selection bias for different treatments. However, many would argue that results from large-scale outcome studies are more applicable to the general population because of their setting and scope. Studies that assess the actual (“real world”) results of treatment are said to measure the effectiveness of treatment versus efficacy measured from controlled trials (3). Furthermore, in addition to their “real-world” setting, the expanded outcome measured (quality of life, etc.) might be more relevant and important to patients than other clinical or biologic outcomes.

There are several basic steps involved in performing patient-based outcomes research (3,4,5). The fundamental steps are as follows: (a) identify and define the disease of interest, (b) create a staging system (clinical-severity index) for disease severity, (c) identify comorbid conditions, and (d) establish the outcomes to be measured (4). Studies can perform only one of those steps or some or all. Therefore, development and validation of an outcomes instrument is one type of outcomes research, and identification of comorbid conditions is another type. Each step is discussed in more detail below.

Outcomes studies in Otolaryngology have yielded many important results, and some brief examples are listed here. There are now validated outcomes instruments for use in chronic sinusitis, nasal obstruction, hearing loss, chronic ear disease, tinnitus, dizziness, head and neck cancer, voice, gastroesophageal reflux, tonsil and adenoid disease, pediatric otitis media, and pediatric sleep apnea, among other diseases. In addition, there is a comprehensive prognostic staging system for obstructive sleep apnea in adults. Studies have shown that global quality of life is significantly worsened in adults with chronic rhinosinusitis and improves to near normal after endoscopic sinus surgery. Similarly, global quality of life is significantly worsened in children with tonsil and adenoid disease. We have learned that comorbid conditions have significant impact on survival and outcome in patients with laryngeal cancer and other head and neck cancers. Disease-specific quality of life has been shown to improve significantly after surgical treatment for the following conditions: chronic rhinosinusitis, pediatric otitis media, vocal cord paralysis, conductive hearing loss, nasal septal deformity, and pediatric sleep apnea, among others. In addition, cochlear implantation has been demonstrated to be very cost-effective relative to other health care interventions.

In addition to patient-based studies, other types of outcomes research have made important contributions, for example, appropriateness studies. In an appropriateness study, records from a large population are reviewed to test the hypothesis that treatment (medical or surgical) was based on appropriate indications. These studies often yield controversial results for two primary reasons. One reason is that it is difficult to achieve consensus on what is an appropriate versus equivocal versus inappropriate indication. An appropriateness study might find that a large percentage of procedures were performed for “equivocal” indications, when there is legitimate contradicting evidence
showing that those indications could have been classified as “appropriate.” The second reason is that those studies rely heavily on medical record documentation, which is often insufficient. So although the patient’s chart might indicate that appropriate indications were not met, in fact that particular patient might have actually met all the criteria. Despite the controversy, these studies are important because they generate discussion and can act as an impetus for further research, collaborative guideline development, etc.

Another type of outcomes research is population studies. Many important findings have been identified from studies of rates of medical care received by different populations. Of particular interest are studies that compare the rates of procedures with controversial indications. Some early examples were studies of population rates of tonsillectomy, hysterectomy, lumbar disk surgery, and carotid endarterectomy: researchers found that populations that seemed remarkably similar in demographics, economics, and health status had markedly different rates of elective procedures performed. The only apparent differences seemed to be where they lived, and the number of specialty physicians per capita in their region. Other studies have shown remarkable differences in rates of hospital admission for the same admitting diagnoses in different cities, even after controlling for many health status and demographic factors that might influence admission rate. While these studies sometimes generate more questions than answers, they give important insight into the delivery of health care.

One important example of a database/populationbased study was one that compared overall survival between patients with obstructive sleep apnea who were treated with continuous positive airway pressure (CPAP) versus patients who underwent uvulopalatopharyngoplasty (9). This study was performed in the Veterans Affairs (VA) system, which has had a national computerized medical record for many years and has a consistent population of patients who receive all their care at the VA. The authors reviewed the database and identified more than 15,000 patients eligible for study. They found that—controlling for a variety of variables including severity and comorbidity —survival was better in the surgery group than the CPAP group. This seemed surprising since in head-to-head comparisons CPAP is more effective than uvulopalatopharyngoplasty. The reason for the population finding seemed to be that many people prescribed CPAP were actually not using it, whereas everyone who had surgery did receive the benefit of surgery. A subanalysis seemed to indicate that regular CPAP users actually had better survival than surgery patients, but again the total group of CPAP patients (including many nonusers) did worse. This sort of population-based outcomes study is very important in determining treatments that actually work in populations. This is also an example of the difference between effectiveness research (in a real-world population) and efficacy research (in a controlled trial).

Meta-analysis is another type of outcomes research. In meta-analysis, the results from several individual studies are combined and a new statistical analysis is performed using the raw data from individual studies. Meta-analysis is more than just a detailed review of the literature, and to perform the analysis, each study must have used very similar methods and report data in a similar format. Despite some methodologic complexity, meta-analysis is a very helpful tool to help achieve an adequate sample power to answer some questions that individual small studies cannot, and also to help resolve conflicting results from individual studies.

Evidence-based medicine (EBM) is an important topic in contemporary medicine. While in fact there is some type of evidence behind many aspects of medical treatment, the explicit techniques of “EBM” have only been recently described and popularized. Dr. David Sackett has been a key developer and leader in EBM and is the lead author on an excellent textbook in the field (10).

More medical schools are teaching their students the techniques of EBM, and articles, books, lectures, and courses on EBM are popular in many fields. Two contemporary disease-related textbooks cover surgical fields: Evidence-based otitis media, by Rosenfeld and Bluestone, and Evidence-based otolaryngology, by Shin, Hartnick and Randolph.

EBM has been defined as the conscientious, explicit, and judicious use of the current best evidence in making decisions about the care of individual patients (11). Furthermore, the practice of EBM has been described as integrating individual clinical expertise with the best available external clinical evidence from systematic research (11,12). Several points—and misconceptions—are worth emphasizing.

First, EBM does not mean only the use of randomized clinical trials. The definition states that it requires the use of the “best available” evidence, not “only the best” evidence. If findings from a randomized clinical trial are available, then that is strong, high-quality evidence and should be used. If, however, randomized trials have not been performed and therefore that type of evidence is not available, you can still practice EBM by using the best available evidence.

Next, EBM does not eliminate the physician’s own experience or knowledge base. The physician should integrate their own clinical experience with the patient’s desire and with the current best clinical evidence when deciding on the best treatment for an individual patient.

The practice of EBM has been likened to a three-legged stool—which would be unstable if one leg were missing (13). The three “legs” of the stool are best evidence, clinical experience, and patient wishes. Along those lines, it would not be practical or reasonable for a physician to rely
only on results from high-quality evidence to make clinical decisions—since there are many questions and issues that have not been addressed with experimental studies. On the other hand, if a physician relies only on personal experience, their practice could become out of date or inappropriate. So the thoughtful and appropriate combination of experience and evidence is a goal to emulate.

There are five steps in practicing EBM (10,12), and we cover each step individually. Those steps are as follows:

Outcomes research can be defined as the scientific study of the outcomes of disease therapies used for a particular disease, condition, or illness (1). While all clinical research measures some type of outcome, such as mortality, morbidity, or some other objective measure, in “outcomes research,” the patient’s perception of their outcome is assessed.

Historically, the movement toward outcomes-based research was started by Dr. Paul Ellwood, who in the 1980s suggested that in the future physicians would assess outcomes by measuring what the patient experienced (2). Subsequently, tools to assess these outcomes were developed and applied across many diseases. However, outcomes research also includes other types of studies in addition to patient-based outcomes studies. Today, outcomes researchers study all aspects of the health care system—from the status of the patient or population at entry, to the organization, delivery, regulation and financing of the health care system, to the status of the patient or population after treatment.

To be inclusive of other aspects of health services research, some have divided outcomes research into record-based outcomes research and patient-based outcomes research. Examples of these different types of studies are shown in Table 8.1.

Prospective, retrospective, or cross-sectional study designs can be used in outcomes research. However, many outcomes studies use an observational prospective design, where outcomes are assessed after diverse treatments—rather than an experimental prospective design, where treatments are carefully controlled or randomized. The differences between these study designs are an important point for discussion.

In experimental or controlled trials, particularly randomized trials, the ideal design uses two groups of patients that are nearly identical in every aspect—except the treatment received. So therefore, any difference in outcome between groups must be due only to the different treatments, since otherwise the groups were the same. Of course, acquiring patient groups that are actually identical is rarely achieved, but nevertheless, that is the basis behind the rigorous design and methodology of controlled trials. In addition, to account for the inherent differences between treatment groups, the randomization process should theoretically distribute those differences (say, in demographics or disease severity) equally between the two groups. Carefully controlled experimental studies can be said to measure the efficacy of treatment, under ideal circumstances. These studies usually yield very reliable results concerning the effects of treatment—in the group that was studied.

However, there are questions about the generalizability of results from rigorously controlled trials to larger populations of patients with a disease because the larger group of patients will not be so carefully controlled and homogeneous. In addition, patient compliance and other factors may lead to different results than in tightly controlled trials. Furthermore, clinical trials are very expensive to perform, especially when considered from the standpoint of cost per patient studied.

Observational study designs are commonly used in outcomes research. All patients with a disease are included, and they are studied in the actual setting in which they receive their health care. This naturally introduces many other factors, which may impact outcome after treatment, including potential selection bias for different treatments. However, many would argue that results from large-scale outcome studies are more applicable to the general population because of their setting and scope. Studies that assess the actual (“real world”) results of treatment are said to measure the effectiveness of treatment versus efficacy measured from controlled trials (3). Furthermore, in addition to their “real-world” setting, the expanded outcome measured (quality of life, etc.) might be more relevant and important to patients than other clinical or biologic outcomes.

There are several basic steps involved in performing patient-based outcomes research (3,4,5). The fundamental steps are as follows: (a) identify and define the disease of interest, (b) create a staging system (clinical-severity index) for disease severity, (c) identify comorbid conditions, and (d) establish the outcomes to be measured (4). Studies can perform only one of those steps or some or all. Therefore, development and validation of an outcomes instrument is one type of outcomes research, and identification of comorbid conditions is another type. Each step is discussed in more detail below.