Hypothesis Generation and Proof (or Disproof) of Concept

In clinical medicine, the need to investigate an individual case is the business of the day. Thus, it is not surprising that many important hypotheses in clinical epidemiology derive from clinical observations. In this manner, single cases or a series of cases often instigate important agendas of clinical investigation leading to valuable therapeutic applications and scientific paradigms. Initial observations particularly are useful when they fit into a hypothesis with biological plausibility, in which case an important criterion of causal inference already is met. Many brilliant clinicians make major contributions by creating such hypotheses based on their clinical observations (for example, observations of Prof J. Donald M. Gass ).

Because clinical trials, cohort studies, and even case-control studies require a considerable investment of cost and effort, characterization of a series of patients to provide proof (or refutation) of concept of the hypothesis in question is a logical first step in a research agenda, often required by funding agencies. When these early results are compelling and interesting, it is appropriate to report these results as pilot investigations, admitting the limitations of the method and recognizing the report as an early step in a line of investigation. Such a series would carry more weight if it did not include the first observations that gave rise to the hypothesis, which would provide some degree of independent support of the initial exceptional observation(s) that provoked the research agenda and would better fit the statistical hypothesis testing paradigm (which requires that observations potentially could refute the hypothesis). In reporting such results, the critical importance of performing a definitive study thereafter must be acknowledged, because there are numerous examples of such studies refuting conclusions based on compelling initial observations (consider the case of grid macular photocoagulation for prevention of complications of age-related macular degeneration ). For the testing of hypotheses, case series are an important early step in the process of investigation, but rarely are definitive.

Recognition of Sentinel Events

Prospective studies, including randomized clinical trials, are limited in their ability to identify rare adverse effects of exposures (such as treatments). Adverse event reporting provides an important safety function both during such trials and after new drugs come to market to identify severe adverse effects as soon as possible. Publication of such events plays a critical role in improving the safety of patients who are candidates for the new treatment. There are many examples of important reports of this nature regarding ocular toxicities of drugs. The World Health Organization has developed a system for assessing potential causality in evaluating drug side-effect associations that should provide guidance for evaluating potential associations in reports of this nature.

Likewise, observations of unexpected clusters of cases may provide clues to emerging epidemics or recognition of previously unrecognized syndromes. Armenian has provided guidance about how one would evaluate highly unusual cases, in pursuit of an explanation. A noteworthy example of an article identifying an emerging epidemic was a small series of 5 cases of an exceptionally rare lung disease ( Pneumocystis pneumonia) among a similar group of individuals (homosexual males in Southern California) ; this was one of the most influential articles ever written, because it was the first sentinel report leading to recognition of the worldwide pandemic of AIDS. An example of a report identifying a previously unrecognized syndrome was the first report of birdshot retinochoroiditis —a condition that presumably was present for generations but was not recognized until 1980. In these examples, although there was no comparison group, the observations were compelling either because occurrence of such diseases in the population of interest was known to be vanishingly rare, or because of establishment of proof of concept that the syndrome existed. Prompt reporting of such observations plays an important role in management of disease outbreaks and recognition of new clinical entities, which can be very important both for population health and clinical practice. Although reporting should await a sufficient number of observations to make the point, waiting for a very large amount of observation time to accumulate in cohorts before reporting such observations would be inappropriate in these circumstances because of the need of health care providers to respond promptly to the new information.

As stated previously, follow-up analytic studies should be performed to ensure that the initial conclusions in reports of this nature were correct and to expand on the observations (as was carried out in these instances). To understand why a compelling set of observations must be considered an exploratory observation rather than confirmation of a hypothesis, remember that if the exceptional observation(s) provoked the hypothesis, then there is no way the hypothesis could have been refuted by those observations. By definition, the observations were hypothesis generating, rather than an activity involving generation of data to test a hypothesis. Because reports of this nature would not have been published had the results not been exceptional (and thus represent a publication bias), further studies generally should be designed to detect a smaller difference than was observed in the initial series.

Studying Outcomes of Rare Diseases or New Treatments

Perhaps the most common form of manuscript encountered by journal editors in ophthalmology is a small case series reporting the outcomes of a novel treatment or of a rare disease. Most of these series are too small to be of much interest, because the risk of an outcome cannot be estimated precisely unless the series is large and the amount of observation long—in which case the case series becomes a cohort study, wherein the case definition defines entry into the cohort. To see this, consider a series of 10 cases that received a novel surgical treatment for a rare disease, 4 of whom had an early adverse outcome (all patients having the same amount of follow-up, so that an exact binomial confidence interval can be used). The 95% confidence interval on the best estimate of risk (40%) would be 12.1% to 73.8%, leaving the reader uncertain as to whether the event is uncommon or highly frequent. Alternatively, consider an alternative scenario in which 0 complications of a new surgical procedure were observed: the 97.5% 1-sided upper confidence limit would be 30.8%, supporting up to a 30% risk of complications as plausible, leaving the safety of the procedure very uncertain. Evaluation of candidate risk factors in this situation would have even less precision. However, if the outcomes of a condition were uniformly dismal, and a series of 10 cases found no instances of a bad outcome, these results would be compelling. Rather than wasting time and energy trying to estimate risk with an inadequate study design, those considering reporting a case series for this purpose first should estimate the precision that is possible based on the data they are likely to find to evaluate the value of such information in comparison with external controls. If an inadequate number of observations are available, collaborative study that allows reasonable sample size goals to be met (for example, a collaborative study of bevacizumab in inflammatory ocular neovascularization ) typically will be far more useful than a small “me-first” report. The communication facility presently available to clinicians provides relative ease in pooling rare observations over large numbers of centers to describe rare but meaningful associations that would not be established by single-center series because of the limited number of observations.