Epidemiologic Approach to Clinical Research: Design Strategies
Emily Y. Chew
John Paul SanGiovanni
Frederick L. Ferris III
Now of first principles we see some by induction, some by perception, some by a certain habituation, and others too in other ways. But each set of principles we must try to investigate in the natural way, and we must take pains to state them definitely, since they have a great influence on what follows. For the beginning is thought to be more than half of the whole, and many questions we ask are cleared up by it.
—Aritistotle, Nicomachean Ethics, Book 1, Chapter 7, ca. 330 b.c.
The passage above is often paraphrased as “well begun is half done”; it is an axiom of great relevance to the clinical researcher, as the best opportunity to optimize potential for obtaining meaningful results and making valid inferences exists within the design phase of a study. Here we present a general overview of key concepts and methods applied within an epidemiologic approach to clinical research design in ophthalmology.
Gaining knowledge of how the body functions in health and disease and of what can be done to intervene in the disease process has been a central pursuit in scholarship and clinical practice throughout the history of medicine. Astute clinicians have cataloged human ailments, developed specific treatments, and drawn meaningful inferences about cause-and-effect relationships. Systematic approaches to recording characteristics or symptoms and testing interventions or potential treatments in groups of healthy, high-risk, or diseased people have emerged with development of the scientific method. One early application of such approaches is noncomparative and focuses on development of treatment modalities; we know it as the case series. In the case-series approach, groups of diseased people are followed over time to estimate the likelihood of various outcomes, either with or without treatment. Adapting a case-series sampling design to clinical research, physicians have been able to develop useful treatments such as penicillin to combat pneumonia and insulin to treat diabetic ketoacidosis. Inference is constrained in the case-series approach when a condition is chronic, progresses at a highly variable rate, or treatment may seem to work well in some people but not in others.
The use of photocoagulation in the treatment of diabetic retinopathy is an example of this. Difficulties in demonstrating the benefits of this highly effective treatment have been reviewed by Ederer and Hiller.1 Although some ophthalmologists reported that photocoagulation was clearly of benefit, others insisted that their results suggested otherwise. The most effective way to resolve such a controversy is to use an “epidemiologic” approach.
Epidemiologic Approaches to Clinical Research
Definitions of epidemiology may be unified to represent the science concerning study of the distribution and determinants of disease. Epidemiologic methods are used in clinical research to measure occurrence of well-characterized events or states and to assess associations of putative causal determinants with such events. Epidemiologic measures are used for (1) describing existence or occurrence of exposures and events, and (2) analysis and inference on the magnitude (strength) and direction (benefit or risk) of determinant-event relationships.
Prevalence and incidence are descriptive epidemiologic measures of event existence and occurrence; in most cases these measures are derived from random samples within defined populations and expressed by their distribution across time, space, and groups of people that vary with respect to exposures or events. Prevalence and incidence estimates for ocular diseases of public health significance exist in a compendium of work published in the April 2004 issue of Archives of Ophthalmology. Descriptive epidemiologic measures provide information for generating hypotheses and planning analytic studies to examine determinant-event relationships.
A central purpose of evaluating determinant-event relationships in clinical research is to obtain meaningful information about the utility of diagnosis, prevention, or treatment in health and disease. The process starts with careful consideration of a plausible concept in disease etiology. Important concepts are commonly spawned from theory, anecdotal reports, or systematic observation in clinical practice or laboratory settings. Concepts are applied in hypothesis formulation. Hypotheses should contain precise language about the suspected relationship of an event of interest with putative etiologic determinants. Since hypotheses specified a priori permit the most objective interpretation of results, this practice should be applied to ensure that all analyses central to the primary study objectives will be evaluated under optimal conditions. As discussed below, the nature of the hypothesis will be a major force in selecting the study design most appropriate for hypothesis testing.
Hypothesis testing involves evaluation of statistical associations that are based on conditional probability statements for determinant-event relationships. The hypothesis is evaluated against a null value that represents equality in rates of event occurrence between study participants exposed or unexposed to the putative causal determinant. Absolute or relative epidemiologic measures of association guide the researcher in making inferences from hypothesis tests. Absolute measures of association include the attributable risk and risk difference. Relative risk, odds ratio, and etiologic/protective fractions are relative measures. Relative measures provide the most unbiased means for estimating magnitude of relationships, since they express occurrence rates on a scale or proportions respective to those of a comparison group (see http://www.pitt.edu/∼super1/courses/epi.htm for details on calculating epidemiologic measures).
Hypothesis testing involves two major elements. The researcher must first decide whether a valid statistical association exists. A valid association is one in which the determinant varies systematically with the event and is unlikely to be explained by flaws in research design, aspects of measurement, data collection, or analysis. Major threats to validity include (1) inaccuracies or imprecision introduced by the magnitude of random sampling variation in the study sample (chance variation); (2) systematic sampling, measurement, or reporting errors (bias); and (3) unmeasured or unanalyzed variation in results derived from factors associated both with the event of interest and the primary exposure (confounding factors). See the section “Threats to Validity of Inference” for additional information on the roles of chance variation, bias, and confounding as alternative explanations for determinant-event relationships.
If the researcher is able to conclude that a determinant-event association is valid, the next step is to consider whether causal inference is appropriate (see Rothman and Greenland2 for a detailed treatment of the subject). A number of factors affect the strength of causal inference; these include but are not limited to the following:
The magnitude of the determinant-event relationship: While stronger relationships argue more compellingly for causal inference, it is important to note that unmeasured or unanalyzed factors may always exert substantial influence on results in any study in which exposure is not randomly assigned.
Biologic credibility: Existence of a biochemical or biophysical pathway linking the putative determinant with the event.
Concordance of results with studies and systematic reviews conducted by different research groups, applying a variety of methods in diverse study populations.
A temporal sequence demonstrating that the determinant existed or changed prior to the event.
Dose-response relationship: A relationship of intensity and/or duration of exposure with occurrence of an event.
Assessment of a valid statistical association and causal inference allows the researcher to make a more informed decision about the suitability of generalizing results from the study population to other populations with similar characteristics. “Externally valid” results are those that may be reasonably generalized. The replication of findings in multiple settings through diverse research designs is essential in considering the likelihood that a study has high external validity. For this reason there are now “global harmonization” movements in clinical research to aimed at use common outcome measures, application of standardized measurement protocols, and formation of research networks designed to conduct work among researchers with common interests; the Diabetic Retinopathy Clinical Research Network (DRCRNet) is one example of such efforts. (Information on the DRCRNet may be found at http://www.niddk.nih.gov.easyaccess1.lib.cuhk.edu.hk/fund/diabetesspecialfunds/consortia/DRCRnet.pdf.) A major benefit of such networks is that they permit access to a more diverse universe of study subjects. As a final comment on these concepts it is extremely important to realize that attaining a high level of internal validity in measures of effect is a necessary precedent for considering external validity of results.
The core concepts presented above should be considered in applying epidemiologic design strategies in clinical research. Choosing the most appropriate strategy depends on the hypotheses being tested, the nature of determinants and events, current understanding of disease pathogenesis, study feasibility, the availability of participants, and research resources. The various epidemiologic approaches to clinical research, their strengths and weaknesses (potential effects on inference), and their appropriate applications are presented in the following section. Detailed discussions are available elsewhere.2,3,4,5,6,7,8,9,10,11,12
Threats to Validity of Inference
Chance Variation
Potential impact of chance variability on inference is assessed with statistical significance tests and confidence intervals. Random variation in the study sample may lead the erroneous conclusion that a determinant-event relationship exists in the universe of people representing the study sample. Measures of central tendency, dispersion, distribution, association, and difference used in clinical research are only estimates of population statistics, as it is usually unfeasible to collect data from all members of the population of interest. Although emphasis on obtaining a study sample representative of this population is paramount, repeated sampling in sets of individuals from the population of interest will not yield exact measures of true variance in determinants under consideration; as such, it is essential to determine the size of a sample necessary to minimize sampling variation since larger samples yield more stable (reliable) estimates. Discussion of sample-size and power calculation techniques is beyond the scope of this chapter. The interested reader is referred to Chow13 for background and http://members.aol.com/johnp71/javastat.html#Power for on-line sample-size and power calculators.
Researchers will often apply a criterion of “statistical significance” representing conditional probability that the magnitude of a determinant-event relationship observed in a study sample is at least as extreme as that possible random variation alone. This criterion, the P value, has been historically set at 0.05 which reflects the researcher’s acceptance of a 5% level of error in concluding that a relationship may not be explained by random variation (e.g., the determinant-event association is not likely to be a chance occurrence). Using the P value alone in assessing the role of chance variation is not appropriate, as the influence of the magnitude of determinant-event relationship cannot be separated from the influence of sample size. With large samples, small differences in epidemiologic measures may be rendered statistically significant, although their clinical relevance may be reasonably questioned. Likewise, a larger difference may not be observed if the study sample does not have sufficient numbers to reduce the influence of random variation. Confidence intervals should be considered with the P value since they provide unique information on magnitude and sample-based variability. Confidence intervals represent limits on the set of values containing the true magnitude of effect within specified degree of certainty. The level of certainty is conventionally set at 95%. The width of the confidence interval reflects sample variation and provides information on the precision of estimate in the epidemiologic measure. In the case that the P value is sufficiently high to prevent the researcher from ruling out chance variation as an explanation, and the confidence intervals are wide, prudence must be exercised in interpreting results; the study may have been underpowered. In the case where P > 0.05 and confidence limits are narrow, the researcher gains assurance in concluding that a determinant-event relationship is unlikely.
Bias
The impact of bias may be most effectively reduced in the design phase of a study; well-defined enrollment criteria, validated instruments and measurement protocols, and staff certification are essential for reducing this threat to validity. Selection bias occurs when the study recruitment or enrollment process varies systematically between participants with a given exposure or event. Information may be biased if the interviewer systematically applies different techniques in data collection or interpretation based upon knowledge of exposure or disease; spurious differences may result from differences in measurement quality, completeness, accuracy, and precision. Information bias may be generated by systemic differences in subject behavior. Participants with a certain exposure or event may systematically recall information differently than those without the exposure or event. When classification of putative determinants or events is biased, the inaccuracies produced may either be distributed randomly or not. Random misclassification will attenuate the magnitude of a true relationship. Nonrandom misclassification may bias results to attenuate or inflate any true relationship, depending on magnitude and direction of the inaccuracy.
Confounding
The impact of confounding may be substantially reduced in design and analytic phases of a study. Design-based techniques involve random allocation of exposure (in experimental designs) and sample restriction (in experimental and observational designs) and matching (in observational designs). Analytic approaches include multivariable analysis and stratification on a potentially confounding factor. Confounding factors are independently associated with the primary exposure and the outcome; assessment of confounding is essential because the direction of determinant-event relationship may be similar to or different than determinant-confounder relationships. In the former case estimates of magnitude of the determinant-event relationship may be increased; in the latter it may be attenuated to the point of nonsignificance.
TYPES OF CLINICAL STUDIES
Study designs are applied for different purposes and the researcher’s choice of an approach will be determined by the nature of the issue under investigation and access to resources. It is therefore essential to understand the value and limitations conferred by various designs, as the sampling scheme applied within a study affects threats to validity and thus the strength of inference.
Clinical research study designs can be classified as observational (the investigator observes the outcome in the persons with self-selected factors) or experimental (the investigator purposely manipulates factors that might influence the outcome). Observational study designs are applied to describe characteristics of populations and to analyze determinant-event relationships. Natural history studies, cross-sectional surveys, correlational studies, and case reports/case series may be applied as descriptive observational designs to obtain information on patterns of disease occurrence across populations, geographic areas, and time. Such information is a useful guide in planning analytic design strategies to elucidate factors or processes implicated in disease pathogenesis or progression. Case-control and cohort studies are the most commonly used analytic observational designs, although under certain circumstances natural history studies and cross-sectional surveys may also be used in determinant-event analyses. The clinical trial is the only experimental type of clinical research design; it is used for analytic purposes. In the following section we present an overview of the major analytic observational design strategies, in the context of extant studies. We then discuss the randomized clinical trial.
Natural History Studies
The natural history study is similar in design to the case series approach. In this approach subjects are identified as having the disease in question and followed over time for the development of preselected events, such as presumed complications of the disease. Although this is ideally done prospectively, it is possible to obtain information from existing health records. Natural history studies permit calculation of rates and proportions of the study population progressing to an event in a specified period of time. They are also useful in identifying risk factors that make certain persons more likely than others to have an event.
A case series of untreated patients that uses defined eligibility and exclusion criteria and complete follow-up is a natural history study. Unfortunately in many case series, eligibility and exclusion criteria are not carefully defined, and the follow-up is incomplete on a sizable proportion of the patients. Examples of prospective natural history studies are the natural history segments of the Diabetic Retinopathy Vitrectomy Study (DRVS)14 and the Age-Related Eye Disease Study (AREDS).15 The main hypothesis in the DRVS was that persons with a specified level of severe diabetic retinopathy have a high risk of losing vision. It was necessary to confirm this prior to considering vitrectomy for these patients. Patients with this degree of diabetic retinopathy were entered into the study and examined at regularly scheduled visits. Rates of severe visual loss, at specific time intervals, could then be calculated. In addition, certain ocular factors were studied to learn whether they would be predictive of severe visual loss. This study demonstrated that eyes with specific characteristics of diabetic retinopathy are at high risk of developing severe visual loss. These characteristics were then used as the major eligibility criteria for entry into one part of a randomized clinical trial designed to evaluate vitrectomy in such eyes. AREDS was designed to evaluate factors influencing the risk of progression to advanced age-related eye diseases and associated vision loss among people with a wide range in the extent and severity of age-related macular degeneration (AMD). Results on the nature and rates of AMD progression were instrumental in identifying the high-to-moderate risk study population for a subsequent clinical trial (AREDS-II). Natural history data from AREDS are also being used to develop a severity scale for defining risk categories for development of AMD.16,17
Surveys
The survey design is applied in clinical research to estimate event prevalence, support concepts, and generate hypotheses for analytic studies designed for making inferences about determinant-event relationships. Exposures and events are evaluated simultaneously in cross-sectional surveys. It is thus difficult to conclusively demonstrate that a determinant preceded an event. Under the exceptional circumstances in which the nature of an exposure is stable (e.g., certain inherited traits) it is possible to test hypotheses with this approach. In addition, information on the characteristics of the disease as well as on characteristics of the population may be used to study possible associations between the disease and various risk factors. However, although this type of study can demonstrate associations, one cannot infer cause and effect because the risk factor found may be the cause of the disease, the result of the disease, or may simply coexist with the disease.
Population-based surveys require either the examination of each person in the population or everyone in a specifically defined subgroup. To the extent that some persons may not be available for examination, the accuracy of the prevalence estimate is diminished. An example of a cross-sectional survey is the Wisconsin Epidemiologic Study of Diabetic Retinopathy.18 In this study, attempts were made to identify all known people with diabetes in several counties in southern Wisconsin. A random sample of this group was then selected and efforts were made to examine each of these persons. The main goal of this study was to determine the prevalence of various types of retinopathy in this population and to identify risk factors such as the degree of hyperglycemia or hypertension that might be associated with more severe retinopathy.
The National Health and Nutrition Examination Survey19 and the Visual Acuity Impairment Study,20 both conducted in cooperation with the US Census Bureau, are examples of studies on eye diseases in the general population. The factors limiting the success of a survey are the techniques used to identify the study population and the methods used to motivate those selected to participate in the examination process and/or interview. The Baltimore Eye Survey was a population based study.21 In this study, the prevalence of primary open-angle glaucoma was compared between black and white residents of east Baltimore. Based on previous clinical studies, there was a growing acceptance that blacks were at a higher risk of glaucoma than whites. Results supported this clinical impression. Using comprehensive examination techniques in population-based samples of blacks and whites, the rate of primary open-angle glaucoma in black Americans was found to be four to five times higher than in whites. The comprehensive examination techniques maximized the sensitivity and specificity of the diagnosis of primary open-angle glaucoma.
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
