7 Curable Sins in Epi

           
           

Says Future of Epidemiology Depends on Doing More Consequential Research
What can we do besides having a better PR (public relations) firm? That's a question asked by Lewis Kuller in his 1996 presidential address in front one of the largest groups of epidemiologists ever to attend a Society for Epidemiologic Research (SER) meeting in Boston in mid-June. Noting all the publicity about epidemiologic findings in the last year, including a Science article highly critical of epidemiology, Kuller offered a list of curable sins which he said might be contributing to epidemiology’s fall from scientific grace.
The seven sins described by Kuller, who is chairman of the Department of Epidemiology at the University of Pittsburgh, are:
  1. the sin of biological implausibility
  2. the sin of confusing reproducibility of a measure with accuracy or validity the parrot effect
  3. the sin of adjusting data unnecessarily the adjustment of data syndrome
  4. the sin of failing to determine temporal associations the epi phenomenon
  5. the sin of using multivariate models to evaluate biological or causal associations
  6. the sin of replicating the same epidemiological studies over and over again with minor variations on the theme
  7. the sin of disassociating academic epidemiology from public health
Biological Implausibility
This occurs when the investigator does not know or understand the biology or pathophysiology of the disease of interest or the measures of the independent and dependent variables. The reported results of the study, i.e. the measurements, are non-physiological. The investigator is unaware of the problem. This is more likely to occur when the data, usually the independent variable, is transformed and the units of measurement are abstruse. Investigators, reviewers, and journal editors have no idea of the range of the true measurement.
"Parrot Effect"
Comparison of different methods of measuring the same variable does not necessarily prove that any of the measures are accurate. It just assumes that the correlations among the measurements are fairly high, i.e. repeatability but not accuracy. According to Kuller, "you can teach a parrot to say good morning. The parrot says good morning all day long very repeatably, but the parrot is unfortunately inaccurate most of the time."
The Adjustment of Data Syndrome
"I do not believe that one can truly adjust for sex, except by a surgical procedure that is very unpopular," says Kuller. Also, he believes that adjustment for host specific factors eliminates host susceptibility as an item to be studied, and calls it "a dreadful mistake." Furthermore, age adjustment presumes similar biological relationships for young and old individuals, and this is "clearly illogical".
The Failure to Determine Temporal Associations
According to Kuller, it is likely that the independent variables are really due to the dependent variables for many of the associations reported in epidemiology. The levels of many risk factors in both cardiovascular disease, cancer, and probably other chronic diseases are modified by the disease in its early subclinical phases.
Multivariate Models To Evaluate Biological or Causal Associations
According to Kuller, epidemiologists often use multivariate models to exclude co-variates because their p-value was less than .10 or .05 in the model. Some of the variables that remain in the model are clearly dependent variables of the independent variables. “There is nothing wrong with generating a new testable model of associations,?says Kuller, “the epidemiologist however must document the nature of the new model, how it relates to existing biological interpretations and how the model can be further tested in both epidemiological and laboratory sciences...The use of complex statistical methodology is not an answer for either good study design or biological plausibility.?
Replicating Epidemiolgic Studies
To make his point, Dr. Kuller asked—Are many epidemiological studies to be classified as inconsequential because they only represent fine tuning of previous studies with slightly different populations with little likelihood of enhancing the science of epidemiology and the public health and preventive medicine implications? According to Kuller, such studies are often methodologically outstanding, the analysis is superior, and the study is dull! In his view, the basic problem is the failure to move from descriptive to observational studies to clinical trials/natural experiments in order to further our understanding of potential etiologic associations.
Schism Between Academic Epi and Public Health
This sin is probably the most important, according to Kuller, and involves failure on the part of both the public health community and the epidemiology community. According to Kuller, the public health establishment in the United States has, to a considerable degree, lost its epidemiology base and similarly epidemiology has lost its public health base to a considerable degree. "It is time for academic epidemiologists to take a very hard look at what is public health epidemiology and preventive medicine and re-focus some of our efforts in establishing a strong scientific basis to evolving public health and preventive medicine programs." The use of epidemiology to identify risk and evaluate public health (and to develop) preventive strategies will continue to be a "big winner,"says Kuller.
Conclusion
Dr. Kuller offered this view in his concluding remarks. “We are very good when we study epidemics, have well-defined and testable hypotheses that will lead to effective public health and preventive medicine techniques that reduce morbidity and mortality and promote society’s health. We are not very good when we wander around in a circle of variables—dependent, independent, confounded, (and) biased, hoping to discover the magical p< .05 or confidence limit above 1. We do even worse when we focus on ill-defined truisms, develop new languages, and presume that we have discovered a new epidemiology. The future, therefore, of epidemiology depends on more consequential epidemiological research. (It depends on) the re-uniting of epidemiology with preventive medicine and public health, and (on) moving epidemiological studies more rapidly from descriptive to analytical observational to experimental clinical trials. Published July 1996 v
Postscript 2000
In my 1996 SER address on the sins of epidemiology, I tried to encourage my fellow epidemiologists to focus more on good, solid, traditional methods of epidemiology such as including the incubation periods of the disease, the modes of disease transmission, biological plausibility, and hypothesis testing. The year 2000 offers new and important opportunities for epidemiology. The Human Genome Project, I believe, will soon make it possible to redefine host susceptibility in the model of host, agent and environment and thus, will make epidemiology more important than it has been in the past. The new molecular methods may provide a technology to better identify agents as has already been done in the Study of Viral Etiology of Disease (Kaposi Sarcoma) and good epidemiological studies based on solid traditional methods may soon bloom in the modern worlds of genetics, molecular biology and new technology.
What, then, are the negative problems? First, we still have substantial problems with the use of epidemiological methods. Many of our colleagues, unfortunately, are trained in statistical analysis but not in good, solid methodology. We still are not focused on specific hypothesis testing but, rather, on collections of data and numerous enthusiastic but irrelevant analyses.
A good epidemiology study of tires and automobile accidents would have likely identified the agent without adjusting for the number of people in the car, age of the driver, type of road pavement, wind speed, outside temperature, rainfall, snowfall, height and weight of the driver, depressive symptoms of the driver, relationship between the driver and other people in the car, whether they were using a cell phone, whether they were smoking, the association with drug abuse, alcohol consumption, and their most recent food frequency questionnaire.
The challenge of epidemiology in the future will be to use our proven methodology to further evaluate causal hypotheses of disease, to test the efficacies of intervention, and to provide the scientific basis of good, solid public health practice. If we do not have good epidemiology, it is very unlikely that we will have good public health and preventive medicine. We should remember that the size of the number of independent variables in a multivariate analysis is most likely inversely related to the likelihood of finding a true association with the dependent variable, i.e., disease.

Links in Epidemiology Resources

Online Journals

Useful Links in the History of Epidemiology

Epidemiologic Methodology Issues

Resournces for Epidemiologists