The hazards of hazard ratios.
TL;DR: The hazard ratio (HR) is the main, and often the only, effect measure reported in many epidemiologic studies, and although the HR may change over time, some studies report only a single HR averaged over the duration of the study’s follow-up; here I review these 2 problems and some proposed solutions.
Abstract: The hazard ratio (HR) is the main, and often the only, effect measure reported in many epidemiologic studies. For dichotomous, non–time-varying exposures, the HR is defined as the hazard in the exposed groups divided by the hazard in the unexposed groups. For all practical purposes, hazards can be thought of as incidence rates and thus the HR can be roughly interpreted as the incidence rate ratio. The HR is commonly and conveniently estimated via a Cox proportional hazards model, which can include potential confounders as covariates. Unfortunately, the use of the HR for causal inference is not straightforward even in the absence of unmeasured confounding, measurement error, and model misspecification. Endowing a HR with a causal interpretation is risky for 2 key reasons: the HR may change over time, and the HR has a built-in selection bias. Here I review these 2 problems and some proposed solutions. As an example, I will use the findings from a Women’s Health Initiative randomized experiment that compared the risk of coronary heart disease of women assigned to combined (estrogen plus progestin) hormone therapy with that of women assigned to placebo. By using a randomized experiment as an example, the discussion can focus on the shortcomings of the HR, setting aside issues of confounding and other serious problems that arise in observational studies. The Women’s Health Initiative followed over 16,000 women for an average of 5.2 years before the study was halted due to safety concerns. The primary result from the trial was a HR. As stated in the abstract and shown in Table 1 of the article, “Combined hormone therapy was associated with a hazard ratio of 1.24.” In addition, Table 2 provided the HRs during each year of follow-up: 1.81, 1.34, 1.27, 1.25, 1.45, and 0.70 for years 1, 2, 3, 4, 5, and 6 , respectively. Thus, the HR reported in the abstract and Table 1 can be viewed as some sort of weighted average of the period-specific HRs reported in Table 2. This bring us to Problem 1: although the HR may change over time, some studies report only a single HR averaged over the duration of the study’s follow-up. As a result, the conclusions from the study may critically depend on the duration of the follow-up. For example, the average HR in the WHI would have been 1.8 if the study had been halted after 1 year of follow-up, 1.7 after 2 years, 1.2 after 5 years, and—who knows—perhaps 1.0 after 10 years. The 24% increase in the rate of coronary heart disease that many researchers and journalists consider as the effect of combined hormone therapy is the result of the arbitrary choice of an average follow-up period of 5.2 years. A trial with a shorter follow-up could have reported an 80% increase, whereas a longer trial might have found little or no increase at all.
Citations
More filters
TL;DR: This study in a nationwide mass vaccination setting suggests that the BNT162b2 mRNA vaccine is effective for a wide range of Covid-19–related outcomes, a finding consistent with that of the randomized trial.
Abstract: Background As mass vaccination campaigns against coronavirus disease 2019 (Covid-19) commence worldwide, vaccine effectiveness needs to be assessed for a range of outcomes across diverse p...
1,660 citations
Stanford University1, New York University2, Duke University3, Boston University4, Saint Louis University5, Northwick Park Hospital6, Imperial College London7, Hospital Universitario La Paz8, Durham University9, NewYork–Presbyterian Hospital10, Albany Medical College11, St. Michael's Hospital12, Montreal Heart Institute13, Auckland City Hospital14, All India Institute of Medical Sciences15, University of British Columbia16, Cedars-Sinai Medical Center17, Harvard University18, Brigham and Women's Hospital19, Saint Francis University20, Columbia University Medical Center21, University of Missouri–Kansas City22, Government Medical College, Thiruvananthapuram23, Sri Jayadeva Institute of Cardiovascular Sciences and Research24, University of São Paulo25, Emory University26, Veterans Health Administration27, Mayo Clinic28, Semmelweis University29, Flinders Medical Centre30, Université Paris-Saclay31, Uppsala University32, Uppsala University Hospital33, Keio University34, National Institutes of Health35, Vanderbilt University36, East Carolina University37, Icahn School of Medicine at Mount Sinai38
TL;DR: Evidence that an initial invasive strategy, as compared with an initial conservative strategy, reduced the risk of ischemic cardiovascular events or death from any cause over a median of 3.2 years is not found.
Abstract: Background Among patients with stable coronary disease and moderate or severe ischemia, whether clinical outcomes are better in those who receive an invasive intervention plus medical ther...
1,324 citations
TL;DR: Further public policy efforts that reduce fine particulate matter air pollution are likely to have continuing public health benefits, and Poisson models produced similar results.
Abstract: Background: Epidemiologic studies have reported associations between fine particles (aerodynamic diameter ≤ 2.5 µm; PM:2.5) and mortality. However, concerns have been raised regarding the sensitivity of the results to model specifications, lower exposures, and averaging time. Objective: We addressed these issues using 11 additional years of follow-up of the Harvard Six Cities study, incorporating recent lower exposures. Methods: We replicated the previously applied Cox regression, and examined different time lags, the shape of the concentration–response relationship using penalized splines, and changes in the slope of the relation over time. We then conducted Poisson survival analysis with time-varying effects for smoking, sex, and education. Results: Since 2001, average PM:2.5 levels, for all six cities, were < 18 µg/m3. Each increase in PM2.5 (10 µg/m3) was associated with an adjusted increased risk of all-cause mortality (PM2.5 average on previous year) of 14% [95% confidence interval (CI): 7, 22], and with 26% (95% CI: 14, 40) and 37% (95% CI: 7, 75) increases in cardiovascular and lung-cancer mortality (PM2.5 average of three previous years), respectively. The concentration–response relationship was linear down to PM2.5 concentrations of 8 µg/m3. Mortality rate ratios for PM2.5 fluctuated over time, but without clear trends despite a substantial drop in the sulfate fraction. Poisson models produced similar results. Conclusions: These results suggest that further public policy efforts that reduce fine particulate matter air pollution are likely to have continuing public health benefits.
846 citations
Cites methods from "The hazards of hazard ratios."
...Because RRs may vary over time and period-specific RRs may be biased, we used the Poisson model to calculate adjusted survival curves (Hernan 2010)....
[...]
TL;DR: Critical concerns regarding this conventional practice of quantifying the underlying differences between groups with respect to a time-to-event end point are summarized and various well-known alternatives are discussed.
Abstract: In a longitudinal clinical study to compare two groups, the primary end point is often the time to a specific event (eg, disease progression, death). The hazard ratio estimate is routinely used to empirically quantify the between-group difference under the assumption that the ratio of the two hazard functions is approximately constant over time. When this assumption is plausible, such a ratio estimate may capture the relative difference between two survival curves. However, the clinical meaning of such a ratio estimate is difficult, if not impossible, to interpret when the underlying proportional hazards assumption is violated (ie, the hazard ratio is not constant over time). Although this issue has been studied extensively and various alternatives to the hazard ratio estimator have been discussed in the statistical literature, such crucial information does not seem to have reached the broader community of health science researchers. In this article, we summarize several critical concerns regarding this conventional practice and discuss various well-known alternatives for quantifying the underlying differences between groups with respect to a time-to-event end point. The data from three recent cancer clinical trials, which reflect a variety of scenarios, are used throughout to illustrate our discussions. When there is not sufficient information about the profile of the between-group difference at the design stage of the study, we encourage practitioners to consider a prespecified, clinically meaningful, model-free measure for quantifying the difference and to use robust estimation procedures to draw primary inferences.
469 citations
TL;DR: In this article, the authors illustrate a model-based method to standardize observed trial results to a specified target population using a seminal human immunodeficiency virus (HIV) treatment trial, and provide Monte Carlo simulation evidence supporting the method.
Abstract: Properly planned and conducted randomized clinical trials remain susceptible to a lack of external validity. The authors illustrate a model-based method to standardize observed trial results to a specified target population using a seminal human immunodeficiency virus (HIV) treatment trial, and they provide Monte Carlo simulation evidence supporting the method. The example trial enrolled 1,156 HIV-infected adult men and women in the United States in 1996, randomly assigned 577 to a highly active antiretroviral therapy and 579 to a largely ineffective combination therapy, and followed participants for 52 weeks. The target population was US people infected with HIV in 2006, as estimated by the Centers for Disease Control and Prevention. Results from the trial apply, albeit muted by 12%, to the target population, under the assumption that the authors have measured and correctly modeled the determinants of selection that reflect heterogeneity in the treatment effect. In simulations with a heterogeneous treatment effect, a conventional intent-to-treat estimate was biased with poor confidence limit coverage, but the proposed estimate was largely unbiased with appropriate confidence limit coverage. The proposed method standardizes observed trial results to a specified target population and thereby provides information regarding the generalizability of trial results.
388 citations
References
More filters
TL;DR: This work argues that the causal structure underlying the bias in each example is essentially the same: conditioning on a common effect of 2 variables, one of which is either exposure or a cause of exposure and the other is either the outcome or acause of the outcome.
Abstract: The term "selection bias" encompasses various biases in epidemiology. We describe examples of selection bias in case-control studies (eg, inappropriate selection of controls) and cohort studies (eg, informative censoring). We argue that the causal structure underlying the bias in each example is essentially the same: conditioning on a common effect of 2 variables, one of which is either exposure or a cause of exposure and the other is either the outcome or a cause of the outcome. This structure is shared by other biases (eg, adjustment for variables affected by prior exposure). A structural classification of bias distinguishes between biases resulting from conditioning on common effects ("selection bias") and those resulting from the existence of common causes of exposure and outcome ("confounding"). This classification also leads to a unified approach to adjust for selection bias.
2,195 citations
"The hazards of hazard ratios." refers methods in this paper
...This built-in selection bias of the HR has also been described using causal diagrams.(3,4) In short, the average HR may be uninformative because of potentially time-varying period-specific HRs, and because the period-specific HRs may be time-varying because of built-in selection bias....
[...]
Harvard University1, George Washington University2, University of Tennessee Health Science Center3, National Institutes of Health4, Brown University5, Rutgers University6, University at Buffalo7, Rush University Medical Center8, University of Washington9, University of California, Los Angeles10, Emory University11, University of California, Irvine12, Wake Forest University13, University of Vermont14
TL;DR: Estrogen plus progestin does not confer cardiac protection and may increase the risk of CHD among generally healthy postmenopausal women, especially during the first year after the initiation of hormone use.
Abstract: Background Recent randomized clinical trials have suggested that estrogen plus progestin does not confer cardiac protection and may increase the risk of coronary heart disease (CHD). In this report, we provide the final results with regard to estrogen plus progestin and CHD from the Women's Health Initiative (WHI). Methods The WHI included a randomized primary-prevention trial of estrogen plus progestin in 16,608 postmenopausal women who were 50 to 79 years of age at base line. Participants were randomly assigned to receive conjugated equine estrogens (0.625 mg per day) plus medroxyprogesterone acetate (2.5 mg per day) or placebo. The primary efficacy outcome of the trial was CHD (nonfatal myocardial infarction or death due to CHD). Results After a mean follow-up of 5.2 years (planned duration, 8.5 years), the data and safety monitoring board recommended terminating the estrogen-plus-progestin trial because the overall risks exceeded the benefits. Combined hormone therapy was associated with a hazard ratio for CHD of 1.24 (nominal 95 percent confidence interval, 1.00 to 1.54; 95 percent confidence interval after adjustment for sequential monitoring, 0.97 to 1.60). The elevation in risk was most apparent at one year (hazard ratio, 1.81 [95 percent confidence interval, 1.09 to 3.01]). Although higher base-line levels of low-density lipoprotein cholesterol were associated with an excess risk of CHD among women who received hormone therapy, higher base-line levels of C-reactive protein, other biomarkers, and other clinical characteristics did not significantly modify the treatment-related risk of CHD. Conclusions Estrogen plus progestin does not confer cardiac protection and may increase the risk of CHD among generally healthy postmenopausal women, especially during the first year after the initiation of hormone use. This treatment should not be prescribed for the prevention of cardiovascular disease.
1,980 citations
"The hazards of hazard ratios." refers methods in this paper
...As an example, I will use the findings from a Women’s Health Initiative randomized experiment that compared the risk of coronary heart disease of women assigned to combined (estrogen plus progestin) hormone therapy with that of women assigned to placebo.(1) By using a randomized experiment as an example, the discussion can focus on the shortcomings of the HR, setting aside issues of confounding and other serious problems that arise in observational studies....
[...]
TL;DR: The findings suggest that the discrepancies between the Women's Health Initiative and Nurses’ Health Study ITT estimates could be largely explained by differences in the distribution of time since menopause and length of follow-up.
Abstract: Causal inferences are drawn from both randomized experiments and observational studies. When estimates from both types of studies are available, it is reassuring to find that they are often similar.1–3 On the other hand, when randomized and observational estimates disagree, it is tempting to attribute the differences to the lack of random treatment assignment in observational studies.
This lack of randomization makes observational effect estimates vulnerable to confounding bias due to the different prognosis of individuals between treatment groups. The potential for confounding may diminish the enthusiasm for other desirable features of observational studies compared with randomized experiments – greater timeliness, less restrictive eligibility criteria, longer follow-up, and lower cost. However, even though randomization is the defining difference between randomized experiments and observational studies, further differences in both design and analysis are commonplace. As a consequence, observational-randomized discrepancies cannot be automatically attributed to randomization itself.
In this paper we assess the extent to which differences other than randomization contribute to discrepant observational versus randomized effect estimates in the well-known example of postmenopausal estrogen plus progestin therapy and the risk of coronary heart disease (CHD). Specifically, we explore discrepancies attributable to different distributions of time since menopause, length of follow-up, and analytic approach.
The published findings on this topic can be briefly summarized as follows. Large observational studies suggested a reduced risk of CHD among postmenopausal hormone users. Two of the largest observational studies were based on the Nurses’ Health Study (NHS)4, 5 in the United States and on the General Practice Research Database6 in the United Kingdom. More recently, the Women’s Health Initiative (WHI) randomized trial7 found a greater incidence of coronary heart disease among postmenopausal women in the estrogen plus progestin arm than in the placebo arm (68% greater in the first two years after initiation, 24% greater after an average of 5.6 years).8, 9
The present paper does not address the complex clinical and public health issues related to hormone therapy, including risk-benefit considerations. Rather, we focus on methodologic issues in the analysis of observational cohort studies. Specifically, we reanalyze the NHS observational data to yield effect estimates of hormone therapy that are directly comparable with those of the randomized WHI trial except for the fact that hormone therapy was not randomly assigned in the NHS. We do this by mimicking the design of the randomized trial as closely as possible in the NHS. As explained below, our approach requires conceptualizing the observational NHS cohort as if it were a sequence of nonrandomized “trials.” Because the randomized trial data were analyzed under the intention-to-treat (ITT) principle, we analyze our NHS “trials” using an observational analog of ITT (see below).
A recent re-analysis of the General Practice Research Database using this strategy could not adjust for lifestyle factors and it yielded wide confidence intervals.10 Further, the estrogen used by women in that study was not the conjugated equine estrogen used by the women in the NHS and WHI studies. Our analysis of the NHS data incorporates lifestyle factors and includes women using the same type of estrogen as in the WHI randomized trial.
682 citations
TL;DR: The authors describe a method and provide a simple worked example using inverse probability weights (IPW) to create adjusted survival curves when the weights are non-parametrically estimated, equivalent to direct standardization of the survival curves to the combined study population.
662 citations
"The hazards of hazard ratios." refers background in this paper
...It is not unexpected that most epidemiologic articles include HRs only, because epidemiology students are traditionally taught to estimate adjusted HRs but not adjusted survival curves.(7) The next paragraph sketches a general procedure to obtain survival curves adjusted for baseline confounders....
[...]
TL;DR: The authors have written a macro in SAS language that produces the adjusted survival estimates and graphs and has the advantage over the average covariate method of allowing for the possibility that theadjusted survival curves differ in shape.
Abstract: When reporting results from survival analysis, investigators often present crude Kaplan-Meier survival curves and adjusted relative hazards from the Cox proportional hazards model. Occasionally, the investigators will also provide a graphical representation of adjusted survival curves based on regression estimates and the average covariate values in the study groups. In this paper, the authors review the limitations of this approach and examine alternative approaches to obtaining adjusted survival curves that have been proposed. Furthermore, a new method to obtain multivariate adjusted survival curves is described. This method is based on direct adjustment of the observed conditional probability of survival at the time of each event. When an unexposed group is used as a standard for adjusting an exposed group, the survival curve in the exposed group is adjusted to the covariate distribution among the unexposed at the time of the event. This method has the advantage over the average covariate method of allowing for the possibility that the adjusted survival curves differ in shape. The method can handle multiple fixed or time-dependent categorical covariates as well as left truncated data, and it allows for estimation of confidence intervals. The authors have written a macro in SAS language that produces the adjusted survival estimates and graphs. This macro is available on request and can be downloaded through the World Wide Web.
197 citations
"The hazards of hazard ratios." refers methods in this paper
...Hazard rate curves were modeled using cubic splines with join-points selected by Akaike’s information criteria(3,4); 95% CIs were applied with bootstrap resampling techniques.(5) Under the null hypothesis of no interaction over time, annual hazard rates for ER-positive and ER-negative breast cancers would be proportional (or similar) with follow-up after initial breast cancer diagnosis....
[...]