LAUSR

In their current article in Annals, VanderWeele and Ding (1) introduce the E-value as a simple measure of the potential for bias arising from unmeasured confounders in observational studies. Bias often poses a greater threat to the validity of reported findings than does the random variability reflected by P values and confidence bounds (2). Although the potential for bias is widely known, reports of observational data often lack sensitivity analyses exploring the possible influence of bias from unobserved factors, perhaps because authors face challenges in specifying the elements and degree of possible confounding in a manner that readers can understand. As a motivating example, consider the report by Park and colleagues (3) of the protective effect on mortality from consuming 4 or more cups of coffee per day compared with no consumption (hazard ratio, 0.82 [CI, 0.78 to 0.87]). Suppose that a higher rate of coffee drinking follows from higher income, a factor not measured in the study. How strong would the incomemortality effect have to be, and how different the incomes of coffee drinkers, to account for the effect of coffee on mortality? We can answer this question using only the reported estimate and CI. First, because the outcome is not rare, we must use the formula in Table 2 in VanderWeele and Ding's article (1) to convert the hazard ratios into approximate relative risks (RRs). Converted to RR, the point estimate equals 0.87. Second, because coffee exposure is protective, we must invert this RR to 1.15. Third, application of the E-value formula leads to an E-value of 1.56. Thus, the observed hazard ratio of 0.82 could be explained away by an unmeasured confounder that was associated with both the exposurehere, coffee consumption of 4 or more cups per dayand the outcomemortalityby a risk ratio of 1.56-fold each, above and beyond the measured confounders, but weaker confounding would not do so. Applying the E-value approach to the reported CI, if income were associated with both coffee consumption and mortality with an RR of 1.43, this level of confounding would render the results no longer significant. The Figure depicts the simple correspondence of the E-value and the RR depending on whether the exposure of interest is protective (RR <1) or not (RR >1). As one would expect, high E-values occur only when the exposureoutcome association is strong. A large E-value would be comforting, because results might be explained away only with substantial confounding. Conversely, a small E-value does not prove that confounding explains the observed association of exposure and outcome. Rather, a small E-value suggests potentially imaginable scenarios in which confounding might explain observed results. The risk of unobserved confounding is not merely a matter of RRs. As Rosenbaum (4) explained, more efficient designs of observational studies can reduce heterogeneity and thereby reduce sensitivity to unobserved sources of bias. The same E-value therefore might reflect different degrees of sensitivity across studies of different designs. Small E-values might call for more elaborate sensitivity analysesfor example, by use of simulationsor for in-depth discussions of the need for enhanced designs and for collection of more data on covariates that are potential confounders. However, the E-value, at the very least, is a suggestion that further investigation is warranted. Figure. The E-value is a function of the RR of the association of exposure (coffee consumption in the example) and outcome (mortality). The E-value rises from the minimum value of 1.0 as the estimated RR decreases away from a value of no association, representing the case of a protective effect of exposure (green curve), or as the RR increases for a harmful exposure (black curve). The dashed green lines reflect the correspondence between RR and the E-value for the particular coffee consumption example in the text. HR= hazard ratio; RR= relative risk. E-values apply easily to exposureoutcome associations expressed as RRs and rate ratios. They also may apply to hazard ratios, albeit with added effort as we just demonstrated, and to odds ratios and differences in binary and continuous outcomes with more assumptions and calculations. For each of these outcome measures, the authors also present either formulas or references to software tools, but a calculator and the Figure will suffice for many sensitivity analyses. With such ease of application, the goal of universal reporting of sensitivity analyses for unmeasured confounding in biomedicine is reasonable. Unmeasured confounding, as VanderWeele and Ding (1) note in passing, is not the only source of uncertainty and potential bias. Measurement error in covariates may lead to bias in either direction (5). Missing covariates might require imputation (6). For longitudinal designs, loss to follow-up might be related to an outcome that is not observed, leading to dropout or attrition bias (7). Among studies based on patient visits and self-reports, the timing of the data collection might be irregular and related to outcome, thereby inducing bias (8). Misspecified statistical models, even if they invoke popular approaches, such as propensity scores, may be a source of bias (9). All these biases, including unobserved confounding that E-values target, likely proliferate in big data, in which large, complex data sets are collected for purposes other than answering specific research questions; big data equals big potential for bias. By publishing this Research and Reporting Methods exposition on sensitivity analyses, Annals hopes to promote more universal use of bias sensitivity analyses. E-values are easy to calculate and apply to common metrics of exposureoutcome association, and result in simple measures of confounding levels that could explain away observed results. With sensitivity results, such as E-values, readers can more readily determine whether observed findings seem to be robust to bias and therefore are likely to be reproducible with alternative designs and new data. Over time, even systematic reviewers and meta-analysts, who combine results of observational studies to limit biases inherent in single studies, might augment pooled point estimates and confidence bounds with E-values to evaluate the potential for remaining bias. We hope that further methodological research will assess the practical utility of E-values across study designs of varying resistance to confounding bias. In the meantime, in our view, access to the simple E-value can, at a minimum, help lead to more thorough discussions of potential bias from unmeasured confounding.