LAUSR

210 www.thelancet.com Vol 393 January 19, 2019 We live in the real world, so it is reasonable to expect that data collected from the real world should help identify effective therapies. Indeed, rapid increases in the availability of registries, electronic health records, and insurance claims, and the ability to access, process, link, and analyse data from these sources at fairly low cost lend support for calls to replace randomised controlled trials (RCTs) with so-called real-world studies to establish the efficacy of a therapy, particularly for common serious diseases with abundant, easily collected data such as diabetes. This push is driven partly by the need to show payers that therapies are working and are therefore of value when used in the real world. Other driving factors include the industry’s wish to reduce costs and time to get results, a mistaken belief that real-world data are somehow more relevant than RCT data for establishing efficacy, and the ease and speed with which registry data can be accessed and publications generated. However, even with the use of sophisticated methods to address various sources of bias, the absence of randomisation precludes protection from confounding and can lead payers and clinicians alike to erroneously infer that a therapy is beneficial or harmful. Observational data from the real world can help to identify associations between drug exposures and outcomes, and they can be used to estimate the strength of such associations. For example, a careful analysis of a Swedish registry containing data prospectively collected from 18 556 people with diabetes showed that secondline therapy with insulin was associated with a 69% higher mortality over a median of 3·9 years than second-line therapy with a dipeptidyl peptidase-4 inhibitor (hazard ratio 1·69, 95% CI 1·45–1·96). Whereas this analysis might lead clinicians and payers to conclude that insulin increases mortality, it could just as easily reflect unaccounted for confounders. Confounders can bias observational findings such that the true results might be diminished or increased. In the example above, if people commencing insulin therapy had a characteristic that led prescribers to select insulin that was in turn linked to mortality, such a characteristic would be a confounder. Statistical adjustments can account for some confounders, and in the foregoing example, such an adjustment was made. However, many confounders cannot be adjusted for because they are either unmeasured or unknown (eg, clinical judgment or uncaptured contraindications). The E value can quantify the vulnerability of an observed relationship to unaccounted for confounders. The problem of confounding is elegantly eliminated by large-scale RCTs in which the randomisation process effectively balances all confounders (known or unknown), thus creating groups with essentially identical characteristics that differ only according to their allocated intervention. Provided that an RCT satisfies three key criteria, any between-group differences in clinical outcomes during follow-up can be confidently attributed to the intervention being evaluated. These criteria are: (1) a large enough sample size to ensure that meaningful baseline imbalances do not occur by chance; (2) intentionto-treat analyses in which the effect of an intervention in a randomised group is analysed in every participant allocated to that group, regardless of adherence to the intervention or treatment with additional therapies; and (3) near-complete follow-up of all randomly assigned participants until the end of the trial. When the effects of insulin on mortality in the observational study described above were evaluated in a large RCT, the effect of insulin on the risk of death was neutral. Thus, apparently clear findings based on observational data from the real world were not corroborated when the possibility of confounding was removed by randomisation. Confounding is unlikely to underlie relationships with extreme relative risks such as those less than 0·25 or greater than 4. When such situations are encountered, an RCT is generally not needed to assess causality. For example, no RCT was required to show that insulin prevented diabetic ketoacidosis in people with type 1 diabetes because it is so effective for this indication. Such situations are, however, rare. Conversely, relationships with relative risks varying between 0·5 and 2 are the ones most commonly reported in analyses of real-world data, and are those most susceptible to unaccounted for confounding that can only be addressed with an RCT. RCTs are therefore irreplaceable, and indeed the plethora of large-scale RCTs done over the past 10 years has dramatically transformed the care of people with Real-world studies no substitute for RCTs in establishing efficacy