LAUSR

To cite: Chiocchia V, White IR, Salanti G. BMJ EvidenceBased Medicine Epub ahead of print: [please include Day Month Year]. doi:10.1136/ bmjebm-2021-111904 © Author(s) (or their employer(s)) 2022. Reuse permitted under CC BY. Published by BMJ. In clinical fields where several competing treatments are available, network metaanalysis (NMA) has become an established tool to inform evidencebased decisions. 2 To determine which treatment is the most preferable, decisionmakers must account for both the quantity and the quality of the available evidence by considering both efficacy and safety outcomes as well as assessing the confidence in the obtained results. It is, however, increasingly common to include in the NMA output a ranking of the competing interventions for a specific outcome of interest. This article focuses on this type of rankings. A hierarchy of treatments (or ranking) is obtained by ordering a specific ranking metric. A ranking metric is a statistic measuring the performance of an intervention and is calculated from the estimated relative treatment effects and their uncertainty in NMA. A commonly used ranking metric is the point estimate of the relative treatment effects against a natural common comparator such as placebo. The rankings are unaffected by choice of comparator, so any comparator may be chosen. Other commonly used metrics are the probability of producing the best outcome value, pBV (sometimes called probability of being the best), and the surface under the cumulative ranking curve (SUCRA) or their frequentist equivalent, the Pscore. Treatment hierarchies are a simple and straightforward way to display the relative performance of an intervention and aid the decisionmaking process, so nowadays most publications and reports present rankings. Furthermore, new ranking metrics are being developed to obtain treatment hierarchies that account for important clinical and methodological aspects, such as multiple outcomes (benefits and risks), clinically important differences and the quality of the evidence. Ranking metrics have been criticised in the literature for their lack of reliability, quoting, among other issues, limited interpretability and ‘instability’. This criticism was based on the disagreement between hierarchies obtained by the different ranking metrics. Consider for example the different treatment hierarchies in figure 1 obtained by different ranking metrics for a network of nine antihypertensives for primary prevention of cardiovascular disease 13 (network graph shown in figure 2). The treatment hierarchy based on pBV disagrees markedly with the other hierarchies, based on relative treatment effects and SUCRA, particularly with respect to the top treatment. Conventional therapy, an illdefined treatment which was evaluated in only one trial, is in the first rank in the hierarchy based on pBV but only in the third/fourth and sixth rank in the hierarchies according to the relative treatment effects and SUCRA, respectively. Although such examples can occur, a recent empirical study showed that they are rather rare and that in general there is a high level of agreement between the hierarchies produced by the most common ranking metrics. Agreement becomes less when, as in the network of antihypertensives, there are large differences in the precision between the treatment effect estimates. These differences in precision could be produced by different data features, such as sparse or poorly connected networks, heterogeneity and inconsistency. Disagreements mostly relate to hierarchies based on pBV . Salanti et al also showed with theoretical examples how the uncertainty in the estimation of the relative treatment effects may affect the order of treatments in a ranking. In particular, they observed how rankings based on pBV are more sensitive to differences in precision across treatment effect estimates than those based on SUCRA. When competing treatments have similar point estimates, pBV tends to rank first the treatment with the most imprecise effect (largest confidence or credible interval); a high pBV , therefore, tends Highlights/key points