LAUSR

We read with interest the paper by Martin et al published in Statistics in Medicine.1 The aim of study was to compare four models including: probabilistic classifier chain, multinomial logistic regression, multivariate logistic regression, and a Bayesian probit model to develop an appropriate model for prediction of multiple binary outcomes. A simulation study and a real-world example were used to assess the marginal and joint risk prediction of multiple outcomes. As a result, the authors claimed that all methods had similar performance to predict the marginal risk of each outcome. Also, they suggested that probabilistic classification chains, multinomial logistic regression or the multivariate probit model might be the most suitable method for developing multivariate clinical prediction models. In our opinion, there are some important methodological limitations regarding the comparison of models in prediction of multiple binary outcomes. First, comparing the performance of marginal risk models has some limitations. The aim of clinical prediction positively impacts clinical decision-making and subsequent patient outcomes, so focusing on the individual risk prediction and using absolute risk prediction are essential criteria that should be considered. It should be noted that the marginal risk is different for each individual2; however, the average marginal effect is reported in these statistical methods. So, despite the type of statistical models that were used, this issue leads to restrict the usefulness of prediction models in decision making especially in personalized medicine. Moreover, the marginal effect is influenced by the role of other variables and the term “interaction” should be considered for both marginal and joint risks.2 Without assessing interaction any result is imprecise and consequently, diminishes the clinical value of prediction models.3 Second, an important assumption that is considered in these statistical methods is independence between different outcomes, while there is no way to deny the role of both additive and multiplicative interaction between different outcomes and predictors. In fact, existing interaction between predictors or even outcome makes a difference in predicting the consequences of changing the value of a predictor, particularly if the predictors are hard to measure or difficult to control.4 Third, using internal validity (model reproducibility) is not sufficient to determine the best model for prediction, since the results of internal validity are often optimistic (overestimated in performance in other individuals) because two datasets are similar. It should be noted that without external validation (transportability), the authors cannot claim that these results can generalize to samples from different but related source populations.5,6 Finally, there is a key point that should be considered by the authors, namely the impact of different types of outcomes (long term vs short term) on the performance of models.