LAUSR

We thank Mackay et al. [1] for their comments on our editorial [2], which accompanied the article by Chiu et al. [3] and agree with the definition of external validation that they provide. In broad terms, internal validation is concerned with reproducibility of a predictionmodel, whereas external validation is concerned with the transportability of model predictions to other settings and populations. Randomly selected patients whose data are not included in the training data used to develop the prediction model and used for validation are similar to, and subject to, the same sampling processes as patients in the development dataset. Internal validation based on such data will address overfitting due to selection of predictor variables and parametrisation of their associations with the outcome, but does not address transportability to different settings. We applaud Mackay et al. for conducting a range of validation exercises; given the importance of validation we encourage authors and editors to include results of such exercises in the main text of published papers rather than in supplementary material. From our reading of the manuscript and supplementary material, it appears that no population from a different source was used to externally validate their model. The populations used for validation were subsets of data from the same four hospitals used in model development (some excluded from the development data set), either: (1) using three hospitals other than Papworth; (2) using data from all four hospitals in 2017; and (3) using the temporally ‘last 10%’ of data from each patient. These discussions illustrate that there is a continuum within the dichotomy ‘internal validation’ and ‘external validation’ [4].