LAUSR

Introduction In silico models proved to be a promising tool to complement and optimize clinical trials. These models should be validated to assess their capacity to reproduce real life behaviors. Opposed to real-life clinical trials where the amount of available data might be low, in silico approaches give us the possibility to simulate virtual populations of thousands of patients. This data size heterogeneity might be an issue. Moreover, in real life data, patients are seen at scheduled visits, leading to an observation time uncertainty (OTU), which is not the case in simulated data.In the context of the validation of a EGFR-mutant Lung Adenocarcinoma mathematical model, we depicted the interest of using combined validation methods to assess the capacity of the model to predict the time to tumor progression, from heterogeneous clinical trials datasets. Methods In this context, to overcome the limitation of using default log-rank test that could lead to biased results, 4 bootstrapped methods have been used: - 2 methods based on the log-rank test where the ratio of significant tests at a given alpha risk level is assessed, taking into account the OTU: The “default Log-rank test” and the “modified test” based on a combination of weighted log-rank tests (MaxCombo). - 2 methods based on prediction intervals. The “raw coverage”, which corresponds to the proportion of the observed time-to-event curve included in the prediction interval and the “juncture metric” which corresponds to the proportion of the observation period where the prediction interval overlaps with an interval bound between the observed data and the same data shifted by the OTU. Results As each one of the validation approaches has its own strengths and weaknesses and conditions of application, the validation process has been based on the combination of well selected methods. Validation metrics showed that the model reproduces successfully real life data depending on a given context. When all the metrics give the same result, we were able to easily conclude. But when it was not the case, this led us to investigate the origin of these discrepancies, taking into account method robustness. Conclusion The validation process is of the utmost importance to assess the level of credibility of the model. We showed that in order to fully evaluate the model, a combination of validation approaches is preferable. The use of multiple metrics highlights the eventual inconsistencies in model predictions and validation dataset’s content that would not have been detected by only a single approach. The drawback of combining metrics is that it complexifies the decision making, implying that the final decision, regarding whether the model is validated or not, is assumed by the user, without forgetting that the validation is not an end objective per se but a stepwise process. Citation Format: Evgueni Jacob, Laura Villain, Nicoletta Ceres, Jean-Louis Palgen, Adele L'Hostis, Claudio Monteiro, Riad Kahoul. The combination of statistical methods to compare observed and simulated data allowed to assess effectively the validity of mathematical model predictions in a context of a EGFR+ lung adenocarcinoma. [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 1 (Regular and Invited Abstracts); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(7_Suppl):Abstract nr 4287.