SHUAI SHAO, PhD: No relevant disclosure to display
Introduction: Model-informed drug development (MIDD) plays an important role in pharmacometrics by combining mathematical models with clinical [1]. Traditional methods such as Nonlinear Mixed Effects Modeling especially developed from NONMEM software have long been the gold standard for population PK analyses. However, the development of artificial intelligence (AI) presents a potential improvement in predictive accuracy and computational efficiency [2]. In this study, we aim to benchmark popPK analysis by comparing the performance of NONMEM with AI/ML models in simulating PK exposure.
Methods: A virtual patient dataset was developed through simulations using the mrgsolve package, based on a two-compartment model with linear elimination and first-order absorption. This dataset incorporates various dosing regimens to mimic intensive PK sampling generally implemented in clinical trials. For the AI-based approach, we employed five machine learning (ML) models: Linear Regression, LASSO, XGBoost, Light Gradient Boosting Machine (LGBM), and Categorical Boosting (CatBoost). These models were trained to predict PK exposure ((AUC, Cmax, Ctrough) on inputs such as time, dose, and PK concentrations. Hyperparameter grid search was conducted for LASSO, XGBoost, LGBM, and CatBoost to enhance model performance. The evaluation was carried out using 10-fold cross-validation and included metrics such as the root mean squared error (RMSE), mean absolute error (MAE), and the coefficient of determination (R^2).
Results: Our findings indicate that AI-based models consistently outperform NONMEM in terms of RMSE and MAE, showing better predictive accuracy. However, NONMEM provides higher R^2 values, suggesting greater interpretability within the context of the current virtual dataset. AI models exhibit lower error rates in predicting concentrations, while NONMEM maintains a superior ability to incorporate physiological processes that govern drug kinetics. Additionally, when comparing PK exposure results from simulations across various dosing regimens, AI/ML approaches and NONMEM yielded comparable outcomes.
Conclusion: Our comparative analysis highlights the complementary strengths of NONMEM and AI-based models in MIDD. The various ML algorithms underscore their potential to enhance predictive accuracy in drug development and NONMEM provides the ability to characterize mechanistic aspects of drug behavior. This study provides evidence for further integration of AI in MIDD and highlights the need to balance accuracy with interpretability in pharmacokinetic modeling., this study shows the importance of combining AI-driven approaches and traditional modeling to improve MIDD in precision therapeutics. Disclosures: Mao B, Shao S, Gao Y, Xu C, Macha S, Ahamadi M: Sanofi - employees. may hold stock and/or stock options in the company of Sanofi.
Citations: [1]Wang, Yaning, et al. "Model‐informed drug development: current US regulatory practice and future considerations." Clinical Pharmacology & Therapeutics 105.4 (2019): 899-911. [2]Liu, Qi, et al. "Application of Machine Learning in Drug Development and Regulation: Current Status and Future Potential." Clinical Pharmacology & Therapeutics 107.4 (2020).
