(W-133) Losing the Forest: Causal Shapley Additive Explanations for Interpretation of Population-Pharmacokinetic models

Elias Clark, Ph.D. – Research Associate II, Metrum Research Group; Curtis Johnston, Pharm.D. – Group Leader PKPD, Principal Scientist II, Metrum Research Group

Objectives: Forest plots, typically formulated as Certeris Paribus, are a commonly used tool to understand covariate relationships in population pharmacokinetic (Pop-PK) modeling. However, these types of forest plots can be misinterpreted leading to unsupported conclusions, particularly for dosing decisions in clinically-relevant subpopulations. This is often due to the “Table 2 Fallacy” where parameter estimates are conflated with causal effects [1]. To address these limitations, we applied multiple types of Shapley Additive Explanations (SHAP), a tool from the field of interpretable machine learning, to Pop-PK models to compare conclusions from different methods and visually interpret model inferences.

Methods: A Pop-PK model was developed with simulated covariates. Covariate distributions were simulated with confounding and mediating variables to mimic the known and assumed biological dependencies between common covariates in pharmacometric analyses and were represented in a causal graph. Forest plots, SHAP, causal SHAP [2], and population simulations were used to interpret the covariate relationships in the model through the effect on exposure summary metrics. The different methods were compared by identifying null versus meaningful covariate effects and recovering the known covariate effect.

Results: Forest plots only captured the direct effects of the covariates on the model and only with respect to the reference values, mimicking a table of parameter estimates. SHAP captured the inter-patient variability in direct covariate effects due to covariate interactions in the nonlinear model. In addition, causal SHAP and population simulations captured indirect effects. Causal SHAP also estimated the total causal effect of a covariate while the population simulations showed non-causal associations between covariates and concentration. Covariates with a null effect in a forest plot had substantial effects in a SHAP analysis and population simulations with the subgroups defined by these covariates having differences in exposure despite null effects in the forest plot.

Conclusions: SHAP characterized covariates in Pop-PK models and yielded different insights than forest plots and population simulations. Population simulations within clinically-relevant subpopulations remain the most applicable method to understand dose adjustments. Forest plots were misleading with respect to the need for dose adjustments and inter-patient variability. Causal graphs and the assumptions they encode contextualized the different conclusions between analysis methods. They can provide an additional perspective on model assessment beyond the current standards of practice and are easily adapted into workflows.

Citations: [1] Westreich D, Greenland S. The table 2 fallacy: presenting and interpreting confounder and modifier coefficients. Am J Epidemiol. 2013 Feb 15;177(4):292-8. doi: 10.1093/aje/kws412. Epub 2013 Jan 30. PMID: 23371353; PMCID: PMC3626058.



[2] Heskes T, Sijben E, Bucur IG, Claassen T. Causal Shapley Values: Exploiting Causal Knowledge to Explain Individual Predictions of Complex Models [Internet]. arXiv [cs.AI]. 2020. Available from: https://arxiv.org/abs/2011.01625