Merging Multimodal with Longitudinal data: Leveraging Graph Neural Networks and Neural-ODEs

Precision medicine aspires to identifying and adjusting treatments for patients depending on the available biomarker data and their therapeutic response. To fulfill this mission, it is essential to construct predictive dynamical models that can seamlessly integrate both longitudinal data as well as high-content data that are possibly multimodal in nature. Since biological and disease knowledge often come in the form of graphs, we propose an innovative approach that merges multimodal data with longitudinal data by leveraging graph neural networks (GNNs) and neural ordinary differential equations (Neural-ODEs). We applied this methodology to an extensive set of patient-derived xenograft (PDX) data, which encompasses a broad spectrum of treatments and their combinations as well as tumors originating from several different organs. The findings from this work highlight the benefits of leveraging a modeling paradigm that is able to discover the dynamical equations from large data sets, resulting in enhanced predictivity of tumor dynamics. Furthermore, it demonstrates the graph encoder's ability to effectively use multimodal data and improve tumor predictions. This work showcases the promise of combining GNNs with Neural-ODEs in offering a deep learning approach to advance new frontiers by discovering dynamical models that have the potential to be causal, generalizable and explainable.