A Multimodal Data Analytics Framework for Early Cancer Detection Using Genomic, Radiomic, and Clinical Big Data Fusion

Abstract

Early detection of cancer has a great impact on increasing patient survival, on reducing treatment intensity and on lowering healthcare costs in the long term. Despite advancements in genomics, imaging, biomarkers and EHR analytics, it's difficult for current tools, due to the various sources of data. This makes it difficult to detect cancers when patients have few, if any, symptoms. In order to address this issue, we propose a Multimodal Data Analytics Framework (MDAF). It integrates genomic sequences, imaging features extracted using radiomics and large clinical data sets. It involves AI and deep-learning pipelines that handle various types of data. The framework has performed data ingestion, automatic feature extraction, hierarchical harmonization, transformer-based fusion, predictive modeling and explainable AI (XAI). It also operates with the privacy-preserving technique of federated learning. Previous studies indicate that data combination from multiple types yields better results than a single-type combination. The accuracy, sensitivity, and specificity can on average be improved by 10-35% using multimodal fusion in different cancers. MDAF is based on the principles of precision oncology, and applicable in real hospitals with their information systems. It is conducive to personalized and evidence-based decisions for clinicians. Future work involves construction of the digital twin simulations for oncology, national federated research registries, and multimodal biomedical knowledge graphs.