Machine Learning-Based Multimodal Molecular Biomarkers for Predictive Health Analytics

Summary

Machine learning with multimodal biomarkers enhances disease prediction and monitoring, offering promise for healthcare advancements in various domains, improving healthcare outcomes through accurate disease prediction.

Abstract

Every year, many people around the world are progressively affected by the devastating conditions of health problems such as heart disease, respiratory infections, neurological dysfunction, cognitive stress, cancer, stroke, diabetes, etc., which lead to severe health complications and associated abnormalities. Thus, early health analytics are crucial, as they enable timely intervention with targeted therapies, potentially providing immediate relief and sustained long-term benefits that may slow disease progression. Due to the complex pathophysiological processes and heterogeneous clinical trials in various health conditions, there is a need for highly sensitive, multimodal biomarkers and effective investigative approaches to accurately detect and monitor patient health outcomes. Therefore, machine learning algorithms with various categories and techniques are considered for predicting outcomes, including prognosis, risk assessment, patient stratification, and disease monitoring. The flow of the proposed work is divided into three stages, as the first stage defines the importance of healthcare with case studies, followed by the traditional Machine Learning (ML) algorithms, traditional Deep Learning (DL) approaches, and modern DL techniques (TabNet and AutoInt) in the second stage. Finally, the experiments are implemented to justify the results. This work highlights the grouping of modalities by integrating molecular protein, chemical, and genetic biomarkers with emerging ML features. The results indicate a significant improvement in predicting the accuracy using the proposed methodology.

Introduction

The healthcare system is increasingly utilizing predictive analytics to monitor patients and diagnose health outcomes in a timely manner, facilitating proactive care and improving patient outcomes. By integrating predictive analytics with biomarkers, clinicians can enhance diagnostic accuracy, identify effective treatments, monitor therapy progress, and gain valuable insights into disease mechanisms, ultimately leading to more informed and effective treatment decisions. Biomarkers refer to the characteristics of biological molecules or indicators found in samples, such as biofluids, tissues, cells, DNA, RNA, blood, and urine, that measure the presence or progression of a disease in the human body, helping to assess the effectiveness of treatment¹.

The advances in the application of molecular biomarkers play a crucial role in personalized medicine by precisely measuring specific molecules to identify unique health problems in patients, enabling targeted treatment strategies from the initial stages². Moving forward, multimodal approaches will help integrate multiple modalities of complex disease patterns, enabling more accurate identification of disease prognosis in cancer and neurodegenerative diseases through advanced predictive analytical techniques³. The cutting-edge methodology utilizes multi-omics data, bioinformatics, and ML algorithms to identify novel biomarkers, enabling patient-specific risk profiling and subjective treatment strategies for cardiovascular diseases (CVDs) and other complex diseases⁴. The combination of molecular biomarkers and multi-omics approaches holds promise for enhancing diagnostic accuracy and therapeutic stratification in neurological diseases⁵.

With the advancement of ML techniques, multimodal molecular biomarkers can integrate diverse data types to identify novel disease signatures in various health conditions, enabling more accurate diagnosis and personalized treatment strategies. Building on this potential, the author explores various ML techniques in healthcare to predict disease diagnosis risk factors and highlights their importance in different healthcare sectors⁶. With ML transforming disease diagnosis and treatment, pharmaceutical practices are increasingly leveraging decision-making algorithms to analyze patient health outcomes and cater to their daily needs, enabling more informed and effective care⁷. Leveraging the potential need of multimodal data integration, this study developed biomarkers using multimodal data (MRI and genetic) to predict the development and progression of Alzheimer's disease, showing that genetic data better predicted future Alzheimer's progression in normal control subjects, while MRI data better characterized stable mild cognitive impairment, and combining both, better-quality classification performance⁸.

Further advancing the application of multimodal data analysis, multimodal deep learning utilizes gene expression data to identify drugs that target specific cell lines of novel biological molecules, thereby elucidating how genomic variations influence treatment response⁹. In conjunction with advances in multimodal data analysis, non-invasive biomarkers, evaluated using AI and ML metrics, are emerging as promising tools for diagnosis, with varying specificity and sensitivity profiles compared to invasive biomarkers, contingent on the specific application and data context¹⁰. In line with the growing importance of multimodal data analysis, Deep Learning (DL) ensemble models can effectively handle complex data by integrating novel biomarkers, including gene-protein interactions, to enhance cancer research and optimize treatment strategies¹¹. As DL ensemble models continue to advance cancer research, predictive analytics plays a crucial role in disease diagnosis and treatment by analyzing large amounts of collective data from multiple sources, including diverse health conditions, to provide a comprehensive overview of the status, such as risk assessment and diagnosis¹².

The objective of the proposed protocol is to implement ML-based and DL-based algorithms that integrate the distinct biological markers to enhance the accuracy of healthcare analytics. Conventional prediction techniques that utilize biomarkers typically rely on single-modality data and linear statistical models, which may not adequately capture the intricate, nonlinear relationships that influence disease development and individual differences. The incorporation of vast amounts of data from observations, e-health records, laboratory readings, physical measurements, and clinical assessments provides a significant opportunity to synthesize and optimize risk assessment in patient diagnosis¹³. Biomarkers play a crucial role in precision medicine, enabling targeted treatment at the right time. While biomarkers are complex and pose challenges in measuring disease subtypes and molecular growth, they are invaluable in assessing toxic reactions under various abnormal conditions, thereby facilitating further analysis. By utilizing biomarkers during clinical observation, healthcare professionals can more accurately predict disease progression and monitor treatment responses. Ultimately, the convergence of multimodal molecular biomarkers in healthcare analytics with AI enables more effective pattern recognition, matching existing characteristics to mitigate clinical impact.

When compared to state-of-the-art approaches, especially the work of Somepalli et al.¹⁴, where the authors proposed ML algorithms on genome data, such as metabolic, epigenetic, and proteomic data, and proposed general guidelines for selecting ML algorithms. However, there is a limitation on the analysis of the data. In the existing work¹⁵, unsupervised integration methods were applied to omics data, focusing on computational challenges. In contrast, there are limitations on the scope and analysis of the methods. Furthermore, Huang et al.¹⁶ covered most of the DL methods for analyzing diverse applications, aiming to understand the power of big data and computational frameworks. The drawbacks included data imbalance, overfitting, and interpretation issues. The proposed protocol is crucial for addressing a critical gap in integrative biomarker research, as current techniques fail to effectively combine high-dimensional and diverse biological data. By utilizing sophisticated ML and DL that excel in pattern recognition, feature selection, and multimodal fusion, the proposed submission aims to identify robust predictive signatures that enhance early diagnosis, prognosis, and individual treatment options. This protocol directly addresses the limitation of disjointed biomarker assessments by presenting a comprehensive, data-driven model that enables scalable, interpretable, and clinically relevant health predictions.

Protocol

1. Experimental workflow

Figure 1 illustrates an organized and modular workflow pipeline designed to categorize molecular biomarkers into Risk or Diagnostic indication categories using machine learning and deep learning techniques. Here is a detailed step-by-step explanation of the process:

1. Dataset acquisition
   The dataset is collected from MarkerDB 2.0, a curated database of molecular biomarkers.
2. Data preprocessing
   This ensures data quality and consistency by handling null values and missing entries, encoding categorical features using LabelEncoder, and scaling numerical features (e.g., entrez_gene_id) using StandardScaler.
3. Feature engineering
   Selecting relevant features that influence biomarker classification, prevent data leakage, and improve model performance is recommended.
4. Model development
   The approaches used are Logistic Regression, Random Forest, XGBoost, whereas in DL: MLP, MLP with LR Scheduler, AutoInt, TabNet. The activities performed are Hyperparameter tuning, Cross-validation, and Optimization for classification tasks.
5. Data splitting
   Here, the training Set (80%) is used for learning model parameters, and the testing Set (20%) is used for evaluating the model on unseen data. At the final stage, evaluation metrics are used to ensure performance is measured holistically, especially in imbalanced class scenarios.

Figure 1: Flow diagram of the proposed work. The workflow of the proposed problem is depicted in this figure. The steps are data preprocessing, feature engineering, model development, splitting, model evaluation using metrics, and biomarker analysis. Please click here to view a larger version of this figure.

2. Dataset description

The dataset name is all_sequence_variants with csv extension. There are columns with names as 'id', 'variation', 'position', 'external_link', 'gene_symbol', 'entrez_gene_id', 'reference', conditions', 'indication_types', where all are object type except id as int type and entrez_gene_id as float type. The total number of rows are 42818 with 9 columns. Further, the dataset has null values for variation, position, external_link, entrez_gene_id, reference, conditions, and indication_types. The dataset is taken from the link given as: https://markerdb.ca/downloads¹⁷

3. Dataset preprocessing

At this stage, data preprocessing begins with gathering dataset information, description, identifying duplicates, and locating missing/null values. There is no impact of the external_link feature when measured by the correlation coefficient, and hence it is removed. The null values of categorical columns and numerical columns are filled with the median and mean, respectively. Finally, the dataset shape of input and output is (42787, 6)(42787,). The input columns are 'id', 'variation', 'position', 'gene_symbol', 'entrez_gene_id', 'conditions', whereas the target column is indication_types.

4. Biomarkers: Categories, data challenges, and the role of molecular risk

A biomarker is a quantifiable trait that signifies either normal or abnormal biological activities or responses to various exposures or interventions. Biomarkers may be molecular, histological, radiographic, or physiological in nature and are divided into seven primary categories: (1) A diagnostic biomarker determines whether an individual has a specific disease or disorder and if it falls within a subtype, (2) A monitoring biomarker indicates the progression of a disease and its response to treatment, (3) A response biomarker shows whether a patient is reacting to a medication or therapy, such as a change in heart rate, (4) A predictive biomarker can ascertain whether an individual is at risk of developing a certain disease and how they may respond to a particular treatment, (5) A prognostic biomarker can offer insights into the likelihood of disease progression or recurrence if a patient is predisposed to a specific health outcome, (6) A risk biomarker assesses the potential level of risk for a condition before a patient receives an official diagnosis, and (7) A safety biomarker evaluates the potential for toxic or adverse reactions that may arise from treatment. This study primarily focuses on the analysis of molecular biomarkers related to risk assessment and diagnostics, as given in Figure 2. Given the vast amount of clinical data accessible worldwide, biomarkers are essential in guiding clinical decisions, advancing research, and facilitating personalized medicine.

Figure 2: Biomarker category for clinical-decision making. The decisions of the biomarker are represented in this diagram. Based on the analytics applied to the biomarkers, clinical decisions are considered, and ML algorithms are implemented. Please click here to view a larger version of this figure.

Finding and comprehending clinically useful biomarker information is challenging due to the wide range of biomarker types, characteristics, and quality. The rapid growth of biomarker studies over the past two decades has further complicated access to comprehensive and up-to-date data, especially for molecular biomarkers. This has led to issues such as inconsistent findings, redundant rediscovery of existing biomarkers, contradictory results, and overlooked opportunities to utilize established biomarkers¹⁸. The emergence of "omics" measurements has exacerbated this problem, resulting in a proliferation of novel molecular biomarkers in clinical testing and the literature, which makes it increasingly difficult to navigate and apply biomarker knowledge effectively¹⁹.

MarkerDB 2.0 provides access to diverse molecular biomarkers for experimental purposes²⁰. Various biomarkers are utilized in clinical research and healthcare applications to diagnose, predict, and monitor diseases. Among these, protein biomarkers use biofluids such as serum, blood, cerebrospinal fluid (CSF), peritoneal fluid, amniotic fluid, plasma, and urine to analyze conditions like diabetes mellitus, tuberculosis, Down syndrome, and myopathy-related disorders. Similarly, genetic biomarkers use gene symbols like LRP1, LPP, MAPT, and SPI1 to analyze conditions such as age-related macular degeneration, allergic diseases, Alzheimer's disease, cancer, and asthma. Furthermore, multimodal molecular biomarkers help measure patient risk and diagnostic indicators²¹. This work highlights the need for healthcare analytics using ML approaches to enhance disease diagnosis, prediction, and personalized treatment.

5. Modelling statistical ML approaches for biomarker discovery

The application of ML approaches has transformed the healthcare field in numerous ways. Specifically, molecular biomarkers have been analyzed using various ML techniques, including Principal Component Analysis (PCA), K-means clustering (KMA), Nearest Neighbor Analysis (NNA), and neural network algorithms²². These models identify intricate patterns from chosen features, manage incomplete or inconsistent data, and support real-time tracking of disease advancement. In addition to diagnosing diseases, ML algorithms provide significant insights into patient issues through data examination and visualization, enabling prompt and targeted care²³. As a result, this enhances patient health outcomes in unusual conditions. Furthermore, the capacity of ML to revolutionize biomarker discovery and therapeutic results across various diseases, including the analysis of molecular biomarkers, is becoming increasingly vibrant.

ML algorithms are categorized into five main types, as given in Figure 3. Supervised learning is the process of training on labeled datasets to understand input-output relationships, making it ideal for tasks such as classification and regression, like decision trees and support vector machines. Unsupervised learning identifies patterns within unlabeled data, and is often applied for clustering and dimensionality reduction, with techniques such as k-means clustering and principal component analysis. Semi-supervised learning merges both labeled and unlabeled data to generate predictions. Reinforcement learning focuses on decision-making informed by feedback from the environment, and is frequently used in gaming and robotics, with methods such as Q-learning and deep reinforcement learning networks. Deep learning employs artificial neural networks with multiple layers to identify intricate patterns in data. This technique aids in predicting biomarkers for common diseases, including cancer, stroke, Alzheimer's disease, and Parkinson's disease. Numerous clinically approved applications make use of this technology^24,25. The precise diagnosis of patients with malignant tumors typically depends on the acquisition and pathological analysis of tissue samples, which yield critical information regarding histological grade, subtype, stage, and various other tumor biomarkers.

Figure 3: Categorization of Machine learning algorithms for clinical-decision making. The types of ML algorithms in the diagram are illustrated to help understand the methodology applied. Please click here to view a larger version of this figure.

Statistics and ML are two separate disciplines that frequently intersect. Statistics involves gathering, analyzing, and interpreting data, typically working with smaller, well-defined datasets, and is primarily focused on comparing and summarizing information. On the other hand, ML is a subset of artificial intelligence that creates predictive models, often dealing with large and intricate datasets. While statistics serve an important role in confirmatory research and comprehending underlying mechanisms, ML is more effective in exploratory research, uncovering complex patterns, and managing missing data. Methods such as logistic regression can be regarded as both statistical models and supervised ML models, illustrating the blurred distinctions between the two domains. Ultimately, researchers must evaluate their research objectives and the nature of their data to select the most suitable method.

6. Modeling MLP and Variants of MLP

MLP, also known as a multilayer perceptron, is a type of neural network architecture featuring multiple neuron layers between the input and output layers, as illustrated in Figure 4. MLP was introduced in the 1950s and gained widespread use with the progress and advancements in backpropagation in the 1980s, which helped minimize the loss. The basic learning principles of MLP are neural nets, weights, and bias values for computing the output. Finally, to know the activation function and threshold value to get the output in the output layer. In the proposed methodology, the nodes in the input layer, hidden layer, and output layer correspond to clinical and gene features, fully connected layers with ReLU activation, and a single neuron with sigmoid activation to predict the binary outcome of indication (Risk, Diagnostic) type, respectively. The major advantages are that they are simple to train and interpret. However, the simple MLP cannot handle large datasets and is sensitive to the learning rate²⁶. Therefore, to overcome these drawbacks, MLP with Learning Rate Scheduler is considered with the architecture given in Figure 3B. MLP with LR is a powerful dynamic mechanism with steep decay, exponential decay, inverse time decay, ReduceLROnPlateau, and cosine annealing. Instead of a fixed LR value, the learning rate changes based on the model performance and training.

Figure 4: Modeling multi-layer Perceptron and MLP with an LR scheduler. (A) Multi-Layer Perceptron: The structure of MLP is illustrated with Input Layer, Hidden Layer, and Output Layer. The first layer receives the input and processes it in the second layer with activations, and the output is received in the last layer. MLP is simple to implement. (B) MLP with LR Scheduler: This is similar to MLP; however, a Learning Rate Scheduler is added to control the training speed for a better convergence rate. The learning rate is reduced at a plateau. This reduces overshooting minima. Please click here to view a larger version of this figure.

This approach is more efficient, reliable, and effective for tabular data. Additionally, there are benefits such as stable convergence improvement on noisy data, improved generalization, and reduced effort in hyperparameter tuning. However, for some prediction tasks, difficulties may arise, such as getting stuck in local minima. Furthermore, in the wide and deep MLP architecture presented in Figure 5, wide and deep components are introduced in the MLP to excel in correlation mapping, memorization of rules, and learning new patterns, while maintaining high standards of generalization. By combining these aspects into the output layer, the binary output is predicted²⁷. However, there are limitations to the generalization of unseen features and overgeneralization. In conclusion, MLP with Batch Norm and Dropout is used to normalize the training process, incorporating dropout for learning and generalization of unknown attributes. In particular, this technique can be applied to high-dimensional data and helps prevent overfitting. Nevertheless, this technique suffers from batch size limitations, increased memory consumption (due to Batch Normalization layers), and is not a universal solution^28,29.

Figure 5: Modeling MLP wide and Deep network. The diagram illustrates the deep path of the MLP, which can capture nonlinear relationships and combines the output with a sigmoid unit for classification. This architecture captures pattern learning and knowledge for heterogeneous biomarker data. Please click here to view a larger version of this figure.

7. TabNet

The TabNet is a deep learning model known as Tabular Network developed by the Google Research team³⁰. The main motivation of the TabNet model is to reconcile differences between the DL approaches for image, text, speech (CNN, Autoencoder, Transformers), and tree-based ML (XGBoost, Random Forest, LightGBM) approaches. Furthermore, the limitations of DL approaches have a huge impact on the TabNet model, with MLP being a parameterized model. Above all, the DL approaches lack interpretation in real-world problems, self-supervised learning, and sequential learning capacity³¹. The TabNet models are confined to applications such as insurance risk prediction³², coal ash content estimation³³, chemical toxicity prediction³⁴, and educational test cheating detection³⁵. The architecture of TabNet is given in Figure 6.

Figure 6: Modeling tabular network. The Tabular network takes input features in a tabular format and applies the transformer technique in a sequential manner in each block. The GLU block, also known as the Gated Linear Unit block, controls the information flow forward through the network, facilitating feature selection and stable learning. The output is presented in the output layer from the accumulated output. Please click here to view a larger version of this figure.

From the above architecture diagram, the system is divided into four components: Input block, Shared Feature Transformer, Sequential Decision Steps, and the loss calculation using optimization. In the categorical features, the features are converted to integers following the embedding layer. At the same time, the continuous features are taken as they are. Finally, the features are converted into a unified feature vector where x ∈ Rd. In the next step, fully connected layers with activation functions are implemented, followed by a transformation to convert the data into a vector, and a decision is made. In the third step, the attention transformer, masking, decision transformer, and prediction accumulation are used in succession. The objectives of this step are selective attention, sequential reasoning, and decision pathways. In the last step, either binary cross-entropy or similar methods are used to find the loss and optimize the value. Further, regularization is performed using sparse attention to prevent overfitting.

8. AutoInt

Automatic Feature Interaction Learning via Self-Attentive Neural Networks (AutoInt) is the full form of AutoInt, introduced in Click-through rate (CTR) prediction³⁶ and later applied to various applications, including software defect prediction³⁷, recommendation systems³⁸, and Genome detection³⁹. There are limitations in the existing ML approaches, including handling high-dimensional sparse data, higher-order combinatorial features, understanding the feature's capability in prediction, and the need for manual feature engineering. All these mentioned challenges are addressed by the AutoInt model with efficiency, effectiveness, and explainability. The objective of AutoInt in the proposed methodology is to predict whether the given variant is of a Risk or Diagnostic indication type. There are seven steps proposed for the given dataset, as shown in Figure 6, along with the source code. Firstly, reading data and preprocessing are implemented to encode all categorical features as integers and map the labels to 0 and 1 for Risk and Diagnostic, respectively. In the second step, feature representation is performed to create the embedding matrix; in turn, the matrix is converted to normalize the numeric features. There are interacting layers where the neutral automatic interaction among the features is learned without manual interaction with the transformers. Furthermore, multiple heads learn different types of interactions, and finally, the features are concatenated, enhancing the representation. Residual connections are present to learn both lower- and higher-order interactions, such as 1 feature, 2 features, 3 features, and so on. Finally, from the residual network, outputs are passed to the sigmoid function for binary value computations, as shown in Figure 7.

Figure 7: Modeling AutoInt network. The AutoInt Network automatically learns features, where the embedding layer takes the features and maps them into the next layer using a self-attention mechanism and residual connections. The last component has the MLP layer and activation functions to produce the output. Please click here to view a larger version of this figure.

Results

The experiments are conducted using ML and DL approaches to compare their performance. At the same time, creating ML models that utilize both supervised and unsupervised learning mechanisms is considered. The supervised techniques used in this methodology are Logistic Regression, Decision Tree, Random Forest, Gradient Boost, Support Vector Machine, and K-Nearest Neighbor algorithms. These algorithms range from simple statistical models to complex tree-based analyses. On the other hand, unsupervised ML algorithms are K-Me...

Discussion

The proposed work clearly discusses the importance of biomarkers in the identification and monitoring of a disease and its characteristics. The suggested approach for biomarker methodology is based on four essential steps: careful data preparation (cleaning, encoding, eliminating identifiers); uniform target encoding (Risk=0, Diagnostic=1); normalization of features and high-quality deep embedding for models such as AutoInt³⁷/TabNet; and a solid evaluation process integrity (appropriate train/test...

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
AutoInt Model	Peking University	https://github.com/shichence/AutoInt	Automatic Feature Interaction Learning via Self-Attentive Neural Networks: Efficient algorithm to automatically learn high-order feature interactions for (sparse) categorical and numerical features
deepctr_torch.models	Open source	https://github.com/shenweichen/DeepCTR-Torch	Modular and Extendible package of deep-learning based CTR models to build own custom model easily.
Google Colaboratory	Google	https://colab.research.google.com/	Cloud-based  environment to write and execute Python code through the browser for machine learning, data analysis
Keras 2.6.0	Keras	https://keras.io/	Provides a Python interface for executing neural networks which runs on top of Tensorflow
MarkerDB 2.0	MarkerDB	https://markerdb.ca/downloads	Publicky avilable Database of Molecular Biomarkers
Matplotlib 3.4.3	Matplotlib	https://matplotlib.org/	Visualization library for python
Numpy 1.21.4	Numpy	https://numpy.org/	Fundamental package for scientific computing with Python
Pandas 1.3.4	Pandas	https://pandas.pydata.org/	Powerful, flexible and easy to use open source data analysis and manipulation tool built on top of the Python programming language.
Python 3.8.10	Python Software Foundation	https://www.python.org/	Popular programming language that integrate deep learning systems more effectively.
pytorch_tabnet.tab_model	Pytorch	https://pypi.org/project/pytorch-tabnet/	For implementing binary/multi-class classification and regression problems.
Scikit-learn	Scikit-learn	https://scikit-learn.org/stable/	Python module for machine learning algorithms
Seaborn	Seaborn	https://seaborn.pydata.org/	High level data visualization library for drawing attractive and informative statistical graphics
TabNet Model	Google	https://github.com/dreamquark-ai/tabnet	TabNet is an end-to-end neural network designed to directly handle tabular data
Tensorflow 2.6.0	Google	https://www.tensorflow.org/	An end-to-end platform for machine learning