Enhancing Heart Disease Prediction Through Cluster-Visualized Distributed Machine Learning Paradigms for Secure Healthcare

Summary

This study develops a scalable framework for heart disease prediction using Hadoop MapReduce and K-Means clustering. Adjusting classification cut-offs affects detection sensitivity. CViHDKNN improves true positive detection while controlling false positives, whereas CViHDDT reduces false positives but may miss some heart disease cases.

Abstract

Heart disease remains one of the leading causes of mortality worldwide, creating an urgent need for accurate and scalable predictive systems that enable early diagnosis and timely clinical intervention. Traditional machine learning approaches often struggle to efficiently process large-scale medical datasets and lack interpretability, limiting their usefulness for supporting clinical decision-making. To address these challenges, this study proposes a Cluster Visualized Distributed Machine Learning framework for heart disease prediction. The framework incorporates two distributed algorithms: Cluster Visualized Hadoop Distributed Decision Tree (CViHDDT) and Cluster Visualized Hadoop Distributed K-Nearest Neighbor (CViHDKNN). The proposed models leverage Hadoop’s MapReduce framework for distributed computation across large datasets, while integrating K-Means clustering for improved data organization and visualization. This cluster-based visualization enhances interpretability by allowing clinicians to better understand relationships among patient risk factors and prediction outcomes. Experimental evaluation was conducted using the UCI Heart Disease dataset in a Hadoop-based distributed environment. The results show that CViHDKNN achieved superior predictive performance, achieving 85.25% accuracy and 88% recall, outperforming the CViHDDT model, which achieved 80.33% accuracy. Adjusting classification cut-off values also influenced sensitivity and detection rates: lower cut-offs improved true-positive detection while maintaining acceptable false-positive levels. These findings demonstrate that clustering-enhanced distributed learning improves scalability, predictive accuracy, and clinical interpretability for heart disease prediction.

Introduction

Heart disease is one of the leading causes of death worldwide and represents a major public health challenge. The rising prevalence of cardiovascular diseases highlights the urgent need for advanced diagnostic models that support early detection and treatment. Traditional diagnostic approaches rely heavily on manual evaluation and clinical judgment, which often struggle to manage large volumes of medical data efficiently. Recently, artificial intelligence (AI) and machine learning (ML) techniques have played an important role in predictive healthcare analytics, enabling data-driven disease prediction and improving risk assessment accuracy in medical systems^1,2.

Conventional diagnostic methods and rule-based decision systems often suffer from limited scalability and lower precision when applied to complex medical datasets. Although machine learning techniques have improved prediction performance, many models still face challenges in processing large-scale medical data and maintaining interpretability for clinical decision-making. Deep learning models may achieve high predictive accuracy but often function as “black-box” systems, making it difficult for healthcare professionals to understand the reasoning behind predictions^3,4,5. This lack of transparency limits their adoption in clinical environments where explainability is essential^6,7,8,9,10.

To overcome these challenges, distributed computing frameworks such as Hadoop have been increasingly adopted for large-scale healthcare analytics. Hadoop’s distributed architecture enables parallel processing of large datasets using the Hadoop Distributed File System (HDFS) and MapReduce, improving computational efficiency and scalability in medical data analysis. However, distributed machine learning models still require mechanisms that enhance interpretability and transparency. Integrating clustering and visualization techniques can help reveal hidden patterns within patient data and support clinicians in understanding predictive results more effectively^{11,12,13,14,15,16}.

Several researchers have explored distributed machine learning techniques for heart disease prediction using algorithms such as decision trees and K-Nearest Neighbor (KNN). Distributed Decision Tree (HDDT) models implemented on Hadoop frameworks have demonstrated improved scalability and classification accuracy compared with traditional decision tree approaches^{17,18,19,20,21,22,23}. Similarly, Hadoop Distributed KNN (HDKNN) methods leverage parallel processing to improve classification efficiency for large, high-dimensional medical datasets^24,25,26. However, these models often lack strong interpretability and visualization mechanisms, which are necessary for effective clinical decision-making and understanding patient risk profiles. Despite these advances, existing distributed models still lack integrated mechanisms for interpretability and visual understanding of patient risk patterns.

To address these limitations, this study proposes a Cluster Visualized Distributed Machine Learning (CVDML) framework for heart disease prediction. The framework introduces two distributed predictive models: Cluster Visualized Hadoop Distributed Decision Tree (CViHDDT) and Cluster Visualized Hadoop Distributed K-Nearest Neighbor (CViHDKNN). CViHDDT applies distributed decision tree construction to optimize the selection of medical features, such as symptoms and prior medical history, while CViHDKNN implements a distributed, parallelized KNN approach to classify patients based on similar medical characteristics. The integration of clustering and visualization techniques improves classification performance and enhances interpretability by grouping patients with similar disease patterns.

The main objective of this research is to develop a scalable and interpretable distributed machine learning framework for accurate heart disease prediction using large medical datasets. The key contributions of this study include: (1) Development of a Hadoop-based distributed machine learning framework capable of efficiently processing large healthcare datasets. (2) Integration of clustering visualization techniques with distributed decision tree and KNN models to improve interpretability and transparency in predictive healthcare analytics²⁷. (3) Demonstration of improved prediction performance through enhanced accuracy, recall, and anomaly detection capability compared with traditional machine learning approaches²⁸.

Protocol

Dataset acquisition

The UCI Heart Disease dataset is a widely used dataset in medical and machine learning research for predicting heart disease. It contains various clinical and diagnostic characteristics of patients, enabling healthcare professionals and researchers to develop data-driven prediction models. The dataset classifies individuals as either likely or unlikely to have heart disease based on several patient attributes, including age, gender, chest pain type, blood pressure, cholesterol levels, and electrocardiogram results (https://archive.ics.uci.edu/dataset/45/heart+disease)²⁹. The overall workflow of the proposed heart disease prediction framework, including data preprocessing, distributed model implementation, and evaluation stages, is illustrated in Figure 1.

Experimental environment setups

The experimental environment was deployed on Apache Hadoop 3.x as the core distributed computing framework for all implementations. The cluster used a master-worker architecture with one dedicated master node and multiple worker nodes. The master node managed job scheduling, resource allocation, and cluster coordination using YARN (Yet Another Resource Negotiator), while worker nodes executed distributed computation tasks in parallel to efficiently process large-scale medical datasets. Each node in the cluster was provisioned with Intel Core i7 processors (or equivalent), 16–32 GB of RAM, and approximately 1 TB of storage.

Data ingestion into HDFS

Dataset Storage

The experimental dataset was stored in HDFS in a block-distributed format, with the target variable indicating the presence or absence of heart disease, separated from the independent feature set, prior to storage across cluster nodes. Feature-specific preprocessing was applied to all stored data blocks using MapReduce workflows. Numerical features, including age, blood pressure, cholesterol levels, and heart rate, were normalized using a robust scaler based on the interquartile range, reducing the influence of outliers that are particularly prevalent in medical datasets where extreme values may represent rare or severe clinical conditions. Categorical variables with more than two categories, such as cp, restecg, and thal, were transformed using one-hot encoding, converting categorical attributes into binary numerical representations compatible with machine learning algorithm inputs^30,31,32. All preprocessing operations were executed as distributed MapReduce jobs across HDFS data blocks, ensuring uniform application of the full pipeline without centralizing raw data at any single point.

Partitioning across nodes

The dataset was partitioned into training and test sets using an 80:20 split, with 80% allocated to training and 20% reserved for evaluation on unseen data. This partitioning was applied consistently across all distributed worker nodes to ensure that each node processed a proportional and representative shard of the full dataset, preventing data skew and supporting balanced model generalization. Scaling ensured that all numerical variables contributed equally during distributed training by preventing features with larger magnitudes from dominating the learning process across nodes. This structured partitioning strategy enhanced predictive reliability and helped prevent overfitting by maintaining a clear separation between training and evaluation data across the distributed cluster.

Data preprocessing

Missing value handling

Medical datasets frequently contain incomplete records due to data entry errors, device malfunctions, or patient non-response during clinical data collection. Prior to model training, all dataset attributes were examined for missing or null values. Rows with missing values in critical clinical features, such as blood pressure, cholesterol, and heart rate, were identified and handled using mean imputation for numerical variables and mode imputation for categorical variables. This approach preserved the dataset's statistical distribution while ensuring that no training sample was discarded unnecessarily, maintaining maximum data availability for model learning across distributed HDFS nodes.

Feature scaling

Numerical features, including age, blood pressure, cholesterol levels, and maximum heart rate, exhibit significantly different value ranges, which can cause features with larger magnitudes to disproportionately influence model training. To address this, a robust scaler based on the interquartile range was applied to all continuous numerical attributes. This scaling strategy is particularly appropriate for medical datasets where extreme clinical values representing rare or severe conditions may otherwise distort the learning process. Scaling ensured that all numerical variables contributed equally during model training and was applied consistently across all distributed worker nodes using MapReduce workflows.

Encoding

Categorical variables with more than two distinct categories, including cp (chest pain type), restecg (resting electrocardiographic results), and thal (thalassemia type), were transformed using one-hot encoding. This process converted each categorical attribute into a set of binary numerical indicator columns, producing representations that machine learning algorithms can process effectively without imposing artificial ordinal relationships between category values. Binary categorical variables were retained in their original numerical form. All encoding operations were executed as distributed MapReduce jobs across HDFS data blocks, ensuring consistent transformation across all partitioned dataset shards.

Train/test split

The preprocessed dataset was partitioned into training and test subsets using an 80:20 split, with 80% allocated to model training and 20% reserved for performance evaluation on unseen data. The target variable, indicating the presence or absence of heart disease, was separated from the independent feature set prior to splitting. This partitioning was applied uniformly across all distributed HDFS nodes to ensure that each worker node processed a proportional and representative shard of the full dataset, preventing data skewing. The 80:20 split strategy enhanced predictive reliability, improved model generalization, and maintained a clear separation between training and evaluation data across the distributed cluster environment, thereby preventing overfitting.

Model implementation

The Cluster Visualized Hadoop Distributed Decision Tree (CViHDDT) model classifies patients into risk categories using a distributed decision tree. The decision tree algorithm recursively splits the dataset based on the most informative features, maximizing the separation between patients with and without heart disease. Within the Hadoop distributed framework, this process is executed across multiple computing nodes, allowing large datasets to be processed efficiently. The distributed architecture reduces computational time while improving scalability. The Cluster Visualized Hadoop Distributed K-Nearest Neighbor (CViHDKNN) algorithm uses the same dataset but applies a different classification strategy. Instead of constructing a decision tree, the model identifies the closest neighboring patients based on medical attributes such as blood pressure, cholesterol levels, and exercise-induced angina. Using distributed computation, the KNN algorithm efficiently clusters patients with similar medical characteristics while managing computational complexity.

The classification principle of the distributed K-nearest neighbor model is illustrated in Figure 2, where a new instance is assigned to a class based on the majority class among its nearest neighbors. Cluster visualization techniques enable healthcare professionals to identify groups of patients with similar clinical characteristics, thereby improving interpretability and supporting personalized treatment recommendations. The proposed heart disease prediction framework integrates data preprocessing, distributed machine learning algorithms, and cluster visualization techniques. By leveraging Hadoop’s distributed computing capabilities, the framework efficiently processes large healthcare datasets while maintaining high prediction accuracy and interpretability, enabling early heart disease detection and improved clinical decision-making.

Cluster visualized Hadoop Distributed decision tree (CViHDDT):

Distributed Decision Tree Training

The proposed Cluster Visualized Hadoop Distributed Decision Tree (CViHDDT) model differs fundamentally from traditional decision tree construction by distributing the tree-building process across multiple nodes in the Hadoop ecosystem rather than building the entire tree on a single machine. Individual worker nodes construct partial decision trees locally on their assigned subset of the dataset using either MapReduce or Apache Spark for parallel processing. These locally constructed partial trees are subsequently combined into a complete global decision tree that encompasses the full distributed dataset. This distributed training strategy significantly accelerates model training, enabling the framework to efficiently handle multi-terabyte medical datasets at scale. The parallel computing infrastructure provided by Hadoop ensures the CViHDDT model is inherently scalable and well-suited for big-data-driven healthcare solutions. Following distributed tree construction, cluster visualization techniques are applied to enhance model interpretability by grouping decision tree nodes into clusters of patients with similar medical conditions using algorithms such as k-means and hierarchical clustering. This clustering process produces clinically meaningful risk categories—such as mild, moderate, and severe heart disease—enabling healthcare professionals to identify patterns in patient data, understand disease progression, and formulate personalized treatment plans.

Feature Selection

Prior to distributed decision tree training, the CViHDDT model applies a structured preprocessing and feature selection pipeline to the raw medical data ingested from HDFS. Missing values are addressed through imputation algorithms to manage incomplete clinical records and prevent data loss without discarding patient samples. Robust Scaler normalization is applied to numerical features such as blood pressure and cholesterol levels to mitigate the disproportionate influence of outliers prevalent in medical datasets. Categorical variables such as gender and family history of heart disease are transformed using one-hot or label encoding to produce numerical representations compatible with machine learning algorithms. After preprocessing, feature extraction is performed to identify the key clinical attributes most predictive of heart disease. This stage eliminates irrelevant and redundant features from the dataset, reducing computational costs in subsequent distributed training stages and ensuring that only the most diagnostically informative attributes — such as chest pain type, resting blood pressure, serum cholesterol, maximum heart rate, and ST depression — are retained as inputs to the distributed decision tree construction process. This systematic feature reduction improves model efficiency, reduces training time across distributed nodes, and enhances the overall predictive reliability of the CViHDDT framework by focusing the learning process on attributes with the strongest clinical discriminative power.

MapReduce Workflow

The MapReduce programming model forms the computational backbone of the CViHDDT distributed training pipeline, enabling parallel processing of the heart disease dataset across all worker nodes in the Hadoop cluster. In the map phase, each worker node independently processes its assigned HDFS data shard, computing partial decision tree structures and local splitting statistics — including Information Gain and Gini Index values — for each candidate attribute, without requiring access to data stored on other nodes. In the reduce phase, the locally computed partial trees and sufficient statistics are aggregated across all nodes to construct the complete global decision tree, consolidating the distributed knowledge learned at each node into a single unified predictive model. This map-reduce decomposition of the tree-building process enables the CViHDDT model to scale linearly with the number of worker nodes, making real-time analysis of large-scale medical datasets computationally feasible. The MapReduce workflow also supports the distributed execution of cluster visualization procedures, in which clustering algorithms are applied in parallel across HDFS data blocks to group patient records into risk categories based on their decision-tree node assignments. Performance evaluation of the resulting model employs precision, recall, F1-score, and classification accuracy as primary metrics, with the distributed cluster visualization further reducing false negatives by enabling finer decision boundaries within the tree — directly improving sensitivity for identifying at-risk patients and enhancing the clinical reliability of the CViHDDT heart disease prediction framework.

Cluster Visualized Hadoop Distributed K-Nearest Neighbor (CViHDKNN)

Clustering

The CViHDKNN (Cluster Visualized Hadoop Distributed K-Nearest Neighbor) framework begins by applying clustering techniques to the heart disease dataset prior to classification, grouping patients with similar medical characteristics into coherent clusters before the KNN search is performed. The heart disease dataset, containing clinical features such as age, cholesterol level, blood pressure, ECG results, and heart rate, is preprocessed and distributed across nodes in the Hadoop cluster using HDFS. Clustering algorithms, including K-Means and Hierarchical Clustering, are then applied across these distributed data partitions to divide the dataset into groups of patients sharing related medical profiles. This pre-classification clustering step serves a critical computational purpose: by restricting the KNN search space to only the most relevant cluster rather than the entire dataset, the algorithm dramatically reduces the number of distance computations required per query instance. Visualizing these clusters provides additional clinical benefit by enabling the identification of patient subgroups with closely related medical characteristics and by supporting more meaningful categorization of risk profiles prior to the nearest-neighbor classification stage. The clustering-based optimization not only reduces computational overhead but also improves classification accuracy by ensuring that each query instance is compared only against the most contextually similar patient records, making the approach particularly well-suited for large-scale heart disease datasets where exhaustive distance computation across the full dataset would be computationally prohibitive.

Distributed KNN

The distributed KNN component of CViHDKNN addresses the fundamental scalability limitation of traditional KNN, which requires loading the entire dataset into memory before computing distances between the query instance and all stored data points. In the CViHDKNN framework, this distance computation is parallelized across multiple worker nodes in the Hadoop cluster using HDFS-distributed data partitions, ensuring that no single node is required to process the complete dataset. Each worker node independently computes the distance between the query instance and the patient records stored in its locally assigned HDFS data shard, identifying the locally nearest neighbors within its partition. By harnessing Hadoop's parallel processing capabilities, CViHDKNN drastically improves scalability and enables efficient management of vast quantities of health-related patient data. This distributed architecture also enhances data security, as sensitive patient records remain within the distributed cluster environment rather than being transferred to external cloud servers or centralized local machines. The combination of clustering-guided search space reduction and Hadoop-distributed distance computation yields a system that achieves both computational efficiency and predictive accuracy, enabling real-time heart disease prediction on large-scale medical datasets. Experimental results confirm that the distributed implementation achieves a classification accuracy of 85.25%, representing a significant performance improvement over the traditional non-distributed KNN baseline, attributable directly to the distributed, clustering-enhanced processing strategy.

Classification

The classification stage of CViHDKNN assigns each query patient instance to a heart disease class based on the majority vote among its K nearest neighbors identified through the distributed search process. The choice of the K value directly influences classification outcomes and predictive precision. When K = 1, the query instance is assigned to the class label of its single closest neighbor, resulting in a highly localized decision boundary that may be sensitive to noise in the training data. When K = 3, the classification is determined by the majority class among the three nearest neighbors — for example, if two neighbors belong to Class 1 (no heart disease) and one belongs to Class 2 (heart disease present), the query instance is classified as Class 1, providing a more robust and noise-tolerant decision. The MapReduce reduce phase aggregates the locally identified nearest neighbors from all worker nodes into a globally ranked list, from which the K nearest neighbors are selected, and then computes the majority vote to produce the final class prediction. Performance of the CViHDKNN classification framework is evaluated using precision, recall, F1-score, and overall classification accuracy as primary metrics. The integration of cluster-constrained search with distributed majority voting produces finer and more accurate decision boundaries than standard KNN, reducing false negatives in at-risk patient identification and improving sensitivity, both critical requirements for clinically reliable heart disease prediction in large-scale distributed healthcare analytics environments.

Results

The experimental evaluation demonstrates that the CViHDKNN model achieves higher predictive performance than the CViHDDT model in heart disease classification. The CViHDKNN model achieved a test accuracy of 85.25%, whereas the CViHDDT model achieved 80.33%, indicating improved predictive capability for the distributed K-nearest neighbor approach. These comparative performance results are summarized in Table 1.

The classification performance of the CViHDDT model shows moderate predictive capability, with precision and recall values indicating a balanced detection of both disease and non-disease cases. In particular, the model achieved a recall of 75% for heart disease cases and 86% for non-disease cases, suggesting it performs relatively well at identifying patients without heart disease but may miss some positive cases. The detailed classification metrics for CViHDDT are presented in Table 2.

In contrast, the CViHDKNN model demonstrates improved sensitivity in detecting heart disease cases. The model achieved a recall of 88% for class 1 (heart disease), indicating it is more effective at identifying patients with heart disease than the CViHDDT model. The classification report for CViHDKNN, including precision, recall, and F1-score values for both classes, is shown in Table 3.

Overall, the results indicate a trade-off between the two models. While CViHDDT exhibits relatively higher precision in identifying non-disease cases, CViHDKNN provides stronger detection of heart disease cases due to its higher recall for the positive class. This improvement in sensitivity is particularly important in medical diagnosis, where failing to detect disease cases may lead to delayed treatment.

A graphical comparison of the evaluation metrics for both models is shown in Figure 3, highlighting the CViHDKNN model's improved predictive performance across multiple metrics.

Data availability:

The dataset used in this study is publicly available and can be accessed without restriction. All experiments were conducted using the Cleveland Heart Disease Dataset, which is publicly available from the UCI Machine Learning Repository at https://archive.ics.uci.edu/ml/datasets/heart+disease. This dataset contains 303 patient records with 14 clinical attributes, including age, sex, chest pain type, resting blood pressure, serum cholesterol, fasting blood sugar, resting electrocardiographic results, maximum heart rate achieved, exercise-induced angina, ST depression, slope of peak exercise ST segment, number of major vessels, thalassemia type, and the target variable indicating the presence or absence of heart disease. No proprietary, restricted, or patient-identifiable data were used at any stage of this research. Researchers wishing to replicate the experimental findings or extend this work may access the dataset directly from the UCI Machine Learning Repository without requiring special permissions or institutional data sharing agreements^33,34. The authors confirm that the dataset used complies with applicable open-access licensing terms and that no ethical restrictions apply to its public availability. All datasets generated and analyzed during this study are included in this article in the form of figures and tables.

Figure 1: Workflow of the proposed heart disease prediction framework. Patient medical data are stored in the Hadoop Distributed File System (HDFS) and undergo preprocessing, including handling missing values, feature scaling, and categorical encoding. Relevant features are selected and used to train two distributed machine learning models, CViHDDT and CViHDKNN. Model performance is evaluated using precision, recall, and F1-score within the Hadoop-based distributed computing environment. Please click here to view a larger version of this figure.

Figure 2: Illustration of the classification principle used in the Cluster Visualized Hadoop Distributed K-Nearest Neighbor (CViHDKNN) model. The example demonstrates KNN classification with different neighbor values (K=1 and K=3), where a new instance is assigned to a class based on the majority class of its nearest neighbors. This concept serves as the basis for the distributed KNN approach used for heart disease prediction. Please click here to view a larger version of this figure.

Figure 3: Graphical comparison of the predictive performance of CViHDDT and CViHDKNN models. The figure illustrates key evaluation metrics, including training accuracy, testing accuracy, precision, recall, and F1-score for both classes, highlighting the improved overall performance of the CViHDKNN model. Please click here to view a larger version of this figure.

Metric	CViHDDT	CviHDKNN
Train Accuracy (%)	75.62%	83.47%
Test Accuracy (%)	80.33%	85.25%
Precision (Class 0)	0.76	0.86
Recall (Class 0)	0.86	0.83
F1-Score (Class 0)	0.81	0.84
Precision (Class 1)	0.86	0.85
Recall (Class 1)	0.75	0.88
F1-Score (Class 1)	0.8	0.86
Overall Accuracy (%)	80.00%	85.00%
Macro Average Precision	0.81	0.85
Macro Average Recall	0.81	0.85
Macro Average F1-Score	0.8	0.85

Table 1: Comparative performance evaluation of the CViHDDT and CViHDKNN models for heart disease prediction. The table summarizes training and testing accuracy along with class-wise precision, recall, F1-score, overall accuracy, and macro-averaged evaluation metrics.

Class	Precision	Recall	F1-Score	Support
0 (No Disease)	0.76	0.86	0.81	29
1 (Heart Disease)	0.86	0.75	0.8	32
Overall Accuracy	0.8	-	-	61
Macro Average	0.81	0.81	0.8	61
Weighted Average	0.81	0.8	0.8	61

Table 2: Classification performance of the CViHDDT model for heart disease prediction. The table reports precision, recall, F1-score, and support values for both classes (presence and absence of heart disease), demonstrating the model’s balanced predictive capability.

Class	Precision	Recall	F1-Score	Support
0 (No Disease)	0.86	0.83	0.84	29
1 (Heart Disease)	0.85	0.88	0.86	32
Overall Accuracy	0.85	-	-	61
Macro Average	0.85	0.85	0.85	61
Weighted Average	0.85	0.85	0.85	61

Table 3: Classification performance of the CViHDKNN model. The table presents precision, recall, F1-score, and support for both classes, showing improved sensitivity in detecting heart disease cases compared with the CViHDDT model.

Discussion

An effective heart disease prediction method based on the clustering technique CViHDDT was developed and evaluated using a Hadoop-based distributed decision tree framework. The model achieved a training accuracy of approximately 75.62% and a testing accuracy of 80.33%, demonstrating its ability to generalize to unseen data. The slightly higher test accuracy suggests that Hadoop’s parallel processing efficiently handles large datasets while minimizing overfitting. Previous studies have similarly reported that Hadoop-based distributed systems improve scalability and processing efficiency in healthcare analytics^1,8. By utilizing the MapReduce framework, the dataset is distributed across multiple nodes, allowing faster computation of entropy and information gain during decision tree construction.

The classification report indicates balanced performance across both classes, with F1-scores of 0.81 for class 0 (absence of heart disease) and 0.80 for class 1 (presence of heart disease). Precision and recall values show that the model identifies heart disease cases with 86% precision and 75% recall, indicating a tendency to produce fewer false positives while missing some positive cases. Macro and weighted averages of precision, recall, and F1-score remain around 0.80, confirming the model's robustness. The detailed performance metrics of the CViHDDT model are summarized in Table 2.

The decision threshold also influences classification performance. A default threshold of 0.5 assigns patients to the heart disease class when the predicted probability exceeds 50%. However, the lower recall for heart disease cases suggests that slightly lowering the threshold could improve sensitivity and detect more positive cases. Conversely, increasing the threshold may reduce false positives at the expense of recall. Because recall is often more critical in medical diagnosis, slightly lowering the threshold may improve the detection of heart disease cases without significantly affecting overall model performance. Similar observations regarding the importance of recall and sensitivity in heart disease prediction have been discussed in previous machine learning studies on cardiovascular diagnosis^6,9,11.

The Cluster Visualized Hadoop Distributed K-Nearest Neighbor (CViHDKNN) model demonstrated stronger predictive performance. The model achieved training accuracy of 83.47% and test accuracy of 85.25%, indicating improved generalization compared with the decision tree approach. Clustering techniques, such as K-Means, were applied prior to KNN classification to group patients with similar medical characteristics, thereby reducing the search space and improving computational efficiency. Visualization-assisted clustering approaches have previously been shown to improve interpretability and predictive performance in healthcare AI systems^2,4. When implemented within the Hadoop environment, MapReduce enables parallel computation of distance calculations and neighbor selection, allowing the algorithm to process large datasets efficiently.

The classification report for CViHDKNN shows balanced performance across both classes. For class 0 (no heart disease), the model achieved a precision of 0.86 and a recall of 0.83; for class 1 (heart disease), the precision and recall were 0.85 and 0.88, respectively. The corresponding F1-scores indicate consistent performance across both classes, and the macro and weighted averages remain around 0.85, confirming the model's reliability. These performance metrics are summarized in Table 3.

Model comparison further highlights the advantage of the clustering-enhanced KNN approach. While CViHDDT provides efficient distributed processing and balanced classification performance, CViHDKNN achieves higher overall accuracy and improved recall for heart disease cases. Specifically, CViHDDT achieved training and testing accuracies of 75.62% and 80.33%, whereas CViHDKNN achieved 83.47% and 85.25%, respectively. In terms of classification performance, CViHDDT produced precision and recall values of 0.76 and 0.86 for class 0, and 0.86 and 0.75 for class 1, whereas CViHDKNN improved heart disease detection, achieving a recall of 0.88. These comparative results are summarized in Table 1. Comparable improvements in predictive performance using optimized and hybrid machine learning approaches for heart disease prediction have also been reported in earlier studies^10,20,30,33.

In medical applications, false negatives can have serious consequences because failing to detect heart disease may delay treatment. Therefore, the higher recall achieved by CViHDKNN is particularly important for early diagnosis and clinical decision support. Several previous cardiovascular prediction studies have similarly emphasized the importance of minimizing false negatives to support timely diagnosis and preventive healthcare strategies^3,6,12. The clustering-based framework also improves interpretability by grouping patients with similar medical characteristics, enabling clinicians to better understand patterns within the dataset. Based on these findings, the CViHDKNN algorithm shows strong potential as an AI-assisted screening tool for early cardiovascular risk detection. By processing large-scale patient datasets within a Hadoop-based distributed environment, the framework can help healthcare systems identify at-risk individuals and support preventive strategies before severe symptoms develop.

Future work may focus on improving model efficiency and enabling real-time clinical decision support. Integrating distributed machine learning models with large-scale healthcare data streams, cloud platforms, and federated learning approaches could further enhance scalability while maintaining patient privacy. Recent studies on hybrid and scalable machine learning systems have also highlighted the growing importance of distributed and privacy-aware healthcare analytics^5,28. Additionally, incorporating data from wearable health monitoring devices and IoT-based medical systems may enable continuous monitoring and earlier detection of cardiovascular risk.

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
Apache Spark	Apache Software Foundation	3.x	Distributed data processing
HDFS	Apache Software Foundation	Included in Hadoop 3.x	Distributed file storage
YARN	Apache Software Foundation	Included in Hadoop 3.x	Resource management and job scheduling
Matplotlib	Matplotlib Development Team		Data visualization
Seaborn	Seaborn Developers		Statistical plotting
Ubuntu OS	Canonical Ltd.	20.04 LTS	Operating system
RobustScaler	Scikit-learn		Feature normalization
UCI Heart Disease Dataset	UCI Machine Learning Repository	Dataset ID 45	Experimental dataset