Assessing the Accuracy of Fitness Smartwatch Data for Cardiovascular and Physical Activity Monitoring: A Validation Study in Digital Health

The research aims to validate their accuracy of low cost-fitness smartwatches by comparing their data with gold standard measurements used in clinical settings. This protocol evaluates the accuracy and reliability of fitness smartwatches, addressing gaps in understanding their real-world performance as their popularity grows. Our findings will provide valuable insights into the accuracy and reliability of fitness smartwatch data, which is crucial for researchers and healthcare professionals using these devices in studies or clinical settings.

To begin, have participants wear the fitness smartwatch on their non-dominant wrist. Attach participants to the apparatus of the polysomnography or PSG machine. With the fitness smartwatch, take two measurements of blood oxygen saturation level while the participant is in a supine position.

Ensure the participant sleep for a maximum of six hours. After the participant awakens, use the fitness smartwatch to take two additional blood oxygen saturation level measurements while they remain in a supine position. Export the sleep trends from both the PSG machine and the fitness smartwatch into a csv file.

Include data on deep sleep duration, light sleep duration, rapid eye movement time, total sleep duration, heart rate, and blood oxygen saturation levels. For the validation of fitness smartwatch step measurements, first have participants wear the fitness smartwatch on their non-dominant wrist and hold a smartphone in their other hand for footstep video recording purposes. Set the step count on the fitness smartwatch to zero, then start the smartphone recording, and have the participant perform the step test.

Record the number of steps measured by the fitness smartwatch. For the three-minute walk test or 3-M WT, instruct the participant to walk at a normal and steady pace for three minutes on a flat surface. For the stair climb or SC test, have the participant take two flights of stairs in ascending and descending order at a normal and steady pace.

Manually calculate the number of steps observed in the video recording. Record this data into a csv file, then export the step counts from the fitness smartwatch into the same file. Import the required packages into Python's workspace, then import the measurement data into Python's workspace.

Replace the file name measurement_data. csv with a file name of choice. Use the drop in a function to remove any empty values for each measurement data.

Perform the Shapiro-Wilk normality test to determine if the data has a normal distribution using the SciPy stats Shapiro package in Python. Now, perform a paired sample T-test using the SciPy stats T-test TestRail package in Python if the data has a normal distribution. If not, perform the Mann-Whitney Wilcoxon test using the SciPy stats Wilcoxon package.

Replace measure1_toolA and measure1_toolB with respective column names of choice in the code. Then use the co and d method to measure the magnitude of the difference between instruments. Employed Bland-Altman analysis to assess the agreement between two instruments, including mean, or median difference, standard deviation, and 95%confidence interval of the bias.

Plot the Bland-Altman analysis for visualization using the Matplotlib pie plot package in Python. Shallow sleep time showed a large bias of 27.77 minutes with wide limits of agreement, indicating significant variability in measurements by the smartwatch compared to PSG. Total sleep time was overestimated by 41.5 minutes, with wider limits of agreement, indicating occasional mismatches in tracking sleep duration.

REM time showed minimal bias and standard deviation, suggesting good interchangeability between the smartwatch and PSG for this parameter. Wake time was overestimated by the smartwatch, with clear clustering and upward trend in differences, indicating measurement bias. Mean heart rate exhibited minimal bias and good clustering near the bias line, indicating strong agreement.

Minimum heart rate displayed some outliers, suggesting difficulty in detecting low heart rates, though overall agreement was reasonable. Maximum heart rate exhibited notable dispersion, reflecting variability in tracking peak values by the smartwatch. Blood oxygen saturation showed minimal bias and narrow limits of agreement, reflecting high agreement between the devices.

The smartwatch significantly underestimated steps for 3-M WT by an average of 31.33 steps and SC steps by 11.