Name: Assessing the Accuracy of Fitness Smartwatch Data for Cardiovascular and Physical Activity Monitoring: A Validation Study in Digital Health (Video) | JoVE Journal
Uploaded: 2025-02-21T13:40:00.000+00:00
Duration: 5 min 51 s
Description: Learn how to validate fitness smartwatch accuracy for cardiovascular and physical activity monitoring using polysomnography and step tests. This protocol compares wearable data with clinical gold standards for reliable health tracking.

ai_agent_IP

ai_agent

maintenance

instantTrialDurationMinutes

isBiopharma

enableNewMyJoveSectionDesign

educationFeatures

seoMetaApi

ai_copilot_institution_blacklist

ai_chat_institution_blacklist

freeAccessWindow

ssoRoleCapture

scheduledMaintenanceBanner

joveLabsFeatures

disable_autoplay_for_bots

extended_trial_duration

5 minutes

We thank the neurotechnologist at the Neurology Lab, University of Malaya Medical Centre, for the help and support in conducting the test for this study. This work was supported by the UM Research Center (IIRG001C-2021IISS).

The study protocol is approved by the Universiti Malaya Medical Centre (UMMC) Ethics Review Board (MREC No: 2021325-9983). The validation study of fitness smartwatch measurements is divided into three parts: (1) validation of sleep measurements against the gold standard using a PSG machine, (2) validation of step measurements by comparing manual calculations from video recordings, and (3) data analysis of the validation tests.
1. Validation of sleep measurements against polysomnography (PSG)
NOTE: The study is conducted in a controlled sleep laboratory to minimize external disruptions. Prior to the test, all participants were briefed in depth about the study and were provided with the study information sheet before written informed consent was obtained. All information gathered is kept anonymous.
<ol>
	<li>Ensure participants arrive 1 h before the test starts for necessary preparation.</li>
	<li>Participants wear the fitness smartwatch on their non-dominant wrist.</li>
	<li>Attach participants to the apparatus of the PSG machine according to the manufacturer&#39;s manual24. 
	NOTE: Use standardized placement of PSG electrodes as per the manufacturer&#39;s manual to ensure minimal discomfort to participants during sleep.</li>
	<li>Take two measurements of blood oxygen saturation level (SpO2) using the fitness smartwatch in a supine position.</li>
	<li>Ensure participants sleep for a maximum 6 h. 
	NOTE: Technical personnel monitor the PSG machine throughout the sleep duration.</li>
	<li>Take two measurements of SpO2 using the fitness smartwatch in a supine position after the participant is awake.</li>
	<li>Export sleep trends from both PSG machine and fitness smartwatch, i.e., deep sleep duration, light sleep duration, rapid eye movements (REM) time and sleep duration, heart rate (minimum, mean, maximum heart rate), and SpO2, into a .csv file. Refer to each device&#39;s manufacturer manual on how to export the data24,25.</li>
	<li>Proceed to section 3 for data analysis.</li>
</ol>
2. Validation of fitness smartwatch step measurements with manual calculation from video recordings
NOTE: Prior to the test, all participants are briefed in depth about the study and are provided with the study information sheet before written informed consent is obtained. All information gathered is kept anonymous. Only participants with good health conditions and no impairment in mobility are chosen for this validation test.
<ol>
	<li>Ensure participants wear the fitness smartwatch on their non-dominant wrist and hold a smartphone for footstep video recording purposes on the other hand.</li>
	<li>Set the step count on the fitness smartwatch to zero. Start recording on the smartphone and have the participant perform the step test as indicated in step 2.3. Stop and save the recording. Record the number of steps measured in the fitness smartwatch.</li>
	<li>Ensure participant completes two types of step tests (i) 3MWT and (ii) SC tests.
	<ol>
		<li>3MWT: Ensure the participant walks at a normal and steady pace for 3 min on a flat surface.</li>
		<li>SC test: Have the participant take two flights of stairs in ascending and descending order at a normal and steady pace.</li>
	</ol>
	</li>
	<li>Replay the recording and calculate the number of steps from the video recording. Record the data into a .csv file. Export the step counts from the fitness smartwatch into the same file. Refer to the fitness smartwatch manufacturer&#39;s manual on how to export the data25.</li>
	<li>Proceed to section 3 for data analysis.</li>
</ol>
3. Data analysis of the validation tests
NOTE: In this protocol, the scipy.stats package in Python was used to perform all analysis tests except the Cohen d test, where the pingouin package was utilized instead. The code is in Supplementary File 1, where each step is mentioned in the comment line.
<ol>
	<li>Import package required into Python&#39;s workspace.</li>
	<li>Import measurement data into the Python&#39;s workspace. Replace the filename measurement_data.csv with a filename of choice.</li>
	<li>Remove any empty values using the dropna() function for each measurement data. Then, perform the Shapiro-Wilk normality test to determine if the data has a normal distribution using the scipy.stats shapiro package in Python.</li>
	<li>Perform paired sample t-test using the scipy.stats ttest_rel package in Python if data has a normal distribution, else, perform Mann-Whitney Wilcoxon using the scipy.stats wilcoxon package in Python. If the result has a p-value &#60; 0.05, it means there is a significant difference between the measurement data from both instruments. In the code, replace measure1_toolA and measure1_toolB with the respective column names of choice.</li>
	<li>Measure the magnitude of the difference between instruments via effect size using the Cohen d method. An output of 0.2 indicates a small effect, 0.5 a moderate effect, and 0.8 a large effect.</li>
	<li>Employ Bland-Altman analysis to assess the agreement between two instruments, such as mean or median difference, standard deviation (SD), and 95% CI of the bias. Plot the Bland-Altman analysis for easy visualization of the output using the matplotlib.pyplot package in Python.</li>
	<li>Repeat step 3.5 until step 3.7 for each pair of the measurement data from both instruments.</li>
</ol>

<ol><li>Sergueeva, K., Shaw, N., Lee, S. H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Understanding+the+barriers+and+factors+associated+with+consumer+adoption+of+wearable+technology+devices+in+managing+personal+health%5BTitle%5D)+AND+%22Can+J+Adm+Sci%22%5BJournal%5D)">Understanding the barriers and factors associated with consumer adoption of wearable technology devices in managing personal health.</a> Can J Adm Sci. 37 (1), 45-60 (2020).</li>
<li>Xu, K., Lu, Y., Takei, K. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Multifunctional+skin-inspired+flexible+sensor+systems+for+wearable+electronics%5BTitle%5D)+AND+%22Adv+Mater+Technol%22%5BJournal%5D)">Multifunctional skin-inspired flexible sensor systems for wearable electronics.</a> Adv Mater Technol. 4 (3), 25(2019).</li>
<li>Ates, H. C., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((End-to-end+design+of+wearable+sensors%5BTitle%5D)+AND+%22Nat+Rev+Mater%22%5BJournal%5D)">End-to-end design of wearable sensors.</a> Nat Rev Mater. 7 (11), 887-907 (2022).</li>
<li>Harrington, J., Schramm, P. J., Davies, C. R., Lee-Chiong, T. L. Jr <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((An+electrocardiogram-based+analysis+evaluating+sleep+quality+in+patients+with+obstructive+sleep+apnea%5BTitle%5D)+AND+%22Sleep+Breath%22%5BJournal%5D)">An electrocardiogram-based analysis evaluating sleep quality in patients with obstructive sleep apnea.</a> Sleep Breath. 17 (3), 1071-1078 (2013).</li>
<li>Mccall, W. V. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((A+rest-activity+biomarker+to+predict+response+to+SSRIs+in+major+depressive+disorder%5BTitle%5D)+AND+%22J+Psychiatr+Res%22%5BJournal%5D)">A rest-activity biomarker to predict response to SSRIs in major depressive disorder.</a> J Psychiatr Res. 64, 19-22 (2015).</li>
<li>High accuracy discrimination of Parkinson's disease participants from healthy controls using smartphones. Arora, S., et al. 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Florence, Italy, , (2014).</li>
<li>Perry, T. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Temporary+tattoos+can+replace+today%27s+biomedical+sensors%5BTitle%5D)+AND+%22IEEE+Spectr%22%5BJournal%5D)">Temporary tattoos can replace today's biomedical sensors.</a> IEEE Spectr. 34, 84-85 (2015).</li>
<li>Lehra, C., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Wearable+applications+in+rugby+for+performance+quantification+and+player+health+assessment%3A+A+brief+review%5BTitle%5D)+AND+%22IJKSS%22%5BJournal%5D)">Wearable applications in rugby for performance quantification and player health assessment: A brief review.</a> IJKSS. 10 (2), 10(2022).</li>
<li>Lacey, A., Whyte, E., O'keeffe, S., O'connor, S., Moran, K. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((A+qualitative+examination+of+the+factors+affecting+the+adoption+of+injury+focused+wearable+technologies+in+recreational+runners%5BTitle%5D)+AND+%22PloS+One%22%5BJournal%5D)">A qualitative examination of the factors affecting the adoption of injury focused wearable technologies in recreational runners.</a> PloS One. 17 (7), 22(2022).</li>
<li>Binyamin, S. S., Hoque, M. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Understanding+the+drivers+of+wearable+health+monitoring+technology%3A+An+extension+of+the+unified+theory+of+acceptance+and+use+of+technology%5BTitle%5D)+AND+%22Sustainability%22%5BJournal%5D)">Understanding the drivers of wearable health monitoring technology: An extension of the unified theory of acceptance and use of technology.</a> Sustainability. 12 (22), 20(2020).</li>
<li>Rao, P., Seshadri, D. R., Hsu, J. J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Current+and+potential+applications+of+wearables+in+sports+cardiology%5BTitle%5D)+AND+%22Curr+Treat+Options+Cardiovasc+Med%22%5BJournal%5D)">Current and potential applications of wearables in sports cardiology.</a> Curr Treat Options Cardiovasc Med. 23 (10), 65(2021).</li>
<li>Casado-Robles, C., Mayorga-Vega, D., Guijarro-Romero, S., Viciana, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Validity+of+the+xiaomi+mi+band+2%2C+3%2C+4+and+5+wristbands+for+assessing+physical+activity+in+12-to-18-year-old+adolescents+under+unstructured+free-living+conditions.+Fit-person+study%5BTitle%5D)+AND+%22J+Sports+Sci+Med%22%5BJournal%5D)">Validity of the xiaomi mi band 2, 3, 4 and 5 wristbands for assessing physical activity in 12-to-18-year-old adolescents under unstructured free-living conditions. Fit-person study.</a> J Sports Sci Med. 2 (2), 196-211 (2023).</li>
<li>Shapovalov, Y. B., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Using+of+personal+smart+tools+in+STEM+education%5BTitle%5D)+AND+%22Proceedings+of+the+1st+Symposium+on+Advances+in+Educational+Technology%22%5BJournal%5D)">Using of personal smart tools in STEM education.</a> Proceedings of the 1st Symposium on Advances in Educational Technology. 2, 192-207 (2020).</li>
<li>Concheiro-Moscoso, P., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Use+of+the+Xiaomi+mi+band+for+sleep+monitoring+and+its+influence+on+the+daily+life+of+older+people+living+in+a+nursing+home%5BTitle%5D)+AND+%22Digit+Health%22%5BJournal%5D)">Use of the Xiaomi mi band for sleep monitoring and its influence on the daily life of older people living in a nursing home.</a> Digit Health. 8, 20552076221121162(2022).</li>
<li>Monitoring system for elderly health care using smart band, Raspberry Pi, and Node-Red. Ardiyanto, Y., Adi, K., Putra, K. T., Utomo, P. 2022 2nd International Conference on Electronic and Electrical Engineering and Intelligent System (ICE3IS), Yogyakarta, Indonesia, , (2022).</li>
<li>Viciana, J., Casado-Robles, C., Guijarro-Romero, S., Mayorga-Vega, D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Are+wrist-worn+activity+trackers+and+mobile+applications+valid+for+assessing+physical+activity+in+high+school+students%3F+Wearfit+study%5BTitle%5D)+AND+%22J+Sports+Sci+Med%22%5BJournal%5D)">Are wrist-worn activity trackers and mobile applications valid for assessing physical activity in high school students? Wearfit study.</a> J Sports Sci Med. 21 (3), 356-375 (2022).</li>
<li>Hao, Y., Ma, X. -K., Zhu, Z., Cao, Z. -B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Validity+of+wrist-wearable+activity+devices+for+estimating+physical+activity+in+adolescents%3A+Comparative+study%5BTitle%5D)+AND+%22JMIR+Mhealth+Uhealth%22%5BJournal%5D)">Validity of wrist-wearable activity devices for estimating physical activity in adolescents: Comparative study.</a> JMIR Mhealth Uhealth. 9 (1), e18320(2021).</li>
<li>Ameen, M. S., Cheung, L. M., Hauser, T., Hahn, M. A., Schabus, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((About+the+accuracy+and+problems+of+consumer+devices+in+the+assessment+of+sleep%5BTitle%5D)+AND+%22Sensors%22%5BJournal%5D)">About the accuracy and problems of consumer devices in the assessment of sleep.</a> Sensors. 19 (19), 4160(2019).</li>
<li>Concheiro-Moscoso, P., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Quality+of+sleep+data+validation+from+the+xiaomi+mi+band+5+against+polysomnography%3A+Comparison+study%5BTitle%5D)+AND+%22J+Med+Internet+Res%22%5BJournal%5D)">Quality of sleep data validation from the xiaomi mi band 5 against polysomnography: Comparison study.</a> J Med Internet Res. 25, e42073(2023).</li>
<li>Topalidis, P., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Evaluation+of+a+low-cost+commercial+actigraph+and+its+potential+use+in+detecting+cultural+variations+in+physical+activity+and+sleep%5BTitle%5D)+AND+%22Sensors%22%5BJournal%5D)">Evaluation of a low-cost commercial actigraph and its potential use in detecting cultural variations in physical activity and sleep.</a> Sensors. 21 (11), 18(2021).</li>
<li>Xu, H., Zheng, X., Jia, W., Yin, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Chromatography%2Fmass+spectrometry-based+biomarkers+in+the+field+of+obstructive+sleep+apnea%5BTitle%5D)+AND+%22Medicine%22%5BJournal%5D)">Chromatography/mass spectrometry-based biomarkers in the field of obstructive sleep apnea.</a> Medicine. 94 (40), e1541(2015).</li>
<li>Michaelson, P. G., Allan, P., Chaney, J., Mair, E. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Validations+of+a+portable+home+sleep+study+with+twelve-lead+polysomnography%3A+Comparisons+and+insights+into+a+variable+gold+standard%5BTitle%5D)+AND+%22Ann+Otol+Rhinol+Laryngol%22%5BJournal%5D)">Validations of a portable home sleep study with twelve-lead polysomnography: Comparisons and insights into a variable gold standard.</a> Ann Otol Rhinol Laryngol. 115 (11), 802-809 (2006).</li>
<li>Berry, R. B., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Rules+for+scoring+respiratory+events+in+sleep%3A+Update+of+the+2007+AASM+manual+for+the+scoring+of+sleep+and+associated+events%5BTitle%5D)+AND+%22J+Clin+Sleep+Med%22%5BJournal%5D)">Rules for scoring respiratory events in sleep: Update of the 2007 AASM manual for the scoring of sleep and associated events.</a> J Clin Sleep Med. 08 (05), 597-619 (2012).</li>
<li>Natus Neurology Incorporated. <a target="_blank" href="http://scholar.google.com/scholar?hl=en&safe=off&q=%22Nicoletone+information+for+use%22">Nicoletone information for use</a>. , https://natus.com/ (2024).</li>
<li>Xiaomi Communications. Mi smart band 6 user manual. , At
 <a target="_blank" href="https://www.manualslib.com/manual/2629650/Xiaomi-Mi-Smart-Band-6.html">https://www.manualslib.com/manual/2629650/Xiaomi-Mi-Smart-Band-6.html</a> (2021).</li>
<li>De Zambotti, M., Cellini, N., Goldstone, A., Colrain, I. M., Baker, F. C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Wearable+sleep+technology+in+clinical+and+research+settings%5BTitle%5D)+AND+%22Med+Sci+Sports+Exerc%22%5BJournal%5D)">Wearable sleep technology in clinical and research settings.</a> Med Sci Sports Exerc. 51 (7), 1538-1557 (2019).</li>
<li>Meltzer, L. J., Montgomery-Downs, H. E., Insana, S. P., Walsh, C. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Use+of+actigraphy+for+assessment+in+pediatric+sleep+research%5BTitle%5D)+AND+%22Sleep+Med+Rev%22%5BJournal%5D)">Use of actigraphy for assessment in pediatric sleep research.</a> Sleep Med Rev. 16 (5), 463-475 (2012).</li>
<li>Paradiso, C., Colino, F., Liu, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((The+validity+and+reliability+of+the+mi+band+wearable+device+for+measuring+steps+and+heart+rate%5BTitle%5D)+AND+%22Int+J+Exerc+Sci%22%5BJournal%5D)">The validity and reliability of the mi band wearable device for measuring steps and heart rate.</a> Int J Exerc Sci. 13 (4), 689-701 (2020).</li>
<li>Benedetto, S., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Assessment+of+the+fitbit+charge+2+for+monitoring+heart+rate%5BTitle%5D)+AND+%22PloS+One%22%5BJournal%5D)">Assessment of the fitbit charge 2 for monitoring heart rate.</a> PloS One. 13 (2), e0192691(2018).</li>
<li>Chow, H. -W., Yang, C. -C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Accuracy+of+optical+heart+rate+sensing+technology+in+wearable+fitness+trackers+for+young+and+older+adults%3A+Validation+and+comparison+study%5BTitle%5D)+AND+%22JMIR+Mhealth+Uhealth%22%5BJournal%5D)">Accuracy of optical heart rate sensing technology in wearable fitness trackers for young and older adults: Validation and comparison study.</a> JMIR Mhealth Uhealth. 8 (4), e14707(2020).</li>
<li>Jachymek, M., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Wristbands+in+home-based+rehabilitation-validation+of+heart+rate+measurement%5BTitle%5D)+AND+%22Sensors%22%5BJournal%5D)">Wristbands in home-based rehabilitation-validation of heart rate measurement.</a> Sensors. 22 (1), 60(2021).</li>
<li>Schalk, M., Schalk, I., Bauernhansl, T., Siegert, J., Schneider, U. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Investigation+of+possible+effects+of+wearing+exoskeletons+during+welding+on+heart+rate%5BTitle%5D)+AND+%22Physiologia%22%5BJournal%5D)">Investigation of possible effects of wearing exoskeletons during welding on heart rate.</a> Physiologia. 2 (3), 94-108 (2022).</li>
<li>Bent, B., Goldstein, B. A., Kibbe, W. A., Dunn, J. P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Investigating+sources+of+inaccuracy+in+wearable+optical+heart+rate+sensors%5BTitle%5D)+AND+%22NPJ+Digit+Med%22%5BJournal%5D)">Investigating sources of inaccuracy in wearable optical heart rate sensors.</a> NPJ Digit Med. 3 (1), 18(2020).</li>
<li>De Cocker, K. A., De Meyer, J., De Bourdeaudhuij, I. M., Cardon, G. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Non-traditional+wearing+positions+of+pedometers%3A+Validity+and+reliability+of+the+Omron+HJ-203-ED+pedometer+under+controlled+and+free-living+conditions%5BTitle%5D)+AND+%22J+Sci+Med+Sport%22%5BJournal%5D)">Non-traditional wearing positions of pedometers: Validity and reliability of the Omron HJ-203-ED pedometer under controlled and free-living conditions.</a> J Sci Med Sport. 15 (5), 418-424 (2012).</li>
<li>Diaz, K. M., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Fitbit%C2%AE%3A+An+accurate+and+reliable+device+for+wireless+physical+activity+tracking%5BTitle%5D)+AND+%22Int+J+Cardiol%22%5BJournal%5D)">Fitbit®: An accurate and reliable device for wireless physical activity tracking.</a> Int J Cardiol. 185, 138-140 (2015).</li>
<li>Mendes, M., Pala, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Type+I+error+rate+and+power+of+three+normality+tests%5BTitle%5D)+AND+%22Inf+Technol+J%22%5BJournal%5D)">Type I error rate and power of three normality tests.</a> Inf Technol J. 2, 135-139 (2003).</li>
<li>Costa, R. R., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Heart+rate+deflection+point+as+an+alternative+to+determining+the+anaerobic+threshold+in+dyslipidaemic+patients%5BTitle%5D)+AND+%22Motriz%3A+Rev+Educ+F%C3%ADs%22%5BJournal%5D)">Heart rate deflection point as an alternative to determining the anaerobic threshold in dyslipidaemic patients.</a> Motriz: Rev Educ Fís. 25 (1), (2019).</li>
<li>Roth, S., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Noninvasive+evaluation+of+the+hemodynamic+status+in+patients+after+heart+transplantation+or+left+ventricular+assist+device+implantation%5BTitle%5D)+AND+%22PloS+One%22%5BJournal%5D)">Noninvasive evaluation of the hemodynamic status in patients after heart transplantation or left ventricular assist device implantation.</a> PloS One. 17 (10), e0275977(2022).</li>
</ol>

The authors declare that the research was conducted in the absence of any commercial or financial compensation, sponsorship, or any relationships with the device manufacturer that could be construed as a potential conflict of interest. The opinions and findings expressed in this article are the author's own and are based solely on their experience with the product from this study.

Several limitations of the fitness smartwatch were identified based on the mixed results in the comparison analysis. The high variability in the smartwatch's measurements may stem from its reliance on movement and heart rate data rather than more detailed measures, such as the electroencephalogram (EEG) used in PSG. Deep sleep and REM time showed better agreement, which suggests that the fitness smartwatch may be suitable for general sleep monitoring. However, it may not be reliable for clinical diagnosis, such as detail...

Wearable technology has grown in popularity, becoming commonplace in various parts of daily life1. These technologies, equipped with sensors and algorithms, have transformed how physiological parameters are monitored and interpreted, providing users with health information, tracking workouts, and allowing users to have a healthier lifestyle. The integration of artificial intelligence and pattern recognition, combined with increasingly popular features like virtual and augmented reality features, not only enhances the functionality of wearable devices but also enables advanced personalized data analysis and more engaging user experience2,3. As wearable devices become more integrated into everyday routines, understanding their accuracy and reliability becomes paramount, especially in industries such as sports performance monitoring and healthcare.
Wearables offer the benefit of real-time patient data logging for chronic illnesses, providing invaluable insights for both patients and healthcare professionals. Metrics such as physical activity, conversation timelines, and sleep duration can help assess the severity of diseases such as depression, sleep apnea, and Parkinson&#39;s disease4,5,6. Emerging technologies, such as electronic stamps that monitor glucose, heart rate, and temperature, are undergoing clinical testing, further expanding the possibilities of advanced wearables in healtchcare7. Furthermore, athletes and fitness enthusiasts utilize wearable technology to optimize training regimens, track performance metrics, and prevent injuries8,9. The integration of regular evaluation of physiological parameters, activity data, and functionalities like electrocardiogram recording has sparked interest regarding wearables&#39; potential to aid in the diagnosis and tracking of cardiovascular diseases8,10. Wearable gadgets have shown promise in enhancing workplace safety, improving lifestyle management, and instilling healthier habits among users11.
Fitness smartwatch bands, such as the Xiaomi Mi Band, a well-known low-cost wearable technology and one of the most widely used smartwatches globally12, have been applied in a variety of applications, including medicine, sports, and education13. For example, research has utilized fitness smartwatches for monitoring the physiological parameters of older individuals14,15, analyzing student behavior during STEM education13, and assessing physical activity in individuals16,17. Despite their extensive use, there are ongoing questions concerning the precision and reliability of the measures provided by these devices. Numerous investigations have attempted to validate the measurements obtained from wearable technology12,18,19,20. Previous studies investigated the accuracy of step counting and the functionality of sleep monitoring, providing insights into the devices&#39; performance in a wide range of settings and user demographics. However, the existing research indicates significant gaps in knowledge, such as contradictory findings across studies and insufficient validation against gold standard methodologies such as polysomnography21,22,23.
The purpose of this study is to validate the data obtained from fitness smartwatches and address the reliability of fitness smartwatches. The goal of evaluating the wearable device&#39;s sleep-tracking accuracy and other relevant metrics such as heart rate, blood oxygen level, and steps is to provide significant insights into the device&#39;s suitability for clinical and research applications that require a high level of accuracy and precision. This is intended to contribute to the growing body of research on wearable technology validation through rigorous methodology and solid statistical analysis, ultimately improving the ability to employ these tools for better health outcomes and well-being.

<table><tbody><tr><td>Sleep Diagnostic System&amp;nbsp;</td><td>Natus Neurology &amp;amp; Compumedics</td><td></td><td>Referred in manuscript as PSG machine</td></tr><tr><td>Xiaomi Mi Band 6</td><td>Xiaomi</td><td></td><td>Referred in manuscript as fitness smartwatch</td></tr></tbody></table>

assessing the accuracy of fitness smartwatch data for cardiovascular and physical activity monitoring: a validation study in digital health

Assessing the Accuracy of Fitness Smartwatch Data for Cardiovascular and Physical Activity Monitoring: A Validation Study in Digital Health

This study aims to validate the accuracy of low-cost fitness smartwatches by comparing their data with gold-standard measurements for cardiovascular and physical activity parameters. The study enrolled 50 subjects, 26 undergoing validation testing for heart rate, blood oxygen saturation (SpO2), and sleep data against polysomnography (PSG). Additionally, 24 subjects participated in the 3-Minute Walk Test (3MWT) and Stairs Climbing (SC), with step counts validated against manual video calculations. Results showed no significant difference between the device&#39;s measurements and gold standard values for shallow sleep, deep sleep, REM time, mean heart rate, minimum heart rate, and SpO2. However, the device significantly underestimated manually counted steps (p = 0.009 (3MWT); p = 0.012 (SC)), total sleep duration (p = 0.004), and wake time (p = 8.94&#160;&#215; 10-8) while overestimating maximum heart rate (p = 0.011). These findings highlight the importance of accurate validation and interpretation of wearable device data in clinical contexts. Given these limitations, excluding the device&#39;s readings in future analyses is recommended to maintain data reliability and research integrity. This study underscores the need for ongoing validation and improvement of wearable technology to ensure its reliability and effectiveness in healthcare.

Table 1 shows significant differences (p-value &lt; 0.05) between data from PSG and the fitness smartwatch during the first part of the validation test. The fitness smartwatch overestimated wake time (p &lt; 0.001), underestimated sleep duration (p = 0.004), and reported a higher maximal heart rate (p = 0.001). However, no significant differences were found between the PSG machine and the fitness smartwatch in the following measurements: shallow sleep time, deep sleep time, REM time, mean heart rate, min...

Wearable technology has low cost and offers convenient monitoring of physiological data. However, the accuracy and reliability of these devices require cautious assessment to ensure their effectiveness and safety for users. This report describes the validation process of a commercial smartwatch in monitoring physiological data and physical activity.

This study aims to validate the accuracy of low-cost fitness smartwatches by comparing their data with gold-standard measurements for cardiovascular and ...

assessing-accuracy-fitness-smartwatch-data-for-cardiovascular

Research

JoVE Journal

Medicine

Pulmonary rehabilitation is widely recognized in the management of respiratory diseases. A key component to successful pulmonary rehabilitation is adherence to the recommended exercise training. The purpose of the present protocol is to describe how continuous data tracking technology can be used to precisely measure adherence to a prescribed aerobic training intensity.

Using Continuous Data Tracking Technology to Study Exercise Adherence in Pulmonary Rehabilitation

A standardized evaluation method was developed for Wearable Mobility Monitoring Systems (WMMS) that includes continuous activities in a realistic daily living environment. Testing with a series of daily living activities can decrease activity recognition sensitivity; therefore, realistic testing circuits are encouraged for valid evaluation of WMMS performance.

Evaluation of a Smartphone-based Human Activity Recognition System in a Daily Living Environment

This protocol describes a method to quantify upper limb performance in daily life using wrist-worn accelerometers.

A Method for Quantifying Upper Limb Performance in Daily Life Using Accelerometers

Unobtrusive sensors and pervasive computing technology incorporated into the daily home life of older adults enables meaningful health and activity changes to be recorded continuously for months to years, providing ecologically valid, high frequency, multi-domain data for research or clinical use.

Methodology for Establishing a Community-Wide Life Laboratory for Capturing Unobtrusive and Continuous Remote Activity and Health Data

This protocol describes a randomized controlled trial as a method to test the effect of a video demonstration on the intra-individual difference between self-reported and accelerometer-based moderate-to-vigorous physical activity.

Visualization of Intensity Levels to Reduce the Gap Between Self-Reported and Directly Measured Physical Activity

The aim is to promote a new approach to cardiac rehabilitation (CR), using a unique remote patient monitoring system that will enable healthcare providers to monitor CR patients at home at low cost, with the intention of making CR services more accessible and improving compliance. The study is currently underway.

A Novel Digital Platform for a Monitored Home-based Cardiac Rehabilitation Program

This innovative device uses magneto-inertial sensors to permit gait and activity analysis in uncontrolled environments. Currently in the qualification process as an outcome measure in the European Medical Agency, one of the applications will be to serve as a clinical endpoint in clinical trials in neuromuscular diseases.

Home-Based Monitor for Gait and Activity Analysis

We describe a protocol for the patient-directed registration of a three lead bipolar electrocardiogram by a smartwatch that functions identically to the Einthoven leads from standard electrocardiograms. This enables patients to record electrocardiograms on their own immediately after the onset of symptoms.

Patient Directed Recording of a Bipolar Three-Lead Electrocardiogram using a Smartwatch with ECG Function

The present protocol introduces a non-commercial self-developed application for collecting real-time on-site data, including psychological scales, GPS location, heart rate, and blood-oxygen saturation level, as well as the application's operating procedures. An empirical study conducted in Taiwan in 2020 was used as an application example.

An Application for Pairing with Wearable Devices to Monitor Personal Health Status

Commercial smartwatches equipped with wearable sensors are increasingly being used in population studies. However, their utility is often constrained by their limited battery duration, memory capacity, and data quality. This report provides examples of cost-effective solutions to real-life technical challenges encountered during studies involving asthmatic children and elderly cardiac patients.

Setup of Consumer Wearable Devices for Exposure and Health Monitoring in Population Studies

M3-BREATHE is a mobile health (mHealth) system that integrates wearable sensors to monitor respiration, behaviour, and environmental exposures for personalized treatment and health evaluation in the real world.

Real-World M3-BREATHE: Toward Multimodal Mobile Monitoring of Behaviour, Respiration, and Exposures for Treatment and Health Evaluation

Learn how to validate fitness smartwatch accuracy for cardiovascular and physical activity monitoring using polysomnography and step tests. This protocol compares wearable data with clinical gold standards for reliable health tracking.

Assessing the Accuracy of Fitness Smartwatch Data for Cardiovascular and Physical Activity Monitoring: A Validation Study in Digital Health (Video) | JoVE Journal

Assessing the Accuracy of Fitness Smartwatch Data for Cardiovascular and Physical Activity Monitoring: A Validation Study in Digital Health

Explore More Videos