Waiting
Login processing...

Trial ends in Request Full Access Tell Your Colleague About Jove

Medicine

Digital Handwriting Analysis of Characters in Chinese Patients with Mild Cognitive Impairment

doi: 10.3791/61841 Published: March 11, 2021
* These authors contributed equally

Summary

This study proposed a digital handwriting analysis of characters in individuals with mild cognitive impairment to find more information than is revealed by traditional pencil–paper handwriting analysis.

Abstract

An increasing amount of evidence shows that cognitive deficits and movement dysfunctions are not separated. Patients with mild cognitive impairment (MCI) can manifest fine motor disorders of the upper extremities. Handwriting is a complex and unique human activity involving both motor and cognitive coordination. Researchers from western countries have discovered that patients with MCI have abnormal handwriting features. However, no relevant studies have been conducted in the Chinese population. Owing to the cross-culture phenomenon of handwriting, the aim of this study is to find new handwriting tasks to demonstrate the differences in handwriting features between elderly patients with MCI and age-matched healthy individuals.

Introduction

or Start trial to access full content. Learn more about your institution’s access to JoVE content here

Mild cognitive impairment (MCI) is considered as a transitional but progressively degenerative cognitive phase that precedes the onset of Alzheimer’s disease (AD)1. It has been reported that the AD progression rate per year is 15% while nearly 75% of moderate and severe MCI cases might remain undiagnosed2. Recent studies have reported that patients with MCI have difficulty in some aspects of fine motor tasks3, and those patients who showed motor disorders, such as slow gait, had a high risk of dementia4.

Handwriting is a complex human activity that entails an intricate blend of cognitive, kinesthetic, and perceptual-motor components including visual and kinesthetic perception, motor planning, eye-hand coordination, visual-motor integration, dexterity, and manual skills1. Handwriting analysis has been utilized to detect cognitive and motor dysfunctions in many types of neurodegenerative diseases, such as AD and Parkinson’s disease (PD)5. In addition, some aspects of handwriting problems have been reported to be an indicator for MCI and related to disease progression6. As a majority of the population uses language, studies that investigate handwriting analysis in Chinese speakers (especially simplified Chinese characters) are still lacking.

There have been several articles that have investigated handwriting abnormalities or “agraphia” in individuals with MCI. For example, by utilizing the traditional pencil–paper methods, Zhou and colleagues tried to unveil the distinct writing abilities between patients with MCI and individuals without MCI. The differences between the groups were not obvious, with the exception of writing errors7. Kawa et al. found handwriting features in patients with MCI using a smart pen, which could dynamically analyze the stroke and pen speed during writing2. WACOM hardware and MovAlyzeR software can detect more real-time information compared with traditional pencil and paper methods and smartpen paper methods. Therefore, dynamic handwriting data, such as pen-down pressure, velocity, acceleration, and jerk, have been found to be a new focus of handwriting analysis relative to static data, such as letter size and space between words2.

However, another phenomenon that cannot be overlooked is the cross-cultural effect of handwriting. The writing systems of different nations are not always the same (e.g., English letters are written from left to right while Hebrew letters are written from right to left)8. In this issue, even reviews have confirmed the effectiveness of handwriting analysis9,10 in alphabetic languages, and the wide gap between Chinese characters and western letters has hindered the interchange capability of handwriting analysis in the methods and results11 of these studies.

There exist several major dissimilarities between western languages (e.g., English) and Chinese. First, there are many more horizonal movements of the pen tip during Chinese character writing compared with letter writing12. Second, unlike the alphabet language, which is associated with phonemes, Chinese is considered to be logographic7. As a result, most Chinese characters have their own unique stroke order, and the width and height of strokes need to be strictly limited. Otherwise, unlimited widths and heights might cause increased illegibility11 (“Equation 1” and “Equation 2” are completely different Chinese characters. Also, “Equation 3”, “Equation 4”, and “Equation 5” are different Chinese characters).

Equation 6” (pronounced “Zheng”) is a typical, simple, and commonly used Chinese character that nearly every Chinese speaker with a two-year educational level can read and write. It has been selected as a writing task in previous Chinese handwriting analysis studies6,12. Researchers decided to use “Equation 6” as the writing task because it is “square-like” and composed of five strokes, all of which are horizonal (#1, #3, #5 stroke, from left to right) or vertical (#2, #4 stroke, from up to down) (Figure 1). According to many fine motor studies, fulfilling the #3 stroke (horizonal) and the #4 stroke (vertical) requires pure wrist and finger movements, respectively6,12,13. As a result, the stroke velocity of both strokes could be a proper manifestation14.

In addition, pen pressure during handwriting is a handwriting feature that has been shown to outperform other kinematic features in reflecting motor control5,15. However, there are no relevant studies in Chinese patients, though positive results have been confirmed by research groups from the Czech Republic, Spain, Israel, and other countries8,16,17.

The signature has been commonly used as a handwriting task in numerous studies5. In general, a signature requires little thinking or in-air time18. “In-air” is defined as when the pressure of the pen tip to the screen is 0 during the handwriting, and “in-air time” is the sum of the time “in-air” during the handwriting. Individuals who suffer from many neuropsychiatric disorders might have deficits in psychomotor control, and thus they exhibit increased in-air time of the signature. For example, Rosenblum et al. found that Israeli patients with depression and Parkinson’s disease showed longer in-air time compared with healthy controls during writing their own name in Hebrew8,19. As Chinese characters have their own shape, in this study, it was decided to use the in-air length tortuosity in segmentation between characters during writing the name as a potential indicative parameter. Tortuosity, defined by the ratio of the arc length to the Euclidean distance between end points, is a measure of curvature, and therefore indexes the smoothness of a specific writing output20.

Subscription Required. Please recommend JoVE to your librarian.

Protocol

or Start trial to access full content. Learn more about your institution’s access to JoVE content here

Our study was approved by the Academic Ethics Committee of the Biological Sciences Division of the Chinese PLA General Hospital in Beijing, China.

1. General aspects of method development

  1. Use a USB digitizer (e.g., Wacom Cintiq Pro 16) and a handheld stylus pen for the handwriting movements. The detailed specifications of the digitizer are as follows: external dimensions (width x depth x height) 410 x 265 x 17.5 mm, spatial resolution 3840 x 2160 dots, pixel size 0.090 x 0.090 mm, temporal resolution 30 ms, and a pressure level of 8,192.
  2. Connect a laptop PC to the digitizer to collect and exhibit the handwriting traces.
  3. Use a software (e.g., Neuroscript MovAlyzeR) for data recording, processing, and analyzing.
  4. Patient inclusion/exclusion criteria
    1. Recruit MCI participants who present with a memory complaint, an objectively impaired memory function, intact activities of daily living, and the absence of dementia21. Besides, they should have the educational level of more than 2 years of preliminary school in mainland China, otherwise, they might have difficulty in writing Chinese characters.
    2. Exclude participants who have obvious visual and upper limb disability.

2. Handwriting task

  1. Run the software and a non-inking stylus pen.
  2. Create an example of Chinese characters on the writing area of the digitizer (see Figure 1).
  3. Allow the subjects to position the writing area into a comfortable position.
  4. Allow the subjects to write on the writing area and accommodate the pen and the surface of the writing area.
  5. Set the sampling rate in the software at 200 Hz.
  6. Instruct the subjects to write his/her name in Chinese with the dominant hand.
    NOTE: A signature in either cursive or a printed version is acceptable, as the subject wished.
  7. Instruct the subjects to write the Chinese character “Equation 6” (pronounced “Zheng”) with the dominant hand.
    NOTE: The Chinese character “Equation 6” in a printed version is acceptable.
    1. Remind the subjects to write in a printed version prior to beginning handwriting.
      NOTE: Ensure that the subject sits and writes in an upright position.
  8. Keep instructions visible during each trial.
    1. Repeat the handwriting trial three times.
    2. If the character “Equation 6” was in written in the wrong stroke order, stop the trial and trace and show the subject how to write the character in the correct stroke order.
    3. If any hesitation was derived from a lack of knowledge, stop the trial and show the subject how to write the character correctly.

3. Data analysis

  1. Run the software; right click on Experiment and select Properties.
  2. Select Processing, and then select Segmentation.
  3. Click on Add first segmentation at any rate, Add last segmentation at any rate, and Move segmentation point to nearest pendown if on a penlift in Segmentation Flags.
  4. Click on At pendown trajectories in Segmentation Methods.
    NOTE: All of these adjustments for the default mode were done to improve the analysis of the Chinese handwriting.

4. Parameter calculation

  1. Run the software, select the subjects in “Equation 6”, and click on Handwriting Trials.
  2. Use the tracing system and trace the handwriting process and stroke order of “Equation 6” step by step.
  3. Find the segmentation of stroke #3 of “Equation 6” and read out the “Average Absolute Velocity” in “extracted data”.
    NOTE: The handwriting analysis software will automatically calculate the “Average Absolute Velocity” of each segmentation.
    CAUTION: Stroke #3 of “Equation 6” is a horizonal movement (from left to right) of the pen tip that is shorter than Character 1 and Character 5 (Figure 1A).
  4. Find the segmentation of stroke #4 of “Equation 6” and read out of the “Average Absolute Velocity” in the “extracted data”.
    CAUTION: Stroke #4 of “Equation 6” is a vertical movement (from up to down) of the pen tip that is shorter than Character 2 (Figure 1).
  5. Read out the “Pen Pressure” of each segmentation in the “extracted data” and obtain an “Average Pen Pressure” of “Equation 6”.
    NOTE: The handwriting analysis software will automatically calculate the “Average Pen Pressure” of each segmentation.
  6. Run the software, select the subjects in “Equation 6”, and click on Handwriting Trials.
  7. Using the tracing system, trace the handwriting process and stroke order of the signature step by step.
  8. Find the segmentation of the stroke between the characters and read out the “Absolute Size” and “Road Length” in the “extracted data”.
  9. Obtain the in-air length tortuosity in the segmentation between characters according to the equation.
    NOTE: The segmentation of the stroke between characters was an in-air segmentation (Figure 2).
    1. Calculate the in-air length tortuosity: 1-Absolute Size/Road Length %.
      NOTE: Tortuosity, defined by the ratio of the arc length to the Euclidean distance between end points, is a measure of curvature, and therefore indexes the smoothness of a specific writing output20. A highly tortuous curve has several bends or curves, whereas a low tortuous curve is one with relatively wide loops/curves and more straightness.
      CAUTION: Most Chinese names are composed of two or three characters. If the signature has two characters, there is only one segmentation of stroke between characters. If the signature has three characters, there are two stroke segmentations between characters. In-air length tortuosity in the segmentation between characters would be an average value.

5. Statistical analysis

  1. Evaluate group differences using a Student’s t-test. A P value less than 0.05 was considered statistically significant. Conduct all statistical analyses using the SPSS 22.0 statistical software package.

Subscription Required. Please recommend JoVE to your librarian.

Representative Results

or Start trial to access full content. Learn more about your institution’s access to JoVE content here

The demographic data of the subjects demonstrated that all the groups matched well in age, gender, educational level, dominant hand, and other parameters.

As shown in Table 1, during the writing of the Chinese character “Equation 6”, elderly subjects with MCI exhibited a lower average absolute velocity of the #3 (2.46 ± 0.40 vs 1.82 ± 0.55, P = 0.001) and #4 stroke (2.61 ± 0.46 vs 1.93 ± 0.50, P < 0.001) and a higher average pen pressure (237.43 ± 39.77 vs 281.99 ± 37.70, P = 0.001) compared with healthy elderly subjects. Additionally, during the signing of Chinese names, the elderly subjects with MCI exhibited a higher in-air length tortuosity in segmentations between the characters compared with the healthy elderly subjects (12.57 ± 6.96 vs 31.66 ± 7.53, P < 0.001).

Figure 1
Figure 1: Chinese character “Equation 6” written in the software. (A). The Chinese character “Equation 6” with only the in-screen segmentations exhibited. The red circles are the beginnings and ends of the segmentations. The blue lines are the in-screen segmentation traces. (B). The Chinese character “Equation 6” with both the in-air and in-screen segmentations exhibited. The blue lines are the in-screen segmentation traces. The grey lines are the in-air segmentation traces. Please click here to view a larger version of this figure.

Figure 2
Figure 2: Chinese signatures of healthy elderly subjects and elderly subjects with MCI. (A). The Chinese signature “Equation 7” (a subject of the healthy elderly group) with only the in-screen segmentations exhibited. The red circles are the beginnings and ends of segmentations. The blue lines are the in-screen segmentation traces. (B). The Chinese signature “Equation 7” (a subject of the healthy elderly group) with both in-air and in-screen segmentations exhibited. The blue lines are the in-screen segmentation traces. The grey lines are the in-air segmentation traces. The red areas emphasize the in-air length segmentations between characters (tortuosity = 5.34%). The left area is the segmentation between “Equation 8” and “Equation 9" (absolute size = 2.2226; road length = 2.4658; tortuosity = 9.98%). The right area is the segmentation between “Equation 9” and “Equation 10” (absolute size = 2.9607; road length = 2.9821; tortuosity = 0.71%). (C). The Chinese signature “Equation 11” (a subject of the elderly with MCI group) with only the in-screen segmentations exhibited. The red circles are the beginnings and ends of the segmentations. The blue lines are the in-screen segmentation traces. (D). The Chinese signature “Equation 11” (a subject of the elderly with MCI group) with both in-air and in-screen segmentations exhibited. The blue lines are the in-screen segmentation traces. The grey lines are the in-air segmentation traces. The red area emphasizes the in-air length segmentations between characters (absolute size = 1.2100; road length = 1.7072; tortuosity = 29.12%). Please click here to view a larger version of this figure.

Healthy Elderly Elderly with MCI P Value
N=20 N=20
Gender (male/female) 10/10 8/12 0.74
Age (years) 69.70±4.51 70.39±3.42 0.602
Dominant hand (Right%) 100 100
Educational (years) 9.60±3.72 8.22±3.30 0.237
MMSE (score) 28.90±0.79 26.33±0.77 <0.001
Average absolute velocity of 3# stroke of “Equation 6 2.46±0.40 1.82±0.55 0.001
Average absolute velocity of 4# stroke of “Equation 6 2.61±0.46 1.93±0.50 <0.001
Average pen pressure of “Equation 6 237.43±39.77 281.99±37.70 0.001
Tortuosity of in air length “Signature” (%) 12.57±6.96 31.66±7.53 <0.001
MCI: Mild Cognitive Impairment
Character 3 ”Equation 6” is a horizonal movement.
Character 4 ”Equation 6” is a vertical movement.

Table 1: Demographic and handwriting analysis data of subjects.

Subscription Required. Please recommend JoVE to your librarian.

Discussion

or Start trial to access full content. Learn more about your institution’s access to JoVE content here

The critical steps in the protocol confirm the legibility of “Equation 6”. In detail, within an entire character, the #3 stroke needs to be shorter than the other horizonal strokes, and the #4 stroke needs to be shorter than the #2 stroke. More specifically, more attentional resources are needed during the writing of the #3 stroke and #4 stroke6,12, and both strokes have a similar length limit. An inappropriate stroke length might have given rise to a bias in the detection of velocity.

The software with the digitizer screen is an on-line data collecting software without a pencil–paper-like shape. To start and stop writing, subjects needed to follow the instructions of the researchers or the prompting instruments. These explicit conditions could be stressors for subjects, distracting their attentional resources and affecting the performance of the handwriting. The troubleshooting of this method is even more severe in patients from rural areas who are not familiar with electronic devices. Enough warm-up time could be helpful. Another way to minimize the effects of this limitation is to place a paper sheet on top of the digitizer. In addition, devices that can collect off-line data with pencil–paper-like shapes, such as Smartpen plus Livescribe notebook, could be another modification. As far as we know, data from MovAlyzeR software and Smartpen will be compatible in the near future.

First, because the primary focus was on the dynamic analysis of handwriting, static parameters, such as character width and height, were not included and analyzed in the current study. Actually, “Equation 6” has been confirmed as a good writing task for detecting micrographia in PD12. Second, some researchers chose to limit the size of the character during the handwriting task (e.g., 1 cm, 2 cm, and 4 cm in amplitude)19. As has been found, subjects require more time to write in a smaller amplitude relative to a larger one. A definite amplitude was not set in this study while the shape of Chinese characters might be an implicit limit for a particular stroke.

To our knowledge, this is the first study of a digital handwriting analysis for simplified Chinese character users. More handwriting tasks regarding simplified Chinese characters can be used to find cognitive deficits and motor dysfunctions in patients with neuropsychiatric disorders.

A digital handwriting analysis can complement traditional pencil–paper cognitive tests, such as Trail-Making Test, MMSE, Montreal Cognitive Assessment, and others17,22. Analyzing handwriting features during a cognitive test is a new paradigm for motor-cognitive dual tasks23. This method might be of help for diagnosing motor cognitive risk syndrome and cerebral small vessel disease.

Subscription Required. Please recommend JoVE to your librarian.

Disclosures

The authors have nothing to disclose.

Acknowledgments

We thank Professor Hans-Leo Teulings from Neuroscript LLD for the digital technology support.

This study was supported by the Wu Jieping Foundation (Grant No.: 320.6750.18456).

Materials

Name Company Catalog Number Comments
Microsoft Surface Pro 2 computer
MovAlyeR 3.4 software
WACOM Cintiq digitizer

DOWNLOAD MATERIALS LIST

References

  1. Werner, P., et al. Handwriting process variables discriminating mild Alzheimer's disease and mild cognitive impairment. Journals of Gerontology. 61, (4), 228-236 (2006).
  2. Kawa, J., et al. Spatial and dynamical handwriting analysis in mild cognitive impairment. Computers in Biology and Medicine. 82, 21-28 (2017).
  3. De Paula, J. J., et al. Impairment of fine motor dexterity in mild cognitive impairment and Alzheimer's disease dementia: association with activities of daily living. Revista Brasilra De Psiquiatria. 38, (3), 235-238 (2016).
  4. Takehiko, D., et al. Combined effects of mild cognitive impairment and slow gait on risk of dementia. Experimental Gerontology. 110, 146-150 (2018).
  5. Impedovo, D., et al. Dynamic handwriting analysis for the assessment of neurodegenerative diseases: a pattern recognition perspective. IEEE Reviews in Biomedical Engineering. 12, 209-220 (2019).
  6. Yu, N. Y., et al. Characterization of the fine motor problems in patients with cognitive dysfunction - A computerized handwriting analysis. Human Movement Science. 65, (17), 30841-30842 (2019).
  7. Zhou, J., et al. Characteristics of agraphia in Chinese patients with Alzheimer's disease and amnestic Mild Cognitive Impairment. Chinese Medical Journal. 129, (13), 1553-1557 (2016).
  8. Rosenblum, S., et al. Handwriting process variables among elderly people with mild major depressive disorder: a preliminary study. Aging Clinical & Experimental Research. 22, (2), 141-147 (2010).
  9. Caligiuri, M. P., et al. Signature dynamics in Alzheimer's disease. Forensic Science International. 302, 109880 (2019).
  10. Thomas, M., et al. Handwriting analysis in Parkinson's disease: current status and future directions. Movement Disorders Clinical Practice. 4, (6), 806-818 (2017).
  11. Leung, S. C., et al. A comparative approach to the examination of Chinese handwriting-The Chinese character. Journal of the Forensic Science Society. 25, 255-267 (1985).
  12. Ma, H. I., et al. Progressive micrographia shown in horizontal, but not vertical, writing in Parkinson's disease. Behavioural Neurology. 27, (2), 169-174 (2013).
  13. Yan, J. H., et al. Alzheimer's disease and mild cognitive impairment deteriorate fine movement control. Journal of Psychiatric Research. 42, 1203-1212 (2008).
  14. Johnson, B. P., et al. Understanding macrographia in children with autism spectrum disorders. Research in Developmental Disabilities. 34, (9), 2917-2926 (2013).
  15. Afonso, O., et al. Writing difficulties in Alzheimer's disease and Mild Cognitive Impairment. Reading and Writing. 32, (1), 217-233 (2019).
  16. Drotár, P., et al. Evaluation of handwriting kinematics and pressure for differential diagnosis of Parkinson's disease. Artificial Intelligence in Medicine. 67, 39-46 (2016).
  17. Garre-Olmo, J., et al. Kinematic and pressure features of handwriting and drawing: preliminary results between patients with mild cognitive impairment, Alzheimer disease and healthy controls. Current Medicinal Chemistry. 14, 960-968 (2017).
  18. Cohen, J., et al. Digital clock drawing: differentiating "thinking" versus "doing" in younger and older adults with depression. Journal of the International Neuropsychological Society. 20, (9), 920-928 (2014).
  19. Rosenblum, S., et al. Handwriting as an objective tool for Parkinson's disease diagnosis. Journal of Neurology. 260, (9), 2357-2361 (2013).
  20. Grace, N., et al. Do handwriting difficulties correlate with core symptomology, motor proficiency and attentional behaviours. Journal of Autism and Developmental Disorders. 47, (4), 1-12 (2017).
  21. Petersen, R. C. Mild cognitive impairment as a diagnostic entity. Journal of Interactive Marketing. 256, (3), 183-194 (2004).
  22. Ishikawa, T., et al. Handwriting features of multiple drawing tests for early detection of Alzheimer's Disease: A preliminary result. Studies in Health Technology and Informatics. 264, 168-172 (2019).
  23. Herold, F., et al. Thinking while moving or moving while thinking-concepts of motor-cognitive training for cognitive performance enhancement. Frontiers in Aging Neuroscience. 10, 228 (2018).
Digital Handwriting Analysis of Characters in Chinese Patients with Mild Cognitive Impairment
Play Video
PDF DOI DOWNLOAD MATERIALS LIST

Cite this Article

Zhào, H., Zhang, Y., Xia, C., Liu, Y., Li, Z., Huang, Y. Digital Handwriting Analysis of Characters in Chinese Patients with Mild Cognitive Impairment. J. Vis. Exp. (169), e61841, doi:10.3791/61841 (2021).More

Zhào, H., Zhang, Y., Xia, C., Liu, Y., Li, Z., Huang, Y. Digital Handwriting Analysis of Characters in Chinese Patients with Mild Cognitive Impairment. J. Vis. Exp. (169), e61841, doi:10.3791/61841 (2021).

Less
Copy Citation Download Citation Reprints and Permissions
View Video

Get cutting-edge science videos from JoVE sent straight to your inbox every month.

Waiting X
Simple Hit Counter