Comparing Sentence Observation Ratios During Reading and Writing in Japanese Students with Handwriting Difficulties

Summary

This article presents a protocol to identify the risk of handwriting difficulties (HD) in Japanese students using eye-tracking methods. Students at risk for HD in school settings can be identified by assessing the proportion of meaningless sentences in their stimulus observations, their writing fluency, and their motor coordination skills.

Abstract

Students with handwriting difficulties (HD) often show visual-motor challenges that affect sentence writing. However, few studies have compared the eye movements of HD and typically developing (TD) children when reading and writing. Recent studies have suggested that the proportion of time spent observing stimuli can help identify behavioral problems in children with developmental disabilities. This study compared the stimulus observation proportions between students with HD and TD peers. Japanese HD (n = 12) and TD participants (n = 15) were examined for differences in eye movement patterns when reading and writing Japanese sentences and compared reading and writing performance for meaningful and meaningless sentences. Eye movements were recorded while participants wrote several Japanese sentences. The results indicated that TD participants read and wrote sentences more fluently than HD peers, although no group differences were found for meaningful writing tasks. HD participants observed sentences more frequently in both reading and writing tasks than their TD peers and showed shorter observation spans and lower observation ratios during handwriting; however, their observation proportions during reading were comparable. Binary logistic regression revealed that letters per minute, the proportion of time spent on meaningless handwriting observations, and motor coordination scores were key indicators for identifying students with HD. These findings suggest that shorter observation spans and lower proportions of stimulus observation when writing reflect an HD student's tendency to merely glance at sentences rather than read them thoroughly.

Introduction

Handwriting is a complex behavior that requires the integration of visual, motor, and cognitive skills^1,2,3,4,5. Recent studies have shown that handwriting difficulties (HD) can be detected by having participants write cycloid loops on paper from left to right^2,3.In these studies, typically developing (TD) participants demonstrated similar visuomotor integration (VMI) skills for spatial and temporal arrangements regardless of whether their eyes were open or closed. However, students with HD showed better spatial and temporal arrangements but poorer writing performance when their eyes were closed. In contrast, despite poorer VMI skills, they wrote less accurately and fluently, with lower legibility, when their eyes were open. These results suggest that visual skill deficits interfere with fluent handwriting performance in students with HD^1,2,3. However, Lopez and Vaivre-Douret^2,3 did not directly measure visual performance data, such as eye-movement patterns, or compare the visual skills of HD and TD students. Therefore, comparing eye movement patterns during handwriting between TD and HD participants may help identify students at risk for HD in daily and school settings.

Few studies have examined eye-movement patterns in HD participants^4,5, despite the importance of comparing such patterns during handwriting⁶. Omori⁶ analyzed eye movement patterns in TD college students during sentence writing. He divided participants into two groups based on their VMI skills and assessed them using a standardized visual-motor integration test⁷. The results indicated that students with lower VMI skills made more and longer fixations and looked back more frequently at the sentence and their hands than students with higher VMI skills. In other words, poor VMI skills are associated with poor observing skills during handwriting. Poor handwriting performance results in longer observing times, more fixations, and longer fixation durations^2,3,6. Furthermore, when the number of looks back at one's hands and the stimuli increases, the stimulus observing ratio, defined as the total duration of stimulus observation relative to total handwriting duration, decreases. Consequently, HD participants spend a smaller proportion of their total writing time on actual handwriting, resulting in a lower observing ratio than TD participants^2,3. TD students with lower VMI skills also looked back at their hand movements and target stimuli more frequently and showed difficulties integrating visuomotor skills⁷. However, previous studies have only evaluated fixation data, not gaze data, which includes both fixations and saccades⁶. To better capture stimulus observation during handwriting, it is advisable to consider both gaze data, which include visit duration and visit count, and fixation data, which include fixation duration and count. Therefore, examining the relationship between visit duration and target stimuli as an observing ratio may help elucidate the visuomotor integration difficulties associated with handwriting.

Sita and Taylor reported that TD adults had fewer fixations, shorter mean fixations, and shorter visit durations when reading compared to writing single words⁸. However, it remains unclear whether TD and HD students show similar eye movement patterns when reading and writing sentences. Furthermore, most prior studies have focused on alphabetic languages^1,2,3,4,5,8, whereas only one study has investigated Japanese writing⁶. Therefore, the present study compared eye movement patterns in TD and HD students when reading and writing Japanese sentences. Because recent work has shown that students with reading difficulties can be identified through nonword reading^9,10, meaningless stimuli were included in this study in addition to meaningful ones. Steacy et al. reported that meaningless word reading skills are strongly associated with the subsequent development of meaningful word reading¹⁰. Although the response modalities differ, comparing meaningful and meaningless sentence writing may similarly help detect students with HD. Therefore, both stimulus types were prepared.

Writing Chinese characters or Japanese Kanji is expected to cause greater handwriting difficulty than writing Roman letters¹¹. Tse et al. compared Chinese and English handwriting development in TD and HD children and found greater difficulty in Chinese, reflecting problems with stroke formation¹¹. They also reported poorer VMI, poorer visual perception (VP), and poorer fine motor skills in HD than TD children, with no group differences in reading performance¹¹. In Japan, approximately 10% of elementary school students in mainstream classes have writing difficulties¹². Kono, Hirabayashi, and Nakamura demonstrated that students' writing fluency increased from 13.08 letters per minute (LPM) in 1st grade to 31.26 LPM in 6th grade¹³, whereas their reading fluency increased from 202.50 LPM in 1st grade to 461.90 LPM in 6th grade¹⁴. Another study using a digital pen reported longer continuous writing spans and fewer downward glances at the paper as handwriting skills developed¹⁵. Noda et al. found that Japanese students with developmental disabilities, including HD, have poorer fine motor skills and poorer sustained writing attention¹⁶. Taken together, these findings^11,13,15,16 suggest that Japanese students with HD exhibit slower writing LPM, shorter continuous writing spans, and more frequent looks back to the stimuli and their hands than TD peers¹³. Although the continuous writing span is typically measured using a digital pen, it can also be estimated from eye tracking by counting fixations. Calculating letters per fixation (L/FX) from eye movement data indicates how often individuals need to fixate on each letter, and the frequency of looking back can be indexed as an observing ratio using total visit duration to the stimuli relative to total writing duration. Therefore, I used an eye tracker to compare L/FX as the observing span and %VD as the observing ratio between HD and TD students.

To investigate eye movement differences when reading and writing Japanese sentences, and to compare performance for meaningful and meaningless sentences between HD and TD students, participants read and wrote several Japanese sentences while their eye movements were recorded. I predicted that HD students would exhibit lower reading and writing LPM¹³, fewer observation spans with shorter fixation durations⁶, and a lower proportion of stimulus observation when reading and writing compared to TD students^2,3,6. I expected distinct eye movement patterns, reduced observing spans, and lower observing ratios in HD relative to TD students.

Protocol

Written informed consent was obtained from participants and their parents or guardians before participation. The Institutional Review Board of Waseda University (No. 2020-176) approved this study.

1. Participants

1. Install the software on a computer from the website (see Table of Materials) to determine the sample size by power analysis.
   1. Click on the software to open it.
   2. In the Test family tab, select F test, and in the Statistical test tab, select ANOVA: Repeated measures, within-between interactions. Under Type of power analysis, choose A priori: Compute required sample size, given α , power, and effect size.
   3. Click Determine, then select Direct in the pop-up window. Enter the partial η² from the chosen previous study (0.20 in this study)⁶. Click Calculate, then Calculate and transfer to main window. Ensure that Effect size f in the Input parameters panel has been updated.
   4. Enter 2 for Number of groups and 4 for Number of measurements for this study. Click Calculate on the main page and confirm that the Total sample size is 12 for this study.
      NOTE: Based on the previously reported effect sizes from time-course analyses⁶, the power analysis indicated that at least 12 participants are required to detect a partial η² of 0.20 or higher with a three-way mixed ANOVA at p = 0.05 and 95% power.
2. Recruit school-aged students and confirm that all are enrolled in mainstream classes at public elementary, junior high, or high schools. In this study, 31 students (chronological age [CA] range, 7-17 years) participated.
3. Present Japanese words referencing "cat[neko]," "dog [inu]," and "bird[tori]", and instruct participants to write them. Confirm that participants can write the presented words correctly; otherwise, exclude them from the study.
4. Exclude any participant who cannot complete the assessments. Based on these criteria, four participants were excluded. Finally, 27 participants (16 boys, 11 girls) participated in this study.
5. Ask participants or their parents about their academic history, especially their reading and writing skills. Request disclosure of any diagnoses, if available, and ensure that diagnoses were made by a pediatrician using clinical criteria¹⁷.
6. Divide participants into two groups based on academic history and clinical criteria: HD (n = 12; 7 with LD and 5 with LD and autism spectrum disorder (ASD)) and TD (n = 15). Ensure that TD participants report no reading or writing problems and have not been diagnosed with any developmental disabilities.
7. Ask participants (or parents) to provide standardized scores, such as intelligence quotient (IQ) or developmental quotient (DQ), previously measured at hospitals or educational institutions, if available. Leave these fields blank if unavailable. In this study, all participants with HD had prior IQ or DQ scores (mean full-scale IQ = 97.80, SD = 24.32). TD participants did not have prior IQ scores; leave these fields blank.

2. Stimulus and apparatus

1. Prepare an eye tracker to record eye movements. Verify that the eye tracker has a measurement performance of 120 Hz or higher.
2. Use a display with 1920 × 1080 resolution and a laptop computer to present and control the stimuli. Ensure the laptop includes software to compute eye-movement coordinates and perform statistical analyses.
3. Instruct participants to sit approximately 70 cm from the eye tracker during recording. Perform a 5-point calibration before presenting the stimulus. This procedure is performed to establish measurement standards for the display and eye tracker. Click the calibration button and observe the colored dots on the display. When a dot has been fixated sufficiently, it will disappear, and a new dot will appear at a random location, such as the corner or center. Repeat the calibration until acceptable results are obtained in the eye-tracking software.
4. Prepare 12 sentences in Hiragana and Kanji, each two to four lines and 32-79 characters. Of these, use eight meaningful sentences describing daily-life content (e.g., Fish ecology, Train delay) and four meaningless sentences written only in Hiragana. Create meaningful sentences with age-appropriate vocabulary, particles, and Kanji at the 2nd-3rd grade level; include three to four Kanji characters per sentence. Create meaningless sentences by combining strings of three or four Hiragana characters. Present all sentences in 28-point font, spanning a single slide. Prepare a blank black slide. Prepare two A4 sheets with four horizontal lines for the writing test for each participant.
5. Convert the 12-sentence slides and the black slide to individual JPEG files. Figure 1 shows example sentence stimuli: panel A is a meaningful sentence, and panel B is a meaningless one. Open the eye-tracking software. In the console, create a new project and name it.
6. Open Design mode. Select Timeline 1 and set the calibration to the default 5-point mode. Add two meaningful sentence slides and one meaningless slide. Then set the proceed mode to key press so the experimenter can advance with the space bar. Insert the blank black slides before, between, and after the sentence slides, also set to key press. Prepare four timelines in the same manner, each with different sentence slides. In Timelines 1 and 3, present the meaningful sentence first, followed by the meaningless sentence; in Timelines 2 and 4, reverse the order.
7. Implement Hiragana-only short-sentence timelines for participants under 10 years of age. For older participants, use mixed Hiragana-Kanji sentence timelines.

Figure 1: Examples of sentence stimuli for reading and writing tasks. (A) A meaningful sentence stimulus, and (B) a meaningless sentence stimulus. Please click here to view a larger version of this figure.

3. Pre-Assessment: PVT-R

1. Sit face-to-face with the participant. Ask their date of birth; if they cannot respond accurately, ask their parents.
2. Present the PVT-R booklet to the participant. The booklet consists of 15 pages, each featuring four visual stimuli. Instruct the participant to listen to the spoken stimulus and touch the corresponding visual stimulus from the four choices.
3. Present each spoken stimulus once according to the instructions and have the participant select the corresponding visual stimulus from the four choices. Record the selection on the sheet, then present the next spoken stimulus. Each page includes six spoken-stimulus presentations.
4. Count the number of correct responses on each page and proceed to the next page until the participant makes three or more errors on two pages. The errors need not be consecutive; do not advance if this criterion is met. Alternatively, stop the assessment if the participant completes page 15 without meeting the error criterion.
5. Count the total number of correct and incorrect responses in the booklet, then calculate the adjusted score, standard score, and verbal age according to the manual.

4. Pre-assessment: Beery-VMI VI

1. Sit face-to-face with the participant. Ask which hand is dominant.
2. Present the VP booklet. Instruct participants to observe the sample geometric figure at the top of the page, then the choice figures at the bottom. Have them select the figure identical to the sample. Repeat this process until the participant has completed all 30 items, then proceed to the next task. Count the number of correct responses.
3. Present the motor coordination (MC) booklet. For the first 20 items, instruct participants to connect the darker dots to the lighter dots to form geometric figures. Instruct the participants to trace lines within the shapes without touching the outlines for the last 10 items. Inform them that the MC task must be completed within 5 min. Then, count the number of correctly connected figures and correctly traced shapes.
4. Count the total number of correct responses in the VP and MC booklets and calculate the standard score for visuomotor integration according to the manual.

5. Eye movement measurement

1. Attach the magnetic mounting bracket to the center of the display's bottom edge.
2. Connect the eye tracker to the display with a USB cable. Attach the eye tracker to the bottom of the display using the included magnetic bracket. Ensure the bracket and eye tracker are securely fastened.
3. Open the configuration software. Select the attached eye tracker and display. Click the calibration button and select the stimulus presentation display. Seat the participant in front of the display and adjust their position to an appropriate distance. Ensure the pre-calibration indicator turns green, confirming approximately 70 cm distance; adjust the seating as needed.
4. Click the calibration button and observe the colored dots on the display. When a dot has been fixated sufficiently, it will disappear, and a new dot will appear at a random location, such as the corner or center. Continue until all seven dots have been completed, then click Accept to finish calibration. Close the configuration software.
5. Open the eye-tracking software and the named project. Select the Recording mode and choose the recording eye tracker from the upper-left menu. Select the stimulus presentation display and enter the participant's name in the Participant tab. Select one of the four timelines and counterbalance the test order (reading vs. writing). For example, for Participant 1, start with reading in Timeline 1 and writing in Timeline 3; for Participant 2, begin with writing in Timeline 4 and reading in Timeline 2. Ensure both tests are completed for each participant. Wait for the participant to be seated in front of the display.
6. Instruct participants to sit approximately 70 cm from the eye tracker. Press Recording to proceed to calibration. Verify the appropriate distance by confirming the eye-tracker status indicator turns green.
7. Click Start calibration to proceed. Instruct participants to follow the white moving dots on the display. If calibration is acceptable, press Start to present the sentence slide for the reading or writing task; otherwise, click Recalibrate and repeat until successful.
8. Start the reading tests (see Figure 2A for details). Select Timeline 1 or 2.
   1. Complete steps 5.5 through 5.7. Instruct participants to read the presented sentence slide aloud as quickly as possible.
   2. Press the space bar to present a blank black slide. Prepare the participant for reading, then press the space bar to present the first sentence slide. Listen carefully and count the number of reading errors on the slide. When the participant finishes the final character, press the space bar to advance to the next blank black slide.
   3. Prepare the participant, then press the space bar to present the second sentence slide. Listen carefully and count the number of reading errors. When the participant finishes the final character, press the space bar to continue. Repeat for the third slide to complete the reading test. Recording stops automatically after the final key press.
   4. Click Save recording in the pop-up window to save the eye-movement data and return to the project home.
   5. Prepare for the writing test by selecting another timeline or finish the session if both tests are completed.
9. Start the writing tests (see Figure 2B for details). Select Timeline 3 or 4.
   1. Complete steps 5.5 through 5.7. Instruct participants to handwrite the presented sentence(s) onto an A4 sheet with four horizontal lines as quickly as possible.
   2. Press the space bar to present a black blank slide. Prepare the participant for writing; then, press the space bar to present the first sentence slide. Observe the writing and count the errors on the slide. Instruct participants to rewrite incorrect words, erasing the previous response with double lines. When the participant finishes the final character, press the space bar to advance to the next black blank slide.
   3. Collect the completed sheet and provide a new sheet to the participant.
   4. Prepare the participant, then press the space bar to present the second sentence slide. Observe the writing and count errors on the slide. Instruct participants to rewrite incorrect words, erasing the previous response with double lines. When the participant finishes the final character, press the space bar to continue. Repeat for the third slide to complete the writing test. Recording stops automatically after the final key press.
   5. Click Save recording in the pop-up window to save the eye-movement data and return to the project home.
   6. Prepare for the reading test by selecting another timeline or finish the eye-movement measurement if both tests are completed.

Figure 2: Eye movement measurements during reading and writing tasks. (A) A reading task trial, and (B) a writing task trial. Please click here to view a larger version of this figure.

6. Exporting eye movement data

1. After collecting the data from all the participants, open the eye-tracking software and go to the project home.
2. Choose the Analyze tab and select the AOI Tool.
   1. Select the rectangle tool to draw an area of interest (AOI) on the stimulus image. Adjust the rectangle to cover the entire text region. Eye-movement data will be calculated within the AOI. Name each AOI "MF_R_##" for meaningful reading, "ML_R_##" for meaningless reading, "MF_W_##" for meaningful writing, or "ML_W_##" for meaningless writing, replacing "##" with the appropriate numbers.
   2. Repeat this process to create and name AOIs for all stimuli used.
3. Choose the Analyze tab and select Metrics export.
   1. Observe the settings panel on the right side of the page.
   2. Select Excel report under Export format, and ensure I-VT filter and Noise reduction are selected for Gaze filter and Pupil diameter filter, respectively.
   3. Select the checkboxes for All recordings in the Recording section, Entire recordings in the Time of interest section, and All areas of interest in the Area of interest section.
   4. On the left side of the window, open Select metrics for report.
   5. Check Select all, then click Export.
   6. Open the exported Excel file and verify all exported data.
   7. Conduct statistical analyses using the exported data.

7. Data analysis

1. Set dependent variables for analysis.
   1. Score the PVT-R to obtain standard scores according to the manual.
   2. Score the Beery-VMI 6 to obtain ability scores for VP and MC according to the manual.
   3. Open the exported Excel file and select the Interval duration tab. Note that interval duration is defined as the time from pressing the space bar to present a sentence stimulus until the final character is produced and the space bar is pressed to advance to the next slide.
   4. Calculate letters per minute (LPM) as a behavioral performance measure. Divide the number of letters in the sentence by the response time, then multiply by 60 (e.g., 60 letters ÷ 15 s × 60 = 240 LPM). Note that higher LPM indicates better performance.
   5. Open the Average fixation duration tab. Multiply by 1,000 to express FD in milliseconds. Organize the data in an Excel sheet by participant, test (reading vs. writing), and sentence condition.
   6. Open the Fixation count tab. Organize the data in an Excel sheet by participant, test (reading vs. writing), and sentence condition. Calculate letters per fixation (L/FX) as the observing span: divide the number of letters in the sentence by the fixation count (e.g., 60 ÷ 15 = 4.00 L/FX). Note that a longer observing span corresponds to a larger L/FX.
   7. Open the Total visit duration tab. Organize the data in an Excel sheet by participant, test (reading vs. writing), and sentence condition. Reuse the interval duration for the next step. Calculate percentage visit duration (%VD) as the observing ratio: divide total visit duration by interval duration within the same condition and multiply by 100 (e.g., 15 s ÷ 30 s × 100 = 50%). Note that a higher percentage of VD indicates a greater observing ratio.
2. Statistical analysis
   1. Open the software on the computer to analyze the data.
   2. Arrange the data format to prepare for statistical analysis.
   3. Conduct independent t-tests to evaluate participants' verbal abilities and visuomotor integration abilities.
   4. Conduct mixed factorial ANOVAs on reading and writing LPM and on eye-movement measures (mean FD, L/FX, %VD) using a 2 (group: HD vs. TD; between-subjects) × 2 (test: reading vs. writing; within-subjects) × 2 (sentence: meaningful vs. meaningless; within-subjects) design.
   5. Conduct binary logistic regression analyses to examine whether screening measures and eye-movement patterns predict HD risk across all participants.

Results

Participant's profile analysis
Table 1 shows the participants' verbal abilities and visuo-motor integration skills. The mean chronological age of HD participants was 11.11 years (range: 8.33-14.83 years) and 11.97 years for TD participants (range: 7.67 - 17.50 years). The PVT-R [18] indicated that HD participants had lower standardized scores than TD participants (8.92 vs. 12.75 [t (18) = 2.25, p = 0.03, Cohen's d = 5.00]). VP and MC scores [8] of the two g...

Discussion

This study examined differential eye-movement patterns in 12 HD and 15 TD participants when reading and writing meaningful and meaningless Japanese sentences. As shown in Figure 3A, HD participants had slower LPM scores for reading and writing compared to TD participants, suggesting that HD participants experience greater difficulty in these tasks. HD participants often also have reading difficulties²¹. Therefore, future studies should use reading materials matched to...

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
Beery VMI-6	Pearson	https://www.pearsonassessments. com/en-us/Store/Professional- Assessments/Motor-Sensory /Beery-Buktenica-Developmental -Test-of-Visual-Motor-Integration -%7C-Sixth-Edition/p/100000663 ?srsltid=AfmBOooTl4STWGQhq ArZwKdX_mSiV0nHgJWPfjddJ 9z0erbWm7Qq6H9v	Standardized test to assess visuo-motor integration skills
Computer display	Dell	P2419H	resolution 1920×1080
Eye tracker	Tobii Technology Japan	X3-120	Portable eye tracker
G*Power	Heinrich-Heine-Universität Düsseldorf	https://www.psychologie.hhu.de /arbeitsgruppen/allgemeine- psychologie-und-arbeitspsychologie/gpower	G*Power analysis software
IBM SPSS statistics	IBM	IBM SPSS statistics	version 29.0.2.0
Laptop computer	Dell	Precision 5540	Windows 11
PVT-R	Nihon Bunkakagakusya	https://www.nichibun.co.jp/seek/kensa/pvt_r.html	Standardized test to assess verbal age
Tobii Pro Lab	Tobii Technology Japan	Tobii Pro Lab	Eye movement analysis software