Name: Using the Visual World Paradigm to Study Sentence Comprehension in Mandarin-Speaking Children with Autism (Video) | JoVE Journal
Uploaded: 2018-10-03T15:00:00.000+00:00
Duration: 6 min 15 s
Description: Learn how to study sentence comprehension in Mandarin-speaking children with autism using the Visual World Paradigm with eye tracking. This method captures real-time processing of morphological cues with minimal cognitive demands.

ai_agent_IP

ai_agent

maintenance

instantTrialDurationMinutes

isBiopharma

enableNewMyJoveSectionDesign

educationFeatures

seoMetaApi

ai_copilot_institution_blacklist

ai_chat_institution_blacklist

freeAccessWindow

ssoRoleCapture

scheduledMaintenanceBanner

joveLabsFeatures

disable_autoplay_for_bots

extended_trial_duration

5 minutes

This work was funded by the National Social Science Foundation of China [16BYY076] to Peng Zhou and the Science Foundation of Beijing Language and Cultural University under the Fundamental Research Funds for the Central Universities [15YJ050003]. The authors are grateful to the children, parents, and teachers at the Enqi Autism Platform and Taolifangyuan Kindergarten in Beijing, China, for their support in running the study.

This study has been approved by the Ethics Committee of the School of Medicine at Tsinghua University. Informed consent has been obtained from all individual participants included in the study.
1. Participant Screening and Study Preparation
<ol>
	<li>Recruit Mandarin-speaking preschool children with autism. 
	NOTE: Their diagnoses should be confirmed by pediatric neurologists at hospitals using DSM-IV-TR40 or DSM-541 and, ideally, the number of participants should be no less than 15. The present study recruited 25 participants with confirmed diagnoses.</li>
	<li>Evaluate each participant independently using gold-standard diagnostic instruments like the Autism Diagnostic Observation Schedule42.</li>
	<li>Measure the verbal IQ of participants using the Wechsler Preschool and Primary Scale of Intelligence-IV (CN), a standardized IQ test designed for Mandarin-speaking children between the ages of 2-6 and 6-1143. 
	NOTE: The verbal IQ scores of the children with autism in the present study were all above 80. They were all high-functioning children with autism.</li>
	<li>Calculate each participant&#39;s mean length of utterance (MLU) by dividing the total number of words by the number of utterances in each speech sample. Record 100 utterances for each participant from either their interactions with parents or with teachers. Then, calculate the MLU by dividing the total number of words in each participant&#39;s utterances by 100. 
	NOTE: MLU indicates the participant&#39;s sentence complexity levels.</li>
	<li>Recruit TD children. Ideally match the TD children to the children with autism for age (TD group 1), MLU (TD group 2), and verbal IQ (TD group 3). 
	NOTE: The present study recruited 50 TD children (25 boys and 25 girls) from local kindergartens. 25 matched to the children with autism for age and 25 matched to the children with autism for both MLU and verbal IQ.</li>
</ol>
2. Warm-up Session
<ol>
	<li>Invite the participants for a warm-up session before the actual test. Introduce the participant to the research environment and interact with him or her to establish a good rapport. 
	NOTE: This can be done on the same day as the testing session or organized on a different day. In the warm-up session, two experimenters are typically involved and interact with the participant using toys and props.</li>
</ol>
3. Conditions and Experimental Design
<ol>
	<li>Construct the test stimuli. Create 12 target items, each comprised of a visual stimulus, and two spoken sentences containing the morphological markers BA and BEI, respectively. Construct the spoken sentences using the same structure: morphological marker + noun phrase (NP) + adverb + verb phrase (VP) (see Examples 1a and 1b below). 
	NOTE: The marker BA indicates that the following NP is the recipient of the holding event (see 2a), and BEI indicates that the following NP is the initiator of the event (see 2b). The subject NP of a sentence in Mandarin can often be omitted when the referent of the NP is contextually available. 
	 
	Example: 
	(1)&#160;a. BA shizi qingqingdi bao-le&#160; &#160; &#160; qilai. 
	&#160; &#160; &#160; &#160; &#160;BA&#160; lion&#160; &#160;gently&#160; &#160; &#160; &#160;hold&#160; &#160; &#160; &#160; &#160;up 
	&#160; &#160; &#160; &#160; &#160;Meaning: Someone gently holds the lion. 
	&#160; &#160; &#160;b. BEI shizi qingqingdi bao-le&#160; &#160; &#160; qilai. 
	&#160; &#160; &#160; &#160; &#160;BEI lion&#160; &#160;gently&#160; &#160; &#160; &#160;hold&#160; &#160; &#160; &#160; &#160; up 
	&#160; &#160; &#160; &#160; &#160;Meaning: Someone is gently held by the lion. 
	(2)&#160;a. BA + [NP]Recipient 
	&#160; &#160; &#160;b. BEI + [NP]Initiator
	<ol>
		<li>Use Pixelmator (or another image editor) to create visual images. Open Pixelmator. Click on the Pixelmator icon. Create a visual image from a template. Click Show Details in the template chooser. Double-click the template to open it. Adjust the width, height, resolution, and color depth from the pop-up menus. Enter the relevant parameters. Click OK.</li>
		<li>Use Praat (or another audio editor) to construct spoken sentences. Set up the microphone. Open Praat. Click on the Praat icon. Select Record Mono sound from the New menu. Set the recording conditions by clicking the sample rate option of 44100. Click the Record button.</li>
		<li>Record the spoken sentences by asking a native Beijing Mandarin-speaker to produce the sentences in a child-directed manner. Save the recordings by clicking Save. 
		NOTE: Typically, 12 to 16 target items are constructed for sentence comprehension studies with children. Test stimuli can be created using other image and audio editors for a visual world study.</li>
	</ol>
	</li>
	<li>Construct visual images, each containing two pictures. The two pictures depict the same event involving the same characters. Reverse the event roles (initiator or recipient) of the two characters in the two pictures. Make one picture compatible with the construction containing BA (BA-target event) and one with the construction containing BEI (BEI-target event). An example is provided in Figure 1. 
	NOTE: This figure has been reprinted with permission from Zhou and Ma (2018)19.</li>
	<li>Counterbalance and randomization: divide the target trials into two experimental lists, with a participant seeing each visual stimulus but listening to only one of the recorded sentences for the stimulus. Counterbalance the spoken sentences containing BA and BEI across the two experimental lists, with 6 constructions containing BA and 6 containing BEI. Add 12 filler items to each experimental list and arrange the target and filler trials in a random order. Randomly assign the participants to the two lists.</li>
</ol>
4. Experimental Procedure
<ol>
	<li>Eye-tracking procedure.
	<ol>
		<li>Invite the participants to sit comfortably in front of the display monitor of the remote eye tracker. Set the distance between the participants&#39; eyes and the monitor around 60 cm. Perform the standard calibration and validation procedures by asking the participants to fixate on a grid of five fixation targets in random succession.</li>
		<li>Present the participants with a spoken sentence while they are seeing a visual image, as done in the standard visual world paradigm10,44. Use the monocular eye-tracking option by tracking the eye that is on the same side as the illuminator of the eye tracker. Record the participant&#39;s eye movements using the eye tracker. 
		NOTE: The eye tracker used in the present study allows remote eye-tracking with a sampling rate of 500 Hz.</li>
	</ol>
	</li>
	<li>Testing and measuring.
	<ol>
		<li>Test the participants individually. Simply tell the participants to listen to the spoken sentences while they are viewing the pictures. Ask one experimenter to monitor the participant on the computer and one to stand behind the participant and gently rest her hands on the participant&#39;s shoulders to minimize the participant&#39;s sudden movements.</li>
		<li>Measure the participant&#39;s eye movements that arise as automatic responses to the linguistic input using the eye tracker. 
		NOTE: The task does not ask participants to make any conscious judgments about the spoken sentences to minimize their computational burden. The eye tracker automatically records the eye movements.</li>
		<li>Monitoring during the test: use the live viewer mode on the computer screen, exhibited by the eye tracker during the test, to observe the participant&#39;s looking behavior. Ask the experimenter who monitors data collection via the live viewer mode to signal to the experimenter who stands behind the participant to reorient the participant if his or her eye gaze wanders off the computer screen.</li>
	</ol>
	</li>
</ol>
5. Data Treatment and Analysis
<ol>
	<li>Code the participants&#39; fixations in two interest areas. Use Data Viewer to draw the two interest areas: BA-target event area and BEI-target event area (see Figure 1). Open Data Viewer. Select one of the interest area shape icons on the tool bar. Use the mouse to drag a box around the region you want to define as an interest area. Save the interest area in the Interest Area Set folder. Apply the interest area to other visual images. 
	NOTE: The depicted event in the upper panel of Figure 1 matches the BA-construction, hence the BA-target event, and the event depicted in the lower panel matches Figure 1b, hence the BEI-target event. The software used for data coding is Data Viewer, which comes with the eye tracker used in the study. Other data analysis software is also available.</li>
	<li>Analyze the eye gaze patterns using Data Viewer.
	<ol>
		<li>Open Data Viewer. Choose the sample report function from the menu to set the time windows for analysis (e.g., every 200 ms for the time window in the present study). Use the same function to time lock the fixation proportions in the interest areas to the onset of the marker for each trial. Export the raw data into an excel file using the export function from the menu.</li>
		<li>Use the excel functions to average the fixation proportions following the onset of the marker for each area. Use the excel functions to compute the fixation proportions in each time window of 200 ms over a period of 5200 ms (the mean length of the target sentences + 200 ms) from the onset of the marker for the two areas. Apply linear mixed-effects models to the eye movement data, detailed in Representative Results below. 
		NOTE: The use of 200 ms as a time window is based on the standard procedure for analyzing child eye gaze data in the literature12,13,18,19,45,46,47, and it is generally assumed that it takes about 200 ms to observe the effects of linguistic markers on eye movements48.</li>
	</ol>
	</li>
</ol>

The authors have nothing to disclose.

In the current study, we present an eye-tracking paradigm that can directly and effectively assess the sentence comprehension abilities of children with autism. We found that 5-year-old children with autism, like their age-matched TD peers, exhibited eye gaze patterns that reflect effective and rapid use of linguistic cues during real-time sentence comprehension.
The findings provide evidence that eye-tracking (in particular, the visual world paradigm) is a sensitive measure of real-time sente...

Sentence comprehension relies on the ability to rapidly integrate different types of linguistic and non-linguistic information1,2,3,4,5,6,7,8,9,10,11. Prior research has found that young typically developing (TD) children incrementally compute the meaning of a sentence using both linguistic and non-linguistic cues12,13,14,15,16,17,18,19. However, there is currently a paucity of research exploring how preschool children with autism understand a sentence using different types of cues. The mechanisms underlying their sentence comprehension remains largely unclear.
It is generally acknowledged that there is enormous variability in the language abilities of children with autism, especially in their expressive language; for example, some children with autism have relatively good structural language, some exhibit deficits in both lexical and grammatical domains, some demonstrate impaired grammar, and some never acquire functional spoken language20,21,22,23,24,25. In addition, prior research seems to suggest that their receptive language is relatively more impaired than their expressive language26,27,28,29. Most research that has assessed sentence comprehension abilities of children with autism have used offline tasks (e.g., standardized tests, caregiver reports), and the findings suggest that their sentence comprehension abilities might be particularly impaired30,31,32,33,34,35,36,37. However, it has been pointed out that poor comprehension abilities are more likely related to these children&#39;s overall lack of social responsiveness than to language processing deficits38,39. Note that these offline tasks used in previous research often require high response demands or interactions with the experimenters, which might pose particular difficulties for children with autism, because they often exhibit various challenging behaviors or symptoms. As a result, this may interact with the high task and communication demands and mask their comprehension abilities [for an overview of methods for assessing receptive language in children with autism, see Kasari et al. (2013)27 and Plesa-Skwerer et al. (2016)29]. Thus, experimental paradigms that can better control these confounding factors are required to further understand the nature of sentence-processing mechanisms in autism.
In the current study, we present an eye-tracking paradigm that can directly and effectively assess sentence comprehension abilities of children with autism. Compared to offline tasks, eye-tracking is a more sensitive testing paradigm to demonstrate children&#39;s comprehension abilities. It is sensitive to the time course of the comprehension process and requires no explicit motor or language responses from the participant, making it a promising method to study younger children and minimally verbal children with autism. In addition, we record eye movements as automatic responses to linguistic input instead of measuring eye movements that accompany conscious responses to linguistic input.

<table><tbody><tr><td>EyeLink 1000 plus eye tracker&amp;nbsp;</td><td>SR Research Ltd.</td><td></td><td>&amp;nbsp;The EyeLink 1000 plus allows remote eye tracking, without a head support. The eye tracker provides information about the participant&amp;rsquo;s point of gaze at a sampling rate of 500 Hz, and it has accuracy of 0.5 degrees of visual angle.&amp;nbsp;</td></tr></tbody></table>

using the visual world paradigm to study sentence comprehension in mandarin-speaking children with autism

Using the Visual World Paradigm to Study Sentence Comprehension in Mandarin-Speaking Children with Autism

Sentence comprehension relies on the ability to rapidly integrate different types of linguistic and non-linguistic information. However, there is currently a paucity of research exploring how preschool children with autism understand sentences using different types of cues. The mechanisms underlying sentence comprehension remains largely unclear. The present study presents a protocol to examine the sentence comprehension abilities of preschool children with autism. More specifically, a visual world paradigm of eye-tracking is used to explore the moment-to-moment sentence comprehension in the children. The paradigm has multiple advantages. First, it is sensitive to the time course of sentence comprehension and thus can provide rich information about how sentence comprehension unfolds over time. Second, it requires minimal task and communication demands, so it is ideal for testing children with autism. To further minimize the computational burden of children, the present study measures eye movements that arise as automatic responses to linguistic input rather than measuring eye movements that accompany conscious responses to spoken instructions.

The present study uses minimal pairs as in Examples 1a and 1b to investigate if and how fast children with autism can use event information encoded in two morphological markers during real-time sentence comprehension. It was predicted that if they are able to rapidly and effectively use event information in the two markers during real-time sentence comprehension, then they should look more at the BA-target event when hearing BA than when hearing BEI. Also, they should fixate more on the B...

We present a protocol to examine the use of morphological cues during real-time sentence comprehension by children with autism.

Sentence comprehension relies on the ability to rapidly integrate different types of linguistic and non-linguistic information. However, there is ...

using-visual-world-paradigm-to-study-sentence-comprehension-mandarin

Research

JoVE Journal

Behavior

Eye tracking has long been used to study gaze patterns in typically-developing individuals, but recent technological advancements have made its use with clinical populations, including autism, more feasible. While eye-tracking young children with autism can offer insight into early symptom manifestations, it involves methodological challenges. Suggestions for best practices are provided.

Eye Tracking Young Children with Autism

A reliable home-based way to assess the language comprehension of very young typically developing children, as well as those with autism, is described.  The method analyzes children's eye gaze while viewing side-by-side images but hearing an audio that matches only one image.  Stimuli are designed with young participants in mind.

Portable Intermodal Preferential Looking (IPL): Investigating Language Comprehension in Typically Developing Toddlers and Young Children with Autism

We describe how to implement a battery of behavioral tasks to examine the processing and integration of sensory stimuli in children with ASD. The goal is to characterize individual differences in temporal processing of simple auditory and visual stimuli and relate these to higher order perceptual skills like speech perception.

Testing Sensory and Multisensory Function in Children with Autism Spectrum Disorder

This paper describes a semantic priming ERP task using within-modality pairs of pictures and words to investigate semantic processing in individuals with autism spectrum disorder (ASD).

A Semantic Priming Event-related Potential (ERP) Task to Study Lexico-semantic and Visuo-semantic Processing in Autism Spectrum Disorder

The present article reviews an eye-tracking methodology for studies on language comprehension. To obtain reliable data, key steps of the protocol must be followed. Among these are the correct set-up of the eye tracker (e.g., ensuring good quality of the eye and head images) and accurate calibration.

Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects

The visual world paradigm monitors participants' eye movements in the visual workspace as they are listening to or speaking a spoken language. This paradigm can be used to investigate the online processing of a wide range of psycholinguistic questions, including semantically complex statements, such as disjunctive statements.

Using Eye Movements Recorded in the Visual World Paradigm to Explore the Online Processing of Spoken Language

This is a qualitative comparative case study analysis of eye-tracking data on the first moments of social video scenes as viewed by three participants: one with autism spectrum disorder, one with comorbid attention deficit-hyperactive disorder, and one neurotypical control.

Comparing Eye-tracking Data of Children with High-functioning ASD, Comorbid ADHD, and of a Control Watching Social Videos

Here, we present a protocol employing dual-modal interference tasks to examine online processing of spoken-Chinese relative clause sentences. Two exemplar experiments involving auditory processing with intra- and extrasentential interferences are described. The paradigm provides a methodology for addressing the nature of working memory and its effects on sentence processing.

Examining Online Syntactic Processing of Spoken Complex Sentences in Chinese Using Dual-Modal Interference Tasks

Here, we present a protocol that manipulates interlocutor visibility to examine its impact on gesture production in interpersonal communication. This protocol is flexible to tasks implemented, gestures examined, and communication modality. It is ideal for populations with communication challenges, such as second language learners and individuals with autism spectrum disorder.

Examining Gesture Production in the Presence of Communication Challenges

The NICU Network Neurobehavioral Scale (NNNS) was developed as an assessment for the at-risk infant. The purpose of this article is to describe the NNNS, provide video examples of the NNNS procedures and discuss the ways in which the exam has been used.

Assessment and Evaluation of the High Risk Neonate: The NICU Network Neurobehavioral Scale

Learn how to study sentence comprehension in Mandarin-speaking children with autism using the Visual World Paradigm with eye tracking. This method captures real-time processing of morphological cues with minimal cognitive demands.

Using the Visual World Paradigm to Study Sentence Comprehension in Mandarin-Speaking Children with Autism

Explore More Videos