Using Eye Movements Recorded in the Visual World Paradigm to Explore the Online Processing of Spoken Language

Summary

The visual world paradigm monitors participants' eye movements in the visual workspace as they are listening to or speaking a spoken language. This paradigm can be used to investigate the online processing of a wide range of psycholinguistic questions, including semantically complex statements, such as disjunctive statements.

Abstract

In a typical eye tracking study using the visual world paradigm, participants' eye movements to objects or pictures in the visual workspace are recorded via an eye tracker as the participant produces or comprehends a spoken language describing the concurrent visual world. This paradigm has high versatility, as it can be used in a wide range of populations, including those who cannot read and/or who cannot overtly give their behavioral responses, such as preliterate children, elderly adults, and patients. More importantly, the paradigm is extremely sensitive to fine grained manipulations of the speech signal, and it can be used to study the online processing of most topics in language comprehension at multiple levels, such as the fine grained acoustic phonetic features, the properties of words, and the linguistic structures. The protocol described in this article illustrates how a typical visual world eye tracking study is conducted, with an example showing how the online processing of some semantically complex statements can be explored with the visual world paradigm.

Introduction

Spoken language is a fast, ongoing information flow, which disappears right away. It is a challenge to experimentally study this temporal, rapidly change speech signal. Eye movements recorded in the visual world paradigm can be used to overcome this challenge. In a typical eye tracking study using the visual world paradigm, participants' eye movements to pictures in a display or to real objects in a visual workspace are monitored as they listen to, or produce, spoken language depicting the contents of the visual world^1,2,3,4. The basic l....

Protocol

All subjects must give informed written consent before the administration of the experimental protocols. All procedures, consent forms, and the experimental protocol were approved by the Research Ethics Committee of the Beijing Language and Culture University.

NOTE: A comprehension study using the visual world paradigm normally consists of the following steps: Introduce the theoretical problems to be explored; Form an experimental design; Prepare the visual and auditory stimuli; Frame the theoretical problem with regard to the experimental design; Select an eye-tracker to track participants' eye movements; Select a software and build a ....

Results

Participants' behavioral responses are summarized in Figure 4. As we described earlier, the correct response to a conjunctive statement (S1 and S2) is the big open box, such as Box A in Figure 1. The correct response to a but-statement (S1 but not S2) is the small open box containing the first mentioned animal, such as Box D in Figure 1. Critically, which box is chosen to t.......

Discussion

To conduct a visual world study, there are several critical steps to follow. First, researchers intend to deduce the interpretation of the auditorily presented language via participants' eye movements in the visual world. Henceforth, in designing the layout of the visual stimuli, the properties of eye movements in a natural task that potentially affect participants' eye movements should be controlled. The effect of the spoken language on participants' eye movements can then be recognized. Second, acoustic cue.......

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
Pixelmator	Pixelmator Team	http://www.pixelmator.com/pro/	image editing app
Praat	Open Sourse	http://www.fon.hum.uva.nl/praat/	Sound analyses and editting software
Eyelink 1000plus	SR-Research, Inc	https://www.sr-research.com/products/eyelink-1000-plus/	remote infrared eye tracker
Experimental Builder	SR-Research, Inc	https://www.sr-research.com/experiment-builder/	eye tracker software
Data Viewer	SR-Research, Inc	https://www.sr-research.com/data-viewer/	eye tracker software
R	Open Sourse	https://www.r-project.org	free software environment for statistical computing and graphics