Name: Gaze in Action: Head-mounted Eye Tracking of Childrens Dynamic Visual Attention During Naturalistic Behavior (Video) | JoVE Journal
Uploaded: 2018-11-14T13:15:00.000+00:00
Duration: 7 min 9 s
Description: Learn head-mounted eye tracking for child development research using infants and toddlers. This method captures dynamic visual attention during natural behaviors like play and reading.

ai_agent_IP

ai_agent

maintenance

instantTrialDurationMinutes

isBiopharma

enableNewMyJoveSectionDesign

educationFeatures

seoMetaApi

ai_copilot_institution_blacklist

ai_chat_institution_blacklist

freeAccessWindow

ssoRoleCapture

scheduledMaintenanceBanner

joveLabsFeatures

disable_autoplay_for_bots

extended_trial_duration

5 minutes

playerTrialDesktop

This research was funded by the National Institutes of Health grants R01HD074601 (C.Y.), T32HD007475-22 (J.I.B., D.H.A.), and F32HD093280 (L.K.S.); National Science Foundation grant BCS1523982 (L.B.S., C.Y.); and by Indiana University through the Emerging Area Research Initiative - Learning: Brains, Machines, and Children (L.B.S.). The authors thank the child and parent volunteers who participated in this research and who agreed to be used in the figures and filming of this protocol. We also appreciate the members of the Computational Cognition and Learning Laboratory, especially Sven Bambach, Anting Chen, Steven Elmlinger, Seth Foster, Grace Lisandrelli, and Charlene Tay, for their assistance in developing and honing this protocol.

<ol><li>Tatler, B. W., Hayhoe, M. M., Land, M. F., Ballard, D. H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Eye+guidance+in+natural+vision%3A+Reinterpreting+salience%5BTitle%5D)+AND+%22Journal+of+Vision%22%5BJournal%5D)">Eye guidance in natural vision: Reinterpreting salience.</a> Journal of Vision. 11 (5), 1-23 (2011).</li>
<li>Hayhoe, M. <a target="_blank" href="https://www.tandfonline.com/doi/abs/10.1080/135062800394676">Vision using routines: A functional account of vision.</a> Visual Cognition. 7 (1-3), 43-64 (2000).</li>
<li>Land, M., Mennie, N., Rusted, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((The+Roles+of+Vision+and+Eye+Movements+in+the+Control+of+Activities+of+Daily+Living%5BTitle%5D)+AND+%22Perception%22%5BJournal%5D)">The Roles of Vision and Eye Movements in the Control of Activities of Daily Living.</a> Perception. 28 (11), 1311-1328 (1999).</li>
<li>Franchak, J. M., Kretch, K. S., Adolph, K. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((See+and+be+seen%3A+Infant-caregiver+social+looking+during+locomotor+free+play%5BTitle%5D)+AND+%22Developmental+Science%22%5BJournal%5D)">See and be seen: Infant-caregiver social looking during locomotor free play.</a> Developmental Science. 21 (4), 12626(2018).</li>
<li>Franchak, J. M., Kretch, K. S., Soska, K. C., Adolph, K. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Head-mounted+eye+tracking%3A+a+new+method+to+describe+infant+looking%5BTitle%5D)+AND+%22Child+Development%22%5BJournal%5D)">Head-mounted eye tracking: a new method to describe infant looking.</a> Child Development. 82 (6), 1738-1750 (2011).</li>
<li>Kretch, K. S., Adolph, K. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((The+organization+of+exploratory+behaviors+in+infant+locomotor+planning%5BTitle%5D)+AND+%22Developmental+Science%22%5BJournal%5D)">The organization of exploratory behaviors in infant locomotor planning.</a> Developmental Science. 20 (4), 12421(2017).</li>
<li>Yu, C., Smith, L. B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Hand-Eye+Coordination+Predicts+Joint+Attention%5BTitle%5D)+AND+%22Child+Development%22%5BJournal%5D)">Hand-Eye Coordination Predicts Joint Attention.</a> Child Development. 88 (6), 2060-2078 (2017).</li>
<li>Yu, C., Smith, L. B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Joint+Attention+without+Gaze+Following%3A+Human+Infants+and+Their+Parents+Coordinate+Visual+Attention+to+Objects+through+Eye-Hand+Coordination%5BTitle%5D)+AND+%22PLoS+One%22%5BJournal%5D)">Joint Attention without Gaze Following: Human Infants and Their Parents Coordinate Visual Attention to Objects through Eye-Hand Coordination.</a> PLoS One. 8 (11), 79659(2013).</li>
<li>Yu, C., Smith, L. B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Multiple+Sensory-Motor+Pathways+Lead+to+Coordinated+Visual+Attention%5BTitle%5D)+AND+%22Cognitive+Science%22%5BJournal%5D)">Multiple Sensory-Motor Pathways Lead to Coordinated Visual Attention.</a> Cognitive Science. 41, 5-31 (2016).</li>
<li>Yu, C., Smith, L. B. <a target="_blank" href="https://www.ncbi.nlm.nih.gov/pubmed/27133869">The Social Origins of Sustained Attention in One-Year-Old Human Infants.</a> Current Biology. 26 (9), 1-6 (2016).</li>
<li>Kretch, K. S., Franchak, J. M., Adolph, K. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Crawling+and+walking+infants+see+the+world+differently%5BTitle%5D)+AND+%22Child+Development%22%5BJournal%5D)">Crawling and walking infants see the world differently.</a> Child Development. 85 (4), 1503-1518 (2014).</li>
<li>Yu, C., Suanda, S. H., Smith, L. B. <a target="_blank" href="http://www.iub.edu/~cogdev/labwork/sa_mcdi_v12.pdf">Infant sustained attention but not joint attention to objects at 9 months predicts vocabulary at 12 and 15 months.</a> Developmental Science. , (2018).</li>
<li>Hennessey, C., Lawrence, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Noncontact+binocular+eye-gaze+tracking+for+point-of-gaze+estimation+in+three+dimensions%5BTitle%5D)+AND+%22IEEE+Transactions+on+Biomedical+Engineering%22%5BJournal%5D)">Noncontact binocular eye-gaze tracking for point-of-gaze estimation in three dimensions.</a> IEEE Transactions on Biomedical Engineering. 56 (3), 790-799 (2009).</li>
<li>Elmadjian, C., Shukla, P., Tula, A. D., Morimoto, C. H. 3D gaze estimation in the scene volume with a head-mounted eye tracker. <a target="_blank" href="https://dl.acm.org/citation.cfm?id=3206351">Proceedings of the Workshop on Communication by Gaze Interaction</a>. , Association for Computing Machinery. New York. 3(2018).</li>
<li>Castellanos, I., Pisoni, D. B., Yu, C., Chen, C., Houston, D. M. Embodied cognition in prelingually deaf children with cochlear implants: Preliminary findings. <a target="_blank" href="http://scholar.google.com/scholar?hl=en&safe=off&q=author%3aCastellanos%2C+I+%22Educating+Deaf+Learners%3A+New+Perspectives%22">Educating Deaf Learners: New Perspectives</a>. Knoors, H., Marschark, M. , Oxford University Press. New York. (2018).</li>
<li>Kennedy, D. P., Lisandrelli, G., Shaffer, R., Pedapati, E., Erickson, C. A., Yu, C. Face Looking, Eye Contact, and Joint Attention during Naturalistic Toy Play: A Dual Head-Mounted Eye Tracking Study in Young Children with ASD. Poster at the International Society for Autism Research Annual Meeting. , (2018).</li>
<li>Yurkovic, J. R., Lisandrelli, G., Shaffer, R., Pedapati, E., Erickson, C. A., Yu, C., Kennedy, D. P. Using Dual Head-Mounted Eye Tracking to Index Social Responsiveness in Naturalistic Parent-Child Interaction. Talk at the International Congress for Infant Studies Biennial Congress. , July (2018).</li>
<li>Holmqvist, K., Nyström, M., Andersson, R., Dewhurst, R., Jarodzka, H., Van de Weijer, J. <a target="_blank" href="http://scholar.google.com/scholar?hl=en&safe=off&q=author%3aHolmqvist%2C+K+%22Eye+tracking%3A+A+comprehensive+guide+to+methods+and+measures%22">Eye tracking: A comprehensive guide to methods and measures</a>. , Oxford University Press. (2011).</li>
<li>Saez de Urabain, I. R., Johnson, M. H., Smith, T. J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((GraFIX%3A+a+semiautomatic+approach+for+parsing+low-+and+high-quality+eye-tracking+data%5BTitle%5D)+AND+%22Behavior+Research+Methods%22%5BJournal%5D)">GraFIX: a semiautomatic approach for parsing low- and high-quality eye-tracking data.</a> Behavior Research Methods. 47 (1), 53-72 (2015).</li>
<li>Franchak, J. M. Using head-mounted eye tracking to study development. <a target="_blank" href="https://www.cambridge.org/core/books/cambridge-encyclopedia-of-child-development/C2C37B38988A7B4BE0DD3BE7C701DE69">The Cambridge Encyclopedia of Child Development 2nd ed</a>. Hopkins, B., Geangu, E., Linkenauger, S. , Cambridge University Press. Cambridge, UK. 113-116 (2017).</li>
<li>Yonas, A., Arterberry, M. E., Granrud, C. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Four-month-old+infants%27+sensitivity+to+binocular+and+kinetic+information+for+three-dimensional-object+shape%5BTitle%5D)+AND+%22Child+Development%22%5BJournal%5D)">Four-month-old infants' sensitivity to binocular and kinetic information for three-dimensional-object shape.</a> Child Development. 58 (4), 910-917 (1987).</li>
<li>Smith, T. J., Saez de Urabain, I. R. Eye tracking. <a target="_blank" href="http://scholar.google.com/scholar?hl=en&safe=off&q=author%3aSmith%2C+TJ+%22The+Cambridge+Encyclopedia+of+Child+Development%22">The Cambridge Encyclopedia of Child Development</a>. Hopkins, B., Geangu, E., Linkenauger, S. , Cambridge University Press. Cambridge, UK. 97-101 (2017).</li>
</ol>

The authors declare that they have no competing or conflicting interests.

This protocol provides guiding principles and practical recommendations for implementing head-mounted eye tracking with infants and young children. This protocol was based on the study of natural toddler behaviors in the context of parent-toddler free play with toys in a laboratory setting. In-house eye-tracking equipment and software were used for calibration and data coding. Nevertheless, this protocol is intended to be generally applicable to researchers using a variety of head-mounted eye-tracking systems to study a ...

The last several decades have seen growing interest in studying the development of infant and toddler visual attention. This interest has stemmed in large part from the use of looking time measurements as a primary means to assess other cognitive functions in infancy and has evolved into the study of infant visual attention in its own right. Contemporary investigations of infant and toddler visual attention primarily measure eye movements during screen-based eye-tracking tasks. Infants sit in a chair or parent&#39;s lap in front of a screen while their eye movements are monitored during the presentation of static images or events. Such tasks, however, fail to capture the dynamic nature of natural visual attention and the means by which children&#39;s natural visual environments are generated - active exploration.
Infants and toddlers are active creatures, moving their hands, heads, eyes, and bodies to explore the objects, people, and spaces around them. Each new development in body morphology, motor skill, and behavior - crawling, walking, picking up objects, engaging with social partners - is accompanied by concomitant changes in the early visual environment. Because what infants do determines what they see, and what they see serves for what they do in visually guided action, studying the natural development of visual attention is best carried out in the context of natural behavior1.
Head-mounted eye trackers (ETs) have been invented and used for adults for decades2,3. Only recently have technological advances made head-mounted eye-tracking technology suitable for infants and toddlers. Participants are outfitted with two lightweight cameras on the head, a scene camera facing outward that captures the first person perspective of the participant and an eye camera facing inward that captures the eye image. A calibration procedure provides training data to an algorithm that maps as accurately as possible the changing positions of the pupil and corneal reflection (CR) in the eye image to the corresponding pixels in the scene image that were being visually attended. The goal of this method is to capture both the natural visual environments of infants and infants&#39; active visual exploration of those environments as infants move freely. Such data can help to answer questions not only about visual attention, but also about a broad range of perceptual, cognitive, and social developments4,5,6,7,8. The use of these techniques has transformed understandings of joint attention7,8,9, sustained attention10, changing visual experiences with age and motor development4,6,11, and the role of visual experiences in word learning12. The present paper provides guiding principles and practical recommendations for carrying out head-mounted eye-tracking experiments with infants and toddlers and illustrates the types of data that can be generated from head-mounted eye tracking in one natural context for toddlers: free-flowing toy play with a parent.

<table><tbody><tr><td>Head-mounted eye tracker</td><td>Pupil Labs</td><td>World Camera and Eye Camera</td><td></td></tr></tbody></table>

gaze in action: head-mounted eye tracking of children's dynamic visual attention during naturalistic behavior

Gaze in Action: Head-mounted Eye Tracking of Children's Dynamic Visual Attention During Naturalistic Behavior

Young children's visual environments are dynamic, changing moment-by-moment as children physically and visually explore spaces and objects and interact with people around them. Head-mounted eye tracking offers a unique opportunity to capture children's dynamic egocentric views and how they allocate visual attention within those views. This protocol provides guiding principles and practical recommendations for researchers using head-mounted eye trackers in both laboratory and more naturalistic settings. Head-mounted eye tracking complements other experimental methods by enhancing opportunities for data collection in more ecologically valid contexts through increased portability and freedom of head and body movements compared to screen-based eye tracking. This protocol can also be integrated with other technologies, such as motion tracking and heart-rate monitoring, to provide a high-density multimodal dataset for examining natural behavior, learning, and development than previously possible. This paper illustrates the types of data generated from head-mounted eye tracking in a study designed to investigate visual attention in one natural context for toddlers: free-flowing toy play with a parent. Successful use of this protocol will allow researchers to collect data that can be used to answer questions not only about visual attention, but also about a broad range of other perceptual, cognitive, and social skills and their development.

The method discussed here was applied to a free-flowing toy play context between toddlers and their parents. The study was designed to investigate natural visual attention in a cluttered environment. Dyads were instructed to play freely with a set of 24 toys for six minutes. Toddlers' visual attention was measured by coding the onset and offset of looks to specific regions of interest (ROIs) -- each of the 24 toys and the parent's face -- and by analyzing the duration and proportion of lo...

Young children do not passively observe the world, but rather actively explore and engage with their environment. This protocol provides guiding principles and practical recommendations for using head-mounted eye trackers to record infants' and toddlers' dynamic visual environments and visual attention in the context of natural behavior.

Young children's visual environments are dynamic, changing moment-by-moment as children physically and visually explore spaces and objects and interact ...

gaze-action-head-mounted-eye-tracking-children-s-dynamic-visual

Research

JoVE Journal

Behavior

Procedures for recording high-density EEG and gaze data during computer game-based cognitive tasks are described.  Using a video game to present cognitive tasks enhances ecological validity without sacrificing experimental control.

Combining Computer Game-Based Behavioural Experiments With High-Density EEG and Infrared Gaze Tracking

Eye tracking has long been used to study gaze patterns in typically-developing individuals, but recent technological advancements have made its use with clinical populations, including autism, more feasible. While eye-tracking young children with autism can offer insight into early symptom manifestations, it involves methodological challenges. Suggestions for best practices are provided.

Eye Tracking Young Children with Autism

A method is described to quantify the quality of visual information processing based on reflexive eye movements in response to specific visual modalities. Reaction times and fixation output parameters are used to characterize visual performance in children with and without visual impairments from 6 months of age.

A Method to Quantify Visual Information Processing in Children Using Eye Tracking

The present article reviews an eye-tracking methodology for studies on language comprehension. To obtain reliable data, key steps of the protocol must be followed. Among these are the correct set-up of the eye tracker (e.g., ensuring good quality of the eye and head images) and accurate calibration.

Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects

This is a qualitative comparative case study analysis of eye-tracking data on the first moments of social video scenes as viewed by three participants: one with autism spectrum disorder, one with comorbid attention deficit-hyperactive disorder, and one neurotypical control.

Comparing Eye-tracking Data of Children with High-functioning ASD, Comorbid ADHD, and of a Control Watching Social Videos

Eye tracking studies using a preferential looking paradigm can be used to study infants' emerging understanding of, and attention to, their external visual world.

Exploring Infant Sensitivity to Visual Language using Eye Tracking and the Preferential Looking Paradigm

Using multimodal sensors is a promising way to understand the role of social interactions in educational settings. This paper describes a methodology for capturing joint visual attention from colocated dyads using mobile eye-trackers.

A Methodology for Capturing Joint Visual Attention Using Mobile Eye-Trackers

We present a protocol for a behavioral analysis of adults (ages 18 to 70-year-old) engaged in learning processes, undertaking tasks designed for Self-Regulated Learning (SRL). The participants, university teachers and students, and adults from the University of Experience, were monitored with eye-tracking devices and the data were analyzed with data-mining techniques.

Eye-tracking Technology and Data-mining Techniques used for a Behavioral Analysis of Adults engaged in Learning Processes

Eye tracking is a non-invasive method to probe information processing. This article describes how eye tracking can be used to study gaze behavior during a flight simulation emergency task in low-time pilots (i.e., &lt;350 flight hours).

Eye Tracking During A Complex Aviation Task For Insights Into Information Processing

Single-pulse transcranial magnetic stimulation over the primary motor cortex, neuronavigation, and registration of electromyographic activity of hand muscles were used in this study to explore corticospinal excitability while participants were observing action sequences.

Corticospinal Excitability Modulation During Action Observation

Learn head-mounted eye tracking for child development research using infants and toddlers. This method captures dynamic visual attention during natural behaviors like play and reading.

Gaze in Action: Head-mounted Eye Tracking of Children's Dynamic Visual Attention During Naturalistic Behavior

Explore More Videos