fmri-validation-of-fnirs-measurements-during-a-naturalistic-task

fMRI Validation of fNIRS Measurements During a Naturalistic Task

We present a method to compare brain activity recorded with near-infrared spectroscopy (fNIRS) in a dance video game task to that recorded in a reduced ...

Meiji University

Visual feedback of hand movement is crucial to accurate reaching. Although previous studies have extensively examined spatial alteration of visual feedback (e.g., prism adaptation), temporal delay of visual feedback has been less explored. In the present study, we investigated the effect of delayed visual feedback of the moving hand in a reaching task. The prefrontal cortical activity was measured by near-infrared spectroscopy (NIRS). Twelve subjects performed reaching tasks under two conditions where visual feedback of their own hand was delayed by 200 ms (delay condition) or 0 ms (normal condition). Introducing the visual feedback delay significantly disrupted the reaching performance, although the subjects gradually adapted to the delay during the experiment. There was a clear tendency to overreach the target in the delay condition, even after the reaching movement had been practiced sufficiently in the normal condition. We observed marked oxy- and total-Hb decreases in the dorsal prefrontal area in the delay conditions. The decrease began shortly after task onset and diminished during the rest period, indicating that the decrease was task-induced. Furthermore, the oxy- and total-Hb decreases were significantly correlated with task performance--the degree of decrease was larger as the subject made more errors. We suggest that the decreases in oxy- and total-Hb at the dorsal prefrontal area are related with the visuomotor recalibration process.

Decrease in prefrontal hemoglobin oxygenation during reaching tasks with delayed visual feedback: a near-infrared spectroscopy study.

One hypothesis on how we recognize an image of, for example, an arm as our own is through the co-occurrence of multiple sensory feedbacks, especially visual and proprioceptive feedbacks, in this process. It has been suggested that the parietal lobe is the region where proprioceptive and visual information of one's own body is integrated. This study investigated parietal cortical activity during a visual-proprioceptive synchrony judgment task in which visual feedback of the subjects' own passively moving hand was delayed. The subjects were required to judge whether or not there was a delay between the proprioceptive and visual feedbacks. Parietal cortical activity, which was measured using a 48-channel near-infrared spectroscopy (NIRS) apparatus, appeared to be modulated by the length of the delay between the visual and proprioceptive feedbacks. The bilateral superior/middle parietal areas were involved in experiencing the synchrony between the visual and proprioceptive feedbacks, whereas the right inferior parietal areas were strongly activated when discrepancy between the two feedbacks was detected. We postulate that the superior portion of the parietal lobe is essential for maintaining one's own body image, while the right inferior portion is involved in detecting movements of others.

The parietal role in the sense of self-ownership with temporal discrepancy between visual and proprioceptive feedbacks.

Whether human infants perceive televised stimuli in the same way to live stimuli largely remains unknown. Action observation, which has been extensively confirmed to elicit activation of internal motor representation, provides a promising framework for investigating this issue. This 'mirror-matching' property has been found in the monkey premotor cortex as well as the premotor and primary motor cortices in human adults. Although larger activation in observing a live action compared to a televised action in adult subjects has been reported, it is unknown whether the same neural response is obtained from human infants. To address this issue, we first measured the activity of motor areas in adult subjects while viewing either a live or televised action of other people by using near-infrared spectroscopy. The motor areas that were activated when the subject themselves performed an action were also activated during action observation in the live setting, while this was not evident in the TV setting. We then conducted qualitatively the same experiment with 6- to 7-month-old infants. The infant's motor areas were significantly activated when observing a live person performing an action. Although we also found activation in the same area during action observation in the TV setting, the difference in activity between action observation and object-motion observation was significant only in the live setting. Our results are the first to demonstrate activation in motor areas during action observation in human infants. We suggest that human brain responds differently to the real world and the virtual world.

Infant's brain responses to live and televised action.

Watching competitive games is a pervasive entertainment in humans and likely to be facilitated by our social cognitive abilities. We investigated the brain activity during observation of a competitive game by using near-infrared spectroscopy. The result showed that activity in the motor area was significantly greater when the individual's favored player won, compared with when the same player lost. The motor area was also activated when the individuals themselves performed the same action, showing the mirror neuron system property. Our results first showed that the motor area activity during observation of competitive games was modulated by the outcome of the game. We suggest that the observer's brain is more likely to share the winner's internal motor representation than that of the loser.

Modulation of the motor area activity during observation of a competitive game.

Rubber hand illusion (RHI) is a subject's illusion of the self-ownership of a rubber hand that was touched synchronously with their own hand. Although previous studies have confirmed that this illusion disappears when the rubber hand was touched asynchronously with the subject's hand, the minimum temporal discrepancy of these two events for attenuation of RHI has not been examined.

Rubber hand illusion under delayed visual feedback.

Temporal congruency between the efference copy of a motor command and the reafferent sensory feedback is crucial for identifying self-generated body movements. We investigated how temporal discrepancy between the efference copy and visual feedback affects the self-body movement recognition process. Subjects experienced active and passive hand movements under conditions of delayed visual feedback (118-352 ms) and judged whether observed hand movements were delayed with respect to the felt movement. The results showed that the discrimination threshold of visual feedback delay (50% detection rate) was not significantly different between active and passive movements. In contrast, the judgment probability curve was significantly steeper for active than passive movements. This indicates that the efference copy enhances the contrast between synchronous and asynchronous movements but does not narrow the time window in this process. We discuss processing of active and passive movements in relation to the senses of self-agency and self-ownership.

Detection of visual feedback delay in active and passive self-body movements.

Observing competitive games such as sports is a pervasive entertainment among humans. The inclination to watch others play may be based on our social-cognitive ability to understand the internal states of others. The mirror neuron system, which is activated when a subject observes the actions of others, as well as when they perform the same action themselves, seems to play a crucial role in this process. Our previous study showed that activity of the mirror neuron system was modulated by the outcome of the subject's favored player during observation of a simple competitive game (rock-paper-scissors). However, whether the mirror neuron system responds similarly in a more complex and naturalistic sports game has not yet been fully investigated.

Modulation of motor area activity by the outcome for a player during observation of a baseball game.

It is well established that several motor areas, called the mirror-neuron system (MNS), are activated when an individual observes other's actions. However, whether the MNS responds similarly to robotic actions compared with human actions is still controversial. The present study investigated whether and how the motor area activity is influenced by appearance (human vs. robot) and/or kinematics (human vs. robot) of the observed action using near-infrared spectroscopy. The results showed that there was a strong interaction between these factors, revealing strong deactivations in sensorimotor areas when the subject saw a human agent performing robotic actions, which was significantly different from responses when observing the human agent acting in a human way and the robot agent performing robotic actions. These results indicate that MNS activity is sensitive to congruency between the appearance and kinematics of the agent, especially when the agent has a human appearance. We discuss the experience-dependent characteristics of MNS sensitivity to observed actions.

Deactivation in the sensorimotor area during observation of a human agent performing robotic actions.

In a previous study, we demonstrated that the mirror neuron system is involved in watching competitive games performed by multiple players [Shimada, S., & Abe, R. (2009). Modulation of the motor area activity during observation of a competitive game. NeuroReport, 20, 979-983]. In the present study, we investigated whether mirror neuron system activity was modulated by the outcome of a competitive game as well as by the spatial relationship between the player and the observer. We measured the activity of motor areas using near-infrared spectroscopy when a subject watched a two-player competitive game, rock-paper-scissors (RPS). Some parts of the motor area that was activated while performing the RPS action were also activated while observing the RPS game. Interestingly, the magnitude of this activity (during observation) was modulated by the outcome and the view of the particular player whom the subject was supporting. In the dorsal anterior region of the motor area measured, the activity was significantly greater when the supported player won rather than when he lost, regardless of the view. It was also greater when the player was presented with an incongruent view than with a congruent view, regardless of the outcome. In contrast, the dorsal posterior region of the motor area was activated only when the supported player won and was observed with a congruent view. We suggest that mirror neuron system activity during observation of a competitive game reflects the subject's internal assimilation of the actions of the observed player.

Outcome and view of the player modulate motor area activity during observation of a competitive game.

The mirror neuron system (MNS) is activated when observing the actions of others. However, it remains unclear whether the MNS responds more strongly to natural bodily actions in the observer's motor repertoire than to unnatural actions. We investigated whether MNS activity is modulated by the unnaturalness of an observed action by inserting short pauses in the middle of the action (0, 2, and 6 pauses; no-pause, pause-1, and pause-2 conditions, respectively). The results indicated that the number of pauses significantly affected motor area activity. Subsequent analyses revealed significant differences between the pause-1 and pause-2 conditions (P<0.01), as well as the no-pause and pause-2 conditions (P<0.05). There was significant activation in the pause-1 condition (P<0.001), while significant deactivation was observed in the pause-2 condition (P<0.05). These results indicate that MNS activity is modulated by the kinematic characteristics of the observed action. We suggest the possibility that a slightly deviated action may enhance the MNS activity during action observation, while a highly unnatural action would lead to a strong attenuation (deactivation) of activity in the MNS.

Modulation of motor area activity during observation of unnatural body movements.

We utilized the high temporal resolution of functional near-infrared spectroscopy to explore how sensory input (visual and rhythmic auditory cues) are processed in the cortical areas of multimodal integration to achieve coordinated motor output during unrestricted dance simulation gameplay. Using an open source clone of the dance simulation video game, Dance Dance Revolution, two cortical regions of interest were selected for study, the middle temporal gyrus (MTG) and the frontopolar cortex (FPC). We hypothesized that activity in the FPC would indicate top-down regulatory mechanisms of motor behavior; while that in the MTG would be sustained due to bottom-up integration of visual and auditory cues throughout the task. We also hypothesized that a correlation would exist between behavioral performance and the temporal patterns of the hemodynamic responses in these regions of interest. Results indicated that greater temporal accuracy of dance steps positively correlated with persistent activation of the MTG and with cumulative suppression of the FPC. When auditory cues were eliminated from the simulation, modifications in cortical responses were found depending on the gameplay performance. In the MTG, high-performance players showed an increase but low-performance players displayed a decrease in cumulative amount of the oxygenated hemoglobin response in the no music condition compared to that in the music condition. In the FPC, high-performance players showed relatively small variance in the activity regardless of the presence of auditory cues, while low-performance players showed larger differences in the activity between the no music and music conditions. These results suggest that the MTG plays an important role in the successful integration of visual and rhythmic cues and the FPC may work as top-down control to compensate for insufficient integrative ability of visual and rhythmic cues in the MTG. The relative relationships between these cortical areas indicated high- to low-performance levels when performing cued motor tasks. We propose that changes in these relationships can be monitored to gauge performance increases in motor learning and rehabilitation programs.

Frontotemporal oxyhemoglobin dynamics predict performance accuracy of dance simulation gameplay: temporal characteristics of top-down and bottom-up cortical activities.

The extrastriate body area (EBA) is a brain region that responds selectively to visual stimuli of human bodies or body parts. Previous studies using functional magnetic resonance imaging (fMRI) have shown that EBA occupies a relatively small region and varied across subjects in its anatomical location. This study investigated whether EBA activity can be detected by functional near-infrared spectroscopy (fNIRS) that imposes few physical constraints on the subjects and has higher temporal but lower spatial resolutions compared to fMRI. For this purpose the subject's brain activity in the occipitotemporal area during observation of images of body parts and objects was measured using fNIRS. The NIRS optode positions were recorded using a 3D digitizer and mapped onto a probabilistic anatomical model. We found that the activity in the occipitotemporal region during observation of body stimuli was significantly greater than that of object stimuli in 11 out of 16 subjects. The group analyses also showed that channels located near the position where the previous studies reported EBA activation were more activated during observation of the body stimuli compared to the object stimuli. The spatial variance of those channels among subjects was relatively small. These results suggest that EBA activity and its anatomical location can be sufficiently measured by using fNIRS and a 3D digitizer.

Localization of extrastriate body area using functional near-infrared spectroscopy and 3D digitizer.

Temporal contingency between motor commands and corresponding auditory feedback is crucial for perception of self-generated sound as well as external auditory events. The present study examined whether delay detection of self-generated sound was affected by the range of delayed auditory feedback used during the experiment. Participants pressed a button with their right index finger and judged whether auditory feedback was delayed compared with the sensation of finger movement. The range of auditory feedback delay was varied across conditions. To calculate the delay detection threshold (DDT), that is, the point at which the delay detection rate was 50%, we fitted a logistic function to the delay-detection probability curve. The DDT was significantly different across conditions (Tukey-Kramer's honestly significant difference test, P<0.01). Specifically, the DDT became longer as the mean delay of the range increased. However, this shift was not observed for the delay range with a minimum delay greater than 250 ms. We propose that the subjective simultaneity of auditory feedback and self-body movement is, to some extent, automatically recalibrated toward the mean delay of the delay range used in the experiment.

Recalibration of subjective simultaneity between self-generated movement and delayed auditory feedback.

The rubber-hand illusion (RHI) is that the subject feels the visually presented tactile stimulation of an artificial (rubber) hand as their own tactile sensation and is caused by stimulating the rubber and real hands synchronously. Our previous study showed that the RHI was greatly reduced as the visual feedback delay of the tactile stimulation of the hand became longer. In the present study, we investigate the relationship between the attenuation of the RHI and the detection of the delay in two experiments: (1) an RHI experiment and (2) a visuotactile asynchrony detection experiment, in which the subjects underwent tactile stimulation of their hand and judged whether visual feedback was consistent with the touch sensation. In line with our previous study, the RHI was significantly reduced as the delay lengthened. Interestingly, proprioceptive drift declined linearly as the delay increased, while the delay detection rate was better fitted by a non-linear (logistic) function. The illusion score showed the intermittent pattern. We suggest that proprioceptive drift is relevant to the processing of the body schema, whereas the delay detection and the subjective feeling of the RHI are more related to the body image processing.

Relationship between sensitivity to visuotactile temporal discrepancy and the rubber hand illusion.

The motor cortical area is often activated to auditory stimuli in the human brain. In this study, we examined whether the motor area shows differential activation for action-related and nonaction-related sounds and whether it is susceptible to the quality of the sounds. A three-dimensional sound field recording and reproduction system based on the boundary surface control principle (BoSC system) was used for this purpose. We measured brain activity during hearing action-related or nonaction-related sounds with electroencephalography using mu rhythm suppression (mu-suppression) as an index of motor cortical activation. The results showed that mu-suppression was observed when the participant heard action-related sounds, but it was not evident when hearing nonaction-related sounds. Moreover, this suppression was significantly larger in the 3D sound field (62-ch loudspeaker condition), which generates a more realistic sound field, than in the 1-ch loudspeaker condition. Our results indicate that the motor area was indeed activated for action-related sounds and that its activation was enhanced with a 3D realistic sound field. We discuss our findings in relation to the mirror neuron system and the possibility of using its activity as an objective measure that reflects the subjective sense of reality in various virtual reality settings when interacting with others.

Article	Year
Decrease in prefrontal hemoglobin oxygenation during reaching tasks with delayed visual feedback: a near-infrared spectroscopy study. Brain research. Cognitive brain research\| PubMed ID: 15268925	2004
The parietal role in the sense of self-ownership with temporal discrepancy between visual and proprioceptive feedbacks. NeuroImage\| PubMed ID: 15670700	2005
Infant's brain responses to live and televised action. NeuroImage\| PubMed ID: 16679032	2006
Modulation of the motor area activity during observation of a competitive game. Neuroreport\| PubMed ID: 19474766	2009
Rubber hand illusion under delayed visual feedback. PloS one\| PubMed ID: 19587780	2009
Detection of visual feedback delay in active and passive self-body movements. Experimental brain research\| PubMed ID: 19830411	2010
Modulation of motor area activity by the outcome for a player during observation of a baseball game. PloS one\| PubMed ID: 19946372	2009
Deactivation in the sensorimotor area during observation of a human agent performing robotic actions. Brain and cognition\| PubMed ID: 20031292	2010
Outcome and view of the player modulate motor area activity during observation of a competitive game. Neuropsychologia\| PubMed ID: 20230845	2010
Modulation of motor area activity during observation of unnatural body movements. Brain and cognition\| PubMed ID: 22626916	2012
Frontotemporal oxyhemoglobin dynamics predict performance accuracy of dance simulation gameplay: temporal characteristics of top-down and bottom-up cortical activities. NeuroImage\| PubMed ID: 23707582	2014
Localization of extrastriate body area using functional near-infrared spectroscopy and 3D digitizer. Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual Conference\| PubMed ID: 24111457	2013
Recalibration of subjective simultaneity between self-generated movement and delayed auditory feedback. Neuroreport\| PubMed ID: 24217393	2014
Relationship between sensitivity to visuotactile temporal discrepancy and the rubber hand illusion. Neuroscience research\| PubMed ID: 24874005	2014
Motor area activity for action-related and nonaction-related sounds in a three-dimensional sound field reproduction system. Neuroreport\| PubMed ID: 25714418	2015

Sotaro Shimada

Biography

JoVE Journal Publications

Other Publications