Conducting Concurrent Electroencephalography and Functional Near-Infrared Spectroscopy Recordings with a Flanker Task

Shi Yang Xu; Lai Ian Cheong; Yin Zhuang; Tania  Alexandra Pinho Couto; Zhen Yuan

doi:10.3791/60669

Neuroscience

Conducting Concurrent Electroencephalography and Functional Near-Infrared Spectroscopy Recordings with a Flanker Task

Published: May 24, 2020 doi: 10.3791/60669

Shi Yang Xu^1,2, Lai Ian Cheong², Yin Zhuang², Tania Alexandra Pinho Couto^1,2, Zhen Yuan^1,2

¹Centre for Cognitive and Brain Sciences, University of Macau, ²Faculty of Health Sciences, University of Macau

Summary

The present protocol describes how to perform concurrent EEG and fNIRS recordings and how to inspect the relationship between the EEG and fNIRS data.

Abstract

Concurrent EEG and fNIRS recordings offer an excellent opportunity to gain a full understanding of the neural mechanism of cognitive processing by inspecting the relationship between the neural and hemodynamic signals. EEG is an electrophysiological technology that can measure the rapid neuronal activity of the cortex, whereas fNIRS relies on the hemodynamic responses to infer brain activation. The combination of EEG and fNIRS neuroimaging techniques can identify more features and reveal more information associated with the functioning of the brain. In this protocol, fused EEG-fNIRS measurements were performed for concurrent recordings of evoked-electrical potentials and hemodynamic responses during a Flanker task. In addition, the critical steps for setting up the hardware and software system as well as the procedures for data acquisition and analysis were provided and discussed in detail. It is expected that the present protocol can pave a new avenue for improving the understanding of the neural mechanisms underlying various cognitive processes by using the EEG and fNIRS signals.

Introduction

This study aims to develop a working protocol to reveal the neural activation pattern underlying the Flanker task by using fused EEG and fNIRS neuroimaging techniques. Interestingly, the concurrent fNIRS-EEG recordings allow for the inspection of the relationship between the hemodynamic signals in the prefrontal cortex and various event-related potential (ERP) components of the whole brain associated with the Flanker task.

The integration of various noninvasive neuroimaging modalities including functional near-infrared spectroscopy (fNIRS), electroencephalography (EEG), and functional magnetic resonance imaging (fMRI) is essential to improve the understanding of where and when information processing is taking place in the brain¹^,²^,³. Additionally, there is the potential to combine fNIRS and EEG to examine the relationship between local neural activity and subsequent changes in hemodynamic responses, in which EEG and fNIRS can be complementary in revealing the neural mechanism of human brain cognitive function. fNIRS is a vascular-based functional neuroimaging technique that relies on the hemodynamic responses to infer brain activation. fNIRS measures the relative oxyhemoglobin (HbO) and deoxyhemoglobin (HbR) concentration changes in the cerebral cortex, which plays an important role in the study of cognitive processing³^,⁴^,⁵^,⁶^,⁷. According to the neurovascular and neurometabolic coupling mechanism⁸, the change of local neural activity associated with cognitive processing is generally accompanied by subsequent alterations in the local blood flow and blood oxygen with a delay of 4-7 seconds. It is shown that the neurovascular coupling is likely a power transducer, which integrates the fast dynamics of neural activity into the vascular input of slow hemodynamics⁹. Specifically, fNIRS is mostly used for inspecting the neurovascular activity in the frontal lobe, especially the prefrontal cortex that is responsible for high cognitive functions, such as executive functions¹⁰^,¹¹^,¹², reasoning and planning¹³, decision making¹⁴, and social cognition and moral judgment¹⁵. However, the hemodynamic responses measured by fNIRS only indirectly capture the neural activity with a low temporal resolution, whereas EEG can offer temporally fine and direct measures of neural activities. Consequently, the combination of EEG and fNIRS recording can identify more features and reveal more information associated with the functioning of the brain.

More importantly, the multi-modal acquisition of EEG and fNIRS signals has been conducted to inspect the brain activation underlying various cognitive tasks¹⁶^,¹⁷^,¹⁸^,¹⁹^,²⁰^,²¹^,²² or brain-computer interface²³^,²⁴. In particular, concurrent ERP (event-related potential) and fNIRS recordings were carried out based on the event-related auditory oddball paradigm¹, in which fNIRS can identify the hemodynamic changes in the frontotemporal cortex several seconds after the appearance of P300 component. Horovitz et al. also demonstrated the simultaneous measurements of fNIRS signals and the P300 component during a semantic processing task²⁵. Interestingly, previous studies based on simultaneous EEG and fNIRS recordings showed that P300 during oddball stimuli exhibited a significant correlation with fNIRS signals²⁶. It was discovered that the multi-modal measures have the potential to reveal the comprehensive cognitive neural mechanism based on the event-related paradigm²⁶. Besides the oddball task, the Flanker task associated with ERP component N200 is also an important paradigm, which can be used for the investigation of cognitive ability detection and evaluation with healthy controls and patients with various disorders. Specifically, N200 was a negative component that peaks 200-350 ms from the anterior cingulated cortex frontal²⁷ and superior temporal cortex²⁸. Although previous studies examined the relationship between the superior frontal cortex and alpha oscillation in the Flanker task²⁹, the correlation between the N200 amplitude and the hemodynamic responses during the Flanker task has not been explored.

In this protocol, a home-made EEG/fNIRS patch based on standard EEG cap was utilized for the concurrent EEG and fNIRS recordings. The arrangements of optodes/electrodes with support were achieved through the placement of fNIRS optodes fused into the EEG cap. The simultaneous EEG and fNIRS data acquisitions were carried out with the same stimuli tasks generated by E-prime software. We hypothesize that ERP components associated with the Flanker task can exhibit a significant correlation with the hemodynamic responses in the prefrontal cortex. Meanwhile, the combined ERP and fNIRS recordings can extract multiple signal indicators to identify the brain activation patterns with enhanced accuracy. To test the hypothesis, the fNIRS setup and EEG machine were integrated to reveal the complex neural cognition mechanism corresponding to the event-related Flanker task.

Subscription Required. Please recommend JoVE to your librarian.

Protocol

Prior to the experimental tests, all participants signed informed consent documents. The protocol for the present study was approved by the Ethics Committee of the University of Macau.

1. Hardware and software setting for concurrent EEG and fNIRS recordings

Construct a head cap for concurrent EEG-fNIRS recordings.
1. Select the appropriate cap size according to the head circumference of participants. In this study, use a medium-size cap since it is suitable for most adolescent and adult participants.
2. Design the layout of fNIRS optodes along with the EEG cap in the prefrontal cortex (Figure 1).
  1. Place EEG electrodes in the middle section of the fNIRS optodes to ensure the measurement of the same brain region by the two techniques¹⁹^,³⁰. However, due to the low spatial resolution of both EEG and fNIRS neuroimaging methods, place the electrodes in the corresponding brain area covered by fNIRS optodes rather than the exact locations of fNIRS channels.
  2. Make 22 holes inside the EEG cap to hold the fNIRS optodes in line with the specific layout in the prefrontal cortex. Identify and mark the locations of fNIRS optodes according to the designed layout of the head cap and then punch holes inside the cap to place and fix the optodes.
  3. Place 21 or 71 EEG electrodes along the surface of the EEG cap (see Table of Materials) according to the 10-20 International System and mount the grids for the optodes.
3. Set the distance between each source-detector pair as 3 cm and then fix the optodes, in which the blue optodes denote the light detectors while the red ones represent the laser sources.
Set the EEG and fNIRS ports in the software.
Use the time triggers generated through the parallel port and serial port to ensure the synchronization of two different signals.
1. Set the parallel port (e.g., H378 in this study) for the EEG system (see Table of Materials).
2. Set the serial port (e.g., 6 9600 in this study) for the fNIRS system (see Table of Materials).
  NOTE: The port type and number should be modified regarding various EEG and fNIRS setups. Please contact the manufacturers for more information.

2. Experimental preparation

Warm up the fNIRS system with lasers switched on for 30 min.
Set all necessary operation parameters for the fNIRS measurement system.
Show the fused experimental setup including the EEG and fNIRS measurement systems to participants.
Measure and mark the Cz point according to the 10-20 International System. Identify the electrode position of Cz at half of the distance between the inion and nasion and half of the distance between the left and right inter-aural indentations.
Place the front part of the cap along the participant’s forehead first and then pull down the back section of the cap towards the neck.
Validate the positions.
1. Measure the distance between the Cz and inion and nasion again with a soft ruler, and double-check whether it is located at the midpoint. Likewise, measure the distance between the Cz and left and right inter-aural, and double-check whether the Cz is located at the midpoint.
Prepare for the EEG recordings.
NOTE: It is highly recommended that the EEG electrodes be set up first and then the fNIRS optodes. If EEG conductive gel covers the holes for the placement of fNIRS optodes, it should be cleaned to prevent the contamination of optodes.
1. Fill conductive gel by inserting a blunt needle through the holes of the EEG electrode grid.
2. Place all electrodes into the EEG electrode grid according to the labels.
3. Open the EEG software and inspect the signal quality of EEG electrodes.
4. Readjust the electrode by refilling conductive gel if the signal quality is not good enough to meet the requirements (40 mV).
5. Readjust the electrode by refilling conductive gel if the impedance could not meet the requirements.
Prepare for the fNIRS recordings.
Caution: Do not expose participants’ eyes to the laser beam of fNIRS sources directly.
1. Place the optical fibers along the holder arms attached to the fNIRS measurement system as well as the holder. Ensure that the fibers are neat and tidy.
2. Insert the optical sources and detectors into the holes according to the layout.
3. Test the signal quality. If a channel does not have a high-level signal-to-noise ratio (i.e., if the channel is marked in yellow), gently inspect the participant’s hair surrounding the optical probes to ensure that nothing exists between the optical probe and scalp.
4. If step 2.8.3 cannot improve the signal quality, turn up the signal intensity. If there is too much signal (i.e., if the channel is marked in red), turn down the signal intensity.

3. Run the experiment

Start the experiment when the signals are stable with excellent signal-to-noise ratio and participants are familiar with the experiment instructions. Use the classic Flanker paradigm for the experimental test²⁹^,³¹.
After the experiment, save and export the data from both EEG and fNIRS.
Remove EEG electrodes and fNIRS optical probes carefully.

4. Measurement of three-dimensional (3D) MNI coordinates of fNIRS optodes with 3D digitizer

Let participants sit in a chair and wear the glasses with the sensor.
Open the digitizer software on the computer. Ensure that the 3D digitizer system is in connection with the computer through an appropriate COM port.
Load the layout of the optodes setting file.
Move the 3D digitizer stylus across the key positions (Nz, Iz, left ear, right ear, Cz) along with the screen and press the button on the stylus.
Localize the optical sources and detectors
Export the 3D coordinates files.

5. Data analysis

fNIRS data analysis
1. Process the 3D MNI coordinates data by using the registration option in NIRS-SPM with MATLAB 2019. Select: stand-alone spatial registration | With 3D Digitize. Choose the previously saved others and origin text files and then select Registration.
2. Pre-process fNIRS signals with Homer2 software³².
  1. Convert the raw data to optical density changes for different wavelengths and further convert to the concentration changes of HbO at different time points using a modified Beer-Lambert Law. Generally, the typically differential path length factor (DPF) value affected by the age, gender, and wavelength, and the distance between the source and decetor³³^,³⁴ is 6, which is similar to the average DPF from previous studies³⁴^,³⁵.
  2. Use the spline motion artifacts detection algorithm from the Homer2 fNIRS processing package for motion correction. Please select the appropriate methods of motion correction based on literature³⁶.
  3. Process the raw hemoglobin continuous data by a low-pass filter of 0.2 Hz and subsequently a high-pass filter of 0.015 Hz.
  4. Normalize hemodynamic signal amplitude by dividing the averaged values.
  5. Generate the fNIRS data for each channel based on the 3D digitizer information. Select the channels that have a registration probability of 100% or more in the superior frontal cortex (SFC)according to the regression calculation of the NIRS-SPM for further analysis.
  6. Export the peak values of oxygen hemoglobin (HbO) concentration changes.
    NOTE: In this study, only HbO signals were analyzed due to their high signal-to-noise ratio. The peak values of run-averaged HbO data were extracted for each channel from each participant for further analysis.
EEG data processing
NOTE: Offline EEG data analysis was performed with the EEGLAB. Only N200 at Fz was the interesting component for the present study. All electrodes were subjected to an automatic artifact correction to remove eye movements by using an internal model of artifact topographies. Continuous EEG data were then segmented into different trials according to target and nontarget stimuli, in which the epoch for each trial lasted 2500 ms, involving a pre-stimulus period of 500 ms (baseline epoch) and a post-stimulus period of 2000 ms (task epoch).
1. Load the raw EEG data folder into the EEGLAB by using the plugins. Choose the BIOSIG plugin for the BDF file in this study.
  NOTE: Please choose a suitable plugin according to the EEG data file format.
2. Set the channel location information for EEGLAB³⁷. Load the corresponding location file of the cap.
3. Re-reference electrodes in the ERPLAB, which is one plugin of EEGLAB. Choose the channels placed in the mastoids as reference electrodes.
4. Extract EEG data epochs based on the event and bin files in the ERPLAB³⁷.
5. Filter the EEG data segments in the ERPLAB by using the FIR filter by filtering the low frequencies with a cutoff of 30Hz and by filtering the high frequencies with a cutoff of 0.1 Hz.
6. Remove ocular EEG artifacts with the Independent Component Analysis in EEGLAB.
7. Reject EEG data segments with amplitude values exceeding ± 100 µV at any channel in ERPLAB.
8. Average the EEG data segments in ERPLAB.
  NOTE: These are the generally used data analysis method and the software for processing EEG and the fNIRS data. There are numerous processing software and methods available.
Correlation calculation
1. Generate the relationship between fNIRS and EEG recordings by using Pearson correlation analysis.

Subscription Required. Please recommend JoVE to your librarian.

Representative Results

Figure 2 shows the HbO signals for all channels while Figure 3 displays the ERPs at Fz and FCz for the two conditions of the Flanker task. Figure 4 illustrated the Pearson correlation analysis results showed that the fNIRS signals in SFC exhibited a significant correlation with the ERP N200 component at Fz for the incongruent condition (P<0.05). However, this is not the case for the congruent conditions (P>0.05).

Figure 1. fNIRS headset placement and channel configuration. The digitized optodes layout are converted into the MNI coordinate system and then overlapped along the brain cortex Please click here to view a larger version of this figure.

Figure 2. HbO signals for all channels associated with the Flanker task. The pink curves denote the incongruent condition while the green ones indicate the congruent condition. Please click here to view a larger version of this figure.

Figure 3. ERP signals for Fz and FCz electrodes. The black curves define the incongruent condition while the red ones denote the congruent condition. Please click here to view a larger version of this figure.

Figure 4: Correlation between the ERP N200 and HbO signals along the superior frontal cortex (SFC) for the incongruent condition. The regression coefficient between the two measurements is 0.59, p = 0.027. Please click here to view a larger version of this figure.

Subscription Required. Please recommend JoVE to your librarian.

Discussion

In this protocol, combined EEG and fNIRS recordings were performed to examine the brain activation patterns involving an event-related Flanker paradigm by recording the neural signals of the whole brain and concurrent hemodynamic responses of the prefrontal cortex. The ERP results showed that N200 at Fz was able to significantly distinguish the congruent and incongruent conditions (P=0.037). Meanwhile, the HbO signals in SFC (channels 21) also exhibited a significant difference between the congruent and incongruent conditions, which demonstrated the important role of the ability to suppress responses that involved the brain cognitive function associated with the Flanker task (P_FDR = 0.041).

In addition, N200 at Fz showed a significant correlation with the hemodynamic response in the SFC (channel 21) for the incongruent condition although this was not the case for the congruent one. The brain activation in the prefrontal cortex is strongly correlated with high cognitive functions, which can be easily identified by fNIRS with the high signal-to-noise ratio in the spatial domain. However, the neural activity (N200) detected by EEG associated with the same Flanker task is mostly revealed in the parietal cortex with high sensitivity and high temporal resolution. N200 at Fz exhibited the cognitive difference between the two conditions, whereas fNIRS signals illustrated the difference of suppression function in the prefrontal region between the two conditions. It was discovered that the cognition showed a significant relationship with executive control during the Flanker task. This might be the main reason why the N200 at Fz exhibited a significant correlation with the hemodynamic response in SFC.

In this protocol, we described how to conduct fused EEG and fNIRS recordings and how to analyze the event-related potential and measure the hemoglobin concentration changes in the prefrontal cortex. The synchronization of different setups is an essential concern for the fusion of two hardware systems. Meanwhile, the event-related trigger is also the crucial mark for the task design of concurrent EEG and fNIRS recordings.

Combined EEG and fNIRS recordings are promising techniques for the investigation of the neural mechanisms underlying various cognitive tasks. In summary, we successfully acquired concurrent EEG and fNIRS data during a Flanker task. The findings indicated that the fNIRS hemodynamic response and ERP component N200 were significantly correlated, which exhibited different perspectives of the cognitive mechanism associated with the Flanker task. The multi-modal neuroimaging results support an essential role of combined EEG and fNIRS technique in contributing to brain cognition with different latencies and activation regions, which paves a new avenue for improving the understanding of the neural mechanisms of Flanker task.

Subscription Required. Please recommend JoVE to your librarian.

Disclosures

The authors have nothing to disclose.

Acknowledgments

This work was performed in part at the high performance computing cluster (HPCC), which is supported by information and communication technology office (ICTO) of the University of Macau. This study was supported by MYRG2019-00082-FHS and MYRG 2018-00081-FHS grants from the University of Macau in Macau, and also funded by The Science and Technology Development Fund, Macau SAR (FDCT 0011/2018/A1 and FDCT 025/2015/A1).