Concurrent EEG and Functional MRI Recording and Integration Analysis for Dynamic Cortical Activity Imaging

Summary

An EEG-fMRI multimodal imaging method, known as the spatiotemporal fMRI-constrained EEG source imaging method, is described here. The presented method employs conditionally-active fMRI sub-maps, or priors, to guide EEG source localization in a manner that improves spatial specificity and limits erroneous results.

Abstract

Electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) are two of the fundamental noninvasive methods for identifying brain activity. Multimodal methods have sought to combine the high temporal resolution of EEG with the spatial precision of fMRI, but the complexity of this approach is currently in need of improvement. The protocol presented here describes the recently developed spatiotemporal fMRI-constrained EEG source imaging method, which seeks to rectify source biases and improve EEG-fMRI source localization through the dynamic recruitment of fMRI sub-regions. The process begins with the collection of multimodal data from concurrent EEG and fMRI scans, the generation of 3D cortical models, and independent EEG and fMRI processing. The processed fMRI activation maps are then split into multiple priors, according to their location and surrounding area. These are taken as priors in a two-level hierarchical Bayesian algorithm for EEG source localization. For each window of interest (defined by the operator), specific segments of the fMRI activation map will be identified as active to optimize a parameter known as model evidence. These will be used as soft constraints on the identified cortical activity, increasing the specificity of the multimodal imaging method by reducing cross-talk and avoiding erroneous activity in other conditionally active fMRI regions. The method generates cortical maps of activity and time-courses, which may be taken as final results, or used as a basis for further analyses (analyses of correlation, causation, etc.) While the method is somewhat limited by its modalities (it will not find EEG-invisible sources), it is broadly compatible with most major processing software, and is suitable for most neuroimaging studies.

Introduction

Electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) can be viewed as neuroimaging modalities with complementary features. FMRI captures brain activity with large temporal scale, as hemodynamic signals indirectly measure the underlying neuronal activity with a poor temporal resolution (on the order of seconds)^1,2. In contrast, EEG directly measures the dynamic electrophysiological activity of the brain with a very high temporal resolution (millisecond level), but poor spatial resolution^3,4. These properties have led to multimodal approaches designed to optimize the favorable aspects of each individual method⁵. Simultaneous use of EEG and fMRI allows for the excellent temporal resolution of EEG to be combined with the high spatial accuracy of fMRI to overcome the limitations associated with unimodal fMRI or EEG.

Methods for EEG and fMRI integration begin with fMRI-informed EEG source localization^6,7. This technique utilizes fMRI-derived spatial information to improve EEG source localization, however, one drawback is the potential spatial bias caused by the application of fMRI as a "hard-constraint" — fMRI-derived spatial information is considered an absolute truth. This poses two large issues that must be reconciled^6-8. First, it must be considered that the use of a static map of Blood Oxygen Level Dependent (BOLD) contrasts may inadvertently strengthen any erroneous activity that falls within it, while damping true activity outside of it. Second, crosstalk from sources occurring outside of the BOLD activation map may influence the presentation of true activity within the results or cause erroneous activity. Despite this, the use of the high spatial resolution of fMRI to provide prior spatial knowledge remains a favorable solution⁵, as the modeling of the EEG inverse problem can be constrained both in the anatomical and functional senses.

In this paper, we demonstrate a spatiotemporal fMRI-constrained EEG source imaging approach that addresses the issue of temporal mismatch between EEG and fMRI by calculating the optimal subset of fMRI priors based on a hierarchical Bayesian model⁹. FMRI-priors are computed in a data-driven manner from particular windows of interest in the EEG data, leading to time-variant fMRI constraints. The proposed approach utilizes the high temporal resolution of EEG to compute a current density mapping of the cortical activity, informed by the high spatial resolution of fMRI in a time-variant, spatially selective manner that accurately images dynamic neural activity.

Protocol

The protocol presented here was designed and performed in accordance with all guidelines for ethical human research as set forth by the respective Institutional Review Boards of the University of Houston and the Houston Methodist Research Institute.

1. Simultaneous EEG/fMRI Recording

1. Obtain informed consent from the participant. Explain to the participant the purpose and procedure of the study, as well as the important safety measures for the simultaneous EEG/fMRI data recording process.
2. Prepare the EEG cap and check impedance outside the MRI-scanner room.
   1. Place an appropriately sized, passive, MRI-compatible EEG cap onto the subject's head. Position the electrodes as per the 10–20 international labeling system¹⁰.
   2. On the EEG recording software, check the impedance of the ground and reference electrodes. To do this, click on the 'impedance' tab and select the electrode type on the software user-interface (see Figure 1).
      NOTE: Exact instructions here are specific to the software used herein (see Table of Materials), and may need to be adapted to other systems.
   3. For each electrode, use a syringe to inject electrolyte gel into the electrode, then use a cotton swab to spread the gel to ensure skin-electrode contact.
      1. As the impedance decreases, continue to monitor the values using the appropriate software (adjust the impedance scale as necessary, depending on the setup) to monitor impedance level properly (see Figure 1). Continue until all electrodes reach impedance levels below 10 kΩ to ensure a high-quality signal.
         NOTE: Per the materials listed and utilized here, it is considered unsafe to have any electrode with an impedance level above 50 kΩ in MR-environment¹¹. This may change depending on the design of the chosen cap and MRI settings, so please consult with the equipment manufacturer and MRI technologists to ensure the safety of the experimental setup.
3. Simultaneous EEG/fMRI hardware setup.
   1. Once the EEG cap preparation is done, have the subject moved to the MR-scanner with the hardware setup described in Figure 2.
      NOTE: Some details of the figure may change, depending on the system in use.
   2. Set up the experimental paradigm display. Use a monitor located in the observation room, behind the glass window facing the front of the MR-scanner (see Figure 2). Use a head coil viewing mirror to allow subjects to view the monitor screen without moving their head or eyes while lying down.
   3. Display a sample image on the computer screen to ensure that subjects can comfortably view the screen, and that the paradigm will display properly. Make any necessary hardware or software adjustments.
4. Experimental paradigm (see Figure 3).
   1. Instruct the subject to remain still, and perform an initial T1-weighted anatomical MRI scan. If possible, use a Field of View that reaches from the bottom of the cerebellum to the top of the head, including the skull and skin.
   2. Start recording the EEG data (see Figure 4).
   3. Simultaneously click the appropriate buttons to begin the MRI recording and initiate the paradigm of interest on the presentation software. Check the EEG data recording to ensure signal quality and, if desired, appropriate markers are being recorded.
      1. When using the set-up described here, first click "Run" in the presentation software and enter the subject number and trial number. The paradigm will initiate upon confirming these settings.
         NOTE: The paradigm employed here consisted of 10 trials in which an emotionally motivated motor response was evoked by means of visual stimulus. For each trial, subjects were asked to first rest for 50 s watching a green screen, after which the image of an unpleasant (images corresponding to surprise, anger, or disgust) or not-unpleasant (images corresponding to happiness or neutrality) face¹² was presented for 10 s. Five images from each category were presented in a randomized order, and subjects were asked to squeeze a ball upon identifying a face as unpleasant, and hold the squeeze until it disappeared.
      2. Use a Gradient-recalled Echo Planar Imaging (GR-EPI) sequence for fMRI recording (recommended); customize to suit the equipment and paradigm.
         NOTE: The sequence used herein included: Echo Time (TE) = 35 ms; Repetition Time (TR) = 1,500 ms; Slice Thickness = 5 mm; Flip Angle = 90 °; Pixel Spacing: 2.75 mm x 2.75 mm. It may be necessary to use an MRI sequence that lasts slightly longer than the display of the paradigm itself, to ensure that the full paradigm is recorded without clipping.

2. Structural MRI Data Analysis and Forward Model Generation

1. Apply full segmentation and reconstruction of various surfaces from the subject's T1-weighted anatomical MRI volume using the Freesurfer image analysis suite^13,14.
   NOTE: A folder containing all segmentation outputs will be generated by Freesurfer.
2. Generate a subject-specific 3-layer Boundary Element Method (BEM) geometrical model following the instructions provided at (https://martinos.org/mne/dev/manual/appendix/bem_model.html)¹⁵ Use the available Graphical User Interface (GUI) to ensure that there is no overlap in the layers.
   1. Open the Freeview Application. Click "File" >> "Load Surface". Navigate to the Subjects directory in the Freesurfer folder. Open the "BEM" folder. Open the "Watershed" folder. Load the four files found here ('outer_skin_surface', 'outer_skull_surface', 'brain_surface', and 'inner_skull_surface').
   2. Move the slice selection sliders and look for overlap in the yellow surface layers. If overlap does occur, double-check the MRI data for anatomical defects or errors, and use the GUI drawing tools to clarify the layers.
      1. Load the original MRI data in the Freeview application by clicking "File" >> "Load Volume". Navigate to the subject folder and open the "mri" folder. Then click on the "orig" directory and open the structural MRI data found there (should be in.mgz or.nii format). Click "OK".
      2. View the greyscale image of the head. Look at the different layers of grey and black around the brain. Ensure that these layers do not have any gaps or irregularities. Use the "Color Picker" tool and select a voxel from the layer to be corrected.
      3. Switch to the "Freehand Voxel Edit", and click to draw on the image. Use this to fill in any defects in the MRI image. Perform correction for all layers and MRI slices, where defects occur.
         NOTE: the "Polywire" and "Livewire" voxel editing tools may also be used in place of the "Freehand".
3. Generate the source space based on the geometry of the pial surface.
4. Perform subject-specific EEG sensor alignment (e.g., translation and rotation) to the MRI space using the Freesurfer head model overlay (Figure 5). Save the transformation.
   1. Open the MNE_analyze application. Click on "File" >> "Load surface". Navigate to the folder containing the subject data and load the pial surface.
   2. Click "File" >> "Load Digitizer Data and select the EEG file of interest (should contain digitizer data). Click "View" >> "Show Viewer". Once the viewer GUI appears, click "Options" and make sure that the "Scalp" and "Digitizer data" options are chosen. Electrodes here are shown in red, with fiducial points in yellow.
   3. On the main window (not the viewer), select "Adjust" >> "Coordinate alignment". Using the 'Coordinate alignment GUI', use the arrow and L/R buttons to shift and rotate the EEG electrodes in the viewer. Adjust as much as necessary. Once the alignment is done, click "Save…" at the bottom of the 'Coordinate Alignment GUI' to save the alignment.
      NOTE: Typically, an even distribution of electrodes throughout the scalp with good fiducial alignment is required.
5. Generate the forward model by providing the subject-specific BEM model, the source space, and the EEG sensor transformation using MNE software¹⁵.

3. Functional MRI Data Analysis

1. Perform first-level (individual subject) fMRI statistical analysis using the General Linear Model (GLM) method to acquire BOLD activation maps for the tasks of interest. Correct for multiple comparisons as necessary¹⁶, using the cluster-based approach that is built into the Freesurfer group-analysis pipeline.
2. Perform group-level analysis on all subjects, if desired, to acquire the BOLD activation map for all subjects in standard space (MNI or Talairach).
   NOTE: The University of Oxford Centre for Functional MRI of the Brain (FMRIB) Software Library (FSL)¹⁷ and Analysis of Functional Neuroimages (AFNI)¹⁸ packages both allow for the analysis of fMRI data on the same surfaces generated by Freesurfer, making them convenient for subsequent analysis.
3. Use the tksurfer visualization tool to perform region-of-interest (ROI) identification by loading the fMRI activation map (both individual-level and group-level), and setting the desired FDR-corrected threshold¹⁹ (p <0.05 is used here).
   ​NOTE: The ROIs identified from individual-level activation maps will serve as the subject-specific fMRI-derived spatial priors for subsequent source localization.
   1. Using the fMRI activation map on the gray matter layer, extract surface patches using a connected-labeling algorithm.
      NOTE: Dulmage-Mendelsohn decomposition was used in this example.
   2. Further sub-divide the patches based on the labeling of a predefined brain atlas, so that any patch of activity covering more than one region is split.
      NOTE: The atlas used here was the DKT40 atlas²⁰ (available from Freesurfer)²¹. Atlases can be specialized or chosen, based on experimental preferences.
4. Project the acquired group-level ROIs (which are currently in standard space) back to the individual source spaces of each subject. After performing the individual subject's structural MRI segmentation (step 2.1), the coordinate transformations between the subject and standard space are provided in the lh.sphere.reg and rh.sphere.reg files, found in the "surf" folder of the subject's Freesurfer output folder.
   NOTE: All subjects will thereby share the same set of ROIs, but in their own specific model. See Figure 6 for examples of the fMRI results and resultant ROIs.

4. EEG Data Analysis

NOTE: Details in this section may be specific to the software used (See Table of Materials for more details). Please refer to the appropriate documentation if using different software packages.

1. Perform scanner gradient artifacts correction through template subtraction. For this, click on the "MR Correction" button in the "Special Signal Processing" menu, and select appropriate parameters in the EEG analysis software GUI (see Figure 7). Input the appropriate parameters to the chosen scanner sequence and experimental design.
   NOTE: Primary parameters include: repetition time (TR) for the MRI scan, scan type (interleaved or continuous), MRI volume markers (or gradient detection method and gradient trigger), channels for correction, and artifact template.
2. Remove cardioballistic artifacts through template subtraction. For this, click on the "CB Correction" button in the "Special Signal Processing" menu, and select appropriate parameters in the analysis software GUI.
   NOTE: Parameters necessary here include minimum and maximum heart rate, artifact template, ECG channel, template correlation, and channels for correction.
3. Apply filtration. Select the button for IIR filtration at the top of the analysis GUI, under "Data Filtration". For example, apply high-pass at 0.05 Hz, low-pass at 40 Hz, and a notch-filter at the power-line frequency (60 Hz), with a roll-off of 48 dB/Hz.
   NOTE: After the application of a low-pass filter at a cutoff frequency of 40 Hz, the 60 Hz notch-filter is not strictly necessary, but is employed as a safeguard against any residual power-line frequencies that may have survived due to the roll-off at filter edges.
4. Perform ocular artifact correction, on top of the analysis GUI: select "Transformation" >> "Artifact Rejection/Reduction" >> "Ocular Correction ICA".
5. Segment the EEG data into epochs based on the specified pre- and post-stimulus time, with respect to the event timing markers. To do this, select, "Transformation" >> "Segment Analysis Functions" >> "Segmentation", then choose the marker of interest and the time segment of interest.
   ​NOTE: Segmentation lengths should be chosen to suit the paradigm and expected brain activity of interest.
6. Perform manual or semi-automatic artifact rejection: select "Transformation" >> "Artifact Rejection/Reduction" >> "Artifact Rejection". When prompted, define criteria for artifacts within the three tabs of the GUI and proceed as instructed on the GUI.
   1. In the 'Inspection Method' tab, select choose "automatically", "semi-automatically", or "manually select artifacts" (semi-automatic mode is recommended). Then select "mark" or "remove artifacts", and specify if the corrections are for a single channel.
   2. In the 'Channel Selection' tab, select the channels which will be corrected for artifacts.
   3. In the 'Criteria' tab, select the basis by which artifacts will be identified. Make selections here to fit experimental needs. Click "OK" after selecting criteria, and artifacts will be identified and/or rejected in accordance with the selections.
7. Perform baseline correction and trial averaging (if applicable).
   1. To perform baseline correction: select "Transformation" >> "Segment Analysis Functions" >> "Baseline Correction". To average the segmented data: select "Transformation" >> "Segment Analysis Functions" >> "Average".

5. Spatiotemporal fMRI Constraints — EG Source Imaging

1. Define window size and window overlapping size (the default setting calls for a 40 ms window size with 50% (20 ms) overlap).
2. Select the subject-specific ROIs set (obtained in step 3) as the spatial prior set. For each EEG segment, the algorithm will then estimate a set of weights for the subset of spatial priors that maximizes the model evidence, and computes the source covariance matrix accordingly.
3. Using the resulting source covariance matrix, perform source localization for the EEG segment being analyzed, yielding the source current density results.
4. Perform steps 5.2 and 5.3 for all EEG segments and, if necessary, summarize the current density results for all time segments into one complete current density time-course by averaging the overlapping portion.
   ​NOTE: This step will result in a current density time-course of cortical activity at every source point defined in step 2.3 (this number is typically on the order of several thousands) (Figure 8).
5. Extract the representative current density time-course at each of the ROIs.
   1. Select the preferred method for summarizing the time-courses from the multiple source points within an ROI into a single signal time-course: average, first eigenvariate, etc.
6. Repeat steps 5.1 to 5.5 for all subjects.

Results

EEG source localization at the basic level involves the solving of the forward and inverse problem. The components required to build and solve the forward problem are shown in Figure 5C. Using a subject-specific T1 image, three layers — brain, skull, and skin — were segmented and meshed. These layers served as the inputs to generate the BEM model. Similarly, the subject's grey-matter layer was segmented from the structural MRI and used to con...

Discussion

We have shown here the necessary steps to use the spatiotemporal fMRI constrained source analysis method for EEG/fMRI integration analysis. EEG and fMRI have become well established as the fundamental methods for non-invasively imaging brain activity, though they face difficulty in their respective spatial and temporal resolutions. While methods have been developed to capitalize on the favorable properties of each, current fMRI-constrained EEG source localization methods frequently rely upon simple fMRI constraints, whic...

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
BrainAmp MR Plus	Brain Products		Amplifiers for EEG recording, MR-compatible
BrainAmp ExG MR	Brain Products		Amplifier for auxilary sensor (EMG), MR-compatible
BrainAmp Power Pack	Brain Products		Provide power to amplifiers in the MR environment
Ribbon Cables	Brain Products		Connects the Power Pack to Amplifiers
SyncBox	Brain Products		Synchronize MR scanner clock with EEG amplifier clock
BrainCap MR	Brain Products		Passive-electrode 64-channel EEG cap, MR-compatible
BrainVision Recorder	Brain Products		EEG data recording software (steps 1.2-1.4.2)
BrainVision Analyzer 2.0	Brain Products		EEG analysis software (steps 4.1-4.6)
USB 2 Adapter (also known as BUA)	Brain Products		Interface between the amplifiers and data acquisition computer
Fiber Optic Cables	Brain Products		Connects the EEG cap in the MR scanner to the Recording Computer
SyncBox Scanner Interface	Brain Products		Synchronize MR scanner clock with EEG amplifier clock
Trigger Cable	Brain Products		Used to send scanner/paradigm triggers to the recording computer
ABRALYT HiCl EEG Electrode Gel	EasyCap		Abrasive EEG gel for passive electrode in MR environment
Ingenia 3.0T MR system	Philips		3.0 T MRI system
Patriot Digitizer	Polhemus		EEG channel location digitization
MATLAB r2014a	MathWorks		Programming base for the DBTN algorithm (steps 3.3-3.4 and 5.1-5.7)
Pictures of Facial Affect	Paul Eckman Group		A series of emotionally valent faces used as stimuli
E-Prime 2.0	Psychology Software Tools, Inc		Presentation Software (step 1.4.3)
Bipolar skin EMG electrode	Brain Products		Used to detect muscle activity.
POLGUI			MATLAB software for digitization
Freesurfer			Software used in steps 2.1-2.4, and steps 3.1-3.2
MNE			Software used in step 2.5