Waiting
Login processing...

Trial ends in Request Full Access Tell Your Colleague About Jove
JoVE Journal Neuroscience

Neuroscience

How to Calculate and Validate Inter-brain Synchronization in a fNIRS Hyperscanning Study

Published: September 8, 2021 doi: 10.3791/62801
Automatic Translation
1, 1, 2, 3, 4, 1, 1
1Institute of Brain and Education Innovation, School of Psychology and Cognitive Science, East China Normal University, 2Department of Musicology, Harbin Conservatory of Music, 3Department of Clinical Neuroscience, Karolinska Institutet, 4School of Psychology, Shenzhen University

Summary

The dynamics between coupled brains of individuals have been increasingly represented by inter-brain synchronization (IBS) when they coordinate with each other, mostly using simultaneous-recording signals of brains (namely hyperscanning) with fNIRS. In fNIRS hyperscanning studies, IBS has been commonly assessed through the wavelet transform coherence (WTC) method because of its advantage on expanding time series into time-frequency space where oscillations can be seen in a highly intuitive way. The observed IBS can be further validated via the permutation-based random pairing of the trial, partner, and condition. Here, a protocol is presented to describe how to obtain brain signals via fNIRS technology, calculate IBS through the WTC method, and validate IBS by permutation in a hyperscanning study. Further, we discuss the critical issues when using the above methods, including the choice of fNIRS signals, methods of data preprocessing, and optional parameters of computations. In summary, using the WTC method and permutation is a potentially standard pipeline for analyzing IBS in fNIRS hyperscanning studies, contributing to both the reproducibility and reliability of IBS.

Abstract

The dynamics between coupled brains of individuals have been increasingly represented by inter-brain synchronization (IBS) when they coordinate with each other, mostly using simultaneous-recording signals of brains (namely hyperscanning) with fNIRS. In fNIRS hyperscanning studies, IBS has been commonly assessed through the wavelet transform coherence (WTC) method because of its advantage on expanding time series into time-frequency space where oscillations can be seen in a highly intuitive way. The observed IBS can be further validated via the permutation-based random pairing of the trial, partner, and condition. Here, a protocol is presented to describe how to obtain brain signals via fNIRS technology, calculate IBS through the WTC method, and validate IBS by permutation in a hyperscanning study. Further, we discuss the critical issues when using the above methods, including the choice of fNIRS signals, methods of data preprocessing, and optional parameters of computations. In summary, using the WTC method and permutation is a potentially standard pipeline for analyzing IBS in fNIRS hyperscanning studies, contributing to both the reproducibility and reliability of IBS.

Introduction

When people coordinate with others, their brains and bodies become a coupled unit through continuous mutual adaption. The coupling between brains can be represented by inter-brain synchronization (IBS) through the hyperscanning approach, which simultaneously records two or more individuals' brain signals1. Indeed, a growing body of fNIRS/EEG hyperscanning studies has found IBS in various collaboration contexts, including finger tapping2, group walking3, playing drums4, guitar playing5, and singing/humming6. fNIRS is widely used for the research of IBS during social interaction, as it less restricts head/body motions in relatively natural settings (compared to fMRI/EEG)7.

The article presents a protocol for calculating IBS via wavelet transform coherence (WTC) method in an fNIRS hyperscanning study. WTC is a method for assessing the cross-correlation between two movement signals on the time-frequency plane and, therefore, can give more information than the traditional correlation analysis (e.g., Pearson correlation and cross-correlation), which is only in the time domain8. In addition, hemodynamic signals are transformed into wavelet components, which can effectively remove the low-frequency noise. Although WTC is time-consuming, it has been the most commonly used method of calculating IBS in action imitation9, cooperative behavior10, verbal communication11, decision making12, and interactive learning13.

The article also presents how to validate IBS with the permutation-based random paring of trials, conditions, and participants. The IBS in hyperscanning studies is always proposed to track online social interaction between individuals, while it can also be interpreted by other explanations, such as the stimulus similarity, motion similarity, or condition similarity14. Permutation test, also called randomization test, can be leveraged to test the above-mentioned null hypotheses through resampling the observed data15. By using permutation, it is useful to investigate whether the identified IBS is specific to interactive behavior, ranging from modulation of IBS within dyads to between groups of partners16.

The protocol described here details how to obtain brain signals via fNIRS technology, calculate IBS through the WTC method, and validate IBS by permutation testing in a hyperscanning study. This study aims to examine whether privileged IBS is elicited by music meters during social coordination. The brain signals were recorded in the frontal cortex, based on the location of the IBS in a previous finding1. The experimental task was originally developed by Konvalinka and her colleges17, in which participants were asked to tap their fingers with the auditory feedback from the partner or themselves after listening to the meter or non-meter stimuli.

Subscription Required. Please recommend JoVE to your librarian.

Protocol

The protocol presented here was approved by the University Committee on Human Research Protection of East China Normal University.

1. Preparation for the experiment

  1. Participants
    1. Recruit a group of undergraduate and graduate students with monetary compensation by the campus advertising.
    2. Ensure that the participants are right-handed and have normal or corrected-to-normal vision and hearing. Ensure that they have not studied music or have studied it for fewer than 3 years before.
    3. Randomly match the students in dyads. To control the potential effect of partner familiarity on social coordination18, ensure that the members of each dyad have not seen or known each other before.
  2. Experimental stimulus
    1. Create the auditory stimuli (440 Hz, 660 ms pure tones) by any free music composition and notation software.
    2. Repeat the tones with an interval of 500-1000 ms and combine them into a tone sequence. Each tone sequence is 12 s longer and composed of 12 tones.
    3. For one tone sequence, accent every first tone (+6 dB) to create the pattern of downbeats and upbeats, defined as the meter stimulus (Supplementary Audio 1). In the second tone sequence, unaccent the tones with equal intensity (40 dB above individual sensation threshold, collected before the experiment task), which corresponded to the non-meter stimulus (Supplementary Audio 2).
  3. Experimental task
    1. Program the experimental task by using a psychological software tool.
    2. Arrange two stages for the experimental task (Figure 1A) as described in steps 1.3.3-1.3.6.
    3. second resting state: Ask participants to remain as motionless as possible with their minds relaxed and eyes closed.
    4. Finger tapping task: Request the participants to complete two parts: a coordination part and an independence part.
    5. During the coordination part, provide auditory feedback (i.e., a drip sound corresponding to one tap) to each participant only for the response generated by the other member of the dyad. Ask the participants to try their best to respond synchronously with the other member.
    6. For the independence part, ensure both participants received the auditory feedback (i.e., a drip sound corresponding to one tap) of their own responses and ask them to respond synchronously with the auditory stimulus as precisely as possible.
      NOTE: Combined with the meter and non-meter stimuli, participants found themselves in one of four different conditions: (i) meter coordination - both participants heard meters, and the responses from each other; (ii) non-meter coordination - both participants heard non-meters and the responses from each other; (iii) meter independence - both participants heard meters and the responses from themselves; (iv) non-meter independence - both participants heard non-meters and the responses from themselves.
    7. For each trial, let the participants first hear a piece of the auditory stimulus (12 s) followed by a sound (262 Hz, 1000 ms) that serves as a cue to start to tap the finger.
    8. Ask the participants to reproduce the stimulus they heard before by tapping their right index finger on the keyboard (participant #1: "f"; participant #2: "j"). Participants must tap 12 times while keeping the same time interval between tones as the previously presented stimulus.
      NOTE: There were 60 trials assigned equally in 4 blocks corresponding to the 4 experimental conditions, namely 15 trials in one block. The order of the blocks was counterbalanced. The total duration of the tapping task was about 26 min.
    9. Between blocks, let the participants rest for 30 s.
    10. During the whole experiment, do not allow the participants to communicate through any language or movement. Separate the participants with the computer monitor to block any visual information that might deliver messages between them.
  4. Homemade fNIRS caps: Buy two elastic swimming caps of normal size. To cover the brain region of interest, mend the swimming caps as described in the following steps:
    1. Put one swimming cap on a headform, and then put a standard 10-20 EEG cap on the swimming cap.
    2. Mark the location of FCz on the swimming cap with a red magic marker.
    3. Take off the EEG cap from the headform.
    4. Put one optode probe patch (3 x 5 setup) on the swimming cap, aligning the middle one of the second probe row of the patch with the marked location of FCz.
      NOTE: The optode probe patch included 15 locations of optode probes (i.e., 8 emitters and 7 detectors), forming 22 measurement channels with 3 cm optode separation (Figure 1B).
    5. Mark the locations of the 15 probes of the patch on the swimming cap.
    6. Take off the patch and the swimming cap from the headform.
    7. Cut 15 small holes on the marked locations of the 15 probes with a pair of scissors.
    8. Mount the patch to the modified swimming cap by embedding the locations of 15 probes into the appropriate 15 holes.
    9. Mend the other swimming cap according to the above process.
      NOTE: The Homemade fNIRS cap, in which the locations of optodes are as per the standard EEG locations, were employed as there were no applicable standard EEG caps for the fNIRS system used in this study. There is no need to make an fNIRS cap if there are suitable standard EEG caps with the fNIRS system.

2. Before participants arrive

NOTE: Ensure to follow steps 2.1-2.5 before participants arrive at the laboratory.

  1. Remind the two participants of one dyad to come to the laboratory per the agreed time schedule.
  2. Start the fNIRS system at least 30 min in advance, leaving the laser turned off.
  3. Insert the optode probes from the fNIRS system into the optode probe patches.
  4. Examine the parameters of fNIRS measurement (i.e., subject ID, the event-related mode, the optode probe arrangement).
  5. Set the experimental apparatus with one table, two chairs, two 19-in computer monitors, and two pairs of headphones (Figure 1C).

3. Participant arrival in the laboratory

NOTE: Sincerely appreciate the two participants of one dyad when they arrive at the fNIRS lab. Request them to put their phone on silent mode and temporarily leave their personal belongings in the cabinet. Then conduct the following processes in sequence:

  1. Before the participants sit down, reconfirm that the two participants have not seen each other before. Ensure that they did not communicate with each other through any language or movement while in the lab.
  2. Provide the participants with informed consent forms approved by the University Committee on Human Research.
  3. Instruct the participants on the details of the experimental task. Ask them to wear headphones and give them several practicing trials.
  4. In the practice trials, allow the two participants of each dyad to practice together.
    NOTE: Before wearing the fNIRS cap, it is worthwhile to measure and determine the head size for each participant by using a flexible rule. Then select the right size cap for the participant according to his/her head size. In this study, such a step was missed as one-size swimming caps were used. It is better to conduct this step on the day before the experiment because the relative operations (i.e., measuring head size, selecting the right size swimming cap, mounting the optode probe patches to the swimming cap, and inserting the optode probes into the probe patches) are time-consuming (about 20-30 min).
  5. Put the fNIRS cap on the head of participants with the center of the cap pointing at the location of CZ, and place the middle optode probe of the second probe row of the patch at FCz.
  6. Operate the fNIRS system to perform the signal calibration with the laser turned on.
  7. If there is an insufficient signal at some channel, adjust the signal intensity with a fiber stick to gently put the hair underneath the surrounding probe tip aside.
  8. If necessary, press the probes gently but ensure not to hurt the participants.
  9. Repeat steps 3.5-3.8 until the quality of the signal is accessible. Make sure that participants feel comfortable during the whole process of signal calibration.
  10. Help the participants to find a comfortable posture for themselves (e.g., comfortable body positions). Remind the participants to keep their heads as motionless as possible during the whole experimental task (i.e., about 26 min).
  11. Examine the quality of NIRS signals again. If there are sufficient signals in all channels, run the experiment procedure on the desktop computer.
  12. Help the participants take off the headphones and fNIRS cap on completion of the experimental procedure. Return their personal belongings and thank them with monetary compensation.
  13. Operate the fNIRS system to save data. Use a disc to export raw fNIRS data (.csv) and use a USB to copy the behavioral data from the computer.
  14. Close the fNIRS system and the computer if no more experimental arrangement.
  15. Keep the lab notebook ready to note down any events, especially abnormalities during the whole experiment.

4. Data analysis

NOTE: Perform all data analysis by using MATLAB software, with the following toolboxes: HOMER219, Hitachi2nirs20, xjView21, Cross Wavelet and Wavelet Coherence toolbox22, and Groppe's scripts in MathWork23.

  1. Data preprocessing
    1. To check the data quality, follow steps 4.1.2-4.1.3
    2. Read the data files (.csv) for each participant with the readHitachData function of xjView.
      NOTE: In this way, the Hitachi measurement data (csv format) is converted into oxyHb/deoxyHb/marker data with the information saved in the measurement (i.e., wavelength, timedata, and channel list).
    3. Visually check the quality in oxyHb and deoxyHb values by plotting all channels' time series in one figure, with the function plotTraces of xjView.
      NOTE: It is easy to identify abnormalities in the data. The channel that has much noise can be excluded in subsequent analysis.
    4. Convert Hitachi files (.csv) to .nirs files format with the csv2nirs function of Hitachi2nirs, which supports further data preprocessing with Homer2.
    5. Transform the raw data to optical density with the function hmrIntensity2OD of Homer2.
    6. Use principal component analyses (PCA)24 to remove the fNIRS global physiological noise by using the function enPCAFilter (nSV = 0.8, that is 80% of the covariance of the data was removed) of Homer2.
    7. Use correlation-based signal improvement method (CBSI)25 to remove head motion artifacts using the function hmrMotionCorrect_Cbsi of Homer2.
    8. Use modified Beer-Lembert law to transform the processed optical density into oxyHb and deoxyHb values with the hmrOD2Conc function of Homer2.
  2. Calculating IBS
    NOTE: For the preprocessed oxyHb values, use WTC to calculate the coherence values for the channel pair that are from the same location of the dyad, including the following pipeline:
    1. Adopt the wtc function of Cross Wavelet and Wavelet Coherence toolbox with default parameters to compute the coherence values at each time and frequency point to obtain a two-axis matrix of coherence values.
    2. For the default parameters, use morlet mother wavelet, to transform each time series into the time and frequency domain by the continuous wavelet transformation.
    3. Select MonteCarloCount to represent the number of surrogate data sets in the significance calculation, and select Auto AR1 to calculate the autocorrelation coefficients of the time series.
    4. Choose frequency band of interest (FOI) as mentioned in steps 4.2.5-4.2.8.
    5. Select and average the coherence values of the frequency band between 0.5-1 Hz (respectively corresponding to period 2 s and 1 s), according to the frequency band used in the finger-motion task of a previous fNIRS hyperscanning study9. Such FOI also corresponded to the period of one tap in the experimental task. Thus, obtain one column of coherence values for each pair.
      NOTE: To further statistically confirm FOI, calculate the coherence values for each dyad across the full frequency range (i.e., 0.008-10 Hz for the data), rather than just confining the selected frequency band (i.e., 0.5-1 Hz).
    6. Average the coherence values of the targeted time windows (same as 4.2.3) for each frequency point.
    7. Next, analyze the average coherence values following the pipeline described in steps 4.2.9-4.2.11 and subsequent statistics (i.e., 4.3.1 - 4.3.2) for each frequency point.
    8. Last, visually inspect FOI by plotting the statistical z values of each channel across frequency.
    9. Select and average the coherence values of the time window during the resting state (time window for 20 s-resting-state) and each experimental condition (i.e., meter coordination, non-meter coordination, meter independence, and non-meter independence), respectively, using the information of mark. Thus, obtain five coherence values for each dyad.
    10. For the task session, only select the duration during which participants tapped to reproduce the auditory stimulus, about 12 s for each trial, thus total 180 s (i.e., 12 s x 15 trials) for each experimental condition.
      NOTE: IBS was calculated as coherence increase (the larger subtracted coherence values than zero), namely the larger coherence values in the task session compared to those in the resting-state session.
    11. Subtract the resting coherence value from the task-related coherence value, respectively, in which the coherence value during the resting-state was used as a baseline in this experiment.
      NOTE: By repeating the above steps (4.2.1-4.2.11) across channels (i.e., 22 channels) and dyads (i.e., 16 dyads), the subtracted coherence values for each dyad at each channel were obtained finally.
  3. Statistics
    1. Compare the subtracted coherence values with zero at each channel for each experimental condition, using the paired samples permutation t-test with the mult_comp_perm_t1 function of Groppe's work (5000 permutations to estimate the distribution of the null hypothesis; desired family-wise alpha level- 0.05; two-tailed test, which means the alternative hypothesis is that the mean of the data is different from 0) as abnormal data distribution and limited sample size in the current experiment26.
      NOTE: The paired samples permutation t-test here is similar to paired t-test, but the latter assumes that the data is normally distributed, whereas the former does not. Such test begins the same way as paired t-test, that is, by computing a t score (i.e., real t score) for the coherence values in different groups (one is the subtracted coherence values in the task condition, the other is zeros). Then, a permutation is generated by exchanging the coherence values of different groups, and a new t score is calculated for the subtracted coherence values and zeros following this permutation. Such permutation is conducted 5000 times. Thus, 5000 t scores are obtained. In the distribution of the 5000 t scores, the relative location of the real t score generates the p-value for the subtracted coherence values.
    2. Correct the p values (i.e., due to the multiple comparison problem, and generate from the comparisons across 22 channels in one patch) by False Discovery Rate method (p < 0.05)27. Perform this correction via the mafdr function of MATLAB toolbox.
      NOTE: If the p-value at any channel was significant (i.e., p < 0.05) after FDR correction, there is IBS at that channel.
    3. Compare the coherence values between different task conditions at the channel where IBS existed, using the paired samples permutation t-test with the mult_comp_perm_t1 function of Groppe's work (same parameters as mentioned in step 4.3.1).
      NOTE: To intuitively examine the IBS during interpersonal coordination regarding meter vs. non-meter stimuli, compare the coherence values of different conditions directly (i.e., meter coordination vs. non-meter coordination; meter coordination vs. meter independence).
    4. Calculate the behavioral performance by the absolute difference between the partners' response time divided by the sum of both partners' responses56.
    5. Evaluate the relationship between the IBS and behavioral performance through the permutation test based on Pearson linear correlation analysis (i.e., the mult_comp_perm_corr function of Groppe's work).
  4. Validating IBS
    NOTE: To exclude the explanations that similar stimuli, motions, or conditions induced the demonstrated IBS, use a permutation test as a validation approach, with three permutations (i.e., within the dyad, between dyad, and between condition permutations), included the followings:
    1. Randomize the label of trials in the meter coordination condition (i.e., within dyad permutation, such as trial #1 and trial #13 in dyad #1) for one dyad at each channel via the randperm function of MATLAB.
    2. Follow the above pipeline of calculating IBS and statistics (i.e., sections 4.2 and 4.3, but excluding the sensitivity analysis for FOI) for the randomized trial label.
      NOTE: Calculate coherence values of the fake pair for each condition separately, and compute coherence increase for the fake pair (i.e., subtract the resting coherence value from the task-related coherence value for the fake pair).
    3. Conduct the permutation 1000 times, followed by the pipeline of calculating IBS and statistics (sections 4.2 and 4.3).
    4. Plot the distribution of statistical z values generated within dyad permutation.
    5. Conduct steps 4.4.2-4.4.4 by randomizing pairing of the participants of the same trial in the meter coordination condition (i.e., between dyad permutation, such as participant #1 in dyad #1 and participant #1 in dyad #3).
    6. Conduct steps 4.4.2-4.4.4 by randomizing the label of conditions for the same members of one dyad in the same trial (i.e., between condition permutation, such as participant #1 in the meter coordination condition and participant #2 in the meter independence condition).

Subscription Required. Please recommend JoVE to your librarian.

Representative Results

The results showed that there was IBS at channel 5 in the meter coordination condition, whereas no IBS existed in other conditions (i.e., meter independence, non-meter coordination, non-meter independence; Figure 2A). At channel 5, the IBS in the meter coordination condition was significantly higher than the coherence values in the non-meter coordination and meter independence condition (Figure 2B). Channel 5 approximately belonged to the left dorsolateral prefrontal cortex (DLPFC; Brodmann Area 9). Moreover, the permutation analysis showed that the observed IBS probably presented in two individuals of one dyad who tried to synchronize with each other in the matched time, but not in the time, partner, or condition of randomly pairing (Figure 2C). Together, these results indicated that music meter induced privileged IBS at DLPFC during interpersonal coordination. Considering the role of DLPFC in social interaction (e.g., modulating attention to other persons28,29) and music (e.g., enhancing cognitive performance in the presence of a musical background30,31), the observed DLPFC-IBS in the meter coordination condition might be related to drive more attention resource to the process involved in interpersonal coordination, such as perceiving and understanding the partner's task and movement.

Figure 1
Figure 1: Experimental design. (A) Experimental procedure and task. (B) Probe configuration. (C) Experimental setup. Please click here to view a larger version of this figure.

Figure 2
Figure 2: Inter-brain synchronization (IBS). (A) The heat maps of the permutation test on the coherence value for each condition. There was IBS at channel 5 in the meter coordination condition. (B) The IBS at channel 5 in the meter coordination condition was significantly greater than those in the meter independence and non-meter coordination condition. **p < 0.01, *p < 0.05. Error bars represent minimum/maximum values. The diamond dots denote extreme values. The shaded area indicates the 95% confidence interval. (C) The effect of IBS (statistical z values) with permutating trial, individual, and condition for all channels. The dashed line indicates the effect of the IBS at channel 5 in the meter coordination condition. The x-axis represents the Z value, and the y-axis represents the number of samples. Please click here to view a larger version of this figure.

Supplementary Audio 1. Please click here to download this File.

Supplementary Audio 2. Please click here to download this File.

Subscription Required. Please recommend JoVE to your librarian.

Discussion

This protocol provides a step-by-step procedure to calculate and validate IBS, using the fNIRS hyperscanning approach to simultaneously collect two participants' brain signals. Some critical issues involved in fNIRS data preprocessing, IBS calculation, statistics, and IBS validation are discussed below.

Data Preprocessing
It is necessary to preprocess fNIRS data in hyperscanning studies to extract real signals from the possible noise (i.e., motion artifacts, systemic components). Although the preprocess is skipped when analyzing IBS in earlier fNIRS hyperscanning studies10,32,33, it has been an essential and standard part in recent ones. In this study, both CBSI and PCA are used to remove noise; the former is reliable to remove head motion artifacts34, while the latter is good at decreasing the global physiological noise (e.g., respiratory, blood pressure, and blood flow variation)35. Of course, there are other motion correction methods for data preprocessing, which perform well in empirical fNIRS studies, such as wavelet filtering36, spline interpolation37, Kalman filtering38, autoregressive algorithms39, and short-channel separation correction40. The comparisons of motion correction methods reported that it is always better to correct motion artifacts than excluding channels or rejecting trials and that each method has emphasis particularly on. It has been proposed that adopting several motion correction methods simultaneously41, as shown in this study, is a realistic solution. In addition, low-pass and high-pass filtering are also usually used in fNIRS data preprocessing to remove physiological noise. Although this method is effective, it may destroy the task effect when the physiological noise and task effect occur in similar frequency bands42. Together, simultaneously using PCA and CBSI might be advisable for data preprocessing in fNIRS hyperscanning studies.

Calculate IBS
It has been proposed that more work is needed to standardize the IBS analysis steps and increase the reproducibility of IBS, as precise algorithms used to calculate IBS are variable across labs and studies43. In this work, the standard pipeline of calculating IBS through WTC is useful for researchers. There are several things needed to be careful. First, WTC commonly falls under the Morlet wavelet family, which is used in this study. However, it is proposed that a Complex Gaussian wavelet is more suitable for fNIRS data than a Morlet wavelet, as the former matches the waveform of the underlying signal (i.e., the multicycle signals rarely occur, especially for the signal of wavelengths around 10 to 20 s)44. More considerations should be directed to the wavelet coherence computations that affect the power of the analysis in subsequent applications for NIRS signals acquired during live social interactions. Second, to be consistent with previous findings of interpersonal coordination with music2,45,46 and music activities4,47,48, the coherence values were computed between the same channels in this study, while some studies have averaged the coherence values of all channels within the same brain region before statistical analysis49,50. In addition, the coherence values were calculated not only between the same channels/regions10,32,51 but also across different channels/regions52,53. These mentioned processes have enriched the pipeline of calculating IBS and might interest future directions of social interaction. Last but not least, only oxyHb values were analyzed in this study since oxyHb values are regarded as the most sensitive indicator of changes in the regional cerebral blood flow54. However, some researchers focused on deoxyHb changes, based on the findings that deoxyHb values are most closely related to the fMRI signal and independent of the global physiological noise55. Anyhow, the results might be more reliable if similar IBS effects are revealed in both oxyHb and deoxyHb changes. Therefore, the analysis of IBS on deoxyHb values is also necessary for future fNIRS hyperscanning studies.

Validate IBS
It is necessary to validate the revealed IBS, as the interpretation of IBS remains complex. For instance, IBS has been explained as a mechanism for information transmission, shared intentionality, behavioral alignment, similar perception, etc. It would help clarify the interpretation of IBS by performing null hypothesis testing with permutation, in which coherence values are either computed for the real dyads but randomly pairing trials or for fake dyads by randomly pairing participants within one condition/group or between conditions/groups16. In this study, permutation was performed by simply conducting a very large number of resamples (i.e., 1000 times). In contrast, coherence values can be calculated for all possible random pairs56. In addition, the above permutation test can be used to generate a null distribution of coherences from all possible coherences in the experiment, to see whether the observed IBS are near the top end of this distribution, which has been commonly used in studies that adopt real-life stimuli and experimental environment57,58. This analysis ensures that the IBS is real-interaction-specific at the sequence level, as the coherence values during matching ones (i.e., trials, individuals, and conditions) must on average statistically exceed an equal-sized random draw of coherences within or between dyads. Such a method is different from the baseline used in the current work (i.e., the resting-state coherence values), which is in line with traditional General Linear Models designs and is selected to compare the current results with the findings in previous studies. It should be noted that the 20-s-resting baseline in this study is shorter than the widely used duration (30 s or more than 1 min), which is used to restrict the total time of the experiment to 30 min to ensure the comfort of participants.

In conclusion, this article provides a specific pipeline of analyzing IBS in fNIRS hyperscanning studies. Such pipeline is a potentially standard data processing approach in the field, which will contribute to both the reproducibility and reliability of IBS. In the future, the details of data processing should be further refined when analyzing IBS for particular groups (i.e., parent-infant, children, and schizophrenia patients) and particular contexts (i.e., nonverbal or verbal communication and teaching situations). Finally, showcasing the protocol of analyzing the inter-brain network for larger groups of participants in natural interactions will benefit the quantification of social interaction.

Subscription Required. Please recommend JoVE to your librarian.

Disclosures

The authors have nothing to disclose.

Acknowledgments

This research was supported by: National Natural Science Foundation of China (31872783, 31800951).

Materials

Name Company Catalog Number Comments
Computer Hewlett-Packard Development Company, L.P. HP S01-pF157mcn
Earphone Royal Philips Electronics, Eindhoven, The Netherlands SHE2405BK/00
EEG cap Compumedics Neuroscan, Charlotte, USA 64-channel Quik-Cap
E-Prime software Psychology Software Tools, Inc., Pittsburgh, USA E-Prime 3
fNIRS system Hitachi Medical Corporation, Tokyo, Japan ETG-7100 Optical Topography System
MATLAB 2014b The MathWorks, Inc., Natick, MA MATLAB 2014b
MuseScore Musescore Company, Belgium MuseScore 3.6.2.548021803
Swimming cap Decathlon Group, Villeneuve-d'Ascq, France 1681552

DOWNLOAD MATERIALS LIST

References

  1. Kingsbury, L., Hong, W. A Multi-Brain Framework for Social Interaction. Trends in Neurosciences. 43 (9), 651-666 (2020).
  2. Konvalinka, I., Bauer, M., Stahlhut, C., Hansen, L. K., Roepstorff, A., Frith, C. D. Frontal alpha oscillations distinguish leaders from followers: multivariate decoding of mutually interacting brains. NeuroImage. 94, 79-88 (2014).
  3. Ikeda, S., et al. Steady Beat Sound Facilitates both Coordinated Group Walking and Inter-Subject Neural Synchrony. Frontiers in Human Neuroscience. 11, 147 (2017).
  4. Duan, L., et al. Cluster imaging of multi-brain networks (CIMBN): a general framework for hyperscanning and modeling a group of interacting brains. Frontiers in Neuroscience. 9, 267 (2015).
  5. Sanger, J., Muller, V., Lindenberger, U. Intra- and interbrain synchronization and network properties when playing guitar in duets. Frontiers in Human Neuroscience. 6, 312 (2012).
  6. Muller, V., Delius, J. A. M., Lindenberger, U. Hyper-frequency network topology changes during choral singing. Frontiers in Physiology. 10, 207 (2019).
  7. Egetemeir, J., Stenneken, P., Koehler, S., Fallgatter, A. J., Herrmann, M. J. Exploring the neural basis of real-life joint action: Measuring brain activation during joint table setting with functional near-infrared spectroscopy. Frontiers in Human Neuroscience. 5, 95 (2011).
  8. Grinsted, A., Moore, J. C., Jevrejeva, S. Application of the cross wavelet transform and wavelet coherence to geophysical time series. Nonlinear Processes in Geophysics. 11 (5-6), 561-566 (2004).
  9. Holper, L., Scholkmann, F., Wolf, M. Between-brain connectivity during imitation measured by fNIRS. NeuroImage. 63 (1), 212-222 (2012).
  10. Cui, X., Bryant, D. M., Reiss, A. L. NIRS-based hyperscanning reveals increased interpersonal coherence in superior frontal cortex during cooperation. NeuroImage. 59 (3), 2430-2437 (2012).
  11. Jiang, J., Dai, B., Peng, D., Zhu, C., Liu, L., Lu, C. Neural synchronization during face-to-face communication. Journal of Neuroscience. 32 (45), 16064-16069 (2012).
  12. Tang, H., Mai, X., Wang, S., Zhu, C., Krueger, F., Liu, C. Interpersonal brain synchronization in the right temporo-parietal junction during face-to-face economic exchange. Social Cognitive and Affective Neuroscience. 11 (1), 23-32 (2016).
  13. Pan, Y., Novembre, G., Song, B., Li, X., Hu, Y. Interpersonal synchronization of inferior frontal cortices tracks social interactive learning of a song. NeuroImage. 183, 280-290 (2018).
  14. Konvalinka, I., Roepstorff, A. The two-brain approach: how can mutually interacting brains teach us something about social interaction. Frontiers in Human Neuroscience. 6, 215 (2012).
  15. Karlsson, A. Permutation, parametric, and bootstrap tests of hypotheses. Journal of the Royal Statistical Society Series a-Statistics in Society. 169, 171 (2006).
  16. Ayrolles, A., et al. HyPyP: a Hyperscanning python pipeline for inter-brain connectivity analysis. Social Cognitive and Affective Neuroscience. 16 (1-2), 72-83 (2021).
  17. Konvalinka, I., Vuust, P., Roepstorff, A., Frith, C. D. Follow you, follow me: continuous mutual prediction and adaptation in joint tapping. Quarterly Journal of Experimental Psychology. 63 (11), 2220-2230 (2010).
  18. Majolo, B., et al. Human friendship favours cooperation in the iterated prisoner's dilemma. Behaviour. 143, 1383-1395 (2006).
  19. Homer2. , Available from: https://www.nitrc.org/projects/hitachi2nirs (2021).
  20. Hitachi2nirs. , Available from: https://www.nitrc.org/projects/hitachi2nirs (2021).
  21. xjview. , Available from: https://www.alivelearn.net/xjview/ (2021).
  22. Cross Wavelet and Wavelet Coherence toolbox. , Available from: http://grinsted.github.io/wavelet-coherence/ (2021).
  23. Groppe's scripts in MathWork. , Available from: https://uk.mathworks.com/matlabcentral/profile/authors/1948879 (2021).
  24. Zhang, Y., Brooks, D. H., Franceschini, M. A., Boas, D. A. Eigenvector-based spatial filtering for reduction of physiological interference in diffuse optical imaging. Journal of Biomedical Optics. 10 (1), 011014 (2005).
  25. Cui, X., Bray, S., Reiss, A. L. Functional near infrared spectroscopy (NIRS) signal improvement based on negative correlation between oxygenated and deoxygenated hemoglobin dynamics. NeuroImage. 49 (4), 3039-3046 (2010).
  26. Lumley, T., Diehr, P., Emerson, S., Chen, L. The importance of the normality assumption in large public health data sets. Annual Review of Public Health. 23 (1), 151-169 (2002).
  27. Benjamini, Y., Yekutieli, D. The control of the false discovery rate in multiple testing under dependency. Annals of Statistics. 29 (4), 1165-1188 (2001).
  28. Miller, B. L., Cummings, J. L. The human frontal lobes: Functions and disorders. , Guilford Press. New York. (2007).
  29. vanden Bos, W., van Dijk, E., Westenberg, M., Rombouts, S. A. R. B., Crone, E. A. What motivates repayment? Neural correlates of reciprocity in the Trust Game. Social Cognitive and Affective Neuroscience. 4 (3), 294-304 (2009).
  30. Corbetta, M., Shulman, G. L. Control of goal-directed and stimulus-driven attention in the brain. Nature Reviews Neuroscience. 3 (3), 201-215 (2002).
  31. Ferreri, L., Aucouturier, J. J., Muthalib, M., Bigand, E., Bugaiska, A. Music improves verbal memory encoding while decreasing prefrontal cortex activity: an fNIRS study. Frontiers in Human Neuroscience. 7, 779 (2013).
  32. Cheng, X., Li, X., Hu, Y. Synchronous brain activity during cooperative exchange depends on gender of partner: A fNIRS-based hyperscanning study. Human Brain Mapping. 36 (6), 2039-2048 (2015).
  33. Hu, Y., Hu, Y., Li, X., Pan, Y., Cheng, X. Brain-to-brain synchronization across two persons predicts mutual prosociality. Social Cognitive and Affective Neuroscience. 12 (12), 1835-1844 (2017).
  34. Delgado Reyes, L. M., Bohache, K., Wijeakumar, S., Spencer, J. P. Evaluating motion processing algorithms for use with functional near-infrared spectroscopy data from young children. Neurophotonics. 5 (2), 025008 (2018).
  35. Zhang, Y., Brooks, D. H., Franceschini, M. A., Boas, D. A. Eigenvector-based spatial filtering for reduction of physiological interference in diffuse optical imaging. Journal of Biomedical Optics. 10 (1), 11014 (2005).
  36. Molavi, B., Dumont, G. A. Wavelet-based motion artifact removal for functional near-infrared spectroscopy. Physiological Measurement. 33 (2), 259-270 (2012).
  37. Scholkmann, F., Spichtig, S., Muehlemann, T., Wolf, M. How to detect and reduce movement artifacts in near-infrared imaging using moving standard deviation and spline interpolation. Physiological Measurement. 31 (5), 649 (2010).
  38. Izzetoglu, M., Chitrapu, P., Bunce, S., Onaral, B. Motion artifact cancellation in NIR spectroscopy using discrete Kalman filtering. Biomedical Engineering Online. 9 (1), (2010).
  39. Barker, J. W., Aarabi, A., Huppert, T. J. Autoregressive model based algorithm for correcting motion and serially correlated errors in fNIRS. Biomedical Optics Express. 4 (8), 1366-1379 (2013).
  40. Gagnon, L., et al. Short separation channel location impacts the performance of short channel regression in NIRS. NeuroImage. 59 (3), 2518 (2012).
  41. Di Lorenzo, R., et al. Brain responses to faces and facial expressions in 5-month-olds: An fNIRS study. Frontiers in Psychology. 10, 1240 (2019).
  42. Duan, L., et al. Wavelet-based method for removing global physiological noise in functional near-infrared spectroscopy. Biomed Opt Express. 9 (8), 3805-3820 (2018).
  43. Hamilton, A. Hype, hyperscanning and embodied social neuroscience. PsyArXiv. , (2020).
  44. Zhang, X., Noah, J. A., Dravida, S., Hirsch, J. Optimization of wavelet coherence analysis as a measure of neural synchrony during hyperscanning using functional near-infrared spectroscopy. Neurophotonics. 7 (1), 015010 (2020).
  45. Ikeda, S., et al. Steady beat sound facilitates both coordinated group walking and inter-subject neural synchrony. Frontiers in Human Neuroscience. 11, 147 (2017).
  46. Osaka, N., et al. How two brains make one synchronized mind in the inferior frontal cortex: fNIRS-based hyperscanning during cooperative singing. Frontiers in Psychology. 6, 1811 (2015).
  47. Abrams, D. A., et al. Inter-subject synchronization of brain responses during natural music listening. European Journal of Neuroscience. 37 (9), 1458-1469 (2013).
  48. Hou, Y., Song, B., Hu, Y., Pan, Y., Hu, Y. The averaged inter-brain coherence between the audience and a violinist predicts the popularity of violin performance. NeuroImage. 211, 116655 (2020).
  49. Baker, J. M., et al. Sex differences in neural and behavioral signatures of cooperation revealed by fNIRS hyperscanning. Scientific Reports. 6, 26492 (2016).
  50. Kruppa, J. A., et al. Brain and motor synchrony in children and adolescents with ASD-a fNIRS hyperscanning study. Social Cognitive and Affective Neuroscience. 16 (1-2), 103-116 (2021).
  51. Liu, T., Duan, L., Dai, R., Pelowski, M., Zhu, C. Team-work, Team-brain: Exploring synchrony and team interdependence in a nine-person drumming task via multiparticipant hyperscanning and inter-brain network topology with fNIRS. NeuroImage. 237, 118147 (2021).
  52. Dai, B., et al. Neural mechanisms for selectively tuning in to the target speaker in a naturalistic noisy situation. Nature Communications. 9 (1), 2405 (2018).
  53. Li, R., Mayseless, N., Balters, S., Reiss, A. L. Dynamic inter-brain synchrony in real-life inter-personal cooperation: A functional near-infrared spectroscopy hyperscanning study. NeuroImage. 238, 118263 (2021).
  54. Boas, D. A., Dale, A. M., Franceschini, M. A. Diffuse optical imaging of brain activation: approaches to optimizing image sensitivity, resolution, and accuracy. NeuroImage. 23, 275-288 (2004).
  55. Ferrari, M., Quaresima, V. A brief review on the history of human functional near-infrared spectroscopy (fNIRS) development and fields of application. NeuroImage. 63 (2), 921-935 (2012).
  56. Mu, Y., Guo, C., Han, S. Oxytocin enhances inter-brain synchrony during social coordination in male adults. Social Cognitive and Affective Neuroscience. 11 (12), 1882-1893 (2016).
  57. Chen, J., et al. Shared memories reveal shared structure in neural activity across individuals. Nature Neuroscience. 20 (1), 115-125 (2017).
  58. Regev, M., et al. Propagation of Information Along the Cortical Hierarchy as a Function of Attention While Reading and Listening to Stories. Cerebral Cortex. 29 (10), 4017-4034 (2019).

Tags

Inter-brain Synchronization FNIRS Hyperscanning Study Calculate Validate Protocol Analysis Pipeline Coherent Method Permutation-based Chest Conditions Participants Data Pre-processing Principle Component Analysis Correlation-based Signal Improvement Method NPCA Filter Head Motion Artifacts HMR Motion Correct Underscore CBSI Wavelet Transform Coherent Function Cross Wavelet And Wavelet Coherent Toolbox Default Parameters Morlet Mother Wavelet Time Series Frequency Domain Continuous Wavelet Transformation Frequency Band
How to Calculate and Validate Inter-brain Synchronization in a fNIRS Hyperscanning Study
Play Video
PDF DOI DOWNLOAD MATERIALS LIST

Cite this Article

Hu, Y., Wang, Z., Song, B., Pan, Y., … More

Hu, Y., Wang, Z., Song, B., Pan, Y., Cheng, X., Zhu, Y., Hu, Y. How to Calculate and Validate Inter-brain Synchronization in a fNIRS Hyperscanning Study. J. Vis. Exp. (175), e62801, doi:10.3791/62801 (2021).

Less
Copy Citation Download Citation Reprints and Permissions
View Video

Get cutting-edge science videos from JoVE sent straight to your inbox every month.

Waiting X
Simple Hit Counter