Measuring Neural and Behavioral Activity During Ongoing Computerized Social Interactions: An Examination of Event-Related Brain Potentials

Summary

Research on social exclusion has grown tremendously in recent years. As the field expands, it is imperative to develop sophisticated methodologies allowing for the simultaneous measurement of neural and behavioral outcomes during social exclusion. This protocol utilizes event-related brain potentials to record ongoing neural activity during computerized social interactions.

Abstract

Social exclusion is a complex social phenomenon with powerful negative consequences. Given the impact of social exclusion on mental and emotional health, an understanding of how perceptions of social exclusion develop over the course of a social interaction is important for advancing treatments aimed at lessening the harmful costs of being excluded. To date, most scientific examinations of social exclusion have looked at exclusion after a social interaction has been completed. While this has been very helpful in developing an understanding of what happens to a person following exclusion, it has not helped to clarify the moment-to-moment dynamics of the process of social exclusion. Accordingly, the current protocol was developed to obtain an improved understanding of social exclusion by examining the patterns of event-related brain activation that are present during social interactions. This protocol allows greater precision and sensitivity in detailing the social processes that lead people to feel as though they have been excluded from a social interaction. Importantly, the current protocol can be adapted to include research projects that vary the nature of exclusionary social interactions by altering how frequently participants are included, how long the periods of exclusion will last in each interaction, and when exclusion will take place during the social interactions. Further, the current protocol can be used to examine variables and constructs beyond those related to social exclusion. This capability to address a variety of applications across psychology by obtaining both neural and behavioral data during ongoing social interactions suggests the present protocol could be at the core of a developing area of scientific inquiry related to social interactions.

Introduction

The scientific examination of social interactions has undergone a renaissance in recent years, with an explosion of new theoretical explanations, models, and paradigms aimed at understanding and exploring the effects of being the target or source of social exclusion and how those interactions lead to the many consequences of exclusion^1-6. Though the literature had made tremendous strides in developing a better understanding of the consequences of social exclusion at behavioral, emotional, cognitive, and neural levels, a great deal remains unknown in relation to the dynamics involved in social exclusion. One notable gap in the literature relates to the measurement of various dynamic social exclusion processes during social interactions. For instance, multiple theoretical models^3,5-8 suggest that the monitoring and assessment of instances of social exclusion is an initial step in a larger self-regulatory system aimed at coping with social exclusion and maintaining healthy and acceptable levels of belonging and social inclusion. These models, and much of the existing literature on exclusion, provide tremendous insights into the consequences of social exclusion and the harmful effects exclusion causes on neural, behavioral, cognitive, and emotional levels. However, the specific processes ongoing in targets of exclusion during social interactions, which lead to both the perception of exclusion and the subsequent emotional and cognitive reactions to exclusion, remain undefined. Researchers have adapted methodologies to obtain self-reported feeling states during social interactions⁹, but these data did not examine the ongoing neural processes that may motivate any self-reported effects.

Accordingly, examinations of exclusion during social interactions were initiated using functional magnetic resonance imaging (fMRI) to “see” what is happening while individuals are being excluded^3,4,10,11. These studies revealed different patterns of neural activation during exclusion compared to inclusion. Though tremendously important in enhancing the understanding of ongoing neural processes present during exclusion and their relations with the self-reported consequences of being excluded, these studies are limited in how they can represent the dynamic nature of social interactions. Specifically, these fMRI methodologies aggregated neural activity across entire social interactions and were unable to examine exclusion on a moment-to-moment basis. This limitation prohibits a complete understanding of the dynamic nature of exclusion-related emotional and cognitive processing that is taking place during social interactions as researchers are unable to determine which moments or events during an exchange are meaningful in relation to the development of one’s perceptions of exclusion and the associated emotional response.

To address these limitations, recent research has implemented the measurement of one class of neural activity, known as event-related brain potentials (ERPs), during the execution of the Cyberball paradigm¹² to examine the moment-to-moment patterns of neural activation present during social exclusion¹³. ERPs refer to neuroelectric activity measured on the scalp that is time-locked to discrete events and represents brain activity in response to or in preparation for a stimulus or response¹⁴. Further, ERPs possess a superior temporal resolution when compared to fMRI, which provides valuable insights into the dynamic responses to social exclusion. As such, neural indices obtained through the event-related examination of brain activity in response to instances of social inclusion and exclusion, which can be implemented and controlled through the Cyberball paradigm and are described in the present protocol, are necessary to evaluate the models and predictions present in current social exclusion theory.

The goal of the current methodology is to measure ongoing neural responses to social events (inclusionary events, exclusionary events) during computerized social interactions in a human participant. In this methodology, researchers have the ability to quantify neural activity in response to each event within the interaction. Further, the current protocol allows for the ongoing examination of each social event as each event is made up of multiple throw images. This allows researchers to look at changes in neural activity as the events unfold. This level of analysis is not available in other methodologies that examine ERPs during social interactions^15,16 as these methodologies only capture neural activity in relation to one image for each event without allowing for the examination of the unfolding event as it occurs. Additionally, the human participant is led to believe that he or she is playing an online game with other people, but is actually playing within a pre-programmed game with a computer. Because the interaction is pre-programmed within the computer, with the flexibility to interact with the decisions made by the human participant, the nature of the social interactions can be pre-determined and programmed to vary depending on the nature of the research question^13,17. For example, the behavior of the computer-generated players during the protocol can be tailored to create instances of social inclusion or social exclusion of any specified duration by altering the pre-programmed schedule of throws (e.g., which player throws the ball to which other player, when those throws occur, the number of throws, and the timing of throws). Accordingly, this allows researchers to measure neural activity in response to events that may or may not match the overall context of the interaction. For example, researchers can quantify a participant’s neural response to an exclusionary social event within an interaction that is largely inclusionary for that participant and potentially compare it to that participant’s neural response to an exclusionary event within a largely exclusionary interaction. These research opportunities are not readily available using fMRI technology given the temporal limitations of fMRI. With this programming flexibility, the current protocol allows researchers from various neuroscientific and psychological backgrounds to address research questions in new ways and obtain dynamic neural and behavioral activity during social interactions.

Protocol

NOTE: The following protocol was developed in accordance with ethical standards approved by the Institutional Review Board at Illinois Wesleyan University.

1. Cyberball Stimulus Preparation

1. Download the Cyberball paradigm^12,18 and install it on the computer (the current protocol utilized images from Cyberball version 3.0). Alternatively, create computerized images to recreate the Cyberball paradigm to meet specific needs.
2. Create individual images for each portion of the throws within Cyberball by using a photo-editing program. For example, break down each of the throws from player to player into the individual throw frames that are shown one after another to create the image of a ball being thrown from player to player on the computer screen (see Figure 1).
3. Add any labels, names, or pictures to each individual throw frame in the photo-editing program, including anything to represent the human participant as the bottom player on the screen (represented by the hand at the bottom of the screen in Figure 1) to create a series of throw frames that are identical except for the movement of the ball from player to player.
4. Note which frame in each throw sequence is the “informational frame” for that throw, or the first frame within the throw sequences that provides information to the players about the specific destination of the throw (i.e., which other player will receive the ball).
5. Ensure that there are throw sequences creating a throw from each player to each other player on the screen (including throws from the human participant to the other players), that each throw sequence has the same number of throw frames, and the informational frame within each throw sequence has been noted.

2. Cyberball Social Interaction Programming

1. Create a sequence file using stimulus presentation software to detail the exact sequence of events within the Cyberball social interaction.
   1. For the sequence file, specify the specific throw frames (in order), the timing of the frames on the screen, the sequencing of the frames, the nature of the event (throw from whom to whom), the response required by the human participant (when necessary), and the overall order of events to create the desired interaction. Explicitly enter all of these specifications into the proper rows, columns, and spaces within the programming code during the creation of the sequence file.
   2. Specify all of the above-mentioned specifics within the programming code for each event within the sequence file and repeat the steps for each sequence file created (e.g., inclusion, exclusion).
      1. Order each of the throw frames in the correct sequence within the sequence file so that the first ball toss is completed without error from one player to the other. Create similar ordered sequences in the file for each type of throw among the players so that each type of throw is represented in the sequence file (e.g., a three-player game consists of six different possible throws).
      2. Space the timing of each throw frame 450 msec apart. In this method, ensure that each frame appears for 450 msec before being replaced by the next frame, which provides an image of motion on the screen for the participant and creates a throw event that lasts a total of 2,700 msec.
      3. Insert an event-related marker each time an informational frame is presented in the sequence file so that the presentation of the informational frame can be marked in time in the file saving the participants’ neural activity. Code this marker to represent the nature of the event by using numbers to represent the players (left player is player “1,” the bottom player is player “2,” the right player is player “3”), which would allow the code “13” to represent a throw from the player on the left to the player on the right.
      4. Copy the entire set of six different throw sequences within the file so that each throw sequence is represented at least twice within the sequence file. This will provide programming flexibility to change the order of events within each block so that they do not look pre-determined.
      5. Create “if, then” statements within the sequence file to allow the human participant to freely select which player will receive the next throw following the human participant. Give the human participant a response pad or mouse to select the next action after receiving a ball toss; potentially using the right mouse button to throw to the player on the right and using the left mouse button to throw to the player on the left in a three-player game.
      6. Ensure that the “if, then” statements lead to the appropriate next throw sequence so that the game play appears seamless (i.e., a human throw to the player on the left should be followed by a throw from the player on the left to another player).
      7. Create loops and “if, then” statements within the sequence file to represent the desired game action and allow the program to appropriately move to the next event regardless of the selections of the human participant.
      8. Initiate counters within the program to change the nature of the game so that the program does not become apparent to the human player (i.e., the same computerized player does not always make the same throw). Use these counters to switch the game action and remove patterns of play throughout the game after the repeated occurrence of a specific event or pattern of events to better give the appearance of spontaneous live play among players on the sides of the screen, not just the actual human participant represented on the bottom of the screen.
2. Develop different sequence files in order to study different types of social interactions. Make these interactions largely inclusive or exclusive, or even partially inclusive or exclusive, for the human participant depending on the nature of the research question by varying the proportion and order of inclusionary events and exclusionary events within each sequence file.
3. Ensure that the event markers appear in the EEG files when collecting neural data to create event-related brain potentials (ERPs) for each event type within each of the different social interactions. These markers should appear in the EEG file as the informational frame is presented to the participant.

3. Neuroelectric Recording

1. Prepare participants for electroencephalography (EEG) assessment in accordance with the guidelines of the Society for Psychophysiological Research¹⁹.
2. Use a lycra electrode cap embedded with 64 sintered Ag-AgCl electrodes (10 mm), arranged in a 10–10 system montage²⁰ to collect EEG data. Fit the cap on the participant’s head and prepare each electrode using conductive gel.
   1. Reference the electrodes online to an electrode placed at the midpoint between Cz and CPz and use AFz as the ground electrode.
      NOTE: Alternative online references may be needed depending upon the nature of the electrode cap used for data collection.
   2. Collect vertical and horizontal bipolar electrooculographic activity (EOG) to monitor eye movements using sintered Ag-AgCl electrodes placed above and below the right orbit and near the outer canthus of each eye.
3. Use a digital bioamplifier to continuously digitize (500 Hz sampling rate), amplify (gain depends on the specific amplifier), and filter (70 Hz low-pass filter, including a 60 Hz notch filter) the raw EEG signal in DC mode. Choose these setting from the available options in the EEG analysis software for the amplifier prior to data collection and vary depending on the specifications of the EEG hardware and software.
4. Record EEG activity using EEG analysis software in order to further process the neural data.

4. Offline Neuroelectric Data Processing

1. Correct eye blinks using a spatial filter, a multi-step procedure that generates an average eye blink, utilizes a spatial singular value decomposition based on principal component analysis (PCA) to extract the first component and covariance values, and then uses those covariance values to develop a filter that is specifically sensitive to eye blinks²¹.
2. Create stimulus-locked epochs relative to the event marker that was inserted in the continuous EEG file in the EEG analysis software by selecting this function from the choices of data transformation options. Run these epochs from -900 msec to 1,800 msec relative to inserted marker, which is equivalent to the entire duration of each six-frame throw and has a timepoint of 0 msec where the event marker was inserted as displayed in Figure 1.
3. Correct for baseline difference between the epochs by removing the average pre-stimulus baseline activity from each epoch (i.e., the 900 msec time window that runs from -900 msec to 0 msec prior to the event marker). This function can be selected or initiated from the data transformation options available in the EEG analysis software.
4. Low-pass filter (30 Hz; 24 dB/octave) the epochs and reject any epochs with electrical artifacts that exceed +75V. Choose these setting from the available options in the EEG analysis software allowing the transformation of EEG data following data collection and vary depending on the specifications of the EEG software.
5. Average the neural responses together for each event type within the Cyberball task blocks.
   NOTE: This averaging process can be adapted to only average the first 20, middle 20, or even last 20 events of a similar type within each interaction to examine dynamic patterns of neural activation over the course of the social interaction¹³.
   1. Combine the various events types to create three major event categories: throws to the participant from either other player, throws from the participant to either other player, and throws not including the participant between the two other players. For example, combine throws from the human to the left player and to the right player into one average waveform.
   2. Combine the events from the computerized players into the event types of most interest: throws to the human participant (inclusionary) and throws away from the human participant (exclusionary).
6. If applicable, quantify the N2 component as the average amplitude in the discrete latency window running from 200 - 320 msec after the event marker at FCz.
7. If applicable, quantify the P3 component as the average amplitude in the discrete latency window running from 320 - 450 msec following the event marker at Pz.
8. If applicable, quantify ERP components to throw frames following the informational frame to examine ongoing differences among patterns of neural activity to different event types with the social interaction.

Representative Results

This protocol has been used in previously published research examining the influence of social exclusion on ongoing neural and behavioral activity¹³. Twenty-two college-aged participants (15 females, 7 males) completed three sessions of the Cyberball task under conditions described above. After providing informed consent, participants were told that they would be playing a computerized ball-tossing game with other undergraduate participants. However, the other participants were not real, they were represented by the computerized players detailed in this protocol. Every human participant completed the same three blocks of the protocol (inclusion, exclusion, re-inclusion). Each block consisted of 80 total throws. In the inclusion and re-inclusion blocks, all players had an equal chance of receiving the ball on each ball toss. In the exclusion block, the human participant had the same equal chance of receiving the ball until receiving 10 throws from the other players. After this initial phase, the human participant was completed excluded for the remainder of the task block.

Representative results from this protocol can include examinations of multiple ERP components for each type of event within a social interaction as well as an examination of ERP components across different types of interactions. Analyses of the N2 component indicate an effect for the type of event, but no effect for the type of social interaction, with larger N2 amplitudes for exclusionary throws regardless of the larger context of the social interaction. Representative findings for the P3 component reveal a similar pattern with an effect for the type of event within the interaction, but not for the type of interaction itself, with larger P3 amplitude for inclusionary events and no overall effects for the nature of the social interaction. Figure 2 provides ERP waveforms by Cyberball block and throw type, highlighting the observed differences in N2 and P3 amplitudes.

Additionally, by utilizing ERPs, this protocol allows for the examination of potential alterations in neural activation over the course of social interactions. Representative analyses can be conducted to examine changes in neural activation to exclusionary events over the course of the entire exclusion process. In examinations of early exclusionary trials compared to later exclusionary trials, analyses of both the N2 and P3 have indicated larger amplitudes for both ERP components during the first 20 exclusionary events following the initial inclusion phase compared to the second 20 exclusionary events following the initial inclusionary phase of the exclusion block (see Figure 3).

Figure 1. Cyberball throw sequence examples with event marker placement. Examples of throw frames along with the placement of ERP markers during different throw sequences in the ongoing Cyberball game. Event markers are inserted as the first informational frame providing information about the nature of each throw is presented to the participant. Please click here to view a larger version of this figure.

Figure 2. Representative ERP waveforms by throw type and block type. This protocol is capable of providing ERP waveforms for each type of social event within each task block of Cyberball. The different patterns of neural activity to each event type can be represented by different waveforms within the same figure, with separate lines for each type of throw (inclusionary, exclusionary) for each block of Cyberball (inclusion, exclusion, re-inclusion). The time point 0 msec represents the timing of the ERP event marker within each throw sequence, with the top graph displaying waveforms at FCz and the bottom graph displaying waveforms at Pz. This figure has been modified from Themanson et al.¹³ with permission.

Figure 3. Representative ERP waveforms displaying component differences over the course of social exclusion. ERP waveforms derived from this protocol parsing the first 20 and second 20 exclusionary events following the initial inclusionary phase of the exclusion block. This capability to show the alterations in neural activity during the course of the social interaction can be applied to different ERP components and electrode sites, as shown by the waveforms for FCz (top) and Pz (bottom). This figure has been modified from Themanson et al.¹³ with permission.

Discussion

In this article, a protocol allowing for the measurement of ongoing event-related neural and behavioral data during social interactions was presented. This procedure creates opportunities to look at multiple different event types (inclusionary, exclusionary) within and across varied social interactions. Specifically, the procedure can quantify moment-to-moment event-related neural activity in response to any event that occurs during a computerized social interaction. This neural activity is independent of any particular movement or action initiated by the participant and the nature of the social interactions can be customized to address a multitude of research questions beyond social exclusion research.

Critical to this protocol is the use of event markers for the collection of neural activity during ongoing social interactions. The insertion of these event markers allows for time-locked analyses of event-related activity within any type of social interaction (inclusionary, exclusionary). This capability is not possible in fMRI methodologies^3,11 due to the limited temporal resolution of hemodynamic measures. An additional critical element is the creation of a series of images reflecting an ongoing and dynamic social interaction among players. This differs from other ERP protocols that present a single image showing the human participant that he/she was the recipient of the throw^15,16,22. In the present protocol, the participants can experience the development of a social event – just like in real-life interactions – and researchers have the capability to examine each frame throughout each throw sequence. This allows for the parsing of neural responses into a component structure, rather than seeing social events as some aggregate occurrence.

Further, because of the open-ended capability to explicitly program the behavior of the computer-generated players within the Cyberball interaction, the researcher has the ultimate flexibility to generate any social interaction for the human participant. This easily allows investigators to examine issues beyond social exclusion research. For example, this protocol could be adapted to address questions pertaining to a number of topics in psychology, across multiple sub-disciplines in the field by varying the nature of the “back-story” surrounding the protocol and the conditions of play within the protocol itself. There is no limit on the number of players or the structure to the game. Accordingly, a variety of conditions or controls could be introduced to assess conformity, social ethics, attentional control, information processing, aggression, cooperation and competition, racial or gender bias, and a host of additional constructs and variables of import to researchers examining human behavior. Additionally, researcher could adapt the present protocol to utilize ERP methodologies in chat room research paradigms²³ or other “online” social interactions²⁴. These paradigms may have more external validity in comparison to Cyberball and the ability to examine ERPs during these chat room interactions, by creating event markers during the presentation of each chat room entry, would allow for the ongoing examination of dynamic neural activity during these more realistic social exchanges while still maintaining a significant level of control over the experimental protocol. This would add to the breadth of understanding related to the process of social exclusion and using these chat room paradigms would allow for much more flexible applications of these ERP techniques to social exclusion research.

Although this protocol has been shown to be effective in obtaining neural and behavioral data from individuals during ongoing social interactions, it is important to mention the limitations of the procedure. Notably, the nature of ERP data requires the averaging of many event trials together in order to improve the signal-to-noise ratio within the data²⁵. Because of this requirement inherent within ERP research, the social interactions in the current protocol can become long and monotonous for the participants as each task block needs to be programmed to include a large number of trials for each type of event. Further, depending on the software program used to implement the presentation of the protocol, the protocol may be dependent upon the human participant’s response within a certain time-window after receiving the ball. If the participant does not response within the programmed timeframe, the software closes the sequence file, which means that the task block has to be restarted. This can lead to differences in how the participants in the same task condition complete the protocol as some participants will be exposed to a longer protocol due to the restart. Finally, neural data by itself is not sufficient for a complete examination of social exclusion processes. The ERP components described in this protocol could be active in response to some other aspect of the social interaction, or other stimuli entirely, depending on the nature of the social exchange and the setting where the participant is completing the research. Accordingly, the neural data need to be linked with self-report questionnaire data to ensure the functional significance of the neural data by showing that the changes in the ERP components are related to social exclusion and not some other processes or stimuli.

The current protocol offers new empirical insights into the dynamic nature of social interactions by exploring the event-related patterns of neural activation present during ongoing social interactions. Future research using this protocol would be well-served to utilize a variety of software platforms and manipulations of the social interactions to more completely assess the patterns of neural activity present during social interactions. Additionally, future investigations can build upon this protocol by adding layers of complexity to the study designs to investigate individual differences, social influences, and contextual effects to uncover important variables and characteristics that may moderate the exclusion-related effects discovered using the present methodology.

Materials

Name	Company	Catalog Number	Comments
Cyberball (Williams et al., 2000) computerized social interaction program	https://cyberball.wikispaces.com		An Alternate set of computerized images can be used or created by the researcher
Neuroscan SynAmps2 64-Channel Amplifier with SCAN 4.3.1 Acquisition and Analysis Software	Compumedics Neuroscan Neuromedical Supplies	9032-0010-01	Alternate amplifiers and EEG acquisition equipment and sofware can be used
STIM2 Complete Version 2.1 Stimulus Presentation Software	Compumedics Neuroscan Neuromedical Supplies	666M	Alternate stimulus presenation software can be used
SynAmps2 Quik-Cap Sintered Ag/AgCl 64 Ch./Medium	Compumedics Neuroscan Neuromedical Supplies	96050255	Alternate EEG caps can be used
Quik-Gel Conductive Gel	Compumedics Neuroscan Neuromedical Supplies	92000016	Alternate EEG conductive electrode gel can be used
NuPrep	Compumedics Neuroscan Neuromedical Supplies	92100025	Alternate skin preparation exfoliants can be used
Blunt needle and syringe kit	Compumedics Neuroscan Neuromedical Supplies	104207	Needle and syringe kit is used to apply conductive gel to electrode embedded in the EEG cap