Measurement of Neurophysiological Signals of Ignoring and Attending Processes in Attention Control

Summary

Attention control comprises enhancement of target signals and attenuation of distractor signals. We describe an approach to measure separately but concurrently, the neurophysiology of attending and ignoring in sustained intermodal attention, utilizing a passive control condition during which neither process is continuously engaged.

Abstract

Attention control is the ability to selectively attend to some sensory signals while ignoring others. This ability is thought to involve two processes: enhancement of sensory signals that are to be attended and the attenuation of sensory signals that are to be ignored. The overall strength of attentional modulation is often measured by comparing the amplitude of a sensory neural response to an external input when attended versus when ignored. This method is robust for detecting attentional modulation, but precludes the ability to assess the separate dynamics of attending and ignoring processes. Here, we describe methodology to measure independently the neurophysiological signals of attending and ignoring using the intermodal attention task (IMAT). This task, when combined with electroencephalography, isolates neurophysiological sensory responses in auditory and visual modalities, when either attending or ignoring, with respect to a passive control. As a result, independent dynamics of attending and of a ignoring can be assessed in either modality. Our results using this task indicate that the timing and cortical sources of attending and ignoring effects differ, as do their contributions to the attention modulation effect, pointing to unique neural trajectories and demonstrating sample utility of measuring them separately.

Introduction

Attention control guides behavior by directing our neural and cognitive resources toward select input signals, while restricting access to other signals, based on a given behavioral goal ¹. For instance, when reading a book, the visual signals corresponding to the book are the target signals to be enhanced, whereas other sensory signals — such as the TV in the next room — are distractor signals to be attenuated. Recordings in both human and non-human primates ^1-4, indicate that neural responses in sensory cortices are enhanced for attended targets relative to ignored distractors during selective attention, indicating that the strength of sensory inputs in the brain is modulated as a function of whether they are classified as targets or distractors ^5-7. We refer to this difference in signal strength when attending versus ignoring as the attention modulation effect.

Of increasing interest is the question of whether and how the neural processes of attending contribute to attention control and its impairments, separately from the neural processes of ignoring. It is increasingly clear that the ability to ignore distractions can be impaired independently from our ability to attend targets. For instance, distractor-suppression can be impaired with increased task load ⁸, cognitive aging ⁹ and sleep deprivation ¹⁰, without a decrement in target enhancement. It is not currently known if a decrement in target enhancement can also exist without a deficit in distractor suppression. Perhaps more importantly, it is not resolved whether deficits of either attending or ignoring, but not both, can elucidate neuropsychiatric conditions in which attention control is impaired. As such, it is valuable to better understand whether attending and ignoring arise from separable cortical pathways, if and how they differ in neural dynamics. By measuring attending and ignoring processes separately, such questions can be addressed.

Here we describe methodology to measure the neurophysiological signals of attending and ignoring separately, but concurrently, in sustained attention. This approach builds on the attention modulation effect: the difference in amplitude of a neural sensory response when the individual is attending versus ignoring to stimuli in that sensory stream. The attention modulation effect is a powerful tool for detecting attention modulation over sensory signals, but it precludes the ability to assess the separate dynamics of attending and ignoring processes. Namely, a difference in neural sensory responses when attending versus ignoring could arise because the process of attention enhances sensory target signals, or because ignoring attenuates sensory distractor signals, or both. To test between these alternatives, the use of an additional control condition is required in which one quantifies the strength of sensory inputs at their natural baseline, when they are neither attended nor ignored. This is similar to walking down a busy street full of cars, but neither actively watching (e.g., for a taxi) nor actively ignoring (e.g., non-taxi cars and buses) the passing cars. By evaluating sensory signals that are attended or ignored, relative to a passive reference condition, the magnitude and timing of attending and ignoring processes can be quantified separately.

Effective uses of such a passive control in measuring attending and ignoring processes have been reported previously in studies of anticipatory attention ^11-13 and memory-attention interactions ^9,10,14-17. Here we describe the use of this approach in the context of sustained attention, in a non-cued, continuous, intermodal (i.e., auditory-visual) attention task (IMAT) ¹⁸. In other words, this method is appropriate to the study of ongoing rather than preparatory control processes, allowing for tracking of these processes across time. This method also quantifies control processes that modulate sensory responses across different sensory modalities (i.e., auditory versus visual), thus focusing on processes that are not specialized to within a particular sensory or content domain. Unlike previous functional magnetic resonance imaging studies ^15,19,20, this method tracks attending and ignoring processes using temporally resolved neurophysiological signals (electroencephalography, EEG), thus providing millisecond resolution on the temporal profiles of attending and ignoring processes. Our representative results demonstrate the use of the technique in identifying direct evidence for separable cortical sources and temporal dynamics of the neural processes of attending and ignoring, and unique contributions to the attention modulation effect.

Protocol

NOTE: This study protocol was developed in accordance with the ethical guidelines approved by the investigational review board at University of California Los Angles.

1. Preparation of Auditory and Visual Stimuli

1. Using software in which visual images can be generated, create two gray scale sinusoid gratings, approximately 5.7 inches in diameter and of any frequency (e.g., 1.36 cycles/degree of visual angle). The images will have an on-screen duration of 100 msec.
   1. Tilt one of the gratings about 10 visual degrees to the right off the median, and tilt the other grating the same amount to the left of the grating.
   2. Ensure that the degree of the tilt is sufficient to allow the participants to distinguish a left tilt from a right tilt without relying on guessing.
2. Using software in which auditory tones can be generated, create two pure tones of 100 msec duration.
   1. Make one of the tones of a higher pitch and the other of a lower pitch. For instance, one pitch can be 750 Hz and the other 900 Hz.
   2. As for the visual stimuli, ensure that the tones are sufficiently distinct such that participants can distinguish between them without relying on guessing.

2. Programming of Stimulus Presentation

1. Using presentation software, create the computer code that will control the presentation of the auditory and visual stimuli during the experiment.
   1. First select the number of stimuli to be presented. Present at least 150 of each of visual and auditory stimuli per experimental condition, to ensure that there are enough repetitions for a reliable neurophysiological response.
   2. Present visual stimuli centrally on a gray background, with the participant sitting at a comfortable viewing distance. Present the auditory stimuli through speakers positioned on either side of the screen.
      Note: For visual stimuli we recommend a gray background, with RGB values at the mid-point (128,128,128) between pure white (255,255,255) and pure black (0,0,0), with the white and black used in generation of the sinusoid stimuli. This ensures that the mean brightness of the background and stimulus are comparable, and the contrast is constant between any point in the stimulus and the background.
   3. For each of auditory and visual stimuli, independently select the timing for the stimuli.
      Note: This prevents participants from anticipating stimuli based on temporal relationships between the two streams.
   4. Use an inter-stimulus interval (ISIs) of approximately 1 sec between sequential presentations of stimuli from the same modality. Slower ISIs will make the task more demanding on vigilance, faster ISIs may make it impossible for participants to make their responses in time.
   5. Vary the precise ISI randomly within a range, such as 0.7 to 2 sec, to make the stimuli unpredictable to participants, preventing neural responses associated with anticipation.
   6. Because cross-modal interactions can arise from simultaneously or near-simultaneously presented stimuli ^21,22, keep the ISI between stimuli from two different streams at no less than 300 msec.
2. Ensure that the auditory and visual stimuli appear to occur interleaved to the participants, but never co-occurring.
3. Last, divide the stimuli into segments of twenty-five. These segments will be preceded by one of the three randomly selected task instructions, described in the next section.

3. Task Instruction

1. Orient the participant to the task prior to collecting neurophysiological measures of brain activity.
   1. Instruct participants to attend and respond to the auditory tones and to ignore the visual stimuli when the instruction is “Listen”. Present this instruction both through audio and visual means.
   2. Assign two buttons for participants to make responses to each tone. For example, “press the left arrow if the tone is high, and right arrow if the tone is low” when the instruction is “Listen”.
   3. Similarly, instruct participants to attend and respond to the visual gratings and to ignore the auditory stimuli when the instruction is “Look”.
   4. Assign two buttons for participants to make responses to the visual gratings. For example, “press the left arrow if the grating is tilted to the left, and right arrow if the grating is tilted to the left”.
      1. Use the same two buttons for visual stimuli as for auditory stimuli to enhance the interference between the modalities and therefore the need to employ attention control mechanisms.
   5. Finally, instruct participants to make no responses when the instruction is “Passive”, but ensure that participants keep their eyes open and focused on the screen.
2. Throughout the task session, alternate the instructions for “Listen” and “Look” between segments to switch a previously attended modality to being irrelevant, thus making it a potent distractor.
3. Remind the participants to keep their eyes fixated on the middle of the screen, or a small dot or crosshair presented at the location of the visual stimulus, and to keep their eyes open throughout the experiment.
4. Build in eight to ten second breaks between segments to mitigate effects of fatigue, allow the participants to rest their eyes, as well as longer 1-2 min breaks every 6-8 min.
5. Finally, provide each participant with ample practice to ensure that they are performing the task correctly. It may be beneficial, especially for participants who have attention difficulties, to practice the visual and the auditory tasks with the attended stream presented in isolation, without the concurrent presentation of the distractor stream.

4. Neurophysiological Data Collection

1. Once the participants are familiar with the task, begin collection of neurophysiological responses to attended and ignored signals during the IMAT.
2. Prepare the electroencephalography (EEG) cap and recording equipment according to manufacturer instructions, and in accordance with current methodology and publication standards for EEG research ^26,27.
   Note: Important EEG recording parameters for EEG recording during the IMAT, that can be user specified, include: (a) sampling rate of 128-1,024 Hz, to capture the low-frequency of ERP signals; (b) alternating current (AC) recording to minimize slow drift; (c) net with sampling of entire scalp and with at minimum of 64 sensors, if source imaging analyses are to be performed.
3. Apply the cap to the participant’s scalp and verify signal impedance and quality at each of the sensors. Pay special attention to ensure that impedances of the recording electrodes are uniform and within the range recommended by manufacturer.
   Note: At this time, also add any additional physiological measurement devices if wanting to collect non-neural physiological signals such as respiration or pulse.
4. Synchronize the neurophysiological recordings with the stimulus presentation software and neurophysiological recording software according to manufacturer’s instructions.
5. Record the neurophysiological signals while the participant performs the task, ensuring that the recording software has precise record of the timing of each stimulus and response for subsequent analysis.

5. Offline Data Analysis

1. Prepare the neurophysiological data for statistical analysis using analysis software.
2. First, remove non-neural signal components that will contribute variability to the neurophysiological recordings of brain responses.
   1. Use a high-pass filter of 0.1-1 Hz, to remove slow drifts such as those caused by changes in impedance of the sensors.
   2. Use a low-pass filter of 30-50 Hz to remove high frequency components introduced by electrical noise.
   3. Identify sensors that show unreliable data, and exclude these or interpolate the signals.
   4. Identify and eliminate large infrequent noise components such as muscle artefact from jaw contraction or movements of the forehead, and systematic, non-neural contributions, such as eye movements.
      Note: Typical algorithms to remove non-neural components include independent components analysis and regression, as well as iterative algorithms based on explicit selection criteria (e.g., voltage changes that exceed a threshold). Follow guidelines of available analysis software, and proceed in accordance with current standards for EEG research ^26,27.
3. Once major non-neural components of the neurophysiological data have been removed, re-reference the data at each electrode by subtracting from each sensor the mean across all other sensors or the mean across left and right mastoid channels.
   Note: This step re-expresses the effects at each sensor relative to a neutral reference that is assumed to contain zero neural signals.
   Note: Sparser electrode montages may not have sufficient sampling to meet the assumptions of this technique ^26,27. In the latter case, the mean of the left and right mastoid may provide a more accurate reference.
4. Next extract temporal epochs of approximately 1 sec surrounding each auditory and each visual event shown. Include 100 msec preceding the stimulus onset to serve as a baseline interval and at least 600 msec following the stimulus onset.
5. Average the data from all epochs that fall into the same condition — attended, ignored, and passively perceived stimuli — to compute the average evoked response potential or “ERP”. Subtract the mean of the data in the pre-stimulus baseline to re-express the ERP amplitudes as changes relative to the pre-stimulus signal.
6. To identify the time course of attending processes, compare the amplitude and timing, as well as spatial distribution of the ERP response after attended stimuli, against those during the passive condition.
7. To identify the time course of ignoring, compare the amplitude and timing, as well as spatial distribution of the ERP response after ignored stimuli, against those during the passive condition.

Representative Results

The IMAT protocol has been used previously to identify the unique contributions of attending and ignoring processes to response speed during sustained attention ¹⁸. In that study, we tested 35 healthy right-handed individuals (22 female, age: x̅ = 21.0, σ = 5.4), recruited through the Psychology department subject pool at the University of California, Los Angeles. All participants provided written informed consent prior to participating in the study. Representative results highlight the value of measuring attend and ignore processes independently. In these results, the IMAT uncovered unique temporal and spatial profiles of attend and ignore processes with respect to the attention modulation effect.

For instance in the auditory sensory modality, attending and ignoring processes contributed at different time points to the attention modulation effect that is obtained by subtracting sensory activities in attend and ignore conditions from one another (Figure 1). The ERP to auditory stimuli at frontocentral sensors, in the first 100 msec after stimulus onset, was modulated by ignoring processes but not by attending processes relative to the passive control. However, in the subsequent 400 msec, modulation of the sensory ERP occurred during the attend condition but not during the ignore condition. Hence, attending and ignoring processes affected sensory auditory responses at different time points following stimulus onset. Since different time points in the ERP can be associated with different stages of processing, such a result indicates that attending and ignoring may implemented at different stages of processing, and are therefore associated with different cortical mechanisms. This differentiation was not evident from the comparison of sensory responses during attended versus ignored conditions (i.e., without a passive control), which showed a significant attention modulation effect throughout the entire sensory processing interval.

Figure 1: Attend and Ignore Temporal Responses to Auditory Stimuli. Auditory evoked-response potentials (ERPs, top plot) to auditory stimuli during attend, ignore, and passive conditions. The inset shows the topography of this response across the scalp. Bottom plot shows difference waves obtained by pairwise subtraction (A = attend, P = passive, I = ignore). Solid bars at top of plot indicates time points during which respective pairs differed in signal amplitude, p <0.05 (false discovery rate corrected). Significance was assessed using a paired t-test. Shading on lines indicates one standard error around the mean.

In the visual sensory modality, a different pattern of results was obtained using the passive control condition that also showed unique contributions of attending and ignoring processes to the attention modulation effect (Figure 2). The visual sensory ERP, at occipital electrodes, showed a significant effect of attentional modulation (attend-ignore) from 180-300 msec following stimulus onset, and after 450 msec. This time period also showed a significant effect of attending when compared to the passive control, but not for ignoring, suggesting that only attending processes modulated sensory processing. Comparing Figure 1 and Figure 2, we can conclude that attending processes during this time period contributed to sensory modulation in both auditory and visual sensory modalities.

Figure 2: Attend and Ignore Temporal Responses to Visual Stimuli. Visual evoked-response potentials (ERPs, top plot) to visual stimuli during attend, ignore and passive conditions. The inset shows the topography of this response across the scalp. Bottom plot shows difference waves obtained by pairwise subtraction (A = attend, P = passive, I = ignore). Solid bars at top of plot indicates time points during which respective pairs differed in signal amplitude, p <0.05 (false discovery rate corrected). Significance was assessed using a paired t-test. Shading on lines indicates one standard error around the mean.

One may also conclude, comparing Figures 1 and 2, that in the visual sensory modality, earlier effects of ignoring were absent, relative to the auditory modality. However, the temporal profiles of these processes depend on the choice of spatial sensors selected for analysis. In Figures 1 and 2, these were evaluated for a cluster of frontocentral and occipital electrodes, respectively ¹⁸. At a different cluster, the temporal patterns of responses may differ. This is compounded by the fact that each electrode measures signals from many different cortical sources, and thus is a mixture of neural signals arising from different cortical locations. For this reason, further resolution of attending and ignoring dynamics, in the spatial domain, may be obtained by projecting the electrode measurements onto a cortical model of the brain. We present here the results of such an analysis, which illustrates further information gain from the IMAT. The methods for cortical projection of EEG data is fully described elsewhere ^18,24.

Figure 3: Cortical Projection of Attend and Ignore Processes. The cortical projections of pair-wise difference waves across the cortex (sPL: superior parietal lobe, latOcc: lateral occipital cortex, sTP: superior temporal plane), at 70 msec following stimulus onset across attend (A), ignore (I) and passive (P) conditions. Significance was assessed using a paired t-test. Cortical maps show resulting t-statistics that pass a significance threshold of p <0.05.

The cortical projection result for auditory and visual attend and ignore processes, at the latency of 70 msec following stimulus onset, is shown in Figure 3. This latency was chosen because it effectively captures variability in attending and ignoring cortical sources in both sensory domains — within a single time period. The data illustrate that, in addition to their different temporal patterns revealed in ERPs, attend and ignore processes across these two modalities had different cortical sources. The overall effect of attentional modulation (A-I) was reliable in superior parietal lobe (sPL), lateral occipital cortex (latOcc), and superior temporal plane (sTP), among other regions.

The effects of attend and ignore processes, when measured against the passive control condition, revealed a more complex pattern that indicated different cortical sources for attending and ignoring. Namely, in the auditory modality, the sTP and sPL decreased in activation when ignoring auditory stimuli (I-P) without an increase in activation when attending auditory stimuli (A-P). In contrast, latOcc increased in activity when attending to auditory stimuli (A-P) but showed no effect when ignoring (I-P). Similarly, cortical sources of attending and ignoring processes differed in the visual modality, following a roughly reversed pattern relative to the auditory modality. Activation decreased in latOcc when ignoring visual stimuli (I-P), but showed no effect for attending (A-P). Whereas, sPL and sTP increased in activation when attending visual stimuli (A-P), but not when ignoring (I-P).

We highlight the presence of attending and ignoring effects at 70 msec in the visual modality, that were not present in the ERP (Figure 2). A possible explanation, in line with the rationale presented above, is that data within the selected electrodes on the scalp represents both a subset and a mixture of the brain signals produced across cortex. The source-imaged results allow for a more accurate analysis of the generators and thus can reveal effects not present in a given subset of electrodes. The combination of the source-imaged data in visual and auditory domain illustrate that attending and ignoring processes have separable cortical sources, in addition to having distinct temporal patterns.

Discussion

Processes related to attending and to ignoring in attention control may involve different neural pathways and time courses. Therefore, it is of value to measure these processes separately. The IMAT is a tool, by which one may capture neurophysiological signals of attending and ignoring separately, but concurrently, in sustained attention. The critical steps include the measurement of sensory neurophysiological responses when the participant is attending, ignoring or passively perceiving stimuli presented in a given modality — either auditory or visual. Most importantly, the use of a reference condition in which neither modality is attended, and in which no responses are made, disambiguates the relative contribution of attending processes and ignoring processes to the attention modulation effect, which measures the degree to which sensory responses are enhanced when attending relative to when ignoring stimuli.

The approach can be modified to include additional controls. For instance, if the auditory and visual tasks differ in difficulty, this may introduce variability in the degree to which attending and ignoring processes are engaged between the two modalities. Furthermore, if such intermodal differences differ between individuals, the group result may be highly variable. A modification to curb this potential confound is to tailor the stimulus properties to each individual before starting the experiment. For example, one may use a stair-case method ²⁵ to establish how much of a pitch difference is required for participants to distinguish between the two tones at 80% accuracy, and how strong a tilt of-vertical is required for participants to distinguish between the two gratings at 80% accuracy. This will ensure comparable difficulty between the modalities. In the presented experiment ¹⁸, accuracy on the attend auditory task (x̅ = 0.94, σ = 0.01) was not significantly different than that on the attend visual task (x̅ = 0.93, σ = 0.01), t(34) <1, p >0.05.

To further establish if one stimulus stream is easier to process than the other, one can also include a condition in which participants respond to only one stimulus stream without the presence of the other (single stream condition) and compare these responses to when the participant attends to the stimuli in the presence of the other stimulus stream (dual stream condition). The comparison of the single stream versus dual stream performance will establish how susceptible the stimulus process is to interference from other modality inputs.

The primary limitation of the passive control condition is that because no responses are sampled, the precise focus of attention is not known. The participants may withdraw from external stimuli during this condition, akin to mind wandering; they may alternate between attending to visual and auditory stimuli; or they may divide their attention between all stimuli. This ambiguity limits the interpretation of mechanisms engaged during the passive condition. It provides, instead, a condition in which the external stimuli are neither continuously attended nor continuously ignored. A further issue is that the passive condition may engage a different level of arousal than during attending and ignoring. Arousal variations may be directly relevant to the mechanisms of attending and ignoring, further study of how these mechanisms manifest are warranted.

The broad significance of the passive reference, regardless of precise mechanism, is in providing a response-free, naturalistic reference point for evaluating attention modulation. This condition approximates the natural state in which we interact with the world when there is nothing specific to which we must attend. The passive condition can be easily integrated within any other attentional paradigm, whether using different sensory modalities or other neurophysiological neuroimaging modalities, such as magneto-encephalography, to quantify attending and ignoring processes. For instance, the visual modality in the IMAT paradigm engages feature-based attention (i.e., features of the sinusoid grating). Adapting the paradigm to spatial attention (i.e., attending to left versus right space) may reveal differences between attending and ignoring at even earlier stages, since visual-spatial attention effects can have earlier onset latency (e.g., 80-120 msec) than feature-based attention. Broader use of this technique in functional magnetic resonance studies ^15,18,19 is likely to reveal new insights regarding the connectivity and network dynamics of attending and ignoring, much in the same way as resting-state analysis of network connectivity has informed our understanding of mind-wandering.

Future implementations of the presented approach will benefit from additional measures during passive perception to better quantify the underlying cognitive strategy of the participant and to better understand what the default-states are of the neural processes of attention. Adaptations of the technique to single-trial analysis will allow one to track attending and ignoring processes across time within a given participant, opening doors to investigation of attention stability across time.

Materials

Name	Company	Catalog Number	Comments
NetStation Software	Electrical Geodesic, Inc.	version 4.5.1	Alternate recording software may be used.
Matlab Software	The MathWorks, Inc.	7.10.0 (R2010a)	Alternate analysis and presentation software may be used.
PsychToolbox Software	http://psychtoolbox.org/	v3.0.8 (2010-03-06)	Open-source software. Alternate stimulus presentation software may be used.
Netstation Amplifier	Electrical Geodesic, Inc.	300	Alternate amplifier may be used.
EEG Net	Electrical Geodesic, Inc.	HCGSN130	Alternate EEG cap may be used.
Saline-Based Electrolyte (Potassium Chloride)	Electrical Geodesic, Inc.	n/a	Electrolyte used in soaking of net for this high-impedance EEG system. Alternate electrolyte mediate can be used.