We present a protocol that uses functional magnetic resonance imaging to investigate the neural correlates of the memory-enhancing effect of emotion. This protocol allows identification of brain activity specifically linked to memory-related processing, contrary to more general perceptual processing, and can be used with healthy and clinical populations.
Emotional events tend to be better remembered than non-emotional events1,2. One goal of cognitive and affective neuroscientists is to understand the neural mechanisms underlying this enhancing effect of emotion on memory. A method that has proven particularly influential in the investigation of the memory-enhancing effect of emotion is the so-called subsequent memory paradigm (SMP). This method was originally used to investigate the neural correlates of non-emotional memories3, and more recently we and others also applied it successfully to studies of emotional memory (reviewed in4, 5-7). Here, we describe a protocol that allows investigation of the neural correlates of the memory-enhancing effect of emotion using the SMP in conjunction with event-related functional magnetic resonance imaging (fMRI). An important feature of the SMP is that it allows separation of brain activity specifically associated with memory from more general activity associated with perception. Moreover, in the context of investigating the impact of emotional stimuli, SMP allows identification of brain regions whose activity is susceptible to emotional modulation of both general/perceptual and memory-specific processing. This protocol can be used in healthy subjects8-15, as well as in clinical patients where there are alterations in the neural correlates of emotion perception and biases in remembering emotional events, such as those suffering from depression and post-traumatic stress disorder (PTSD)16, 17.
1. Experimental Design, Methodological Issues, & Stimuli
The SMP consist of two tasks, an encoding task and a retrieval task, and recording of brain activity may occur during the encoding task or during both encoding and retrieval tasks (see Figure 1 for a diagram illustrating the design of the SMP).
Figure 1. General Diagram of the SMP design. Brain regions whose activity during encoding is greater for subsequently remembered than for subsequently forgotten items (R > F) are associated with encoding success (ES). Similarly, if this difference in memory (Dm) 3 is measured based on comparing brain activity recorded during retrieval (Hits > Misses) it identifies regions associated with retrieval success (RS).
Several important aspects must be considered in studies using the SMP with emotional stimuli:
- It is important to control for the emotional properties of stimuli, such as emotional valence and arousal/intensity, as they have different effects on emotional perception and thus on emotional memory10, 26, 27. Commonly used emotional stimuli are pictures from the International Affective Picture System (IAPS)23 and words from the Affective Norms for English Words (ANEW)24. IAPS pictures and ANEW words are controlled on two major affective dimensions (i.e., valence - the degree of pleasantness or unpleasantness, and arousal - the emotional intensity) that define the emotional properties of a stimulus (see 25). Another category of stimuli frequently used in emotion research are human faces with emotional expressions, such us those from the Pictures of Facial Affect (http://www.paulekman.com) and the NimStim (http://www.macbrain.org/) sets.
- Other factors whose manipulations may also affect memory performance, and hence contribute to the versatility of this protocol include: a. variation in the type of encoding task used (i.e. incidental11, 13, 19 or intentional 8, 20), b. the encoding focus with respect to the emotional content of the stimuli (i.e. with emotional 8, 11 or non-emotional aspects as the primary focus 13, 19), and c. the type of retrieval task used (i.e. recall 8, 11 or recognition 12, 20). In addition, stimulus duration also affects memory performance 21, and hence it should be considered as well, along with the factors mentioned above.
- Other possible confounding variables, such as the human presence in and visual complexity of emotional and neutral pictures 11 should also be controlled for.
- It is critical that memory performance for the conditions of interest falls in reasonable ranges that allow fair statistical comparisons of both behavioral and brain imaging data. For example, in our cued-recall tasks, the proportion of remembered and forgotten trials was around 50% 8, 11. However, because in two-choice recognition tasks this performance level would be close to chance, higher overall performance level is expected. In this case, fair comparison between remembered and forgotten conditions can be accomplished by randomly selecting a smaller number of remembered trials to equate the number of forgotten trials. In addition, an enhancing effect of emotion on memory should also be present in the behavioral data, reflected in greater proportion of remembered trials for emotional than for neutral items.
- It is also advised that mood induction be avoided as this may also affect performance 22. This can be done by pseudo-randomizing the emotional stimuli to ensure that no more than 2-3 stimuli of the same valence are consecutively presented 11.
2. Preparing the Subject for the Scan
All Subjects provide written informed consent prior to running the experimental protocol, which has been approved by an Ethics Board.
Prior to Entering the Scanning Room
- Approximately 45 minutes prior to the scan each subject completes questionnaires to assess overall emotional state and anxiety level 28, 29. These initial assessments are used in combination with post-session assessments to screen for changes in mood/anxiety as a result of participating in the study.
- After the completion of the questionnaires, the subject is informed of the scanning procedures and given specific instructions for the behavioral task. The subject also completes a short practice session to become familiar with the task.
Entering the Scanning Room
- The subject is instructed to lie supine on the scanning bed, with additional cushioning for the head, and is given ear protection. Also, the non-adhesive side of a length of tape may be wrapped lightly around the subject’s forehead to help minimize movement.
- Prior to scanning, the subject’s right hand is positioned on a response box and the response box is tested. Before starting data collection, it should also be ensured that the subject can see the screen projection clearly. An emergency stop button is placed nearby the subject’s free hand so that s/he may indicate any urgent need to stop the scanner.
- In our studies, stimuli were presented using CIGAL, an in-house data presentation software program 30, but other software are also available, including E-prime, SuperLab, Presentation, and the Matlab-based Psychtoolbox.
- Based on our investigations, we recommend that the duration of scanned tasks involving emotion stimulation should be no longer than 1h, and the overall scanning time should not exceed 2 hrs; also, if possible the latter should be reduced in clinical groups 17.
3. Data Recording and Processing
In the original study by Dolcos and colleagues 11, MRI data were collected on a 1.5 Tesla GE scanner. Full-brain coverage was acquired via 34 contiguous gradient-echo echoplanar images (EPIs) sensitive to blood oxygen level dependent (BOLD) contrast, defined parallel to the AC-PC plane in the axial plane (TR = 3000ms; TE = 40ms; FOV = 642 image matrix; FA = 90°; slice thickness = 3.75 mm). The scanning parameters used here have been well suited to identify activations in the medial-temporal lobe (MTL) structures, but other scanning parameters optimized for higher spatial resolution can also be used 33-35.
- The subject completes questionnaires to assess state mood and anxiety levels and, if not performed before the scanning session, assessments of personality traits (e.g. neuroticism, emotional arousability) can also be collected.
- If retrieval activity is not recorded during the scanning session, the retrieval task may be performed following the scanning session, or at a later time. If not already performed in the scanner, an emotional rating task may be used to assess the subject’s emotional response to the stimuli. This way, the separation of emotional and neutral categories can be done based on the participants’ ratings, rather than based on the normative scores. However, this task should be performed following the retrieval task.
- To maximize the effect of emotion on memory, a minimum delay of ~20 minutes needs to occur between encoding and retrieval 18. However, the delay can be much longer, from minutes to days, weeks, and months, as the impact of emotion on memory can be long-lasting and emotional recollection tends to persist overtime compared to neutral recollection 12, 13.
The Memory Task
The main focus in the present protocol is on findings from a study using a cued-recall memory test 11, in which participants were provided with verbal cues for each picture and they had to provide in writing details about the pictures they could remember. However, other retrieval tasks can also be used. Of particular relevance is the so-called R-K paradigm 31, a memory recognition task in which participants categorize each retrieved item based on whether they recollect specific contextual details from encoding (Remember responses) or just have the feeling of familiarity that they encountered the items (Know responses). Used with fMRI and with emotional stimuli, this task allows not only dissociation of the neural correlates of recollection- vs. familiarity-based retrieval 32 but also identification of differential effects of emotion on these two types of retrieval, and the associated neural correlates 12.
fMRI Data Analysis
Statistical Parametric Mapping (SPM: (http://www.fil.ion.uc.uk/spm/) in combination with in-house Matlab-based tools were used for data analysis. Pre-processing typically involves: quality assurance, image alignment, motion correction, co-registration, normalization, and smoothing (8 mm3 Kernel), but if analyses involve anatomically defined regions of interest (ROIs), increased spatial specificity is obtained if data are not smoothed and normalized (see 11). Individual and group-level statistical analyses involve comparisons of encoding/retrieval brain activity based on subsequent memory performance. The modulatory effect of emotion on memory-related activity can be seen in areas showing greater ES / Dm or RS activity for emotional than for neutral stimuli (see Figure 2). Investigations targeting MTL activity allow identification of functional dissociations, which are linked to the emotional content of stimuli (emotional vs. neutral memory; 11), the type of memory being studied (item vs. source memory; 19), or the type of retrieval task (recollection- vs. familiarity-based; 12, 36). Investigation of brain regions outside of the MTL particularly target areas in the prefrontal cortex (PFC), that are differentially involved in emotional vs. neutral memory 10, 20.
Figure 2. Identification of Memory-Related Brain Activity Susceptible to Emotion Modulation. In addition to allowing identification of memory-related brain activity that is sensitive to emotion modulation (EmoDm / RS > NeuDm / RS; i.e., showing emotion x memory interactions), this design also allows identification of brain regions whose activity reflects emotion effects on general/perceptual processing [(Emotion R + Emotion F) > (Neutral R + Neutral F ); i.e., showing main effects of emotion], as well as overall effects of memory [(Emotion R + Neutral R) > (Emotion F + Neutral F); i.e., showing main effects of memory].
4. Representative Results:
The employment of the SMP in studies of emotional memory has proved successful in shedding light on the neural correlates of the memory-enhancing effect of emotion. Providing strong support for the modulation hypothesis 37, 38, studies of memory encoding have found evidence supporting the idea that emotion enhances memory through modulatory influences from the amygdala, a brain region involved in emotion processing, on activity in memory-related brain regions, such as the medial-temporal lobe (MTL 11, see Figure 3) and the prefrontal cortex (PFC10). Moreover, studies of memory retrieval provide further evidence that the modulation hypothesis also applies to retrieval, and that various MTL regions are differentially involved in recollection- vs. familiarity-based retrieval of emotional memories 12.
Figure 3. Evidence for the Modulation Hypothesis. Supporting the modulation hypothesis, the memory-enhancing effect of emotion was associated with increased ES/Dm activity (A) in and interactions (B) between the amygdala and the MTL memory system (illustrated in C & D). The bar graphs compare the percent signal change for emotional and neutral ES / Dm, as extracted from the peak activation slice and averaged across hemispheres. The scattergrams show the co-variation between ES / Dm activity in the amygdala and the entorhinal cortex for emotional and neutral pictures. ES / Dm = Encoding Success / Difference due to Memory. From Dolcos et al. 11, with permission.
This experimental protocol allows investigation of the neural correlates of the memory-enhancing effect of emotion, by specifically separating the memory-related brain activity that is susceptible to modulation by emotion from activity in brain regions susceptible to emotion modulation during general/perceptual processing. This design has been instrumental in advancing our understanding of the neural mechanisms underlying the impact of emotion on memory, and given its versatility it is expected that SMP will remain crucial in determining how various aspects present at the time an event is experienced will later affect one’s ability to recollect and re-experience a memory for that event and the associated emotions. Furthermore, this protocol can be easily adapted to investigate the neural correlates associated with biases in emotional processing and memory that are typically found in specific clinical populations (e.g. depression and PTSD patients) 16, 17. Successful implementation of the SMP to investigate the effect of emotional memory depends on properly considering the various factors mentioned above, including the manipulations that may influence responses during encoding and/or retrieval tasks.
No conflicts of interest declared.
FD was supported by a Young Investigator Award from the National Alliance for Research on Schizophrenia and Depression and a CPRF Award from the Canadian Psychiatric Research Foundation.
- Bradley, M. M., Greenwald, M. K., Petry, M. C., Lang, P. J. Remembering pictures: pleasure and arousal in memory. J Exp Psychol Learn Mem Cogn. 18, 379-390 (1992).
- Christianson, S. A. Emotional stress and eyewitness memory: a critical review. Psychol Bull. 112, 284-309 (1992).
- Paller, K. A., Wagner, A. D. Observing the transformation of experience into memory. Trends Cogn Sci. 6, 93-102 (2002).
- Dolcos, F. The Impact of Emotion on Memory: Evidence from Brain Imaging Studies. VDM Verlag Dr. Muller. (2009).
- Dolcos, F., Denkova, E. Neural correlates of encoding emotional memories: a review of functional neuroimaging evidence. Cell Science Reviews. 5, 78-122 (2008).
- Dolcos, F., LaBar, K. S., Cabeza, R. The memory-enhancing effect of emotion: Functional neuroimaging evidence. Memory and Emotion: Interdisciplinary Perspectives. Uttl, B., Ohta, N. Blackwell Publishing. 107-134 (2006).
- LaBar, K. S., Cabeza, R. Cognitive neuroscience of emotional memory. Nat Rev Neurosci. 7, 54-64 (2006).
- Dolcos, F., Cabeza, R. Event-related potentials of emotional memory: encoding pleasant, unpleasant, and neutral pictures. Cogn Affect Behav Neurosci. 2, 252-263 (2002).
- Dolcos, F., Graham, R., LaBar, K. S., Cabeza, R. Coactiviation of the amygdala and hippocampus predicts better recall for emotional than for neutral pictures. Brain & Cognition. 51, 221-223 (2003).
- Dolcos, F., LaBar, K. S., Cabeza, R. Dissociable effects of arousal and valence on prefrontal activity indexing emotional evaluation and subsequent memory: an event-related fMRI study. Neuroimage. 23, 64-74 (2004).
- Dolcos, F., LaBar, K. S., Cabeza, R. Interaction between the amygdala and the medial temporal lobe memory system predicts better memory for emotional events. Neuron. 42, 855-863 (2004).
- Dolcos, F., LaBar, K. S., Cabeza, R. Remembering one year later: role of the amygdala and the medial temporal lobe memory system in retrieving emotional memories. Proc Natl Acad Sci U S A. 102, 2626-2631 (2005).
- Ritchey, M., Dolcos, F., Cabeza, R. Role of amygdala connectivity in the persistence of emotional memories over time: an event-related FMRI investigation. Cereb Cortex. 18, 2494-2504 (2008).
- Jacques, S. t, Dolcos, P. L., Cabeza, F., R, Effects of aging on functional connectivity of the amygdala for subsequent memory of negative pictures: a network analysis of functional magnetic resonance imaging data. Psychol Sci. 20, 74-84 (2009).
- Jacques, S. t, Dolcos, P., Cabeza, F., R, Effects of aging on functional connectivity of the amygdala during negative evaluation: a network analysis of fMRI data. Neurobiol Aging. 31, 315-327 (2010).
- Hayes, J. P., LaBar, K. S., McCarthy, G. Reduced hippocampal and amygdala activity predicts memory distrotions for trauma reminders in combat-related PTSD. J Psychiatr Res. (2011).
- Ritchey, M., Dolcos, F., Eddington, K. M., Strauman, T. J., Cabeza, R. Neural correlates of emotional processing in depression: Changes with cognitive behavioral therapy and predictors of treatment response. Journal of Psychiatric Research. (2010).
- Kleinsmith, L. J., Kaplan, S. Paired-associate learning as a function of arousal and interpolated interval. J Exp Psychol. 65, 190-193 (1963).
- Kensinger, E. A., Schacter, D. L. Amygdala activity is associated with the successful encoding of item, but not source, information for positive and negative stimuli. J Neurosci. 26, 2564-2570 (2006).
- Kensinger, E. A., Corkin, S. Two routes to emotional memory: distinct neural processes for valence and arousal. Proc Natl Acad Sci U S A. 101, 3310-3315 (2004).
- Kensinger, E. A., Garoff-Eaton, R. J., Schacter, D. L. Memory for specific visual details can be enhanced by negative arousing content. Journal of Memory and Language. 54, 99-112 (2006).
- Bradley, M. M., Cuthbert, B. N., Lang, P. J. Picture media and emotion: effects of a sustained affective context. Psychophysiology. 33, 662-670 (1996).
- Lang, P. J., Bradley, M. M., Cuthbert, B. N. Technical Report A-8. International affective picture system (IAPS): Affective ratings of pictures and instruction manual. University of Florida. Gainesville, FL. (2008).
- Lang, P. J., Bradley, M. M., Cuthbert, B. N. Technical report C-1. Affective norms for English workds (ANEW): Stimuli, instruction manual and affective ratings. The Center for Research in Psychophysiology, University of Florida. Gainesville, FL. (2008).
- Bradley, M. M., Lang, P. J. Measuring emotion: the Self-Assessment Manikin and the Semantic Differential. J Behav Ther Exp Psychiatry. 25, 49-59 (1994).
- Kensinger, E. A. Remembering emotional experiences: the contribution of valence and arousal. Rev Neurosci. 15, 241-251 (2004).
- Hamann, S. B., Ely, T. D., Grafton, S. T., Kilts, C. D. Amygdala activity related to enhanced memory for pleasant and aversive stimuli. Nat Neurosci. 2, 289-293 (1999).
- Watson, D., Clark, L. A., Tellegen, A. Development and validation of brief measures of positive and negative affect: the PANAS scales. J Pers Soc Psychol. 54, 1063-1070 (1988).
- Spielberger, C. D., Gorsuch, R. L., Lushene, R. E. Manual for the State-Trait Anxiety Inventory. Consulting Psychologists Press. Palo Alto, CA. (1970).
- Voyvodic, J. T. Real-Time fMRI Paradigm Control, Physiology, and Behavior Combined with Near Real-Time Statistical Analysis. NeuroImage. 10, 91-106 (1999).
- Tulving, E. Memory and consciousness. Canadian Psychology. 26, 1-12 (1985).
- Eldridge, L. L., Knowlton, B. J., Furmanski, C. S., Bookheimer, S. Y., Engel, S. A. Remembering episodes: a selective role for the hippocampus during retrieval. Nat Neurosci. 3, 1149-1152 (2000).
- Kirwan, C. B., Jones, C. K., Miller, M. I., Stark, C. E. L. High-resolution fMRI investigation of the medial temporal lobe. Human Brain Mapping. 28, 959-966 (2007).
- Preston, A. R., Bornstein, A. M., Hutchinson, J. B., Gaare, M. E., Glover, G. H., Wagner, A. D. High-resolution fMRI of Content-sensitive Subsequent Memory Responses in Human Medial Temporal Lobe. Journal of Cognitive Neuroscience. 22, 156-173 (2010).
- Robinson, S., Pripfl, J., Bauer, H., Moser, E. The impact of EPI voxel size on SNR and BOLD sensitivity in the anterior medio-temporal lobe: a comparative group study of deactivation of the Default Mode. Magnetic Resonance Materials in Physics, Biology and Medicine. 21, 279-290 (2008).
- Daselaar, S. M., Fleck, M. S., Cabeza, R. Triple Dissociation in the Medial Temporal Lobes: Recollection, Familiarity, and Novelty. Journal of Neurophysiology. 96, 1902-1911 (2006).
- McGaugh, J. L. Memory--a century of consolidation. Science. 287, 248-251 (2000).
- McGaugh, J. L. The amygdala modulates the consolidation of memories of emotionally arousing experiences. Annu Rev Neurosci. 27, 1-28 (2004).