Reading in color is a new method for training letter-color associations that are typically found only in grapheme-color synesthetes. It involves an implicit form of training that has potential for long-term associative training methods because the training is a byproduct of reading and any text can be colored.
Synesthesia is a rare condition in which a stimulus from one modality automatically and consistently triggers unusual sensations in the same and/or other modalities. A relatively common and well-studied type is grapheme-color synesthesia, defined as the consistent experience of color when viewing, hearing and thinking about letters, words and numbers. We describe our method for investigating to what extent synesthetic associations between letters and colors can be learned by reading in color in nonsynesthetes. Reading in color is a special method for training associations in the sense that the associations are learned implicitly while the reader reads text as he or she normally would and it does not require explicit computer-directed training methods. In this protocol, participants are given specially prepared books to read in which four high-frequency letters are paired with four high-frequency colors. Participants receive unique sets of letter-color pairs based on their pre-existing preferences for colored letters. A modified Stroop task is administered before and after reading in order to test for learned letter-color associations and changes in brain activation. In addition to objective testing, a reading experience questionnaire is administered that is designed to probe for differences in subjective experience. A subset of questions may predict how well an individual learned the associations from reading in color. Importantly, we are not claiming that this method will cause each individual to develop grapheme-color synesthesia, only that it is possible for certain individuals to form letter-color associations by reading in color and these associations are similar in some aspects to those seen in developmental grapheme-color synesthetes. The method is quite flexible and can be used to investigate different aspects and outcomes of training synesthetic associations, including learning-induced changes in brain function and structure.
Synesthesia is a rare condition typically described as a 'crossing of the senses.' There are many forms of synesthesia. One of the most common types is grapheme-color synesthesia, the consistent and automatic experience of color upon seeing, hearing, or thinking about letters, words or numbers. The prevalence of grapheme-color synesthesia is about 1% of the population1. Research has shown that synesthetic experiences are genuine, not imagined, and are not related to neurological or psychiatric illness2-11. Recent neuroimaging research has shown that the function and structure of the brains of synesthetes differ from matched nonsynesthetic controls12. There is a genetic component to synesthesia13,14, however, the majority of synesthetic inducers are culturally determined, for example by language15,16. This interaction between genes and environment begs the question of why some people develop synesthesia and others do not. Is this difference entirely determined by genetics, or is it possible to train synesthesia in individuals who may not possess 'synesthetic' genes? A first step to answering this question requires a long-term training method that has potential for developing associations of the type found in synesthesia. Here we describe a method for training letter-color associations by reading text with colored letters (Figure 1). We will also discuss a method that may be used to predict individual differences in the learning effect.
Previous synesthetic training methods have focused on directed training using a computer17-21. Computer training in the lab requires the participants to come into the lab over the course of several days, although computer training at home is also now feasible. Training associations by reading in color can be done at home and at the participants' convenience. Perhaps the biggest advantages of reading in color are that participants are able to choose what they would like to read and that any text can be colored. Importantly, the learned associations between letters and colors are a 'byproduct' of normal reading, since the participants are not instructed to nor are they trying to remember which letter is paired with which color (although of course they are aware of these colored letters). We have previously shown that it is possible to develop letter-color associations by reading in color, and these are similar in their automatic nature to those seen in developmental grapheme-color synesthesia, although we are not claiming that the trainees should be considered synesthetes22.
The most common behavioral task used to objectively measure the presence or strength of a synesthetic association is a modified version of the classic Stroop paradigm23. However, this type of task cannot be used as a diagnostic marker of synesthesia since it is possible to train a Stroop effect in nonsynesthetes24. (A test of consistency or test of genuineness is better suited for diagnosis of synesthesia2,25). In the classic Stroop task, a color word (e.g. 'RED') is shown in color to a participant and he or she must say what color the word is printed in as fast as possible. When the word is presented in the same color as the color word itself (e.g. 'RED' printed in red ink), it is considered to be congruent, and when the word is presented in different color as the color word (e.g. 'RED' printed in green ink), it is considered to be incongruent (Figure 2). People react slower and are less accurate on incongruent trials compared to congruent trials, and this difference is called 'the Stroop effect.' The cause of the Stroop effect is that reading the word itself is an automatic process that cannot be completely inhibited or ignored and therefore interferes with the task at hand. The 'synesthetic' version of this task uses letters instead of words, and the instructions are the same: indicate as fast as possible the color that the letter is presented in26. A difference between incongruent and congruent conditions is found in individuals who have an existing association between letters (or numbers) and colors, most of all synesthetes (Figure 2), but also trainees18,20-22,27. Genuine synesthesia involves more than just the presence of a Stroop effect, the most important characteristics being the experience of percepts (e.g. color) in a modality that is not being stimulated (e.g. black text) that are highly specific and consistent across time2,25. The presence of a 'synesthetic' Stroop effect shows that the associations between letters and colors are automatic to the point that they interfere with task demands, but it cannot prove that a letter-color association is perceptual in nature or whether an individual is a synesthete18.
Individuals are all different, and some trainees may develop large Stroop effects after reading in color, while others may not. We have developed a set of questions about subjective color experience that may predict how strong these learned associations become at the individual level. Additionally, learning letter-color associations may interact with pre-existing preferences for the letter-color pairs in nonsynesthetes. These results point towards underlying factors that may determine how sensitive an individual is to developing letter-color associations. For example, sensitivity may be determined by vividness of visual mental imagery28 or memory ability29-31, both of which have been found to be related to grapheme-color synesthesia. The reading in color method may help researchers to further explore these hypotheses by providing a feasible long-term training program for nonsynesthetes. It remains to be seen if reading consistently colored letters over long periods of time may lead some nonsynesthetic individuals to experience real color percepts in response to black letters that remain automatic and also consistent over years or a lifetime.
The overall procedure involves recruiting motivated participants to read colored books. The participants must be motivated to read because they will not be observed outside of the lab. Furthermore, they are trusted to accurately report how much they read. Participants must be tested both before and after reading for the presence of learned letter-color associations. Each participant should be assigned a unique set of letter-color pairs based on their individual preferences. Four high-frequency letters (e.g. a, e, n, and r) are paired with four-high frequency colors (e.g. red, orange, green, and blue). Learned associations are tested with a synesthetic version of the classic Stroop task, comparing congruent and incongruent conditions of letter-color pairs. Magnetic resonance images (MRI) are acquired before and after reading in order to test for learning-induced changes in brain function and structure32,33. In addition, it is important to evaluate the experience of the participants using questionnaires because the subjective experiences of the participants are of interest, such as possible color experiences in the presence of black text. The number of words read will vary between participants because they may choose their own reading materials. Post-session testing should always be conducted as soon as possible after the participant is finished reading (to date there is no data suggesting how long learning effects may endure as measured with the Stroop task). This protocol can be adjusted in many ways, but we consider it to be the basis for a synesthetic training method using reading in color.
We have described a basic method for training and testing letter-color associations through reading in color. An important aspect for this method to work properly is that the individual participants are each motivated to read the colored books, have done so, and reported the duration and amount of reading honestly, since they cannot be directly observed in the lab. It helps to recruit people who are reading enthusiasts because they normally do a lot of reading in their own time. In case the expected results are not obtained, it is important to rule out that a null effect (or lack of a Stroop effect) is simply due to the fact that the participants did not actually read the colored text. One way to ensure that participants have done the reading is for the experimenter(s) to read the books in order to converse with the participants about their contents. It is not always the case that each participant will develop a significant Stroop effect based on reading, but with a comparable sample size (N ~ 15), the representative results are typically found at the group level.
In terms of data analyses, decide before beginning the experiment whether outliers will be removed from the reaction time data. Our standard is that response times less than 150 msec and greater than 2.5 times the standard deviation per condition and session be removed. Before testing begins, it is good practice to determine a minimum criterion for the amount of reading necessary (we suggest around 40,000 words as a guideline) and to exclude participants who have not read at least this amount. Also, consider excluding participants who have stopped reading a certain number of days before the testing session. After the data has been collected, make sure that only correct trials are included in the reaction time analysis and check for speed-accuracy tradeoffs. Also, check the accuracy on the color-key response training to make sure that participants learned which key goes with which color. Watching the participants when doing the color-key training to make sure that they are able to do the task without looking at their fingers ensures the quality of the data to be collected. Consider checking whether the Stroop effect is present in most or all of the participants or whether only a few participants with very strong effects seem to be driving the group-level effects. Similarly, check whether one of the preference groups is driving the effects (in our experience, preference group 2 typically showed a larger change in the Stroop effect due to reading).
In the brain imaging functional data, ensure that the quality of the data is good enough by testing for significant (de)activation for all stimuli regardless of congruency. Furthermore, consider adding respiration and heart rate as confound regressors in your GLM analysis. Another control measure during scanning is the use of an eye-tracker to ensure that the participants are indeed watching the screen, doing the task properly and not asleep. Short runs (less than 10 min) help to avoid concentration and fatigue issues. Finally, the characteristics of the (subject) sample tested will also have an effect on the results and should always be taken into account if not a variable of interest because age and gender affect brain function and structure34,35. The interpretation of brain activation related to trained letter-color associations in the Stroop task remains an interesting line for future research. Regarding the classic Stroop task, the contrast incongruent > congruent is typically associated with frontal lobe and anterior cingulate activation that is thought to reflect cognitive control mechanisms because it is necessary to inhibit an automatic response in the incongruently colored condition compared to the congruent condition, which involves response and conflict monitoring46. The contrast congruent > incongruent has received less attention in the conflict processing literature47. The synesthetic version of the Stroop task has not been employed in many fMRI studies on grapheme-color synesthesia12. Two studies have tested synesthetes using a comparable synesthetic Stroop task in an fMRI paradigm, but they did not report comparable contrasts48,49.
Concerning the brain imaging structural data, consider using the significant clusters of activation found during the Stroop task as masks for regions of interests in the statistical analyses of the structural images. In addition, consider correlating white and grey matter properties with behavior on the Stroop task in order to test whether macroscopic brain structures predict inter-individual differences in performance and learning.
There are many aspects of the current protocol that can be changed based on the research goal of the experiment at hand. In the Stroop task, for example, the number of letters and colors trained can vary. It should be noted that increasing the number of colors tested would increase the necessary number of response options during the Stroop task affecting the experimental paradigm. Some researchers use a microphone to record vocal responses in the computer lab, although this is not as practical in the MRI scanner. A baseline condition can be added to the Stroop task (in addition to the congruent and incongruent conditions), in which letters that have not been colored in the books are presented in the same colors used for the congruent and incongruent conditions. Using a baseline condition can allow better inference as to how the congruent and incongruent conditions change over time in comparison to any change in the baseline condition. The duration and amount of reading may vary and may also be a variable of interest decided upon by the researchers, as well as the time in between reading the last colored book and the last testing session. Many other cognitive and perceptual tasks and questionnaires can be added to the protocol. Lastly, the protocol may be easily combined with other neuroimaging techniques.
The presence of a synesthetic Stroop effect in trainee data mimics the behavior on the same task found in grapheme-color synesthetes18,27,50. We would like to stress the fact that the presence of a Stroop effect is not enough to claim that trainees have genuine synesthesia. Developmental synesthesia is defined by more than just a synesthetic Stroop effect or perceptual color experiences, such as consistency of the letter-color associations over time with an onset in early childhood51, differences in the function and structure of the brain12, and a possible genetic predisposition13,14 (for a discussion about defining synesthesia, see references52-55). Therefore, we prefer the term 'pseudo-synesthesia', in order to differentiate trainees (even if they would report color experiences) from grapheme-color synesthetes, who report the experience of synesthetic associations for as long as they can remember.
Individual differences in the learning effect (i.e. the size of the post-reading Stroop effect) may be predicted by the self-report rating of how much the color experience is internalized (i.e. 'in the mind's eye'), indicating that the subjective experience of color plays a role in training synesthetic associations. Such a correlation does not provide definitive proof that color experiences related to the trained letters are perceptual in nature and not solely semantic (propositional) associations. It is possible that the questionnaire is biased because it excludes the possibility to indicate that an individual experiences a semantic association without having any experience of color. Following the suggestion of an anonymous reviewer on this point, we propose to add an additional question to the questionnaire: “When I see certain letters in black text, I do not see color or experience color (in the mind's eye), but I have an automatic association to color." Further research is necessary in order to understand whether sensitivity to forming letter-color associations is related to other processes, such as visual mental imagery. It remains to be seen whether trained letter-color associations go beyond semantic associations by having similar qualities as visual mental images or veridical perception.
Reading in color is a method for training letter-color associations. This method can be used to examine the extent to which cognitive advantages seen in synesthesia, such as better memory for certain stimuli categories29-31 may be trained in nonsynesthetes and whether this training may induce changes in the brains of the trainees. It is promising in the sense that there is still much to learn concerning the effects of both short-term and long-term 'reading in color' training methods. We hope that by providing the basic protocol that we have developed, others will use it to progress not just the field of (training) synesthesia research, but the cognitive neuroscience of learning and memory as well.
Appendix A: Code for coloring books in Microsoft Word using a Visual Basic macro
Sub Letters2Colors()
Dim a_color As Long
Dim e_color As Long
Dim n_color As Long
Dim r_color As Long
e_color = RGB(230, 0, 0) 'red
n_color = RGB(255, 143, 0) 'orange
a_color = RGB(0, 181, 0) 'green
r_color = RGB(0, 155, 255) 'blue
Selection.Find.ClearFormatting
Selection.Find.Replacement.ClearFormatting
With Selection.Find
.Text = "a"
.Replacement.Text = "a"
.Forward = True
.Wrap = wdFindContinue
.Format = True
.MatchCase = True
.MatchWholeWord = False
.MatchByte = False
.CorrectHangulEndings = True
.MatchWildcards = False
.MatchSoundsLike = False
.MatchAllWordForms = False
.Replacement.Font.Color = a_color
End With
Selection.Find.Execute Replace:=wdReplaceAll
Selection.Find.ClearFormatting
Selection.Find.Replacement.ClearFormatting
With Selection.Find
.Text = "e"
.Replacement.Text = "e"
.Forward = True
.Wrap = wdFindContinue
.Format = True
.MatchCase = True
.MatchWholeWord = False
.MatchByte = False
.CorrectHangulEndings = True
.MatchWildcards = False
.MatchSoundsLike = False
.MatchAllWordForms = False
.Replacement.Font.Color = e_color
End With
Selection.Find.Execute Replace:=wdReplaceAll
Selection.Find.ClearFormatting
Selection.Find.Replacement.ClearFormatting
With Selection.Find
.Text = "n"
.Replacement.Text = "n"
.Forward = True
.Wrap = wdFindContinue
.Format = True
.MatchCase = True
.MatchWholeWord = False
.MatchByte = False
.CorrectHangulEndings = True
.MatchWildcards = False
.MatchSoundsLike = False
.MatchAllWordForms = False
.Replacement.Font.Color = n_color
End With
Selection.Find.Execute Replace:=wdReplaceAll
Selection.Find.ClearFormatting
Selection.Find.Replacement.ClearFormatting
With Selection.Find
.Text = "r"
.Replacement.Text = "r"
.Forward = True
.Wrap = wdFindContinue
.Format = True
.MatchCase = True
.MatchWholeWord = False
.MatchByte = False
.CorrectHangulEndings = True
.MatchWildcards = False
.MatchSoundsLike = False
.MatchAllWordForms = False
.Replacement.Font.Color = r_color
End With
Selection.Find.Execute Replace:=wdReplaceAll
End Sub
Appendix B: Example reading experience questionnaire
The following questions should be given in random order:
Q1) I enjoy reading
Q2) I do not enjoy reading
Q3) I enjoyed reading in color
Q4) I did not enjoy reading in color
Q5) I enjoyed the content of the book
Q6) I did not enjoy the content of the book
Q7) The colored text was pretty
Q8) The colored text was ugly
Q9) The colored text was distracting
Q10) The colored text was not distracting
Q11) The colored text became less distracting over time
Q12) The colored text became more distracting over time
Q13) I felt as if I was reading faster in color (by the end of the book)
Q14) I felt as if I was reading slower in color (by the end of the book)
Q15) I was more motivated to read this book compared to a book with normal black text
Q16) I was less motivated to read this book compared to a book with normal black text
Q17) I like the color red
Q18) I like the color orange
Q19) I like the color green
Q20) I like the color blue
Q21) I read more than an average person
Q22) I read less than an average person
Q23) I tend to read books from the same genre
Q24) I tend to read books from a variety of genres
Q25) When I see certain letters in black text (e,n,a,r), I see them in color
Q26) When I see certain letters in black text (e,n,a,r), I experience them in color (i.e. in the mind's eye)
Q27) When I think about certain letters (e,n,a,r), I experience them in color (i.e. in the mind's eye)
Q28) When I think about certain letters (e,n,a,r), I see them in color
Q29) Whenever I see or think about letters, I have no color experience
Q30) Choose one of the following statements that best describes your experience (A to E)…
(A) When I see certain letters in black text (e,n,a,r), I see them in color
(B) When I see certain letters in black text (e,n,a,r), I experience them in color (i.e. in the mind's eye)
(C) When I think about certain letters (e,n,a,r), I experience them in color (i.e. in the mind's eye)
(D) When I think about certain letters (e,n,a,r), I see them in color
(E) Whenever I see or think about letters, I have no color experience
Open question: Have you noticed any changes in behavior or experience since you started reading the book(s)?
The authors have nothing to disclose.
We thank Nick Daems for writing the Visual Basic Word macro. We would like to acknowledge and thank the publishers Nijgh and van Ditmar (Amsterdam, the Netherlands) for providing Dutch language materials for our research protocol. We would also like to thank all of our participants.
Achieva 3T MRI scanner | Philips | website for information: http://www.healthcare.philips.com/main/products/mri/systems/achievaTX | |
Presentation | Neurobehavioral Systems | Software for conducting psychological experiments: www.neurobs.com |