Lexical Decision Task for Studying Written Word Recognition in Adults with and without Dementia or Mild Cognitive Impairment

Summary

This article describes how to implement a simple lexical decision experiment to assess written word recognition in neurologically healthy participants and in individuals with dementia and cognitive decline. We also provide a detailed description of reaction time analysis using principal components analysis (PCA) and mixed-effects modeling.

Abstract

Older adults are slower at recognizing visual objects than younger adults. The same is true for recognizing that a letter string is a real word. People with Alzheimer's disease (AD) or Mild Cognitive Impairment (MCI) demonstrate even longer responses in written word recognition than elderly controls. Despite the general tendency towards slower recognition in aging and neurocognitive disorders, certain characteristics of words influence word recognition speed regardless of age or neuropathology (e.g., a word’s frequency of use). We present here a protocol for examining the influence of lexical characteristics on word recognition response times in a simple lexical decision experiment administered to younger and older adults and people with MCI or AD. In this experiment, participants are asked to decide as quickly and accurately as possible whether a given letter string is an actual word or not. We also describe mixed-effects models and principal components analysis that can be used to detect the influence of different types of lexical variables or individual characteristics of participants on word recognition speed.

Introduction

Words are stored in the mental lexicon in a highly interconnected network. The connections between words may reflect shared properties, such as semantic similarity (e.g., dog and cat), form similarity (dog and fog), or frequent co-occurrence in common language use (e.g., dog and leash). Cognitive theories of language, such as usage-based theory¹, argue that every encounter of a word by a language user has an effect on the word’s mental representation. According to Exemplar Theory, a word’s representation consists of many exemplars, which are built up from individual tokens of language use and which represent the variability that exists for a given category. The frequency of use² impacts representations in memory by contributing to the strength of an exemplar¹.

Word recognition speed can reveal the characteristics of the mental lexicon. A commonly used experimental paradigm for measuring the speed of word recognition is the lexical decision task. In this task, participants are presented with letter strings on a monitor, one at a time. They are instructed to decide as quickly as possible whether the letter string on the screen is a real word or not by pressing the corresponding button.

By examining reaction times for real words, researchers can address a number of important questions about language processing. For example, identifying which factors make recognition faster can test hypotheses about the structure of the mental lexicon and reveal its architecture. Moreover, comparisons of performance across different groups of participants can help us understand the influence of various types of language experience, or, in the case of aging or neurodegenerative diseases (e.g., Alzheimer’s disease), the role of cognitive decline.

Some factors (e.g., the frequency of use) exhibit greater influence on word recognition than other factors (e.g., word length). With advancing age, the way people recognize written words might change^3,4. Younger adults tend to rely heavily on semantic (meaning-based) aspects of a word, such as how many compounds (e.g., bulldog) or derived words (e.g., doggy) share aspects of both form and meaning with the target word (in this case, dog). Word recognition for older adults appears to be more influenced by form-based aspects, such as the frequency that two subsequent letters co-occur in the language (e.g., the letter combination st occurs more often in English words than the combination sk).

To determine the factors that influence the word recognition speed across different groups, the researcher can manipulate certain variables in the stimulus set and then test the power of these variables to predict word recognition speed. For example, to test whether word recognition is driven by semantic or form-based factors, the stimulus set should include variables that reflect the degree of connectivity of a word to its semantic neighbors in the mental lexicon or its connectivity to other words that share part of its form.

This method was used in the current study to investigate whether word recognition speed is influenced by different factors in younger and older adults and in individuals with Alzheimer’s disease (AD) or mild cognitive impairment (MCI)³. The method described here is based on visual word recognition but can be adapted to the auditory modality. However, some variables that are significant predictors of reaction times in a typical visual lexical decision experiment might not predict response latencies in an auditory lexical decision or may have the opposite effect. For example, the phonological neighborhood has the opposite effect across these two modalities⁵:  words with larger phonological neighborhoods exhibit a facilitatory effect on visual word recognition but result in longer response latencies in auditory lexical decision⁶.

Word-finding difficulties in older adults⁷ have been generally attributed to difficulty accessing the phonological word form rather than a breakdown of the semantic representation⁸. However, AD research has primarily focused on semantic declines^{9,10,11,12,13,14}. It is important to disentangle how semantic and orthographic factors influence the recognition of written words in aging with and without cognitive decline. The influence of form-related factors is more pronounced in older than in younger adults, and it remains significant in people with MCI or AD³. Thus, this methodology can help us uncover features of the mental lexicon across different populations and identify changes in the lexicon’s organization with age and neuropathology. One concern when testing patients with neuropathology is that they may have difficulties accessing task-related knowledge. However, the lexical decision task is a simple task with no burden on working memory or other complex cognitive skills that many patients exhibit problems with. It has been considered appropriate for AD and MCI populations.

Protocol

The protocol follows the guidelines of the Ethics Committee of the Hospital District of Northern Savo (IRB00006251).

1. Participant screening

1. Recruit younger and older adults who have normal or corrected-to-normal vision and are native speakers of the language tested unless the study addresses specific research questions regarding second language acquisition.
2. For healthy control groups, exclude participants who have a history of neurological or psychiatric disorders.
3. For the clinical groups, recruit individuals who have been diagnosed with Alzheimer’s disease¹⁵ or mild cognitive impairment^16,17. Recruit only individuals who are able to give informed consent, according to the clinician's judgment. For accurate comparisons, match the age range and mean of the clinical groups with that of the healthy older adult participants.
4. Measure the severity of dementia, for example, using the Clinical Dementia Rating Scale¹⁸ (CDR, 0=no dementia, 0.5=very mild, 1=mild, 2=moderate, 3=severe). Exclude patients with severe dementia because the task may be too difficult for them. Do not include participants who seem unable to follow instructions, despite their severity rating.

2. Stimulus construction

1. Select word stimuli to address specific research questions, for example, whether semantic or orthographic/phonological variables have a stronger influence on word recognition¹⁹ in different populations.
2. Calculate from a corpus²⁰ or retrieve from a database²¹ variables reflecting semantic, phonological, and orthographic characteristics of the stimuli so they can be used either as theoretically motivated predictors explaining word recognition reaction times or as control variables. Also, use participants’ gender, age, and years of education as explanatory or control variables.
3. In addition to the real words, build a set of matched pseudo-words. Pseudo-words resemble real words in that they conform to the language’s norms for placement of certain letters in certain word positions (phonotactics). In order to control for phonotactics, create pseudo-words, for example, by randomly recombining the first syllables from some words with the second syllables from other words. Remove any items that happened to produce a real word through this recombination and all the items that violate the phonotactics of the language.
4. Match the pseudo-words with the target words in terms of the word length in letters and bigram frequency, which is the average number of times that all combinations of two subsequent letters occur in a text corpus. These variables have been shown to influence recognition speed.
   NOTE: Manipulating the pseudo-word ratio (e.g., the number of real words relative to the number of pseudo-words) may lead to different results, with responses to the less probable stimuli being slower and less accurate²².
5. Add a set of real-word fillers in order to decrease participant’s expectancy of the next stimulus belonging to a certain type (e.g., a certain inflectional class). Choose them, for instance, from different word categories (e.g., inflectional classes) than the ones used to construct stimuli according to the characteristics of interest.

3. Experimental design

1. Present letter strings horizontally, one at a time, subtending a visual angle of about 5°.
2. Begin the experiment with a practice session that includes a small number of trials, with one word presented per trial (e.g., 15 words and 15 pseudo-words not included in the actual experiment). This is to familiarize the participant with the task and the response buttons. If the participant is not responding accurately (‘yes’ button for real words and ‘no’ button for pseudo-words) during the practice trials, provide feedback and redo the practice session.
3. Divide the experiment into blocks and give short breaks after the practice session and between the blocks. These breaks allow participants to rest their eyes and will reduce fatigue.
4. Start each new block with a few filler items that will not be included in the analysis (e.g., common nouns such as dog, sister, year) because the first few trials of the block are sometimes ignored by participants with MCI or AD.
5. Present the experimental items in a random order for each participant.
6. Begin each trial with a fixation mark (e.g., a + sign) appearing in the center of the screen for 500 ms, followed by a blank screen for a fixed (e.g., 500 ms) or variable amount of time (e.g., 500-800 ms).
7. Immediately after the blank screen, present a letter string (word or pseudo-word) for 1,500 ms or until the participant responds.
8. After a response is made or after 1,500 ms from the onset of the word (whichever comes first), follow again with a blank screen until 3000 ms has passed from the beginning of the trial.
9. Repeat this sequence until all of the items in the experiment have been presented.
   NOTE: Times for the delay between the stimuli serve as an example. Changing them may affect the pattern of results.

4. Experimental procedure

1. Place the participant in front of a computer monitor at a viewing distance of about 80 cm in a normally lit room.
2. Instruct the participant to decide as quickly and accurately as possible whether the letter string on the screen is a real word or not by pressing one of two corresponding buttons with their dominant hand (e.g., the index finger for real words and the middle finger for pseudo-words) or using the index finger of each hand.
   NOTE: Participants try to optimize their performance in line with the instructions. Thus, their responses will be affected by stressing speed over accuracy or vice versa²³.

5. Analyzing data with a mixed-effects model in R

NOTE: Many different statistical programs can be used to perform the analysis. This section describes steps for analyzing data in R²⁴.

1. Obtain the reaction time (RT) measured in milliseconds for each trial from the output file of the presentation program (e.g., E-Studio software).
2. Install the packages lme4²⁸ and lmerTest²⁹. Attach packages with the function library or require.
3. Import data into R by using, e.g., the read.table function.
4. Check the need for transformation, e.g., with the boxcox function from the MASS package²⁵, as the distribution of RT data is typically highly skewed.
   > library (MASS)
   > boxcox(RT ~ Expnanatory_variable, data = yourdata)
   NOTE: The graph produced by the boxcox function shows a 95% confidence interval for the boxcox transformation parameter. Depending on the lambda values located within this interval, the needed transformation can be chosen, e.g., λ=−1 (inverse transformation), λ=0 (logarithmic transformation), λ=1/2 (square root transformation), and λ=1/3 (cube root transformation).
   1. Transform the RT values using inverted transformed RTs (e.g., -1000/RT) or binary logarithms of RTs (e.g., log2(RT)) since these transformations tend to provide more normal-like distributions for reaction times in lexical decision experiments than raw RTs²⁶.
   2. Alternatively, use statistical methods that do not rely on normal distributions and fit robust linear mixed-effects models and provide estimates on which outliers or other sources of contamination have little influence²⁷.
5. Since reaction time analyses are typically conducted on accurate responses, exclude trials in which the participants’ response was incorrect (a response of “no” to real words) as well as omissions.
   1. Also, exclude responses to pseudo-words and fillers unless there are specific hypotheses about them.
   2. Exclude trials with response times faster than 300 ms because they typically indicate that the participant was too late responding to a previous stimulus or that he or she accidentally pressed the response button before reading the stimulus.
6. Build a basic linear mixed-effects model that identifies RT as the outcome measure and Subject, Item, and Trial as random effects. Note that variables whose values are randomly sampled from a larger set (population) of values are included as random effects and variables with a small number of levels or for which all levels are included in the data are fixed effects. Add the random effects in the form (1 | Subject) in order to estimate random intercepts for each of the random effects.
   > g1 = lmer (RT ~ (1 | Subject) + (1 | Item) + (1 | Trial), data = yourdata)
   > summary (g1)
7. Add explanatory variables in a theoretically motivated order. For instance, add words’ base frequency as a fixed effect. Some variables, such as base or surface frequency, have Zipfian distributions, so insert them in the model with a transformation that results in a more Gaussian distribution shape, e.g., logarithmic transformation.
   > g2 = lmer (RT ~ log(BaseFrequency + 1) + (1 | Subject) + (1 | Item) + (1 | Trial), data = yourdata)
   > summary (g2)
8. Check with the Anova function if adding each predictor (e.g., BaseFrequency) significantly improved the predictive power of the model compared to a model without the predictor.
   > anova (g1, g2)
   1. If there is no significant difference in the fit of the new model over the simpler model, prefer the simplest model with fewer predictors. Also, check the Akaike Information Criterion (AIC)³⁰ of each model. AIC is a measure of how well statistical models fit a set of data according to maximum likelihood. Lower values indicate a better fit for the data³¹.
      > AIC (g1); AIC (g2)
9. Repeat steps 5.7. and 5.8. by adding other explanatory variables, e.g., some of those that are presented in Table 1, one by one in a theoretically motivated order and keeping only those that significantly improve the predictive power of the model. If variable stimulus onset asynchrony was used, include it as a fixed-effect variable in the model.
10. Check for theoretically motivated interactions between predictors. For instance, add a term of interaction the log of Base Frequency by Age.
    > g3 = lmer (RT ~ log(BaseFrequency + 1) + Age + log(BaseFrequency + 1) : Age + (1 | Subject) + (1 | Item) + (1 | Trial), data = yourdata)
    NOTE: It is possible that a predictor is significant as a term of interaction with another variable, but not significant as the main predictor. In this case, do not remove this predictor from the model (include it also as the main effect).
11. Add by-participant random slopes³² for predictors by including “1 +” before the variable name, then “| Subject”, e.g., (1 + log(BaseFrequency  + 1) | Subject), because participants’ response times might be affected by words' lexical characteristics in different ways.
    NOTE: If there are many continuous predictors, allowing them all to have random slopes is unrealistic because random slope models require large amounts of data to accurately estimate variances and covariances^33,34. In case the maximal model does not converge (in other words, successfully compute), simplify the model³³. Alternatively, implement Bayesian versions of multilevel modeling³⁵.
12. Run the analysis for each participant group separately. Alternatively, run an analysis on all data, with group as a fixed-effect predictor, and then test for an interaction of group by significant predictors.
    > g4 = lmer (RT ~ log(BaseFrequency + 1) + Age + log(BaseFrequency + 1) : Age  + Group + log(BaseFrequency + 1) : Group +  (1 + log(BaseFrequency + 1) | Subject) +  (1 | Item) + (1 | Trial), data = yourdata)
13. In order to remove the influence of possible outliers, exclude data points with absolute standardized residuals exceeding, e.g., 2.5 standard deviations²⁶, and re-fit the model with the new data (yourdata2).
    > yourdata2 = yourdata [abs(scale(resid(g4))) < 2.5, ]
    > g5 = lmer (RT ~ log(BaseFrequency + 1) + Age + log(BaseFrequency + 1) : Age  + Group + log(BaseFrequency + 1) : Group + (1 + log(BaseFrequency +1) | Subject) +  (1 | Item) + (1 | Trial), data = yourdata2)
    NOTE: Not all extreme data points are harmful for the model – only those that have excessive leverage over the model.
14. In the case of exploratory (data-driven) analysis, use backward stepwise regression: include all variables in the initial analysis and then remove non-significant variables from the model in a step-by-step fashion. Alternatively, use the automatic procedure of eliminating non-significant predictors with the step function provided by the package lmerTest²⁹.
    > step (g4)

Results

Table 1 shows a list of variables that were obtained from three different sources (a corpus, a dictionary, and pilot testing of test items) that are included in the analysis as fixed-effect predictors. Many of these variables have been previously reported to affect word recognition speed.

Corpus:
Base frequency	the numbe...

Discussion

By using a simple language task that does not require language production, the present study investigated the impact of various lexical variables on word recognition in neurologically healthy younger and older adults, as well as in people with Alzheimer’s disease or Mild Cognitive Impairment. The age range used for recruiting “older adults” might depend on the specific research interests; however, the range for the healthy elderly group should match as closely as possible the age range and distribution ...

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
E-Prime	Psychology Software Tools	version 2.0.10.356.
PC with Windows and Keyboard
R	R Foundation for Statistical Computing	R Core Team (2018). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/.