Experimental Paradigm for Measuring the Effect of Induced Emotion on Grammar Learning

Summary

Here, we present a protocol to measure the effect of positive induced emotion on grammar learning in foreign language learners using a semi-artificial language that integrates the grammatical rules of a foreign language with the lexicon of the learners' native language.

Abstract

Previous studies have found that emotion has significant influence on the learning of foreign language vocabulary and textual comprehension. However, little attention has been given to the effect of induced emotion on grammar learning. This research examined the influence of positive induced emotion on the learning of Japanese grammatical rules among a group of learners with Chinese as their native language, using a semi-artificial language (i.e. Chipanese), which combines the grammatical rules of Japanese and the vocabulary of Chinese. Music was used to invoke positive emotional conditions in participants. Participants were required to learn Chipanese sentences in a training session through practice and then a grammaticality judgment task was administered to measure learning outcomes. We found that participants in positive emotional states performed less accurately and efficiently than those in the control group. The findings suggest that the protocol is effective in identifying the effect of positive induced emotion on grammar learning. The implications of this experimental paradigm for investigating foreign language learning are discussed.

Introduction

Emotions play a crucial role in various cognitive activities such as perception, learning, reasoning, memorization, and problem solving. Because language learning requires attention, reasoning, and memorization, emotions may have significant influence on language learning outcomes¹. Several prior studies have explored the effect of induced emotions on word production or text comprehension^2,3, and have consistently found that emotion had a crucial influence on the two language processes. For example, Egidi and Caramazza found that positive emotion increased the sensitivity to inconsistency in text comprehension in the brain areas specific for inconsistency detection, whereas negative emotion increased the sensitivity to inconsistency in less specific areas². Hinojosa et al. examined the effect of induced mood on word production and discovered that negative mood impaired the retrieval of phonological information during word production³. In spite of the evidence showing that emotions have a marked impact on text comprehension and word production, it is still not clear whether emotions affect grammar learning, one of the essential aspects of language learning. The present study aimed to explore the effect of learners' emotional states on grammar learning.

Language and emotion are two primary components of human experience⁴. Their relationships have mostly been explored by studies in affective neurolinguistics. At the single word level, previous studies have consistently found that emotional features, such as arousal or valence, significantly affect the processing of individual words^5,6,7. Specifically, some studies have identified a significant advantage for positive words⁵, and other studies have found an advantage for both positive and negative words⁷. Although some studies have reported an interaction between valence and arousal, a lack of significant interaction was reported in other research⁴. The picture is more complex at the level of sentence processing. Previous studies have explored issues concerning the interaction between emotional content and syntactic or semantic unification processes during sentence comprehension. Emotional information has been found to exert different influences on the processing of gender or number features⁴. Further, positive and negative emotion was connected with different agreement effects⁴. For instance, positive emotional features facilitated number agreement processing, whereas negative emotional features inhibited these processes⁴. At the semantic level, emotional features influenced semantic unification processes in both sentence and discourse contexts through the activation of the brain regions involved in single word processing and combinatorial semantic processes⁴. A review of the previous literature indicates that most prior research has focused on the effects of emotional information on the comprehension of words, sentences, and texts^8,9, or the neural basis of emotional effects on language production^10,11. However, how individuals' affective states might influence language processing or learning has been largely overlooked.

The most frequently used approach to the studies of emotions in grammar learning is the artificial grammar learning paradigm. Several studies have used artificial grammar tasks to examine the effect of emotion on the learning of a new language¹². First introduced by Reber in 1967¹³, the artificial grammar learning paradigm is characterized by the use of non-meaningful materials, such as number strings or non-word letter strings, which are in fact generated by an underlying grammar. Researchers usually exposed participants in different emotional states (positive, neutral, or negative) to the number strings or letter strings presented either visually or audibly and measured their learning outcomes. Studies with the artificial grammar approach typically consist of a training session and a testing session. In the training session, participants are instructed to observe or memorize a list of symbol sequences that are generated from a finite state grammar. Participants are informed that the sequences follow a particular set of rules, but they are not given any details regarding these rules. In the testing session, participants are presented with new symbol sequences, some of which are grammatical and others are not. They are then required to judge whether the strings are grammatical or not. Artificial grammar tasks allow the instantiation of various theories of learning, such as rules, similarity, and associative learning theories¹⁴. This approach can effectively minimize the influence of lexical factors on the learning of grammatical rules, as artificial languages are made up of numbers, letters, or other meaningless symbols, rather than words in natural languages. However, many researchers have argued that the knowledge acquired in artificial grammar learning may represent statistical properties that are different from the features of natural grammar used by human beings¹⁵. Evidence from neurological studies shows that the grammars in natural languages are processed differently from the finite-state grammars used in artificial grammar learning tasks^16,17. Therefore, artificial grammar learning tasks may not reflect the learning of human languages. Studies of the emotion effect on grammar learning using artificial grammars are more likely to reveal how emotion influences statistical learning, rather than the learning of natural grammars in human languages. It is not entirely clear whether findings from the meaningless stimuli can be generalized to foreign language learning.

The present study intended to adopt a semi-artificial language paradigm to investigate the effect of emotion on grammar learning. Semi-artificial language tasks were first introduced by Williams and Kuribara to examine language learning. A semi-artificial language is generated with the combination of lexicon in the learners' native language and the grammar of a different language. An example of such language can be found in Williams and Kuribara's study¹⁸. Williams and Kuribara designed a novel semi-artificial language, Japlish, which followed the word order and case-marking rules of Japanese but used English vocabulary¹⁸. Sample Japlish sentences in their study are provided in Table 1.

Structure	Examples
SV	Horse-ga when fell?
SOV	Pilot-ga that runway-o saw
SIOV	Student-ga dog-ni what-o offered?
S when what-o V?	Bill-ga when what-o sang?
S who-ni what-o V?	That doctor-ga who-ni what-o showed?
S [SOV]V	John-ga angrily Mary-ga that ring-o lost that said.
OS[SV]V	That disease-o vet-ga cow-ga have that declared.

Table 1: Sample sentences in a semi-artificial language. The sentences were generated with English lexis and Japanese syntax. The sentences in the table are from Williams and Kuribara's study¹⁸.

As shown in the table, although English words are used, they are combined into sentences in accordance with the Japanese word order and case-marking rules. The Japlish sentences are all verb-final and nouns are case-marked for subject (-ga), indirect object (-ni), or object (-o). A detailed description of Japlish can be found in Grey et al.'s study¹⁹. Semi-artificial language tasks involve a training phase and a testing phase. During the training phase, participants are instructed to learn a new language, and in the testing phase, they are required to perform acceptability judgment tasks or sentence-picture matching tasks. The accuracy and reaction times (RTs) of their responses are recorded to assess their learning performance.

Semi-artificial language tasks have mainly three advantages: First, as semi-artificial languages are created using grammatical rules in a new language, the tasks can minimize the influence of prior knowledge of the structures as well as language transfer¹⁹. Second, the tasks enable us to control and manipulate the type and amount of exposure participants receive¹⁹. In this way, they allow for more accurate assessment of the learning effects. Finally, as the grammars used in semi-artificial language tasks are from human languages, the tasks allow us to measure how participants acquire natural grammars, rather than artificial ones. In this aspect, they are more advantageous than artificial grammar tasks in which sequences of numbers or letters are used instead of real words. The use of natural grammar makes us more confident to conclude that the findings obtained are applicable to natural language learning. Given that prior studies have demonstrated learning effects using the semi-artificial language paradigm^20,21,22, it is a useful approach to investigating issues in language learning that are difficult to isolate in the complex context of natural language research. However, semi-artificial language tasks are only applicable to foreign languages that are structurally different from learners' native languages. If the tested language is structurally similar to the learners' native language, it might make the former indistinguishable from the latter.

Compared with the tasks using natural languages, semi-artificial language tasks allow for a more objective assessment of the effects of emotion on grammar learning. This is because words in natural languages are closely associated with specific grammatical functions. For example, inanimate nouns (e.g., desk, nail) are more likely to function as the patients of verbs. Thus, it is difficult to differentiate the performance of vocabulary learning from that of grammar learning because the two are interrelated and inseparable in natural languages. As emotions have been found to have vital influence on word processing^23,24, they may have indirect influence on grammar learning. Therefore, it is not easy to clearly differentiate the effect of emotion on vocabulary learning from that on grammar learning. This problem can be easily solved in semi-artificial language tasks because these tasks allow for the separation of vocabulary from grammar, and thus enable us to identify the effect of emotion on grammar learning, without having to worry about the interference from lexical learning.

Although the semi-artificial language paradigm has been used in some studies to investigate linguistic knowledge in second language acquisition^25,26, this approach has rarely been used to explore learners' individual differences in emotional conditions in foreign language learning. In this study, we intended to explore how positive induced emotion influences grammar learning using a semi-artificial language. Findings from this study have important implications for foreign language teaching and learning.

Protocol

The experiment was approved by the Ethics Committee of Beijing Foreign Studies University and it complied with the guideline for experiments with human subjects. All subjects in this research provided written informed consent.

1. Stimuli construction

1. Design experimental stimuli based on the specific research questions. As this study is intended to examine foreign language learning using a semi-artificial language, create the experimental sentences by rearranging sentences in the participants' native language in accordance with the grammatical rules of the tested foreign language. Sample experimental sentences are provided in Table 2.
   NOTE: As our study was intended to investigate the learning of Japanese grammar by Chinese learners, a semi-artificial language (Chipanese) was generated with the vocabulary of Chinese and the syntax of Japanese. The experimental sentences were adapted from the experimental stimuli used in Liu, Xu, and Wang²⁷.

Table 2: Sample experimental sentences used in this study. Sentence (a) is a Chinese sentence and (b) is its Japanese equivalence. Sentence (c) is the experimental stimuli generated by rearranging sentence (a) in accordance with the syntactic structure of (b). This semi-artificial language was first designed by Liu, Xu, and Wang²⁷.

2. Select representative grammatical structures in the tested language. Make sure to include a variety of grammatical structures to maintain the grammatical complexity and diversity of a natural language.
   NOTE: Four grammatical structures were tested in our study, including two simple grammatical structures (SOV, SIOV) and two complex ones ([SOV]SV, [OSV]SV). The lexical frequency of nouns and verbs was matched between different types of sentences.
3. For the training phase, design at least 20 sentences for each grammatical structure in the tested language for the grammaticality judgment task. The grammatical and ungrammatical sentences should be counterbalanced and controlled for lexical variables such as lexical frequency and number of strokes.
4. For the testing phase, design at least 12 sentences for each grammatical structure in the tested language. Make half of the sentences grammatical and the other half ungrammatical. The grammatical and ungrammatical sentences should be counterbalanced and controlled for lexical variables.
5. Randomize the sentences and design the experiment.
   1. Randomize the sentences before presenting them to the participants in both the training phase and the testing phase. Find the 'Selection' tab on the Property Page in the stimulus presentation software and set the selection method to 'Random'.
   2. Divide the experiment into blocks with each lasting for no more than 10 min.

2. Participant recruitment and preparation for the experiment

1. Recruit participants who have no reading difficulties and no background in the foreign language that will be tested. Make sure participants have normal or corrected-to-normal vision.
2. Randomly assign the participants into two groups (an experimental group and a control group), with each group containing at least 30 members. Ensure the two groups do not differ in years of education or gender ratio.
3. Inform the participants that they should be free from exhaustion, hunger, illness, or other conditions that make them uncomfortable on the day of the experiment.
4. Invite the participants individually or in groups to the laboratory.

3. Procedure

1. Take the participants to the laboratory room and instruct them to sit down at computer workplaces.
2. Ask the participants to read and sign the written informed consent forms.
3. Have the participants complete emotion induction.
   1. Give the participants the pencil-and-paper version of the Self-Assessment Manikin (SAM) pictorial rating scale^28,29. Ask participants to rate their emotion by using pencils to mark the corresponding manikin.
      NOTE: The Self-Assessment Manikin scale is a non-verbal assessment tool to measure the three dimensions of emotion (i.e., valence, arousal, and dominance). The scale we administered was a nine-point scale that ranges from one to nine in each of the three dimensions.
   2. Instruct participants in the experimental group to listen to the positively-valenced music through headphones for 10 min.
      NOTE: In this study, Good Time was used to induce positive emotion. A pilot study was implemented in a group of 20 participants to test the validity of the music in emotional induction, and results indicated that participants rated their emotion as significantly more positive after the induction, which showed that the music was effective in placing participants in the positive affective state. Participants in the control group were not exposed to any emotionally-loaded materials. Rather, they proceeded directly to the learning task.
   3. Give participants the pencil-and-paper version of the Self-Assessment Manikin scale^28,29 and ask them to rate their emotional states.
4. Ask participants to perform the training task.
   1. Present the following written instructions on the computer screens to inform participants that they will learn a language containing the vocabulary in their native language and a new grammar: "Welcome to our experiment! In this experiment, you will be required to learn a new language that contains Chinese words and a new grammar. When you are ready, press any key to proceed to the experiment."
   2. Present the following written instructions on the computer screens to instruct participants to perform the learning task: "Next, you will see some sentences on the screen. Please observe them carefully and decide whether they are grammatical or not. Press '1' if the sentences are grammatical, and '0' if they are ungrammatical. After each response, you will receive a feedback reply ('CORRECT!' or 'INCORRECT!') and the correct sentence will be presented. Press any key to proceed to the learning task."
   3. Have the participants judge the grammatical acceptability of the sentences on the computer screen with a press of the button ('1' for grammatical and '0' for ungrammatical) and present a reply ('CORRECT!' or 'INCORRECT!') on the screen after each response. Present the correct structures after the replies to reinforce the effect of learning.
   4. When all trials are completed, present the following written instructions on the computer screens to inform participants that the training task is over: "The learning task is completed!"
5. Give participants the pencil-and-paper version of the Self-Assessment Manikin scale^28,29 and ask them to rate their emotional conditions.
6. Ask participants to perform the grammaticality judgment task for the testing session.
   1. Present the following written instructions on the computer screens: "Please continue to answer similar questions. Please decide whether the grammatical structures of the sentences are correct. Press '1' if you think they are correct and press '0' if you think they are incorrect. You will be given seven seconds to respond. If you fail to respond within the time limit, the next question starts automatically. Press any key to start the experiment."
   2. Have participants read the sentences shown on the screen and judge the grammaticality of the sentences with a button press ("1" for grammatical and "0" for ungrammatical).
   3. When all trials are completed, present the following written instruction on the computer screens to inform participants that the task is over: "The end. Thank you for your participation!"
7. Ask the participants to fill in a questionnaire about their demographic details. Prepare their payment during the questionnaire.
8. Provide the monetary compensation or reward to participants.

4. Data analysis

1. Collect data from the output files from the stimulus presentation software.
2. Perform an ANOVA analysis with emotion (positive, control) and sentence type (SOV, SIOV, OSVSV, SOVSV) as the independent variables and the average accuracy of learning as the dependent variable to explore the effect of emotion on accuracy.
3. Perform an ANOVA analysis with emotion (positive, control) and sentence type (SOV, SIOV, OSVSV, SOVSV) as the independent variables and the average RTs as the dependent variable to explore the effect of emotion on RTs.

Representative Results

The aim of this study is to explore the effect of positive induced emotion on foreign language grammar learning. For this purpose, two groups of participants were recruited to participate in the experiment, including a positive-emotion group (15 female, M_age = 20.20, age range: 18–27) and a control group (16 female, M_age = 20.33, age range: 18–26). Each group consisted of 30 participants. The two groups did not differ significantly in age, t (58) = -0.215, p = 0.831 or years of education, t (58) = -0.830, p = 0.410, although the control group had higher level of English language proficiency than the other group. None of them reported to have learned Japanese before. Music was used to induce positive emotions. After emotion induction, participants were required to learn a novel language, Chipanese, which was generated with Chinese vocabulary and Japanese grammar. The participants learned the language through a grammaticality judgment task during which they were asked to judge the grammaticality of the sentences, with feedback provided after each response. To assess the learning outcomes, we administered an acceptability judgment task during which participants were presented with new Chipanese sentences, some of which were grammatical and others were not. The participants were asked to decide whether the sentences were grammatical or not. The accuracy and RTs of their responses were recorded and analyzed to assess their learning performance.

To find out whether emotion induction was successful, a repeated-measures ANOVA was performed with time (before induction vs. after induction) and group (positive vs. control) as predictors and rating scores as the dependent variable. The analysis revealed a significant main effect of time, F(1, 58) = 25.91, p < 0.001; a significant main effect of group, F(1, 58) = 12.62, p < 0.001; and a significant interaction effect between time and group, F(1, 58) = 28.03, p < 0.001. Pair-wise comparison showed that the valence ratings for the control group did not change significantly after induction. Participants in the control group were in a relatively neutral emotional condition both before and after the induction. The ratings for the positive-emotion group increased significantly after the induction with the positively-valenced music (p < 0.001). Participants in the positive-emotion group were significantly happier than those in the control group (p < 0.001). Therefore, our emotion induction was successful (Figure 1).

Figure 1: Results of emotion induction. The figure presents the mean rating scores for the emotional conditions of the two groups. The error bars represent the standard deviation of the data. Please click here to view a larger version of this figure.

To explore whether emotional states were sustained during the training phase, we performed a repeated-measures ANOVA with group (positive vs. control) and time (after induction vs. after training) as predictors and rating scores as the dependent variable. Results indicate that there was a significant main effect of group, F(1, 58) = 52.96, p < 0.001. The average rating scores for the positive-emotion group were significantly higher than those for the control group, which suggested that the positive-emotion group was significantly happier than the control group. No significant effect of time was found, F(1, 58) = .61, p = 0.436, which indicated that valence ratings for the two groups did not change significantly during the training phase and the positive-emotion participants remained happier than the controls during this period. Results indicated that participants' emotional conditions were sustained throughout the experiment.

Figure 2: Accuracy of grammar learning by group. The figure presents the mean proportion of correct answers by the positive-emotion group and the control group. The error bars represent the standard errors of the data. Please click here to view a larger version of this figure.

The mean accuracy of grammar learning by the two groups is presented in Figure 2. ANOVA was performed with emotion (positive, control) and sentence type (SOV, SIOV, OSVSV, SOVSV) as the predictors and mean accuracy as the dependent variable. Results revealed a significant effect of emotion, F(1, 58) = 62.68, p < 0.001; and a significant effect of sentence type, F(1, 58) = 35.21, p < 0.001. The interaction between emotion and sentence type was not statistically significant, F(1, 58) = 1.71, p = 0.165. The participants in the control group performed significantly more accurately than those in the positive emotion condition. SIOV and SOVSV structures were comprehended significantly less accurately than SOV and OSVSV structures. The control group performed better than the positive-emotion group regardless of the grammatical structures.

Figure 3: Reaction times in grammar learning by group. The figure presents the mean RTs by the positive-emotion group and the control group. The error bars represent the standard errors of the data. Please click here to view a larger version of this figure.

The mean RTs for the two groups are shown in Figure 3. Before analysis, the RTs for incorrect responses were removed. RTs above three standard deviations from the mean were considered as outliers and excluded from analysis. ANOVA was performed with emotion (positive, control) and sentence type (SOV, SIOV, OSVSV, SOVSV) as the predictors and RTs as the dependent variable. Results showed that there was a significant effect of emotion, F(1, 58) = 600.81, p < 0.001; and a significant effect of sentence type, F(1, 58) = 77.03, p < 0.001. The interaction between emotion and sentence type was not significant, F(1, 58) = 1.70, p = 0.165. The participants in the control group reacted faster than those in the positive emotional states. SOV structures were recognized faster than the other three structures, and SIOV structures were recognized more slowly than other structures. However, the effect of emotion did not differ significantly across different structures. The positive-emotion group reacted significantly more slowly than the control group in all the structures tested.

Discussion

The results indicate that participants rated their emotions to be significantly more positive after being exposed to the positively-valenced music. These subjects were significantly happier than the control group. This suggests that our emotion manipulation was successful. Participants in the positive-emotion group were found to be significantly less accurate and efficient than those in the control group. One possible reason is that participants employed an inductive strategy in grammar learning, resulting in a strong reliance on analytical and bottom-up processing. Inductive processing involves the careful consideration of detailed information and the use of analytical strategies to process information. Deductive processing, on the other hand, involves the use of heuristic strategies which feature a strong reliance on previous experience to speed up the problem-solving process. As grammar learning is a process that requires the consideration of the relationships between different linguistic units and the extraction of the rules governing these relationships, inductive processing is a more facilitative processing style than deductive processing. Previous studies have found that the inductive approach is an effective approach to learning foreign language grammar³⁰. In the present study, we manipulated the word order and case-marking of the experimental sentences, which are essential aspects of grammar. This learning task required participants to analyze and identify the relationships between different words in the sentences and form judgments about the legitimate order of the words in the sentences. Most importantly, these judgments must be based on the careful observation and analysis of specific linguistic details such as syntactic categories or semantic roles. Therefore, word order learning benefited more from inductive strategies and the corresponding bottom-up processing style. As positive emotion fosters a deductive or top-down processing style^31,32, which is incompatible with the inductive approach used by participants, it may have an inhibitory effect on participants' performance in grammar learning. Another possible reason is that individuals in positive emotional conditions tend to process information in a less effortful way^33,34. According to the motivational principles, individuals in positive affective conditions are less motivated to make great efforts because they are more inclined to maintain their affective states by avoiding cognitive efforts³⁵. Therefore, learners in the positive emotional condition may be less motivated than the controls, which might explain why they performed worse in the experimental task. The findings are consistent with Politis and Houtz's study³⁶ and Liu et al.'s study²⁷, which found that learners in positive emotional conditions performed less accurately and carefully in learning activities. The findings from this study showed that the protocol was successful in identifying the effect of artificially induced positive emotion on foreign language grammar learning.

In terms of methodology, the experiment was based on the semi-artificial language paradigm. The semi-artificial language was chosen in order to ensure that the tested structures were novel to participants. This study is different from traditional artificial language studies in the following two aspects: First, we adopted a target language system that followed the grammatical rules of a natural language (Japanese). Second, the lexicon used in the target language system consisted of real words (Chinese), rather than meaningless number or letter strings. These two features increased the similarity of the semi-artificial language to natural languages used by human beings. The use of semi-artificial languages can retain the grammatical complexity and the semantic information in natural languages, and thus can better reveal how participants' individual differences in affective states influence their learning of grammatical rules in natural languages. Therefore, the findings from the present study are generalizable to foreign language learning outside the laboratory.

By combining the vocabulary in the learners' native language with the grammar in a foreign language, semi-artificial language tasks can effectively separate the two learning processes and differentiate the effect of emotion on grammar learning from that on vocabulary learning. Compared with natural languages, semi-artificial languages can minimize the interference of lexical factors in grammar learning, and thus allow for more objective assessment of learning outcomes. Semi-artificial language tasks have been used in several studies and were found to be an effective approach to studying the implicit or incidental learning of syntax or morphosyntax in non-native languages¹⁹. The present study demonstrated that this approach is also applicable to the investigations into learners' individual differences in non-native language learning.

A major problem with this protocol is the lack of a priori measures that ensure similar processing of grammaticality and of learning in the absence of any emotion manipulation in the two groups. An ideal protocol should include a series of pretest manipulations that allow for unambiguously demonstrating that the two groups learn grammar similarly in the absence of emotion induction. For example, we would administer a grammaticality judgment task and a word-order learning task using another foreign language and compare the performance of the two groups. They should only be allowed to proceed to the experiment if no significant difference is found between the two groups. Results of this study would be more convincing if we were able to demonstrate, prior to the experiment, that the two groups did not differ in basic aspects that could have biased their behavioral responses. This is a step that has been ignored in our protocol and should be closely considered in future studies. Another limitation is that it is only applicable to the languages that are grammatically different from the learners' native languages. If the foreign language has similar grammatical structures as the learner's native language, the resulting semi-artificial language will contain the same vocabulary and similar grammar as their native language, making it difficult to distinguish between the two. For those languages that are similar to learners' native languages, it is preferable to use artificial language tasks in which sequences of letters or numbers are used instead of words. Besides, as this research has only examined the effect of artificially induced positive emotion on grammar learning, the results may not necessarily hold the same for negative affective states. Further studies are needed to explore how negative affective states might influence foreign language learning using the semi-artificial language paradigm.

Materials

Name	Company	Catalog Number	Comments
E-prime	PST	2.0.8.22	Stimulus presentation software
Computer	N/A	N/A	Used to present stimuli and record subjects' responses
Self-Assessment Manikin (SAM)	N/A	N/A	Used to assess subjects' affective states. From Lang (1980)29