This paper introduces a method for obtaining somatosensory event-related potentials following orofacial skin stretch stimulation. The current method can be used to evaluate the contribution of somatosensory afferents to both speech production and speech perception.
Cortical processing associated with orofacial somatosensory function in speech has received limited experimental attention due to the difficulty of providing precise and controlled stimulation. This article introduces a technique for recording somatosensory event-related potentials (ERP) that uses a novel mechanical stimulation method involving skin deformation using a robotic device. Controlled deformation of the facial skin is used to modulate kinesthetic inputs through excitation of cutaneous mechanoreceptors. By combining somatosensory stimulation with electroencephalographic recording, somatosensory evoked responses can be successfully measured at the level of the cortex. Somatosensory stimulation can be combined with the stimulation of other sensory modalities to assess multisensory interactions. For speech, orofacial stimulation is combined with speech sound stimulation to assess the contribution of multi-sensory processing including the effects of timing differences. The ability to precisely control orofacial somatosensory stimulation during speech perception and speech production with ERP recording is an important tool that provides new insight into the neural organization and neural representations for speech.
Speech production is dependent on both auditory and somatosensory information. The auditory and somatosensory feedback occur in combination from the earliest vocalizations produced by an infant and both are involved in speech motor learning. Recent results suggest that somatosensory processes contribute to perception as well as production. For example, the identification of speech sounds is altered when a robotic device stretches the facial skin as participants listen to auditory stimuli1. Air puffs to the cheek that coincide with auditory speech stimuli alter participants' perceptual judgments2.
These somatosensory effects involve the activation of cutaneous mechanoreceptors in response to skin deformation. The skin is deformed in various ways during movement, and cutaneous mechanoreceptors are known to contribute to kinesthetic sense3,4. The kinesthetic role of cutaneous mechanoreceptors is demonstrated by recent findings5-7 that the movement-related skin strains are appropriately perceived as flexion or extension motion depending on the pattern of skin stretch6. Over the course of speech motor training, which is the repetition of specific speech utterance with concomitant facial skin stretch speech, articulatory patterns change in an adaptive manner7. These studies indicate that modulating skin stretch during action provides a method for assessing the contribution of cutaneous afferents to the kinesthetic function of the sensorimotor system.
The kinesthetic function of orofacial cutaneous mechanoreceptors has been studied mostly using psychophysiological methods7,8 and microelectrode recoding from sensory nerves9,10. Here, the current protocol focuses on the combination of orofacial somatosensory stimulation associated with facial skin deformation and event related potential (ERP) recording. This procedure has precise experimental control over the direction and timing of facial skin deformation using a computer-controlled robotic device. This allows us to test specific hypotheses about the somatosensory contribution to speech production and perception by selectively and precisely deforming facial skin in a wide range of orientations during both speech motor learning and directly in speech production and perception. ERP recording are used to noninvasively evaluate the temporal pattern and timing of the influence of somatosensory stimulation on orofacial behaviors. The current protocol then can evaluate the neural correlates of kinesthetic function and assess the contribution of the somatosensory system to both speech processing, speech production and speech perception.
To show the utility of the application of skin stretch stimulation to ERP recording, the following protocol focuses on the interaction of somatosensory and auditory input in speech perception. The results highlight a potential method to assess somatosensory-auditory interaction in speech.
The current experimental protocol follows the guidelines of ethical conduct according to the Yale University Human Investigation Committee.
1. Electroenchephalopgaphy (EEG) Preparation
2. Somatosensory Stimulation
Note: The current protocol applies facial skin stretch for the purpose of somatosensory stimulation. The experimental setup with the EEG system is represented in Figure 1. The details of the somatosensory stimulation device have been described in the previous studies1,7,12-14. Briefly, two small plastic tabs (2 cm wide and 3 cm height) are attached with double-sided tape to the facial skin. The tabs are connected to the robotic device using string. The robot generates systematic skin stretch loads according to experimental designs. The setup protocol for ERP recording is as follows:
3. ERP Recording
This section presents representative event-related potentials in response to somatosensory stimulation resulting from facial skin deformation. The experimental setup is represented in Figure 1. Sinusoidal stimulation was applied to the facial skin lateral to the oral angle (See Figure 3A as reference). One hundred stretch trials were recorded for each participant with 12 participants tested in total. After removing the trials with blinks and eye movement artifacts offline on the basis of the horizontal and vertical electro-oculography signals (over ±150 µV), more than 85% of trials were averaged. EEG signals were filtered with a 0.5-50 Hz band-pass filter and re-referenced to the average across all electrodes. Figure 2 shows the average somatosensory ERP from selected representative electrodes. In frontal regions, peak negative potentials were induced at 100-200 msec post stimulus onset followed by a positive potential at 200-300 msec. The largest response was observed in the midline electrodes. Different from the previous studies of somatosensory ERP15-18, there is no earlier latency (<100 msec) potentials. This temporal pattern is rather similar to the typical N1-P2 sequence following auditory stimulation19. In comparison between the corresponding pair of electrodes in left and right hemisphere, the temporal pattern is quite similar probably due to the bilateral stimulation.
Figure 1. Experimental setup. Please click here to view a larger version of this figure.
Figure 2. Event related potentials in response to somatosensory stimulation produced by facial skin stretch. The ERPs were obtained from representative electrodes. Please click here to view a larger version of this figure.
The first result shows how the timing of stimulation affects multisensory interaction during speech processing12. In this study, neural response interactions were found by comparing ERPs obtained using somatosensory-auditory stimulus pairs with the algebraic sum of ERPs to the unisensory stimuli presented separately. The pattern of auditory-somatosensory stimulations are represented in Figure 3A. Figure 3B shows the pattern of event-related potentials in response to somatosensory-auditory stimulus pairs (Red line). The black line represents the sum of individual unisensory auditory and somatosensory ERPs. The three panels correspond to the time lag between two stimulus onsets: 90 msec lead of the somatosensory onset (Left), simultaneous (Center) and 90 msec lag (Right). When somatosensory stimulation was presented 90 msec before the auditory onset, there is a difference between paired and summed responses (the left panel in Figure 3B). This interaction effect gradually decreases as a function of the time lag between the somatosensory and auditory inputs (see the change between the two dotted lines in Figure 3B). The results demonstrate that the somatosensory-auditory interaction is dynamically modified with the timing of stimulation.
Figure 3. Event-related potentials reflect a somatosensory-auditory interaction in the context of speech perception. This Figure has been modified from Ito, et al.12 (A) temporal pattern of somatosensory and auditory stimulations. (B) Event-related potentials for combined somatosensory and auditory stimulation in three timing conditions (lead, simultaneous, and lag) at electrode Pz. The red line represents recorded responses to paired ERPs. The dashed line represents the sum of somatosensory and auditory ERPs. The vertical dotted lines define an interval 160-220 msec after somatosensory onset in which differences between "pair" and "sum" responses are assessed. Arrows represent auditory onset. Please click here to view a larger version of this figure.
The next result demonstrates that the amplitude of the somatosensory ERP increases in response to listening to speech13. The pattern of somatosensory stimulation is the same as noted above. Figure 4 shows somatosensory ERPs, which are converted into scalp current density20 in off-line analysis, at electrodes (FC3, FC5, C3) over the left sensorimotor area. Somatosensory event-related potentials were recorded while participants listen to speech in the presence of continuous background sounds. The study tested four background conditions: speech, non-speech sounds, pink-noise and silent13. The results indicated the amplitude of somatosensory event-related potentials during listening to speech sounds was significantly greater than the other three conditions. There was no significant difference in amplitude for the other three conditions. Figure 4B shows normalized peak amplitudes in the different conditions. The result indicates that listening to speech sounds alters the somatosensory processing associated with facial skin deformation.
Figure 4. Enhancement of somatosensory event-related potentials due to speech sounds. The ERPs were recorded under four background sound conditions (Silent, Pink noise, Speech and Non-speech). This Figure has been modified from Ito, et al.13 (A) Temporal pattern of somatosensory event-related potentials in the area above left motor and premotor cortex. Each color corresponds to a different background sound condition. The ERPs were converted to scalp current density20. (B) Differences in z-score magnitudes associated with the first peak of the somatosensory ERPs. Error bars are standard errors across participants. Each color corresponds to different background sound conditions, as in Panel A. Please click here to view a larger version of this figure.
The studies reported here provide evidence that precisely controlled somatosensory stimulation that is produced by facial skin deformation induces cortical ERPs. Cutaneous afferents are known as a rich source of kinesthetic information3,4 in human limb movement5,6 and speech movement7,8,21. Stretching the facial skin in a manner that reflects the actual movement direction during speaking induces a kinesthetic sense similar to the corresponding movement. The current method combining precisely controlled skin stretch and ERP recordings can be used to investigate the neural basis of orofacial function during a wide range of speech behaviors.
Using mechanical stimulation and simultaneous EEG recording, it is important to monitor the ongoing signals for artifact. In particular, since the strings used to stretch the skin are located close to the EEG electrodes and cables, there is the possibility of electrical and motion artifacts being induced in the EEG signals. This artifact is distinguishable because of relatively large amplitude and lower frequency compared with the electrophysiological response. Before recording, the stimulation setup including the string configuration needs to be checked carefully to identify and eliminate any mechanical artifacts due to the stimulation. Although artifacts can be removed by post signal processing, such as filtering or independent component analysis22 similar to eye movement and blinking, cleaner signals are always more desirable.
The previous studies of somatosensory event-related potentials have mostly used brief somatosensory stimuli that were produced using mechanical23, electrical18 or laser nociceptive stimulation15. Somatosensory inputs arising from these kinds of stimulation are not associated with any particular articulatory motion in speech, and hence, they may not be suitable for investigating speech-related cortical processing. Möttönen, et al.17 had failed to show a change of magnetoenchalographic somatosensory potentials using simple lip tapping during listening to speech sounds. In contrast, deformation of the facial skin provides kinesthetic input similar to that which occurs in conjunction with speech articulatory motion21 and sensorimotor adaption7. These stimuli also interact with speech perceptual processing1,14. The somatosensory ERP from the current skin stretch perturbation is more suitable for the investigation of speech-related cortical processing than the other methods currently available for somatosensory stimulation. Several different characteristics were found between the current skin stretch stimulation and the previous methods. Further investigation including the source location is required.
Although deformation of the facial skin occurs to varying degrees during speech motion8, the skin lateral to the oral angle is densely innervated with cutaneous mechanoreceptors10,24 and may be predominantly responsible for the detection of skin stretch during speech. The skin at the corners of the mouth may be especially important for speech motor control and speech motor learning. The current approach is somewhat limited because the stretch of the skin can only be done in one direction and at one location per EEG session. Using a more complex skin deformation and evaluating multiple directions and/or multiple locations in one EEG session will provide further insight into the specific role of somatosensation in speech processing.
There are long-standing interests in speech communication studies concerning the nature of representations and processing in speech production and perception25-27. The discovery of mirror neurons28,29 reinforced the idea that motor functions are involved in speech perception. The involvement of the motor system (or the motor and premotor cortex) has also been investigated30-35 in the perception of speech sounds. Nevertheless, the link between speech production and perception is still poorly understood. Exploring possible somatosensory influences on speech perception can help us understand the neural bases of speech perception and production, and whether they overlap or link. The current technique for modulating somatosensory function has provided a new tool to investigate this important area of inquiry. The current technique has the additional advantage that it can be used in investigations of somatosensory function more generally and how it interacts with other sensory modalities in neural processing.
The authors have nothing to disclose.
This work was supported by National Institute on Deafness and Other Communication Disorders Grants R21DC013915 and R01DC012502, the Natural Sciences and Engineering Research Council of Canada and the European Research Council under the European Community’s Seventh Framework Programme (FP7/2007-2013 Grant Agreement no. 339152).
EEG recording system | Biosemi | ActiveTwo | |
Robotic decice for skin stretch | Geomagic | Phantom Premium 1.0 | |
EEG-compatible earphones | Etymotic research | ER3A | |
Software for visual and auditory stimulation | Neurobehavioral Systems | Presentation | |
Electrode gel | Parker Laboratories, INC | Signa gel | |
Double sided tape | 3M | 1522 | |
Disposable syringe | Monoject | 412 Curved Tip | |
Analog input device | National Instuments | PCI-6036E | |
Degital output device | Measurement computing | USB-1208FS |