Multi-Modal Signals for Analyzing Pain Responses to Thermal and Electrical Stimuli

Summary

This article focuses on the experimental elicitation of pain through heat (thermal) and electrical stimulation while recording physiological, visual, and paralinguistic responses. It aims at collecting valid multimodal data for analyzing pain based on its intensity, quality, and duration. 

Abstract

The assessment of pain relies mostly on methods that require a person to communicate. However, for people with cognitive and verbal impairments, existing methods are not sufficient as they lack reliability and validity. To approach this problem, recent research focuses on an objective pain assessment facilitated by parameters of responses derived from physiology, and video and audio signals. To develop reliable automated pain recognition systems, efforts have been made in creating multimodal databases in order to analyze pain and detect valid pain patterns. While the results are promising, they only focus on discriminating pain or pain intensities versus no pain. In order to advance this, research should also consider the quality and duration of pain as they provide additional valuable information for more advanced pain management. To complement existing databases and the analysis of pain regarding quality and length, this paper proposes a psychophysiological experiment to elicit, measure, and collect valid pain reactions. Participants are subjected to painful stimuli that differ in intensity (low, medium, and high), duration (5 s / 1 min), and modality (heat / electric pain) while audio, video (e.g., facial expressions, body gestures, facial skin temperature), and physiological signals (e.g., electrocardiogram [ECG], skin conductance level [SCL], facial electromyography [EMG], and EMG of M. trapezius) are being recorded. The study consists of a calibration phase to determine a subject’s individual pain range (from low to intolerable pain) and a stimulation phase in which pain stimuli, depending on the calibrated range, are applied. The obtained data may allow refining, improving, and evaluating automated recognition systems in terms of an objective pain assessment. For further development of such systems and to investigate pain reactions in more detail, additional pain modalities such as pressure, chemical, or cold pain should be included in future studies. Recorded data of this study will be released as the “X-ITE Pain Database”.

Introduction

Pain is a very personal and unpleasant sensation that is perceived differently by everyone. It lasts from seconds to months and may vary in its quality (throbbing, sharp, burning, etc.). If treated inadequately, pain influences physical and psychological functions of the body, reduces the quality of life, and bears the risk of becoming a chronic condition. In clinical care, the accurate assessment of pain intensity and quality is highly relevant to provide successful pain management^1,2. Gold standard methods for assessing pain, such as the visual analog scales (VAS), the numeric rating scale (NRS), or the McGill Pain Questionnaire³, rely on self-reports of patients and, thus, only work sufficiently with cognitively and verbally unimpaired persons. Consequently, all those established methods lack validity and reliability when it comes to neonates⁴, delirious, somnolent, sedated, or ventilated patients⁵, or people suffering from dementia^6,7. In addition to or as an alternative to self-report scales, methods to measure pain through observation by trained personnel (e.g., the Zurich Observation Pain Assessment⁸ or the Abbey Pain Scale⁹) have been developed in recent years. Nevertheless, even these tools suffer from limitations in reliability and validity, as even trained raters cannot guarantee an objective assessment. Furthermore, the application is often too time-consuming for clinical staff when pain assessment should be done on a regular basis. 

Several research teams have focused on developing automated pain recognizing systems, which allow for measuring pain by means of physiological, visual, and/or paralinguistic signal sets as new approaches for assessing and monitoring pain and its intensities objectively. Previous studies show promising results in detecting and differentiating pain^{10,11,12,13,16,17,18} or discriminating pain from basic emotions^14,15 based solely on one of the signal sets^{10,11,12,13,14,15} as well as on a combination/fusion^16,17,19 of the sets. The abovementioned modalities react almost autonomously to stressful stimuli such as pain. Using them has the advantage that they do not require a person’s ability to report her/his pain. Such individuals would greatly benefit from an objective pain recognition system which incorporates such modalities. Data sets consisting of reactions to elicited pain provide precious information for analyzing pain patterns and developing practical applications for detecting and monitoring pain. Amongst others, Walter et al.²⁰ created the “BioVid Heat Pain Database”, a multimodal database that is publicly available and provides data from short-time induced painful heat stimuli and corresponding psychophysiological and visual reactions. The “SenseEmotion Database” of Velana et al.²¹ includes biosignals, videos, and paralinguistic information from volunteers affected by phasic heat pain and emotional stimuli.

While these databases are well suited for examining pain reactions, they are mostly based on one specific pain model. As pain differs in its quality (supposedly depending on the pain model) and in its duration, it also may differ in its physiological, visual, and paralinguistic correlates. To the best of the authors’ knowledge, no multimodal studies or databases exist that combine two or more pain models and vary pain stimuli in intensity and duration in order to not only detect pain patterns but also distinguish between pain qualities.

This paper provides a protocol on how to conduct a complex psychophysiological experiment to elicit pain and simultaneously record physiological responses (e.g., ECG, EMG of Musculus trapezius, corrugator supercilii, and zygomaticus major, SCL) as well as video (e.g., facial expressions, body gestures, facial skin temperature) and audio data. Participants are stimulated with short (phasic) and longer lasting (tonic) heat and electrical pain stimuli that differ in intensity. A calibration phase prior to the experiment determines pain thresholds for each subject individually.

The study aims at collecting multimodal data for investigating pain (patterns) regarding intensity, quality and length by means of statistical methods, machine learning algorithms, etc. Additionally, the already collected data is planned to be published for academic research purposes under the name “X-ITE (Experimentally Induced Thermal and Electrical) Pain Database”. It may extend existing databases, such as BioVid Heat Pain and SenseEmotion^20,21, and contribute to the further development, improvement, and/or evaluation of automated pain recognition systems in matters of validity, reliability, and real-time recognition. 

The rest of the paper is organized in the following way. The protocol describes how to carry out the pain elicitation study step-by-step. Then, the representative results present the outcome of the experiment. Finally, the discussion covers critical steps, limitations, and benefits of the study followed by suggestions for future extensions.

Protocol

The study was conducted in accordance with the ethical guidelines laid down in the World Medical Association Declaration of Helsinki (ethical committee approval was granted: 196/10-UBB/bal) and approved by the ethics committee of the University of Ulm (Helmholtzstraße 20, 89081 Ulm, Germany).

1. Subject recruitment and selection

1. Recruit an equal number of healthy female and male subjects between 18 and 50 years of age through posters, handouts, local press advertisements, and social media to achieve a mostly general sample. Advertise the scientific benefit of the study and offer a monetary compensation. Provide telephone number or contact email address for further information.
   NOTE: Age-related effects in pain sensitivity are well reported²² and should be considered in the sample selection. To avoid confounding the results with age effects, we choose a younger group as considered by Lautenbacher et al.²².
2. Exclude potential subjects who meet any of the following criteria: suffering from chronic pain, depression, or a history of psychiatric disorders; having neurological conditions, headache syndrome, or cardiovascular disease; regularly taking pain medication or using painkillers directly before the experiment.

2. General preparations of the pain elicitation experiment

NOTE: The pain elicitation experiment consists of two temporally successive parts: the calibration part and the pain stimulation part. The calibration part determines a participant’s individual pain threshold and pain tolerance level in terms of thermal and electrical stimuli. The pain stimulation part performs the pain induction adjusted to the individual thresholds. Each part of the experiment takes place in a different room: the calibration room and the experimental room. The calibration room also serves as a monitoring room for the experimenter during the pain stimulation part (see Figure 1). 

Figure 1: Schematic representation of room setup. The right side shows the calibration/monitoring room where the calibration part takes place. Later on, it also serves as a signal monitoring room during the pain stimulation part, which follows the calibration part. The left side shows the experimental room where the pain stimulation part takes place. Both rooms are connected by a conduit pipe, which the thermode, the electrodes’ cable of the electrical stimulator and computer wires can be passed through. Please click here to view a larger version of this figure.

1. Welcome the arriving subject and lead her/him to the calibration room. Inform the participant in detail about the 3 h experiment and the possibility of termination at any time without any negative consequences. Obtain a written informed consent for the experiment and a written confirmation that none of the exclusion criteria apply. Prepare a receipt and the monetary compensation.
2. Use CE-marked stimulators that allow for inducing highly controlled thermal and electrical stimuli. Use analogue-to-digital converters that convert the analog thermal and electrical stimuli into digital signals to capture. Turn on the thermal and the electrical stimulator.
3. For the calibration and the main part of the experiment, employ appropriate software, which enables the manual and/or automatic triggering of thermal and electrical stimuli. Launch the software of the thermal and electrical stimulator. Prepare pencil and paper to write down stimuli intensities and the corresponding pain intensities reported by the participant.
4. Let the subject sit down comfortably on a chair with a right-sided armrest. Place a printed rating scale going from 0 to 100 in steps of 5 with, on the left, anchor point 0/no pain and, on the right, anchor point 100/intolerable pain in front of the subject. Instruct the subject to verbally rate the pain intensity of a stimulus immediately when asked to do so, using the scale provided. Accurately explain that only zero means “no pain” while 100 equals a stimulus that cannot be tolerated anymore.
   NOTE: In this study, “pain tolerance” is understood as the intensity of a stimulus that a subject cannot bear anymore, meaning here: cannot tolerate anymore. Thus, the anchor point 100 is marked as intolerable pain in contrast to a numerical rating scale used in clinical practice.

3. Calibration of electrical pain threshold and tolerance (parts 1 and 2)

NOTE: Only one experimenter should conduct the calibration part to minimize the social effects on pain sensitivity. Choose an experimenter with the same sex as the participant to minimize cross-sex effects on pain sensitivity²³. Part 1 determines pain threshold and tolerance in terms of short (phasic) electrical stimuli and part 2 in terms of longer lasting (tonic) electrical stimuli. Those values serve as a basis for calculating the phasic and tonic electrical pain stimuli applied in the pain stimulation part.

1. Clean the participant’s skin of the right index and middle finger with alcohol solution. Place one disposable Ag/AgCl electrode (skin contact size: 34 mm in diameter) on the upper side of the intermediate phalanx of the right index finger (anode) and another one on the upper side of the proximal phalanx of the right middle finger (cathode). Connect the electrodes to the electrical stimulator.
2. Ask the subject to rest their right arm comfortably on the armrest and provide them with instructions for the following procedure.
   NOTE: Here is an example of how to formulate the instruction for the electrical calibration part 1: “You are going to experience short electrical stimuli of different intensities. We start with a very low intensity. First, you are required to indicate the first time you feel low pain. This will be your ‘Pain Threshold’. In order to determine this threshold, I will start an electrical stimulus, and shortly before it ends, I’m going to say ‘Now.’ When this happens, you should immediately report whether the stimulus was painful or not by rating it on a scale from zero to one hundred. If the stimulus was not painful, please report ‘Zero.’ I will then go on with an increased intensity. After the first time you report a number greater than zero, I will reduce the intensity a few levels and we repeat the whole procedure until you indicate a number greater than zero again. We do this to validate the ‘Pain Threshold’. After that, I will slowly increase the intensity to the point where you rate the stimulus with ‘One hundred,’ meaning you cannot tolerate or stand the pain anymore. This will be your ‘Pain Tolerance Level’. Again, to validate this level, I will go back a few levels and we will repeat the procedure until you report ‘One hundred’ for the second time. As soon as you say ‘One hundred,’ I will immediately stop the stimulus.”
3. Begin electrical calibration part 1 by starting a stimulus of 0.5 mA (400 V) with a duration of 5 s by clicking the start button of the electrical stimulator (software).
   NOTE: Each electrical stimulus of the electrical calibration part 1 has a duration of 5 s. The stimulus consists of 100 single electroshocks of 2 ms duration, each distributed equally over the 5 s.
   CAUTION: The electrical calibration always starts with 0.5 mA and has a cutoff at 25 mA in order to prevent unconsciousness and life threatening situations.
4. Say “Now” at the 4th second of the stimulus. Write down the number the subject reports for the corresponding stimulus intensity. Conduct a pause of 10 s long.
5. If the subject reports zero, increase the intensity by 0.5 mA, start the stimulus, and go back to step 3.4. Otherwise, reduce the intensity by 1.5 mA (minimum: 0.5 mA), start the stimulus, and go back to step 3.4. When the subject reports a number greater than zero for the second time, calculate the mean of the two intensities corresponding to zero, write it down and mark it as “phasic Electrical Pain Threshold” (pEPTh). Afterward, continue with step 3.6.
6. Increase the current intensity by 0.5 mA and start the stimulus.
7. Say “Now” after 4 s from starting the stimulus. Write down the number the subject reports for the corresponding stimulus intensity. Pause for 10 s.
8. If the subject reports a value below 100, increase the intensity by 0.5 mA, start the stimulus, and go back to step 3.7. Otherwise, reduce the intensity by 1.5 mA (minimum: 0.5 mA), start the stimulus, and go back to step 3.7. When the subject reports 100 for the second time, calculate the mean of the two intensities corresponding to 100, write it down, and mark it as “phasic Electrical Pain Tolerance” (pEPTo). Continue with step 3.9.
9. Inform the participant about part 2 of the electrical calibration.
   NOTE: This is a possible instruction for electrical calibration part 2: “Again we are starting with a low intensity, but this time, the stimuli will be longer. I am going to say ‘Now’ two times, right after the beginning and shortly before the ending of a stimulus. Every time I say ‘Now,’ you report a number as you did in the first part. After the first time you report a number greater than zero, I will reduce the intensity a few levels and we repeat the whole procedure for validation purposes, until you indicate a number greater than zero again. Following that, I will slowly increase the intensity to the point where you rate the stimulus with ‘One hundred.’ Again, to validate this threshold, I will go back a few levels and we will repeat the procedure until you report ‘One hundred’ for the second time. As soon as you say ‘One hundred,’ I will immediately stop the stimulus.”
10. Begin electrical calibration part 2 by starting a stimulus of 0.5 mA (400 V) with a duration of 10 s by clicking the start button of the electrical stimulator (software).
    NOTE: Each electrical stimulus of the electrical calibration part 2 has a duration of 10 s. The stimulus consists of 200 single electrical shocks of 2 ms of duration each, distributed equally over the 10 s.
11. Say “Now” after 1 s from starting the stimulus. Write down the number the subject reports for the corresponding stimulus intensity. Say “Now” 1 s before the stimulus ends and, again, write down the number the subject now reports for the corresponding stimulus intensity. Pause for 10 s.
12. If both of the subject’s reports are zero, increase the intensity by 0.5 mA, start the stimulus, and go back to step 3.11. Otherwise, reduce the intensity by 1.5 mA (minimum: 0.5 mA), start the stimulus, and go back to step 3.11. When the subject reports a number greater than zero for the second time, calculate the mean of the two intensities corresponding to zero, write it down, and mark it as “tonic Electrical Pain Threshold” (tEPTh). Afterward, continue with step 3.13.
13. Increase the current intensity by 0.5 mA and start the stimulus.
14. Say “Now” after 1 s from starting the stimulus. Write down the number the subject reports for the corresponding stimulus intensity. Say “Now” 1 s before the stimulus ends. Again, write down the number the subject now reports for the corresponding stimulus intensity. Pause for 10 s.
15. If both of the subject’s reports are below 100, increase the intensity by 0.5 mA, start the stimulus, and repeat steps 3.14‒3.15. Otherwise if any of the subject’s reports is exactly 100, reduce the intensity by 1.5 mA (minimum: 0.5 mA), start the stimulus, and repeat steps 3.14‒3.15. When the subject reports 100 for the second time, calculate the mean of two intensities corresponding to 100, write it down, and mark it as “tonic Electrical Pain Tolerance” (tEPTo). Continue then with step 3.16.
16. Disconnect the electrodes from the electrical stimulator and remove both Ag/AgCl electrodes from the participant’s fingers. Clean the fingers with alcohol solution to wash off the electrode gel remains.

4. Calibration of thermal pain threshold and tolerance (parts 1 and 2)

NOTE: The thermal pain calibration is divided into two parts. Part 1 determines pain threshold and tolerance in terms of short (phasic) thermal stimuli and part 2 does so in terms of longer lasting (tonic) thermal stimuli. Those values serve as basis for calculating the phasic and tonic thermal pain stimuli applied during the pain stimulation part.

1. Apply a 30 mm x 30 mm thermode to the upper side of the subject’s right forearm, about 30 mm proximal to the wrist, by a hook-and-loop fastener strap. Ask the subject to rest their right arm comfortably on the armrest.
   NOTE: A thermode is the probe/part of the thermal stimulator which is attached to the subject’s skin and induces the actual thermal stimulus.
2. Inform the subject about the procedure of the thermal calibration part 1.
   NOTE: A formulation for the instruction for thermal calibration part 1 might be: “You are now going to experience short but constant thermal stimuli of different intensities. We start with a temperature just above your body temperature. A thermal stimulus will start and shortly before it ends, I will say ‘Now.’ When this happens, you should quickly report whether the stimulus was painful or not by rating it on a scale from zero to one hundred, like you did in the electrical calibration part. Similarly, if the stimulus was not painful, please report ‘Zero.’ I will then go on with an increased intensity. There is always a pause of a few seconds between two stimuli. The calibration phase is finished when you report ‘One hundred’ or the cutoff temperature is reached.”
3. Begin thermal calibration part 1 by starting a stimulus of 39 °C with a duration of 5 s by clicking the start button of the thermal stimulator (software).
   CAUTION: The thermal calibration part 1 has a cutoff temperature of 50 °C in order to prevent skin burns.
   NOTE: Each thermal stimulus of the thermal calibration part 1 has a duration of 5 s.
4. Say “Now” at the 4th second of the stimulus. Write down the intensity number the subject gives for the corresponding temperature. Pause for 10 s.
5. If the indication of the subject is below 100, increase the temperature by 1 °C, start the stimulus, and repeat steps 4.4‒4.5. Otherwise, if the indication is 100, or the cutoff temperature of 50 °C is reached, terminate thermal calibration part 1 by continuing with the next step.
6. Check the noted numbers and mark the first temperature with a corresponding intensity number greater than zero as “phasic Heat Pain Threshold” (pHPTh). Mark the temperature with an indication of 100 as “phasic Heat Pain Tolerance” (pHPTo).
   NOTE: If the subject reports a number below 100 at the cutoff temperature (50 °C), mark 50 °C as pHPTo.
7. Inform the participant about part 2 of the thermal calibration.
   NOTE: Here is an exemplary formulation for the instruction for this thermal calibration phase: “We start with a temperature above your body temperature, but this time the thermal stimuli will be longer. I will say ‘Now’ two times: right after the beginning and shortly before the ending of the stimuli. Every time I say ‘Now,’ you report a number that equals your pain experience. If the stimulus was not painful, please report ‘Zero.’ I will then go on with an increased temperature. There is always a pause of a few seconds after each stimulus. The calibration phase is finished when you report ‘One hundred’ or the cutoff temperature is reached.”
   CAUTION: The thermal calibration part 3 has a cutoff temperature of 49.5 °C in order to prevent skin burns, due to the longer stimulus duration.
8. Begin thermal calibration part 2 with a stimulus of 39 °C for 10 s long by clicking the start button of the thermal stimulator (software).
   NOTE: Each thermal stimulus of the thermal calibration part 2 has a duration of 10 s.
9. Say “Now” after 1 s from starting the stimulus. Write down the intensity number the subject reports for the corresponding temperature. Say “Now” 1 s before the stimulus ends. Again, write down the intensity number the subject now reports for the corresponding temperature. Pause for 60 s.
10. If both indications of the subject are below 100, increase the temperature by 1 °C (exception: see note below), start the next stimulus, and repeat step 4.16. Otherwise, if any of the reports is exactly 100 or the cutoff temperature of 49.5 °C is reached, terminate thermal calibration part 2 by continuing with the following step.
    NOTE: In this calibration phase, the temperature steps are: 39 °C, 40 °C, 41 °C, 42 °C, 43 °C, 44 °C, 45 °C, 46 °C, 47 °C, 48 °C, 49 °C, 49.5 °C.
11. Check the noted numbers and mark the first temperature at which at least one of the corresponding numbers is greater than zero as “tonic Heat Pain Threshold” (tHPTh). Mark the temperature with a first noted report of 100 as “tonic Heat Pain Tolerance” (tHPTo).
    NOTE: If the subject reports both numbers below 100 at the cutoff temperature (49.5 °C), mark 49.5 °C as tHPTo.
12. Remove the thermode from the participant’s underarm. Ask the subject whether they need a short break and/or want to use the sanitary facility. To perform a check if the participant is suitable for the pain stimulation part in terms of thermal stimuli, follow the instructions described in Supplementary File 1.

5. Preparation of the pain stimulation experiment

1. Conduct the pain stimulation in a camera-monitored, temperature-controlled, and low-noise experimental room next to the calibration/monitoring room (see Figure 1). Connect both rooms via a conduit pipe (80 mm in diameter). 
   NOTE: Camera monitoring provides information about the subject’s health status and allows for rapid intervention in case of sudden unconsciousness or circulatory collapse.
2. Set up an examination couch on which the subject lies during the experiment. Place it with the long side next to a wall near the conduit pipe. Provide a pillow for the head.
3. Attach a mirror to the wall next to the examination couch where the head of the subject will rest.
4. For capturing physiological data (ECG, 3 x EMG, and SCL), audio, videos (frontal and side view of the face, the facial skin temperature, and a full body view), and thermal and electrical stimulator outputs during the experiment, use appropriate recording computers, software, and recording devices (a biosignal recorder, three high-resolution color cameras, one thermal camera, and one directional microphone). 
   1. Develop a solution for synchronizing the recorded modalities. It may comprise a hardware triggering of devices, a recording of trigger signals by the biosignal and audio recorders, computer clock synchronization (e.g., via NTP) and a recording of timestamps along with the data streams, and postprocessing of the recorded data streams to compensate for temporal offsets and clock drift.
5. Install the full body view camera in a way that it captures the whole body of the subject. Install the frontal face view camera approximately 1 m above the head of the participant. Mount the microphone on the left side and the thermal camera on the right side, next to the frontal face camera. Attach the side view camera to the ceiling. Adjust it to a point where it is able to record one side of the subject’s face, as well as the opposite side reflected in the mirror (see Figure 2).

Figure 2: Schematic representation of camera and microphone setup. The frontal face camera, thermal camera and microphone are set up approx. 1 m above the head of the participant. A side view camera captures both sides of the face with the help of a mirror. A body view camera mounted to the wall allows for the recording of body movement. Please click here to view a larger version of this figure.

NOTE: Due to a small experimental room, combining a side view camera with a mirror is a very elegant solution to capture both sides of the subject’s face with just one camera.

6. Duplicate the graphical output of the biosignal recording computer to a computer monitor set up in the calibration/monitoring room. 
7. Set up active PC speakers in the calibration/monitoring room. Connect them via the conduit pipe with the audio-recording computer. Make sure to hear the participant via the microphone in case they need assistance during the experiment.
8. Pass the thermode and the electrodes’ cable of the electrical stimulator through the conduit pipe to the experimental room. Put rolled up acoustic foam in both sides of the conduit pipe (or a similar material that absorbs sound).
   NOTE: Leave the thermal stimulator in the calibration/monitoring room. It contaminates the audio signal recording due to regularly starting its internal ventilator in order to cool down. 
9. Set the recording sampling rates as follows: (a) audio at 44.1 kHz; (b) frontal and side view cameras at 25 Hz; (c) full body view camera at 30 Hz; (d) thermal camera at 120 Hz; (e) SCL, EMG, and ECG at 1,000 Hz. Save all settings.
10. Get a cold gel pack (100 mm x 100 mm) and put it into a freezer. Prepare a 200 mm x 200 mm hygienic nonwoven towel or something similar (e.g., a thin paper towel) and an ointment.
11. Calculate 12 individual stimulus intensities, six each for the heat and electrical pain induction, as follows: (a) phasic electrical pain intensity 3 (pE₃) = 90% of pEPTo; (b) phasic electrical pain intensity 2 (pE2) = (pE3 + pEPTh)/2; (c) phasic electrical pain intensity 1 (pE₁) = pEPTh; (d) tonic electrical pain intensity 3 (tE₃) = 90% of tEPTo; (e) tonic electrical pain intensity 2 (tE₂) = (tE₃ + tEPTh)/2; (f) tonic electrical pain intensity 1 (tE₁) = tEPTh; (g) phasic heat pain intensity 3 (pH₃) = pHPTo - 0.5 °C if the subject reported 100 for pHPTo—otherwise, pH₃ = pHPTo; (h) phasic heat pain intensity 2 (pH₂) = (pH₃ + pHPTh)/2; (i) phasic heat pain intensity 1 (pH₁) = pHPTh; (j) tonic heat pain intensity 3 (tH₃) = tHPTo - 0.5 °C if the subject reported 100 for tHPTo—otherwise, tH₃ = tHPTo; (k) tonic heat pain intensity 2 (tH₂) = (tH₃ + tHPTh)/2; (l) tonic heat pain intensity 1 (tH₁) = tHPTh.
12. Enter the values of the phasic electrical (pE₁‒pE₃) and heat pain intensities (pH₁‒pH₃) and tonic electrical (tE₁‒tE₃) and heat pain intensities (tH₁‒tH₃)—calculated in step 5.11 based on the calibration performed as per sections 3 and 4—into the software of the thermal and electrical stimulator. Set the baseline (no pain) temperature to 32 °C and the temperature rate of increase to 8 °C/s. Save all settings.
13. Use a scripting language-based computer software, which communicates with the thermal and electrical stimulator. Make sure it allows for controlling and triggering pain stimuli based on a pain elicitation script.
    NOTE: A pain elicitation script triggers the randomized pain stimuli and controls timing and duration. In this study, the software of the thermal stimulator provides the possibility for preparing a pain elicitation script. The software triggers thermal stimuli automatically and sends signals when an electrical stimulus shall be triggered. Electrical stimuli are triggered by a script prepared in a second software.
14. Prepare the pain elicitation script (see Figure 3) as follows. Set the number of each phasic stimulus intensity (pE₁, pE₂, pE₃, pH₁, pH₂, and pH₃) to 30 and the number of each tonic stimulus intensity (tE₁, tE₂, tE₃, tH₁, tH₂, and tH₃) to 1. Set the duration of each phasic stimulus to 5 s and the duration of each tonic stimulus to 60 s. Randomize the order of all stimuli. Randomize the pauses between the phasic stimuli to 8–12 s. Set the pauses after the tonic stimuli to 300 s. Save all settings.
    NOTE: Due to the longer duration, the number of the different tonic heat stimuli is set to 1 to avoid skin burns. All pauses after tonic stimuli must be 300 s in order to allow physiological signals to return to baseline and, thereby, not contaminate subsequent signals. 

Figure 3: Graphical illustration of the pain stimulation part. (A) Exemplary pain elicitation script with randomized phasic (blue) and tonic (red) pain stimuli. (B) Excerpt from the pain elicitation script above: Three phasic stimuli with a duration time of 5 seconds and subsequent pauses. The duration of pauses varies between 8 and 12 seconds. (pH₁, pH₂, pH₃ = phasic heat pain with intensity 1, 2, 3; tH₁, tH₂, tH₃ = tonic heat pain with intensity 1, 2, 3; pE₁, pE₂, pE₃ = phasic electrical pain with intensity 1, 2, 3; tE₁, tE₂, tE₃ = tonic electrical pain with intensity 1, 2, 3; s = seconds). Please click here to view a larger version of this figure.

6. Pain stimulation

1. Lead the participant to the experimental room and tell them about the upcoming procedure. Explain the experimental setup and the functionalities of the instruments. Inform the subject again about the possibility to abort the experiment anytime by either pressing a provided emergency button or asking to stop.
2. Ask the subject to lie down comfortably on the examination couch. Instruct them to keep lying on their back during the experiment.
3. Clean all skin areas where the electrodes will be attached with alcohol solution. Remove any dead skin cells on the surface of the left cheek, behind the left ear, and above the left eyebrow with abrasive gel. Reclean these areas with an alcohol solution.
4. For the measurement of the SCL, attach two pregelled, nonpolarizable Ag/AgCl electrodes to the underside of the distal phalanx of the right index and middle finger by Velcro straps. Make sure the straps are not too tight. They are too tight if the subject reports a throbbing sensation in their fingertips.
5. To record the ECG, use three pregelled, adhesive Ag/AgCl snap electrodes with circular contact areas (34 mm in diameter). Place one electrode (cathode) on the chest, approximately 6 cm below the right collarbone. Place the second one (anode) on the left ninth and tenth rib. Attach the third electrode (ground/reference) to the right-side waist next to the pelvic bone.
6. To record the EMG of M. trapezius, also use three pregelled, adhesive Ag/AgCl snap electrodes with circular contact areas (34 mm in diameter). Place two electrodes (cathode and anode) side by side on the trapezius muscle left of the neck. Place the third one (reference) below on the left collarbone.
7. Use six reusable, shielded Ag/AgCl electrodes with 4 mm in recording diameter to measure the EMGs of M. corrugator supercilii and M. zygomaticus major. Fill the cavities of the electrodes with electrolyte gel. 
   1. Attach the electrodes by means of double-sided adhesive collars as follows: for the corrugator supercilii, place one electrode (anode) directly above the left eyebrow, next to the (left) glabella line. Place the second electrode (cathode) 1 cm lateral to the first one. 
   2. Attach a third electrode (reference) to the middle of the frontal bone just below the hairline. For the zygomaticus major, draw an imaginary line from the left oral commissure to the left earlobe. Place one electrode (anode) slightly below the middle of the line and a second one (cathode) 1 cm medial next to it. Attach the third electrode (reference) to the left mastoid. Connect all electrodes to the corresponding inputs of the biosignal recording device.
8. Perform a visual check by means of the biosignal recording software if all physiological signals are of good/excellent quality. Ask the subject to move certain muscles and check the respective signal. Adjust/improve any unsatisfying signals by appropriate means/actions.
9. Place one Ag/AgCl electrode (34 mm diameter) on the upper side of the intermediate phalanx of the left index finger (anode) and another one on the upper side of the proximal phalanx of the left middle finger (cathode). Fixate the electrodes with medical tape as subjects might sweat during the procedure, reducing the adhesiveness of the electrodes. Connect the electrodes to the electrical stimulator.
10. Apply the thermode to the upper side of the subject’s left forearm about 30 mm proximal to the wrist by a Velcro strap. Ensure that the strap does not constrict the skin.
11. Start all cameras. Ensure that the participant is perfectly visible in the camera images. If necessary, ask the subject to adjust the position. Especially take care of facial cameras with a small field of view. Ideally, the face should be in the center of the image to reduce risk that the participant moves the head out of the field of view during the experiment.
12. Check if microphone is on and the recording volume is satisfactory.
13. Ask the subject if she has any further questions and if she is ready for the experiment. Instruct her to act completely naturally and not to suppress and exaggerate any pain reactions during the experiment.
14. Start all recording devices (cameras, microphone, biosignal recorder) complying with requirements for data synchronization. 
15. Leave the experimental room and enter the calibration/monitoring room. Wait 5 minutes in order to allow physiological signals of the subject to normalize. Run the pain elicitation script.
16. Carefully monitor the subject and the progress of the pain stimulation part. Write down any timestamps of obvious unusual/unnatural behavior, technical problems or biosignal artifacts due to extreme movement, coming-off of electrodes, etc.
17. After the ending of the pain elicitation script, stop all recording devices. Save/export all data in a preferred format. Turn off the thermal and electrical stimulator.
18. Check if the participant is alright and detach all electrodes and the thermode. Clean all skin areas with alcohol solution to remove electrode gel remains.
19. Get the cold gel pack from the freezer and wrap it into the hygienic non-woven towel (or something similar, e.g. thin paper towel). Ask the participant to apply it for at least 5 minutes to the skin area where the thermode was placed. 
20. Offer the subject the opportunity to have their individual pain levels demonstrated and explained.
21. Apply ointment to the skin area where the thermode was placed.
    NOTE: Cold gel pack and ointment is used to minimize (a potential) redness and irritation of the skin.
22. Hand over the monetary compensation and have it acknowledged with a receipt. Provide contact information in case of emerging issues. Thank the participant and say goodbye.
23. Remove the acoustic foam from both sides of the conduit pipe. Pass the thermode and the electrodes’ cable of the electrical stimulator back to the calibration/monitoring room. Dispose all disposable electrodes, clean all reusable electrodes of gel remains and clean the examination couch with a suitable surface sanitizer. Put the cold gel pack back into the freezer.

Results

Pain is perceived differently by any person and may express itself diversely in facial expressions, paralinguistic and/or physiological signals. The design of this study is suitable to analyze pain responses in numerous ways with respect to the underlying aims. The obtained data may allow answering research questions, such as: Are there specific pain response patterns? Do they differ regarding pain model and duration?

A total of 134 subjects participated in our experiment. The sex ratio was 50/50. We divided them in the following age groups: 1) 18-29 years (N = 49, 23 men, 26 women), 2) 30-39 years (N = 45, 23 men, 22 women), 3) 40-50 years (N = 40, 21 men, 19 women). The average age of all subjects was 31.4 (SD = 9.7), of all men = 33.4 (SD = 9.3) and of all women = 32.9 (SD = 10.2) years. The study took place at the Department of Medical Psychology of the University of Ulm, Germany.

The main outcome of this protocol is a data set of audio, video and psychophysiological signals reflecting the subjects’ responses to pain stimuli. Table 1 provides a general overview on the technical features of the recorded signals and on the numbers of induced pain stimuli in the study. 

Technical Features
Signal:	Sampling Rate:		Attributes:
Audio	44100 Hz		Mono, MP3 320 kbps
Camera 1 (face, frontal view)	25 Hz		Color video: resolution 1384 x 1032, HEVC encoded with libx265 (CRF 16, preset medium)
Camera 2 (face, side view)	25 Hz		Color video: resolution 1620 x 840, HEVC encoded with libx265 (CRF 16, preset medium)
Body Camera	ca. 30 Hz		Color video: resolution 1500 x 600, HEVC encoded with libx265 (CRF 16, preset medium); Depth video: resolution 500 x 200, lossless encoding
Thermal Camera	ca. 120.8 Hz		Surface temperature video: resolution 120 x 160, grayscale MPEG-4-AVC encoded with  libx264 (CRF 0,  preset veryfast), encoded temperature range 26.5-52.0 °C (steps of 0.1)
ECG	1000 Hz		Hardware filtered via BioPac: 35 Hz LP, 0.5 Hz HP, 50 Hz notch filter
SCL	1000 Hz		Hardware filtered via BioPac: 10 Hz LP, no HP, no notch filter
EMG M. trapezius	1000 Hz		Hardware filtered via BioPac: 500 Hz LP, 10 Hz HP, no notch filter
EMG M. corrugator supercilii	1000 Hz		Hardware filtered via BioPac: 500 Hz LP, 10 Hz HP, no notch filter
EMG M. zygomaticus major	1000 Hz		Hardware filtered via BioPac: 500 Hz LP, 10 Hz HP, no notch filter
Stimuli	Thermal		Electrical
Subjects:	Phasic Stimuli (5 s):	Tonic Stimuli (60 s):	Phasic Stimuli (5 s):	Tonic Stimuli (60 s):
Per subject	90 (30 per intensity)	3 (1 per intensity)	90 (30 per intensity)	3 (1 per intensity)
All (N = 134)	12060 (4020 per intensity)	402 (134 per intensity)	12060 (4020 per intensity)	402 (134 per intensity)
Men (n = 67)	6030 (2010 per intensity)	201 (67 per intensity)	6030 (2010 per intensity)	201 (67 per intensity)
Women (n = 67)	6030 (2010 per intensity)	201 (67 per intensity)	6030 (2010 per intensity)	201 (67 per intensity)

Table 1: Technical features and number of induced stimuli. The upper half (Technical Features) shows the sampling rates and attributes of the specific signals. The lower half (Stimuli) shows the numbers of the induced specific (thermal/electrical) pain stimuli for one subject, for all subjects and for each gender. (MP3 = Moving Picture Experts Group Layer-3 Audio, kbps = kilobits per second, HEVC = High Efficiency Video Coding, CRF = Constant Rate Factor, MPEG-4-AVC = Motion Picture Experts Group Layer-4 Video Advanced Video Coding, Hz = Hertz, °C = degrees Celsius, s = seconds, ECG = Electrocardiogram, SCL = Skin Conductance Level, EMG = Electromyography, LP = low-pass filter, HP = high-pass filter, M. = Musculus).

A secondary outcome concerning the calibration phase of the study is presented in Table 2. It shows the mean stimulation temperatures and currents of pain intensities 1 and 3 (as calculated in step 5.11 of the protocol) for all subjects and additionally for the male and female subgroup. 

Stimuli	Thermal [in °C] mean (SD)				Electrical [in mA] mean (SD)
Subjects	pH1	pH3	tH1	tH3	pE1	pE3	tE1	tE3
All (N = 134)	44.03 (2.25)	49.17 (1.20)	42.50 (2.14)	47.76 (1.02)	1.63 (0.94)	5.64 (2.72)	1.69 (1.12)	5.70 (2.59)
Men (n = 67)	44.56 (2.18)	49.48 (0.89)	43.11 (1.98)	47.93 (1.04)	1.94 (1.01)	6.83 (3.02)	1.96 (1.16)	6.90 (2.72)
Women (n = 67)	43.51 (2.74)	48.87 (1.39)	41.89 (2.14)	47.59 (0.98)	1.32 (0.75)	4.45 (1.70)	1.43 (1.01)	4.51 (1.80)

Table 2: Mean stimulation temperatures and currents of pain intensities 1 and 3. (pH₁, pH₃ = phasic heat pain with intensity 1, 3; tH₁, tH₃ = tonic heat pain with intensity 1, 3; pE₁, pE₃ = phasic electrical pain with intensity 1, 3; tE₁, tE₃ = tonic electrical pain with intensity 1, 3; °C = degrees Celsius; mA = milliampere, SD = standard deviation).

If all steps of the protocol are conducted carefully and no technical problems occur (in terms of computer or recording device crashes, etc.), a successful outcome may look similar as depicted in Figure 4. All signals are of high quality and not affected by external sources of interferences. The participant is clearly visible in every camera. 

Figure 4: Example data from a successful experiment. The figure depicts recorded signals a few seconds before, during and after an intense pain stimulus. All signals are non-filtered and synchronized in time. For clarity, only representative screenshots of the video signals are shown here. (EMG = Electromyography, SCL = Skin Conductance Level, ECG = Electrocardiogram, M. = Musculus, s = seconds). Please click here to view a larger version of this figure.

However, unexpected incidents may cause the data to become noisy or corrupted. Besides computer or recording device crashes, the coming-off of electrodes (especially reusable electrodes with small diameter which are attached by means of double-sided adhesive collars) mostly leads to unusable signals. As an example for a sub-optimal data set, Figure 5 shows the moment when an EMG electrode comes off and renders the corresponding signal useless. 

Figure 5: Example data from a sub-optimal experiment. The red circle indicates the time one of the EMG electrodes (M. zygomaticus major) fell off the subject’s cheek. This might have been due to sweat or head movement. From this moment on, the signal was lost. (EMG = Electromyography, SCL = Skin Conductance Level, ECG = Electrocardiogram, M. = Musculus, s = seconds). Please click here to view a larger version of this figure.

Due to ethical guidelines, the maximum intensities of thermal and electrical stimuli had to be restricted. Regarding thermal calibration control part (see supplementary file), 37 subjects (31 men, 6 women) reached the given cutoff of 50.5 °C (ratio = 37/134 = 27.61 %). As for thermal calibration part 1, 60 participants (39 men, 21 women) reached the cutoff of 50.0 °C (ratio = 60/134 = 44.78 %) and concerning part 2, 57 persons (37 men, 20 women) reached the cutoff of 49.5 °C (ratio = 57/134 = 42.54 %). The cutoff for both electrical calibration parts was 25 mA. None of the 134 subjects reached it.

As we plan to publish the data (see next paragraph), the data sets of participants who have reached the cutoffs will additionally be marked and their subjective pain ratings for the corresponding cutoffs will be included.

We would like to point out that the main focus of the protocol is obtaining multi-modal signals for analyzing thermal and electrical pain. Therefore, no other results are discussed here. After checking and excluding data sets due to missing data or rejected written consent for data sharing, the data sets of this study will be made available under the name “X-ITE Pain Database”. For further information on when and how to obtain the X-ITE Pain DB please visit https://github.com/philippwerner/pain-database-list.

Supplementary File 1. Please click here to download this file.

Discussion

The presented protocol focuses on the experimental elicitation of thermal (heat) and electrical pain while recording physiological, visual and paralinguistic signals. This novel approach, combining two pain models with different stimuli intensities and two different stimuli durations (phasic and tonic), offers a broad perspective about the psychophysiological patterns and expressions of pain. However, for the realization of this protocol several steps need to be considered. 

In general, if working with pain stimuli it is crucial to ensure the safety of the subjects. All pain stimuli have to be highly controlled and should only be carried out by experienced experimenters. 
Furthermore, for recording and collecting reliable and high quality data, the proper attachment of devices (electrodes), the perfect functioning of recording devices and a smooth communication between computers is highly recommended. All sources of interferences should be eliminated or reduced to a minimum. To guarantee consistency between participants, it is important to provide standardized instructions and unvarying experimental conditions. 

According to our experience, finding suitable participants who meet all criteria and are willing to receive numerous painful stimuli, takes a long time and is quite challenging. In addition to that, the monetary compensation has to be high enough to attract subjects to the study. Especially persons between 30 and 50 years are hard to find. This may be because the experiment is too long (ca. 4 hours, including arrival and departure) and they have to take half a day off from work.

Because the safety of the participants is of top priority, pain induction may need to be restricted. Due to ethical guidelines, the stimulus intensities must not exceed certain levels to prevent burns and unconsciousness in terms of thermal and electrical pain induction, respectively. A general cutoff of intensities may result in a ceiling effect as some subjects may reach the intensity limits before feeling intolerable pain. In this study, approximately 42% (considering thermal calibration part 1 and 2) of the participants reached the thermal cutoffs (see representative results). As they did not reach their “real” pain tolerances, their physiological responses to the highest thermal stimuli might behave differently in contrast to physiological responses of subjects who reached them. If so, mixing these two groups could influence classification results in terms of pain recognition.

An important point to address is the pain modalities in this experiment. Participants are subjected only to thermal and electrical pain stimuli (due to the fact that these are highly controllable in an experimental setting). Thus, if examining pain patterns regarding quality, findings may not translate to other pain modalities such as pressure, chemical or visceral pain.
The same consideration on transferability of results applies to the study sample. The protocol is ethically restricted to healthy adults. For example, it does not include children or cognitively and verbally impaired persons. Furthermore, in our study only European people participated. Also here, analytical results may not apply to groups not considered in this experiment.

Another limitation may concern the Hawthorne effect²⁴: The subjects are aware that they are being filmed/observed in the study. This might change their behavior.

Compared to existing pain databases, the protocol provides significant advantages for analyzing pain response patterns as it combines two pain models and two time courses (phasic and tonic): Besides the intensity and duration of pain, it also considers the quality of pain. As thermal pain is described differently than electrical pain (e.g., burning vs. sharp), it may also differ in the pain reactions. If so, those findings could link a pain response pattern to the underlying source of pain. Furthermore, the study is multi-modal to widen the range of pain investigation opportunities: Employing 5 psychophysiological signals, 2 face (front/side) camera signals, 1 body view camera signal, 1 thermal camera and 1 audio signal, pain may be analyzed and assessed more precisely. 

For a more complex investigation of pain response patterns, future extensions of this method should include more biosignals such as electroencephalography (EEG), body temperature and respiration. It would also be of great benefit to employ controlled pressure as a further pain model. Researchers aiming at automatic pain recognition via data gathered with this protocol should further test promising machine learning models with clinical control groups. 

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
PATHWAY Model ATS	Medoc Ltd., Ramat Yishai, Israel		Thermal Stimulator
30 mm x 30 mm ATS Thermode	Medoc Ltd., Ramat Yishai, Israel		Thermode
PATHWAY Software Arbel 6.3.7.22.1	Medoc Ltd., Ramat Yishai, Israel		Thermal Stimulator Software
Digitimer DS7A Current Stimulator	Digitimer Ltd., Hertfordshire, UK		Electrical Stimulator
Inquisit 5	Millisecond Software, Seattle, WA, USA		Software for triggering electrical stimuli
Analogue-To-Digital Converter	Wissenschaftliche Werkstatt Elektronik, University of Ulm, Ulm, Germany	custom built
BIOPAC MP150 System	BIOPAC Systems, Inc., Goleta, CA, USA		Biosignal Recording Hardware
AcqKnowledge Software 4.1.1	BIOPAC Systems, Inc., Goleta, CA, USA		Biosignal Recording Software
NTG-2 Dual Powered Directional Condenser Microphone	RØDE Microphones, Silverwater, Australia		Audio Recording Microphone
Kinect v2	Microsoft, Redmond, WA, USA		Body View Camera
AV Pike F-145C	Allied Vision Technologies GmbH, Stadtroda, Germany		Face Camera (frontal view)
AV Prosilica GT 1600C	Allied Vision Technologies GmbH, Stadtroda, Germany		Face Camera (side view)
PIR uc 180 Thermal Camera	InfraTec GmbH, Dresden, Germany		Thermal Face Camera
Synchronization Hardware	Werkstatt, IIKT, University of Magdeburg, Magdeburg, Germany	custom built	Hardware triggering of cameras, trigger signal is recorded by BIOPAC and Audacity
Recording and Synchronization Software	Philipp Werner, Neuro-Information Technology, University of Magdeburg, Magdeburg, Germany	custom software	Real-time recording, offline video encoding, and offline synchronization
Examination Couch	ClinicalCare GmbH, Bremen, Germany
Ag-AgCl Electrodes EL254 / EL254S (Reusable, 4mm recording diameter)	BIOPAC Systems, Inc., Goleta, CA, USA		Used to record EMG M. corrugator and M. zygomaticus
Ag-AgCl Electrodes BlueSensor P (Disposable, skin contact size: 34 mm diameter, measuring area 154 mm2)	Ambu GmbH, Bad Nauheim, Germany		Used to record ECG and EMG M. trapezius. Also used for electrical stimulation
Audacity 2.1.2	Dominic Mazzoni (Audacity)		Audio Recording Software
Cold Gel Pack	C+V Pharma Depot GmbH, Versmold, Germany
Panthenol 50mg/g	ratiopharm GmbH, Ulm, Germany		Ointment
Alumnium Profiles	item Industrietechnik GmbH, Solingen, Germany		Used to install all cameras and microphone
Electrode Gel GEL1	BIOPAC Systems, Inc., Goleta, CA, USA
ELPREP Skin Preparation Gel	BIOPAC Systems, Inc., Goleta, CA, USA