A Cross-Disciplinary and Multi-Modal Experimental Design for Studying Near-Real-Time Authentic Examination Experiences

An experimental design was developed to investigate the real-time influences of an examination experience to assess the emotional realities students experience in higher education settings and tasks. This design is the result of a cross-disciplinary (e.g., educational psychology, biology, physiology, engineering) and multi-modal (e.g., salivary markers, surveys, electrodermal sensor) approach.

The protocol allows researchers in social sciences, psychology, education, engineering, and other disciplines interested in using biometrics and multiple techniques to understand examination, classroom, and similar authentic experiences. The main advantage of this technique is that it aims to find the potential sources of noise due to motion artifacts in the data. Also, the protocol is simple to implement and minimizes the use of expensive software.

While diagnosing a disease isn't our intended goal, the technologies we use through electrodermal sensors and salivary biomarkers have the potential to detect physiological and biological abnormalities in a participant. This method can provide insight into educational psychology research exploring motivational and emotional constructs in the classroom. It could also be used in marketing, business, design, and other research areas that involve an individual's cognitive and emotional processes.

The main struggle a researcher will have with this method is to time sync the events they aim to study while accounting for the limitations of existing physiological and biological techniques. It is important to understand the process of electrodermal activity, especially the working process of electrodermal sensors, as the existing open-source filtering techniques are limited. Given that this technique is aimed for quasi-experimental designs using multiple techniques and disciplinary expertise, a visual demonstration of this method is essential to allow researchers to witness the parallel and coordinated steps needed to successfully execute this method.

Begin by setting up a workspace with minimal distractions. An unintentionally activated startle response can create a significant spike in electrodermal activity, or EDA. Ensure that the presentation of the survey question is synchronized to the electrodermal sensor.

Use the internal clock of the computer to establish a data collection time frame, and if using any Bluetooth-enabled electrodermal sensors, sync times in Greenwich Meridian Time. It is critical that all instruments, including surveys, exam protocol, electrodermal sensors, and web cameras are synced to the same time, so there is no shift in recording times for each data source. If needed, set up web cameras to measure the participant's face at different angles.

Refer to the manuscript for recommendations on camera positioning. For electrodermal sensor readings, designate a period of time during which participants are in a relaxed state to establish baseline EDA data. Instruct participants to stare at the testing shield for five to 15 minutes at the beginning of the exam, or program this cue into the computer.

To collect saliva samples, prepare a cooler with dry ice that has an internal temperature of minus 20 degrees Celsius. This will prevent room-temperature degradation of enzymes for the cortisol samples. If using the swab collection method, instruct participants to keep the swab either in the inner cheek or under the tongue for 60 seconds.

Develop a flagging system that will allow participants to notify the researcher when the salivary sample is ready to be collected. In preparation for the experiment, pre-schedule participants to an examination session, assess any medical information and dietary habits, note hand dominance for EDA collection, and remind participants to avoid consumption of sugary or caffeinated products on the day of the experiment. Prior to participant arrival, make sure that the sensors are properly calibrated, software updates have been taken care of, and the sensors have been wiped with 70%alcohol wipes.

Fit the EDA sensor on the participant's non-dominant wrist by placing the sensor with the button facing down towards the thumb. Have the participant draw an imaginary line from the space between the second and third finger to their mid-wrist area, and place the sensor electrodes there. Ask the participants to adjust the sensor straps.

When starting the sensor, follow the manufacturer's protocols to ensure that they are set up to collect data. For the devices used in this protocol, depress the sensor button for three seconds. A green light will blink intermittently, followed by a red blinking light, and then fade-out.

During fade-out, press the button once for one second, and if it blinks red, it is recording data. To turn the sensor off, press the button for three seconds. The green light at the bottom of the wristband will fade.

Retrieve data from the sensor by connecting it to the computer and uploading it in the managing software system. If collecting salivary samples, pour one ounce of water for participants upon arrival to the experimental room, and ask them to swish and swallow it. During examination, use time prompts to cue participants when it is time to collect saliva.

Allow 60 seconds for sample collection, and return them to the examination. When collecting salivary samples, wear fresh nitrile gloves to minimize any dust particulate or contaminant from hand oils to be transferred to the sample vial. Immediately transfer the samples to the prepared cooler with the internal temperature of minus 20 degrees Celsius.

To provide an authentic testing experience, consult with the course instructor to align the exam content with course content, and select an evaluation of the course content that can be replicated in an experimental setting. Also, ensure that the testing environment parallels the experimental setup. Accommodate for interruptions in test-taking by allotting additional test-taking time and designing the examination program to return students to a particular exam problem if a survey prompt interrupted them.

Make sure that interruption time is consistent across participants. Include calibration and relaxation periods throughout the exam experience by starting with a simple question and allowing participants 30 seconds to rest in between each exam question. Finally, ensure that the time-stamping program accounts for any changes in the examination to allow for data source matching.

It is important that calibration and relaxation periods are accounted for in the protocol, especially for electrodermal sensors and salivary biomarkers, as baseline values are needed to determine signal noise or outlier data. Two experimental designs were tested, design A with EDA sensors and surveys and design B with added salivary biomarkers. Although exam difficulty was the same, opposing differences in the mean EDA values were found between the correct and incorrect responses across different self-efficacy scores.

For experimental design B, the mean EDA, as well as cortisol and salivary alpha-amylase, or sAA, were measured at the beginning, middle, end, and 20 minutes after the exam. The EDA levels decreased from beginning to end of the exam but recovered 20 minutes after, a trend that was paralleled by both cortisol and sAA. Statistical significance was found between EDA and sAA at the beginning and middle of the exam, whereas EDA and cortisol showed significance between the middle and end of the exam.

By the 20-minute mark, EDA and sAA, as well as cortisol and sAA, showed significance between each other. Please ensure that synchronization of instruments and protocols are automated as much as possible, and check for errors before the start of the experiment. This procedure allows for the integration of other biometrics, such as eye triggers, electromyographs, as well as other forms of service that include open-ended questions and video prompts, among others.

This technique can be used by researchers to explore the connections between cognition, affect, physiology, and performance.