SSVEP-based Experimental Procedure for Brain-Robot Interaction with Humanoid Robots

The overall goal of this method is to establish an S-S-V-E-P based experimental procedure by integrating multiple software programs to enable the study of brain robot interaction with humanoid robots. This is achieved by first place in EEG electrodes on human subject to measure the brain responses through an EEG data acquisition system. A user interface is used to elicit S-S-V-E-P responses and to display video feedback in the closed loop control experiments.

The second step is to record the EEG signals of the first time subjects to analyze your S-S-V-E-P feature offline and to train the classifier for each subject. Next, the online signal processor and the robot controller are configured for the online control of a humanoid Robert. As the final step, the subject completes three specific closed loop control experiments within different environments.

In order to evaluate the brain Robert interaction performance, the results show the normalized power spectral density of brain signals generated while staring at the stimuli flickering at different frequencies. Our system translates these brainwave patterns into commands to control for Robert behaviors. Using this approach, the subject is capable of interacting with the humanoid Robert Bio brainin signals.

This allows the humanoid Robert to perform typical tasks that are popular in robotic research and are helpful in assisting the disabled. The advantage of this approach is its reliability and flexibility as it is developed by integrating multiple software programs such as open wide graph and central software, as well as user developed programs related in c plus plus and match lab. This method enables to study brain robot in action with the humanoid robot.

This study is prospective in assisting the sick and elderly as well as performing unsanitary or dangerous jobs. My teammates, Lim and Han, will now demonstrate the placement of EEG sensors and show the method of connecting to the human no robot respectively. Explain experiment to procedure to the subject.

Obtain consent to participate in experiments. Measure the circumference of the subject, select the E EEG cap size that is close to the measurement. Measure the distance between the NAS mark 10%of distance as a reference for lining the cap.

Mark the midpoint of the distance at the vertex on the subject cul position. The electrode cz of the E EEG cap on the vertex align the 10%mark with the midpoint of the FP one and FP two electrodes. Make sure that the fz, cz, PZ, and OZ electrodes are on the midline of the head.

Smear the reference reference to electrodes with conductive gel. Place the reference electrodes on the left and right OIDs using medical tape. Tighten the chain strap.

Place the blunt tipped syringe in electrode toters and inject conductive gels into each electrode in the following order. First, the ground electrode and second, the five electrodes used in experiment O 2 0 1 OZ.PZ and cz. Sit the subject in a comfortable chair, 60 centimeter in front of a stimulus monitor.

Connect the electrode wires to the EEG data acquisition system. Configure the sampling rate to one kilohertz. Examine the EEG signal quality on the dedicated data processing computer.

If there is a problem with the particular electrode. Re-inject gels to adjust the impedance of the channel. Flicker four robots images that different frequencies as visual stimuli on the user interface.

For the first time subject, conduct the offline training experiment to establish their S-S-V-E-P feature vectors and to train the classifier. Conduct 32 trials for each subject and record your brain signal throughout this process. When a trial starts, randomly select one stimulus as the target and display a yellow arrow above it.

One second. Later flicker the four visual stimuli at different frequencies on the user interface for five seconds, request the subject to focus on the selected stimulus target while keeping the body movement to a minimum. After each trial, give the subject three seconds to relax and then start the next trial.

When all trials are finished, read the saved data. Vertical orange bars indicate triggers to select a stimulus as the target the stimuli are flickering during the five second period. In each trial, extract a three second data apo between two seconds and five seconds after the trigger.

Calculate the CCA coefficient of the multi-channel EEG data with a reference signal y, which is set to be the periodic stimulus signal at certain frequencies. The CCA method finds the linear combination vectors to maximize the correlation coefficient between the combined signals. Spatially filter the multi-channel EEG data using the calculated CCA coefficients.

Calculate the power spectral density of the filter data using fast foyer transform. Normalize the power spectral density with respect to its mean value between three hutz and 40 hutz. Then calibrate the classification parameters for each stimulus frequency.

Configure the classification parameters for the subject through an open wipe setting window. Start the online signal processor on the data processing computer. Acquire EEG data from channels O 2 0 1 OZ PZ and CZ of the EEG system every 0.5 seconds and extract the data segment of the last three seconds for online processing.

Process the data segment using the algorithms described in the offline process. Calculate the real-time power spectral density for classification. When the amplitude of one feature frequency is above a given threshold, the stimulus flickering at the corresponding frequency is classified.

As the SSV EEP target, start up the human Noro robot and establish its wifi connection to the data processing computer. Configure and run the robot controller, which received the classification results from the online signal processor and controls the corresponding behaviors of the humanoid Robert via wireless connection. Run the choreograph program on the dedicated stimulus presentation computer to display live video feedback from the Roberts camera on the user interface.

Request the subject to perform three specific closed loop control tests within different environments in order to evaluate the brain Robert interaction performance. These tests are popular in robotic research and are helpful in assisting the disabled and elderly in their daily lives. Before a new task inform the subject of the objective of the task and the available behaviors for control.

These results show the procedure of processing EEG data, including extracting the multi-channel data epoch spatially filtering the data using CCA coefficients and calculating the normalized power spectral density. The normalized power spectral density obtained in single trials in which the subject stares at different flickering targets are shown here. In the first closed loop control experiments, the human Noah Robert was controlled by brain signals to walk through obstacles and push a light switch on the wall.

Brain signals were possessed in real time. To classify the subject's control intent, the subject watched the live video feedback and controlled the Robert behavior by staring at the corresponding stimuli. After reaching the switch, the subject controlled the Robert to turn on the light.

In the second experiment, the human robot was navigated continuously towards the staircase following the exit sign based on video feedback. When confronting passer spy, the subject controlled the Robert to say, excuse me, and wait for the passer spy to give way. The Robert was then controlled to walk towards the staircase.

In the third experiment, the humanoid Robert was controlled with full body movement to reach and pick up the balloon target and deliver it to the subject's hand based on the live video feedback. Because human intent is perceived by interpreting LU time EEG signals, it is critical to verifies electric connections and e EEG signal qualities before conducting experiments. Another common issue while acquiring EEG signals is an interference or artifacts and noises.

The loins, our me such interference is improved by reducing the body movement subject and utilizing CCA based multichannel technique. Our method is effective to design a variety of experimental procedure and flexible to investigate algorithms and techniques for brain robot interaction especially is, can be easily used to explore new BRI applications.