An Experimental Platform to Study the Closed-loop Performance of Brain-machine Interfaces

We use the activity of a visual inter neuron in the fly brain, the H one cell, to control the motors of a mobile robot. The robot is placed on a constantly rotating turntable and the cell's activity is used to stabilize the robot relative to the environment against external movements. Images of pattern motion recorded from the robot are sampled and sent to two computer CRT monitors, which are positioned in front of the fly.

The signals of the H one cell measured in spikes per second indicate the speed of the pattern motion. Different control laws are then applied to convert the recorded spikes per second into a control signal for the robot's motors. Hi, I'm Navita Josh from Holger Crops Lab in the department of Bioengineering Imperial College London.

I'm Chris Peterson, also from Holger Cups Lab. Hi, and I am Holger Cup. Today we'll show you a procedure for creating a brain machine interface between individual cells in the fly visual system and a robot.

We use this procedure to test the performance of different control strategies using neuronal signals to control robotic systems under closed lip conditions. So let's get started. To begin preparing the fly, cool it on ice, and then use blunted cocktail sticks to hold the wings down and fix the back of the fly to a piece of double-sided tape on a microscope Slide next, use B wax to attach the wings to the slide and also to block the action of the flight motor.

This step requires quick and accurate handling so that the fly does not warm up during the procedure. Now, under the microscope, hold each leg with forceps and use a pair of small scissors to cut them off at the joints closest to the body. Repeat this for the proboscis.

To prevent the fly from drying out, the holes must be sealed with wax. Next, cut one of the wings off, and then turn the fly on its side. Remove any remaining pieces of wing while leaving the katra, covering the Hal tears and seal the hole with wax.

Repeat this procedure for the other wing. To stimulate a target neuron in a defined way, the fly's head has to be properly aligned with the computer monitors. To do this, you will need a customized holder that has a broad space for the fly's body and an appendage on one end with a notch cut where the fly's neck will be placed.

Place the fly onto the holder with its neck in the notch, pressing it down while gluing the abdomen in place. Now, place the fly holder in a stand so that you can see the front of the fly's head through the microscope. Viewing the fly with red light and optical phenomenon called the pseudo pupil can be seen in each eye.

If the pseudo pupil assumes a certain shape, then the orientation of the fly's head is perfectly defined. Use a micro manipulator to correctly orient the fly's head and then use wax to glue it to the holder. Next, press the thorax down flat and wax it to the holder.

This allows the rear head capsule to be opened so that electrodes can be inserted into the fly Brain use a micro scalpel or a fine injection needle to carefully cut a window into the cuticle of the right head capsule. Be careful not to cut the neural tissue right underneath the cuticle. Once the piece of cuticle is removed, add a few drops of ringer solution.

Use forceps to remove any floating hairs, fat deposits, or muscle tissue that may cover the LOA plate. The LOA plate can be identified by a characteristic branching pattern of silvery trachea that covers its posterior surface. Cut a small hole into the cuticle of the left rear head capsule for positioning a reference electrode with the flying prepared.

Let's see how to position the recording electrode. The recording electrode must be placed in close proximity to the H one neuron. The H one neuron mainly responds to horizontal back to front motion presented to its receptive field.

To position the recording electrode, use the trachea as a visual landmark. Initially place the electrode between the uppermost trachea. It helps to use an audio amplifier to convert the recorded electrical potentials into acoustic signals.

Each individual spike is turned into a characteristic clicking sound. The closer the electrode gets to an individual neuron, the clearer the clicking sound becomes. To identify the H one neuron by means of its motion preference, stimulate it with motion in the horizontal direction.

With the recording electrode in place, let's move on to visual stimulation and recordings. To begin place a fly in front of two CRT computer monitors. Because the fly visual system is 10 times faster than the humans, the monitors must display 200 frames per second position.

The centers of the monitors at plus or minus 45 degrees relative to the flies orientation. As seen from the flies eye equator, each monitor sub tens an angle of plus or minus 25 degrees in the horizontal and plus or minus 19 degrees in the vertical plane. Input to the computer monitors is provided by two video cameras mounted on a small two-wheeled SRO robot that has been modified for the experiment.

Position the robot on a turntable within a cylindrical area whose walls are lined with a pattern of vertically oriented black and white stripes. By rotating the turntable in the horizontal plane, the movements of the robot are limited to only one degree of freedom. Initially, both the turntable and robot are at rest.

When the turntable starts moving its rotation carries the robot in the same direction, and the video cameras record the relative motion between the robot and the striped pattern of the arena. The battery powered video cameras on the robot are mounted at an orientation of plus or minus 45 degrees. They capture 200 images per second to match the frame rate of the computer monitors in front of the fly log.

The images presented to the computer monitors at 200 frames per second at a resolution of six 40 by four 80 gray scale. While the fly is watching the movements of the striped pattern record, the band passed filtered. For example, between 302 kilohertz electrical signals with a digital acquisition board using a sampling rate of at least 10 kilohertz, A threshold is applied to the band passed filtered electrical signals to separate the spikes from the background activity.

A causal half Gaussian filter is convolved with the spikes to obtain a smooth spiking activity estimate for the H one cell to close the loop of the brain machine interface. A control algorithm is used to convert the spiking rate of the H one cell to a robot speed, which is fed back via a Bluetooth interface to control the two DC motors driving the wheels of the robot. Pure sign waves are chosen as velocity profiles for the turntable.

The sign waves have a DC offset such that the turntable only rotates in the direction which stimulates the H one neuron along its preferred direction. The whole control system is set up such that stimulation of the H one neuron results in the robot compensating for the movement of the turntable when set up correctly. Visual stabilization is achieved when the counter rotation of the robot matches the rotation of the turntable, resulting in little or no pattern movement on the computer monitors.

The overall performance of the system depends on the control algorithm being used to close the loop. The first algorithm we test is a proportional controller where the updated robot speed is proportional to the difference in angular velocities between the robot omega R and the turntable Omega P.Different values for the static gain. KP and input frequencies for the turntable signal omega P are chosen to test the performance of the controller.

Sample traces for omega P and omega R are shown here for KP equals one and an input frequency of 0.6 hertz for omega P, the robot in green follows the turntable in blue with a lag and a smaller peak amplitude. The horizontal component of the pattern motion that stimulates the H one cell is shown on the right in red input frequencies for the turntable signal. Omega P are chosen between 0.03 and three hertz, and the corresponding robot signal omega R is recorded.

Both signals are transformed into the frequency domain by a fast four-year transform, and the amplitude and phase values are calculated at the input frequency. The BO d magnitude plot for the proportional controller with KP equals one shows the response of the system over the tested input frequencies. The performance of the controller generally decreases with increasing frequencies.

The slightly increased gain at one hertz is as a result of oscillations in the robot signal due to using only one H one cell whose dynamic output range mainly covers horizontal back to front motion. The Bodhi phase plot shows a controller phase lag less than PI for input frequencies less than 0.6 hertz. This shows that the controller is stable for frequencies less than 0.6 hertz and unstable for input frequencies greater than or equal to one hertz.

The performance of the proportional controller with a static KP was compared with an adaptive controller where the value for KP is updated every 50 milliseconds. Based on the peak spike rate. F max calculated over time interval T minus 500 milliseconds to T.As a result of the large integration time window, the proportional controller performs better than the adaptive controller for the parameter range tested, the adaptive controller had a similar phase characteristic as the proportional controller, the grading pattern around the turntable was removed and the lab environment was used as an approximation of naturalistic visual input for the fly H one cell.

On average, the Bodhi magnitude plot for the naturalistic visual input showed slightly higher gains than the one with grading visual input, probably because the wider range of spatial frequencies in naturalistic visual images is exploited. The Bodhi phase plot characteristics for grading versus naturalistic visual inputs were similar. We've just shown you how to create a brain machine interface between a cell and the five visual system and a robot.

There are a few critical steps during this procedure. First, avoid deep cuts during opening the head capsule to prevent from injuring the brain. Second, position the electrode carefully so that it records from only one cell we record from the H one cell.

Third, keep the brain moist at all times and prevent it from drying out. So that's it. Thanks for watching and good luck with the experiments.