Designing and Implementing Nervous System Simulations on LEGO Robots

Summary

An approach to neural network modeling on the LEGO Mindstorms robotics platform is presented. The method provides a simulation tool for invertebrate neuroscience research in both the research lab and the classroom. This technique enables the investigation of biomimetic robot control principles.

Abstract

We present a method to use the commercially available LEGO Mindstorms NXT robotics platform to test systems level neuroscience hypotheses. The first step of the method is to develop a nervous system simulation of specific reflexive behaviors of an appropriate model organism; here we use the American Lobster. Exteroceptive reflexes mediated by decussating (crossing) neural connections can explain an animal's taxis towards or away from a stimulus as described by Braitenberg and are particularly well suited for investigation using the NXT platform.¹ The nervous system simulation is programmed using LabVIEW software on the LEGO Mindstorms platform. Once the nervous system is tuned properly, behavioral experiments are run on the robot and on the animal under identical environmental conditions. By controlling the sensory milieu experienced by the specimens, differences in behavioral outputs can be observed. These differences may point to specific deficiencies in the nervous system model and serve to inform the iteration of the model for the particular behavior under study. This method allows for the experimental manipulation of electronic nervous systems and serves as a way to explore neuroscience hypotheses specifically regarding the neurophysiological basis of simple innate reflexive behaviors. The LEGO Mindstorms NXT kit provides an affordable and efficient platform on which to test preliminary biomimetic robot control schemes. The approach is also well suited for the high school classroom to serve as the foundation for a hands-on inquiry-based biorobotics curriculum.

Introduction

Neurophysiological investigations over the past 100 years have broadened tremendously our knowledge of nervous system structure and function. However, the majority of nervous system research to date has relied on the use of isolated preparations or restrained subjects. While there have been many successful efforts to record neural activity from freely behaving animals^2-5, the biorobotic approach provides a valuable tool to allow for nervous system manipulation in order to test systems level neuroscience hypotheses⁶. Simulated nervous systems operating on robots can be experimentally manipulated and allow for the extension of software modeling to the physical world. This approach has been well implemented in the academic world^7,8 but the process of building a biomimetic robot for hypothesis testing can be expensive and time consuming. We present a method to execute the biorobotic approach using a commercially available robotics kit (LEGO Mindstorms NXT 2.0). The goal of this method is to provide a rapid and efficient way to test systems level neuroscience hypotheses on robotic⁹ or bio-hybrid¹⁰ embodied neural network simulations. Accelerating the process from hypothesis to experiment improves research productivity. The simple LEGO Mindstorms platform provides a test bed for biomimetic sensors and neural networks which we demonstrate using the American Lobster (Homarus americanus) as a model organism. The method also provides a powerful hands-on educational tool in the classroom as students can design and manipulate nervous systems for their own robots¹¹.

Protocol

1. Building the Robot Model

1. Choose a model organism to study that is well represented in the neuroethological literature. Invertebrates generally make good candidates because their relatively simple nervous systems have been well studied and are primarily comprised of innate reflexes. We will demonstrate this approach using the American Lobster, Homarus americanus.
2. Select well-studied reflexive behaviors for modeling purposes. We have chosen the lobster's responses to antennal bend and to claw bump detection as the animal reflexively responds to bending of antennae to elicit rheotaxis (orientation to water flow) while claw contacts mediate obstacle avoidance. Simple reflexes relying on decussating neural connections from bilaterally symmetrical sensors are well suited for this type of study¹.
3. Build or choose a robotic platform with appropriate sensors for the selected behaviors in order to run the neural network simulation. While a dedicated robot can be constructed such as RoboLobster¹² or the Sprawl (cockroach) robot family¹³, here we use a commercially available kit to expedite the experimental process. The LEGO Mindstorms NXT 2.0 kit, providing templates, modular components and established sensors, allows for the rapid physical construction of the robot.
4. Select or build sensors to match the modalities of the behaviors selected in step 1.2. Premade sensors from the LEGO Mindstorms kit can be used or homebrew sensors can be engineered^14,15. We have selected the included LEGO touch sensor and a custom antennal bend sensor consisting of a FlexPoint bend sensor spliced to an NXT connector. To build a custom resistive connector, cut an NXT connector wire and solder the black and white cables to the leads of the sensor.
5. Use previously published neuroethological studies to find or develop a hypothetical neural network for modeling purposes. The literature has proposed various neural networks to explain behavior in various model organisms, from lobsters¹⁶ to lamprey¹⁷. Functional neural units should be identified and their synaptic connections theorized. Novel neuroethological experiments can also be developed and implemented if laboratory circumstances allow.

2. Programming the Nervous System

1. Use LabVIEW software along with the LabVIEW Module for LEGO MINDSTORMS NXT to create Virtual Instruments (VIs) that run equations for a mathematical neuron and synapse model. While any published model can be used, we recommend the discrete time map-based (DTM) model¹⁸. This allows for computationally efficient real time operation while maintaining a variety of neuronal firing output regimes. Two control parameters, alpha and sigma, determine the dynamics of the simulated neuron to produce varied neural output including tonic spiking, chaotic firing, bursting and intrinsic silence. LabVIEW VIs for LEGO Mindstorms for this neuron model can be found online here: http://myfiles.neu.edu/blustein.d/neurobots
2. Place VIs into a While Loop so that the network will update iteratively.
3. Connect presynaptic neuron spike outputs through synapses to postsynaptic neurons.
4. Pass iteratively updating parameters through shift registers to the next calculation cycle. The DTM model updates fast (x) and slow (y) dynamical variables that are passed to the next iteration of calculations.
5. Set synapse parameters to define network characteristics. For each synapse, the synaptic strength (gsyn), the relaxation rate (gamma), and the reversal potential (xrp) are coefficients in the DTM model equations that determine synaptic behavior. The value of gsyn is a multiplier that determines the magnitude of the synaptic current effect from a spike event, ranging from 0.0 (no effect) to 1.0 (100% effect). The value of gamma is a multiplier that determines the rate of decay of the synaptic current regardless of spike events, ranging from 0.0 (complete decay) to 1.0 (no decay). The value of xrp determines the level of excitation or inhibition of the synapse, values typically range from -3.0 to 3.0 with positive values representing excitation and negative values representing inhibition. Values can be tuned for your specific network characteristics. Here we use the following settings: gamma= 0.95, gsyn= 0.60 and xrp = 2.2 (excitatory synapse) or -1.8 (inhibitory synapse).
6. Adjust alpha and sigma control parameters to modify baseline neuron activity. The dynamics of the model for varied control parameters are described in detail elsewhere¹⁸ but typically, alpha < 4 produces intrinsically silent neural activity for low sigma values or tonic spiking for high sigma values. The transition between 'low' and 'high' sigma values generally occurs between -1.0 and 1.0 depending on the alpha value. Bursting neuron activity is produced for alpha > 4 when sigma values are near zero. Here we use alpha = 4.05 and sigma = -3.10.
7. Use LabVIEW VIs to input sensor information into your network. For provided sensors, the VIs are already available within the Mindstorms LabVIEW module. For custom resistive sensors, use the Light Sensor VI as a programming framework. Information about creating homebrew sensors can be found elsewhere¹⁴.
8. Insert graphical charts on the front panel of the LabVIEW program to visualize neuronal activity.
9. Temporarily replace sensor inputs with front panel control boxes in order to manually manipulate the sensor information going to the neural network.
10. Run the neuronal network and manually adjust the front panel controls to present varied sensory input. Confirm that the network qualitatively functions as predicted for known conditions.
11. Adjust the network as needed by changing the neuron and synapse parameters. First attempts at tuning the network should be done by changing synaptic strength (gsyn) of the various synapses. Other parameters may need to be adjusted as well.
12. Once the network functions with qualitative accuracy, replace the front panel control boxes with the sensor input VIs.

3. Nervous System Simulation Testing

1. Set up a controlled environment in which to observe the animal and the robot under similar conditions. We use a rectangular tank containing rock obstacles (Figure 1). The tank can be emptied to accommodate the LEGO robot in the same space.
2. Mount a video camera overhead to record the animal/robot behavior. Make sure the contrast between the subject and the background is high to allow for automatic tracking. We painted the bottom of the test tank white.
3. Place the animal in the controlled environment and video record the behavior. Be sure to limit the sensory capabilities of the lobster to match those of the robot. Here we restrict the lobster's sensory capabilities by covering the eyes with an aluminum foil mask and deafferenting chemoreceptors with freshwater¹⁹.
4. Adjust the environment to suit a LEGO robot (removing water if necessary) and run the robot in the same controlled environment.
5. Use a MATLAB script to automatically track the markers on the specimens. We use MouseLabTracker²⁰.
6. Create and compare vector plots of the animal and robot movements.
7. Modify the nervous system and repeat steps 3.3-3.6 to observe how different aspects of the electronic nervous system affects the robot's behavior. Here we adjust the synaptic strength between the claw bump sensor neurons and the motor system and monitor robot performance compared to that of a lobster. Alternatively, the sensory conditions in the test arena can be modified and the effects observed in robot and animal. For example, the arena described here could be subjected to different water flow rates.

Results

Inputs from a lobster's claws into its nervous system mediate obstacle negotiation in a novel environment. Figure 1 shows a screenshot of the video used to analyze the behavior of a LEGO robot (Figure 1A) and a lobster (Figure 1B) in the test arena. The test arena was unmodified between animal and robot tests except that the water was emptied from the tank for the robot trials.

Video tracking results are displayed in Figure 2....

Discussion

When initiating biorobotic nervous system simulation experiments, there are a few important guidelines to follow. Choosing the right model organism is critical: pick an organism that is easy to obtain and maintain. Invertebrates are ideal because they do not usually require institutional approval for experimentation and their husbandry needs are often less demanding than those of vertebrates. From a scientific perspective, it is beneficial to choose an animal that has an established neuroethological literature record fro...