Designing and Implementing Nervous System Simulations on LEGO Robots

Summary

An approach to neural network modeling on the LEGO Mindstorms robotics platform is presented. The method provides a simulation tool for invertebrate neuroscience research in both the research lab and the classroom. This technique enables the investigation of biomimetic robot control principles.

Abstract

We present a method to use the commercially available LEGO Mindstorms NXT robotics platform to test systems level neuroscience hypotheses. The first step of the method is to develop a nervous system simulation of specific reflexive behaviors of an appropriate model organism; here we use the American Lobster. Exteroceptive reflexes mediated by decussating (crossing) neural connections can explain an animal's taxis towards or away from a stimulus as described by Braitenberg and are particularly well suited for investigation using the NXT platform.¹ The nervous system simulation is programmed using LabVIEW software on the LEGO Mindstorms platform. Once the nervous system is tuned properly, behavioral experiments are run on the robot and on the animal under identical environmental conditions. By controlling the sensory milieu experienced by the specimens, differences in behavioral outputs can be observed. These differences may point to specific deficiencies in the nervous system model and serve to inform the iteration of the model for the particular behavior under study. This method allows for the experimental manipulation of electronic nervous systems and serves as a way to explore neuroscience hypotheses specifically regarding the neurophysiological basis of simple innate reflexive behaviors. The LEGO Mindstorms NXT kit provides an affordable and efficient platform on which to test preliminary biomimetic robot control schemes. The approach is also well suited for the high school classroom to serve as the foundation for a hands-on inquiry-based biorobotics curriculum.

Introduction

Neurophysiological investigations over the past 100 years have broadened tremendously our knowledge of nervous system structure and function. However, the majority of nervous system research to date has relied on the use of isolated preparations or restrained subjects. While there have been many successful efforts to record neural activity from freely behaving animals^2-5, the biorobotic approach provides a valuable tool to allow for nervous system manipulation in order to test systems level neuroscience hypotheses⁶. Simulated nervous systems operating on robots can be experimentally manipulated and allow for the extension of software modeling to the physical world. This approach has been well implemented in the academic world^7,8 but the process of building a biomimetic robot for hypothesis testing can be expensive and time consuming. We present a method to execute the biorobotic approach using a commercially available robotics kit (LEGO Mindstorms NXT 2.0). The goal of this method is to provide a rapid and efficient way to test systems level neuroscience hypotheses on robotic⁹ or bio-hybrid¹⁰ embodied neural network simulations. Accelerating the process from hypothesis to experiment improves research productivity. The simple LEGO Mindstorms platform provides a test bed for biomimetic sensors and neural networks which we demonstrate using the American Lobster (Homarus americanus) as a model organism. The method also provides a powerful hands-on educational tool in the classroom as students can design and manipulate nervous systems for their own robots¹¹.

Protocol

1. Building the Robot Model

1. Choose a model organism to study that is well represented in the neuroethological literature. Invertebrates generally make good candidates because their relatively simple nervous systems have been well studied and are primarily comprised of innate reflexes. We will demonstrate this approach using the American Lobster, Homarus americanus.
2. Select well-studied reflexive behaviors for modeling purposes. We have chosen the lobster's responses to antennal bend and to claw bump detection as the animal reflexively responds to bending of antennae to elicit rheotaxis (orientation to water flow) while claw contacts mediate obstacle avoidance. Simple reflexes relying on decussating neural connections from bilaterally symmetrical sensors are well suited for this type of study¹.
3. Build or choose a robotic platform with appropriate sensors for the selected behaviors in order to run the neural network simulation. While a dedicated robot can be constructed such as RoboLobster¹² or the Sprawl (cockroach) robot family¹³, here we use a commercially available kit to expedite the experimental process. The LEGO Mindstorms NXT 2.0 kit, providing templates, modular components and established sensors, allows for the rapid physical construction of the robot.
4. Select or build sensors to match the modalities of the behaviors selected in step 1.2. Premade sensors from the LEGO Mindstorms kit can be used or homebrew sensors can be engineered^14,15. We have selected the included LEGO touch sensor and a custom antennal bend sensor consisting of a FlexPoint bend sensor spliced to an NXT connector. To build a custom resistive connector, cut an NXT connector wire and solder the black and white cables to the leads of the sensor.
5. Use previously published neuroethological studies to find or develop a hypothetical neural network for modeling purposes. The literature has proposed various neural networks to explain behavior in various model organisms, from lobsters¹⁶ to lamprey¹⁷. Functional neural units should be identified and their synaptic connections theorized. Novel neuroethological experiments can also be developed and implemented if laboratory circumstances allow.

2. Programming the Nervous System

1. Use LabVIEW software along with the LabVIEW Module for LEGO MINDSTORMS NXT to create Virtual Instruments (VIs) that run equations for a mathematical neuron and synapse model. While any published model can be used, we recommend the discrete time map-based (DTM) model¹⁸. This allows for computationally efficient real time operation while maintaining a variety of neuronal firing output regimes. Two control parameters, alpha and sigma, determine the dynamics of the simulated neuron to produce varied neural output including tonic spiking, chaotic firing, bursting and intrinsic silence. LabVIEW VIs for LEGO Mindstorms for this neuron model can be found online here: http://myfiles.neu.edu/blustein.d/neurobots
2. Place VIs into a While Loop so that the network will update iteratively.
3. Connect presynaptic neuron spike outputs through synapses to postsynaptic neurons.
4. Pass iteratively updating parameters through shift registers to the next calculation cycle. The DTM model updates fast (x) and slow (y) dynamical variables that are passed to the next iteration of calculations.
5. Set synapse parameters to define network characteristics. For each synapse, the synaptic strength (gsyn), the relaxation rate (gamma), and the reversal potential (xrp) are coefficients in the DTM model equations that determine synaptic behavior. The value of gsyn is a multiplier that determines the magnitude of the synaptic current effect from a spike event, ranging from 0.0 (no effect) to 1.0 (100% effect). The value of gamma is a multiplier that determines the rate of decay of the synaptic current regardless of spike events, ranging from 0.0 (complete decay) to 1.0 (no decay). The value of xrp determines the level of excitation or inhibition of the synapse, values typically range from -3.0 to 3.0 with positive values representing excitation and negative values representing inhibition. Values can be tuned for your specific network characteristics. Here we use the following settings: gamma= 0.95, gsyn= 0.60 and xrp = 2.2 (excitatory synapse) or -1.8 (inhibitory synapse).
6. Adjust alpha and sigma control parameters to modify baseline neuron activity. The dynamics of the model for varied control parameters are described in detail elsewhere¹⁸ but typically, alpha < 4 produces intrinsically silent neural activity for low sigma values or tonic spiking for high sigma values. The transition between 'low' and 'high' sigma values generally occurs between -1.0 and 1.0 depending on the alpha value. Bursting neuron activity is produced for alpha > 4 when sigma values are near zero. Here we use alpha = 4.05 and sigma = -3.10.
7. Use LabVIEW VIs to input sensor information into your network. For provided sensors, the VIs are already available within the Mindstorms LabVIEW module. For custom resistive sensors, use the Light Sensor VI as a programming framework. Information about creating homebrew sensors can be found elsewhere¹⁴.
8. Insert graphical charts on the front panel of the LabVIEW program to visualize neuronal activity.
9. Temporarily replace sensor inputs with front panel control boxes in order to manually manipulate the sensor information going to the neural network.
10. Run the neuronal network and manually adjust the front panel controls to present varied sensory input. Confirm that the network qualitatively functions as predicted for known conditions.
11. Adjust the network as needed by changing the neuron and synapse parameters. First attempts at tuning the network should be done by changing synaptic strength (gsyn) of the various synapses. Other parameters may need to be adjusted as well.
12. Once the network functions with qualitative accuracy, replace the front panel control boxes with the sensor input VIs.

3. Nervous System Simulation Testing

1. Set up a controlled environment in which to observe the animal and the robot under similar conditions. We use a rectangular tank containing rock obstacles (Figure 1). The tank can be emptied to accommodate the LEGO robot in the same space.
2. Mount a video camera overhead to record the animal/robot behavior. Make sure the contrast between the subject and the background is high to allow for automatic tracking. We painted the bottom of the test tank white.
3. Place the animal in the controlled environment and video record the behavior. Be sure to limit the sensory capabilities of the lobster to match those of the robot. Here we restrict the lobster's sensory capabilities by covering the eyes with an aluminum foil mask and deafferenting chemoreceptors with freshwater¹⁹.
4. Adjust the environment to suit a LEGO robot (removing water if necessary) and run the robot in the same controlled environment.
5. Use a MATLAB script to automatically track the markers on the specimens. We use MouseLabTracker²⁰.
6. Create and compare vector plots of the animal and robot movements.
7. Modify the nervous system and repeat steps 3.3-3.6 to observe how different aspects of the electronic nervous system affects the robot's behavior. Here we adjust the synaptic strength between the claw bump sensor neurons and the motor system and monitor robot performance compared to that of a lobster. Alternatively, the sensory conditions in the test arena can be modified and the effects observed in robot and animal. For example, the arena described here could be subjected to different water flow rates.

Representative Results

Inputs from a lobster's claws into its nervous system mediate obstacle negotiation in a novel environment. Figure 1 shows a screenshot of the video used to analyze the behavior of a LEGO robot (Figure 1A) and a lobster (Figure 1B) in the test arena. The test arena was unmodified between animal and robot tests except that the water was emptied from the tank for the robot trials.

Video tracking results are displayed in Figure 2. Animal and robot travel paths were automatically tracked using MouseLabTracker²⁰ in MATLAB. Representative paths for lobster (black lines) and LEGO robot trials (colored lines) are shown. In the robot trials, synaptic strength from the claw bump sensory neurons into the nervous system was varied and represented by different colored paths (red, low excitation; blue, medium excitation; green, high excitation). The hypothetical nervous system with mid level excitatory connections from the claw bump sensory neurons produced similar behaviors to the animal. Quantitative parameters can be extracted from the data such as path length and average walking speed (Table I).

Figure 1. Overhead video frames of the test arena for LEGO robot (A) and lobster (B) trials. Conditions were kept identical except that water was added to the tank for the animal trials. The high contrast video image facilitates automatic tracking using MATLAB software.

Figure 2. Tracking of lobster (black lines) and LEGO robot (colored lines) paths through the test arena shown in Figure 1. On the robot, the synaptic strength from claw bump sensor neurons was varied (red, low excitation; blue, medium excitation; green, high excitation). The * symbol denotes the starting position and filled black shapes show the location of obstacles. Since head position was being tracked, apparent obstacle contact varies due to variability in claw position.

	Average path distance (cm)	Average travel speed (cm)
Lobster	177.37	1.03
LEGO Robot (high excitation)	162.72	0.94
LEGO Robot (mid excitation)	165.32	0.85
LEGO Robot (low excitation)	51.02	0.69

Table I. Averaged parameters from analysis of video behavior data for lobsters and for LEGO robots with varied strengths of input synapses from the claw bump sensory system.

Discussion

When initiating biorobotic nervous system simulation experiments, there are a few important guidelines to follow. Choosing the right model organism is critical: pick an organism that is easy to obtain and maintain. Invertebrates are ideal because they do not usually require institutional approval for experimentation and their husbandry needs are often less demanding than those of vertebrates. From a scientific perspective, it is beneficial to choose an animal that has an established neuroethological literature record from which to draw hypotheses for testing²¹.

Just as important as choosing a suitable organism is the selection of appropriate behavioral reflexes for study. The LEGO NXT platform can only handle four sensors without custom modifications so selecting a relatively simple sensory system with limited sensors is important. Simple behavioral reflexes are often well described and provide more flexibility in the hypotheses that can be tested. The computing power of the NXT is a limiting factor too so efforts should be made to reduce the scope of the nervous system simulation. Scientists often publish proposed neural networks explaining specific behaviors and it is easiest to select one of these. Simple taxes and kineses are well suited to this biorobotic approach²².

While this approach is useful for rapid testing of nervous system hypotheses, there are several limitations to keep in mind. The scope of testing possible is limited. Complicated sensory systems cannot be used with this platform. For example, complex visual processing is beyond the capability of the NXT processor but related questions could be addressed by investigating simpler components of the visual system, such as optical flow processing. The sensor suite for the NXT provides some inherent limitations as to the scope of investigations available. While custom sensors can be developed and used on the NXT, this is a time consuming process that requires electronics expertise, possibly negating the advantages of the platform for rapid experimental implementation.

While there are limitations on the range of hypotheses that can be tested, this approach provides a unique tool to early stage implementation of embodied nervous system simulations. The LEGO Mindstorms NXT platform can serve as a stepping stone to a more thorough testing of systems level hypotheses as demonstrated by such robots as RoboLamprey⁹ and Barbara Webb's robotic cricket²³. Simple components of extensive biomimetic control architectures can be tested in a rapid and straightforward manner. The method can also advance bio-hybrid simulations that control virtual agents to include embodied vehicle control²⁴, and it can serve as a biomimetic control framework to interface between brain and machine²⁵.

Using the LEGO Mindstorms platform allows for the transfer of this biorobotic approach to a broader audience in schools and laboratories outside of robotics. The tools have been used by high school and college students¹¹ and provide an exciting inquiry-based method to teach neuroscience, robotics, and the value of interdisciplinary science. Laboratories without the equipment or background needed to build robots can use this well-supported mainstream robotics kit to use robots as a tool in their work. The biorobotic approach could apply to research in a range of fields, from neuroscience and ethology to biomechanics and social behavior.