design-use-an-apparatus-for-presenting-graspable-objects-3d

Design and Use of an Apparatus for Presenting Graspable Objects in 3D Workspace

Reaching and grasping are highly-coupled movements, and their underlying neural dynamics have been widely studied in the last decade. To distinguish ...

Zhejiang University

A practical brain-machine interface (BMI) requires real-time decoding algorithms to be realised in a portable device rather than a personal computer. In this article, a field-programmable gate array (FPGA) implementation of a probabilistic neural network (PNN) is proposed and developed to decode motor cortical ensemble recordings in rats performing a lever-pressing task for water rewards. A chronic 16-channel microelectrode array was implanted into the primary motor cortex of the rat to record neural activity, and the pressure signal of the lever were recorded simultaneously. To decode the pressure value from neural activity, both Matlab-based and FPGA-based mapping algorithms using a PNN were implemented and evaluated. In the FPGA architecture, training data of the network were stored in random access memory (RAM) blocks and multiply-add operations were realised by on-chip DSP48E slices. In the approximation of the activation function, a Taylor series and a look-up table (LUT) are used to achieve an accurate approximation. The results of FPGA implementation are as accurate as the realisation of Matlab, but the running speed is 37.9 times faster. This novel and feasible method indicates that the performance of current FPGAs is competent for portable BMI applications.

Field-programmable gate array implementation of a probabilistic neural network for motor cortical decoding in rats.

Real-time computation, portability and flexibility are crucial for practical brain-machine interface (BMI) applications. In this work, we proposed Hardware Processing Modules (HPMs) as a method for accelerating BMI computation. Two HPMs have been developed. One is the field-programmable gate array (FPGA) implementation of spike sorting based on probabilistic neural network (PNN), and the other is the FPGA implementation of neural ensemble decoding based on Kalman filter (KF). These two modules were configured under the same framework and tested with real data from motor cortex recording in rats performing a lever-pressing task for water rewards. Due to the parallelism feature of FPGA, the computation time was reduced by several dozen times, while the results are almost the same as those from Matlab implementations. Such HPMs provide a high performance coprocessor for neural signal computation.

FPGA implementation of hardware processing modules as coprocessors in brain-machine interfaces.

Recently, local field potentials (LFPs) have been successfully used to extract information of arm and hand movement in some brain-machine interfaces (BMIs) studies, which suggested that LFPs would improve the performance of BMI applications because of its long-term stability. However, the performance of LFPs in different frequency bands has not been investigated in decoding hand grasp types. Here, the LFPs from the monkey's dorsal premotor cortices were collected by microelectrode array when monkey was performing grip-specific grasp task. A K-nearest neighbor classifier performed on the power spectrum of LFPs was used to decode grasping movements. The decoding powers of LFPs in different frequency bands, channels and trials used for training were also studied. The results show that the broad high frequency band (200-400Hz) LFPs achieved the best performance with classification accuracy reaching over 0.9. It infers that high frequency band LFPs in PMd cortex could be a promising source of control signals in developing functional BMIs for hand grasping.

Decoding grasp types with high frequency of local field potentials from primate primary dorsal premotor cortex.

Local field potentials (LFP) are valuable signals for decoding motor kinematics in brain machine interfaces (BMIs). To take full advantage of LFPs, however, more systematic investigation of the relationship between LFPs and ipsilateral limb movement is required. In this paper we investigated the decoding power of LFPs for the ipsilateral wrist movement from two monkeys performing a 2D center-out task. The results show that LFPs could predict the ipsilateral wrist position and velocity with high accuracy, which is comparable to the accuracy of decoding the contralateral kinematics. Furthermore, similar to contralateral decoding, the low (0.3-5 Hz, 5-15 Hz) and high (100-200 Hz, 200-400 Hz) frequency bands resulted in significantly better decoding performance than the medium frequency bands. These results suggest that ipsilateral LFPs could be used to build better BMIs in similar ways of using contralateral LFPs.

Decoding wrist kinematics from local field potentials of the ipsilateral primary motor and dorsal premotor cortices.

The high-dimensional neural recordings bring computational challenges to movement decoding in motor brain machine interfaces (mBMI), especially for portable applications. However, not all recorded neural activities relate to the execution of a certain movement task. This paper proposes to use a local-learning-based method to perform neuron selection for the gesture prediction in a reaching and grasping task.

Local-learning-based neuron selection for grasping gesture prediction in motor brain machine interfaces.

Previous studies have shown that the dorsal premotor cortex (PMd) neurons are relevant to reaching as well as grasping. In order to investigate their specific contribution to reaching and grasping, respectively, we design two experimental paradigms to separate these two factors. Two monkeys are instructed to reach in four directions but grasp the same object and grasp four different objects but reach in the same direction. Activities of the neuron ensemble in PMd of the two monkeys are collected while performing the tasks. Mutual information (MI) is carried out to quantitatively evaluate the neurons' tuning property in both tasks. We find that there exist neurons in PMd that are tuned only to reaching, tuned only to grasping, and tuned to both tasks. When applied with a support vector machine (SVM), the movement decoding accuracy by the tuned neuron subset in either task is quite close to the performance by full ensemble. Furthermore, the decoding performance improves significantly by adding the neurons tuned to both tasks into the neurons tuned to one property only. These results quantitatively distinguish the diversity of the neurons tuned to reaching and grasping in the PMd area and verify their corresponding contributions to BMI decoding.

Information analysis on neural tuning in dorsal premotor cortex for reaching and grasping.

Recent studies have shown that dorsal premotor cortex (PMd), a cortical area in the dorsomedial grasp pathway, is involved in grasp movements. However, the neural ensemble firing property of PMd during grasp movements and the extent to which it can be used for grasp decoding are still unclear.

Distinct neural patterns enable grasp types decoding in monkey dorsal premotor cortex.

 Previous studies have demonstrated that target direction information presented by the dorsal premotor cortex (PMd) during movement planning could be incorporated into neural decoder for achieving better decoding performance. It is still unknown whether the neural decoder combined with only target direction could work in more complex tasks where obstacles impeded direct reaching paths.  In this study, spike activities were collected from the PMd of two monkeys when performing a delayed obstacle-avoidance task. We examined how target direction and intended movement selection were encoded in neuron population activities of the PMd during movement planning. The decoding performances of movement trajectory were compared for three neural decoders with no prior knowledge, or only target direction, or both target direction and intended movement selection integrated into a mixture of trajectory model (MTM).  We found that not only target direction but also intended movement selection was presented in neural activities of the PMd during movement planning. It was further confirmed by quantitative analysis. Combined with prior knowledge, the trajectory decoder achieved the best performance among three decoders.  Recruiting prior knowledge about target direction and intended movement selection extracted from the PMd could enhance the decoding performance of hand trajectory in indirect reaching movement.

Prior Knowledge of Target Direction and Intended Movement Selection Improves Indirect Reaching Movement Decoding.

It has been suggested that the brain adopts a simplified strategy to coordinate a large number of degrees of freedom in motor control. Synergies have been proposed as a strategy to produce movements by recruitment of a small number of fixed modular patterns. However, there is no direct support for a synergistic organization of the brain itself. In this study, we recorded neural activities from the dorsal premotor cortex (PMd) of monkeys trained to reach and grasp differently shaped objects (grasping task) or the same object in different positions (reaching task). Non-negative matrix factorization (NNMF) was applied to the neural data to extract neural synergies, whose functional roles were verified in several ways. We found that motor cortex used similar neural synergies for grasping different objects; combining only a few of the synergies accounted for most of the variance in the original data. When used for single-trial task decoding, the synergy coefficients performed as well and robustly as the original data in both tasks. The synergy amplitudes for each unit were significantly correlated with the corresponding neuron's firing rate. In addition, we also observed synergies shared between tasks and task-specific synergies, as shown before for muscle synergies. Altogether, we demonstrated that neural synergies are effective in describing neural population activity during reach to grasp movements and provide a new tool for interpreting neural data for movement control.

Neural synergies for controlling reach and grasp movement in macaques.

The neural principles underlying reaching and grasping movements have been studied extensively in primates for decades. However, few experimental apparatuses have been developed to enable a flexible combination of reaching and grasping in one task in three-dimensional (3D) space.

Article	Year
Field-programmable gate array implementation of a probabilistic neural network for motor cortical decoding in rats. Journal of neuroscience methods\| PubMed ID: 19879294	2010
FPGA implementation of hardware processing modules as coprocessors in brain-machine interfaces. Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual Conference\| PubMed ID: 22255365	2011
Decoding grasp types with high frequency of local field potentials from primate primary dorsal premotor cortex. Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual Conference\| PubMed ID: 23366234	2012
Decoding wrist kinematics from local field potentials of the ipsilateral primary motor and dorsal premotor cortices. Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual Conference\| PubMed ID: 23367398	2012
Local-learning-based neuron selection for grasping gesture prediction in motor brain machine interfaces. Journal of neural engineering\| PubMed ID: 23428877	2013
Information analysis on neural tuning in dorsal premotor cortex for reaching and grasping. Computational and mathematical methods in medicine\| PubMed ID: 23781275	2013
Distinct neural patterns enable grasp types decoding in monkey dorsal premotor cortex. Journal of neural engineering\| PubMed ID: 25380169	2014
Prior Knowledge of Target Direction and Intended Movement Selection Improves Indirect Reaching Movement Decoding. Behavioural neurology\| PubMed ID: 28490836	2017
Neural synergies for controlling reach and grasp movement in macaques. Neuroscience\| PubMed ID: 28647390	2017
An automated behavioral apparatus to combine parameterized reaching and grasping movements in 3D space. Journal of neuroscience methods\| PubMed ID: 30502371	2019

Yaoyao Hao

Biography

JoVE Journal Publications

Other Publications