A Step-by-Step Implementation of DeepBehavior, Deep Learning Toolbox for Automated Behavior Analysis

Summary

The purpose of this protocol is to utilize pre-built convolutional neural nets to automate behavior tracking and perform detailed behavior analysis. Behavior tracking can be applied to any video data or sequences of images and is generalizable to track any user-defined object.

Abstract

Understanding behavior is the first step to truly understanding neural mechanisms in the brain that drive it. Traditional behavioral analysis methods often do not capture the richness inherent to the natural behavior. Here, we provide detailed step-by-step instructions with visualizations of our recent methodology, DeepBehavior. The DeepBehavior toolbox uses deep learning frameworks built with convolutional neural networks to rapidly process and analyze behavioral videos. This protocol demonstrates three different frameworks for single object detection, multiple object detection, and three-dimensional (3D) human joint pose tracking. These frameworks return cartesian coordinates of the object of interest for each frame of the behavior video. Data collected from the DeepBehavior toolbox contain much more detail than traditional behavior analysis methods and provides detailed insights to the behavior dynamics. DeepBehavior quantifies behavior tasks in a robust, automated, and precise way. Following the identification of behavior, post-processing code is provided to extract information and visualizations from the behavioral videos.

Introduction

A detailed analysis of behavior is key to understanding the brain and behavior relationships. There have been many exciting advances in methodologies for recording and manipulating neuronal populations with high temporal resolution, however, behavior analysis methods have not developed at the same rate and are limited to indirect measurements and a reductionist approach¹. Recently, deep learning based methods have been developed to perform automated and detailed behavior analysis^2,3,4,5. This protocol provides a step-....

Protocol

1. GPU and Python Setup

1. GPU Software
   When the computer is first setup for deep learning applications, GPU-appropriate software and drivers should be installed which can be found on the GPU's respective website. (see the Table of Materials for those used in this study).
2. Python 2.7 Installation
   Open a command line prompt on your machine.
   Command line: sudo apt-get install python-pip python-dev python-virtualenv

2. TENSORBOX

1. Tensorbox Setup
   1. Create Virtual Environment for Tensorbox

Results

When the protocol is followed, the data for each network architecture should be similar to the following. For TensorBox, it outputs a bounding box around the object of interest. In our example, we used videos from a food pellet reaching task, and labeled the right paws to track their movement. As seen in Figure 1, the right paw can be detected in different positions in both the front view and side view cameras. After post-processing with camera calibration, 3.......

Discussion

Here, we provide a step-by-step guide for implementation of DeepBehavior, our recently developed deep learning based toolbox for animal and human behavior imaging data analysis². We provide detailed explanations for each step for installation of the frameworks for each network architecture, and provide links for installation of the open-source requirements to be able to run these frameworks. We demonstrate how to install them, how to create training data, how to train the network, and how to proce.......

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
CUDA v8.0.61	NVIDIA	n/a	GPU Software
MATLAB R2016b	Mathworks	n/a	Matlab
Python 2.7	Python	n/a	Python Version
Quadro P6000	NVIDIA	n/a	GPU Processor
Ubuntu v16.04	Ubuntu	n/a	Operating System