a-micro-ct-based-method-for-characterizing-lesions-locating

A Micro-CT-based Method for Characterizing Lesions and Locating Electrodes in Small Animal Brains

Lesion and electrode location verification are traditionally done via histological examination of stained brain slices, a time-consuming procedure that ...

Harvard University

The human visual system is able to recognize objects despite tremendous variation in their appearance on the retina resulting from variation in view, size, lighting, etc. This ability--known as "invariant" object recognition--is central to visual perception, yet its computational underpinnings are poorly understood. Traditionally, nonhuman primates have been the animal model-of-choice for investigating the neuronal substrates of invariant recognition, because their visual systems closely mirror our own. Meanwhile, simpler and more accessible animal models such as rodents have been largely overlooked as possible models of higher-level visual functions, because their brains are often assumed to lack advanced visual processing machinery. As a result, little is known about rodents' ability to process complex visual stimuli in the face of real-world image variation. In the present work, we show that rats possess more advanced visual abilities than previously appreciated. Specifically, we trained pigmented rats to perform a visual task that required them to recognize objects despite substantial variation in their appearance, due to changes in size, view, and lighting. Critically, rats were able to spontaneously generalize to previously unseen transformations of learned objects. These results provide the first systematic evidence for invariant object recognition in rats and argue for an increased focus on rodents as models for studying high-level visual processing.

A rodent model for the study of invariant visual object recognition.

Primates can easily identify visual objects over large changes in retinal position--a property commonly referred to as position "invariance." This ability is widely assumed to depend on neurons in inferior temporal cortex (IT) that can respond selectively to isolated visual objects over similarly large ranges of retinal position. However, in the real world, objects rarely appear in isolation, and the interplay between position invariance and the representation of multiple objects (i.e., clutter) remains unresolved. At the heart of this issue is the intuition that the representations of nearby objects can interfere with one another and that the large receptive fields needed for position invariance can exacerbate this problem by increasing the range over which interference acts. Indeed, most IT neurons' responses are strongly affected by the presence of clutter. While external mechanisms (such as attention) are often invoked as a way out of the problem, we show (using recorded neuronal data and simulations) that the intrinsic properties of IT population responses, by themselves, can support object recognition in the face of limited clutter. Furthermore, we carried out extensive simulations of hypothetical neuronal populations to identify the essential individual-neuron ingredients of a good population representation. These simulations show that the crucial neuronal property to support recognition in clutter is not preservation of response magnitude, but preservation of each neuron's rank-order object preference under identity-preserving image transformations (e.g., clutter). Because IT neuronal responses often exhibit that response property, while neurons in earlier visual areas (e.g., V1) do not, we suggest that preserving the rank-order object preference regardless of clutter, rather than the response magnitude, more precisely describes the goal of individual neurons at the top of the ventral visual stream.

What response properties do individual neurons need to underlie position and clutter "invariant" object recognition?

While many models of biological object recognition share a common set of "broad-stroke" properties, the performance of any one model depends strongly on the choice of parameters in a particular instantiation of that model--e.g., the number of units per layer, the size of pooling kernels, exponents in normalization operations, etc. Since the number of such parameters (explicit or implicit) is typically large and the computational cost of evaluating one particular parameter set is high, the space of possible model instantiations goes largely unexplored. Thus, when a model fails to approach the abilities of biological visual systems, we are left uncertain whether this failure is because we are missing a fundamental idea or because the correct "parts" have not been tuned correctly, assembled at sufficient scale, or provided with enough training. Here, we present a high-throughput approach to the exploration of such parameter sets, leveraging recent advances in stream processing hardware (high-end NVIDIA graphic cards and the PlayStation 3's IBM Cell Processor). In analogy to high-throughput screening approaches in molecular biology and genetics, we explored thousands of potential network architectures and parameter instantiations, screening those that show promising object recognition performance for further analysis. We show that this approach can yield significant, reproducible gains in performance across an array of basic object recognition tasks, consistently outperforming a variety of state-of-the-art purpose-built vision systems from the literature. As the scale of available computational power continues to expand, we argue that this approach has the potential to greatly accelerate progress in both artificial vision and our understanding of the computational underpinning of biological vision.

A high-throughput screening approach to discovering good forms of biologically inspired visual representation.

Much of neurophysiology and vision science relies on careful measurement of a human or animal subject's gaze direction. Video-based eye trackers have emerged as an especially popular option for gaze tracking, because they are easy to use and are completely non-invasive. However, video eye trackers typically require a calibration procedure in which the subject must look at a series of points at known gaze angles. While it is possible to rely on innate orienting behaviors for calibration in some non-human species, other species, such as rodents, do not reliably saccade to visual targets, making this form of calibration impossible. To overcome this problem, we developed a fully automated infrared video eye-tracking system that is able to quickly and accurately calibrate itself without requiring co-operation from the subject. This technique relies on the optical geometry of the cornea and uses computer-controlled motorized stages to rapidly estimate the geometry of the eye relative to the camera. The accuracy and precision of our system was carefully measured using an artificial eye, and its capability to monitor the gaze of rodents was verified by tracking spontaneous saccades and evoked oculomotor reflexes in head-fixed rats (in both cases, we obtained measurements that are consistent with those found in the literature). Overall, given its fully automated nature and its intrinsic robustness against operator errors, we believe that our eye-tracking system enhances the utility of existing approaches to gaze-tracking in rodents and represents a valid tool for rodent vision studies.

A self-calibrating, camera-based eye tracker for the recording of rodent eye movements.

'High-level' vision lacks a single, agreed upon definition, but it might usefully be defined as those stages of visual processing that transition from analyzing local image structure to analyzing structure of the external world that produced those images. Much work in the last several decades has focused on object recognition as a framing problem for the study of high-level visual cortex, and much progress has been made in this direction. This approach presumes that the operational goal of the visual system is to read-out the identity of an object (or objects) in a scene, in spite of variation in the position, size, lighting and the presence of other nearby objects. However, while object recognition as a operational framing of high-level is intuitive appealing, it is by no means the only task that visual cortex might do, and the study of object recognition is beset by challenges in building stimulus sets that adequately sample the infinite space of possible stimuli. Here I review the successes and limitations of this work, and ask whether we should reframe our approaches to understanding high-level vision.

Do we understand high-level vision?

Brains are, at a fundamental level, biological computing machines. They transform a torrent of complex and ambiguous sensory information into coherent thought and action, allowing an organism to perceive and model its environment, synthesize and make decisions from disparate streams of information, and adapt to a changing environment. Against this backdrop, it is perhaps not surprising that computer science, the science of building artificial computational systems, has long looked to biology for inspiration. However, while the opportunities for cross-pollination between neuroscience and computer science are great, the road to achieving brain-like algorithms has been long and rocky. Here, we review the historical connections between neuroscience and computer science, and we look forward to a new era of potential collaboration, enabled by recent rapid advances in both biologically-inspired computer vision and in experimental neuroscience methods. In particular, we explore where neuroscience-inspired algorithms have succeeded, where they still fail, and we identify areas where deeper connections are likely to be fruitful. 

Neural networks and neuroscience-inspired computer vision.

Editorial: What can simple brains teach us about how vision works.

Lesion verification and quantification is traditionally done via histological examination of sectioned brains, a time-consuming process that relies heavily on manual estimation. Such methods are particularly problematic in posterior cortical regions (e.g. visual cortex), where sectioning leads to significant damage and distortion of tissue. Even more challenging is the post hoc localization of micro-electrodes, which relies on the same techniques, suffers from similar drawbacks and requires even higher precision. Here, we propose a new, simple method for quantitative lesion characterization and electrode localization that is less labor-intensive and yields more detailed results than conventional methods. We leverage staining techniques standard in electron microscopy with the use of commodity micro-CT imaging. We stain whole rat and zebra finch brains in osmium tetroxide, embed these in resin and scan entire brains in a micro-CT machine. The scans result in 3D reconstructions of the brains with section thickness dependent on sample size (12-15 and 5-6 microns for rat and zebra finch respectively) that can be segmented manually or automatically. Because the method captures the entire intact brain volume, comparisons within and across studies are more tractable, and the extent of lesions and electrodes may be studied with higher accuracy than with current methods.

A micro-CT-based method for quantitative brain lesion characterization and electrode localization.

Machine learning is a field of computer science that builds algorithms that learn. In many cases, machine learning algorithms are used to recreate a human ability like adding a caption to a photo, driving a car, or playing a game. While the human brain has long served as a source of inspiration for machine learning, little effort has been made to directly use data collected from working brains as a guide for machine learning algorithms. Here we demonstrate a new paradigm of "neurally-weighted" machine learning, which takes fMRI measurements of human brain activity from subjects viewing images, and infuses these data into the training process of an object recognition learning algorithm to make it more consistent with the human brain. After training, these neurally-weighted classifiers are able to classify images without requiring any additional neural data. We show that our neural-weighting approach can lead to large performance gains when used with traditional machine vision features, as well as to significant improvements with already high-performing convolutional neural network features. The effectiveness of this approach points to a path forward for a new class of hybrid machine learning algorithms which take both inspiration and direct constraints from neuronal data.

Using human brain activity to guide machine learning.

Thus far, optical recording of neuronal activity in freely behaving animals has been limited to a thin axial range. We present a head-mounted miniaturized light-field microscope (MiniLFM) capable of capturing neuronal network activity within a volume of 700 × 600 × 360 µm at 16 Hz in the hippocampus of freely moving mice. We demonstrate that neurons separated by as little as ~15 µm and at depths up to 360 µm can be discriminated.

High-speed volumetric imaging of neuronal activity in freely moving rodents.

In the version of this Brief Communication originally published online, ref. 21 included details for a conference paper (Pegard, N. C. et al. Paper presented at Novel Techniques in Microscopy: Optics in the Life Sciences, Vancouver, BC, Canada, 12-15 April 2015). The correct reference is the following: Pégard, N. C. et al. Optica 3, 517-524 (2016). This error has been corrected in the print, HTML and PDF versions of the paper.

Article	Year
A rodent model for the study of invariant visual object recognition. Proceedings of the National Academy of Sciences of the United States of America\| PubMed ID: 19429704	2009
What response properties do individual neurons need to underlie position and clutter "invariant" object recognition? Journal of neurophysiology\| PubMed ID: 19439676	2009
A high-throughput screening approach to discovering good forms of biologically inspired visual representation. PLoS computational biology\| PubMed ID: 19956750	2009
A self-calibrating, camera-based eye tracker for the recording of rodent eye movements. Frontiers in neuroscience\| PubMed ID: 21152259	2010
Do we understand high-level vision? Current opinion in neurobiology\| PubMed ID: 24552691	2014
Neural networks and neuroscience-inspired computer vision. Current biology : CB\| PubMed ID: 25247371	2014
Editorial: What can simple brains teach us about how vision works. Frontiers in neural circuits\| PubMed ID: 26483639	2015
A micro-CT-based method for quantitative brain lesion characterization and electrode localization. Scientific reports\| PubMed ID: 29581439	2018
Using human brain activity to guide machine learning. Scientific reports\| PubMed ID: 29599461	2018
High-speed volumetric imaging of neuronal activity in freely moving rodents. Nature methods\| PubMed ID: 29736000	2018
Author Correction: High-speed volumetric imaging of neuronal activity in freely moving rodents. Nature methods\| PubMed ID: 29786093	2018

David D. Cox

Biography

JoVE Journal Publications

Other Publications