serial-two-photon-tomography-whole-marmoset-brain-for-neuroanatomical

Serial Two-Photon Tomography of the Whole Marmoset Brain for Neuroanatomical Analyses

Serial Two-photon tomography (STPT) is a technique to image a mass of tissue in its three-dimensional shape by combining two-photon imaging with automatic ...

RIKEN Center for Brain Science

We describe our connectomics pipeline for processing anterograde tracer injection data for the brain of the common marmoset (Callithrix jacchus). Brain sections were imaged using a batch slide scanner (NanoZoomer 2.0-HT) and we used artificial intelligence to precisely segment the tracer signal from the background in the fluorescence images. The shape of each brain was reconstructed by reference to a block-face and all data were mapped into a common 3D brain space with atlas and 2D cortical flat map. To overcome the effect of using a single template atlas to specify cortical boundaries, brains were cyto- and myelo-architectonically annotated to create individual 3D atlases. Registration between the individual and common brain cortical boundaries in the flat map space was done to absorb the variation of each brain and precisely map all tracer injection data into one cortical brain space. We describe the methodology of our pipeline and analyze the accuracy of our tracer segmentation and brain registration approaches. Results show our pipeline can successfully process and normalize tracer injection experiments into a common space, making it suitable for large-scale connectomics studies with a focus on the cerebral cortex.

The NanoZoomer artificial intelligence connectomics pipeline for tracer injection studies of the marmoset brain.

Diffusion-weighted magnetic resonance imaging (dMRI) allows non-invasive investigation of whole-brain connectivity, which can reveal the brain's global network architecture and also abnormalities involved in neurological and mental disorders. However, the reliability of connection inferences from dMRI-based fiber tracking is still debated, due to low sensitivity, dominance of false positives, and inaccurate and incomplete reconstruction of long-range connections. Furthermore, parameters of tracking algorithms are typically tuned in a heuristic way, which leaves room for manipulation of an intended result. Here we propose a general data-driven framework to optimize and validate parameters of dMRI-based fiber tracking algorithms using neural tracer data as a reference. Japan's Brain/MINDS Project provides invaluable datasets containing both dMRI and neural tracer data from the same primates. A fundamental difference when comparing dMRI-based tractography and neural tracer data is that the former cannot specify the direction of connectivity; therefore, evaluating the fitting of dMRI-based tractography becomes challenging. The framework implements multi-objective optimization based on the non-dominated sorting genetic algorithm II. Its performance is examined in two experiments using data from ten subjects for optimization and six for testing generalization. The first uses a seed-based tracking algorithm, iFOD2, and objectives for sensitivity and specificity of region-level connectivity. The second uses a global tracking algorithm and a more refined set of objectives: distance-weighted coverage, true/false positive ratio, projection coincidence, and commissural passage. In both experiments, with optimized parameters compared to default parameters, fiber tracking performance was significantly improved in coverage and fiber length. Improvements were more prominent using global tracking with refined objectives, achieving an average fiber length from 10 to 17 mm, voxel-wise coverage of axonal tracts from 0.9 to 15%, and the correlation of target areas from 40 to 68%, while minimizing false positives and impossible cross-hemisphere connections. Optimized parameters showed good generalization capability for test brain samples in both experiments, demonstrating the flexible applicability of our framework to different tracking algorithms and objectives. These results indicate the importance of data-driven adjustment of fiber tracking algorithms and support the validity of dMRI-based tractography, if appropriate adjustments are employed.

Optimization and validation of diffusion MRI-based fiber tracking with neural tracer data as a reference.

In building biological neural network models, it is crucial to efficiently convert diverse anatomical and physiological data into parameters of neurons and synapses and to systematically estimate unknown parameters in reference to experimental observations. Web-based tools for systematic model building can improve the transparency and reproducibility of computational models and can facilitate collaborative model building, validation, and evolution. Here, we present a framework to support collaborative data-driven development of spiking neural network (SNN) models based on the Entity-Relationship (ER) data description commonly used in large-scale business software development. We organize all data attributes, including species, brain regions, neuron types, projections, neuron models, and references as tables and relations within a database management system (DBMS) and provide GUI interfaces for data registration and visualization. This allows a robust "business-oriented" data representation that supports collaborative model building and traceability of source information for every detail of a model. We tested this data-to-model framework in cortical and striatal network models by successfully combining data from papers with existing neuron and synapse models and by generating NEST simulation codes for various network sizes. Our framework also helps to check data integrity and consistency and data comparisons across species. The framework enables the modeling of any region of the brain and is being deployed to support the integration of anatomical and physiological datasets from the brain/MINDS project for systematic SNN modeling of the marmoset brain.

A Spiking Neural Network Builder for Systematic Data-to-Model Workflow.

We present MiniVess, the first annotated dataset of rodent cerebrovasculature, acquired using two-photon fluorescence microscopy. MiniVess consists of 70 3D image volumes with segmented ground truths. Segmentations were created using traditional image processing operations, a U-Net, and manual proofreading. Code for image preprocessing steps and the U-Net are provided. Supervised machine learning methods have been widely used for automated image processing of biomedical images. While much emphasis has been placed on the development of new network architectures and loss functions, there has been an increased emphasis on the need for publicly available annotated, or segmented, datasets. Annotated datasets are necessary during model training and validation. In particular, datasets that are collected from different labs are necessary to test the generalizability of models. We hope this dataset will be helpful in testing the reliability of machine learning tools for analyzing biomedical images.

A dataset of rodent cerebrovasculature from in vivo multiphoton fluorescence microscopy imaging.

Understanding prefrontal cortex projections to diencephalic-mesencephalic junction (DMJ), especially to subthalamic nucleus (STN) and ventral mesencephalic tegmentum (VMT) helps our comprehension of Deep Brain Stimulation (DBS) in major depression (MD) and obsessive-compulsive disorder (OCD). Fiber routes are complex and tract tracing studies in non-human primate species (NHP) have yielded conflicting results. The superolateral medial forebrain bundle (slMFB) is a promising target for DBS in MD and OCD. It has become a focus of criticism owing to its name and its diffusion weighted-imaging based primary description.

Tomographic tract tracing and data driven approaches to unravel complex 3D fiber anatomy of DBS relevant prefrontal projections to the diencephalic-mesencephalic junction in the marmoset.

Magnetic resonance imaging (MRI) is a non-invasive neuroimaging technique that is useful for identifying normal developmental and aging processes and for data sharing. Marmosets have a relatively shorter life expectancy than other primates, including humans, because they grow and age faster. Therefore, the common marmoset model is effective in aging research. The current study investigated the aging process of the marmoset brain and provided an open MRI database of marmosets across a wide age range. The Brain/MINDS Marmoset Brain MRI Dataset contains brain MRI information from 216 marmosets ranging in age from 1 and 10 years. At the time of its release, it is the largest public dataset in the world. It also includes multi-contrast MRI images. In addition, 91 of 216 animals have corresponding high-resolution ex vivo MRI datasets. Our MRI database, available at the Brain/MINDS Data Portal, might help to understand the effects of various factors, such as age, sex, body size, and fixation, on the brain. It can also contribute to and accelerate brain science studies worldwide.

Multi-modal brain magnetic resonance imaging database covering marmosets with a wide age range.

The prefrontal cortex (PFC) has dramatically expanded in primates, but its organization and interactions with other brain regions are only partially understood. We performed high-resolution connectomic mapping of the marmoset PFC and found two contrasting corticocortical and corticostriatal projection patterns: "patchy" projections that formed many columns of submillimeter scale in nearby and distant regions and "diffuse" projections that spread widely across the cortex and striatum. Parcellation-free analyses revealed representations of PFC gradients in these projections' local and global distribution patterns. We also demonstrated column-scale precision of reciprocal corticocortical connectivity, suggesting that PFC contains a mosaic of discrete columns. Diffuse projections showed considerable diversity in the laminar patterns of axonal spread. Altogether, these fine-grained analyses reveal important principles of local and long-distance PFC circuits in marmosets and provide insights into the functional organization of the primate brain.

Local and long-distance organization of prefrontal cortex circuits in the marmoset brain.

The primate brain has unique anatomical characteristics, which translate into advanced cognitive, sensory, and motor abilities. Thus, it is important that we gain insight on its structure to provide a solid basis for models that will clarify function. Here, we report on the implementation and features of the Brain/MINDS Marmoset Connectivity Resource (BMCR), a new open-access platform that provides access to high-resolution anterograde neuronal tracer data in the marmoset brain, integrated to retrograde tracer and tractography data. Unlike other existing image explorers, the BMCR allows visualization of data from different individuals and modalities in a common reference space. This feature, allied to an unprecedented high resolution, enables analyses of features such as reciprocity, directionality, and spatial segregation of connections. The present release of the BMCR focuses on the prefrontal cortex (PFC), a uniquely developed region of the primate brain that is linked to advanced cognition, including the results of 52 anterograde and 164 retrograde tracer injections in the cortex of the marmoset. Moreover, the inclusion of tractography data from diffusion MRI allows systematic analyses of this noninvasive modality against gold-standard cellular connectivity data, enabling detection of false positives and negatives, which provide a basis for future development of tractography. This paper introduces the BMCR image preprocessing pipeline and resources, which include new tools for exploring and reviewing the data.

The Brain/MINDS Marmoset Connectivity Resource: An open-access platform for cellular-level tracing and tractography in the primate brain.

Pairwise image registration is a necessary prerequisite for brain image comparison and data integration in neuroscience and radiology. In this work, we explore the efficacy of implicit neural representations (INRs) in improving the performance of brain image registration in magnetic resonance imaging. In this setting, INRs serve as a continuous and coordinate based approximation of the deformation field obtained through a multi-layer perceptron. Previous research has demonstrated that sinusoidal representation networks (SIRENs) surpass ReLU models in performance. In this study, we first broaden the range of activation functions to further investigate the registration performance of implicit networks equipped with activation functions that exhibit diverse oscillatory properties. Specifically, in addition to the SIRENs and ReLU, we evaluate activation functions based on snake, sine+, chirp and Morlet wavelet functions. Second, we conduct experiments to relate the hyper-parameters of the models to registration performance. Third, we propose and assess various techniques, including cycle consistency loss, ensembles and cascades of implicit networks, as well as a combined image fusion and registration objective, to enhance the performance of implicit registration networks beyond the standard approach. The investigated implicit methods are compared to the VoxelMorph convolutional neural network and to the symmetric image normalization (SyN) registration algorithm from the Advanced Normalization Tools (ANTs). Our findings not only highlight the remarkable capabilities of implicit networks in addressing pairwise image registration challenges, but also showcase their potential as a powerful and versatile off-the-shelf tool in the fields of neuroscience and radiology.

Exploring the performance of implicit neural representations for brain image registration.

Neurons receive, process, and integrate inputs. These operations are organized by dendrite arbor morphology, and the dendritic arborization (da) neurons of the  peripheral sensory nervous system are an excellent experimental model for examining the differentiation processes that build and shape the dendrite arbor. Studies in da neurons are enabled by a wealth of fly genetic tools that allow targeted neuron manipulation and labeling of the neuron's cytoskeletal or organellar components. Moreover, as da neuron dendrite arbors cover the body wall, they are highly accessible for live imaging analysis of arbor patterning. Here, we outline the structure and function of different da neuron types and give examples of how they are used to elucidate central mechanisms of dendritic arbor formation.

Study of Dendrite Differentiation Using  Dendritic Arborization Neurons.

Neurons have a complex dendritic architecture that governs information flow through a circuit. Manual quantification of dendritic arbor morphometrics is time-consuming and can be inaccurate. Automated quantification systems such as DeTerm help to overcome these limitations. DeTerm is a software tool that automatically recognizes dendrite branch terminals with high precision. It uses an artificial neural network to label the terminals, count them, and provide each terminal's positional data. DeTerm can recognize the dendritic terminals of  dendritic arborization (da) neurons, and it can also examine other types of neurons, including mouse Purkinje cells. It is freely available and works on Mac, Windows, and Linux. Here, we describe the use of DeTerm.

Use of DeTerm for Automated  Dendrite Arbor Terminal Counts.

In this study, we introduce "instance loss functions", a new family of loss functions designed to enhance the training of neural networks in the instance-level segmentation and detection of objects in biomedical image data, particularly those of varied numbers and sizes. Intended to be utilized conjointly with traditional loss functions, these proposed functions, prioritize object instances over pixel-by-pixel comparisons. The specific functions, the instance segmentation loss (L), the instance center loss (L), the false instance rate loss (L), and the instance proximity loss (L), serve distinct purposes. Specifically, L improves instance-wise segmentation quality, L enhances segmentation quality of small instances, L minimizes the rate of false and missed detections across varied instance sizes, and L improves detection quality by pulling predicted instances towards the ground truth instances. Through the task of segmenting white matter hyperintensities (WMH) in brain MRI, we benchmarked our proposed instance loss functions, both individually and in combination via an ensemble inference models approach, against traditional pixel-level loss functions. Data were sourced from the Alzheimer's Disease Neuroimaging Initiative (ADNI) and the WMH Segmentation Challenge datasets, which exhibit significant variation in WMH instance sizes. Empirical evaluations demonstrate that combining two instance-level loss functions through ensemble inference models outperforms models using other loss function on both the ADNI and WMH Segmentation Challenge datasets for the segmentation and detection of WMH instances. Further, applying these functions to the segmentation of nuclei in histopathology images demonstrated their effectiveness and generalizability beyond WMH, improving performance even in contexts with less severe instance imbalance.

A new family of instance-level loss functions for improving instance-level segmentation and detection of white matter hyperintensities in routine clinical brain MRI.

How actin filaments (F-actins) are dynamically reorganized in motile cells at the level of individual filaments is an open question. To find the answer, a high-speed atomic force microscopy (HS-AFM) system has been developed to live-imagine intracellular dynamics of the individual F-actins. However, noise and low resolution made it difficult to fully recognize individual F-actins in the HS-AFM images. To tackle this problem, we developed a new machine learning method that quantitatively recognizes individual F-actins. The method estimates F-actin orientation from the image while improving the resolution. We found that F-actins were oriented at ±35° toward the membrane in the lamellipodia, which is consistent with Arp2/3 complex-induced branching. Furthermore, in the cell cortex our results showed non-random orientation at four specific angles, suggesting a new mechanism for F-actin organization demonstrating the potential of our newly developed method to fundamentally improve our understanding of the structural dynamics of F-actin networks.

Machine learning-guided reconstruction of cytoskeleton network from live-cell AFM images.

Predicting the evolution of white matter hyperintensities (WMH), a common feature in brain magnetic resonance imaging (MRI) scans of older adults (i.e., whether WMH will grow, remain stable, or shrink with time) is important for personalised therapeutic interventions. However, this task is difficult mainly due to the myriad of vascular risk factors and comorbidities that influence it, and the low specificity and sensitivity of the image intensities and textures alone for predicting WMH evolution. Given the predominantly vascular nature of WMH, in this study, we evaluate the impact of incorporating stroke lesion information to a probabilistic deep learning model to predict the evolution of WMH 1-year after the baseline image acquisition, taken soon after a mild stroke event, using T2-FLAIR brain MRI. The Probabilistic U-Net was chosen for this study due to its capability of simulating and quantifying the uncertainties involved in the prediction of WMH evolution. We propose to use an additional loss called volume loss to train our model, and incorporate stroke lesions information, an influential factor in WMH evolution. Our experiments showed that jointly segmenting the disease evolution map (DEM) of WMH and stroke lesions, improved the accuracy of the DEM representing WMH evolution. The combination of introducing the volume loss and joint segmentation of DEM of WMH and stroke lesions outperformed other model configurations with mean volumetric absolute error of 0.0092 ml (down from 1.7739 ml) and 0.47% improvement on average Dice similarity coefficient in shrinking, growing and stable WMH.

Article	Year
The NanoZoomer artificial intelligence connectomics pipeline for tracer injection studies of the marmoset brain. Brain structure & function\| PubMed ID: 32367264	2020
Optimization and validation of diffusion MRI-based fiber tracking with neural tracer data as a reference. Scientific reports\| PubMed ID: 33339834	2020
A Spiking Neural Network Builder for Systematic Data-to-Model Workflow. Frontiers in neuroinformatics\| PubMed ID: 35909884	2022
A dataset of rodent cerebrovasculature from in vivo multiphoton fluorescence microscopy imaging. Scientific data\| PubMed ID: 36932084	2023
Tomographic tract tracing and data driven approaches to unravel complex 3D fiber anatomy of DBS relevant prefrontal projections to the diencephalic-mesencephalic junction in the marmoset. Brain stimulation\| PubMed ID: 37028755	2023
Multi-modal brain magnetic resonance imaging database covering marmosets with a wide age range. Scientific data\| PubMed ID: 37105968	2023
Local and long-distance organization of prefrontal cortex circuits in the marmoset brain. Neuron\| PubMed ID: 37196659	2023
The Brain/MINDS Marmoset Connectivity Resource: An open-access platform for cellular-level tracing and tractography in the primate brain. PLoS biology\| PubMed ID: 37384809	2023
Exploring the performance of implicit neural representations for brain image registration. Scientific reports\| PubMed ID: 37833464	2023
Study of Dendrite Differentiation Using Dendritic Arborization Neurons. Cold Spring Harbor protocols\| PubMed ID: 38148165	2024
Use of DeTerm for Automated Dendrite Arbor Terminal Counts. Cold Spring Harbor protocols\| PubMed ID: 38148169	2024
A new family of instance-level loss functions for improving instance-level segmentation and detection of white matter hyperintensities in routine clinical brain MRI. Computers in biology and medicine\| PubMed ID: 38599072	2024
Machine learning-guided reconstruction of cytoskeleton network from live-cell AFM images. iScience\| PubMed ID: 39391724	2024
Prediction of white matter hyperintensities evolution one-year post-stroke from a single-point brain MRI and stroke lesions information. Scientific reports\| PubMed ID: 39774013	2025

Henrik Skibbe

Biography

JoVE Journal Publications

Other Publications