JoVE Visualize What is visualize?
Related JoVE Video
Pubmed Article
Evolution with reinforcement learning in negotiation.
PUBLISHED: 01-01-2014
Adaptive behavior depends less on the details of the negotiation process and makes more robust predictions in the long term as compared to in the short term. However, the extant literature on population dynamics for behavior adjustment has only examined the current situation. To offset this limitation, we propose a synergy of evolutionary algorithm and reinforcement learning to investigate long-term collective performance and strategy evolution. The model adopts reinforcement learning with a tradeoff between historical and current information to make decisions when the strategies of agents evolve through repeated interactions. The results demonstrate that the strategies in populations converge to stable states, and the agents gradually form steady negotiation habits. Agents that adopt reinforcement learning perform better in payoff, fairness, and stableness than their counterparts using classic evolutionary algorithm.
Authors: C. R. Gallistel, Fuat Balci, David Freestone, Aaron Kheifets, Adam King.
Published: 02-26-2014
We describe a high-throughput, high-volume, fully automated, live-in 24/7 behavioral testing system for assessing the effects of genetic and pharmacological manipulations on basic mechanisms of cognition and learning in mice. A standard polypropylene mouse housing tub is connected through an acrylic tube to a standard commercial mouse test box. The test box has 3 hoppers, 2 of which are connected to pellet feeders. All are internally illuminable with an LED and monitored for head entries by infrared (IR) beams. Mice live in the environment, which eliminates handling during screening. They obtain their food during two or more daily feeding periods by performing in operant (instrumental) and Pavlovian (classical) protocols, for which we have written protocol-control software and quasi-real-time data analysis and graphing software. The data analysis and graphing routines are written in a MATLAB-based language created to simplify greatly the analysis of large time-stamped behavioral and physiological event records and to preserve a full data trail from raw data through all intermediate analyses to the published graphs and statistics within a single data structure. The data-analysis code harvests the data several times a day and subjects it to statistical and graphical analyses, which are automatically stored in the "cloud" and on in-lab computers. Thus, the progress of individual mice is visualized and quantified daily. The data-analysis code talks to the protocol-control code, permitting the automated advance from protocol to protocol of individual subjects. The behavioral protocols implemented are matching, autoshaping, timed hopper-switching, risk assessment in timed hopper-switching, impulsivity measurement, and the circadian anticipation of food availability. Open-source protocol-control and data-analysis code makes the addition of new protocols simple. Eight test environments fit in a 48 in x 24 in x 78 in cabinet; two such cabinets (16 environments) may be controlled by one computer.
22 Related JoVE Articles!
Play Button
Extinction Training During the Reconsolidation Window Prevents Recovery of Fear
Authors: Daniela Schiller, Candace M. Raio, Elizabeth A. Phelps.
Institutions: Mt. Sinai School of Medicine, New York University , New York University .
Fear is maladaptive when it persists long after circumstances have become safe. It is therefore crucial to develop an approach that persistently prevents the return of fear. Pavlovian fear-conditioning paradigms are commonly employed to create a controlled, novel fear association in the laboratory. After pairing an innocuous stimulus (conditioned stimulus, CS) with an aversive outcome (unconditioned stimulus, US) we can elicit a fear response (conditioned response, or CR) by presenting just the stimulus alone1,2 . Once fear is acquired, it can be diminished using extinction training, whereby the conditioned stimulus is repeatedly presented without the aversive outcome until fear is no longer expressed3. This inhibitory learning creates a new, safe representation for the CS, which competes for expression with the original fear memory4. Although extinction is effective at inhibiting fear, it is not permanent. Fear can spontaneously recover with the passage of time. Exposure to stress or returning to the context of initial learning can also cause fear to resurface3,4. Our protocol addresses the transient nature of extinction by targeting the reconsolidation window to modify emotional memory in a more permanent manner. Ample evidence suggests that reactivating a consolidated memory returns it to a labile state, during which the memory is again susceptible to interference5-9. This window of opportunity appears to open shortly after reactivation and close approximately 6hrs later5,11,16, although this may vary depending on the strength and age of the memory15. By allowing new information to incorporate into the original memory trace, this memory may be updated as it reconsolidates10,11. Studies involving non-human animals have successfully blocked the expression of fear memory by introducing pharmacological manipulations within the reconsolidation window, however, most agents used are either toxic to humans or show equivocal effects when used in human studies12-14. Our protocol addresses these challenges by offering an effective, yet non-invasive, behavioral manipulation that is safe for humans. By prompting fear memory retrieval prior to extinction, we essentially trigger the reconsolidation process, allowing new safety information (i.e., extinction) to be incorporated while the fear memory is still susceptible to interference. A recent study employing this behavioral manipulation in rats has successfully blocked fear memory using these temporal parameters11. Additional studies in humans have demonstrated that introducing new information after the retrieval of previously consolidated motor16, episodic17, or declarative18 memories leads to interference with the original memory trace14. We outline below a novel protocol used to block fear recovery in humans.
Neuroscience, Issue 66, Medicine, Psychology, Physiology, Fear conditioning, extinction, reconsolidation, emotional memory, spontaneous recovery, skin conductance response
Play Button
Recording Single Neurons' Action Potentials from Freely Moving Pigeons Across Three Stages of Learning
Authors: Sarah Starosta, Maik C. Stüttgen, Onur Güntürkün.
Institutions: Ruhr-University Bochum.
While the subject of learning has attracted immense interest from both behavioral and neural scientists, only relatively few investigators have observed single-neuron activity while animals are acquiring an operantly conditioned response, or when that response is extinguished. But even in these cases, observation periods usually encompass only a single stage of learning, i.e. acquisition or extinction, but not both (exceptions include protocols employing reversal learning; see Bingman et al.1 for an example). However, acquisition and extinction entail different learning mechanisms and are therefore expected to be accompanied by different types and/or loci of neural plasticity. Accordingly, we developed a behavioral paradigm which institutes three stages of learning in a single behavioral session and which is well suited for the simultaneous recording of single neurons' action potentials. Animals are trained on a single-interval forced choice task which requires mapping each of two possible choice responses to the presentation of different novel visual stimuli (acquisition). After having reached a predefined performance criterion, one of the two choice responses is no longer reinforced (extinction). Following a certain decrement in performance level, correct responses are reinforced again (reacquisition). By using a new set of stimuli in every session, animals can undergo the acquisition-extinction-reacquisition process repeatedly. Because all three stages of learning occur in a single behavioral session, the paradigm is ideal for the simultaneous observation of the spiking output of multiple single neurons. We use pigeons as model systems, but the task can easily be adapted to any other species capable of conditioned discrimination learning.
Neuroscience, Issue 88, pigeon, single unit recording, learning, memory, extinction, spike sorting, operant conditioning, reward, electrophysiology, animal cognition, model species
Play Button
Novel Apparatus and Method for Drug Reinforcement
Authors: Allison A. Feduccia, Christine L. Duvauchelle.
Institutions: University of Texas at Austin.
Animal models of reinforcement have proven to be useful for understanding the neurobiological mechanisms underlying drug addiction. Operant drug self-administration and conditioned place preference (CPP) procedures are expansively used in animal research to model various components of drug reinforcement, consumption, and addiction in humans. For this study, we used a novel approach to studying drug reinforcement in rats by combining traditional CPP and self-administration methodologies. We assembled an apparatus using two Med Associate operant chambers, sensory stimuli, and a Plexiglas-constructed neutral zone. These modifications allowed our experiments to encompass motivational aspects of drug intake through self-administration and drug-free assessment of drug/cue conditioning strength with the CPP test. In our experiments, rats self-administered cocaine (0.75 mg/kg/inj, i.v.) during either four (e.g., the "short-term") or eight (e.g., the "long-term") alternating-day sessions in an operant environment containing distinctive sensory cues (e.g., olfactory and visual). On the alternate days, in the other (differently-cued) operant environment, saline was available for self-infusion (0.1 ml, i.v.). Twenty-four hours after the last self-administration/cue-pairing session, a CPP test was conducted. Consistent with typical CPP findings, there was a significant preference for the chamber associated with cocaine self-administration. In addition, in animals undergoing the long-term experiment, a significant positive correlation between CPP magnitude and the number of cocaine-reinforced lever responses. In conclusion, this apparatus and approach is time and cost effective, can be used to examine a wide array of topics pertaining to drug abuse, and provides more flexibility in experimental design than CPP or self-administration methods alone.
Neuroscience, Issue 42, conditioned place preference (CPP), self-administration, rat, behavioral neuroscience, drug reinforcement, cocaine, animal models
Play Button
Cortical Source Analysis of High-Density EEG Recordings in Children
Authors: Joe Bathelt, Helen O'Reilly, Michelle de Haan.
Institutions: UCL Institute of Child Health, University College London.
EEG is traditionally described as a neuroimaging technique with high temporal and low spatial resolution. Recent advances in biophysical modelling and signal processing make it possible to exploit information from other imaging modalities like structural MRI that provide high spatial resolution to overcome this constraint1. This is especially useful for investigations that require high resolution in the temporal as well as spatial domain. In addition, due to the easy application and low cost of EEG recordings, EEG is often the method of choice when working with populations, such as young children, that do not tolerate functional MRI scans well. However, in order to investigate which neural substrates are involved, anatomical information from structural MRI is still needed. Most EEG analysis packages work with standard head models that are based on adult anatomy. The accuracy of these models when used for children is limited2, because the composition and spatial configuration of head tissues changes dramatically over development3.  In the present paper, we provide an overview of our recent work in utilizing head models based on individual structural MRI scans or age specific head models to reconstruct the cortical generators of high density EEG. This article describes how EEG recordings are acquired, processed, and analyzed with pediatric populations at the London Baby Lab, including laboratory setup, task design, EEG preprocessing, MRI processing, and EEG channel level and source analysis. 
Behavior, Issue 88, EEG, electroencephalogram, development, source analysis, pediatric, minimum-norm estimation, cognitive neuroscience, event-related potentials 
Play Button
Barnes Maze Testing Strategies with Small and Large Rodent Models
Authors: Cheryl S. Rosenfeld, Sherry A. Ferguson.
Institutions: University of Missouri, Food and Drug Administration.
Spatial learning and memory of laboratory rodents is often assessed via navigational ability in mazes, most popular of which are the water and dry-land (Barnes) mazes. Improved performance over sessions or trials is thought to reflect learning and memory of the escape cage/platform location. Considered less stressful than water mazes, the Barnes maze is a relatively simple design of a circular platform top with several holes equally spaced around the perimeter edge. All but one of the holes are false-bottomed or blind-ending, while one leads to an escape cage. Mildly aversive stimuli (e.g. bright overhead lights) provide motivation to locate the escape cage. Latency to locate the escape cage can be measured during the session; however, additional endpoints typically require video recording. From those video recordings, use of automated tracking software can generate a variety of endpoints that are similar to those produced in water mazes (e.g. distance traveled, velocity/speed, time spent in the correct quadrant, time spent moving/resting, and confirmation of latency). Type of search strategy (i.e. random, serial, or direct) can be categorized as well. Barnes maze construction and testing methodologies can differ for small rodents, such as mice, and large rodents, such as rats. For example, while extra-maze cues are effective for rats, smaller wild rodents may require intra-maze cues with a visual barrier around the maze. Appropriate stimuli must be identified which motivate the rodent to locate the escape cage. Both Barnes and water mazes can be time consuming as 4-7 test trials are typically required to detect improved learning and memory performance (e.g. shorter latencies or path lengths to locate the escape platform or cage) and/or differences between experimental groups. Even so, the Barnes maze is a widely employed behavioral assessment measuring spatial navigational abilities and their potential disruption by genetic, neurobehavioral manipulations, or drug/ toxicant exposure.
Behavior, Issue 84, spatial navigation, rats, Peromyscus, mice, intra- and extra-maze cues, learning, memory, latency, search strategy, escape motivation
Play Button
Characterization of Complex Systems Using the Design of Experiments Approach: Transient Protein Expression in Tobacco as a Case Study
Authors: Johannes Felix Buyel, Rainer Fischer.
Institutions: RWTH Aachen University, Fraunhofer Gesellschaft.
Plants provide multiple benefits for the production of biopharmaceuticals including low costs, scalability, and safety. Transient expression offers the additional advantage of short development and production times, but expression levels can vary significantly between batches thus giving rise to regulatory concerns in the context of good manufacturing practice. We used a design of experiments (DoE) approach to determine the impact of major factors such as regulatory elements in the expression construct, plant growth and development parameters, and the incubation conditions during expression, on the variability of expression between batches. We tested plants expressing a model anti-HIV monoclonal antibody (2G12) and a fluorescent marker protein (DsRed). We discuss the rationale for selecting certain properties of the model and identify its potential limitations. The general approach can easily be transferred to other problems because the principles of the model are broadly applicable: knowledge-based parameter selection, complexity reduction by splitting the initial problem into smaller modules, software-guided setup of optimal experiment combinations and step-wise design augmentation. Therefore, the methodology is not only useful for characterizing protein expression in plants but also for the investigation of other complex systems lacking a mechanistic description. The predictive equations describing the interconnectivity between parameters can be used to establish mechanistic models for other complex systems.
Bioengineering, Issue 83, design of experiments (DoE), transient protein expression, plant-derived biopharmaceuticals, promoter, 5'UTR, fluorescent reporter protein, model building, incubation conditions, monoclonal antibody
Play Button
Protein WISDOM: A Workbench for In silico De novo Design of BioMolecules
Authors: James Smadbeck, Meghan B. Peterson, George A. Khoury, Martin S. Taylor, Christodoulos A. Floudas.
Institutions: Princeton University.
The aim of de novo protein design is to find the amino acid sequences that will fold into a desired 3-dimensional structure with improvements in specific properties, such as binding affinity, agonist or antagonist behavior, or stability, relative to the native sequence. Protein design lies at the center of current advances drug design and discovery. Not only does protein design provide predictions for potentially useful drug targets, but it also enhances our understanding of the protein folding process and protein-protein interactions. Experimental methods such as directed evolution have shown success in protein design. However, such methods are restricted by the limited sequence space that can be searched tractably. In contrast, computational design strategies allow for the screening of a much larger set of sequences covering a wide variety of properties and functionality. We have developed a range of computational de novo protein design methods capable of tackling several important areas of protein design. These include the design of monomeric proteins for increased stability and complexes for increased binding affinity. To disseminate these methods for broader use we present Protein WISDOM (, a tool that provides automated methods for a variety of protein design problems. Structural templates are submitted to initialize the design process. The first stage of design is an optimization sequence selection stage that aims at improving stability through minimization of potential energy in the sequence space. Selected sequences are then run through a fold specificity stage and a binding affinity stage. A rank-ordered list of the sequences for each step of the process, along with relevant designed structures, provides the user with a comprehensive quantitative assessment of the design. Here we provide the details of each design method, as well as several notable experimental successes attained through the use of the methods.
Genetics, Issue 77, Molecular Biology, Bioengineering, Biochemistry, Biomedical Engineering, Chemical Engineering, Computational Biology, Genomics, Proteomics, Protein, Protein Binding, Computational Biology, Drug Design, optimization (mathematics), Amino Acids, Peptides, and Proteins, De novo protein and peptide design, Drug design, In silico sequence selection, Optimization, Fold specificity, Binding affinity, sequencing
Play Button
Membrane Potentials, Synaptic Responses, Neuronal Circuitry, Neuromodulation and Muscle Histology Using the Crayfish: Student Laboratory Exercises
Authors: Brittany Baierlein, Alison L. Thurow, Harold L. Atwood, Robin L. Cooper.
Institutions: University of Kentucky, University of Toronto.
The purpose of this report is to help develop an understanding of the effects caused by ion gradients across a biological membrane. Two aspects that influence a cell's membrane potential and which we address in these experiments are: (1) Ion concentration of K+ on the outside of the membrane, and (2) the permeability of the membrane to specific ions. The crayfish abdominal extensor muscles are in groupings with some being tonic (slow) and others phasic (fast) in their biochemical and physiological phenotypes, as well as in their structure; the motor neurons that innervate these muscles are correspondingly different in functional characteristics. We use these muscles as well as the superficial, tonic abdominal flexor muscle to demonstrate properties in synaptic transmission. In addition, we introduce a sensory-CNS-motor neuron-muscle circuit to demonstrate the effect of cuticular sensory stimulation as well as the influence of neuromodulators on certain aspects of the circuit. With the techniques obtained in this exercise, one can begin to answer many questions remaining in other experimental preparations as well as in physiological applications related to medicine and health. We have demonstrated the usefulness of model invertebrate preparations to address fundamental questions pertinent to all animals.
Neuroscience, Issue 47, Invertebrate, Crayfish, neurophysiology, muscle, anatomy, electrophysiology
Play Button
Perceptual and Category Processing of the Uncanny Valley Hypothesis' Dimension of Human Likeness: Some Methodological Issues
Authors: Marcus Cheetham, Lutz Jancke.
Institutions: University of Zurich.
Mori's Uncanny Valley Hypothesis1,2 proposes that the perception of humanlike characters such as robots and, by extension, avatars (computer-generated characters) can evoke negative or positive affect (valence) depending on the object's degree of visual and behavioral realism along a dimension of human likeness (DHL) (Figure 1). But studies of affective valence of subjective responses to variously realistic non-human characters have produced inconsistent findings 3, 4, 5, 6. One of a number of reasons for this is that human likeness is not perceived as the hypothesis assumes. While the DHL can be defined following Mori's description as a smooth linear change in the degree of physical humanlike similarity, subjective perception of objects along the DHL can be understood in terms of the psychological effects of categorical perception (CP) 7. Further behavioral and neuroimaging investigations of category processing and CP along the DHL and of the potential influence of the dimension's underlying category structure on affective experience are needed. This protocol therefore focuses on the DHL and allows examination of CP. Based on the protocol presented in the video as an example, issues surrounding the methodology in the protocol and the use in "uncanny" research of stimuli drawn from morph continua to represent the DHL are discussed in the article that accompanies the video. The use of neuroimaging and morph stimuli to represent the DHL in order to disentangle brain regions neurally responsive to physical human-like similarity from those responsive to category change and category processing is briefly illustrated.
Behavior, Issue 76, Neuroscience, Neurobiology, Molecular Biology, Psychology, Neuropsychology, uncanny valley, functional magnetic resonance imaging, fMRI, categorical perception, virtual reality, avatar, human likeness, Mori, uncanny valley hypothesis, perception, magnetic resonance imaging, MRI, imaging, clinical techniques
Play Button
A Proboscis Extension Response Protocol for Investigating Behavioral Plasticity in Insects: Application to Basic, Biomedical, and Agricultural Research
Authors: Brian H. Smith, Christina M. Burden.
Institutions: Arizona State University.
Insects modify their responses to stimuli through experience of associating those stimuli with events important for survival (e.g., food, mates, threats). There are several behavioral mechanisms through which an insect learns salient associations and relates them to these events. It is important to understand this behavioral plasticity for programs aimed toward assisting insects that are beneficial for agriculture. This understanding can also be used for discovering solutions to biomedical and agricultural problems created by insects that act as disease vectors and pests. The Proboscis Extension Response (PER) conditioning protocol was developed for honey bees (Apis mellifera) over 50 years ago to study how they perceive and learn about floral odors, which signal the nectar and pollen resources a colony needs for survival. The PER procedure provides a robust and easy-to-employ framework for studying several different ecologically relevant mechanisms of behavioral plasticity. It is easily adaptable for use with several other insect species and other behavioral reflexes. These protocols can be readily employed in conjunction with various means for monitoring neural activity in the CNS via electrophysiology or bioimaging, or for manipulating targeted neuromodulatory pathways. It is a robust assay for rapidly detecting sub-lethal effects on behavior caused by environmental stressors, toxins or pesticides. We show how the PER protocol is straightforward to implement using two procedures. One is suitable as a laboratory exercise for students or for quick assays of the effect of an experimental treatment. The other provides more thorough control of variables, which is important for studies of behavioral conditioning. We show how several measures for the behavioral response ranging from binary yes/no to more continuous variable like latency and duration of proboscis extension can be used to test hypotheses. And, we discuss some pitfalls that researchers commonly encounter when they use the procedure for the first time.
Neuroscience, Issue 91, PER, conditioning, honey bee, olfaction, olfactory processing, learning, memory, toxin assay
Play Button
Appetitive Associative Olfactory Learning in Drosophila Larvae
Authors: Anthi A. Apostolopoulou, Annekathrin Widmann, Astrid Rohwedder, Johanna E. Pfitzenmaier, Andreas S. Thum.
Institutions: University of Konstanz, University of Fribourg.
In the following we describe the methodological details of appetitive associative olfactory learning in Drosophila larvae. The setup, in combination with genetic interference, provides a handle to analyze the neuronal and molecular fundamentals of specifically associative learning in a simple larval brain. Organisms can use past experience to adjust present behavior. Such acquisition of behavioral potential can be defined as learning, and the physical bases of these potentials as memory traces1-4. Neuroscientists try to understand how these processes are organized in terms of molecular and neuronal changes in the brain by using a variety of methods in model organisms ranging from insects to vertebrates5,6. For such endeavors it is helpful to use model systems that are simple and experimentally accessible. The Drosophila larva has turned out to satisfy these demands based on the availability of robust behavioral assays, the existence of a variety of transgenic techniques and the elementary organization of the nervous system comprising only about 10,000 neurons (albeit with some concessions: cognitive limitations, few behavioral options, and richness of experience questionable)7-10. Drosophila larvae can form associations between odors and appetitive gustatory reinforcement like sugar11-14. In a standard assay, established in the lab of B. Gerber, animals receive a two-odor reciprocal training: A first group of larvae is exposed to an odor A together with a gustatory reinforcer (sugar reward) and is subsequently exposed to an odor B without reinforcement 9. Meanwhile a second group of larvae receives reciprocal training while experiencing odor A without reinforcement and subsequently being exposed to odor B with reinforcement (sugar reward). In the following both groups are tested for their preference between the two odors. Relatively higher preferences for the rewarded odor reflect associative learning - presented as a performance index (PI). The conclusion regarding the associative nature of the performance index is compelling, because apart from the contingency between odors and tastants, other parameters, such as odor and reward exposure, passage of time and handling do not differ between the two groups9.
Neuroscience, Issue 72, Developmental Biology, Neurobiology, Biochemistry, Molecular Biology, Physiology, Behavior, Drosophila, fruit fly, larvae, instar, olfaction, olfactory system, odor, 1-octanol, OCT, learning, reward, sugar, feeding, animal model
Play Button
A Novel Procedure for Evaluating the Reinforcing Properties of Tastants in Laboratory Rats: Operant Intraoral Self-administration
Authors: AnneMarie Levy, Cheryl L. Limebeer, Justin Ferdinand, Ucal Shillingford, Linda A. Parker, Francesco Leri.
Institutions: University of Guelph.
This paper describes a novel method for studying the bio-behavioral basis of addiction to food. This method combines the surgical component of taste reactivity with the behavioral aspects of operant self-administration of drugs. Under very brief general anaesthesia, rats are implanted with an intraoral (IO) cannula that allows delivery of test solutions directly in the oral cavity. Animals are then tested in operant self-administration chambers whereby they can press a lever to receive IO infusions of test solutions. IO self-administration has several advantages over experimental procedures that involve drinking a solution from a spout or operant responding for solid pellets or solutions delivered in a receptacle. Here, we show that IO self-administration can be employed to study self-administration of high fructose corn syrup (HFCS). Rats were first tested for self-administration on a progressive ratio (PR) schedule, which assesses the maximum amount of operant behavior that will be emitted for different concentrations of HFCS (i.e. 8%, 25%, and 50%). Following this test, rats self-administered these concentrations on a continuous schedule of reinforcement (i.e. one infusion for each lever press) for 10 consecutive days (1 session/day; each lasting 3 hr), and then they were retested on the PR schedule. On the continuous reinforcement schedule, rats took fewer infusions of higher concentrations, although the lowest concentration of HFCS (8%) maintained more variable self-administration. Furthermore, the PR tests revealed that 8% had lower reinforcing value than 25% and 50%. These results indicate that IO self-administration can be employed to study acquisition and maintenance of responding for sweet solutions. The sensitivity of the operant response to differences in concentration and schedule of reinforcement makes IO self-administration an ideal procedure to investigate the neurobiology of voluntary intake of sweets.
Behavior, Issue 84, Administration, Oral, Conditioning, Operant, Reinforcement (Psychology), Reinforcement Schedule, Taste, Neurosciences, Intraoral infusions, operant chambers, self-administration, high fructose corn syrup, progressive ratio, breakpoint, addiction
Play Button
Creating Objects and Object Categories for Studying Perception and Perceptual Learning
Authors: Karin Hauffen, Eugene Bart, Mark Brady, Daniel Kersten, Jay Hegdé.
Institutions: Georgia Health Sciences University, Georgia Health Sciences University, Georgia Health Sciences University, Palo Alto Research Center, Palo Alto Research Center, University of Minnesota .
In order to quantitatively study object perception, be it perception by biological systems or by machines, one needs to create objects and object categories with precisely definable, preferably naturalistic, properties1. Furthermore, for studies on perceptual learning, it is useful to create novel objects and object categories (or object classes) with such properties2. Many innovative and useful methods currently exist for creating novel objects and object categories3-6 (also see refs. 7,8). However, generally speaking, the existing methods have three broad types of shortcomings. First, shape variations are generally imposed by the experimenter5,9,10, and may therefore be different from the variability in natural categories, and optimized for a particular recognition algorithm. It would be desirable to have the variations arise independently of the externally imposed constraints. Second, the existing methods have difficulty capturing the shape complexity of natural objects11-13. If the goal is to study natural object perception, it is desirable for objects and object categories to be naturalistic, so as to avoid possible confounds and special cases. Third, it is generally hard to quantitatively measure the available information in the stimuli created by conventional methods. It would be desirable to create objects and object categories where the available information can be precisely measured and, where necessary, systematically manipulated (or 'tuned'). This allows one to formulate the underlying object recognition tasks in quantitative terms. Here we describe a set of algorithms, or methods, that meet all three of the above criteria. Virtual morphogenesis (VM) creates novel, naturalistic virtual 3-D objects called 'digital embryos' by simulating the biological process of embryogenesis14. Virtual phylogenesis (VP) creates novel, naturalistic object categories by simulating the evolutionary process of natural selection9,12,13. Objects and object categories created by these simulations can be further manipulated by various morphing methods to generate systematic variations of shape characteristics15,16. The VP and morphing methods can also be applied, in principle, to novel virtual objects other than digital embryos, or to virtual versions of real-world objects9,13. Virtual objects created in this fashion can be rendered as visual images using a conventional graphical toolkit, with desired manipulations of surface texture, illumination, size, viewpoint and background. The virtual objects can also be 'printed' as haptic objects using a conventional 3-D prototyper. We also describe some implementations of these computational algorithms to help illustrate the potential utility of the algorithms. It is important to distinguish the algorithms from their implementations. The implementations are demonstrations offered solely as a 'proof of principle' of the underlying algorithms. It is important to note that, in general, an implementation of a computational algorithm often has limitations that the algorithm itself does not have. Together, these methods represent a set of powerful and flexible tools for studying object recognition and perceptual learning by biological and computational systems alike. With appropriate extensions, these methods may also prove useful in the study of morphogenesis and phylogenesis.
Neuroscience, Issue 69, machine learning, brain, classification, category learning, cross-modal perception, 3-D prototyping, inference
Play Button
Operant Sensation Seeking in the Mouse
Authors: Christopher M. Olsen, Danny G. Winder.
Institutions: Vanderbilt University Medical Center.
Operant methods are powerful behavioral tools for the study of motivated behavior. These 'self-administration' methods have been used extensively in drug addiction research due to their high construct validity. Operant studies provide researchers a tool for preclinical investigation of several aspects of the addiction process. For example, mechanisms of acute reinforcement (both drug and non-drug) can be tested using pharmacological or genetic tools to determine the ability of a molecular target to influence self-administration behavior1-6. Additionally, drug or food seeking behaviors can be studied in the absence of the primary reinforcer, and the ability of pharmacological compounds to disrupt this process is a preclinical model for discovery of molecular targets and compounds that may be useful for the treatment of addiction3,7-9. One problem with performing intravenous drug self-administration studies in the mouse is the technical difficulty of maintaining catheter patency. Attrition rates in these experiments are high and can reach 40% or higher10-15. Another general problem with drug self-administration is discerning which pharmacologically-induced effects of the reinforcer produce specific behaviors. For example, measurement of the reinforcing and neurological effects of psychostimulants can be confounded by their psychomotor effects. Operant methods using food reinforcement can avoid these pitfalls, although their utility in studying drug addiction is limited by the fact that some manipulations that alter drug self-administration have a minimal impact on food self-administration. For example, mesolimbic dopamine lesion or knockout of the D1 dopamine receptor reduce cocaine self-administration without having a significant impact on food self-administration 12,16. Sensory stimuli have been described for their ability to support operant responding as primary reinforcers (i.e. not conditioned reinforcers)17-22. Auditory and visual stimuli are self-administered by several species18,21,23, although surprisingly little is known about the neural mechanisms underlying this reinforcement. The operant sensation seeking (OSS) model is a robust model for obtaining sensory self-administration in the mouse, allowing the study of neural mechanisms important in sensory reinforcement24. An additional advantage of OSS is the ability to screen mutant mice for differences in operant behavior that may be relevant to addiction. We have reported that dopamine D1 receptor knockout mice, previously shown to be deficient in psychostimulant self-administration, also fail to acquire OSS24. This is a unique finding in that these mice are capable of learning an operant task when food is used as a reinforcer. While operant studies using food reinforcement can be useful in the study of general motivated behavior and the mechanisms underlying food reinforcement, as mentioned above, these studies are limited in their application to studying molecular mechanisms of drug addiction. Thus, there may be similar neural substrates mediating sensory and psychostimulant reinforcement that are distinct from food reinforcement, which would make OSS a particularly attractive model for the study of drug addiction processes. The degree of overlap between other molecular targets of OSS and drug reinforcers is unclear, but is a topic that we are currently pursuing. While some aspects of addiction such as resistance to extinction may be observed with OSS, we have found that escalation 25 is not observed in this model24. Interestingly, escalation of intake and some other aspects of addiction are observed with self-administration of sucrose26. Thus, when non-drug operant procedures are desired to study addiction-related processes, food or sensory reinforcers can be chosen to best fit the particular question being asked. In conclusion, both food self-administration and OSS in the mouse have the advantage of not requiring an intravenous catheter, which allows a higher throughput means to study the effects of pharmacological or genetic manipulation of neural targets involved in motivation. While operant testing using food as a reinforcer is particularly useful in the study of the regulation of food intake, OSS is particularly apt for studying reinforcement mechanisms of sensory stimuli and may have broad applicability to novelty seeking and addiction.
Neuroscience, Issue 45, novelty seeking, self-administration, addiction, motivation, reinforcement
Play Button
Drosophila Adult Olfactory Shock Learning
Authors: Bilal R. Malik, James J.L. Hodge.
Institutions: University of Bristol.
Drosophila have been used in classical conditioning experiments for over 40 years, thus greatly facilitating our understanding of memory, including the elucidation of the molecular mechanisms involved in cognitive diseases1-7. Learning and memory can be assayed in larvae to study the effect of neurodevelopmental genes8-10 and in flies to measure the contribution of adult plasticity genes1-7. Furthermore, the short lifespan of Drosophila facilitates the analysis of genes mediating age-related memory impairment5,11-13. The availability of many inducible promoters that subdivide the Drosophila nervous system makes it possible to determine when and where a gene of interest is required for normal memory as well as relay of different aspects of the reinforcement signal3,4,14,16. Studying memory in adult Drosophila allows for a detailed analysis of the behavior and circuitry involved and a measurement of long-term memory15-17. The length of the adult stage accommodates longer-term genetic, behavioral, dietary and pharmacological manipulations of memory, in addition to determining the effect of aging and neurodegenerative disease on memory3-6,11-13,15-21. Classical conditioning is induced by the simultaneous presentation of a neutral odor cue (conditioned stimulus, CS+) and a reinforcement stimulus, e.g., an electric shock or sucrose, (unconditioned stimulus, US), that become associated with one another by the animal1,16. A second conditioned stimulus (CS-) is subsequently presented without the US. During the testing phase, Drosophila are simultaneously presented with CS+ and CS- odors. After the Drosophila are provided time to choose between the odors, the distribution of the animals is recorded. This procedure allows associative aversive or appetitive conditioning to be reliably measured without a bias introduced by the innate preference for either of the conditioned stimuli. Various control experiments are also performed to test whether all genotypes respond normally to odor and reinforcement alone.
Neuroscience, Issue 90, Drosophila, Pavlovian learning, classical conditioning, learning, memory, olfactory, electric shock, associative memory
Play Button
The Dig Task: A Simple Scent Discrimination Reveals Deficits Following Frontal Brain Damage
Authors: Kris M. Martens, Cole Vonder Haar, Blake A. Hutsell, Michael R. Hoane.
Institutions: Southern Illinois University at Carbondale.
Cognitive impairment is the most frequent cause of disability in humans following brain damage, yet the behavioral tasks used to assess cognition in rodent models of brain injury is lacking. Borrowing from the operant literature our laboratory utilized a basic scent discrimination paradigm1-4 in order to assess deficits in frontally-injured rats. Previously we have briefly described the Dig task and demonstrated that rats with frontal brain damage show severe deficits across multiple tests within the task5. Here we present a more detailed protocol for this task. Rats are placed into a chamber and allowed to discriminate between two scented sands, one of which contains a reinforcer. The trial ends after the rat either correctly discriminates (defined as digging in the correct scented sand), incorrectly discriminates, or 30 sec elapses. Rats that correctly discriminate are allowed to recover and consume the reinforcer. Rats that discriminate incorrectly are immediately removed from the chamber. This can continue through a variety of reversals and novel scents. The primary analysis is the accuracy for each scent pairing (cumulative proportion correct for each scent). The general findings from the Dig task suggest that it is a simple experimental preparation that can assess deficits in rats with bilateral frontal cortical damage compared to rats with unilateral parietal damage. The Dig task can also be easily incorporated into an existing cognitive test battery. The use of more tasks such as this one can lead to more accurate testing of frontal function following injury, which may lead to therapeutic options for treatment. All animal use was conducted in accordance with protocols approved by the Institutional Animal Care and Use Committee.
Neuroscience, Issue 71, Medicine, Neurobiology, Anatomy, Physiology, Psychology, Behavior, cognitive assessment, dig task, scent discrimination, olfactory, brain injury, traumatic brain injury, TBI, brain damage, rats, animal model
Play Button
Interview: HIV-1 Proviral DNA Excision Using an Evolved Recombinase
Authors: Joachim Hauber.
Institutions: Heinrich-Pette-Institute for Experimental Virology and Immunology, University of Hamburg.
HIV-1 integrates into the host chromosome of infected cells and persists as a provirus flanked by long terminal repeats. Current treatment strategies primarily target virus enzymes or virus-cell fusion, suppressing the viral life cycle without eradicating the infection. Since the integrated provirus is not targeted by these approaches, new resistant strains of HIV-1 may emerge. Here, we report that the engineered recombinase Tre (see Molecular evolution of the Tre recombinase , Buchholz, F., Max Planck Institute for Cell Biology and Genetics, Dresden) efficiently excises integrated HIV-1 proviral DNA from the genome of infected cells. We produced loxLTR containing viral pseudotypes and infected HeLa cells to examine whether Tre recombinase can excise the provirus from the genome of HIV-1 infected human cells. A virus particle-releasing cell line was cloned and transfected with a plasmid expressing Tre or with a parental control vector. Recombinase activity and virus production were monitored. All assays demonstrated the efficient deletion of the provirus from infected cells without visible cytotoxic effects. These results serve as proof of principle that it is possible to evolve a recombinase to specifically target an HIV-1 LTR and that this recombinase is capable of excising the HIV-1 provirus from the genome of HIV-1-infected human cells. Before an engineered recombinase could enter the therapeutic arena, however, significant obstacles need to be overcome. Among the most critical issues, that we face, are an efficient and safe delivery to targeted cells and the absence of side effects.
Medicine, Issue 16, HIV, Cell Biology, Recombinase, provirus, HeLa Cells
Play Button
Predicting the Effectiveness of Population Replacement Strategy Using Mathematical Modeling
Authors: John Marshall, Koji Morikawa, Nicholas Manoukis, Charles Taylor.
Institutions: University of California, Los Angeles.
Charles Taylor and John Marshall explain the utility of mathematical modeling for evaluating the effectiveness of population replacement strategy. Insight is given into how computational models can provide information on the population dynamics of mosquitoes and the spread of transposable elements through A. gambiae subspecies. The ethical considerations of releasing genetically modified mosquitoes into the wild are discussed.
Cellular Biology, Issue 5, mosquito, malaria, popuulation, replacement, modeling, infectious disease
Play Button
Using Learning Outcome Measures to assess Doctoral Nursing Education
Authors: Glenn H. Raup, Jeff King, Romana J. Hughes, Natasha Faidley.
Institutions: Harris College of Nursing and Health Sciences, Texas Christian University.
Education programs at all levels must be able to demonstrate successful program outcomes. Grades alone do not represent a comprehensive measurement methodology for assessing student learning outcomes at either the course or program level. The development and application of assessment rubrics provides an unequivocal measurement methodology to ensure a quality learning experience by providing a foundation for improvement based on qualitative and quantitatively measurable, aggregate course and program outcomes. Learning outcomes are the embodiment of the total learning experience and should incorporate assessment of both qualitative and quantitative program outcomes. The assessment of qualitative measures represents a challenge for educators in any level of a learning program. Nursing provides a unique challenge and opportunity as it is the application of science through the art of caring. Quantification of desired student learning outcomes may be enhanced through the development of assessment rubrics designed to measure quantitative and qualitative aspects of the nursing education and learning process. They provide a mechanism for uniform assessment by nursing faculty of concepts and constructs that are otherwise difficult to describe and measure. A protocol is presented and applied to a doctoral nursing education program with recommendations for application and transformation of the assessment rubric to other education programs. Through application of these specially designed rubrics, all aspects of an education program can be adequately assessed to provide information for program assessment that facilitates the closure of the gap between desired and actual student learning outcomes for any desired educational competency.
Medicine, Issue 40, learning, outcomes, measurement, program, assessment, rubric
Play Button
Operant Learning of Drosophila at the Torque Meter
Authors: Bjoern Brembs.
Institutions: Free University of Berlin.
For experiments at the torque meter, flies are kept on standard fly medium at 25°C and 60% humidity with a 12hr light/12hr dark regime. A standardized breeding regime assures proper larval density and age-matched cohorts. Cold-anesthetized flies are glued with head and thorax to a triangle-shaped hook the day before the experiment. Attached to the torque meter via a clamp, the fly's intended flight maneuvers are measured as the angular momentum around its vertical body axis. The fly is placed in the center of a cylindrical panorama to accomplish stationary flight. An analog to digital converter card feeds the yaw torque signal into a computer which stores the trace for later analysis. The computer also controls a variety of stimuli which can be brought under the fly's control by closing the feedback loop between these stimuli and the yaw torque trace. Punishment is achieved by applying heat from an adjustable infrared laser.
Neuroscience, Issue 16, operant, learning, Drosophila, fruit fly, insect, invertebrate, neuroscience, neurobiology, fly, conditioning
Play Button
Spatial Multiobjective Optimization of Agricultural Conservation Practices using a SWAT Model and an Evolutionary Algorithm
Authors: Sergey Rabotyagov, Todd Campbell, Adriana Valcu, Philip Gassman, Manoj Jha, Keith Schilling, Calvin Wolter, Catherine Kling.
Institutions: University of Washington, Iowa State University, North Carolina A&T University, Iowa Geological and Water Survey.
Finding the cost-efficient (i.e., lowest-cost) ways of targeting conservation practice investments for the achievement of specific water quality goals across the landscape is of primary importance in watershed management. Traditional economics methods of finding the lowest-cost solution in the watershed context (e.g.,5,12,20) assume that off-site impacts can be accurately described as a proportion of on-site pollution generated. Such approaches are unlikely to be representative of the actual pollution process in a watershed, where the impacts of polluting sources are often determined by complex biophysical processes. The use of modern physically-based, spatially distributed hydrologic simulation models allows for a greater degree of realism in terms of process representation but requires a development of a simulation-optimization framework where the model becomes an integral part of optimization. Evolutionary algorithms appear to be a particularly useful optimization tool, able to deal with the combinatorial nature of a watershed simulation-optimization problem and allowing the use of the full water quality model. Evolutionary algorithms treat a particular spatial allocation of conservation practices in a watershed as a candidate solution and utilize sets (populations) of candidate solutions iteratively applying stochastic operators of selection, recombination, and mutation to find improvements with respect to the optimization objectives. The optimization objectives in this case are to minimize nonpoint-source pollution in the watershed, simultaneously minimizing the cost of conservation practices. A recent and expanding set of research is attempting to use similar methods and integrates water quality models with broadly defined evolutionary optimization methods3,4,9,10,13-15,17-19,22,23,25. In this application, we demonstrate a program which follows Rabotyagov et al.'s approach and integrates a modern and commonly used SWAT water quality model7 with a multiobjective evolutionary algorithm SPEA226, and user-specified set of conservation practices and their costs to search for the complete tradeoff frontiers between costs of conservation practices and user-specified water quality objectives. The frontiers quantify the tradeoffs faced by the watershed managers by presenting the full range of costs associated with various water quality improvement goals. The program allows for a selection of watershed configurations achieving specified water quality improvement goals and a production of maps of optimized placement of conservation practices.
Environmental Sciences, Issue 70, Plant Biology, Civil Engineering, Forest Sciences, Water quality, multiobjective optimization, evolutionary algorithms, cost efficiency, agriculture, development
Play Button
Population Replacement Strategies for Controlling Vector Populations and the Use of Wolbachia pipientis for Genetic Drive
Authors: Jason Rasgon.
Institutions: Johns Hopkins University.
In this video, Jason Rasgon discusses population replacement strategies to control vector-borne diseases such as malaria and dengue. "Population replacement" is the replacement of wild vector populations (that are competent to transmit pathogens) with those that are not competent to transmit pathogens. There are several theoretical strategies to accomplish this. One is to exploit the maternally-inherited symbiotic bacteria Wolbachia pipientis. Wolbachia is a widespread reproductive parasite that spreads in a selfish manner at the extent of its host's fitness. Jason Rasgon discusses, in detail, the basic biology of this bacterial symbiont and various ways to use it for control of vector-borne diseases.
Cellular Biology, Issue 5, mosquito, malaria, genetics, infectious disease, Wolbachia
Copyright © JoVE 2006-2015. All Rights Reserved.
Policies | License Agreement | ISSN 1940-087X
simple hit counter

What is Visualize?

JoVE Visualize is a tool created to match the last 5 years of PubMed publications to methods in JoVE's video library.

How does it work?

We use abstracts found on PubMed and match them to JoVE videos to create a list of 10 to 30 related methods videos.

Video X seems to be unrelated to Abstract Y...

In developing our video relationships, we compare around 5 million PubMed articles to our library of over 4,500 methods videos. In some cases the language used in the PubMed abstracts makes matching that content to a JoVE video difficult. In other cases, there happens not to be any content in our video library that is relevant to the topic of a given abstract. In these cases, our algorithms are trying their best to display videos with relevant content, which can sometimes result in matched videos with only a slight relation.