JoVE Visualize What is visualize?
Related JoVE Video
Pubmed Article
Reinforcement learning or active inference?
PLoS ONE
PUBLISHED: 02-12-2009
This paper questions the need for reinforcement learning or control theory when optimising behaviour. We show that it is fairly simple to teach an agent complicated and adaptive behaviours using a free-energy formulation of perception. In this formulation, agents adjust their internal states and sampling of the environment to minimize their free-energy. Such agents learn causal structure in the environment and sample it in an adaptive and self-supervised fashion. This results in behavioural policies that reproduce those optimised by reinforcement learning and dynamic programming. Critically, we do not need to invoke the notion of reward, value or utility. We illustrate these points by solving a benchmark problem in dynamic programming; namely the mountain-car problem, using active perception or inference under the free-energy principle. The ensuing proof-of-concept may be important because the free-energy formulation furnishes a unified account of both action and perception and may speak to a reappraisal of the role of dopamine in the brain.
Authors: Karin Hauffen, Eugene Bart, Mark Brady, Daniel Kersten, Jay Hegdé.
Published: 11-02-2012
ABSTRACT
In order to quantitatively study object perception, be it perception by biological systems or by machines, one needs to create objects and object categories with precisely definable, preferably naturalistic, properties1. Furthermore, for studies on perceptual learning, it is useful to create novel objects and object categories (or object classes) with such properties2. Many innovative and useful methods currently exist for creating novel objects and object categories3-6 (also see refs. 7,8). However, generally speaking, the existing methods have three broad types of shortcomings. First, shape variations are generally imposed by the experimenter5,9,10, and may therefore be different from the variability in natural categories, and optimized for a particular recognition algorithm. It would be desirable to have the variations arise independently of the externally imposed constraints. Second, the existing methods have difficulty capturing the shape complexity of natural objects11-13. If the goal is to study natural object perception, it is desirable for objects and object categories to be naturalistic, so as to avoid possible confounds and special cases. Third, it is generally hard to quantitatively measure the available information in the stimuli created by conventional methods. It would be desirable to create objects and object categories where the available information can be precisely measured and, where necessary, systematically manipulated (or 'tuned'). This allows one to formulate the underlying object recognition tasks in quantitative terms. Here we describe a set of algorithms, or methods, that meet all three of the above criteria. Virtual morphogenesis (VM) creates novel, naturalistic virtual 3-D objects called 'digital embryos' by simulating the biological process of embryogenesis14. Virtual phylogenesis (VP) creates novel, naturalistic object categories by simulating the evolutionary process of natural selection9,12,13. Objects and object categories created by these simulations can be further manipulated by various morphing methods to generate systematic variations of shape characteristics15,16. The VP and morphing methods can also be applied, in principle, to novel virtual objects other than digital embryos, or to virtual versions of real-world objects9,13. Virtual objects created in this fashion can be rendered as visual images using a conventional graphical toolkit, with desired manipulations of surface texture, illumination, size, viewpoint and background. The virtual objects can also be 'printed' as haptic objects using a conventional 3-D prototyper. We also describe some implementations of these computational algorithms to help illustrate the potential utility of the algorithms. It is important to distinguish the algorithms from their implementations. The implementations are demonstrations offered solely as a 'proof of principle' of the underlying algorithms. It is important to note that, in general, an implementation of a computational algorithm often has limitations that the algorithm itself does not have. Together, these methods represent a set of powerful and flexible tools for studying object recognition and perceptual learning by biological and computational systems alike. With appropriate extensions, these methods may also prove useful in the study of morphogenesis and phylogenesis.
22 Related JoVE Articles!
Play Button
The Dig Task: A Simple Scent Discrimination Reveals Deficits Following Frontal Brain Damage
Authors: Kris M. Martens, Cole Vonder Haar, Blake A. Hutsell, Michael R. Hoane.
Institutions: Southern Illinois University at Carbondale.
Cognitive impairment is the most frequent cause of disability in humans following brain damage, yet the behavioral tasks used to assess cognition in rodent models of brain injury is lacking. Borrowing from the operant literature our laboratory utilized a basic scent discrimination paradigm1-4in order to assess deficits in frontally-injured rats. Previously we have briefly described the Dig task and demonstrated that rats with frontal brain damage show severe deficits across multiple tests within the task5. Here we present a more detailed protocol for this task. Rats are placed into a chamber and allowed to discriminate between two scented sands, one of which contains a reinforcer. The trial ends after the rat either correctly discriminates (defined as digging in the correct scented sand), incorrectly discriminates, or 30 sec elapses. Rats that correctly discriminate are allowed to recover and consume the reinforcer. Rats that discriminate incorrectly are immediately removed from the chamber. This can continue through a variety of reversals and novel scents. The primary analysis is the accuracy for each scent pairing (cumulative proportion correct for each scent). The general findings from the Dig task suggest that it is a simple experimental preparation that can assess deficits in rats with bilateral frontal cortical damage compared to rats with unilateral parietal damage. The Dig task can also be easily incorporated into an existing cognitive test battery. The use of more tasks such as this one can lead to more accurate testing of frontal function following injury, which may lead to therapeutic options for treatment. All animal use was conducted in accordance with protocols approved by the Institutional Animal Care and Use Committee.
Neuroscience, Issue 71, Medicine, Neurobiology, Anatomy, Physiology, Psychology, Behavior, cognitive assessment, dig task, scent discrimination, olfactory, brain injury, traumatic brain injury, TBI, brain damage, rats, animal model
50033
Play Button
Progressive-ratio Responding for Palatable High-fat and High-sugar Food in Mice
Authors: Sandeep Sharma, Cecile Hryhorczuk, Stephanie Fulton.
Institutions: University of Montreal.
Foods that are rich in fat and sugar significantly contribute to over-eating and escalating rates of obesity. The consumption of palatable foods can produce a rewarding effect that strengthens action-outcome associations and reinforces future behavior directed at obtaining these foods. Increasing evidence that the rewarding effects of energy-dense foods play a profound role in overeating and the development of obesity has heightened interest in studying the genes, molecules and neural circuitry that modulate food reward1,2. The rewarding impact of different stimuli can be studied by measuring the willingness to work to obtain them, such as in operant conditioning tasks3. Operant models of food reward measure acquired and voluntary behavioral responses that are directed at obtaining food. A commonly used measure of reward strength is an operant procedure known as the progressive ratio (PR) schedule of reinforcement.4,5 In the PR task, the subject is required to make an increasing number of operant responses for each successive reward. The pioneering study of Hodos (1961) demonstrated that the number of responses made to obtain the last reward, termed the breakpoint, serves as an index of reward strength4. While operant procedures that measure changes in response rate alone cannot separate changes in reward strength from alterations in performance capacity, the breakpoint derived from the PR schedule is a well-validated measure of the rewarding effects of food. The PR task has been used extensively to assess the rewarding impact of drugs of abuse and food in rats (e.g.,6-8), but to a lesser extent in mice9. The increased use of genetically engineered mice and diet-induced obese mouse models has heightened demands for behavioral measures of food reward in mice. In the present article we detail the materials and procedures used to train mice to respond (lever-press) for a high-fat and high-sugar food pellets on a PR schedule of reinforcement. We show that breakpoint response thresholds increase following acute food deprivation and decrease with peripheral administration of the anorectic hormone leptin and thereby validate the use of this food-operant paradigm in mice.
Neuroscience, Issue 63, behavioral neuroscience, operant conditioning, food, reward, obesity, leptin, mouse
3754
Play Button
Novel Apparatus and Method for Drug Reinforcement
Authors: Allison A. Feduccia, Christine L. Duvauchelle.
Institutions: University of Texas at Austin.
Animal models of reinforcement have proven to be useful for understanding the neurobiological mechanisms underlying drug addiction. Operant drug self-administration and conditioned place preference (CPP) procedures are expansively used in animal research to model various components of drug reinforcement, consumption, and addiction in humans. For this study, we used a novel approach to studying drug reinforcement in rats by combining traditional CPP and self-administration methodologies. We assembled an apparatus using two Med Associate operant chambers, sensory stimuli, and a Plexiglas-constructed neutral zone. These modifications allowed our experiments to encompass motivational aspects of drug intake through self-administration and drug-free assessment of drug/cue conditioning strength with the CPP test. In our experiments, rats self-administered cocaine (0.75 mg/kg/inj, i.v.) during either four (e.g., the "short-term") or eight (e.g., the "long-term") alternating-day sessions in an operant environment containing distinctive sensory cues (e.g., olfactory and visual). On the alternate days, in the other (differently-cued) operant environment, saline was available for self-infusion (0.1 ml, i.v.). Twenty-four hours after the last self-administration/cue-pairing session, a CPP test was conducted. Consistent with typical CPP findings, there was a significant preference for the chamber associated with cocaine self-administration. In addition, in animals undergoing the long-term experiment, a significant positive correlation between CPP magnitude and the number of cocaine-reinforced lever responses. In conclusion, this apparatus and approach is time and cost effective, can be used to examine a wide array of topics pertaining to drug abuse, and provides more flexibility in experimental design than CPP or self-administration methods alone.
Neuroscience, Issue 42, conditioned place preference (CPP), self-administration, rat, behavioral neuroscience, drug reinforcement, cocaine, animal models
1998
Play Button
RNAi-mediated Double Gene Knockdown and Gustatory Perception Measurement in Honey Bees (Apis mellifera)
Authors: Ying Wang, Nicholas Baker, Gro V. Amdam.
Institutions: Arizona State University , Norwegian University of Life Sciences.
This video demonstrates novel techniques of RNA interference (RNAi) which downregulate two genes simultaneously in honey bees using double-stranded RNA (dsRNA) injections. It also presents a protocol of proboscis extension response (PER) assay for measuring gustatory perception. RNAi-mediated gene knockdown is an effective technique downregulating target gene expression. This technique is usually used for single gene manipulation, but it has limitations to detect interactions and joint effects between genes. In the first part of this video, we present two strategies to simultaneously knock down two genes (called double gene knockdown). We show both strategies are able to effectively suppress two genes, vitellogenin (vg) and ultraspiracle (usp), which are in a regulatory feedback loop. This double gene knockdown approach can be used to dissect interrelationships between genes and can be readily applied in different insect species. The second part of this video is a demonstration of proboscis extension response (PER) assay in honey bees after the treatment of double gene knockdown. The PER assay is a standard test for measuring gustatory perception in honey bees, which is a key predictor for how fast a honey bee's behavioral maturation is. Greater gustatory perception of nest bees indicates increased behavioral development which is often associated with an earlier age at onset of foraging and foraging specialization in pollen. In addition, PER assay can be applied to identify metabolic states of satiation or hunger in honey bees. Finally, PER assay combined with pairing different odor stimuli for conditioning the bees is also widely used for learning and memory studies in honey bees.
Neuroscience, Issue 77, Genetics, Behavior, Neurobiology, Molecular Biology, Chemistry, Biochemistry, biology (general), genetics (animal and plant), animal biology, RNA interference, RNAi, double stranded RNA, dsRNA, double gene knockdown, vitellogenin gene, vg, ultraspiracle gene, usp, vitellogenin protein, Vg, ultraspiracle protein, USP, green fluorescence protein, GFP, gustatory perception, proboscis extension response, PER, honey bees, Apis mellifera, animal model, assay
50446
Play Button
Irrelevant Stimuli and Action Control: Analyzing the Influence of Ignored Stimuli via the Distractor-Response Binding Paradigm
Authors: Birte Moeller, Hartmut Schächinger, Christian Frings.
Institutions: Trier University, Trier University.
Selection tasks in which simple stimuli (e.g. letters) are presented and a target stimulus has to be selected against one or more distractor stimuli are frequently used in the research on human action control. One important question in these settings is how distractor stimuli, competing with the target stimulus for a response, influence actions. The distractor-response binding paradigm can be used to investigate this influence. It is particular useful to separately analyze response retrieval and distractor inhibition effects. Computer-based experiments are used to collect the data (reaction times and error rates). In a number of sequentially presented pairs of stimulus arrays (prime-probe design), participants respond to targets while ignoring distractor stimuli. Importantly, the factors response relation in the arrays of each pair (repetition vs. change) and distractor relation (repetition vs. change) are varied orthogonally. The repetition of the same distractor then has a different effect depending on response relation (repetition vs. change) between arrays. This result pattern can be explained by response retrieval due to distractor repetition. In addition, distractor inhibition effects are indicated by a general advantage due to distractor repetition. The described paradigm has proven useful to determine relevant parameters for response retrieval effects on human action.
Behavior, Issue 87, stimulus-response binding, distractor-response binding, response retrieval, distractor inhibition, event file, action control, selection task
51571
Play Button
Recording Single Neurons' Action Potentials from Freely Moving Pigeons Across Three Stages of Learning
Authors: Sarah Starosta, Maik C. Stüttgen, Onur Güntürkün.
Institutions: Ruhr-University Bochum.
While the subject of learning has attracted immense interest from both behavioral and neural scientists, only relatively few investigators have observed single-neuron activity while animals are acquiring an operantly conditioned response, or when that response is extinguished. But even in these cases, observation periods usually encompass only a single stage of learning, i.e. acquisition or extinction, but not both (exceptions include protocols employing reversal learning; see Bingman et al.1 for an example). However, acquisition and extinction entail different learning mechanisms and are therefore expected to be accompanied by different types and/or loci of neural plasticity. Accordingly, we developed a behavioral paradigm which institutes three stages of learning in a single behavioral session and which is well suited for the simultaneous recording of single neurons' action potentials. Animals are trained on a single-interval forced choice task which requires mapping each of two possible choice responses to the presentation of different novel visual stimuli (acquisition). After having reached a predefined performance criterion, one of the two choice responses is no longer reinforced (extinction). Following a certain decrement in performance level, correct responses are reinforced again (reacquisition). By using a new set of stimuli in every session, animals can undergo the acquisition-extinction-reacquisition process repeatedly. Because all three stages of learning occur in a single behavioral session, the paradigm is ideal for the simultaneous observation of the spiking output of multiple single neurons. We use pigeons as model systems, but the task can easily be adapted to any other species capable of conditioned discrimination learning.
Neuroscience, Issue 88, pigeon, single unit recording, learning, memory, extinction, spike sorting, operant conditioning, reward, electrophysiology, animal cognition, model species
51283
Play Button
A Novel Procedure for Evaluating the Reinforcing Properties of Tastants in Laboratory Rats: Operant Intraoral Self-administration
Authors: AnneMarie Levy, Cheryl L. Limebeer, Justin Ferdinand, Ucal Shillingford, Linda A. Parker, Francesco Leri.
Institutions: University of Guelph.
This paper describes a novel method for studying the bio-behavioral basis of addiction to food. This method combines the surgical component of taste reactivity with the behavioral aspects of operant self-administration of drugs. Under very brief general anaesthesia, rats are implanted with an intraoral (IO) cannula that allows delivery of test solutions directly in the oral cavity. Animals are then tested in operant self-administration chambers whereby they can press a lever to receive IO infusions of test solutions. IO self-administration has several advantages over experimental procedures that involve drinking a solution from a spout or operant responding for solid pellets or solutions delivered in a receptacle. Here, we show that IO self-administration can be employed to study self-administration of high fructose corn syrup (HFCS). Rats were first tested for self-administration on a progressive ratio (PR) schedule, which assesses the maximum amount of operant behavior that will be emitted for different concentrations of HFCS (i.e. 8%, 25%, and 50%). Following this test, rats self-administered these concentrations on a continuous schedule of reinforcement (i.e. one infusion for each lever press) for 10 consecutive days (1 session/day; each lasting 3 hr), and then they were retested on the PR schedule. On the continuous reinforcement schedule, rats took fewer infusions of higher concentrations, although the lowest concentration of HFCS (8%) maintained more variable self-administration. Furthermore, the PR tests revealed that 8% had lower reinforcing value than 25% and 50%. These results indicate that IO self-administration can be employed to study acquisition and maintenance of responding for sweet solutions. The sensitivity of the operant response to differences in concentration and schedule of reinforcement makes IO self-administration an ideal procedure to investigate the neurobiology of voluntary intake of sweets.
Behavior, Issue 84, Administration, Oral, Conditioning, Operant, Reinforcement (Psychology), Reinforcement Schedule, Taste, Neurosciences, Intraoral infusions, operant chambers, self-administration, high fructose corn syrup, progressive ratio, breakpoint, addiction
50956
Play Button
Drosophila Adult Olfactory Shock Learning
Authors: Bilal R. Malik, James J.L. Hodge.
Institutions: University of Bristol.
Drosophila have been used in classical conditioning experiments for over 40 years, thus greatly facilitating our understanding of memory, including the elucidation of the molecular mechanisms involved in cognitive diseases1-7. Learning and memory can be assayed in larvae to study the effect of neurodevelopmental genes8-10 and in flies to measure the contribution of adult plasticity genes1-7. Furthermore, the short lifespan of Drosophila facilitates the analysis of genes mediating age-related memory impairment5,11-13. The availability of many inducible promoters that subdivide the Drosophila nervous system makes it possible to determine when and where a gene of interest is required for normal memory as well as relay of different aspects of the reinforcement signal3,4,14,16. Studying memory in adult Drosophila allows for a detailed analysis of the behavior and circuitry involved and a measurement of long-term memory15-17. The length of the adult stage accommodates longer-term genetic, behavioral, dietary and pharmacological manipulations of memory, in addition to determining the effect of aging and neurodegenerative disease on memory3-6,11-13,15-21. Classical conditioning is induced by the simultaneous presentation of a neutral odor cue (conditioned stimulus, CS+) and a reinforcement stimulus, e.g., an electric shock or sucrose, (unconditioned stimulus, US), that become associated with one another by the animal1,16. A second conditioned stimulus (CS-) is subsequently presented without the US. During the testing phase, Drosophila are simultaneously presented with CS+ and CS- odors. After the Drosophila are provided time to choose between the odors, the distribution of the animals is recorded. This procedure allows associative aversive or appetitive conditioning to be reliably measured without a bias introduced by the innate preference for either of the conditioned stimuli. Various control experiments are also performed to test whether all genotypes respond normally to odor and reinforcement alone.
Neuroscience, Issue 90, Drosophila, Pavlovian learning, classical conditioning, learning, memory, olfactory, electric shock, associative memory
50107
Play Button
Studying Food Reward and Motivation in Humans
Authors: Hisham Ziauddeen, Naresh Subramaniam, Victoria C. Cambridge, Nenad Medic, Ismaa Sadaf Farooqi, Paul C. Fletcher.
Institutions: University of Cambridge, University of Cambridge, University of Cambridge, Addenbrooke's Hospital.
A key challenge in studying reward processing in humans is to go beyond subjective self-report measures and quantify different aspects of reward such as hedonics, motivation, and goal value in more objective ways. This is particularly relevant for the understanding of overeating and obesity as well as their potential treatments. In this paper are described a set of measures of food-related motivation using handgrip force as a motivational measure. These methods can be used to examine changes in food related motivation with metabolic (satiety) and pharmacological manipulations and can be used to evaluate interventions targeted at overeating and obesity. However to understand food-related decision making in the complex food environment it is essential to be able to ascertain the reward goal values that guide the decisions and behavioral choices that people make. These values are hidden but it is possible to ascertain them more objectively using metrics such as the willingness to pay and a method for this is described. Both these sets of methods provide quantitative measures of motivation and goal value that can be compared within and between individuals.
Behavior, Issue 85, Food reward, motivation, grip force, willingness to pay, subliminal motivation
51281
Play Button
Combining Computer Game-Based Behavioural Experiments With High-Density EEG and Infrared Gaze Tracking
Authors: Keith J. Yoder, Matthew K. Belmonte.
Institutions: Cornell University, University of Chicago, Manesar, India.
Experimental paradigms are valuable insofar as the timing and other parameters of their stimuli are well specified and controlled, and insofar as they yield data relevant to the cognitive processing that occurs under ecologically valid conditions. These two goals often are at odds, since well controlled stimuli often are too repetitive to sustain subjects' motivation. Studies employing electroencephalography (EEG) are often especially sensitive to this dilemma between ecological validity and experimental control: attaining sufficient signal-to-noise in physiological averages demands large numbers of repeated trials within lengthy recording sessions, limiting the subject pool to individuals with the ability and patience to perform a set task over and over again. This constraint severely limits researchers' ability to investigate younger populations as well as clinical populations associated with heightened anxiety or attentional abnormalities. Even adult, non-clinical subjects may not be able to achieve their typical levels of performance or cognitive engagement: an unmotivated subject for whom an experimental task is little more than a chore is not the same, behaviourally, cognitively, or neurally, as a subject who is intrinsically motivated and engaged with the task. A growing body of literature demonstrates that embedding experiments within video games may provide a way between the horns of this dilemma between experimental control and ecological validity. The narrative of a game provides a more realistic context in which tasks occur, enhancing their ecological validity (Chaytor & Schmitter-Edgecombe, 2003). Moreover, this context provides motivation to complete tasks. In our game, subjects perform various missions to collect resources, fend off pirates, intercept communications or facilitate diplomatic relations. In so doing, they also perform an array of cognitive tasks, including a Posner attention-shifting paradigm (Posner, 1980), a go/no-go test of motor inhibition, a psychophysical motion coherence threshold task, the Embedded Figures Test (Witkin, 1950, 1954) and a theory-of-mind (Wimmer & Perner, 1983) task. The game software automatically registers game stimuli and subjects' actions and responses in a log file, and sends event codes to synchronise with physiological data recorders. Thus the game can be combined with physiological measures such as EEG or fMRI, and with moment-to-moment tracking of gaze. Gaze tracking can verify subjects' compliance with behavioural tasks (e.g. fixation) and overt attention to experimental stimuli, and also physiological arousal as reflected in pupil dilation (Bradley et al., 2008). At great enough sampling frequencies, gaze tracking may also help assess covert attention as reflected in microsaccades - eye movements that are too small to foveate a new object, but are as rapid in onset and have the same relationship between angular distance and peak velocity as do saccades that traverse greater distances. The distribution of directions of microsaccades correlates with the (otherwise) covert direction of attention (Hafed & Clark, 2002).
Neuroscience, Issue 46, High-density EEG, ERP, ICA, gaze tracking, computer game, ecological validity
2320
Play Button
A Proboscis Extension Response Protocol for Investigating Behavioral Plasticity in Insects: Application to Basic, Biomedical, and Agricultural Research
Authors: Brian H. Smith, Christina M. Burden.
Institutions: Arizona State University.
Insects modify their responses to stimuli through experience of associating those stimuli with events important for survival (e.g., food, mates, threats). There are several behavioral mechanisms through which an insect learns salient associations and relates them to these events. It is important to understand this behavioral plasticity for programs aimed toward assisting insects that are beneficial for agriculture. This understanding can also be used for discovering solutions to biomedical and agricultural problems created by insects that act as disease vectors and pests. The Proboscis Extension Response (PER) conditioning protocol was developed for honey bees (Apis mellifera) over 50 years ago to study how they perceive and learn about floral odors, which signal the nectar and pollen resources a colony needs for survival. The PER procedure provides a robust and easy-to-employ framework for studying several different ecologically relevant mechanisms of behavioral plasticity. It is easily adaptable for use with several other insect species and other behavioral reflexes. These protocols can be readily employed in conjunction with various means for monitoring neural activity in the CNS via electrophysiology or bioimaging, or for manipulating targeted neuromodulatory pathways. It is a robust assay for rapidly detecting sub-lethal effects on behavior caused by environmental stressors, toxins or pesticides. We show how the PER protocol is straightforward to implement using two procedures. One is suitable as a laboratory exercise for students or for quick assays of the effect of an experimental treatment. The other provides more thorough control of variables, which is important for studies of behavioral conditioning. We show how several measures for the behavioral response ranging from binary yes/no to more continuous variable like latency and duration of proboscis extension can be used to test hypotheses. And, we discuss some pitfalls that researchers commonly encounter when they use the procedure for the first time.
Neuroscience, Issue 91, PER, conditioning, honey bee, olfaction, olfactory processing, learning, memory, toxin assay
51057
Play Button
Automated, Quantitative Cognitive/Behavioral Screening of Mice: For Genetics, Pharmacology, Animal Cognition and Undergraduate Instruction
Authors: C. R. Gallistel, Fuat Balci, David Freestone, Aaron Kheifets, Adam King.
Institutions: Rutgers University, Koç University, New York University, Fairfield University.
We describe a high-throughput, high-volume, fully automated, live-in 24/7 behavioral testing system for assessing the effects of genetic and pharmacological manipulations on basic mechanisms of cognition and learning in mice. A standard polypropylene mouse housing tub is connected through an acrylic tube to a standard commercial mouse test box. The test box has 3 hoppers, 2 of which are connected to pellet feeders. All are internally illuminable with an LED and monitored for head entries by infrared (IR) beams. Mice live in the environment, which eliminates handling during screening. They obtain their food during two or more daily feeding periods by performing in operant (instrumental) and Pavlovian (classical) protocols, for which we have written protocol-control software and quasi-real-time data analysis and graphing software. The data analysis and graphing routines are written in a MATLAB-based language created to simplify greatly the analysis of large time-stamped behavioral and physiological event records and to preserve a full data trail from raw data through all intermediate analyses to the published graphs and statistics within a single data structure. The data-analysis code harvests the data several times a day and subjects it to statistical and graphical analyses, which are automatically stored in the "cloud" and on in-lab computers. Thus, the progress of individual mice is visualized and quantified daily. The data-analysis code talks to the protocol-control code, permitting the automated advance from protocol to protocol of individual subjects. The behavioral protocols implemented are matching, autoshaping, timed hopper-switching, risk assessment in timed hopper-switching, impulsivity measurement, and the circadian anticipation of food availability. Open-source protocol-control and data-analysis code makes the addition of new protocols simple. Eight test environments fit in a 48 in x 24 in x 78 in cabinet; two such cabinets (16 environments) may be controlled by one computer.
Behavior, Issue 84, genetics, cognitive mechanisms, behavioral screening, learning, memory, timing
51047
Play Button
Appetitive Associative Olfactory Learning in Drosophila Larvae
Authors: Anthi A. Apostolopoulou, Annekathrin Widmann, Astrid Rohwedder, Johanna E. Pfitzenmaier, Andreas S. Thum.
Institutions: University of Konstanz, University of Fribourg.
In the following we describe the methodological details of appetitive associative olfactory learning in Drosophila larvae. The setup, in combination with genetic interference, provides a handle to analyze the neuronal and molecular fundamentals of specifically associative learning in a simple larval brain. Organisms can use past experience to adjust present behavior. Such acquisition of behavioral potential can be defined as learning, and the physical bases of these potentials as memory traces1-4. Neuroscientists try to understand how these processes are organized in terms of molecular and neuronal changes in the brain by using a variety of methods in model organisms ranging from insects to vertebrates5,6. For such endeavors it is helpful to use model systems that are simple and experimentally accessible. The Drosophila larva has turned out to satisfy these demands based on the availability of robust behavioral assays, the existence of a variety of transgenic techniques and the elementary organization of the nervous system comprising only about 10,000 neurons (albeit with some concessions: cognitive limitations, few behavioral options, and richness of experience questionable)7-10. Drosophila larvae can form associations between odors and appetitive gustatory reinforcement like sugar11-14. In a standard assay, established in the lab of B. Gerber, animals receive a two-odor reciprocal training: A first group of larvae is exposed to an odor A together with a gustatory reinforcer (sugar reward) and is subsequently exposed to an odor B without reinforcement 9. Meanwhile a second group of larvae receives reciprocal training while experiencing odor A without reinforcement and subsequently being exposed to odor B with reinforcement (sugar reward). In the following both groups are tested for their preference between the two odors. Relatively higher preferences for the rewarded odor reflect associative learning - presented as a performance index (PI). The conclusion regarding the associative nature of the performance index is compelling, because apart from the contingency between odors and tastants, other parameters, such as odor and reward exposure, passage of time and handling do not differ between the two groups9.
Neuroscience, Issue 72, Developmental Biology, Neurobiology, Biochemistry, Molecular Biology, Physiology, Behavior, Drosophila, fruit fly, larvae, instar, olfaction, olfactory system, odor, 1-octanol, OCT, learning, reward, sugar, feeding, animal model
4334
Play Button
Training Synesthetic Letter-color Associations by Reading in Color
Authors: Olympia Colizoli, Jaap M. J. Murre, Romke Rouw.
Institutions: University of Amsterdam.
Synesthesia is a rare condition in which a stimulus from one modality automatically and consistently triggers unusual sensations in the same and/or other modalities. A relatively common and well-studied type is grapheme-color synesthesia, defined as the consistent experience of color when viewing, hearing and thinking about letters, words and numbers. We describe our method for investigating to what extent synesthetic associations between letters and colors can be learned by reading in color in nonsynesthetes. Reading in color is a special method for training associations in the sense that the associations are learned implicitly while the reader reads text as he or she normally would and it does not require explicit computer-directed training methods. In this protocol, participants are given specially prepared books to read in which four high-frequency letters are paired with four high-frequency colors. Participants receive unique sets of letter-color pairs based on their pre-existing preferences for colored letters. A modified Stroop task is administered before and after reading in order to test for learned letter-color associations and changes in brain activation. In addition to objective testing, a reading experience questionnaire is administered that is designed to probe for differences in subjective experience. A subset of questions may predict how well an individual learned the associations from reading in color. Importantly, we are not claiming that this method will cause each individual to develop grapheme-color synesthesia, only that it is possible for certain individuals to form letter-color associations by reading in color and these associations are similar in some aspects to those seen in developmental grapheme-color synesthetes. The method is quite flexible and can be used to investigate different aspects and outcomes of training synesthetic associations, including learning-induced changes in brain function and structure.
Behavior, Issue 84, synesthesia, training, learning, reading, vision, memory, cognition
50893
Play Button
Operant Sensation Seeking in the Mouse
Authors: Christopher M. Olsen, Danny G. Winder.
Institutions: Vanderbilt University Medical Center.
Operant methods are powerful behavioral tools for the study of motivated behavior. These 'self-administration' methods have been used extensively in drug addiction research due to their high construct validity. Operant studies provide researchers a tool for preclinical investigation of several aspects of the addiction process. For example, mechanisms of acute reinforcement (both drug and non-drug) can be tested using pharmacological or genetic tools to determine the ability of a molecular target to influence self-administration behavior1-6. Additionally, drug or food seeking behaviors can be studied in the absence of the primary reinforcer, and the ability of pharmacological compounds to disrupt this process is a preclinical model for discovery of molecular targets and compounds that may be useful for the treatment of addiction3,7-9. One problem with performing intravenous drug self-administration studies in the mouse is the technical difficulty of maintaining catheter patency. Attrition rates in these experiments are high and can reach 40% or higher10-15. Another general problem with drug self-administration is discerning which pharmacologically-induced effects of the reinforcer produce specific behaviors. For example, measurement of the reinforcing and neurological effects of psychostimulants can be confounded by their psychomotor effects. Operant methods using food reinforcement can avoid these pitfalls, although their utility in studying drug addiction is limited by the fact that some manipulations that alter drug self-administration have a minimal impact on food self-administration. For example, mesolimbic dopamine lesion or knockout of the D1 dopamine receptor reduce cocaine self-administration without having a significant impact on food self-administration 12,16. Sensory stimuli have been described for their ability to support operant responding as primary reinforcers (i.e. not conditioned reinforcers)17-22. Auditory and visual stimuli are self-administered by several species18,21,23, although surprisingly little is known about the neural mechanisms underlying this reinforcement. The operant sensation seeking (OSS) model is a robust model for obtaining sensory self-administration in the mouse, allowing the study of neural mechanisms important in sensory reinforcement24. An additional advantage of OSS is the ability to screen mutant mice for differences in operant behavior that may be relevant to addiction. We have reported that dopamine D1 receptor knockout mice, previously shown to be deficient in psychostimulant self-administration, also fail to acquire OSS24. This is a unique finding in that these mice are capable of learning an operant task when food is used as a reinforcer. While operant studies using food reinforcement can be useful in the study of general motivated behavior and the mechanisms underlying food reinforcement, as mentioned above, these studies are limited in their application to studying molecular mechanisms of drug addiction. Thus, there may be similar neural substrates mediating sensory and psychostimulant reinforcement that are distinct from food reinforcement, which would make OSS a particularly attractive model for the study of drug addiction processes. The degree of overlap between other molecular targets of OSS and drug reinforcers is unclear, but is a topic that we are currently pursuing. While some aspects of addiction such as resistance to extinction may be observed with OSS, we have found that escalation 25 is not observed in this model24. Interestingly, escalation of intake and some other aspects of addiction are observed with self-administration of sucrose26. Thus, when non-drug operant procedures are desired to study addiction-related processes, food or sensory reinforcers can be chosen to best fit the particular question being asked. In conclusion, both food self-administration and OSS in the mouse have the advantage of not requiring an intravenous catheter, which allows a higher throughput means to study the effects of pharmacological or genetic manipulation of neural targets involved in motivation. While operant testing using food as a reinforcer is particularly useful in the study of the regulation of food intake, OSS is particularly apt for studying reinforcement mechanisms of sensory stimuli and may have broad applicability to novelty seeking and addiction.
Neuroscience, Issue 45, novelty seeking, self-administration, addiction, motivation, reinforcement
2292
Play Button
Protein WISDOM: A Workbench for In silico De novo Design of BioMolecules
Authors: James Smadbeck, Meghan B. Peterson, George A. Khoury, Martin S. Taylor, Christodoulos A. Floudas.
Institutions: Princeton University.
The aim of de novo protein design is to find the amino acid sequences that will fold into a desired 3-dimensional structure with improvements in specific properties, such as binding affinity, agonist or antagonist behavior, or stability, relative to the native sequence. Protein design lies at the center of current advances drug design and discovery. Not only does protein design provide predictions for potentially useful drug targets, but it also enhances our understanding of the protein folding process and protein-protein interactions. Experimental methods such as directed evolution have shown success in protein design. However, such methods are restricted by the limited sequence space that can be searched tractably. In contrast, computational design strategies allow for the screening of a much larger set of sequences covering a wide variety of properties and functionality. We have developed a range of computational de novo protein design methods capable of tackling several important areas of protein design. These include the design of monomeric proteins for increased stability and complexes for increased binding affinity. To disseminate these methods for broader use we present Protein WISDOM (http://www.proteinwisdom.org), a tool that provides automated methods for a variety of protein design problems. Structural templates are submitted to initialize the design process. The first stage of design is an optimization sequence selection stage that aims at improving stability through minimization of potential energy in the sequence space. Selected sequences are then run through a fold specificity stage and a binding affinity stage. A rank-ordered list of the sequences for each step of the process, along with relevant designed structures, provides the user with a comprehensive quantitative assessment of the design. Here we provide the details of each design method, as well as several notable experimental successes attained through the use of the methods.
Genetics, Issue 77, Molecular Biology, Bioengineering, Biochemistry, Biomedical Engineering, Chemical Engineering, Computational Biology, Genomics, Proteomics, Protein, Protein Binding, Computational Biology, Drug Design, optimization (mathematics), Amino Acids, Peptides, and Proteins, De novo protein and peptide design, Drug design, In silico sequence selection, Optimization, Fold specificity, Binding affinity, sequencing
50476
Play Button
Perceptual and Category Processing of the Uncanny Valley Hypothesis' Dimension of Human Likeness: Some Methodological Issues
Authors: Marcus Cheetham, Lutz Jancke.
Institutions: University of Zurich.
Mori's Uncanny Valley Hypothesis1,2 proposes that the perception of humanlike characters such as robots and, by extension, avatars (computer-generated characters) can evoke negative or positive affect (valence) depending on the object's degree of visual and behavioral realism along a dimension of human likeness (DHL) (Figure 1). But studies of affective valence of subjective responses to variously realistic non-human characters have produced inconsistent findings 3, 4, 5, 6. One of a number of reasons for this is that human likeness is not perceived as the hypothesis assumes. While the DHL can be defined following Mori's description as a smooth linear change in the degree of physical humanlike similarity, subjective perception of objects along the DHL can be understood in terms of the psychological effects of categorical perception (CP) 7. Further behavioral and neuroimaging investigations of category processing and CP along the DHL and of the potential influence of the dimension's underlying category structure on affective experience are needed. This protocol therefore focuses on the DHL and allows examination of CP. Based on the protocol presented in the video as an example, issues surrounding the methodology in the protocol and the use in "uncanny" research of stimuli drawn from morph continua to represent the DHL are discussed in the article that accompanies the video. The use of neuroimaging and morph stimuli to represent the DHL in order to disentangle brain regions neurally responsive to physical human-like similarity from those responsive to category change and category processing is briefly illustrated.
Behavior, Issue 76, Neuroscience, Neurobiology, Molecular Biology, Psychology, Neuropsychology, uncanny valley, functional magnetic resonance imaging, fMRI, categorical perception, virtual reality, avatar, human likeness, Mori, uncanny valley hypothesis, perception, magnetic resonance imaging, MRI, imaging, clinical techniques
4375
Play Button
From Voxels to Knowledge: A Practical Guide to the Segmentation of Complex Electron Microscopy 3D-Data
Authors: Wen-Ting Tsai, Ahmed Hassan, Purbasha Sarkar, Joaquin Correa, Zoltan Metlagel, Danielle M. Jorgens, Manfred Auer.
Institutions: Lawrence Berkeley National Laboratory, Lawrence Berkeley National Laboratory, Lawrence Berkeley National Laboratory.
Modern 3D electron microscopy approaches have recently allowed unprecedented insight into the 3D ultrastructural organization of cells and tissues, enabling the visualization of large macromolecular machines, such as adhesion complexes, as well as higher-order structures, such as the cytoskeleton and cellular organelles in their respective cell and tissue context. Given the inherent complexity of cellular volumes, it is essential to first extract the features of interest in order to allow visualization, quantification, and therefore comprehension of their 3D organization. Each data set is defined by distinct characteristics, e.g., signal-to-noise ratio, crispness (sharpness) of the data, heterogeneity of its features, crowdedness of features, presence or absence of characteristic shapes that allow for easy identification, and the percentage of the entire volume that a specific region of interest occupies. All these characteristics need to be considered when deciding on which approach to take for segmentation. The six different 3D ultrastructural data sets presented were obtained by three different imaging approaches: resin embedded stained electron tomography, focused ion beam- and serial block face- scanning electron microscopy (FIB-SEM, SBF-SEM) of mildly stained and heavily stained samples, respectively. For these data sets, four different segmentation approaches have been applied: (1) fully manual model building followed solely by visualization of the model, (2) manual tracing segmentation of the data followed by surface rendering, (3) semi-automated approaches followed by surface rendering, or (4) automated custom-designed segmentation algorithms followed by surface rendering and quantitative analysis. Depending on the combination of data set characteristics, it was found that typically one of these four categorical approaches outperforms the others, but depending on the exact sequence of criteria, more than one approach may be successful. Based on these data, we propose a triage scheme that categorizes both objective data set characteristics and subjective personal criteria for the analysis of the different data sets.
Bioengineering, Issue 90, 3D electron microscopy, feature extraction, segmentation, image analysis, reconstruction, manual tracing, thresholding
51673
Play Button
A Fully Automated and Highly Versatile System for Testing Multi-cognitive Functions and Recording Neuronal Activities in Rodents
Authors: Weimin Zheng, Edgar A. Ycu.
Institutions: The Neurosciences Institute, San Diego, CA.
We have developed a fully automated system for operant behavior testing and neuronal activity recording by which multiple cognitive brain functions can be investigated in a single task sequence. The unique feature of this system is a custom-made, acoustically transparent chamber that eliminates many of the issues associated with auditory cue control in most commercially available chambers. The ease with which operant devices can be added or replaced makes this system quite versatile, allowing for the implementation of a variety of auditory, visual, and olfactory behavioral tasks. Automation of the system allows fine temporal (10 ms) control and precise time-stamping of each event in a predesigned behavioral sequence. When combined with a multi-channel electrophysiology recording system, multiple cognitive brain functions, such as motivation, attention, decision-making, patience, and rewards, can be examined sequentially or independently.
Neuroscience, Issue 63, auditory behavioral task, acoustic chamber, cognition test, multi-channel recording, electrophysiology, attention, motivation, decision, patience, rat, two-alternative choice pitch discrimination task, behavior
3685
Play Button
Human Fear Conditioning Conducted in Full Immersion 3-Dimensional Virtual Reality
Authors: Nicole C. Huff, David J. Zielinski, Matthew E. Fecteau, Rachael Brady, Kevin S. LaBar.
Institutions: Duke University, Duke University.
Fear conditioning is a widely used paradigm in non-human animal research to investigate the neural mechanisms underlying fear and anxiety. A major challenge in conducting conditioning studies in humans is the ability to strongly manipulate or simulate the environmental contexts that are associated with conditioned emotional behaviors. In this regard, virtual reality (VR) technology is a promising tool. Yet, adapting this technology to meet experimental constraints requires special accommodations. Here we address the methodological issues involved when conducting fear conditioning in a fully immersive 6-sided VR environment and present fear conditioning data. In the real world, traumatic events occur in complex environments that are made up of many cues, engaging all of our sensory modalities. For example, cues that form the environmental configuration include not only visual elements, but aural, olfactory, and even tactile. In rodent studies of fear conditioning animals are fully immersed in a context that is rich with novel visual, tactile and olfactory cues. However, standard laboratory tests of fear conditioning in humans are typically conducted in a nondescript room in front of a flat or 2D computer screen and do not replicate the complexity of real world experiences. On the other hand, a major limitation of clinical studies aimed at reducing (extinguishing) fear and preventing relapse in anxiety disorders is that treatment occurs after participants have acquired a fear in an uncontrolled and largely unknown context. Thus the experimenters are left without information about the duration of exposure, the true nature of the stimulus, and associated background cues in the environment1. In the absence of this information it can be difficult to truly extinguish a fear that is both cue and context-dependent. Virtual reality environments address these issues by providing the complexity of the real world, and at the same time allowing experimenters to constrain fear conditioning and extinction parameters to yield empirical data that can suggest better treatment options and/or analyze mechanistic hypotheses. In order to test the hypothesis that fear conditioning may be richly encoded and context specific when conducted in a fully immersive environment, we developed distinct virtual reality 3-D contexts in which participants experienced fear conditioning to virtual snakes or spiders. Auditory cues co-occurred with the CS in order to further evoke orienting responses and a feeling of "presence" in subjects 2 . Skin conductance response served as the dependent measure of fear acquisition, memory retention and extinction.
JoVE Neuroscience, Issue 42, fear conditioning, virtual reality, human memory, skin conductance response, context learning
1993
Play Button
Laser Capture Microdissection of Mammalian Tissue
Authors: Robert A Edwards.
Institutions: University of California, Irvine (UCI).
Laser capture microscopy, also known as laser microdissection (LMD), enables the user to isolate small numbers of cells or tissues from frozen or formalin-fixed, paraffin-embedded tissue sections. LMD techniques rely on a thermo labile membrane placed either on top of, or underneath, the tissue section. In one method, focused laser energy is used to melt the membrane onto the underlying cells, which can then be lifted out of the tissue section. In the other, the laser energy vaporizes the foil along a path "drawn" on the tissue, allowing the selected cells to fall into a collection device. Each technique allows the selection of cells with a minimum resolution of several microns. DNA, RNA, protein, and lipid samples may be isolated and analyzed from micro-dissected samples. In this video, we demonstrate the use of the Leica AS-LMD laser microdissection instrument in seven segments, including an introduction to the principles of LMD, initializing the instrument for use, general considerations for sample preparation, mounting the specimen and setting up capture tubes, aligning the microscope, adjusting the capture controls, and capturing tissue specimens. Laser-capture micro-dissection enables the investigator to isolate samples of pure cell populations as small as a few cell-equivalents. This allows the analysis of cells of interest that are free of neighboring contaminants, which may confound experimental results.
Issue 8, Basic Protocols, Laser Capture Microdissection, Microdissection Techniques, Leica
309
Play Button
Protocols for Microapplicator-assisted Infection of Lepidopteran Larvae with Baculovirus
Authors: Huarong Li, Wendy Sparks, Bryony Bonning.
Institutions: Iowa State University.
Baculoviruses are widely used both as protein expression vectors and as insect pest control agents. . This video shows how lepidopteran larvae can be infected with microapplicator techniques in the gut with baculovirus polyhedra and in the hemolymph with budded virus. This accompanying Springer Protocols section provides an overview of the baculovirus lifecycle and use of baculoviruses as insecticidal agents. Formulation and application of baculoviruses for pest control purposes are described elsewhere.
Plant Biology, Issue 18, Springer Protocols, Baculovirus insecticides, recombinant baculovirus, insect pest management
889
Copyright © JoVE 2006-2015. All Rights Reserved.
Policies | License Agreement | ISSN 1940-087X
simple hit counter

What is Visualize?

JoVE Visualize is a tool created to match the last 5 years of PubMed publications to methods in JoVE's video library.

How does it work?

We use abstracts found on PubMed and match them to JoVE videos to create a list of 10 to 30 related methods videos.

Video X seems to be unrelated to Abstract Y...

In developing our video relationships, we compare around 5 million PubMed articles to our library of over 4,500 methods videos. In some cases the language used in the PubMed abstracts makes matching that content to a JoVE video difficult. In other cases, there happens not to be any content in our video library that is relevant to the topic of a given abstract. In these cases, our algorithms are trying their best to display videos with relevant content, which can sometimes result in matched videos with only a slight relation.