Operant Protocols for Assessing the Cost-benefit Analysis During Reinforced Decision Making by Rodents

Summary

A cost-benefit analysis is a weighing-scale approach that the brain performs during the course of decision making. Here, we propose a protocol to train rats on an operant-based decision-making paradigm where rats choose higher rewards at the expense of waiting for 15 s to receive them.

Abstract

Reinforcement-guided decision making is the ability to choose between competing courses of action based on the relative value of the benefits and their consequences. This process is integral to the normal human behavior and has been shown to be disrupted by neurological and psychiatric disorders such as addiction, schizophrenia, and depression. Rodents have long been used to uncover the neurobiology of human cognition. To this end, several behavioral tasks have been developed; however, most are non-automated and are labor-intensive. The recent development of the open-source microcontroller has enabled researchers to automate operant-based tasks for assessing a variety of cognitive tasks, standardizing the stimulus presentation, improving the data recording and consequently, improving the research output. Here, we describe an automated delay-based reinforcement-guided decision-making task, using an operant T-maze controlled by custom-written software programs. Using these decision-making tasks, we show the changes in the local field potential activities in the anterior cingulate cortex of a rat whilst it performs a delay-based cost-and-benefit decision-making task.

Introduction

Decision making is the process of recognizing and selecting choices based on the values and preferences of the decision maker and the consequences of the selected action¹. Although decision making has been extensively studied in different fields (i.e., economics, psychology, and neuroscience), neural mechanisms underlying such cognitive abilities are not yet fully understood. Two subcategories of decision making are perceptual decision making and reinforcement-guided decision making. Though they incorporate considerable overlapping elements and concepts, perceptual decision making relies on the available sensory information^1,2, whereas reinforcement-guided decision-making deals with the relative value of actions gained over a specific timescale³. One important aspect of reinforced decision making is the cost-benefit analysis which is performed intuitively by the brain by computing the benefits of the given choices and subtracting the associated costs of each alternative¹.

The T-maze (or the variant Y-maze) is one of the most-used mazes in cognitive experiments using rodents. Animals are placed in the start arm (the base of the T) and permitted to choose the goal arm (one of the side arms). Tasks such as a forced alternation or left-right discrimination are mainly used with rodents in the T-maze to test reference and working memory⁴. T-mazes are also widely used in decision-making experiments^5,6,7. In the simplest design, the reward is placed in only one goal arm. The choice is predictable, and animals would certainly prefer the reward rather than nothing, regardless of the reward value. Another option is to place rewards in both goal arms and then let the animals make a choice of which path to take depending on several parameters (i.e., the natural preference of the animal, the difference in the value of the rewards, and the costs to be paid). In the value-based design, the task is more complicated by having weighing-scale properties. In this way, an animal receives differently valued rewards by choosing between the two alternatives, as well as between the costs of the actions [i.e., the amount of waiting (delay-based) or the amount of effort (effort-based) needed to receive rewards], each contributing to the decision that is made^5,6.

In traditional delay-based T-maze decision making, animals are trained to select the high reward arm (HRA) and avoid the opposite low reward arm (LRA). The sides of the HRA and the LRA remain unchanged throughout the experiment. Although the task described above has been well documented in the literature, it suffers from several procedural drawbacks. Firstly, by having a fixed goal arm, the animal knows which arm to choose from the beginning of each trial. In this scenario, animals may select the goal arm based on their memory rather than on decision making. Hence, in a delay-based decision-making paradigm, if an animal selects the low reward because of the study intervention, it will not be clear whether this is due to a loss of memory or to the study intervention. A memory control group to segregate the observed behavior from the memory problem might be considered, but this burdens researchers and animals alike because of the additional work⁷. A second concern is the moment of decision making by the animal: once animals reach the decision zone (the junction of all three arms), they usually look to the left and to the right, weigh the costs and benefits regarding each arm, and then make their decision. However, after a few trials, they perform such a computation prior to arriving at the decision zone and simply run directly to the reward arm. As a result, these two drawbacks—a pre-bias to one arm and finding the moment of decision making—both highly interrupt the interpretation of electrophysiological and neuroimaging data.

In the method explained in this paper, the preferred arm (HRA) is cued by an auditory cue and may vary from trial to trial. Animals initiate the trials by entering the test zone (Figure 1) and triggering the auditory cue by "nose-poking" an infrared gate that has been placed at the junction of the three arms. The audio signal (20 dB, between 500 and 1,000 ms) is played from a speaker at the end of the goal arm.

Protocol

All procedures explained here were approved and carried out in accordance with the Guide for the Care and Use of Laboratory Animals and were approved by the Florey Institute Animal Ethics Committee or the Neuroscience Research Center.

1. Housing, Handling, and Food Restriction

1. Use adult (normally 8 weeks old) male rats (any strains) and keep them in the room with a 12-h light/dark cycle.
2. Restrict their food access to encourage the animals to perform the task.
   Note: Housing the animals singularly is suggested, as it provides a better control of each one’s food intake.
3. On days 1 - 3, handle the animals for approximately 5 min, 2x a day.
   Note: Handling familiarizes the animals with human contact and decreases the level of stress and anxiety in the animals throughout the experiments.
4. Weigh the animals after each handling session. Use the first day’s weight as the free-feeding amount and keep the animals on about 80 - 85% of their free-feeding amount.

2. Experimental Set-up

1. Use a T-maze equipped with 3 partitions or arms, 2 stimulus speakers, 5 retractable doors, and 5 infrared motion sensors or infrared beam (IRB) sensors.
   Note: The T-maze can be constructed of medium-density fiberboard (MDF) or polyvinyl chloride (PVC).
2. Control the maze constructed here by an Arduino microcontroller.
3. Copy the Arduino code into the computer.
4. Download the Arduino software from the software website and install it on a computer.
5. Connect the microcontroller to the computer using a USB port.
6. On the computer, click the software icon, go to Tools, and then select Port.
7. From the drop-down menu, choose the COM port (Communication port) which connects the software to the computer.
8. Go to Tools and select Boards. From the drop-down menu, select the type of Arduino that controls the T-maze.
9. Click Upload on the top left of the interface window. Select the Arduino code. Wait until the process finishes.
10. Click Serial monitor at the top right of the interface window. Then, in a new pop-out window, change the baud rate to 115200.

3. Habituation to the Maze

1. Prior to each session of habituation, bring the animals to the experiment room at least 1 h prior to the experiment.
2. Leave 10 sugar pellets at each goal arm and leave all doors open.
   Note: At this stage, there is no need to track the animals’ movements (Figure 1).
3. Days 1 - 3: Place the animals individually in the maze for periods of 10 min, 1x per day, to explore the maze with no restrictions.
4. Days 4 - 5: Place the animals individually in the maze. Leave 2 pellets in each arm and allow them to sample the food from both sides. Remove the animal from the maze immediately after eating pellets from both sides or after 5 min of being in the maze.
   Note: The main difference between this stage and the previous stage is that animals need to be removed from the maze immediately after eating pellets from both sides. This would familiarize an animal with being removed from the maze at the end of a trial.
5. After each session, clean the maze’s floors with 70% ethanol and make sure the ethanol has evaporated before placing the next animal in the maze.

4. Discrimination Training

1. Maze preparation
   1. Day 6: Bring the animals to the experiment room at least 1 h prior to the experiment.
   2. Run the Arduino program and set the trial number to 14 of which the first 4 trials will be forced-choice and the rest will be choice trials.
      Note: The program will randomly generate an equal number of trials to be assigned to the left and right side of the T-maze.
   3. Before each trial begins, place 4 pellets in the goal arm which is to be cued as the HRA and 2 pellets in the other arm which is the LRA.
2. Forced-choice trial training (4 trials)
   1. Pseudo-randomly block one arm before each trial so that the animal is forced to select the other arm.
      Note: The blocked arm can be a high reward arm or a low reward arm whilst the auditory signal highlights the side of the HRA.
   2. Place an animal in the start box (Figure 1). After 5 - 7 s, simultaneously open the start gate and click Start in the Arduino’s interface window.
   3. Collect the animal immediately after eating pellets or after 5 min of being in the maze.
   4. Leave the animal in the home cage for 2 min.
3. Choice trial training (10 trials)
   1. Prior to each trial, close door-B in the arm which is selected by the software to be the HRA. Leave door-A open in the opposite arm (the LRA).
   2. Place an animal in the start box (Figure 1). After 5 - 7 s, simultaneously open the start gate and click Start.
   3. Let the animal freely choose either arm. If the animal chooses the HRA, open door-A, let the animal enter the chamber, shut door-A, and open door-B immediately to give the animal access to the selected food well.
   4. If the animal chooses the LRA, open door-B in order to give the animal access to the food well.
   5. Remove the animal after it has eaten all the food in the selected food well and leave it in the home cage for 2 min.
   6. Complete the 10 choice trials for each animal and record the animal’s choice (HRA or LRA) in each trial.
   7. Calculate the percentage of high reward choice (HRC) for all choice-trials after each training session.
   8. Following the completion of 14 trails, the individual times obtained by each infrared sensor are shown in the Arduino interface window.

5. Delay Training

1. Once each animal has reached 80% of HRC in the discrimination training (step 4), begin the delay training by running 10 trials per day for each animal. In this stage, door-B is to be opened after only a 5 s delay when the animal chooses the HRA.
2. If the animal reaches an HRC of 80% at the end of a training session with a 5 s delay, increase the delay to 10 s for the next training session.
3. Once the animal has reached an HRC of 80% with a 10 s delay, increase the delay to 15 s for the next training session.
4. Record the animal’s choice for each trial to calculate the HRC after each training session.
5. Following the completion of the trials, the individual times obtained by each infrared sensor are shown in the Arduino interface window.
   Note 1: The timer starts when you click Enter. The ‘Time_decision’ is the time after the nose-poke to the IRB-1. ‘Time_left_1’ is the time when an animal breaks IRB-1L and ‘Time_left_2’ is when IRB-2L is broken. If the behavioral study is combined with electrophysiology or neural imaging (e.g., calcium imaging) methods, film recording is suggested for a better synchronization of the animal’s behavior to neural signals.

6. Electrophysiology (Electrode Fabrication)

1. Use single or bipolar electrodes to record the local field potentials (LFPs).
   Note: The use of bipolar electrodes is suggested because it enables local re-referencing to reduce the impact of the volume conduction. Therefore, the recorded LFP is the differential voltage between 2 electrodes with a ~200 μm distance between the tips.
2. To make a bipolar electrode, glue 2 tungsten electrodes together or wind 2 stainless-steel wires together (normally, PFA-coated stainless-steel wires with a 50 μm diameter are used).
3. Solder the grounding and recording wires to a mini plug/socket to become a connector to the electrophysiology recording system.

7. Anesthesia

1. To implant the electrodes, anesthetize the rats by an intraperitoneal injection of a mixture of ketamine (100 mg/kg) and xylazine (8 mg/kg), or by administering isoflurane as described in the following steps.
   Note: Isoflurane is recommended due to the improved control of its anesthetic depth.
2. Use an anesthetic machine with a controlled oxygen and anesthetic vapor flow.
3. To anesthetize the animal by isoflurane, place it in the induction chamber and set isoflurane at a rate of around 4% and oxygen at 500 - 1,000 mL/min on the anesthetic machine.
4. Continue the induction until the righting reflex of the animal disappears.
5. Set the anesthetic maintenance rate at 1 - 2.5% isoflurane and 300 - 400 mL/min oxygen on the machine.
6. Place a heat-pad with insulation (cotton wool) on a stereotaxic device and then immobilize the animal in the stereotaxic device.
7. Measure the level of anesthesia by pinching the toes of the animal.

8. Surgical procedure

1. Using an electric shaver or a pair of scissors, shave the animal’s head where the incision is to be made.
2. Scrub the surgery area with ethanol and then with povidone–iodine, 4%.
3. Apply eye lubricant to protect the animal’s cornea during the surgery.
4. Using tissue forceps, gently pull the skin between the animal’s eyes and ears. Using scissors, cut the skin that is pinched between the forceps and remove the periosteum.
5. Use hydrogen peroxide to clean and disinfect the skull.
6. Find the location of the electrode insertion point by using stereotaxic coordinates and mark it on the skull with a permanent marker.
7. Use a handheld drill to make one hole for the grounding electrode, one for anchoring the implant, and one hole per recording electrode.
8. Insert the electrodes at a very slow rate to minimize the possible cortical and subcortical damage.
9. Connect the grounding electrode to the skull using a small screw.
10. Secure the connector plug/socket in place by using a thin layer of dental acrylic cement.
11. Inject buprenorphine (15 µg/kg, subcutaneously) or meloxicam (1 - 3 mg/kg, subcutaneously) as a painkiller.
12. Leave the animal in a cage on a heated mat to recover. Observe the animal at least every 30 min until it has recovered from the anesthetic.

9. Post-procedure Training

1. Following the surgery, allow at least a 10-day recovery period. House the animals individually with free access to food and water ad libitum and monitor them on a daily basis (see Rudebeck, Walton, Smyth, Bannerman, and Rushworth⁷ for more information).
2. Following the recovery, start restricting the animal’s diet to 85% of their free-feeding amount.
3. Retrain the animals in the maze to recall the discrimination phase (step 4).
4. Continue re-training the mice until they have achieved 80% of HRC.

Results

The data presented here is the recorded LFP from the left orbitofrontal cortex (OFC) and the anterior cingulate cortex (ACC) of six male Wistar rats using bipolar electrodes (of PFA-coated stainless steel). Table 1 shows the behavioral acquisition length for each training stage. The coordinates for the target locations were determined from a rat brain atlas⁹ and are as follows: for the AAC, 1.2 mm anterior to the bregma, 0.8 mm lateral to the midli...

Discussion

Rodents have long been used in neuroscientific studies that deal with different topics, from cognitive abilities such as learning and memory^2,14 and reinforced behavior^7,15,16 to the central control of organs^17,18 and neuropharmacology^19,20. The proposed ...

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
T-maze			Self made
Dustless Precision Sugar Pellets	TSE Systems Intl. Group	F0023	45 mg, Sucrose
Ketamine Hydrochloride Injection, USP	Sigma-Aldrich	6740-87-0
Xylazine	Sigma-Aldrich	7361-61-7
stereotaxic device	Stoelting
Isofluran	Santa Cruz Biotechnology	sc-363629Rx
PFA-coated stainless-steel wires	A-M systems
acrylic cement	Vertex, MA, USA
(wooden or PVC (polyvinyl chloride)-made)	local suppliers
Mini-Fit Power Connector	Molex	15243048
ethannol 70%	Local suppliers
buprenorphine	diamondback drugs
Arduino UNO	Arduino		https://www.arduino.cc/
Infrared emitting diode	Sharp	GL480E00000F	http://www.sharp-world.com/
Chronux Toolbox			Chronux.org
Arduino codes			https://github.com/dechuans/arduino-maze