Recording Single Neurons' Action Potentials from Freely Moving Pigeons Across Three Stages of Learning

Summary

Learning new stimulus-response associations engages a wide range of neural processes which are ultimately reflected in changing spike output of individual neurons. Here we describe a behavioral protocol allowing for the continuous registration of single-neuron activity while animals acquire, extinguish, and reacquire a conditioned response within a single experimental session.

Abstract

While the subject of learning has attracted immense interest from both behavioral and neural scientists, only relatively few investigators have observed single-neuron activity while animals are acquiring an operantly conditioned response, or when that response is extinguished. But even in these cases, observation periods usually encompass only a single stage of learning, i.e. acquisition or extinction, but not both (exceptions include protocols employing reversal learning; see Bingman et al.¹ for an example). However, acquisition and extinction entail different learning mechanisms and are therefore expected to be accompanied by different types and/or loci of neural plasticity.

Accordingly, we developed a behavioral paradigm which institutes three stages of learning in a single behavioral session and which is well suited for the simultaneous recording of single neurons' action potentials. Animals are trained on a single-interval forced choice task which requires mapping each of two possible choice responses to the presentation of different novel visual stimuli (acquisition). After having reached a predefined performance criterion, one of the two choice responses is no longer reinforced (extinction). Following a certain decrement in performance level, correct responses are reinforced again (reacquisition). By using a new set of stimuli in every session, animals can undergo the acquisition-extinction-reacquisition process repeatedly. Because all three stages of learning occur in a single behavioral session, the paradigm is ideal for the simultaneous observation of the spiking output of multiple single neurons. We use pigeons as model systems, but the task can easily be adapted to any other species capable of conditioned discrimination learning.

Introduction

Learning new stimulus-response-outcome associations engages a wide range of neural plasticity processes. These processes are ultimately reflected in the changing spike output of individual neurons. Arguably, one of the most frequently employed learning paradigms is Pavlovian fear conditioning conducted with rodents. In this setting, the acquisition and extinction of a conditioned response take place within a few dozen trials². The rapid development of conditioned fear can be advantageous because it allows running a large number of animals within a short time. Also, acquisition and extinction can be observed within a few tens of trials on a single day in naive animals^3,4 or spread across 2 to 3 days^2,5-8. However, the insights gained about the changes of neural activity during learning in these experiments do not necessarily apply outside the domain of fear conditioning. For example, goal-directed behavior driven by positive reinforcement is more adequately modeled by operant rather than Pavlovian conditioning procedures, and may in part depend on different neural substrates^9,10. Also, fear conditioning develops so rapidly that neural responses to the CS can only be observed for a few dozen trials, placing severe limits on the analysis of changes of neural activity during learning.

Unfortunately, the acquisition and extinction of operant responding usually takes many days. This is detrimental for neurophysiological investigations, because it is notoriously difficult to record the activity of single cells over more than a few hours. Due to the high similarity of the waveforms of extracellularly recorded action potentials, it is problematic to claim that spikes recorded on one day are generated from the same cell as spikes with similar waveforms recorded on the next^11,12, especially in areas with a high cell density such as the hippocampus.

To address these issues, we developed a novel behavioral paradigm utilizing 3 learning conditions within one experimental session on a single day. This requires that the experimental animal is willing to perform hundreds of trials under varying conditions on a thin schedule of reinforcement. Homing pigeons (Columbia livia forma domestica) are classic model organisms in experimental psychology^13-17. These birds are able to perform complex visual discriminations¹⁸, can flexibly adapt behavior to changing reinforcement contingencies^19,20, and are uniquely avid workers, performing 1,000 trials with minimal amount of reinforcement. These characteristics make them especially suitable for the experiments described below.

Protocol

Ethics Statement

All experiments were conducted in accordance with the German guidelines for the care and use of animals in science. Procedures were approved by a national ethics committee of the state of North Rhine-Westphalia, Germany.

System overview

Operant Testing Chamber

The operant chamber (Figure 1) measures 34 cm x 34 cm x 50 cm. Three translucent response keys (4 cm x 4 cm, located approximately 20 cm above floor level) are recessed into the back wall of the chamber. Stimuli are shown through an LCD flat screen mounted behind the response keys. Two 2-Watt light bulbs located at the side walls provide dim illumination. The chamber is housed in a sound-attenuating cubicle to mask extraneous sounds. Loudspeakers provide white noise at all times. Food (grain) is provided by a food hopper located below the center key. Experimental hardware is controlled by custom-written MATLAB code²¹. Animals are constantly monitored through a digital camera attached to the front wall of the chamber.

Custom-built Microdrives

Microdrives housing 16 electrode wires are custom-built in our laboratory; the design is based on work by Bilkey and colleagues^22,23, and the reader is referred to these articles for a detailed description. We modified their design to allow for a larger number of electrodes (16 instead of 8; 25 µm nichrome wires), and we connect the electrode wires via conductive silver glue to the headstage socket. Additionally, we use gold-plating of the electrode tips to reduce impedance and to achieve better signal-to-noise ratios (apply -3 V for ~3 sec; impedances should drop to <100 kΩ).

Once the microdrive is assembled, electrodes are cut to the desired length, tips are cleaned in an ultrasonic bath (Tergazyme in distilled water) for 20 min and rinsed another 20 min in distilled water. Gold-plating of electrode tips should take place immediately before implantation. For grounding, we use a silver ball electrode placed above the lateral cerebellum. Specification of materials is provided in the Materials table which accompanies this article.

An important issue when working with freely moving animals is movement artifacts. We found that movement artifacts in our setups are largely due to a) high electrode impedances (>500 kΩ) and b) imperfect attachment of the contacts between the plug (implant) and the socket (headstage) while the animal is moving. A variety of commercially available microconnectors does not perform satisfactorily for recording from freely moving birds, because the mechanical contact between plug and socket rapidly deteriorates through vigorous movements of the pigeons (head-bobbing, key-pecking). The best mechanical connection between implant and headstage was achieved with headplug assemblies from Ginder Scientific. These plug-socket assemblies feature 18 contacts and are firmly affixed to each other by a ring nut.

Electrophysiological Recording Setup

The electrophysiology setup comprises the following components: 1) a custom-built headstage with unity gain (operational amplifier) 2) 15 differential amplifier modules housed in two rack mount units (DPA-2FS and EPMS-07, respectively; npi electronic GmbH, Germany), 3) a 16-channel analog-to-digital converter (power 1401 mark I). Raw signals are amplified 1,000x and band-pass filtered (500-5,000 Hz, 1^st order filter), digitized with a sampling rate of 16-20 kHz and stored with Spike2 Version 7.06a for offline processing. Event times (such as stimulus onset or individual key pecks of the animal) are captured via a laboratory-built parallel port IO box (see Rose et al.²¹) and forwarded to the AD converter for storage along with the neurophysiological data (see Figure 1). Offline processing is described further below.

Figure 1. System overview. Information flow is symbolized by colored arrows. Computer 1 controls hardware pertaining to behavioral output (stimulus display via the flat screen monitor, house light, food hopper, feeder light, response keys) and sends event timestamps to the AD converter. Computer 2 stores neurophysiological signals obtained from the A/D converter and event timestamps received from Computer 1. The photograph on the left shows the conditioning chamber inside the sound-attenuating cubicle. Its elements are: 1) Sound-attenuating shell, 2-4) response keys, 5) food hopper, 6) feeder light 7) house light, 8) observation camera.

Single-Interval-Forced-Choice (SIFC) Discrimination Task

For clarity, we will describe the final SIFC task here and then explain the steps needed to train animals on this task below.

The SIFC task is outlined in Figure 2. After the intertrial interval (ITI) has elapsed, the center key is transilluminated green for up to 5 sec ('initialization phase'). Immediately following the third response of the animal within 5 sec, one out of several sample stimuli is presented on the center key for 2 sec ('sample phase'; example stimuli are shown in the inset to Figure 2). After 2 sec, the center key is again transilluminated green, and the animal has to respond once more before the two side keys are transilluminated ('confirmation phase'). Depending on the identity of the stimulus shown in the sample phase, the animal is required to direct a single response to either the left or the right key ('choice phase'). If it chooses the correct destination, access to reward (grain) is granted for 2 sec. Thus, the core of the task consists of responding to the left choice key after presentation of one particular stimulus on the center key, and responding to the right choice key after presentation of another stimulus. The reason that the sample phase is bracketed by an initialization and a confirmation phase is to keep the animals' head in front of the center key while the sample stimulus is presented.

Once the animal masters this task for a single pair of stimuli (henceforth, 'familiar' stimuli, FS), it is presented with a novel stimulus (NS) pair in every new session, and has to learn which of the two novel stimuli is to be followed by a response to the left or the right choice key. The FS pair continues to be presented during those experiments to serve as suitable control condition. Adequate performance on the final task hinges crucially on the animals' willingness to perform >1,000 trials at overall reinforcement probabilities <0.5. The following paragraphs describe a training procedure in which task complexity is gradually increased until the animal reaches the level of the SIFC; at the same time, reinforcement probability and the number of trials per session need to be increased to ensure consistently high performance on the final task.

1. Animal Training

1. Food Restriction
   1. Weigh the animals after at least two weeks of free access to food. Take this weight as the free-feeding weight. Restrict food access over the next 1-2 weeks until animals reach 85% of their free-feeding weight.
   2. It is critical that pigeons maintain a healthy physical appearance and normal activity throughout the duration of the entire experiment. To that end, carefully monitor the animals' appearance and weight across the entire duration of behavioral training and testing. Weigh animals before and after each experimental session to assess daily food intake. Supply additional food if necessary to prevent further weight loss. Provide unrestricted access to food over the weekend.
2. Autoshaping
   Autoshaping serves to habituate the animal to the experimental chamber and establish conditioned responding.
   1. Present a 5-sec visual stimulus (henceforth, initialization stimulus, IS) on the center key. Immediately upon termination of the IS or a single peck to the response key (whatever comes first), switch off key illumination and present food reward (2 sec activation of the food hopper).
   2. Keep the ITI considerably longer than the sample presentation time to facilitate learning²⁴. Use values of 120 sec for ITI and run 40 trials per day. Later reuse the IS as the initialization, confirmation, and choice key stimulus in the final task (see Figure 2). This phase of training will take the animals approximately one week.
   3. Once the animal responds reliably (in >85% of trials), decrease the ITI stepwise down to 10 sec and the sample presentation time down to 2 sec. At the same time, increase the number of responses required for reinforcement to 3 (fixed ratio of 3, FR 3). Additionally, increase the total number of trials per day. Choose parameters such that the animal is trained every day for approximately 1 hr. This phase of training will last roughly 2 weeks.
   4. Repeat Steps 1.2.1 - 1.2.3 for the left and right response keys until subjects reliably respond to the IS on all 3 keys. Alternate trials with activation of the left, right, and center key randomly.
   5. Now present the IS first at the center and then, conditional on a response, at either side key (omit reinforcement for the center key response). Alternate activated side keys randomly from trial to trial. Terminate trials in which the subject does not respond to the center key after 5 sec. Repeat until the animal performs reliably (~3 days).
   6. Introduce 2 new stimuli which will later serve as FS in the final task (see Figure 2, inset, for examples). Repeat Steps 1.2.1 - 1.2.3 with these stimuli. Responding will be established more rapidly than with the first stimulus, usually within 4 days.

Figure 2. Illustration of the behavioral paradigm. After an ITI of 5 sec, the center key is transilluminated green for up to 5 sec (initialization). If the animal responds 3x within these 5 sec, 1 out of the 4 sample stimuli is presented at the same position. After a fixed sample presentation time of 2 sec during which the animal has to respond at least once, the central pecking key is transilluminated green again (confirmation). After another peck, the 2 side keys are transilluminated green. The subject indicates its choice by responding once to one of the side keys. During acquisition and reacquisition, correct responses are followed by 2 sec food access accompanied by activation of the feeder light, or activation of the feeder light alone. If incorrect, house lights are turned off for 3 sec. During extinction, both correct and incorrect responses to the extinction stimulus remain inconsequential. Inset shows example novel and familiar stimulus pairs.

3. Training of a Single-Interval Forced-Choice (SIFC) Task for Familiar Stimuli
   1. Establish the full sequence of initialization, sample, confirmation, and choice: on each trial, present first the IS (FR 3), then either of the two FS (2 sec fixed duration), then again the IS (FR 1). Use a prompted-choice design: in each trial, transilluminate only the choice key which is correct for the given FS. This phase of training should take approximately 1 week.
   2. Once the subject performs reliably (>85% responses to the respective side key), introduce free-choice trials (both side keys transilluminated during choice phase). If the animal responds to the correct side, provide food access for 2 sec. Incorrect responses are followed by time-out punishment (houselights off for 2 sec). If no response is given within 3 sec, terminate trial and re-start ITI. Animals usually learn these subcomponents of the task within 2 weeks.
   3. Gradually increase the fraction of free-choice trials during subsequent sessions from 20% to 100%.
   4. If the subject performs >90% correct in free-choice trials, decrease reward probability for correct responses to 0.5 while in parallel increasing the number of trials per session to 1,000. Do not change parameters every day/session but choose them flexibly depending on the performance level of the subject concerning initialization omissions and percentage of correct responses. This phase of training will last about 4 weeks.
   5. Pigeons tend to refuse responding to unfamiliar stimuli. Therefore, once the animal reliably performs >1,000 trials, autoshape responses to a large set of visual stimuli (see Section 1.2). However, do not preexpose the visual stimuli destined for later use as novel stimuli in the final paradigm but uniform color displays.
4. Final Single-Interval Forced-Choice Task with Novel Stimuli Under Different Reinforcement Conditions
   1. Warm-up
      Let the subject perform 50 trials with the FS only. Set reward probability for these stimuli to <1 (say, 0.5 - 0.8) during all phases to prevent premature satiation and, therefore, lack of motivation to respond.
   2. Acquisition Stage
      Randomly alternate trials with presentation of FS and NS. Assign different response keys as correct for the two NS and reinforce every correct response. Compute percentage of correct responses as a running average over the last 120 trials. Acquisition is considered complete once performance for each of the NS exceeds 85%, but not before a minimum of 150 trials have been executed.
   3. Extinction Stage
      Stop reinforcing correct and punishing incorrect responses to a random NS (extinction stimulus). Begin reacquisition phase when correct responding to the extinction stimulus drops below 60% and the animal has experienced at least 150 trials in this phase in total.
   4. Reacquisition stage
      Again reinforce correct and punish incorrect responses to the extinction stimulus, as during the acquisition stage. Terminate the session when performance for this stimulus exceeds 85% and the animals performed at least 150 trials in this phase in total.

2. Electrophysiology

1. Electrode Implantation
   Implantation surgery takes place after animals repeatedly (3-4x) completed the entire acquisition-extinction-reacquisition sequence and is described in more detail elsewhere²⁵.
   1. Place 5-6 stainless steel microsrews on the skull for anchoring a dental cement head mount including the microdrive.
   2. Perform a craniotomy just above the brain region of interest; then carefully dissect the dura and lower the electrodes to the desired position.
   3. Before anchoring the microdrive to the head mount, apply Vaseline around the guide cannula; this will prevent dental cement from encasing the guide tube.
   4. Use an insulated silver ball electrode placed underneath the skull overlying the cerebellum as ground.
   5. Provide animals with painkillers (Carprofen, 10 mg/kg, injected twice daily) for three days following surgery. Allow animals to recover for at least 2 weeks.
2. Recordings While Animals Perform the Task
   1. Use a new pair of novel stimuli for each session and advance electrodes at least 125 µm (half a revolution of the drive screw) before starting. If no action potentials of sufficient signal-to-noise ratio are observed, abort the session, place the animal in the home cage and try again the next day.
   2. Arrange the headstage cable such that it does not interfere with the animal's normal pecking and feeding behavior. This can be achieved by attaching the cable with several elastic straps to the top of the conditioning chamber and habituating the birds to the attached cable for some hours.
   3. If available, make use of a commutator to provide extra freedom of movement for the birds.
3. Offline signal analyses
   1. Band-pass-filter all channels from 500 to 5,000 Hz with steep roll-offs offline using Spike2. Extract spikes with amplitude thresholds and sort them manually employing principal component analysis.
   2. Examine sorting results with custom-written MATLAB code (available at MATLAB Central File Exchange, File ID #37339). A well-isolated single unit (Figure 3) should meet all of the following criteria: a) a clearly separated cluster in principal component space, b) no sign of multiple units when all recorded waveforms are overlaid and plotted as heat map (Figure 3A), c) symmetrically distributed peak waveform amplitudes (Figure 3B), d) stable recording throughout the session as evidenced by unchanging spike amplitude (Figure 3C), e) no or very few spike events that occur during the refractory period of the preceding spike (Figure 3D), and a signal-to-noise ratio (SNR) of at least 2 (SNR is here defined as the difference between the minimum and maximum of the averaged spike waveform, divided by the trimmed width of the noise band (2.5^th and 97.5^th percentiles of the distribution of values from the first bin of all waveforms)). SNR of the unit shown in Figure 3 is 3.9.
   3. Inspect raw channels offline for movement-related artifacts. Discard channels when indicated.
   4. Electrical artifacts occurring during key pecking can in rare cases be confused with proper spike waveforms. Test for the contamination of the recordings by examining the time histogram of spike counts relative to each registered key peck (peri-peck time histogram, PPTH, Figure 3E). Pecking-induced artifacts show up as a peak in the histogram close (±50 msec) to time 0. As an extra check, plot the waveforms of all putative spike events registered within ±20 msec of a key peck separately and compare it to spike waveforms detected outside this window (Figure 3F).

Figure 3. Quality metrics for unit isolation. A) Heat map of all waveforms' time-voltage values. B) Distributions of maximum (red), minimum (green), and noise (blue) voltage values of all waveforms. The distributions are well separated, indicating excellent unit isolation. C) Spontaneous firing rate (red, calculated from 2-sec segments in all intertrial intervals) and spike amplitudes (peak-to-peak) as a function of time in session. Both curves were smoothed with a boxcar function (width: 50 data points). D) Interspike-interval distribution for this unit. Bin width, 10 msec (inset: 1 msec). Very short intervals are nearly absent (<0.1% of intervals below 4 msec). E) PSTH triggered to key pecks. Event counts close to the key peck (±20 msec) are highlighted red. F) All 157 waveforms recorded within ±20 msec of key peck events. The waveforms compare favorably to overall waveform shape shown in panel A.

Results

Behavior

Figure 4A shows the behavioral performance of an animal in one example session. The performance level of the animal reaches criterion for NS 2 within 180 trials (45 stimulus presentations) and is close to 100% for the NS 1 from the beginning. This strategy - first responding to the same key for both new stimuli, and then adjusting responses for one of the stimuli - is about as often observed as initial random responding to both NS. In this session, the NS 2 was randomly ...

Discussion

This protocol describes a complex behavioral task suitable for concurrent single-unit recordings. We have described the SIFC task for pigeons, but it can be easily adapted to rodents by requiring nose pokes or lever pressing rather than key pecks, and substituting visual by olfactory, auditory, or tactile stimuli.

Perhaps the most critical steps during the training procedure are 1) gradual reduction of reward probability and 2) increase in trial number. Regarding intermittent reinforcemen...

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
Resistance wire (for use as electrodes)	California Fine Wire, Grover Beach, CA, USA		Stablohm 675; formvar-coated nichrome wires (outer diameter 25 µm)
Microconnectors	Ginder Scientific, Nepean, Ontario, Canada		GS18PLG-220 (plug) & GS18SKT-220 (socket to build headstage)
Cannulae	Henke Sass Wolf, Tuttlingen, Germany		0.4 x 20 mm/ 27 Gx3/4"
Gold solution for plating	Neuralynx, Bozeman, MT, USA		SIFCO Process Gold Non-Cyanide, Code 5355
Solution for ultrasonic bath	Alconox, Inc., New York, USA	1304	Tergazyme
Conductive glue	Henkel Loctite		LOCTITE 3888 Silver filled, conductive, adhesive
Stainless steel screws	J.I. Morris, Southbridge, MA, USA		F0CE125 self-tapping miniature screws, body length 1/8 inches
Light-curing dental cement	van der Ven Dental, Duisburg, Germany		Omniceram Evo Flow A2
Light-curing unit	van der Ven Dental, Duisburg, Germany		Jovident Excelled 215 Curing Light (wireless LED light curing unit)
Filter amplifiers	npi electronic GmbH, Germany	DPA-2FS
A/D converter	Cambridge Electronic Design, Cambridge, UK	power 1401
Spike2 software	Cambridge Electronic Design, Cambridge, UK		Version 7.06a
MATLAB	The Mathworks, Natick, MA, USA		R2012a