Detecting and Assigning Ultrasonic Vocalizations to Individual Mice during Social Interactions

Summary

This protocol describes a method for recording synchronized video and audio data during multi-mouse social interactions. The method uses a microphone array and sound source localization to assign ultrasonic vocalizations (USVs) to individual mice, allowing quantitative analysis of vocal behavior across animals and social contexts.

Abstract

Mice (Mus musculus) communicate using ultrasonic vocalizations (USVs) during social interactions. However, because mouse USVs are not associated with clear visual indicators and occur at frequencies outside the human auditory range, it remains challenging to identify individual vocalizing mice within a group. This protocol describes a method for recording synchronized video and ultrasonic audio data during multi-animal social interactions, enabling simultaneous capture of behavior and vocal activity. A computational pipeline is outlined that integrates multi-animal tracking, vocalization detection, sound-source localization, and assignment of vocalizations to individual animals. The validation procedures used to assess tracking accuracy, vocalization extraction, localization precision, and assignment confidence at each stage of the pipeline are further described. Application of this protocol to a four-animal demonstration dataset yields individual-resolved vocalization tracking, spatially precise sound-source estimates, and quantitative measures of vocal output and acoustic features. Together, these approaches provide a robust framework for linking vocal communication to individual behavior and enable downstream analyses of sex differences, individual variability, and social communication in freely interacting mice.

Introduction

Social communication across the animal kingdom is an essential component of behavioral dynamics, aiding in processes such as survival and reproduction^1,2. Among mammals, mice communicate acoustically via USVs, which span frequencies from 35 to 110 kHz³. USVs exhibit complex features, conveying information depending on the behavioral context⁴. Female mice exhibit a preference for male mouse USVs^5,6, and male and female mice interact vocally during courtship behaviors⁷, indicating that USVs function as key social signals. Because specific USV features are linked to distinct mouse behavioral states⁸, accurately assigning vocalizations to individuals is essential for understanding social interactions.

Despite extensive efforts to elucidate the functions of USVs and their relationship with social behavior⁸, an ongoing challenge in the field has been assigning USVs to individual mice during group interactions. Difficulties arise due to both a lack of visible mouth movements during USV production⁴ and the frequency range of USVs being inaudible to the human ear³. As a result, prior studies on the relationship between USVs and social behavior have been restricted to simplified assays or constrained environments that limit the range of observable group behaviors⁹, providing only partial insight into how vocal behavior unfolds in freely interacting animals.

To refine the understanding of the relationship between auditory communication and social behavior, it is necessary to capture a full range of mouse behaviors and to assign vocalizations to individuals within a group. Resolving vocalizer identities is essential for the purposes of determining sex- and context-dependent vocal ranges across individuals⁸, testing causal hypotheses about how specific vocal features drive conspecific behavior^10,11,12, and, where applicable, correlating vocal output with simultaneously measured brain signals (e.g., electrophysiology)¹³. Historically, it was believed that male mice were the sole vocalizers; however, sound-source localization revealed that female mice also produce USVs in group settings⁷ or when paired with a male¹⁴. Without determining which animal emitted a USV, linking vocal features to individual behavior and social outcomes in freely interacting groups remains limited. Therefore, utilizing a localization system that enables assignment of vocalizations to individual animals while maintaining natural interaction dynamics is essential.

The microphone-array sound-source localization system described here, which was previously published^7,15, estimates the spatial origin of USVs within an arena and tracks the positions of all animals present at the time of vocalization, enabling vocalizations to be assigned to the emitter. Sound-source estimates and tracked animal positions are then integrated using a probability-based metric to support the assignment of USVs to individual animals. Vocalization assignment converts previously ambiguous group vocal recordings into labeled animal vocal data^7,15 enabling quantitative analyses that link vocalization type and timing to behavior and subsequent conspecific action. The purpose of the present work is to provide a detailed protocol for implementing this method, which was previously quantified and validated¹⁵. The following protocol outlines the setup, calibration, and implementation of this system, including microphone-array geometry, audio-video synchronization, automated sound detection and probabilistic source estimation, multi-animal position tracking, and the assignment procedure used for attributing USVs to the emitting animal^7,15.

Protocol

All protocols involving animals were approved by the University of Delaware Animal Care and Use Committee (protocol number: 1275) and conducted in accordance with National Institutes of Health guidelines. The reagents and the equipment used are listed in the Table of Materials.

1. Constructing the sound-source localization rig

1. Prepare the recording chamber.
   1. Assemble a cuboid frame (76.2 cm × 76. 2 cm × 61 cm; width × length × height), which serves as the recording arena, and place it in a sound-attenuating enclosure.
   2. Use nylon mesh to create the walls and line the outside of the enclosure with acoustic foam to reduce sound reflections.
   3. Place a thermometer outside, but near, the recording chamber the enclosure to monitor ambient temperature during recordings.
   4. Perform all recordings under dark conditions.
      NOTE: Maintain stable environmental conditions across recording sessions, and record environmental parameters.
2. Configure the video and audio recording systems.
   1. Install infrared illumination above the arena to enable video recording in darkness.
   2. Position a video camera above the arena and configure it to record continuously at 30 Hz.
   3. Configure the audio acquisition system to record ultrasonic signals from an eight-channel microphone array at a sampling rate of 250 kHz.
      NOTE: Eight microphones were used in this system, but the number of microphones can be adjusted depending on experimental needs.
   4. Synchronize the video and audio recordings using a shared external trigger to ensure temporal alignment.
   5. Confirm that video and audio data are recorded concurrently and stored for each session.
      ​NOTE: Hardware and acquisition software are listed in the Table of Materials and Table 1, respectively.
3. Arrange the microphones and arena.
   1. Evenly space microphones along the walls of the arena (two per wall).
   2. Position the microphone centers 38.1 cm apart, 19.05 cm from each corner, and 10.8 cm above the arena floor.
   3. Mark microphone locations visually to facilitate later calibration and validation.
      NOTE: LEDs may be used to mark microphone positions.

2. Preparations for recording social behavior

1. Setup the experimental animals.
   1. Select the mice to be recorded. Here, groups of 2 male and 2 female C57BL/6J mice between 8 and 12 weeks of age at the time of recording were used.
      NOTE: The strain, sex ratio, age range of mice, and the number of recording days in the current experimental paradigm are adjustable for different experimental goals. For the demonstration of this method, a subset of experimental mice was reused across two recording days. C57BL/6J mice older than 12 weeks may exhibit reduced sensitivity to high-frequency sounds¹⁶.
2. House the animals.
   1. House each mouse individually for 14 days prior to recording.
      NOTE: Individual housing minimizes the influence of prior social experience on group interactions.
   2. Provide each cage with bedding, enrichment, and ad libitum access to food and water.
3. Mark the animals for visual identification.
   1. Two days prior to recording, anesthetize each mouse using 2.5% isoflurane in oxygen at a flow rate of 1 L/min via a precision vaporizer in a sealed chamber. Once each mouse is anesthetized, transfer it out of the chamber, laying flat with its back accessible, receiving the anesthesia from a nose cone for the duration of the marking procedure.
      NOTE: The anesthesia protocol described in the current experimental paradigm is approved in our animal protocol and was applied to each mouse in serial, but is adjustable for different animals or experimental goals.
   2. While each mouse is under anesthesia, apply a unique pattern to it’s back using non-toxic hair dye to enable visual identification during tracking.
   3. Monitor each animal continuously while the dye sets, maintaining anesthesia for approximately 15 min, and record the duration of anesthesia.
   4. Rinse and place each mouse in a warmed recovery cage until fully ambulatory.
   5. Return each mouse to an individual clean cage with bedding, enrichment, and access to food and water.
      ​NOTE: Apply markings to female mice before males to minimize the transfer of male scent cues that could influence subsequent behavior.
4. Perform opposite-sex exposure.
   1. One day prior to recording, prepare clean cages without bedding.
   2. For each experimental mouse, place the animal into a cage followed by a novel mouse of the opposite sex.
   3. Allow animals to interact for 10 min.
   4. Observe interactions, and manually record behavioral events.
   5. Following exposure, return experimental animals to individual housing following exposure.
      NOTE: Opposite-sex exposure is specific to the current experimental paradigm, as it increases the likelihood of vocal behavior during subsequent group recordings¹⁷; however, it is not a required component of the methods described. Separate animals if copulatory behavior is observed.
5. Determine the estrous states of female mice.
   1. Approximately 2 h prior to recording, assess the estrous state of each female mouse (if using female subjects).
   2. Perform vaginal lavage using 30 µL of phosphate-buffered saline.
   3. Transfer the lavage sample onto a clean microscope slide, and spread evenly.
   4. Allow the slides to dry completely.
   5. Apply crystal violet stain to each slide, and use a pipette to rinse gently with water. 
   6. Examine slides under a light microscope.
   7. Classify estrous state based on the relative proportions of nucleated epithelial cells, cornified epithelial cells, and leukocytes¹⁸.
   8. Record the estrous state for each female.
      NOTE: As USVs may impact female receptivity during estrus^19,20, determining estrous state is specific to the current experimental paradigm.; however, it is not a required component of the methods described.

3. Social interaction recordings

1. Prepare the recording arena.
   1. Add white paper-based bedding to the arena floor to a uniform depth of approximately 1.25 cm.
   2. Ensure that bedding covers all corners evenly.
   3. Verify that all microphones are aligned, unobstructed, and oriented toward the arena.
   4. Prior to experimental recordings, place a ruler and a high-contrast object in the center of the arena to verify camera focus and establish spatial scaling.
   5. Remove the calibration objects before behavioral recordings begin.
      NOTE: The white paper-based bedding provides contrast for darker-furred animals, facilitating tracking. If lighter-furred animals are being used, then a black paper-based bedding would help tracking. If recording durations are extended, add food and water to the arena. Adding North/South and East/West directional markers to the ruler facilitates identification of the eight microphones.
2. Perform pre-behavior system recordings.
   1. Immediately before behavioral recordings, record 15 s of background noise with infrared lights on.
   2. Record 15 s of a microphone localization test with infrared lights off and microphone indicator lights on.
   3. Save all pre-behavior recordings for later verification of system performance.
      ​NOTE: These recordings are used to verify microphone function and baseline noise levels. Scripts used for recording are available in the associated code repository.
3. Record group social behavior.
   1. Prepare all recording settings in advance to minimize animal interaction prior to data collection.
   2. Place the female mice into the arena.
   3. Place the male mice into the arena.
      ​NOTE: If recording multiple animals of the same sex, place both animals simultaneously.
   4. Close the enclosure, and begin recording.
   5. Record group social behavior continuously for 10 min.
      NOTE: Recording duration is adjustable.
   6. Verify that video and audio files are readable and free of corruption prior to preprocessing.
   7. At the conclusion of the recording, remove all animals except one.
4. Record individual animals.
   1. Record the remaining mouse alone in the arena for 10 min.
   2. Repeat individual recordings for each of the remaining mice.
   3. Weigh each mouse immediately after individual recordings.
      NOTE: Individual recordings are used to train and validate multi-animal tracking. Document the temperature inside the rig prior to each recording session, as temperature affects sound propagation. Repeat the recording session if video or audio files are corrupted or incomplete.

4. Video preprocessing

1. Prepare the video files.
   1. Verify that all video recordings are stored in a single directory associated with the experiment.
2. Adjust the video intensity for tracking.
   1. If needed, apply a uniform brightness adjustment to all video frames by adding a constant value (brightness_adjustment x 255) to each pixel, where brightness_adjustment is a scalar typically ranging from 0.1 to 0.6 depending on illumination conditions.
   2. If needed, apply a global contrast adjustment using intensity remapping (e.g., MATLAB’s imadjust function) with a defined input range (e.g., [0.2, 0.8]), which maps pixel intensities at 20% of maximum to black and 80% of maximum to white and linearly scales intermediate values. This step is done to increase the separation between light-colored identification markings and the background.
   3. Clip pixel intensities to the valid range for the video bit depth (i.e., 0 to 255 for 8-bit video).
      NOTE: Identical preprocessing parameters must be applied to all frames within a recording and across recordings to ensure consistency. These adjustments are optional and depend on recording conditions. For the representative data presented here, no brightness or contrast adjustments were applied. When adjustments are needed, users should preview the processed first frame to confirm adequate contrast between animals and the arena before processing the entire video.
3. Export the preprocessed videos.
   1. Save the preprocessed video files using the same frame rate and temporal alignment as the original recordings.
   2. Retain the original video files for reference and verification.
      NOTE: Scripts used for video preprocessing and the specific parameter values applied are available in the associated code repository.

5. Multi-animal tracking

1. Prepare the tracking software and data.
   1. Install a multi-animal tracking software and all required dependencies. This study used MOuse TRacker (MoTR; Table 1) because the format of output is compatible with Muse (Table 1), another software in our pipeline used for sound-source localization calculations²¹.
      NOTE: Other automated tracking software, such as Social LEAP Estimates Animal Poses (SLEAP)²² or DeepLabCut²³, could be used to track animals, as long as the position of the animal, heading direction, and position of the nose are generated.
   2. Organize preprocessed video files so that single-animal recordings and multi-animal recordings are clearly separated.
   3. Verify that all video files have consistent frame rates and spatial resolutions.
      ​NOTE: Software installation instructions and dependency requirements are provided in the associated code repository.
2. Train the single-animal models.
   1. Select single-animal recordings corresponding to each experimental subject.
   2. Train a separate identity classifier for each animal using features extracted from that animal’s single-animal recording. During group recordings, these per-animal classifier outputs are combined with trajectory information across frames to maintain animal identity.
   3. Verify that training completes successfully before proceeding.
      ​NOTE: Single-animal recordings are used to initialize identity-specific tracking in multi-animal sessions.
3. Track multi-animal recordings.
   1. Apply the trained tracking model to multi-animal social interaction recordings.
   2. Generate frame-by-frame estimates of animal position, orientation, and body shape for each subject.
   3. Save tracking outputs for downstream analyses.

6. Acoustic segmentation

1. Prepare the audio data.
   1. Verify that all audio recordings from individual microphone channels are present and accessible.
2. Detect vocalizations.
   1. Run the acoustic segmentation software, Ax (Table 1), on each recording to detect USVs.
   2. Configure time-frequency parameters defined in an input file considered by Ax. The microphone sampling frequency was set to 250 kHz.
      NOTE: Three window sizes for Non-uniform Fast Fourier Transform were considered, including 64, 128, and 256 samples. A multi-taper time-bandwidth product of 3 with 5 tapers was applied, along with a significance threshold of 0.05. A minimum duration of 0 samples was used, with a low frequency cutoff of 30 kHz and a high frequency cutoff of 110 kHz. The convolution size was set to cover 1001 Hz by 0.001 s. A flag was included to merge harmonically related signals. For specific details, see Neunuebel et al.⁷.
   3. Extract candidate USVs based on these time-frequency features.
   4. Save segmentation outputs as a spreadsheet for each recording.
      ​NOTE: Software installation instructions, scripts, and parameter settings are provided in the associated code repository⁷.
3. Compile the segmentation outputs.
   1. Generate a consolidated list of detected vocalization events containing onset times, offset times, and frequency bounds.
   2. Save consolidated vocalization files for downstream localization analyses.

7. Sound-source localization

1. Prepare the inputs for localization.
   1. Confirm that acoustic segmentation outputs and multi-animal tracking results are available for each recording.
   2. Organize localization inputs so that audio data, tracking data, and arena geometry information are accessible to the localization software.
      ​NOTE: Scripts used to arrange input files, define required directory structures, and gather arena geometry information are provided in the associated code repository.
2. Estimate the sound-source locations.
   1. Apply the sound-source localization algorithm, MUSE (Table 1), to each detected vocalization⁷.
   2. Using microphone array recordings, estimate the spatial origin of each vocalization within the arena.
   3. For each vocalization, estimate the sound-source location using jackknife resampling by repeating the localization procedure eight times, each time omitting one microphone so that each estimate is based on the remaining seven microphones. This generates eight estimates per vocalization.
      ​​NOTE: Differences in arrival time across microphones are critical for estimating the sound-source location. Jackknife resampling is used to assess localization reliability and reduce sensitivity to individual microphone channels. Scripts used to estimate the sound-source location are provided in the associated code repository⁷.
3. Compute the sound-source location probability distributions.
   1. For each vocalization, calculate a covariance matrix²⁴, and use this and the average jackknife point estimates to compute probability density across the arena floor.
4. Integrate localization with animal positions.
   1. Extract tracked head or nose positions for all animals at the time of each vocalization.
   2. For each vocalization, evaluate the localization probability density at the location of each animal’s nose.
   3. For each vocalization, compute an assignment confidence metric for each animal based on the probability values. The assignment confidence metric is calculated for each animal as the density value at that animal’s nose divided by the sum of density values of all animals. We refer to this metric as the mouse probability index (MPI)¹⁵.
5. Assign vocalizations. 
   1. If the assignment confidence metric for a single mouse exceeds 0.95, the signal is assigned to that mouse^7,15.
   2. Label vocalizations that do not meet assignment criteria as unassigned.
   3. Save localization and assignment outputs for downstream validation and analysis.
      NOTE: Assignment thresholds, confidence criteria, and code are provided in the associated analysis scripts.

8. Validation of results

1. Validate multi-animal tracking.
   1. Load finalized tracking outputs for each recording.
   2. Overlay tracked animal positions and body shape onto corresponding video frames.
   3. Visually inspect tracking results to confirm accurate mouse position and consistent identity assignment across frames.
      NOTE: Tracking results include color-coded ellipses indicating head direction overlaid on each mouse in each video frame. Ellipse size and body orientation should match mouse size and orientation, respectively, in each frame. Ellipse color should be consistent for each mouse throughout a recording.
   4. Identify and correct tracking errors.
      Save corrected tracking outputs for downstream analyses.
      NOTE: Common tracking errors occur during close social interactions or near arena boundaries. Correct all cases of head-tail flips, identity switches, and ellipse jumps and drifts.
      NOTE: Programs for viewing tracking output and correcting errors are provided in the code repository.
2. Validate vocalization extraction.
   1. Load acoustic segmentation outputs for each recording.
   2. Generate spectrograms from raw audio data for representative time windows.
   3. Overlay detected vocalization events, including individual vocalization time and frequency ranges, onto spectrograms.
   4. Visually confirm correspondence between detected events and vocalizations observed in the spectrograms.
   5. If systematic errors are observed, adjust segmentation parameters before rerunning extraction and localization steps, or manually adjust any case of segmentation errors (i.e., misidentification of vocalization time or frequency range).
3. Validate vocalization assignment.
   1. Load sound-source localization and assignment outputs.
   2. For each vocalization, visualize localization estimates, spatial probability distributions, and animal positions overlaid on video frames.
   3. Confirm that localization probability densities are centered near the head of the assigned animal and are clearly separated from other animals.
   4. Verify that vocalizations failing assignment criteria are labeled as unassigned.
      NOTE: Checks for assignment output include verifying that all expected outputs have been generated and that the probability map for each vocalization is located within arena bounds and surrounding the mouse assigned the vocalization.
   5. Save the validated assignment results for downstream analyses.
      NOTE: Assignment confidence thresholds and validation criteria are defined in the associated analysis scripts.

9. Data analyses

1. Visualize the sound-source localization.
   1. For single- or multi-animal recording sessions, select video frames during which vocalizations were detected.
   2. Generate schematized representations of the selected video frames.
   3. Generate a spatial schematic from the MUSE output by plotting the positions of all microphones, the individual sound-source point estimates, the estimated sound-source location, the spatial probability distribution, the mouse position(s), the inferred vocalizer identity, and a scale bar.
   4. Generate spectrograms of the corresponding vocalization as recorded by each microphone channel.
      ​NOTE: These visualizations are helpful for qualitatively assessing localization performance in both individual and group contexts.
2. Quantify the localization error and precision.
   1. Compute the distance between the estimated sound-source location and the position of the recorded animal to quantify localization error. Use single mouse recordings for these calculations, as there is only a single emitter and the source is known.
   2. To assess assignment precision, transform sound-source estimates and corresponding animal positions into a reference frame centered on the vocalizing animal.
   3. Visualize the spatial distribution of sound-source estimates relative to the centered animal position.
3. Generate spectrograms to visualize vocalizations.
   1. Identify vocalization bouts of interest using vocal extraction outputs containing vocalizer identity, vocalization onset and offset times, and high and low frequency bounds.
   2. Select time windows spanning from before the onset of the first vocalization to after the offset of the last vocalization in each bout.
   3. Extract voltage traces from the audio channel files for the selected time windows.
   4. Generate spectrograms by transforming the extracted voltage traces from the time domain to the frequency domain.
   5. Mark time points corresponding to assigned and unassigned vocalizations using distinct colors on the spectrogram.
   6. For social interaction sessions, identify vocalization bouts produced by different individuals, and generate spectrograms with vocalizer identities indicated using identity-specific colors.
4. Quantify the vocal output by individual.
   1. Use the output of the sound-source localization code to assign each vocalization an identity label, such that unlocalized, localized-but-unassigned, and individual animal identities are encoded numerically.
   2. Sum the number of vocalizations produced by each individual animal.
   3. Compare vocalization counts across individuals and between sexes.
5. Analyze the vocal characteristics.
   1. Extract frequency contours for each vocalization.
   2. Quantify vocal parameters such as maximum frequency, minimum frequency, mean frequency, bandwidth, and vocalization duration.
   3. Compare vocal parameters between sexes.
   4. Perform appropriate statistical analyses based on the distributional properties of the data.
      NOTE: Select statistical methods to match the parametric properties of the measured variables. Here, the number of subjects is too low for statistical comparisons.

Results

The results presented in this article are from recordings conducted over 2 days and are intended to illustrate the output of the protocol, originally published in Warren et al.¹⁵. These data are not intended to support statistical inference or generalized conclusions.

Extraction and assignment of USVs
USVs were recorded during both individual and group social interaction sessions using the microphone-array localization system. During each social int...

Discussion

Reliable attribution of USVs to individual animals during group interactions has been a major technical limitation in the study of mouse vocal communication. Although mice are widely used to study social communication, the inability to assign vocalizations to specific individuals has limited quantitative analysis of vocal behavior in naturalistic social settings. A method that enables animal-specific vocalization tracking during social interactions is therefore essential for studying vocal production in freely interactin...

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
Alpha-Dri Bedding	Shepherd Specialty Papers	ALPHA-dri	Needed for sound source localization system
Aluminum Extrusion	8020	Custom built to match recording arena	Needed for sound source localization system
BNC-2110	National Instruments	777643-01	Needed for sound source localization system
Clear Acrylic Panel	Home Depot	MC-24	Needed for sound source localization system
CM16/CMPA40-5V	Avisoft	40014	Needed for sound source localization system
Computer	Hewlett–Packard	Z620	Needed for sound source localization system
Grasshopper 3	FLIR	GS3-U3-41C6M-C	Needed for sound source localization system
IR lights	GANZ	IR-LT30	Needed for sound source localization system
NI PXIe-1073	National Instruments	781161-01	Needed for sound source localization system
NI PXIe-6356	National Instruments	781053-01	Needed for sound source localization system
Nice ‘N Easy, Born Blonde Maxi	Clairol	2523721	Needed for sound source localization system
Nylon mesh for cage walls	McMaster-Carr	9318T25	Needed for sound source localization system
Power Cord	National Instruments	763000-01	Needed for sound source localization system
SCHC68-68-EPM	National Instruments	192061-02	Needed for sound source localization system
Sonex foam	Pinta Acoustic	VLW-35	Needed for sound source localization system