Automated Interactive Video Playback for Studies of Animal Communication

The overall goal of the following experiment is to use an automated, interactive video playback system to study courtship behavior in female sword tail fish. This is accomplished by modeling a male digital fish, adding motion and interactivity, and creating rules for how it will behave relative to the subject fish. In this demonstration, a virtual male will be tracking the movements of a real female to set up the system.

An aquarium flanked by two monitors is constructed and connected to the by observed viewer system to track subjects in real time. Motion data is then passed to the interactive video playback program on a separate server to determine the motion of the animated stimulus on screen. Once the system is in place, females are given a simultaneous choice between according male con specific and according male hetero specific on opposite sides of the aquarium.

Ultimately, the preference in courtship behavior that female fish show towards males of a similar species can be influenced by the addition of an interactive male stimulus. Demonstrating the procedural will be Tricia Bakowski, former grad student from my lab and current employee at Dreamworks SKG. Position a camera over the tank and connect it to a by observe video card.

Make adjustments so that the entire aquarium is in view. Configure the byob Observe viewer system to receive tracking information from the camera. Next, enable the plugin to the BYOBs Observe viewer that sends coordinates to the animation server via a specified network IP address.

To begin calibration, turn on the animation server and open interactive display setup txt. Enter screen with and screen height in pixels under number monitor, info under number program type enter calibration. Be sure to save these changes.

Next, double click on the interactive video playback or IVP program icon to start the program to ensure correct correspondence between viewer data and the interactive output from IVP. Enter the tank dimensions as they were set up in the viewer system earlier to correct for the ends of the fish tank being smaller than the monitors. Use the keyboard to move the viewing area or view port of the stimuli to align with the ends of the testing tank viewed on the animation server.

The pink view port is echoed on one testing monitor and the blue view port on the other. Next scale the display of the male fish on both view ports until it reaches the desired stimulus size. Use the standard calibration stimulus and scale to 66.7 millimeters using a standard benchmark.

Move the boxes in the display so that they align to the edges of the test tank. Use the queue or w keys to move the boxes toward or away from the center of the fish to set up the system. Place a CRT monitor at each end of an 80 liter aquarium filled with clean water and ensure that the brightness of the two monitors is matched.

Load the model to be tested in this demonstration. Files corresponding to according males, phosphorus, birman, and according male ex MechE are selected. Choose from non-interactive or interactive set the side and stage in which each stimulus will appear and the desired size of each stimulus.

Next choose of the movements of the two stimuli are to be mirrored or non-med. Finally, the male's courtship displays can be made interactive by choosing either the default setting in which the dorsal fin is raised only during a lateral courtship display or a distance. The subject must be from the mail in order to trigger a response.

To begin a test session, open interactive display setup, txt and set number program type to live testing and set number, experiment name to the same file name that contains the experiment data. Turn on the IVP program, but don't press start. This will leave a blank screen for the acclimatization period.

Gently place the subject fish into the aquarium and allow it 10 minutes to settle. Before testing, start the by observe viewer 2.0 and the IVP. The video stimuli are displayed to subjects in a trial consisting of two 20 minute stages.

A stage consists of four or five minute segments, an example of which is seen here. The first segment acclimatizes the female to the test tank for five minutes by displaying a monochromatic screen on both monitors. In the second segment, a different video stimulus is displayed on the left and right monitor for an additional five minutes.

Immediately following the display of the video stimuli, a monochromatic screen is again displayed for five minutes on both monitors. In the fourth and final segment, stimuli from the second segment are presented, but the position of each stimulus is switched. This provides a within female control for side bias.

The second stage can repeat the same pattern with a different set of simultaneously presented stimuli. After all stages are complete, remove the subject and introduce the next fish to be tested to control for side biases and order effects systematically vary side and order of presentation across subjects. The interactive stimulus shown here in red closely tracks the female's horizontal position shown in blue in real time.

The three rules guiding male behavior are as follows, where the X direction refers to the length of the tank and the direction refers to the width of the tank. Rule one, the simulated male always follows the subject across the screen tracking her in the Z direction. Rule two, the simulated male's dorsal fin is only raised when it is performing a lateral courtship display.

Rule three, the simulated male sword tail fish will only perform a lateral courtship display for 50%of the total time it is being displayed to the female. The lateral courtship display is triggered by the male fish being within one fourth male body length of the female fish in the Z direction. The lateral courtship display is independent of how close the female is to the monitor.

In the X Direction Association, time is automatically computed by the by observe viewer and is defined as the amount of time that a female spends within 10 centimeters of one monitor or the other non-interactive stimuli Created by the IVP show that female X experts mon strongly preferred visual signals of their own species. However, when females were tested with interactive conspecific and hetero specific males, they failed to show a preference After its development. This technique allowed researchers in the field of animal behavior to explore social interactions using quantitative algorithms.