Enabling High Grayscale Resolution Displays and Accurate Response Time Measurements on Conventional Computers

Summary

Conventional computer hardware can not generate visual stimuli with sufficiently high grayscale resolution and measure response times with sufficient accuracy. We describe how to use the VideoSwitcher to produce high-resolution monochromatic displays, and the RTbox to measure response times with high accuracy on conventional computer hardware.

Abstract

Display systems based on conventional computer graphics cards are capable of generating images with 8-bit gray level resolution. However, most experiments in vision research require displays with more than 12 bits of luminance resolution. Several solutions are available. Bit++ ¹ and DataPixx ² use the Digital Visual Interface (DVI) output from graphics cards and high resolution (14 or 16-bit) digital-to-analog converters to drive analog display devices. The VideoSwitcher ³ described here combines analog video signals from the red and blue channels of graphics cards with different weights using a passive resister network ⁴ and an active circuit to deliver identical video signals to the three channels of color monitors. The method provides an inexpensive way to enable high-resolution monochromatic displays using conventional graphics cards and analog monitors. It can also provide trigger signals that can be used to mark stimulus onsets, making it easy to synchronize visual displays with physiological recordings or response time measurements.

Although computer keyboards and mice are frequently used in measuring response times (RT), the accuracy of these measurements is quite low. The RTbox is a specialized hardware and software solution for accurate RT measurements. Connected to the host computer through a USB connection, the driver of the RTbox is compatible with all conventional operating systems. It uses a microprocessor and high-resolution clock to record the identities and timing of button events, which are buffered until the host computer retrieves them. The recorded button events are not affected by potential timing uncertainties or biases associated with data transmission and processing in the host computer. The asynchronous storage greatly simplifies the design of user programs. Several methods are available to synchronize the clocks of the RTbox and the host computer. The RTbox can also receive external triggers and be used to measure RT with respect to external events.

Both VideoSwitcher and RTbox are available for users to purchase. The relevant information and many demonstration programs can be found at http://lobes.usc.edu/.

Protocol

1. VideoSwitcher

1.1. Connect VideoSwitcher to a computer

The VideoSwitcher (Figure 1) receives analog video (VGA) inputs and is used to drive cathode ray tube (CRT) color monitors. Before using the VideoSwitcher, make sure the computer graphics card has either a VGA port (Figure 2A) or a Digital Visual Interface (DVI-I) port (Figure 2B) that transmits both digital and analog video signals. A DVI-to-VGA adaptor (Figure 2C) is necessary to connect the DVI-I port to the VideoSwitcher.

Turn off the computer and monitor and disconnect the monitor from the computer graphics card.

Connect the VGA or the DVI-I port of the computer graphics card to the input port of the VideoSwitcher using the provided VGA cable. Use a small screw driver to secure the connections.

Connect the monitor's VGA cable to the output port of the VideoSwitcher and secure the connections.

Connect the VideoSwitcher to a power source using the provided power adaptor.

Turn on the computer and the monitor.

1.2. Two display modes of the video switcher

The switch button on the VideoSwitcher allows the user to switch back-and-forth in two display modes. In the color mode, the VideoSwitcher does not alter the video signal from the computer graphics card. In the grayscale mode, the display becomes monochromatic, with its luminance determined mostly by the video signal from the blue channel of the graphics card, slightly affected by the signal from the red channel, and no contribution from the green channel. This is because the resister network inside the VideoSwitcher attenuates the signal from the red channel by a factor of about 128, and the signal from the green channel doesn't contribute at all. The ratio of the contributions of the blue and red channels to the output of the VideoSwitcher is called blue-to-red ratio (BRratio). The BRratio is property of a Video Switcher, independent of the display setup. It can be determined experimentally for each VideoSwitcher (see 1.3).

The display mode can also be switched by sending a special signal through the green channel of the graphics card using a function call in Matlab:

PsychVideoSwitcher('SwitchMode', whichScreen, ToGrayMode);

where ToGrayMode of 1 or 0 means to switch to grayscale or color mode.

Figure 3 shows the schematic of a VideoSwitcher.

1.3. Measure BRratio

The BRratio can be measured for each VideoSwitcher using a square wave grating detection task. The two levels of the square wave are assigned RGB values such as (0, 0, 40) and (x, 0, 39). One can adjust the value of x until the square wave grating disappears, at which point the x value is the BRratio --- the magnitude of the signal in the R-channel that leads to the intensity produced by one unit of signal in the B-channel. Although one can measure the BRratio electronically, for example to measure the ratio of the voltage outputs of a VideoSwitcher at RGB values of (0, 0, 255) and (255, 0, 0), the psychophysical method described here is very simple and accurate.

1.4. Display Calibration

For most monitors, display luminance doesn't increase linearly with the pixel gray level specified by the computer graphics card. The relationship can be described as:

where L_max, L_min and γ are determined with a Gamma correction procedure, either by using photometers or by a combination of psychophysical procedures and photometric measurements ^7,8. Gamma calibration is important to produce accurate contrasts on a display; it must be performed for each display device.

With a photometer, one can simply display a large uniform square at different pixel graylevels (e.g., from 0 to 255), measure the luminance of the center of the square, and fit the results with Equation 1 to obtain γ.

The psychophysical procedure involves creating, in one area of the viewing surface, as uniform as possible a mixture (in space and time) of equal quantities of zero- and full-intensity pixels and determining the graylevel value of pixels in an adjacent homogeneous area (in which all the pixels have the same intensity) that produces a psychophysical match to the luminance of the mixed-pixel area. The first match determines the graylevel value for 1/2 of maximum luminance. A mixture of pixels with luminance intensities 1/2 and 1 is used to determine the 3⁄4 value, and this procedure is repeated until seven values from 1⁄8 to 7⁄8 have been determined. These matches are repeated, and various checks for consistency are made, such as verifying that a mixture of 3⁄4 and 1⁄4 matches 1/2. In this procedure, both the B and R levels of the homogeneous area are adjusted, and U=B+R/BRratio. The results can be fit with Equation 1 to obtain γ.

After we obtain values of L_min, L_max and γ, we can set the background luminance L₀ = (L_max +L_min)/2, The relationship between pixel contrast c(U) and pixel gray level U is:

We can rewrite the equation to solve for the required gray level for a given contrast:

where U is normally a non-integer number. The RGB value of the corresponding pixel is: R=BRratio*[U-int(U)], G=0, B=int(U), where int() takes the integer value of its input.

1.5. Demo of a sine wave grating

The effect of increased grayscale resolution can be shown by comparing two sine wave gratings. The grating on the left uses the full capacity of the VideoSwitcher with each pixel intensity specified by the appropriate R and B values. The grating on the right is the same as the left one except that only the rounded U is used as B value of each pixel, mimicking the output of an 8-bit display device.

2. RTbox

2.1. Connect the RTbox to a computer

Connect the RTbox (Figure 4) to a USB port of a computer with the provided cable. If needed, download the driver of the device from http://www.ftdichip.com/Drivers/VCP.htm. Install the driver following the instructions on the website.

2.2. Use the RTbox to measure response time

The RTbox can be used in two different ways.

If the user's program can return reliable stimulus onset times, such as the time returned by Screen('Flip') and PsychPortAudio in Psychtoolbox [8], the RTbox can be used just like a keyboard. Because stimulus onset times and RTbox button times are based the clocks of the host computer and the RTbox, respectively, we must synchronize the two clocks. This is done with a function call RTBox('clear') prior to each stimulus onset to clear unwanted events in the buffer. The response time can be simply measured as t_resp= t - t_onset, where tonset is the stimulus returned from the user program, and t is the button time returned by the function call [t, button]=RTBox(timeout). This is because the RTBox driver automatically converts the button time from the RTbox time to host computer time.

If the user's program can't return reliable stimulus onset times, trigger signals indicating stimulus onsets must be provided to the RTbox. One solution is to use a light sensor to detect the onset of the stimulus or a light patch that is synchronized to the stimulus. The output of the light sensor is connected to the light port of the RTbox. A second solution is to use the trigger output of the VideoSwitcher. The trigger can be directly connected to the pulse port of the RTbox. A third solution is to connect the audio signal of the host computer to the sound port of the RTbox.

To generate a stimulus onset trigger from the VideoSwitcher, we "turn on" the green channel of the pixels in the central portion of the first frame of the visual stimulus. Because it does not contribute to the resistor network, the signal in the green channel is invisible to the subject. A function call [t, button]=RTBox('pulse', timeout) returns the response time, where 'pulse' instructs the RTbox to compute the response time relative to the trigger. The same function call is used for sound trigger. For light sensor triggers, the function call is [t, button]=RTBox('light', timeout).

3. Integrate VideoSwitcher and RTbox

We have programmed a demonstration program that uses the VideoSwitcher to control the display, the RTbox to collect accurate response time, and the RTbox to send event code and responses to an ERP system. The program can be downloaded from http://lobes.usc.edu/videoswitcher/VideoSwitcherRTBoxERP_demo.m.

4. Representative Results

Figure 1. Picture of a video switcher. It has VGA input and output ports (both female), a button to switch between two display modes, and a trigger output port.

Figure 2. Pictures of VGA (A) and DVI-I (B) ports and a DVI-I to VGA adaptor (C).

Figure 3. The schematic of a video switcher. The multiplexer has two input modes, controlled by a voltage level. When it is in the mode shown in the schematic, the RGB channels of a monitor receive the same signal, so the display is in grayscale mode. The switching voltage is generated by a CPLD chip, controlled by signals in the G-channel and the switch button. The trigger is also controlled by the input in the G-channel.

Figure 4. Picture of an RTbox. It has a USB-B port, a light input port, a pulse/sound input port, an external button input port, and a TTL output port.

Discussion

The video switcher provides a compact, inexpensive, and easy-to-use method to generate high luminance resolution displays using conventional computer graphics cards and analog color monitors. It is connected between the analog outputs of the graphics card and the monitor. It can also switch between monochromatic and chromatic display modes, providing additional convenience. Its trigger channel allows researchers to synchronize other equipment to visual displays.

The RTbox is a compact, inexpensive, and easy-to-use hardware/software solution for accurate response time measurement. If external triggers synchronized with stimulus onsets are available, RT measurements are totally independent of the host computer. For visual stimuli, we designed a built-in light port to receive triggers generated by photodiodes. External triggers can also be used to calibrate the stimulus onset time returned by user program. If the user program can provide precise stimulus onset time, accurate synchronization between the host computer and the RTbox can be achieved by the RTbox driver, so the RTbox can be used to obtain accurate RT measurements without any external trigger. It can also receive external button signals, such as outputs from an MRI compatible button box, and transmit all input signals and event markers to other devices, such as an EEG system. It can serve as an interface for simultaneous EEG/MRI recording systems.