パイロット テスト

Before administering any experiment it is beneficial to perform a smaller preliminary analysis, or a pilot study, to test and refine the experimental procedure. Pilot testing is the assessment of one aspect of a pilot study.

In psychological experiments, researchers have the challenge of creating experiences for participants that are consistent and authentic; however, there are many ways to manipulate any one variable. For example, if you want participants to feel sad you can have them think of their own sad memory, watch a sad video, or read a sad story.

Researchers must find the best way to modify a psychological construct in order to produce the most effective manipulation. Often, before running the main study, researchers will “pilot test” their manipulations to check their effectiveness.

This video demonstrates how to design and execute a pilot test with an example that seeks to identify the best sound to play during a difficult task to maximally manipulate acute stress.

Before performing a pilot test, proper experimental design is essential. This process includes creation of operational definitions, or clear descriptions of specific concepts.

In this experiment, several stressful sounds are being piloted as acute stress inducers for the participant while they complete a difficult task.

For the purposes of this experiment, the operational definition of a stressful sound is any noise that creates a feeling of tension, immediacy, or anxiety within participants.

This will be manipulated through three different sounds—static, ticking clock, and crying baby—which are subjected to three experimental groups.

Next an operational definition is created for acute stress. For the purposes of this experiment, acute stress is defined as the stress or feeling of tension and strain resulting from recent demands or pressures.

Here, pressure is applied through administration of complex math problems.

In order to measure acute stress accurately, participants will be asked about their own stress levels using a straightforward question on a survey.

The item “stressed” appears embedded within several other distractor items and is rated on a 1 to 7 scale.

Distractor items are not related to the present study but are included to make the true purpose of the study less obvious.

From this scale, the experimenter will evaluate the different sounds to find which one most consistently results in the highest level of acute stress in the participant.

To conduct the study, first meet the participant at the lab. Provide the participant with informed consent. This is a brief description of the research, a sense of the procedure, an indication of potential risks and benefits, the freedom of withdrawal at any time, and a manner to get help if they experience discomfort.

To run the condition, tell the participant that they will receive a series of math problems that should be easy to solve, and they are to complete as many as possible in the 2 min time limit. Tell the participant to try to concentrate and ignore any sounds they may hear.

Next, give the participant the math problem sheet. Start the timer and immediately play the sound being tested. Now, indicate to the participant that they may start.

Following the math task, measure the dependent variable of acute stress by giving the participant a form that asks them to indicate how they currently feel.

After the experiment, debrief the participant by telling them the nature of the study. Specifically, explain that to determine what type of sound led to the most stress, three different groups worked on the same math problems for 2 min while listening to one of three sounds.

Further debrief the participant by explaining that deception was used in this experiment by indicating that the study was about concentration and that the math problems were easy.

Explain that in both cases, deception was necessary to capture the participants’ natural reaction. Divulging that the study was actually about inducing stress and that difficult math problems would be used would likely increase initial stress levels.

In a large-scale pilot study, 40 participants were tested for each sound used. This number of participants was necessary to ensure that the results are reliable and reflective of the greater population.

After collecting data from 120 people, an analysis of variance comparing the static, ticking clock, and crying baby conditions was performed to see how they influenced stress level. The numbers presented here reflect the mean reported stress levels that participants indicated on the 1 to 7 scale for the “stressed” item in each condition.

As seen in the figure, the crying baby condition reported the most stress as hypothesized.

Now that you are familiar with how researchers conduct pilot experiments, let’s take a look at how these data are incorporated into future studies.

For example, based on the results of this pilot study, researchers expanded upon the type of math problems given—easy or difficult—to induce different levels of stress and then examined the effects on relationship behaviors.

The results indicated that those under more stress were more likely to pay attention to alternate partners and were less likely to give their own partner compliments.

In another study, pilot data were collected to test the feasibility of using a multimedia mobile phone program for youth smoking cessation intervention.

As the program’s content was found to be appropriate and technically easy to distribute to the target audience, the study was designed for a larger population of youth and revealed its benefits to help youth quit smoking.

You’ve just watched JoVE’s introduction to pilot testing. Now you should have a good understanding of how to modify a variable to determine the best manipulation for an experiment. The video demonstrated how to conduct a pilot test, as well as how to evaluate the results, and concluded with examples of larger studies informed by pilot tests.