Evaluating the Accuracy of Snap Judgments

1. Organize materials.

1. Create videos, which includes previously filmed footage of 10 college instructors. The content of the teaching should cover a broad array of subject areas.
2. For each teacher, identify three separate, 10-s clips. The three clips should be taken from the beginning, middle, and end of class, respectively, and feature the teacher alone in the video frame.
3. Following Latin-square designs, combine the three clips in random order; do this for each teacher. This will result in 30 total video clips.
4. Compile the end-of-semester student evaluations for each of the 10 instructors in the videos. These evaluations are from the actual courses that correspond to the video footage.

2. Participant Recruitment

1. Conduct a power analysis and recruit a sufficient number of participants to watch and rate the video clips.
   1. Women were preferred in the original study because prior research supports the notion that they are better than men at decoding nonverbal behavior.

3. Data Collection

1. Tell participants to provide ratings of the instructor based on overall nonverbal behavior. In particular, ask participants to judge the teacher on 15 teaching-related dimensions (e.g., accepting, active, attentive, dominant, honest, likable, warm, professional, etc.) on a 9-point Likert scale (1= not at all; 9 = very).
   1. Participants should receive no other training.
2. To assess molecular nonverbal behavior (i.e., specific nonverbal actions), have two paid, trained coders watch the same video clips.
3. For each clip, tally the number of nods, headshakes, smiles, laughs, yawns, frowns, biting of the lips, downward gazes, self-touches, fidgets, emphatic gestures, and weak gestures the teachers made.
4. Have the two raters also indicate the teacher's symmetry and body posture.
5. To account for attractiveness effects, also have two coders judge the physical attractiveness of each teacher based on a 5-point Likert scale (1 = not at all; 5 = very) based on a single photo of each teacher taken from the video.
6. Fully debrief participants.

4. Data Analysis

1. Note that the primary outcome variable of interest is teaching effectiveness, which is assessed via ratings made by the teachers' students at the end of the semester.
   1. This includes two items that asked the students to rate the instructor's performance and the quality of the course overall.
2. Convert these ratings to percentages.
   1. The mean percentage of the two evaluation items serves as the primary dependent variable of interest.
3. Analyze the ratings of molar nonverbal behavior for reliability.
4. From these data, compute an overall mean, which represents the molar nonverbal behavior composite score.
   1. Both the composite score and each of the individual 15 ratings are considered as potential predictors of teaching effectiveness.
5. Analyze coders' ratings of molecular nonverbal behaviors for reliability.
6. Analyze ratings of teachers' attractiveness for reliability.