A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments

This is an efficient method for collecting and analyzing similarity judgments, and it doesn't assume anything about the geometric properties of subjects underlying mental representations. The main advantages of the method are its flexibility and minimization of assumptions about the nature of the perceptual representation. The kinds of stimuli and the complexity of trials can be varied, and a wide range of geometric models can be fit to the data.

This method could be useful to researchers interested in characterizing the mental representations of low and high-level aspects of visual stimuli. Select an experiment to run. Navigate to the word experiment by clicking similarities, then experiments, then word_exp, or to the image experiment by clicking similarities, then experiments, then image_exp.

Finalize the experimental stimuli. If the word experiment is being run, prepare a list of words. And for the image experiment, make a new directory and place all the stimulus images in it.

In the experiments directory, find the configuration file called config. yaml by clicking similarities, then experiments, and then config.yaml. Open the file in a source code editor and update the value of the file variable to the path to the directory containing the stimulus set.

This is where PsychoPy will look for the image stimuli. Create trial configurations by opening the config. yaml file in the analysis directory, then set the value of the path_2_stimulus_list parameter to the path to stimuli.txt.

From the similarities directory, run the script by executing the commands displayed in the window one after the other. This creates a file called trial_conditions. csv in similarities in which each row contains the names of the stimuli appearing in a trial along with their positions.

Break the full set of 222 trials generated into sessions and randomize the trial order by executing the commands displayed in the window. In the typical design, sessions comprise 111 trials, each of which requires approximately one hour to run. When prompted, enter the displayed input parameters.

Rename and save each of the generated files as conditions. csv in its own directory. Copy the conditions.

csv file and paste it into the current directory containing the psyexp file. Open PsychoPy and open the psyexp or py file in the relevant experiments directory. In PsychoPy, click on the green play button to run the experiment.

In the modal pop-up, enter the subject name or ID and session number and click OK to start. Instructions will be displayed at the start of each session. Allow the subject about one hour to complete the task and as the task is self-paced, encourage the subjects to take breaks if needed.

After all the sessions are completed, combine the raw data files and reformat them into a single JSON file for further processing by running preprocess. py in the terminal using the commands visible on the screen. When prompted, enter the requested input parameters including the path to the subject data directory, subject IDs to pre-process the data and the experiment name used to name the output file, then press Enter.

This will create a JSON file in the output directory that combines responses across repeats for each trial. To determine the pair-wise choice probabilities from rank order judgments, go to similarities, then analysis and run describe_data. py in the command line.

When prompted, enter the path to subject data and the list of subjects to run the analysis. This will create three kinds of plots. Generate low-dimensional Euclidean models of the perceptual spaces using the choice probabilities by running model_fitting.

py using the command line displayed on the screen. When prompted, provide the inputs for the directory to the subject-data/preprocessed, the number of stimuli, which will be 37 by default, the number of iterations, the output directory, and the amount of Gaussian noise, which will be 0.18 by default. Visualize the log likelihood of the obtained models and assess their fit by running similarities, analysis, model_fitting_figure.py.

When prompted, input the needed path to the CSV files containing log likelihoods. Visualize the perceptual spaces for each subject and generate scatter plots showing the points from the 5D model projected onto the first two principle components by running the displayed commands. When prompted, enter the input parameters and the path to the NPY file containing the 5D points.

After the script is executed, exit the virtual environment. In the file generated for the word experiment, the first row corresponds to a trial in which eight stimuli appear around the reference stimulus monkey. The rank judgments were decomposed into pair-wise choices.

The distribution of choice probabilities was highly consistent across the subjects. The data clustering near the diagonal in each panel indicates a great deal of consistency in the choice probabilities between the subjects and for the judgements that are not at the extremes. The dominant diagonal indicates that the choice probabilities in the two contexts, including the intermediate choice probabilities between zero and one, are close to identical for each subject.

The log likelihoods are shown relative to the log likelihood of the best model, that is a model that assigns the observed choice probability to each comparison without constraining these probabilities by any geometric consideration. Principle component analysis was performed on the points from the 5D model of the perceptual space where the animals perceived as similar were denoted by points near each other. It is crucial to decide on all parameters when designing an experiment and to set them in the config files before beginning step two.

Keeping careful track of each subject's data is also very important. The method provides a large number of similarity judgment, so a number of analyses could be applied, such as cluster analysis, a focus on context, or modeling with different geometric spaces.