estimation-contact-regions-between-hands-objects-during-human-multi

Estimation of Contact Regions Between Hands and Objects During Human Multi-Digit Grasping

To grasp an object successfully, we must select appropriate contact regions for our hands on the surface of the object. However, identifying such regions ...

Technical University of Darmstadt

The capability of directing gaze to relevant parts in the environment is crucial for our survival. Computational models have proposed quantitative accounts of human gaze selection in a range of visual search tasks. Initially, models suggested that gaze is directed to the locations in a visual scene at which some criterion such as the probability of target location, the reduction of uncertainty or the maximization of reward appear to be maximal. But subsequent studies established, that in some tasks humans instead direct their gaze to locations, such that after the single next look the criterion is expected to become maximal. However, in tasks going beyond a single action, the entire action sequence may determine future rewards thereby necessitating planning beyond a single next gaze shift. While previous empirical studies have suggested that human gaze sequences are planned, quantitative evidence for whether the human visual system is capable of finding optimal eye movement sequences according to probabilistic planning is missing. Here we employ a series of computational models to investigate whether humans are capable of looking ahead more than the next single eye movement. We found clear evidence that subjects' behavior was better explained by the model of a planning observer compared to a myopic, greedy observer, which selects only a single saccade at a time. In particular, the location of our subjects' first fixation differed depending on the stimulus and the time available for the search, which was well predicted quantitatively by a probabilistic planning model. Overall, our results are the first evidence that the human visual system's gaze selection agrees with optimal planning under uncertainty.

Multi-step planning of eye movements in visual search.

While interacting with objects during every-day activities, e.g. when sliding a glass on a counter top, people obtain constant feedback whether they are acting in accordance with physical laws. However, classical research on intuitive physics has revealed that people's judgements systematically deviate from predictions of Newtonian physics. Recent research has explained at least some of these deviations not as consequence of misconceptions about physics but instead as the consequence of the probabilistic interaction between inevitable perceptual uncertainties and prior beliefs. How intuitive physical reasoning relates to visuomotor actions is much less known. Here, we present an experiment in which participants had to slide pucks under the influence of naturalistic friction in a simulated virtual environment. The puck was controlled by the duration of a button press, which needed to be scaled linearly with the puck's mass and with the square-root of initial distance to reach a target. Over four phases of the experiment, uncertainties were manipulated by altering the availability of sensory feedback and providing different degrees of knowledge about the physical properties of pucks. A hierarchical Bayesian model of the visuomotor interaction task incorporating perceptual uncertainty and press-time variability found substantial evidence that subjects adjusted their button-presses so that the sliding was in accordance with Newtonian physics. After observing collisions between pucks, which were analyzed with a hierarchical Bayesian model of the perceptual observation task, subjects transferred the relative masses inferred perceptually to adjust subsequent sliding actions. Crucial in the modeling was the inclusion of a cost function, which quantitatively captures participants' implicit sensitivity to errors due to their motor variability. Taken together, in the present experiment we find evidence that our participants transferred their intuitive physical reasoning to a subsequent visuomotor control task consistent with Newtonian physics and weighed potential outcomes with a cost functions based on their knowledge about their own variability.

Intuitive physical reasoning about objects' masses transfers to a visuomotor decision task consistent with Newtonian physics.

The efficient coding hypothesis posits that sensory systems are tuned to the regularities of their natural input. The statistics of natural image databases have been the topic of many studies, which have revealed biases in the distribution of orientations that are related to neural representations as well as behavior in psychophysical tasks. However, commonly used natural image databases contain images taken with a camera with a planar image sensor and limited field of view. Thus, these images do not incorporate the physical properties of the visual system and its active use reflecting body and eye movements. Here, we investigate quantitatively, whether the active use of the visual system influences image statistics across the visual field by simulating visual behaviors in an avatar in a naturalistic virtual environment. Images with a field of view of 120° were generated during exploration of a virtual forest environment both for a human and cat avatar. The physical properties of the visual system were taken into account by projecting the images onto idealized retinas according to models of the eyes' geometrical optics. Crucially, different active gaze behaviors were simulated to obtain image ensembles that allow investigating the consequences of active visual behaviors on the statistics of the input to the visual system. In the central visual field, the statistics of the virtual images matched photographic images regarding their power spectra and a bias in edge orientations toward cardinal directions. At larger eccentricities, the cardinal bias was superimposed with a gradually increasing radial bias. The strength of this effect depends on the active visual behavior and the physical properties of the eye. There were also significant differences between the upper and lower visual field, which became stronger depending on how the environment was actively sampled. Taken together, the results show that quantitatively relating natural image statistics to neural representations and psychophysical behavior requires not only to take the structure of the environment into account, but also the physical properties of the visual system, and its active use in behavior.

Looking for Image Statistics: Active Vision With Avatars in a Naturalistic Virtual Environment.

Which strategy people use to guide locomotor interception remains unclear despite considerable research and the importance of an answer with ramification into the heuristics and biases debate. Because the constant bearing (CB) strategy corresponds to the target-heading (CTH) strategy with an additional constraint, these two strategies can be confounded experimentally. But, the two strategies are distinct in the information they require: while the CTH strategy only requires access to the relative angle between the direction of motion and the target, the CB strategy requires access to a stable allocentric reference frame. Here, we manipulated the visual information about allocentric reference frames in three virtual environments and asked participants to steer a car to intercept a moving target. Participants' interception paths showed different degrees of curvature and their target-heading angles were approximately constant, consistent with the CTH strategy. By contrast, the target's bearing angle continuously changed in all participants except one. This particular participant produced linear interception paths with little change in the target's bearing angle, seemingly consistent with both strategies. This participant continued this pattern of steering even in the environment without any visual information about allocentric reference frames. Therefore, this pattern of steering is attributed to the CTH strategy rather than the CB strategy. The overall results add important evidence for the conclusion that locomotor interception is better accounted for by the CTH strategy and that experimentally observing a straight interception trajectory with a CB angle is not sufficient evidence for the CB strategy.

How do people steer a car to intercept a moving target: Interceptions in different environments point to one strategy.

Psychophysical methods are a cornerstone of psychology, cognitive science, and neuroscience where they have been used to quantify behavior and its neural correlates for a vast range of mental phenomena. Their power derives from the combination of controlled experiments and rigorous analysis through signal detection theory. Unfortunately, they require many tedious trials and preferably highly trained participants. A recently developed approach, continuous psychophysics, promises to transform the field by abandoning the rigid trial structure involving binary responses and replacing it with continuous behavioral adjustments to dynamic stimuli. However, what has precluded wide adoption of this approach is that current analysis methods do not account for the additional variability introduced by the motor component of the task and therefore recover perceptual thresholds that are larger compared to equivalent traditional psychophysical experiments. Here, we introduce a computational analysis framework for continuous psychophysics based on Bayesian inverse optimal control. We show via simulations and previously published data that this not only recovers the perceptual thresholds but additionally estimates subjects' action variability, internal behavioral costs, and subjective beliefs about the experimental stimulus dynamics. Taken together, we provide further evidence for the importance of including acting uncertainties, subjective beliefs, and, crucially, the intrinsic costs of behavior, even in experiments seemingly only investigating perception.

Article	Year
Multi-step planning of eye movements in visual search. Scientific reports\| PubMed ID: 30644423	2019
Intuitive physical reasoning about objects' masses transfers to a visuomotor decision task consistent with Newtonian physics. PLoS computational biology\| PubMed ID: 33075051	2020
Looking for Image Statistics: Active Vision With Avatars in a Naturalistic Virtual Environment. Frontiers in psychology\| PubMed ID: 33692732	2021
How do people steer a car to intercept a moving target: Interceptions in different environments point to one strategy. Quarterly journal of experimental psychology (2006)\| PubMed ID: 33749396	2021
Putting perception into action with inverse optimal control for continuous psychophysics. eLife\| PubMed ID: 36173094	2022

Constantin A. Rothkopf

Biography

JoVE Journal Publications

Other Publications