Approximate Bayesian Inference of Parametric Cost Functions in Continuous Decision-Making

Abstract

Bayesian observer and actor models have provided normative explanations for behavior in many perception & action tasks including discrimination tasks, cue combination, and sensorimotor control by attributing behavioral variability and biases to factors such as perceptual and motor uncertainty, prior beliefs, and behavioral costs.
However, it is unclear how to extend these models to more complex tasks such as continuous production and reproduction tasks, because inferring behavioral parameters is often difficult due to analytical intractability. Here, we overcome this limitation by approximating Bayesian actor models using neural networks. Because Bayesian actor models are analytically tractable only for a very limited set of probability distributions, e.g. Gaussians, and cost functions, e.g. quadratic, one typically uses numerical methods. This makes inference of their parameters computationally difficult. To address this, we approximate the optimal actor using a neural network trained on a wide range of different parameter settings. The pre-trained neural network is then used to efficiently perform sampling-based inference of the Bayesian actor model’s parameters with performance gains of up to three orders of magnitude compared to numerical solution methods. We validated our proposed method on synthetic data, showing that recovery of sensorimotor parameters is feasible. Importantly, individual behavioral differences can be attributed to differences in perceptual uncertainty, motor variability, and internal costs.
We finally analyzed real data from a task in which participants had to throw beanbags towards targets at different distances and from a task in which subjects needed to propel puck to different target distances. Behaviorally, subjects differed in how strongly they undershot and overshot different targets and whether they showed a regression to the mean over trials. We could attribute these complex behavioral patterns to changes in priors because of learning and undershoots and overshoots to behavioral costs and motor variability. Taken together, we present a new analysis method applicable to continuous production and reproduction tasks, which remains computationally feasible even for complex cost functions and probability distributions.

Type
This was the thesis for my M.Sc. in Autonomous Systems.
Tobias Niehues
Tobias Niehues
PhD Student in Cognitive Science

My research interests include statistical modeling of human behavior, especially regarding human decision-making and (the interplay of) perception & action.