Active-sampling policies & their relation to attention & curiosity: Sampling & search depend on individual preferences over cognitive states, including attitudes towards uncertainty, learning progress & types of information

Towards a neuroscience of active sampling and curiosity. Jacqueline Gottlieb & Pierre-Yves Oudeyer. Nature Reviews Neuroscience, volume 19, pages758–770 (2018), https://www.nature.com/articles/s41583-018-0078-0

Abstract: In natural behaviour, animals actively interrogate their environments using endogenously generated ‘question-and-answer’ strategies. However, in laboratory settings participants typically engage with externally imposed stimuli and tasks, and the mechanisms of active sampling remain poorly understood. We review a nascent neuroscientific literature that examines active-sampling policies and their relation to attention and curiosity. We distinguish between information sampling, in which organisms reduce uncertainty relevant to a familiar task, and information search, in which they investigate in an open-ended fashion to discover new tasks. We review evidence that both sampling and search depend on individual preferences over cognitive states, including attitudes towards uncertainty, learning progress and types of information. We propose that, although these preferences are non-instrumental and can on occasion interfere with external goals, they are important heuristics that allow organisms to cope with the high complexity of both sampling and search, and generate curiosity-driven investigations in large, open environments in which rewards are sparse and ex ante unknown.