Predictions shape our perception: fMRI and multivoxel pattern analysis show that non-stimulated regions of early visual areas contain information about the conscious perception of an ambiguous visual stimulus

Non-stimulated regions in early visual cortex encode the contents of conscious visual perception. Bianca M. van Kemenade, Gregor Wilbertz, Annalena Müller, Philipp Sterzer. Human Brain Mapping, December 3 2021. https://doi.org/10.1002/hbm.25731

Abstract: Predictions shape our perception. The theory of predictive processing poses that our brains make sense of incoming sensory input by generating predictions, which are sent back from higher to lower levels of the processing hierarchy. These predictions are based on our internal model of the world and enable inferences about the hidden causes of the sensory input data. It has been proposed that conscious perception corresponds to the currently most probable internal model of the world. Accordingly, predictions influencing conscious perception should be fed back from higher to lower levels of the processing hierarchy. Here, we used functional magnetic resonance imaging and multivoxel pattern analysis to show that non-stimulated regions of early visual areas contain information about the conscious perception of an ambiguous visual stimulus. These results indicate that early sensory cortices in the human brain receive predictive feedback signals that reflect the current contents of conscious perception.

4 DISCUSSION

Our findings show that the current perceptual state during bistability can be decoded from fMRI signal patterns not only in stimulated early visual regions, which is in line with previous studies (Haynes & Rees, 2005), but crucially also in non-stimulated retinotopic visual cortex, which did not receive any bottom-up input. This suggests that non-stimulated regions of early visual cortex contain information not only about visual stimulation in the surrounding context, as previously shown (Smith & Muckli, 2010), but even about conscious perception independent of visual stimulation per se. This is in line with current theories that model bistable perception within the framework of predictive processing (Brascamp, Sterzer, Blake, & Knapen, 2018; Hohwy et al., 2008). According to this view, ambiguous stimuli (such as the bistable moving plaids used here) provide equally strong sensory evidence for two different percepts, but the currently dominant percept establishes an implicit prediction regarding the cause of the sensory input. This prediction is thought to stabilize the current perceptual state through feedback from higher to lower hierarchical levels, while sensory evidence for the currently suppressed perceptual interpretation elicits prediction errors that act to destabilize the current percept, eventually leading to a perceptual change (Weilnhammer et al., 2021; Weilnhammer, Stuke, Hesselmann, Sterzer, & Schmack, 2017). Here, we provide evidence supporting the notion of feedback signalling of predictions in bistable perception.

There have been other studies that showed neural activity in visual areas that were not directly stimulated. These include studies on object perception (Williams et al., 2008), feature-based attention (Serences & Boynton, 2007), visual scene perception (Smith & Muckli, 2010), and illusions like the Kanizsa triangle (Kok, Bains, van Mourik, Norris, & de Lange, 2016), apparent motion (Chong, Familiar, & Shim, 2016; Muckli, Kohler, Kriegeskorte, & Singer, 2005), or the bistable Gestalt illusion (Grassi, Zaretskaya, & Bartels, 2017). Our study is in line with this earlier work, which underlines the idea that long-range connections carry feedback signals from higher areas back to early visual cortex. However, it is distinct from these findings in the key aspect that it shows that such feedback signals in non-stimulated visual areas carry information about the subjective interpretation of an ambiguous stimulus, where the physical properties of the stimulus are stable, while the conscious perception of the participant alternates between two alternative interpretations. Bistable motion quartets inducing apparent motion also show activity along the non-stimulated motion path depending on conscious interpretation, but this activity underlies the reconstruction of an illusory percept, that is, of a stimulus that is not actually there. In our study, the activity reflected feedback signals about a stimulus that was always physically present, but was interpreted in different ways over time. As such, our results do not only support the general idea that predictions are sent back to early visual cortex, but importantly that they are involved in the subjective interpretation of an ambiguous stimulus.

Our univariate results showed significantly more activation for patterns than components in non-stimulated early visual areas. Increased activation for patterns in early visual cortex has been reported in previous studies as well (Grassi et al., 2018; Wilbertz, Ketkar, Guggenmos, & Sterzer, 2018). We observed this pattern only in non-stimulated areas, which resembles the results by Grassi et al. (2017) that a global Gestalt percept induced more activity in the illusory percept regions in early visual cortex than a local Gestalt percept. The fact that we observed this effect in non-stimulated regions only seems to support the hypothesis that it is driven by feedback mechanisms, as indicated by findings from Kok et al. (2016) who found enhanced activity for illusory percepts only in deep cortical layers that process feedback signals. As such, our univariate results support our multivariate results. Since it has been shown that attentional mechanisms can also drive perceptual effects in non-stimulated areas (Serences & Boynton, 2007), it is possible that attention to the current percept might have contributed to the results. However, since we found opposite univariate patterns in early visual cortex (more activity for pattern percepts) and area hMT+/V5 (more activity for component percepts), feedback mechanisms seem a more likely explanation. On a similar note, it has been reported that people blink more during pattern perception compared to component perception (Brych, Murali, & Händel, 2021), which could be an alternative explanation for the increased BOLD response in visual cortex (Hupé et al., 2012). However, again the opposite pattern in early visual cortex versus hMT+/V5 seems to rather point at the involvement of feedback mechanisms.

We suggest that the percept-related information that we found in non-stimulated regions of early visual areas most likely arises from feedback signalling that originates from higher-level areas concerned with the computation of component vs. pattern motion perception, such as area hMT+/V5 (Castelo-Branco et al., 2002; Duarte, Costa, Martins, & Castelo-Branco, 2017; Grassi et al., 2018). Research on bistable plaid motion has shown that hMT+/V5 is concerned with the disambiguation of bistable plaids into pattern and component motion (Castelo-Branco et al., 2002), and that it sends information back to early visual cortex during this process (Duarte et al., 2017). Furthermore, effective connectivity analyses have shown that apparent motion induced activation of non-stimulated visual regions along the illusory apparent motion path is associated with enhanced feedback signalling from area hMT+/V5 (Sterzer, Haynes, & Rees, 2006), which has been shown to be causally involved in such apparent motion perception in a later TMS study (Vetter, Grosbras, & Muckli, 2015). Considering these studies, it seems plausible that area hMT+/V5 is also involved in predictive feedback signalling to non-stimulated areas during bistable plaid motion perception, and that our results thus reflect predictive feedback signalling coming from this area. Our significant decoding results in hMT+/V5 support the idea that this area generates the predictions that are sent back to early visual areas during bistable perception, though future studies will have to provide direct causal evidence. There are other potential origins of feedback signalling in bistable plaid perception, as several studies have shown involvement of frontoparietal areas in bistable perception (Brascamp et al., 2018; Grassi et al., 2018; Weilnhammer et al., 2021). Recent evidence suggests that hMT+/V5 might signal perceptual conflict to and receive signals from frontal areas to resolve this conflict, making hMT+/V5 a hub for receiving and relaying feedback signals from and to frontal cortex (Weilnhammer et al., 2021). As our study was focused on visual cortex, we were unable to verify the involvement of areas outside visual cortex. However, our results support the idea of hMT+/V5 as a source of feedback signals to early visual cortex in bistable perception.

In conclusion, our current results provide compelling support for the notion that conscious perception reflects an internal model that generates predictions about the current state of the world, and that these predictions are fed back to the lowest levels of sensory processing to enable inferences regarding the sensory input.