The Self in the Mind’s Eye: Revealing How We Truly See Ourselves Through Reverse Correlation. Lara Maister et al. Psychological Science, November 11, 2021. https://doi.org/10.1177/09567976211018618
Abstract: Is there a way to visually depict the image people “see” of themselves in their minds’ eyes? And if so, what can these mental images tell us about ourselves? We used a computational reverse-correlation technique to explore individuals’ mental “self-portraits” of their faces and body shapes in an unbiased, data-driven way (total N = 116 adults). Self-portraits were similar to individuals’ real faces but, importantly, also contained clues to each person’s self-reported personality traits, which were reliably detected by external observers. Furthermore, people with higher social self-esteem produced more true-to-life self-portraits. Unlike face portraits, body portraits had negligible relationships with individuals’ actual body shape, but as with faces, they were influenced by people’s beliefs and emotions. We show how psychological beliefs and attitudes about oneself bias the perceptual representation of one’s appearance and provide a unique window into the internal mental self-representation—findings that have important implications for mental health and visual culture.
Keywords: self-representation, body, appearance, reverse correlation, personality, self-face, open data
Discussion
We investigated how we see ourselves in our mind’s eye by creating visual images of individual participants’ mental representations of both their faces and their body shapes in a data-driven, unconstrained way, minimizing participant biases and experimenter assumptions. This technique produced rich, holistic, and multidimensional visual representations of the face and body, which we found not only carried accurate information about physical appearance but also provided novel insights into the way in which participants’ thoughts and feelings about themselves can color their self-image.
We observed clear interactions between the physical and psychological aspects of the self: Self-portraits of both the face and the body were significantly related to higher level, more abstract self-beliefs and attitudes. In Experiment 1, representations of one’s facial appearance were influenced by beliefs regarding one’s personality traits; for example, if a participant believed that they were highly extraverted, they also held an internal representation of their face that had exaggerated stereotypically extraverted facial features compared with their true appearance. In Experiment 2, we demonstrated similar results for perceptual representations of body shape: Participants with negative attitudes toward their bodies also held visual representations of their body’s physical appearance as wider and typical peers as slimmer, compared with participants who had more positive attitudes.
Until now, there has been little investigation of the interaction between physical and psychological selves, and most of the work that has been done has focused on the bottom-up effects of multisensory and sensorimotor contingencies on higher-level psychological self-representations (Preston & Ehrsson, 2014). Our work uniquely focuses on self-representations stored in long-term memory to point to a close, interactive relationship between physical and psychological representations of the self, consistent with an interactive hierarchical model of self-representation (as proposed by Sugiura, 2013). Higher level self-beliefs and attitudes may influence the perceptual quality of the self-portraits (via a top-down modulation during the reconstruction of these images; see Kosslyn, 2005), but conversely, the perceptual features of the physical self-representation might also lead to congruent inferences about one’s self-beliefs and attitudes. Indeed, evidence from studies on social perception supports a bidirectional causal relationship for our representations of others (Dotsch et al., 2008; Todorov et al., 2015); therefore, a similar bidirectional relationship with regard to self-representations may also be likely.
Although the results with regard to the relationship between physical and psychological self-representations were similar for faces and bodies, there were interesting differences. Participants’ representations of their facial appearance were clearly related to their real facial characteristics, showing a significant level of self-specificity. Classification studies, both using human participants and simulated using a face-recognition algorithm, confirmed that identity could be correctly classified from the self-portraits at well-above-chance levels. In contrast, participants’ perceptual representations of their bodies were less related to real body characteristics (e.g., actual body size) and were more strongly influenced by affective attitudes toward the self. This is consistent with previous evidence using single-dimension measures of body parts (Ben-Tovim et al., 1990) and brings into question the wide literature attempting to characterize perceptual body representations in eating disorders in terms of overestimation or underestimation biases (for a review, see Mölbert et al., 2017). However, it will be important to replicate the findings of both experiments using larger samples of more diverse participants before drawing conclusions. The generalizability of the present study may be limited. In Experiment 1, only young Caucasian adults were tested, and therefore it is necessary to follow up with studies using a wider range of ethnicities. Furthermore, in Experiment 2, only young adult women were tested, and their body size may have been relatively homogeneous compared with the general population.
Interestingly, individual differences in objective accuracy of the facial self-portraits were correlated with self-esteem, specifically with regard to social confidence. The higher an individual’s social self-esteem, the more objectively accurate their self-portrait was. This raises interesting considerations regarding the causal role of social interaction in the development and maintenance of self-representations. Social interactions are an important source of information about our appearance, via feedback on our appearance and via social comparisons (Cash et al., 1983). Therefore, individuals with higher social self-esteem may have engaged in more frequent, close social interactions and thus received more social input about their appearance, leading to more accurate self-perception. Alternatively, individuals with more accurate perception of their appearance may also have smoother, more reciprocal, and more predictable social relationships, leading to greater social confidence. For example, having an accurate perception of one’s own attractiveness may lead to more successful romantic interactions and a lower chance of being rebuffed by someone poorly matched (see Le Lec et al., 2017), leading to higher social self-esteem. Both of these potential explanations appeal to a long-term relationship between self-esteem and the development of an accurate self-face representation. However, it is important to note that in our study, we assessed state self-esteem rather than trait self-esteem. Although it is likely that state and trait self-esteem measures are highly correlated (e.g., see Heatherton & Polivy, 1991), future research may explore whether this finding holds for more stable aspects of self-esteem.
Our results are consistent with the findings of a very recent study, which also used the reverse-correlation technique to create visual self-face representations (Moon et al., 2020). In this study, links were found between the valence of the self-face representations generated, as rated by external observers, and various self-reported traits. Self-esteem, explicit self-evaluation, and extraversion were found to be linked to more positive or pleasant-appearing self-portraits, and social anxiety was related to more negative or unpleasant-appearing self-portraits. The authors concluded that the valence of self-face representations created in this manner was able to reflect the attitude toward self. In the present study, consistent with Moon et al.’s findings, our results also showed a significant association between self-reported psychological traits and the physical features of the self-face representation. However, our results further refine our understanding of this relationship by demonstrating that self-reported personality traits were not merely linked with the perceptual valence of self-face representations, as in Moon et al.’s study, but that individual personality traits were linked to specific facial configurations in the self-portraits that were recognizable as such by independent raters.
Our study further extends existing knowledge in several key ways. First, although Moon et al. (2020) measured participants’ perceptions of self-similarity with their own self-portraits, no work has yet been done to explore the actual accuracy of self-representations or to provide a well-controlled, unbiased assessment of their links to self-beliefs and attitudes. Here, we confirmed the validity of the reverse-correlation method in self-face representation research, demonstrating that the resulting images contain enough visual information to support recognition using subjective ratings from an independent sample of raters as well as objectively using simulated experiments implementing a face-recognition algorithm. Furthermore, when exploring whether these self-face representations are influenced by higher level self-processing, we controlled for real facial features, which is crucial to avoid confounds and to provide a valid, strict test of our hypothesis. Finally, we extended our investigation to consider not only face representations but also body shapes, which enriched and generalized our findings to lend support to a broader mechanism whereby beliefs and attitudes influence perceptual body representations.
In this study, we used a combination of objective, algorithm-based techniques and subjective personality ratings from human observers to analyze both the self-portraits and real photographs. It is possible that the human ratings of the real photographs may have been informed by superficial features of the faces, such as makeup, facial hair, and grooming habits, despite the participants providing the ratings being instructed to ignore such features. However, it is important to note that the effects of this potential source of information could not explain the key results reported here. Such effects would serve only to increase the correlation found between the personality ratings of participants’ real faces and their self-reported personalities. Importantly, it could not alter the relationship between the personality ratings of the self-portraits and the self-reported personality ratings, which is key for our hypothesis, because superficial features such as facial hair and makeup were not represented in the reverse-correlation images. This issue further reiterates the importance of carefully controlling for participants’ real facial ratings, which we ensured was done in each key analysis.
Both the approach we used to produce the self-portraits and our findings are highly relevant to our understanding of clinical disorders of body image, such as anorexia nervosa and body dysmorphia. Previous studies into these disorders have normally focused on online perception of the body or have used distorted images of the patients’ own bodies as stimuli, which did not allow for unbiased measurement (Smeets et al., 1999). Our approach could be used as a unique, direct method of assessing distortions in visual memory in these patients, allowing us to reveal whether they stem from higher level self-beliefs and attitudes or even a disorder in the link between these attitudes and the physical self-representation. This approach will also allow us to compare the effects of different treatments (e.g., those targeting perceptual distortions and those targeting emotional or cognitive aspects of the disorder) as well as assess the effects of treatment across time.
In conclusion, we present a novel way to visually depict how people see themselves in their mind’s eye and, in doing so, revealed visual clues to people’s deeply held self-beliefs and attitudes. Our mental images of our own appearance are fundamental to our understanding of some of the most severe mental disorders that are clustered under the term of body-image disorders. In addition, at a time when our culture is powered by images at an unprecedented level, and our obsession with our own image is evidenced in our social media use (Storr, 2018), our approach and the novel insights presented here pave the way for future explorations, in a data-driven, unconstrained, and richly detailed way, of how we mentally see ourselves.
Posts
