Evaluating the Replicability of Social Priming Studies: The strongest predictor of replication success was whether or not the replication team included at least one of the authors of the original paper

Mac Giolla, Erik, Simon Karlsson, David A. Neequaye, and Magnus Bergquist. 2022. “Evaluating the Replicability of Social Priming Studies.” PsyArXiv. July 8. doi:10.31234/osf.io/dwg9v

Abstract: To assess the replicability of social priming findings we reviewed the extant close replication attempts in the field. In total, we found 65 close replications, that replicated 46 unique findings. Ninety-four percent of the replications had effect sizes smaller than the effect they replicated, only 18% of the replications reported a significant p-value in the original direction, and the 95% confidence interval of the replication effects included the original effects only 26% of the time. The strongest predictor of replication success was whether or not the replication team included at least one of the authors of the original paper. Twelve of the 16 replications with at least one original author produced a significant effect in the original direction and the meta-analytic average of these studies suggest a significant priming effect (d = 0.33, 95% CI[0.26; 0.65]). In stark contrast, none of the 49 replications by independent research teams produced a significant effect in the original direction and the meta-analytic average was virtually zero (d = 0.001, 95% CI[-0.03; 0.03]). We argue that these results have shifted the burden of proof back onto advocates of social priming. Successful replications from independent research teams will likely be required to convince sceptics that social priming exists at all.

UK: Ethnic Minority Victories Mobilize White Voters

Zonszein, Stephanie, and Guy Grossman. 2022. “Turnout Turnaround: Ethnic Minority Victories Mobilize White Voters.” OSF Preprints. July 5. doi:10.31219/osf.io/w2dg8

Abstract: In many countries, the number of ethnic minority representatives has been steadily increasing. How is such a trend shaping electoral behavior? Past work has generally focused on the political engagement of ethnic minorities as a response to having a co-ethnic on the ballot. In contrast, less attention has been devoted to assessing whether an ethnic minority incumbent shapes the electoral behavior of majority-group members. We argue that increased political representation of minorities can be experienced as an external threat to a historically white dominant political context. This in turn may politically activate white constituents aiming to revert their (perceived) disempowerment. We test this argument employing a novel dataset that characterizes candidates' ethnicity, covering four UK Parliamentary general elections, and a regression discontinuity design of close elections between ethnic minority and majority-group candidates. Comparing constituencies that are otherwise identical, except for being represented by a minority Member of Parliament (MP), we find that an MP's ethnicity matters for electoral participation: turnout in constituencies narrowly represented by an ethnic minority MP is 3.6 percentage points larger than in constituencies narrowly represented by a white MP. Consistent with our argument, we find that such difference in turnout is driven by majority-white constituencies, and that voters in these constituencies choose the party of the minority incumbent’s strongest white opponent. However, this dynamic does not overpower minorities' incumbency advantage, but it contributes to polarizing the electorate along ethnic lines. Our findings have important implications for intergroup relations, political behavior, and recent political dynamics more broadly.

Women are somewhat better at divergent thinking, while men show greater variability, meaning there are more likely to fall in the extremes (& previous research say that this is hardly related to creative performance)

Abdulla Alabbasi, A. M., Thompson, T. L., Runco, M. A., Alansari, L. A., & Ayoub, A. E. A. (2022). Gender differences in creative potential: A meta-analysis of mean differences and variability. Psychology of Aesthetics, Creativity, and the Arts. Jul 2022. https://doi.org/10.1037/aca0000506

Abstract: The current study examined gender differences in divergent thinking (DT) using meta-analyses of mean difference and variation. The main objective of the meta-analysis of mean difference was to resolve contradictory findings in the creativity literature regarding the prevalence of creativity among males or females in creative potential. The meta-analysis of variation aimed to test the greater male variability hypothesis (GMVH) in DT. To test gender differences in means (i.e., Hedges’ g), results from 213 studies (k = 1,251; N = 115,289) were analyzed using a three-level approach. Females slightly outperformed males in DT, g = −.065, 95% CI [−.095, −.034], p = ≤ .001. Three-level multiple regression analyses showed that the mean effect size significantly varied by (a) country, (b) DT subscale, (c) type of task, and (d) ability (gifted vs. nongifted). In the second meta-analysis, the GMVH in creative potential was tested by synthesizing the results of 1,152 effect sizes from 187 studies (k = 1,152; N = 101,328). The results confirmed the existence of greater male variability (GMV) in DT, (InVR) = 1.216, 95% CI [1.14, 1.29], p ≤ .001, indicating 21.6% GMV in DT. Multiple regression analyses explained 29.82% of variability in the mean effect (InVR) at Level-2 (within-studies variance), and 5% of the variability in the mean effect at Level-3 (between-studies variance). The mean difference findings support the gender similarity hypothesis, while variation results tend to support the gender differences hypothesis. Limitations and recommendations for future studies are discussed.

Discussion

Gender Differences in Means

Although seminal review research on gender differences in mean DT scores supported the gender similarity hypothesis (e.g., Baer, 2012; Baer & Kaufman, 2008; Kogan, 1974; Runco et al., 2010), an empirical quantitative investigation was warranted. The current results show a small effect size (Cohen, 1988), favoring females. Interestingly, previous findings indicated that females mostly outperformed males in DT abilities (Thompson et al., 2021), yet males have shown a slight advantage in terms of creative performance (Hora et al., 2021). Such a pattern of findings suggest that females may initially show greater creative potential, but that males are able to apply their potential more fruitfully in terms of achievements.

As expected, the overall mean effect size showed high heterogeneity, requiring moderator analysis. Moderator selection was based on possible sources of variability identified by previous meta-analyses in creativity (Abdulla Alabbasi et al., 2021; Abraham, 2016; Acar & Runco, 2012; Acar, Runco, & Park, 2020; Baer & Kaufman, 2008; Paek et al., 2021; Reiter-Palmon et al., 2019; Runco et al., 2010; Said-Metwaly et al., 2020; Kogan, 1974; Thompson et al., 2021). These included year of publication (Baer, 2012; Mar’i & Karayanni, 1983), cross-cultural comparisons (Shao et al., 2019; Storme et al., 2017), age (Cheung & Lau, 2010; Palmiero et al., 2014), DT test (Runco et al., 2016), DT subscale (Karwowski et al., 2016), type of task (Abdulla Alabbasi et al., 2021; Taylor & Barbot, 2021), and ability (Abdulla Alabbasi et al., 2021; Runco, 1993). The three-level multiple regression analysis showed that Level-2 and Level-3 together explained 61.6% of the total variance. The multiple regression analysis indicated that the mean effect size significantly varied by (a) country, (b) DT subscale, (c) type of task, and (d) ability. The smallest gender difference in DT was observed in Asian countries, while slightly larger differences were observed in the Middle East, the United States, and Canada. This finding is consistent with that of some studies on the differences in intellectual abilities across different cultures (Feingold, 1994; Gray et al., 2019; He et al., 2013); however, there was a small difference in DT between participants in different countries (less than g = .10; −.085–.013).

Regarding DT subscales, the results showed a significantly larger mean effect size for fluency, in favor of females, while the smallest gender difference was observed for originality. This is one of the most interesting and significant findings, given that originality is the central feature of creativity (Acar et al., 2019; Runco & Jaeger, 2012). Females also scored higher than males in the composite DT score. Although the composite score is a useful index, it might be misleading since it offers an incomplete assessment of individual differences in creative potential. For educators, the emphasis should be on originality more than any other DT index, first because it is an essential element in any creative work or behavior (Runco, 2014) and, second, because the current findings showed no significant gender difference in originality, unlike fluency, which may be biased against males.

Another important consideration for educators is the type of task used in DT tests when assessing students’ creative potential (e.g., to identify gifted students). The current finding is consistent with some previous studies reporting that females outperform males in verbal tasks (Abraham, 2016; Halpern et al., 2007). This is not to say that educators (or researchers) should avoid using verbal DT tests; in fact, we believe that both task types elicit unique information about an individual’s creative potential. Our recommendation is that both verbal and figural tasks be used to screen students for special programs (e.g., gifted student programs), as using both mediums seem to capture a fuller spectrum of gender strengths. Finally, the comparison between gifted and nongifted samples showed that both gifted females and nongifted females outperformed gifted males and nongifted males, although the magnitude of the effect size was larger in gifted females than gifted males, supporting some previous giftedness and DT findings (e.g., Abdulla Alabbasi et al., 2021; Bahar & Ozturk, 2018).

Gender Differences in Variability

The GMVH was initially proposed as a possible explanation for greater male superiority in different cognitive domains throughout human history (see Feingold, 1992 for a historical review). Several meta-analytic reviews of GMVH in intellectual abilities supported greater male variability in most of the tested intellectual abilities (Feingold, 1992; Gray et al., 2019; Hedges & Friedman, 1993). For creativity, studies on gender differences in variability have been conducted in both Eastern (He & Wong, 2011; He et al., 2013; Ju et al., 2015; Lau & Cheung, 2015) and Western cultural contexts (Karwowski, Jankowska, Gajda, et al., 2016; Taylor & Barbot, 2021), and one study was conducted with an African sample (Karwowski Jankowska, Gralewski, et al., 2016). These investigations attributed different findings to cultural differences (e.g., He et al., 2013), type of task (e.g., Taylor & Barbot, 2021), and the obtained or reported variance ratio (VR). For instance, whereas He and Wong (2011) reported a VR of 1.62 for the composite score of the test for Creative Thinking-Drawing Production (TCT-DP), He et al. (2013) reported a VR of 1.30, Ju et al. (2015) reported a VR of 1.06, and Karwowski, Jankowska, Gralewski, et al. (2016) reported a VR of 1.82. However, earlier studies were limited in terms of the assessments used (all used the TCT-DP except Lau & Cheung, 2015; Taylor & Barbot, 2021), type of task (all used figural tasks except Lau & Cheung, 2015; Taylor & Barbot, 2021), the capacity for cultural comparisons, and sample size. By retrieving the raw data from 187 studies, we were able to calculate the (InVR) of a sample of 101,328, providing a clearer picture of GMVH in creative potential. Moreover, we were able to test different moderators (see Tables 4 and 5) to explain the high heterogeneity observed in the mean effect size. The mean effect size obtained in the current study was less than most previous studies (except Ju et al., 2015). Note here that a VR between .90 and 1.10 indicates a small effect size, while a VR greater than 1.10 would indicate GMV (Karwowski, Jankowska, Gralewski, et al., 2016; Lau & Cheung, 2015). The major findings from the moderator analysis were: (a) greater male DT variability was observed in verbal tasks (InVR = 1.249); (b) among DT subscales, GMV was observed in the elaboration subscale (InVR = 1.429); and (c) among DT tests, a GMV was observed in Wallach and Kogan’s tests (InVR = 1.316).

First, regarding type of task, the current findings differ from previous meta-analyses on GMVH in other cognitive abilities. For instance, Feingold (1992) reported that males and females did not differ in verbal tests such as short-term memory (STM), abstract reasoning, and perceptual speed, whereas a large male variability was found in mechanical reasoning, for example. Hedges and Nowell (1995) reported a negligible difference between males and females in the VR for vocabulary (VR = 1.00–1.08) and reading comprehension (VR = 1.03–1.16), compared with spatial ability (VR = 1.27) and mechanical reasoning (VR = 1.45–1.74). The current finding (i.e., GMV in verbal DT) is also inconsistent with Lau and Cheung (2015), who concluded that GMV was supported in figural tasks, while not much variability was observed in verbal tasks. Greater male variability in elaboration is one area that deserves further future investigation, given that none of the previous studies on GMVH in creative potential targeted or assessed elaboration. The same is true for differences in variability between DT tests, which was not tested before.

Limitations and Future Directions

The limitations of meta-analyses often originate in the limitations of the primary studies. First, DT tests are not synonymous with creativity. Studies show mixed evidence on DT tests’ predictive validity, with some suggesting test scores are unrelated to real-world creative achievement (Baer, 1993), and others suggesting that they account for up to half of the variance in creative achievement (Plucker, 1999). Nevertheless, DT tests have consistently been the most popular way to measure creativity (Abdulla & Cramond, 2017; Plucker & Makel, 2010), and thus, synthesis of these results continues to be a useful metric of the state of the creativity literature.

The overall sample of the meta-analysis was also limited because creativity research has tended to be conducted more often with youth rather than adults. The mean age of the overall sample was 13.91, and only 25.4% of the included studies consisted of participants above the age of 18 (see Figure 3).

Fig 3

As such, there was an age ceiling that limits the generalizability of these findings. This is important because there is some evidence indicating that DT increases with age (Fusi et al., 2021; Shah & Gustafsson, 2021), at least up to middle-age (until about 40 years-old; Massimiliano, 2015; Reese et al., 2001). The creativity literature overall would benefit from extending data collection to older samples to gain a better understanding of life span creativity.

Additional steps to diversify primary samples would further improve the generalizability of future meta-analyses of DT data. Though the current study attempted to emphasize cultural variability and included some studies in Arabic, the sample still primarily comprised Western, English-speaking participants. Similarly, few to no studies allowed participants to self-identify outside of the female-male binary. Future investigations from more diverse countries, including those speaking different languages, and with data collection that allows for the representation of nonbinary gender identities would provide richer data.

Nudges didn't work with vaccination intentions against COVID-19

Can vaccination intentions against COVID-19 be nudged? Elena Kantorowicz-Reznichenko, Jaroslaw Kantorowicz, Liam Wells. Behavioural Public Policy, July 8 2022. https://doi.org/10.1017/bpp.2022.20

Abstract: Once vaccines against COVID-19 became available in many countries, a new challenge has emerged – how to increase the number of people who vaccinate? Different policies are being considered and implemented, including behaviourally informed interventions (i.e., nudges). In this study, we have experimentally examined two types of nudges on representative samples of two countries – descriptive social norms (Israel) and saliency of either the death experience from COVID-19 or its symptoms (UK). To increase the legitimacy of nudges, we have also examined the effectiveness of transparent nudges, where the goal of the nudge and the reasons of its implementation (expected effectiveness) were disclosed. We did not find evidence that informing people that the vast majority of their country-people intend to vaccinate enhanced vaccination intentions in Israel. We also did not find evidence that making the death experience from COVID-19, or its hard symptoms, salient enhanced vaccination intentions in the UK. Finally, transparent nudges as well did not change the results. We further provide evidence for the reasons why people choose not to vaccinate, and whether different factors such as gender, belief in conspiracy theories, political ideology, and risk perception, play a role in people's intentions to vaccinate or susceptibility to nudges.

Discussion and conclusion

In this article, we aimed to examine a number of soft interventions to increase people's intention to vaccinate. Vaccination is currently considered the main solution to the global pandemic and vaccinating the majority of the population is a key public policy goal. At least in democratic countries, governments do not wish to force people to vaccinate, respecting their freedoms and rights over their bodies. Therefore, other methods to encourage vaccination are necessary. Nudging is one popular method, which has been used in many countries around the world for different public policies (e.g., increasing tax compliance, organ donation, savings). Therefore, it has been also discussed and considered in the context of vaccination against COVID-19.

We have experimentally examined three nudges, in two countries, which at the initial period of vaccination availability appeared to be leading in rates of vaccination. Those countries are also similar in terms of their public being generally supportive of different nudges (Reisch & Sunstein, Reference Reisch and Sunstein2016; Pe'er et al., Reference Pe'er, Feldman, Gamliel, Sahar, Tikotsky, Hod and Schupak2019). General support of nudges was also found in many other countries (e.g., Jung & Mellers, Reference Jung and Mellers2016; Sunstein et al., Reference Sunstein, Reisch and Rauber2018, Reference Sunstein, Reisch and Kaiser2019), thus constituting an instrument that has a potential to direct people's behaviour across different countries. First, we examined the effectiveness of a descriptive social norm on a representative sample of the Israeli (Jewish)Footnote¹⁸ population. In particular, we have utilized findings from psychology indicating that people follow the behaviour of others, in order to encourage vaccination intentions. Overall, the average intention of people to vaccinate was high, which is a promising result. However, the nudge itself did not make a difference. Also, a more ‘legitimate’ nudge which was transparent about its method and goal did not change people's choices.

We have also run an experiment to examine two additional nudges on a representative sample of the population in the UK. This experiment used the saliency nudge, utilizing findings from psychology that making certain factors more salient might affect how people treat probabilities, and in turn, which choices they make. The experiments made either the death experience very vivid and alarming or stressed the symptoms of COVID-19. Also, in the UK already in the baseline people had high intentions to vaccinate on average. The saliency nudges had very limited effect to none at all. In particular, when considering the full sample, none of the nudges changed the choices. Looking at the restricted sample (those participants who passed the manipulation check) showed a very small effect of the transparent death saliency nudge. Even though statistically significant, the small effect does not seem to be promising. In addition, we have examined the choices of different subgroups in the two countries but found no differences in the effectiveness of the nudge.

Our studies were well powered as we have based our sample sizes on a power analysis. Therefore, the null results are unlikely to be the result of lack of statistical power. One explanation might be that nudges are effective when people do not have strong preferences either way. In that case, it is not costly to follow a certain nudge. Some studies suggested that the inability of the nudge to change the behaviour of some people might be derived from a stronger preference of those people against the direction of the nudge (Bronchetti et al., Reference Bronchetti, Dee, Huffman and Magenheim2013; Beshears et al., Reference Beshears, Choi, Laibson, Madrian and Milkman2015; Jachimowicz et al., Reference Jachimowicz, Duncan, Weber and Johnson2019). For example, Bronchetti et al. (Reference Bronchetti, Dee, Huffman and Magenheim2013) raised the possibility that people with lower income are more resistant to default nudges that direct them to allocate money for savings because they already have plans how to use this money.

The context of the new vaccination is sensitive. On the one hand, there is an ongoing (threatening) pandemic with many uncertainties in respect of its long-term effects. On the other hand, the developed vaccine (which at the time of the study was not approved yet) is novel and entails many uncertainties with respect to the long-term effects. People are either more afraid of the former, or more of the latter. Therefore, it is difficult to affect their choices with nudges that target their intuitive system of decision making, instead of the deliberative process of decision making. In other words, it might be necessary to first address people's concerns, before having an effect on their behaviour. Our results seem to support the recently expressed opinion of one of the ‘founders’ of the concept of nudges himself, that nudges might not be a sufficient instrument to enhance vaccination to end the current pandemic,Footnote¹⁹ even though there is evidence that at least using reminders and simplifying the process of vaccination has a positive effect (Dai et al., Reference Dai, Saccardo, Han, Roh, Raja, Vangala, Modi, Pandya, Sloyan and Croymans2021).

Our results demonstrate that many people are willing to vaccinate. But there is also a smaller group which is hesitant. From our investigation of the reasons for this hesitation, it seems that the primary reason is the concern about the side effects. Even at the stage of our experiments it was clear that once the vaccine will be available, there will not be sufficient evidence of their long-term effects. This is due to the urgency in approving this vaccine and saving the world from the pandemic. Therefore, this concern is understandable.

Consequently, to encourage vaccination, governments should invest more in understanding people's concerns and trying to address them. For example, by investing in campaigns where people receive more information on the trade-offs between the uncertain long-term effects of the vaccine, and the uncertain (probably worse) long-term effects of contracting COVID-19. Relying solely on soft interventions such as nudges seems not to be sufficient.

One limitation of this study is that we focus on people's intentions rather than vaccination uptake, thus potentially facing the problem of intention-behaviour gap (Sheeran, Reference Sheeran2002). In the specific context of vaccinations for example, several studies have found that even when people intend to vaccinate, they do not always follow through (e.g., Bronchetti et al., Reference Bronchetti, Dee, Huffman and Magenheim2013; Chang et al., Reference Chang, Jacobson, Shah, Pramanik and Shah2021). Nevertheless, there are many studies demonstrating that an intention to vaccinate is generally a strong predictor of an actual uptake of vaccines (daCosta DiBonaventura & Chapman, Reference daCosta DiBonaventura and Chapman2005; Lehmann et al., Reference Lehmann, Ruiter, Chapman and Kok2014; Fall et al., Reference Fall, Izaute and Chakroun-Baggioni2018; Jensen et al., Reference Jensen, Ayers and Koskan2022). In the specific context of this article, at the time of the study we have found that 63.6% of the Israeli participants and 75.3% of the UK participants (control groups), respectively, either ‘strongly agreed’ or ‘agreed’ that they would vaccinate themselves against COVID-19. Looking at the most recent data to date of vaccination uptake we see that nearly 77 persons per 100 population have taken the first dose and around 69 per 100 are fully vaccinated in Israel. In the UK, almost 78 persons per 100 population have taken the first dose, and nearly 73 in 100 are fully vaccinated.Footnote²⁰ Therefore, the baseline intentions in our studies (which were expressed before the vaccine became available) seem to be overall aligned with the actual rate of vaccination as reported by WHO for the beginning of the year 2022.

Furthermore, even though we do not have a way to directly translate our results from intentions to behaviour, our findings seem to be conservative in this respect. From the studies of the intention-behaviour gap, it seems that the gap is mostly driven by people who intend to act but eventually fail to do so (Sheeran, Reference Sheeran2002).Footnote²¹ Therefore, it is reasonable to expect that if the investigated nudges in this article did not change people's intention, it probably would not change people's behaviour.

A related potential concern is that currently there is more knowledge about the effectiveness of the vaccine and its short-term safety.Footnote²² The fact that uncertainty regarding those two factors were the main reasons for some of our subjects not to intend to vaccinate, might suggest our results would change at this stage. However, the level of uncertainty was similar for all participants in our studies. And yet many indicated an intention to vaccinate. Those who were hesitant or reluctant did not seem on average to change their minds in response to the employed nudges. Whereas the new information may have on itself encouraged people to vaccinate, there is no immediate reason to believe it would influence the level of effectiveness of the nudge. For example, in Israel, even after the vaccines against COVID-19 were made available, and evidence of its efficacy and short-term safety had emerged, the two main concerns of the people who were still hesitant about vaccination remained its effectiveness and safety (e.g., Heller et al., Reference Heller, Chun, Shlomo, Gewirtz-Meydan, Acri, Kulkarni and Grinstein-Weiss2022). Also in the UK, the long-term safety was still a major concern for people who were choosing not to vaccinate themselves (Majeed et al., Reference Majeed, Papaluca and Molokhia2021). However, Majeed et al. (Reference Majeed, Papaluca and Molokhia2021) stated that the emerging data on the benefits of the vaccine also reduce vaccine hesitancy. Therefore, it might be reasonable to assume, that those who are still hesitating at this stage, more than a year after vaccines were introduced and with the current reliable information on the effectiveness and short-term safety of vaccines, are those who hold stronger preference against vaccinating. Consequently, our results might again be viewed as conservative, and suggest that other strategies, which are addressing the persisting concerns are needed, rather than simply using ‘system 1’ nudges to enhance vaccination.

Amygdala damage associated with smaller social network size

Amygdala but not hippocampal damage associated with smaller social network size. Janelle N.Beadle et al. Neuropsychologia, July 8 2022, 108311. https://doi.org/10.1016/j.neuropsychologia.2022.108311

Abstract: Social network size has been associated with complex socio-cognitive processes (e.g., memory, perspective taking). Supporting this idea, recent neuroimaging studies in healthy adults have reported a relationship between social network size and brain volumes in regions related to memory and social cognition (e.g., hippocampus, amygdala). Lesion-deficit studies in neurological patients are rare and have been inconclusive due to differences in participant sampling and measurement. The present study uses a multiple case study approach. We investigated patients with focal damage to the hippocampus and/or amygdala (two neural structures thought to be critical for social networks), and examined the patients’ social network size, loneliness, and life satisfaction relative to a non-injured comparison group. Patients with amygdalar damage had smaller social networks and reported higher levels of loneliness and lower life satisfaction, on average, than comparison participants. Patients with damage to the hippocampus reported more friends than the comparison participants, but did not differ in their ratings of loneliness or life satisfaction. This lesion study offers new evidence that the amygdala is critical for social networks, life satisfaction, and reduced loneliness.

Keywords: Social networksLesion studiesLonelinessAmygdalaHippocampus