Evaluating the Replicability of Social Priming Studies: The strongest predictor of replication success was whether or not the replication team included at least one of the authors of the original paper

Mac Giolla, Erik, Simon Karlsson, David A. Neequaye, and Magnus Bergquist. 2022. “Evaluating the Replicability of Social Priming Studies.” PsyArXiv. July 8. doi:10.31234/osf.io/dwg9v

Abstract: To assess the replicability of social priming findings we reviewed the extant close replication attempts in the field. In total, we found 65 close replications, that replicated 46 unique findings. Ninety-four percent of the replications had effect sizes smaller than the effect they replicated, only 18% of the replications reported a significant p-value in the original direction, and the 95% confidence interval of the replication effects included the original effects only 26% of the time. The strongest predictor of replication success was whether or not the replication team included at least one of the authors of the original paper. Twelve of the 16 replications with at least one original author produced a significant effect in the original direction and the meta-analytic average of these studies suggest a significant priming effect (d = 0.33, 95% CI[0.26; 0.65]). In stark contrast, none of the 49 replications by independent research teams produced a significant effect in the original direction and the meta-analytic average was virtually zero (d = 0.001, 95% CI[-0.03; 0.03]). We argue that these results have shifted the burden of proof back onto advocates of social priming. Successful replications from independent research teams will likely be required to convince sceptics that social priming exists at all.