Does Honesty Require Time? Shalvi, Eldar, and Bereby-Meyer (2012) may have overestimated the true effect of time pressure on cheating and the generality of the effect beyond the original context

Does Honesty Require Time? Two Preregistered Direct Replications of Experiment 2 of Shalvi, Eldar, and Bereby-Meyer (2012). Ine Van der Cruyssen et al. Psychological Science, March 10, 2020. https://doi.org/10.1177/0956797620903716

Abstract: Shalvi, Eldar, and Bereby-Meyer (2012) found across two studies (N = 72 for each) that time pressure increased cheating. These findings suggest that dishonesty comes naturally, whereas honesty requires overcoming the initial tendency to cheat. Although the study’s results were statistically significant, a Bayesian reanalysis indicates that they had low evidential strength. In a direct replication attempt of Shalvi et al.’s Experiment 2, we found that time pressure did not increase cheating, N = 428, point biserial correlation (rpb) = .05, Bayes factor (BF)01 = 16.06. One important deviation from the original procedure, however, was the use of mass testing. In a second direct replication with small groups of participants, we found that time pressure also did not increase cheating, N = 297, rpb = .03, BF01 = 9.59. These findings indicate that the original study may have overestimated the true effect of time pressure on cheating and the generality of the effect beyond the original context.

Keywords: intuition, cheating, lying, honesty, replication, moral decision making, time pressure, open data, open materials, preregistered

General Discussion

What is people’s automatic tendency in a tempting situation? Shalvi et al. (2012) found that time pressure, a straightforward manipulation to spark “thinking fast” over “thinking slow,” provoked more cheating, and they concluded that people’s initial response is to serve their self-interest and cheat. We found no evidence that time pressure increased cheating in the die-roll paradigm. There are three possible reasons why replication studies do not produce the same results as the original study: (a) methodological problems in the replication study, (b) overestimation of the true effect size in the original study, or (c) differences between the studies that moderate the effect (Wicherts, 2018).
The first possibility is that methodological limitations in the replication study produced different results. In our first replication study, participants may not have fully appreciated the financial benefits of cheating. In our second replication study, relying on two test sites and offering the task in two languages may have increased error variance. But even for participants who performed the task in their native language, there was anecdotal support for the absence of a time-pressure effect (BF₀₁ = 2.90).
The second possible explanation is that the original study overestimated the true effect size. The use of between-session rather than within-session randomization in the original study makes the experimenter aware of condition assignment and raises the possibility that the experimenter influenced the results (Rosenthal et al., 1963). Also, a single observation (in this case, a single reported die-roll outcome) per participant is likely to provide for a noisy measure. With low reliability, the results are more likely to vary per sample.
The third possible explanation is that the time-pressure effect on cheating is influenced by the context and that differences between the studies explain the different results. Our replications differed in several ways from the original, the most prominent being the country where the study was run, namely Israel in the original versus The Netherlands in the replications. The difference in test site raises the possibility of cross-cultural differences in intuitive dishonesty. Perceived country corruption, for instance, is related to the amount of cheating in the die-under-the-cup game (Gächter & Schulz, 2016). Then again, the large meta-analysis by Abeler, Nosenzo, & Raymond (2019) found that cheating behavior varies little by country. Still, it seems worthwhile to explore whether the automatic tendency to cheat may vary with culture.
In both our PDRs, people were predominantly honest, and we in fact found no evidence of cheating.⁴ Whereas Shalvi et al. (2012) originally reasoned that “time pressure evokes lying even in settings in which people typically refrain from lying” (p. 1268), our findings point to the possibility that the time-pressure effect is bound to settings that produce more pronounced cheating (e.g., when providing justifications for cheating).
In sum, our findings indicate that the original study by Shalvi et al. (2012) may have overestimated the true effect of time pressure on cheating or the generality of the effect beyond the original context. The vast majority of our participants were honest—even under time pressure. This finding casts doubt on whether people’s intuitive tendency is to cheat and fits better with a preference for honest behavior.