Small effect: Financial incentives can boost critical thinking skills

Revisiting the effect of incentivization on cognitive reflection: A meta-analysis. Eldad Yechiam, Dana Zeif. Journal of Behavioral Decision Making, May 17 2022. https://doi.org/10.1002/bdm.2286

Abstract: The effect of performance-based incentives on judgment biases is a controversial issue. A recent extensive meta-analysis of Brañas-Garza et al. (2019, https://doi.org/10.1016/j.socec.2019.101455) found no effect of incentives on performance in the cognitive reflection test (CRT), a test commonly used for assessing heuristic versus deliberative judgments. We conducted a meta-analysis of studies that directly compared an incentive versus no-incentive conditions (n = 2836) and found a small positive effect of incentivization (Cohen's d = .21) on CRT performance. In addition, we re-examined the Brañas-Garza et al. (2021) dataset (n = 39,385), taking into consideration not only whether incentivization was used but also the nominal and fiscal size of the incentive. The results show that surprisingly when including only studies that used monetary incentives, the effect of incentivization and incentive size are significant, even when taking into consideration other aspects of the studies. The results thus suggest a small but robust effect of incentives on judgment biases.