The non-effects of repeated exposure to the Cognitive Reflection Test: We do not improve scores

The non-effects of repeated exposure to the Cognitive Reflection Test. Andrew Meyer, Elizabeth Zhou, Shane Frederick. Judgment and Decision Making, Vol. 13, No. 3, May 2018, pp. 246-259. http://journal.sjdm.org/18/18228a/jdm18228a.html

Abstract: We estimate the effects of repeated exposure to the Cognitive Reflection Test (CRT) by examining 14,053 MTurk subjects who took the test up to 25 times. In contrast with inferences drawn from self-reported prior exposure to the CRT, we find that prior exposure usually fails to improve scores. On average, respondents get only 0.024 additional items correct per exposure, and this small increase is driven entirely by the minority of subjects who continue to spend time reflecting on the items. Moreover, later scores retain the predictive validity of earlier scores, even when they differ, because initial success and later improvement appear to measure the same thing.

Keywords: Cognitive Reflection Test, repeated testing