Preregistration... Comparing Dream to Reality: We observed deviations from the plan in all studies, and, more importantly, in all but one study, at least one of these deviations was not fully disclosed

Claesen, Aline, Sara L. B. T. Gomes, Francis Tuerlinckx, and wolf vanpaemel. 2019. “Preregistration: Comparing Dream to Reality.” PsyArXiv. May 9. doi:10.31234/osf.io/d8wex

Abstract: Doing research inevitably involves making numerous decisions that can influence research outcomes in such a way that it leads to overconfidence in statistical conclusions. One proposed method to increase the interpretability of a research finding is preregistration, which involves documenting analytic choices on a public, third-party repository prior to any influence by data. To investigate whether, in psychology, preregistration lives up to that potential, we focused on all articles published in Psychological Science with a preregistered badge between February 2015 and November 2017, and assessed the adherence to their corresponding preregistration plans. We observed deviations from the plan in all studies, and, more importantly, in all but one study, at least one of these deviations was not fully disclosed. We discuss examples and possible explanations, and highlight good practices for preregistering research.