Publication bias and the canonization of false facts (2017)

Publication bias and the canonization of false facts. Silas Boye Nissen, Tali Magidson, Kevin Gross, Carl T Bergstrom. eLife Sciences, 2016, https://elifesciences.org/articles/21451

Abstract: Science is facing a “replication crisis” in which many experimental findings cannot bereplicated and are likely to be false. Does this imply that many scientific facts are false as well? To find out, we explore the process by which a claim becomes fact. We model the community’s confidence in a claim as a Markov process with successive published results shifting the degree of belief. Publication bias in favor of positive findings influences the distribution of published results. We findthat unless a sufficient fraction of negative results are published, false claims frequently can become canonized as fact. Data-dredging, p-hacking, and similar behaviors exacerbate the problem. Should negative results become easier to publish as a claim approaches acceptance as a fact, however, true and false claims would be more readily distinguished. To the degree that the model reflects the real world, there may be serious concerns about the validity of purported facts in some disciplines.

Check also Raising awareness for the replication crisis in clinical psychology by focusing on inconsistencies in psychotherapy research: how much can we rely on published findings from efficacy trials? Michael P. Hengartner. Front. Psychol. 2018 http://www.bipartisanalliance.com/2018/02/psychotherapy-was-marvellous-invention.html

Also Replication in experimental economics: A historical and quantitative approach focused on public good game experiments. Nicolas Vallois, Dorian Jullien. GREDEG Working Paper No. 2017-21. 2017. http://www.bipartisanalliance.com/2018/03/replication-in-experimental-economics.html

And Psychology's Renaissance. Leif D. Nelson, Joseph P. Simmons, and Uri Simonsohn. Annual Review of Psychology, forthcoming. http://www.bipartisanalliance.com/2017/11/the-scientific-practices-of.html