Estimating data corruption by publication selection bias: The presence of an effect in economics decreased from 99.9% to 29.7% after adjusting for the bias: in psychology, 98.9% → 55.7%; in medicine, 38.0% → 27.5%

Footprint of publication selection bias on meta-analyses in medicine, economics, and psychology. František Bartoš, Maximilian Maier, Eric-Jan Wagenmakers, Franziska Nippold, Hristos Doucouliagos, John P. A. Ioannidis, Willem M. Otte, Martina Sladekova, Daniele Fanelli, T.D. Stanley. https://arxiv.org/pdf/2208.12334.pdf

Abstract: Publication selection bias undermines the systematic accumulation of evidence. To assess the extent of this problem, we survey over 26,000 meta-analyses containing more than 800,000 effect size estimates from medicine, economics, and psychology. Our results indicate that meta-analyses in economics are the most severely contaminated by publication selection bias, closely followed by meta-analyses in psychology, whereas meta-analyses in medicine are contaminated the least. The median probability of the presence of an effect in economics decreased from 99.9% to 29.7% after adjusting for publication selection bias. This reduction was slightly lower in psychology (98.9% → 55.7%) and considerably lower in medicine (38.0% → 27.5%). The high prevalence of publication selection bias underscores the importance of adopting better research practices such as preregistration and registered reports.