From the Natural Selection of Bad Science to the Intelligent Design of Better Science

Practical Methodological Reform Needs Good Theory. Will M.  Gervais. Perspectives on Psychological Science, January 29, 2021. https://doi.org/10.1177/1745691620977471

Abstract: In the face of unreplicable results, statistical anomalies, and outright fraud, introspection and changes in the psychological sciences have taken root. Vibrant reform and metascience movements have emerged. These are exciting developments and may point toward practical improvements in the future. Yet there is nothing so practical as good theory. This article outlines aspects of reform and metascience in psychology that are ripe for an injection of theory, including a lot of excellent and overlooked theoretical work from different disciplines. I review established frameworks that model the process of scientific discovery, the types of scientific networks that we ought to aspire to, and the processes by which problematic norms and institutions might evolve, focusing especially on modeling from the philosophy of science and cultural evolution. We have unwittingly evolved a toxic scientific ecosystem; existing interdisciplinary theory may help us intelligently design a better one.

Keywords: metascience, methods reform, philosophy of science, cultural evolution, modeling, norms, diversity and inclusion, theory

From the Natural Selection of Bad Science to the Intelligent Design of Better Science

We’re all making it up as we go along, to the best of our ability, hoping not to make a mess of everything. Under the circumstances, I think a little modesty in our scientific and statistical claims would be in order, no?

— Danielle Navarro (2019, p. 11)

Our current scientific ecosystem is unhealthy. Cheap, low-effort, unreliable science can spread at the expense of slower, more reliable work (Bakker et al., 2012; Gervais et al., 2015; Smaldino & McElreath, 2016). Fortunately, we have the opportunity to clean up the mess we’ve evolved.

We are an evolved species, genetically and culturally. But we also have the intelligence to guide the evolution of our cultures. An understanding of evolutionary forces can shed light on how societal ills are maintained, inside (Smaldino & McElreath, 2016) and outside (O’Connor, 2019b) of science. But knowledge is power! Once we understand the forces that created problems, solutions might be more possible. Guided cultural evolution can be practiced at various levels of social organization (Atkins et al., 2019; Bicchieri, 2016; Wilson, 2011) and is well worth attempting in science (O’Connor, 2019a; Smaldino, 2019; Stewart & Plotkin, 2020).

A vibrant methods reform and metascience community has sprung up in psychology. We seek to reshape the scientific ecology that we have (likely unwittingly) allowed to evolve, an ecosystem in which the factors driving individual success erode the collective enterprise of science. Our metascience and reform movement is characterized by lofty goals and a tireless passion for science. I argue that it can maximize its odds of success by drawing on all available theoretical tools, especially turning to tools that have originated and fermented in areas adjacent to psychology, including philosophy of science and cultural evolution.

Given the cultural evolutionary forces that drive the spread of substandard science (Smaldino & McElreath, 2016), it only makes sense to turn to core evolutionary principles to turn the tide and intentionally evolve or design a better scientific ecosystem. This endeavor, ultimately, is a project of guided cultural evolution (Atkins et al., 2019; Bicchieri, 2016; Wilson, 2011). So let us turn to the best available theories to sharpen our metascientific projects, tune our scientific aspirations, and change the norms and institutions we have inherited.

Theory gives us a clue how to proceed. Theory can spur the evolution of better science in domains in which technical, methodological, and statistical tweaks will likely prove insufficient (O’Connor & Weatherall, 2020; Smaldino, 2019; Stewart & Plotkin, 2020; Szollosi et al., 2019; van Rooij, 2019). Theory can help us choose and interpret replication projects (Field et al., 2019). It can help us hone our statistical intuitions about what replication rates are or ought to be. Theory can help us set goals for reform of the field to maximize the scientific desiderata we most value (Devezer et al., 2019). It can make our forensic assays of the field more efficient and meaningful (Field et al., 2019). Theory can even give us hints as to what cues we may (even inadvertently) be sending observers, perhaps undermining the types of communities most likely to actually solve the practical challenges science faces today (O’Connor, 2019a; O’Connor & Weatherall, 2018; Zollman, 2010).

We have passively evolved a toxic scientific ecosystem. Perhaps by embracing relevant theory, including work from outside psychology, we can intelligently design a healthier one for future generations of scientists.