Brain Development and Physical Aggression: Regardless of gender, children’s physical aggressiveness peaks at 2-4 years of age but then starts diverging, as girls learn more quickly than boys to suppress such behaviors

Brain Development and Physical Aggression: How a Small Gender Difference Grows into a Violence Problem. Lise Eliot. Current Anthropology, Volume 62, Number S23, February 2021. https://www.journals.uchicago.edu/doi/full/10.1086/711705

Abstract: Of the various behavioral differences between males and females, physical aggression is one of the largest. Regardless of gender, children’s physical aggressiveness peaks between two and four years of age but then starts diverging, as girls learn more quickly than boys to suppress such overt behaviors. By puberty there is a sizable gender difference in physical aggression and violence. Studies in rodents suggest that sex differences in aggression and rough-and-tumble play are mediated by the amygdala, whose volume in males is enhanced by prenatal testosterone. However, efforts to extend this model to humans have shown limited success. This paper takes a critical look at current assumptions about gender differences in aggression and their neural basis. Aggression and empathy are competing impulses that engage much of the same cortico-limbic circuitry and are highly sensitive to social factors and early adversity. Learning, or neuroplasticity, is both a primary cause and key intervention for minimizing male aggression and violence.

Rerouting the Neurodevelopment of Male Violence

This proposed trajectory for the incubation of violence accords with our understanding of brain development and neuroplasticity. Given the potency of early experience, it is likely that an initial biologic tilt, compounded by social factors and environmental stressors, leads to a divergence in neural circuitry and increased risk for violence among the minority of highly aggressive boys. Research on the neurobiology of aggression and violence has uncovered various neural correlates but no clear-cut sex difference in the limbic circuitry that underlies such behaviors. Again, the hub of this circuit is the amygdala, which integrates a wide range of sensory and contextual signals to detect emotion and drive motivated behavior through both autonomic and cortical outputs. The amygdala is about 10% larger in adult men compared to women, but this difference is proportional to overall brain volume and disappears when analyses correct for individual differences in brain or body size (Marwha, Halari, and Eliot 2017). Similarly, boys’ amygdala volume is about 7% larger than girls’ in infancy, but this is proportional to their larger head and brain size at birth. Another developmental difference is that the amygdala reaches its peak volume about 1.5 years later in boys compared to girls at the end of the prepubertal period (Uematsu et al. 2012), which also parallels the sex difference in bodily growth that peaks one to two years earlier in girls.

In principle, males’ larger absolute amygdala volume could contribute to the gender difference in physical aggression and violence. However, studies of adult humans have actually detected an inverse correlation between aggression and amygdala volume (Rosell and Siever 2015). That is, the amygdala is reliably reduced in volume in association with aggression in both sexes—contrary to the findings in rats. The amygdala, in turn, is controlled by the orbital prefrontal cortex (PFC), a largely inhibitory connection that appears weaker in individuals who tend toward high levels of anger expression. Like most of the frontal lobe, the orbital PFC matures quite slowly, so only gradually gains potency at suppressing anger and aggression as children grow.

Early research suggested that women have a proportionately larger orbitofrontal cortex than men (Gur et al. 2002), which was believed to underlie their stronger emotional regulation. However, more recent research using very large samples has failed to confirm this; if anything, the medial orbitofrontal cortex is slightly larger in men, and there is no sex difference in volume of the lateral orbitofrontal cortex (Potvin, Dieumegarde, and Duchesne 2017, 2018; Ritchie et al. 2018). Furthermore, studies of infants from zero to two years of age have failed to detect sex differences at the level of brain structure or connectivity in the amygdala, orbitofrontal cortex, and their associated circuits, suggesting that any such male-female brain differences emerge later in development, coincident with the period of gender socialization and segregation across childhood and adolescence (Geng et al. 2017; Gilmore, Knickmeyer, and Gao 2018).

Other research has focused on neurochemical differences among highly aggressive individuals, which include reduced serotonin levels and greater prevalence of the short-allele form of the serotonin uptake transporter (the 5-HT transporter-linked polymorphic region, or 5-HTTLPR), which results in less efficient serotonin reuptake from the synapse (Rosell and Siever 2015). Importantly, research on both the 5-HTTLPR and the serotonin catabolic enzyme, monoamine oxidase A (MAOA), has identified relationships between low-activity alleles and aggressive or antisocial behavior, but only through interaction with adverse childhood experience (maltreatment). In the case of MAOA, this interaction between short allele (lower serotonin degradation) and maltreatment predicted antisocial behavior in males but trended the opposite way in females, suggesting an early-seeded sex difference in the impact of adversity on the development of antisocial behavior (Byrd and Manuck 2014).

These are just some of the neuroanatomical and neurochemical pathways that participate in violent behavior and are being shaped during the developmental period that normally sees a suppression in overt aggression. Given the high rate of adverse childhood experiences (ACEs) among incarcerated juveniles (Baglivio et al. 2014) and the many links between such adversity and neurobiological dysfunction, it is clear that violence can be one result of neuroplasticity gone awry—a failure to develop the inhibition that increasingly holds aggression in check as children grow. The impact of ACEs on risk-taking, mental health, and violence has become a particular focus of public health efforts, with the goal of identifying and evaluating strategies to counteract their threat to healthy limbic system development (Centers for Disease Control and Prevention, US 2016).

It is clear that such interventions must begin very early, if not a full generation in advance. Antisocial violence is transmitted intergenerationally, and by the time an abused child becomes a parent it is often too late for their own children to escape its shadow. Nonetheless, some interventions are beginning to show promise. Evaluating these is complex, especially when the outcome of interest, violent aggression, is relatively rare and not apparent until decades later. But a few strategies have proven beneficial for reducing child abuse and improving long-term behavior outcomes, mostly involving parent support and education about child development and positive discipline (Tremblay, Vitaro, and Côté 2018).

One program out of Dublin that has been heavily researched is called Preparing for Life. It works with parents from pregnancy through the first four to five years of life using a combination of home visiting, infant massage, and positive parenting education. A large, randomized trial of Preparing for Life (Côté et al. 2018) found benefits of treatment to children’s cognitive development and externalizing behaviors, a common precursor to aggression and violence. Another intervention conducted in the United States, the Nurse-Family Partnership, has been found to reduce child abuse (Chen and Chan 2016) and improve cognitive outcomes, especially for boys (Heckman et al. 2017). Additionally, high-quality preschool has been found to lower the incidence of crime among both boys and girls (García, Heckman, and Ziff 2019). Such interventions cannot end at preschool but must continue when children enroll in elementary and high school. In Canada, a two-year project was developed for seven- to nine-year-olds that included home-based parent training and school-based pairings of at-risk boys with a handful of carefully chosen, prosocial peers. Long-term benefits of this program included a reduction in delinquency and criminal records by age 24, although the intervention did not impact violent crime rates (Tremblay, Vitaro, and Côté 2018).

I conclude with another promising program: a low-cost, holistic intervention developed in Canada and known as Roots of Empathy. At the heart of this program is a human infant, recruited with his or her parent to join an elementary school classroom once per month and serve as the emotional “teacher.” Students cluster around a large blanket and watch as the infant’s abilities and personality unfold month by month. The curriculum is centered on attachment and the importance of a caring, trusted nurturer for every human to grow and thrive physically and emotionally. Most importantly, the children in the class—boys and girls alike—develop a genuine bond and protective affection for their infant teacher. Rigorous assessments of Roots of Empathy have found several benefits in reducing bullying and aggression and increasing empathy and prosocial behaviors within classrooms. Importantly, the benefits are equally great for boys as girls, contrary to teachers’ initial expectations (Connolly et al. 2018).

The Human Brain Is Best Described as Being on a Female/Male Continuum: Evidence from a Neuroimaging Connectivity Study

The Human Brain Is Best Described as Being on a Female/Male Continuum: Evidence from a Neuroimaging Connectivity Study. Yi Zhang et al. Cerebral Cortex, bhaa408; January 20 2021, https://doi.org/10.1093/cercor/bhaa408

Abstract: Psychological androgyny has long been associated with greater cognitive flexibility, adaptive behavior, and better mental health, but whether a similar concept can be defined using neural features remains unknown. Using the neuroimaging data from 9620 participants, we found that global functional connectivity was stronger in the male brain before middle age but became weaker after that, when compared with the female brain, after systematic testing of potentially confounding effects. We defined a brain gender continuum by estimating the likelihood of an observed functional connectivity matrix to represent a male brain. We found that participants mapped at the center of this continuum had fewer internalizing symptoms compared with those at the 2 extreme ends. These findings suggest a novel hypothesis proposing that there exists a neuroimaging concept of androgyny using the brain gender continuum, which may be associated with better mental health in a similar way to psychological androgyny.

Keywords: androgyny, brain functional network, sex difference

Discussion

In the present study, we identified an age-dependent pattern of sex differences in the brain functional architecture using the fMRI data of nearly 10 000 participants from teenagers to older adults, and systematically examined the potentially confounding effects on these findings. Based on the identified sex differences, we trained an SVM classifier that achieved a 77.75% accuracy in an independent test sample. Using the continuous output of this SVM, we constructed a brain gender continuum and defined an androgynous brain to be at the middle of this continuum. Indeed, we showed that the patterns of functional connectivity, at the 2 extreme ends of this brain gender continuum, represented predominantly either more female or male features as compared with the center of the continuum. Finally, we used this brain gender continuum to uncover a U-shaped relationship between the neuroimaging-defined brain gender and mental health, particularly the participants with an androgynous brain indeed had fewer internalizing symptoms.

The age-dependency of the sex differences may be associated with a number of factors such as the behavior, genetics, and hormones. Research has shown that different environmental contexts, experiences, and behaviors, throughout the lifespan may alter the structural and functional architecture of the brain, in addition to modulation by neurotransmitters (Kolb and Gibb 2011). Genetic factors may also have differential expression across the lifespan, for example Deary et al. (2006) have shown different rates of heritability of intelligence across age. In addition, the sex hormones have nonlinear developmental trajectories (Haimov-Kochman and Berger 2014; Mcewen and Milner 2017) which increase during childhood and adolescence (Nottelmann et al. 1987) but decrease during aging (Rosario et al. 2004; Cui et al. 2013). Particularly, testosterone, a sex hormone, has been implicated in the developmental change of the DMN (Nota et al. 2016), and in our study we found that 3 brain regions (i.e., the cingulate cortex, angular cortex, and precuneus) with the most differences in their functional connectivity were all identified within the DMN and these differences were also supported by previous studies using smaller samples at different age groups (Lombardo et al. 2018; Ritchie et al. 2018; Ernst et al. 2019). Furthermore, in the trained SVM, a multivariate classifier, we also found that the DMN contributed the most to the classification accuracy of this model. Our findings suggest that the patterns of functional connectivity in the brain are unlikely to be entirely determined by the sex hormone levels. In the UKB sample, we showed that the greater the number of years since menopause, presumably reflecting decreased estrogen levels, the larger the gender brain continuum score, suggesting a shift towards the male end. However, the effect size of this association was small (r = 0.048). Therefore, while sex hormones influence the brain’s functional connectivity many other factors, including those discussed above, also have an impact.

After systematically testing the potential confounders, we confirmed the findings of sex differences in the brain’s functional connectivity. Based on the differences identified, we trained an SVM classifier and mapped each brain onto a brain gender continuum by using the continuous output of the SVM classifier. Some previous studies using cross-validation within the training samples achieved a high classification accuracy (~90%) (Wang et al. 2012; Luo et al. 2019). However, applying such classifiers to the independent test samples, only moderate classification accuracies could be achieved (~75%) (Satterthwaite et al. 2014; Weis et al. 2019), which were comparable with the classification accuracy of 77.75% achieved in the current study. Compared with the low classification accuracy (i.e., 65.7%) in a previous study using a test sample from a different age group compared with the training sample (Weis et al. 2019), our classifier achieved a better accuracy after regressing out age and its higher order terms from the functional connectivity matrix (77.75%). This result was in support of the finding that the sex difference in brain functional connectivity was age dependent.

The moderate classification accuracy of the multivariate classifier indicated that the brain functional architecture was unlikely to be conceptualized as binary, as is the case with biological sex, but was more likely to be continuously represented on a brain gender spectrum. At the behavioral level, Bem had hypothesized that an androgynous gender role would lead to higher self-esteem and better mental health (Bem 1974), since individuals identifying with androgyny are free to act in both masculine and feminine ways without many constraints of gender appropriateness (Bem 1977). In particular, the androgynous group reported having fewer internalizing symptoms (Pauletti et al. 2017). However, previous studies provided only the behavioral observations, therefore there was a need to understand the neural mechanism of such observations. Our results demonstrated that the participants whose brain functional connectivity mapped onto the androgynous segment of the brain gender continuum had fewer internalizing problems, which is advantageous for mental health. This U-shaped association was seen for both males and females, although it was most prominent in males. These findings may indicate that being more compassionate and sociable (traditionally female traits) could potentially improve self-esteem of men, thereby potentially reducing internalizing problems; but being more aggressive and confrontational (traditionally male traits) might not boost self-esteem of women (Pauletti et al. 2017). Future research should include self-report data on male/female behavioral traits within different contexts, for example work, home and social settings, which could further elucidate the relationship between psychological androgyny and the concept of brain androgyny.

However, the current study also has several limitations. First, no single large dataset exists that contains samples covering the entire lifespan, from infancy to old age. In our study, we first analyzed the large-scale multicenter samples from different age groups, and then validated the findings using a single-center sample covering a wider age range but with a smaller sample size. Across this age range, there will inevitably be many environmental factors which will have changed and may have some influence. Second, although the sex hormones have been implicated in the sex dimorphism of the brain’s functional architecture (Bao and Swaab 2011), we need the lifespan measurements of the sex hormones to further investigate the molecular mechanisms underlying the brain gender continuum.

Can You Ever Be Too Smart for Your Own Good? Comparing Linear and Nonlinear Effects of Cognitive Ability on Life Outcomes

Can You Ever Be Too Smart for Your Own Good? Comparing Linear and Nonlinear Effects of Cognitive Ability on Life Outcomes. Matt I. Brown, Jonathan Wai, Christopher F. Chabris. Perspectives on Psychological Science, March 8, 2021. https://doi.org/10.1177/1745691620964122

Abstract: Despite a long-standing expert consensus about the importance of cognitive ability for life outcomes, contrary views continue to proliferate in scholarly and popular literature. This divergence of beliefs presents an obstacle for evidence-based policymaking and decision-making in a variety of settings. One commonly held idea is that greater cognitive ability does not matter or is actually harmful beyond a certain point (sometimes stated as > 100 or 120 IQ points). We empirically tested these notions using data from four longitudinal, representative cohort studies comprising 48,558 participants in the United States and United Kingdom from 1957 to the present. We found that ability measured in youth has a positive association with most occupational, educational, health, and social outcomes later in life. Most effects were characterized by a moderate to strong linear trend or a practically null effect (mean R2 range = .002–.256). Nearly all nonlinear effects were practically insignificant in magnitude (mean incremental R2 = .001) or were not replicated across cohorts or survey waves. We found no support for any downside to higher ability and no evidence for a threshold beyond which greater scores cease to be beneficial. Thus, greater cognitive ability is generally advantageous—and virtually never detrimental.

Keywords: individual differences, cognition, cognitive ability, intelligence, IQ, curvilinear, 

Estimating the Prevalence of Transparency and Reproducibility-Related Research Practices in Psychology (2014–2017)

Estimating the Prevalence of Transparency and Reproducibility-Related Research Practices in Psychology (2014–2017). Tom E. Hardwicke et al. Perspectives on Psychological Science, March 8, 2021. https://doi.org/10.1177/1745691620979806

Abstract: Psychologists are navigating an unprecedented period of introspection about the credibility and utility of their discipline. Reform initiatives emphasize the benefits of transparency and reproducibility-related research practices; however, adoption across the psychology literature is unknown. Estimating the prevalence of such practices will help to gauge the collective impact of reform initiatives, track progress over time, and calibrate future efforts. To this end, we manually examined a random sample of 250 psychology articles published between 2014 and 2017. Over half of the articles were publicly available (154/237, 65%, 95% confidence interval [CI] = [59%, 71%]); however, sharing of research materials (26/183; 14%, 95% CI = [10%, 19%]), study protocols (0/188; 0%, 95% CI = [0%, 1%]), raw data (4/188; 2%, 95% CI = [1%, 4%]), and analysis scripts (1/188; 1%, 95% CI = [0%, 1%]) was rare. Preregistration was also uncommon (5/188; 3%, 95% CI = [1%, 5%]). Many articles included a funding disclosure statement (142/228; 62%, 95% CI = [56%, 69%]), but conflict-of-interest statements were less common (88/228; 39%, 95% CI = [32%, 45%]). Replication studies were rare (10/188; 5%, 95% CI = [3%, 8%]), and few studies were included in systematic reviews (21/183; 11%, 95% CI = [8%, 16%]) or meta-analyses (12/183; 7%, 95% CI = [4%, 10%]). Overall, the results suggest that transparency and reproducibility-related research practices were far from routine. These findings establish baseline prevalence estimates against which future progress toward increasing the credibility and utility of psychology research can be compared.

Keywords: transparency, reproducibility, meta-research, psychology, open science

Discussion

Our evaluation of transparency and reproducibility-related research practices in a random sample of 250 psychology articles published between 2014 and 2017 shows that, although many articles were publicly available, crucial components of research—protocols, materials, raw data, and analysis scripts—were rarely made publicly available alongside them. Preregistration remained a nascent proposition with minimal adoption. The disclosure of funding sources and conflicts of interest was modest. Replication or evidence synthesis via meta-analysis or systematic review was infrequent (although, admittedly, only a relatively short time had elapsed since the articles had been published). Although there is evidence that some individual methodological reform initiatives have been effective in specific situations (e.g., Hardwicke et al., 2018; Nuijten et al., 2017; for review, see Hardwicke, Serghiou, et al., 2020), the findings of the current study imply that their collective, broader impact on the psychology literature during the examined period was still fairly limited in scope.

For most of the articles (65%) we examined, we could access a publicly available version (open access). This is higher than recent open-access estimates obtained for biomedicine (25%; Wallach et al., 2018) and the social sciences (40%; Hardwicke, Wallach, et al., 2020), as well as a large-scale automated analysis that suggested that 45% of the scientific literature published in 2015 was publicly available (Piwowar et al., 2018). Limiting access to academic publications reduces opportunities for researchers, policymakers, practitioners, and the general public to evaluate and make use of scientific evidence. One step psychologists can take to improve the public availability of their articles is to upload them to the free preprint server PsyArXiv (https://psyarxiv.com/). Uploading a preprint does not preclude publication at most journals (Bourne et al., 2017), although specific policies regarding open access can be checked on the Sherpa/Romeo database (http://sherpa.ac.uk/romeo/index.php).

The reported availability of research materials was modest in the articles we examined (14%), which is comparable to recent estimates in the social sciences (11%; Hardwicke, Wallach, et al., 2020) and lower than in biomedicine (33%; Wallach et al., 2018). Several reportedly available sets of materials were in fact not available because of broken links, an example of the “link-rot” phenomenon that has been observed by others trying to access research resources (Evangelou et al., 2005; Rowhani-Farid & Barnett, 2018). We also did not find any study protocols (an additional document detailing the study methods); however, it is unclear to what extent this results from a difference in norms between, for example, biomedicine (in which prespecified protocols are increasingly promoted; Ioannidis, Greenland, et al., 2014) and psychology (in which there may not be an expectation to provide methodological details in a separate protocol document). We did not examine whether sufficient methodological information was provided in the Method sections of articles, as this would have required domain-specific expertise in the many topics addressed by the articles in our sample. The availability of original research materials (e.g., survey instruments, stimuli, software, videos) and protocols enables the comprehensive evaluation of research (during traditional peer review and beyond; Vazire, 2017) and high-fidelity independent replication attempts (Open Science Collaboration, 2015; Simons, 2014), both of which are important for the verification and systematic accumulation of scientific knowledge (Ioannidis, 2012). Furthermore, reusing materials and protocols reduces waste and enhances efficiency (Chalmers & Glasziou, 2009; Ioannidis, Greenland, et al., 2014). Psychologists can share their materials and protocols online in various third-party repositories that use stable permalinks, such as the Open Science Framework² (OSF; see Klein et al., 2018). One observational study found that when the journal Psychological Science offered authors an open-materials badge there was a subsequent increase in the sharing of materials (Kidwell et al., 2016).

Data-availability statements in the articles we examined were extremely uncommon. This is consistent with accumulating evidence that suggests that the data underlying scientific claims are rarely immediately available (Alsheikh-Ali et al., 2011; Iqbal et al., 2016), although some modest improvement has been observed in recent years in biomedicine (Wallach et al., 2018). Although we did not request data from authors directly, such requests to psychology researchers typically have a modest yield (Vanpaemel et al., 2015; Wicherts et al., 2006). Most data appear to be effectively lost, including for some of the most influential studies in psychology and psychiatry (Hardwicke & Ioannidis, 2018b). Vanpaemel et al. (2015), for example, could not obtain 62% of the 394 data sets they requested from authors of papers published in four American Psychological Association journals in 2012. The sharing of raw data, which is the evidence on which scientists base their claims, enables verification through the independent assessment of analytic or computational reproducibility (Hardwicke, Bohn, et al., 2020; Hardwicke et al., 2018; LeBel et al., 2018) and analytic robustness (Steegen et al., 2016). Data sharing also enhances evidence synthesis, such as through individual participant-level meta-analysis (Tierney et al., 2015), and can facilitate discovery, such as through the merging of data sets and reanalysis with novel techniques (Voytek, 2016). Psychologists can improve data availability by uploading raw data to third-party repositories such as the OSF (Klein et al., 2018). Data sharing must be managed with caution if there are ethical concerns, but such concerns do not always preclude all forms of sharing or necessarily negate ethical motivations for sharing (Meyer, 2017). Furthermore, when data cannot be made available it is always possible to explicitly declare this in research articles and explain the rationale for not sharing (Morey et al., 2016). Journal policies that use badges to encourage data sharing (Kidwell et al., 2016) or mandate data sharing (Hardwicke et al., 2018; Nuijten et al., 2017) have been associated with marked increases in data availability in the journals that adopted them.

Of the articles we examined, only one shared an analysis script, a dearth consistent with assessments in biomedicine (Wallach et al., 2018), the social sciences (Hardwicke, Wallach, et al., 2020), and biostatistics (Rowhani-Farid & Barnett, 2018). Analysis scripts (a step-by-step description of the analysis in the form of computer code or instructions for recreating the analysis in point-and-click software) provide the most veridical documentation of how the raw data were filtered, summarized, and analyzed. Verbal descriptions of analysis procedures are often ambiguous, contain errors, or do not adequately capture sufficient detail to enable analytic reproducibility (Hardwicke, Bohn, et al., 2020; Hardwicke et al., 2018; Stodden et al., 2018). Psychologists can share their analysis scripts on a third-party repository, such as the OSF (Klein et al., 2018), and educational resources are available to help researchers improve the quality of their analysis code (Wilson et al., 2017). Sharing the computational environment in which analysis code successfully runs may also help to promote its longevity and trouble-free transfer to other researchers’ computers (Clyburne-Sherin et al., 2018).

Preregistration, which involves making a time-stamped, read-only record of a study’s rationale, hypotheses, methods, and analysis plan on an independent online repository, was rare in the articles we examined. Preregistration fulfills a number of potential functions (Nosek et al., 2019), including clarifying the distinction between exploratory and confirmatory aspects of research (Kimmelman et al., 2014; Wagenmakers et al., 2012) and enabling the detection and mitigation of questionable research practices such as selective-outcome reporting (Franco et al., 2016; John et al., 2012; Simmons et al., 2011). Preregistration is relatively new to psychology (Nosek et al., 2018, 2019), but similar concepts of registration have a longer history in the context of clinical trials in biomedicine (Dickersin & Rennie, 2012), in which they have become the expected norm (Zarin et al., 2017). However, clinical trials represent only a minority of biomedical research, and estimates suggest that preregistration is rare in biomedicine overall (Iqbal et al., 2016; Wallach et al., 2018). Preregistration is also rare in the social sciences (Hardwicke, Wallach, et al., 2020). There is no doubt that the number of preregistrations (and the related Registered Reports article format) is increasing in psychology (Hardwicke & Ioannidis, 2018a; Nosek et al., 2018); however, our findings suggest that efforts to promote preregistration may not yet have had widespread impact on routine practice. It is important to note that because there is a time lag between registration and study publication, our measures may underestimate adoption. Although norms and standards for preregistration in psychology are still evolving (Nosek et al., 2019), several dedicated registries, such as the OSF, will host preregistrations, and detailed guidance is available (Klein et al., 2018).

Our findings suggest that psychology articles were more likely to include funding statements (62%) and conflict-of-interest statements (39%) than social-science articles in general (31% and 15%, respectively; Hardwicke, Wallach, et al., 2020) but less likely than biomedical articles (69% and 65%, respectively; Wallach et al., 2018). It is possible that these disclosure statements are more common than most other practices we examined because they are often mandated by journals (Nutu et al., 2019). Disclosing funding sources and potential conflicts of interest in research articles helps readers to make informed judgments about the risk of bias (Bekelman et al., 2003; Cristea & Ioannidis, 2018). In the absence of established norms or journal mandates, authors may often assume that such statements are not relevant to them (Chivers, 2019). However, because the absence of a statement is ambiguous, researchers should ideally always include one, even if it is to explicitly declare that there were no funding sources and no potential conflicts of interest.

Of the articles we examined, 5% claimed to be a replication study—slightly higher than a previous estimate in psychology of 1% (Makel et al., 2012) and a similar estimate of 1% in the social sciences (Hardwicke, Wallach, et al., 2020) but comparable to a 5% estimate in biomedicine (Wallach et al. 2018). Only 1% of the articles we examined were cited by another article that claimed to be a replication attempt; of these articles, 11% were included in a systematic review, and 7% were included in a meta-analysis. Replication and evidence synthesis through systematic reviews and meta-analyses help to verify and build on the existing evidence base. However, it is unclear what an ideal frequency of these activities would be because they depend on many factors, such as how often studies are sufficiently similar to be amenable to synthesis methods. Although the current findings suggest that routine replication and evidence synthesis is relatively rare in psychology, many high-profile replication attempts have been conducted in recent years (Open Science Collaboration, 2015; Pashler & Wagenmakers, 2012). In addition, because the articles we examined were published relatively recently, there may be some time lag before relevant replication and evidence-synthesis studies emerge. For example, in biomedicine at least, there is a geometric growth in the number of meta-analyses, and in many fields multiple meta-analyses are often conducted once several studies appear on the same research question (Ioannidis, 2016).

The current study has several caveats and limitations. First, our findings are based on a random sample of 250 articles, and the obtained estimates may not necessarily generalize to specific contexts, such as other disciplines, subfields of psychology, or articles published in particular journals. However, this target sample size was selected to balance informativeness with tractability, and the observed estimates have reasonable precision. Second, although the focus of this study was transparency and reproducibility-related practices, this does not imply that the adoption of these practices is sufficient to promote the goals they are intended to achieve. For example, poorly documented data may not enable analytic reproducibility (Hardwicke, Bohn, et al., 2020; Hardwicke et al., 2018), and inadequately specified preregistrations may not sufficiently constrain researcher degrees of freedom (Claesen et al., 2019; Bakker et al., 2020). Third, we relied only on published information. Direct requests to authors may have yielded additional information; however, as noted earlier, such requests to research psychologists are often unsuccessful (Hardwicke & Ioannidis, 2018a; Vanpaemel et al., 2015; Wicherts et al., 2006). Fourth, a lack of transparency may have been justified in some cases if there were overriding practical, legal, or ethical concerns (Meyer, 2017). However, no constraints of this kind were declared in any of the articles we examined. Last, the study can gauge the prevalence of the assessed practices only during a particular time period. The effect of reform initiatives introduced after the examined time period, such as the founding of the Society for Improving Psychological Science (http://improvingpsych.org), will not be represented in our findings.

The current findings imply the minimal adoption of transparency and reproducibility-related practices in psychology during the examined time period. Although researchers appear to recognize the problems of low credibility and reproducibility (Baker, 2016) and endorse the values of transparency and reproducibility in principle (Anderson et al., 2010), they are often wary of change (Fuchs et al., 2012; Houtkoop et al., 2018) and routinely neglect these principles in practice (Hardwicke, Wallach, et al., 2020; Iqbal et al., 2016; Wallach et al., 2018). There is unlikely to be a single remedy to this situation. A multifaceted approach will likely be required, with iterative evaluation and careful scrutiny of reform initiatives (Hardwicke, Serghiou, et al., 2020). At the educational level, guidance and resources are available to aid researchers (Crüwell et al., 2019; Klein et al., 2018). At the institutional level, there is evidence that funder and journal policies can be effective at fomenting change (Hardwicke et al., 2018; Nuijten et al., 2017), and these initiatives should be translated and disseminated where relevant. Heterogeneous journal policies (Nutu et al., 2019) may currently be disrupting efforts to establish norms and promote better standards in routine practice. The Transparency and Openness Promotion initiative promises to encourage the adoption and standardization of journal policies related to transparency and reproducibility (Nosek et al., 2015), but it remains to be seen how effective this initiative will be in practice. Aligning academic rewards and incentives (e.g., funding awards, publication acceptance, promotion, and tenure) with better research practices may also be instrumental in encouraging wider adoption of these practices (Moher et al., 2018).

The current study is one of several to examine the prevalence of transparency and reproducibility-related research practices across scientific disciplines (Hardwicke, Wallach, et al., 2020; Iqbal et al., 2016; Wallach et al., 2018). Here, we have sketched out some of the topography of psychology’s territory. Additional studies will be required to fill in areas of the map that have yet to be explored and increase the resolution in specific areas (e.g., subfields of psychology). Future studies can also add a temporal dimension by comparing new data with the baseline established here, allowing us to explore the evolution of this landscape over time.