A Model for Public Access to Trustworthy and Comprehensive Reporting of Research. Marina Broitman, Harold C. Sox, Jean Slutsky. JAMA, March 28, 2019. doi:10.1001/jama.2019.2807
The Patient-Centered Outcomes Research Institute (PCORI) was authorized by Congress in 2010 to fund comparative clinical effectiveness research. The legislation required the institute to guarantee peer review of all research results and to make those results publicly accessible within 90 days of their receipt, requirements that were the first of their kind for a US-based research funding organization. The authorizing legislation further stipulated that the peer review should assess the scientific integrity of each study and its adherence to the methodological standards established by the PCORI methodology committee.1 The law broadly defined the forms of peer review that would be acceptable.
The PCORI board of governors, after comment from patients, clinicians, professional interest groups, and journal editors, established a peer review process2 requiring all awardees to submit a comprehensive final report for PCORI-based external peer review. After final approval, the institute would post on its website lay and technical abstracts of the report; the complete, approved final report and study protocol; and a summary of the peer review critiques and the authors’ responses to those critiques. This Viewpoint describes the peer review system and the comprehensive report, which are the essential elements of this program, and discusses the problems each raise and the potential benefits.
The PCORI peer review process is similar to the review process at some journals in its use of methods experts and clinician-scientists as external reviewers. However, PCORI peer review is unique in its recruiting of patients, caregivers, health care professionals, and policy makers to critique the relevance and usefulness of the research. The institute engaged a contractor to manage the external peer review process to help ensure its impartiality. Each report receives 4 to 5 external reviews and a comprehensive review from the peer review editor.
The final research report is structured like a journal article, but is 3 to 4 times longer to accommodate a complete account of all study aims, methods, and results. The report also includes an account of the study’s adherence to the PCORI methodology standards1 and a section for patient engagement in research, in which the authors describe their partnership and collaborations with interested groups and individuals (including patients and other health care decision makers) in the development, implementation, and interpretation of the study. Per legislative requirements, the report must describe the study limitations and how the comparative effectiveness of the study interventions differs among various subgroups of study participants.
From October 6, 2016, to February 28, 2019, there were 275 final reports submitted for peer review and 206 had completed all revisions following peer review. As of February 28, 2019, PCORI has posted 177 publicly available lay and technical abstracts based on those reports as well as 41 of the full reports. Posting the study results could jeopardize journal publication; therefore, the institute waits 12 months after completion of peer review to post the final research report on its website unless the main study results have already been published in a journal article.
The institute met its legislative mandates. It established a peer review process and the requirement to write a comprehensive report, and it publishes lay and technical abstracts of the final report within 90 days of completing peer review. The vision of the sponsors of the legislation (ie, prompt public access to all study results following peer review) is achievable but at what cost and with what benefits? It is possible now to describe some of the challenges and speculate about potential benefits.
To assess the effects of requiring a comprehensive report, the institute surveyed 191 investigators about their experiences with final research report preparation and peer review and 52% responded. Some awardees raised complaints about the burden of writing a comprehensive final report and responding to peer review when they are preparing manuscripts to submit to journals. Others reported that writing the final report helped them to identify topics, text, and tables for focused articles. Many did not see an added value from the comprehensive final report, which increased their frustration with the time and effort involved. Survey respondents estimated that their research teams spent between 160 to 255 hours preparing the final report and responding to reviewer critiques.
The peer review process is taking longer than originally expected. Instead of the 4 to 6 months estimated by the PCORI board of governors,2 peer review takes a median of 8 to 9 months from submission to final acceptance. Unlike journals that can quickly reject unsuitable articles, the institute must accept and publicly post all reports. Some reports require many rounds of review and revision, which delays completion of peer review and public release of the results.
These problems are present and real, whereas the most substantial benefits lie in the future. First, the public is likely to benefit from a full report of research even when the results are unlikely to change practice. Because inconclusive studies are the least likely to be published in a scientific journal,3 the published body of evidence is incomplete and is likely biased toward positive results,4 which is a problem for systematic reviewers, guideline developers, and many others who rely on evidence to make decisions, including clinicians and patients. Journal articles and research registries each play a key role in informing the public, but a comprehensive final research report can fill a gap in public reporting by providing an in-depth account of a study, including all results and lessons learned from doing the research, regardless of study success.
Second, study authors may benefit from the institute’s peer review of their final research report before they submit manuscripts to a journal. Researchers can use the critiques by the PCORI peer reviewers to anticipate the concerns of journal reviewers and shape journal manuscripts accordingly. When the final research report for the institute is prepared (after publication of a journal article), authors can include information that journals typically do not publish such as a narrative account of how the study unfolded (including explanations of key decisions) and a full account of the patients and others who partnered with the researchers to conduct the study.
Dual peer review and publication could cause problems, especially when the description of research in a journal article conflicts with the final report. These differences are best handled by transparent reporting of the reasons for the discrepancies. When report authors disagree with the recommendations from the peer reviewers, both sides of the disagreement are captured in the peer review summary.
Third, a comprehensive final report supports the goal of increasing the efficiency of research by providing essentially unlimited space in which to describe how a 3- to 5-year study developed and evolved and to transmit all results and lessons learned to other researchers and funding agencies.5 Posted final research reports that provide all study results, including unpublished secondary or inconclusive results, may provide leads for others to investigate, describe pitfalls to avoid, or describe a line of investigation that led nowhere and should be avoided.
Fourth, the funder that requires a final research report can learn from systematically examining the research that it has supported, especially if reports have been peer reviewed. Ongoing review of its body of funded research could inform many aspects of the work of a funding agency such as the choice of high-priority research topics for funding announcements, instructions to applicants, the evaluation of research proposals, postaward negotiations to strengthen the study, and monitoring research while it is in progress. Peer review of adherence to the PCORI methodology standards has identified several recurring methodological shortcomings that the institute must address.
In the future, these potential benefits of requiring a peer-reviewed, publicly available comprehensive final report will be weighed against the costs. In a few years, it will be possible to determine if these final reports have expanded the reach of each research study, whether they are used in systematic reviews, and how lessons learned from peer review of the final reports have changed the way that the institute does its work. For the present, it may be enough to hope that an openly accessible peer-reviewed comprehensive final research report will help to increase public trust in research. It is possible that the sponsors of the legislation were on to something important when they set the institute on this path. However, the realization of their vision is in its infancy and its putative benefits are largely speculative. There is still much to learn.
Female
ovarian cycles have been thoroughly studied in many species of mammals.
A great deal of research on endocrine mechanisms in relation to
behavioral expression rates throughout different cycle stages has
elucidated both the proximate regulation and the functional significance
of female sexual and reproductive strategies in mammals. Female
sexuality involves complex interactions between neuroendocrine
mechanisms in relation to neurotransmitter activities to modulate
behavior. Such neurophysiological processes are only partially
understood in primates.
In most mammals,
sexual activities are limited to the periovulatory period of the female
cycle. This period is characterized by elevated estrogen concentrations
leading to ovulation, followed by an increase of progesterone secretion
that facilitates the implantation of the egg into the uterus. In
general, sexual activity is correlated with estrogen increase and is
reduced at elevated progesterone levels. The period of the cycle in
which females are sexually receptive and sexually active is termed
estrus.
The best-investigated female sexual
behavior in terms of physiology and neurobiology is the lordosis reflex
in rodents. This is a posture in which females allow male intromission.
Shortly before ovulation, males approach their mating partners and
mount them. This sensory interaction enables the lordosis reflex, which
is controlled by the sex steroids estradiol and progesterone. The
behavioral aspect of this reflex is regulated by subcortical
hypothalamic brain structures such as the ventromedial nucleus and the
periacqueductal gray, where ovarian hormones find acting sites to
facilitate the reflex (Flanagan-Cato 2011; Uphouse 2014).
The regulation of lordosis behavior involves complex neurobiological
circuits and their underlying neurochemistry. As illustrated by Beach (1976),
females advertise their sexual readiness to males with their
attractiveness, followed by proceptivity (behavioral signals to males
such as solicitation to copulate), and then by receptivity to copulatory
behavior with subsequent ejaculation.
These
three aspects of female sexuality are related to hormonal changes
during the female cycle and ultimately serve the process of
reproduction. The Beach paradigm was thought to encompass more or less
all non-primate mammalian species. In primates, instead, it has long
been known that females do not limit their sexual behavior to specific
cycle stages and to the corresponding hormone fluctuations (Dixson 2009). In humans in particular, copulations can occur during all cycle stages (Dixson 2009).
This has led some to believe that human sexuality is completely
emancipated from its biological regulation and entirely dependent on
volitional and cultural factors. It is, however, very unlikely that
humans continued to evolved by natural selection up to the Pleistocene,
but then suddenly stopped after settling and farming (approx.
10,000 years ago). Rather, it is more likely that cultural and genetic
processes mutually interacted throughout all human evolution (Richerson
et al. 2010). Natural selection is still acting on certain traits in contemporary humans (Byars et al. 2009)
and producing adaptations through culture-gene co-evolution. This
process may operate much more quickly than previously thought (Field et
al. 2016).
Although there are some important differences between human female
sexuality and female sexuality in nonhuman animals, the prevalent view
emphasizes continuity rather discontinuity between humans and other
animals, especially other primates. Therefore, in this article, we
highlight the many similarities between humans and nonhuman primates in
hormone secretions, neurophysiological subcortical organization, and
female sexual behavior during the reproductive cycle, with particular
emphasis on the signaling of Secondary Sexual Characteristics (SSC).