Papers and patents are becoming less disruptive over time

Papers and patents are becoming less disruptive over time. Michael Park, Erin Leahey & Russell J. Funk. Nature volume 613, pages138–144, Jan 4 2023. https://www.nature.com/articles/s41586-022-05543-x

Abstract: Theories of scientific and technological change view discovery and invention as endogenous processes1,2, wherein previous accumulated knowledge enables future progress by allowing researchers to, in Newton’s words, ‘stand on the shoulders of giants’3,4,5,6,7. Recent decades have witnessed exponential growth in the volume of new scientific and technological knowledge, thereby creating conditions that should be ripe for major advances8,9. Yet contrary to this view, studies suggest that progress is slowing in several major fields10,11. Here, we analyse these claims at scale across six decades, using data on 45 million papers and 3.9 million patents from six large-scale datasets, together with a new quantitative metric—the CD index12—that characterizes how papers and patents change networks of citations in science and technology. We find that papers and patents are increasingly less likely to break with the past in ways that push science and technology in new directions. This pattern holds universally across fields and is robust across multiple different citation- and text-based metrics1,13,14,15,16,17. Subsequently, we link this decline in disruptiveness to a narrowing in the use of previous knowledge, allowing us to reconcile the patterns we observe with the ‘shoulders of giants’ view. We find that the observed declines are unlikely to be driven by changes in the quality of published science, citation practices or field-specific factors. Overall, our results suggest that slowing rates of disruption may reflect a fundamental shift in the nature of science and technology.

Discussion

In summary, we report a marked decline in disruptive science and technology over time. Our analyses show that this trend is unlikely to be driven by changes in citation practices or the quality of published work. Rather, the decline represents a substantive shift in science and technology, one that reinforces concerns about slowing innovative activity. We attribute this trend in part to scientists’ and inventors’ reliance on a narrower set of existing knowledge. Even though philosophers of science may be correct that the growth of knowledge is an endogenous process—wherein accumulated understanding promotes future discovery and invention—engagement with a broad range of extant knowledge is necessary for that process to play out, a requirement that appears more difficult with time. Relying on narrower slices of knowledge benefits individual careers53, but not scientific progress more generally.

Moreover, even though the prevalence of disruptive works has declined, we find that the sheer number has remained stable. On the one hand, this result may suggest that there is a fixed ‘carrying capacity’ for highly disruptive science and technology, in which case, policy interventions aimed at increasing such work may prove challenging. On the other hand, our observation of considerable churn in the underlying fields responsible for producing disruptive science and technology suggests the potential importance of factors such as the shifting interests of funders and scientists and the ‘ripeness’ of scientific and technologicalknowledge for breakthroughs, in which case the production of disruptive work may be responsive to policy levers. In either case, the stability we observe in the sheer number of disruptive papers and patents suggests that science and technology do not appear to have reached the end of the ‘endless frontier’. Room remains for the regular rerouting that disruptive works contribute to scientific and technological progress.

Our study is not without limitations. Notably, even though research to date supports the validity of the CD index12,34, it is a relatively new indicator of innovative activity and will benefit from future work on its behaviour and properties, especially across data sources and contexts. Studies that systematically examine the effect of different citation practices54,55, which vary across fields, would be particularly informative.

Overall, our results deepen understanding of the evolution of knowledge and may guide career planning and science policy. To promote disruptive science and technology, scholars may be encouraged to read widely and given time to keep up with the rapidly expanding knowledge frontier. Universities may forgo the focus on quantity, and more strongly reward research quality56, and perhaps more fully subsidize year-long sabbaticals. Federal agencies may invest in the riskier and longer-term individual awards that support careers and not simply specific projects57, giving scholars the gift of time needed to step outside the fray, inoculate themselves from the publish or perish culture, and produce truly consequential work. Understanding the decline in disruptive science and technology more fully permits a much-needed rethinking of strategies for organizing the production of science and technology in the future.