Understanding the onset of hot streaks across artistic, cultural, and scientific careers

Understanding the onset of hot streaks across artistic, cultural, and scientific careers. Lu Liu, Nima Dehmamy, Jillian Chown, C. Lee Giles, Dashun Wang. arXiv Mar 2021. https://arxiv.org/abs/2103.01256

Hot streaks dominate the main impact of creative careers. Despite their ubiquitous nature across a wide range of creative domains, it remains unclear if there is any regularity underlying the beginning of hot streaks. Here, we develop computational methods using deep learning and network science and apply them to novel, large-scale datasets tracing the career outputs of artists, film directors, and scientists, allowing us to build high-dimensional representations of the artworks, films, and scientific publications they produce. By examining individuals' career trajectories within the underlying creative space, we find that across all three domains, individuals tend to explore diverse styles or topics before their hot streak, but become notably more focused in what they work on after the hot streak begins. Crucially, we find that hot streaks are associated with neither exploration nor exploitation behavior in isolation, but a particular sequence of exploration followed by exploitation, where the transition from exploration to exploitation closely traces the onset of a hot streak. Overall, these results unveil among the first identifiable regularity underlying the onset of hot streaks, which appears universal across diverse creative domains, suggesting that a sequential view of creative strategies that balances experimentation and implementation may be particularly powerful for producing long-lasting contributions, which may have broad implications for identifying and nurturing creative talents.

---

Individual behavior At the individual level, the ‘essential tension’ hypothesis by Thomas Kuhn [60] illustrates the choice between exploiting existing ideas and exploring newyet risky opportunities. The sociology of science offers severalfundamentaltheoretical discussions [61,62]. Morerecently, empiricalanlaysishasbeenconductedtoquantitatively understand the ‘essential tension’ hypothesis. For example, Foster et al. [53] analyzed millions of abstracts from MEDLINE, and identified topics from the clusters on the chemical networkto trace the researchstrategy ofbiomedical researchers [63]. In addition,thePACS code in American Physical Society (APS) dataset has also been widely used to quantify exploration and exploitation for scientific careers [18, 52, 64].

Researchers have also studied various environmental, social and individualfactors that may influence one’s choice between exploration and exploitation [48]. Environmentalfactors include resource status of a local position [49, 65], cost and reward of exploration and exploitation [65, 66], available information on different options [67], and more. Discussions centered around how long individuals should stay in the exploitation/exploration phase and when to change their behaviors under different environmental settings. For example, the probability of exploration increases when the resource is depleted, when the cost of exploration decreases, or when individuals are uncertain about the options. The social factors are widely discussed in social learning strategies and collective intelligence [68–72], ranging fromtask complexity [73], to past success and failure [71, 73]tonetwork structures [74, 75]. Individuals can update their strategies like exploration, exploitation or copying others to increase theirpayoffsunderdifferent settings.Individual factors such as personalities [76], cognitive capacity [77], and aspiration level[78], also influence one’s propensity to explore or exploit.

In the literature of strategic management and organization theory, scholars have examined exploration and exploitation behaviors ofindividuals and firms, particularly focusing ontheeffects thishasonorganizationaloutcomes. Forexample,Singh&Agrawal[79]found that when scientists begin working within a new organization, the organization increases their use of the new recruit’s prior work and that the majority of the effect is due to the employee’s own exploitation of their prior work. Groysberg & Lee [80] found that when star security analysts were hired to explore (i.e., to initiate new activities for the organization), they experienced a drop in performance; whereas star security analysts hired to engage in exploitation (i.e., to reinforce the organization’s existing activities) experienced a boost in performance. Other research has looked at the antecedents of individuals’ exploration and exploitation behaviors. For example, Lee & Meyer-Doyle [81] examined how financialincentives shapedthebehavior of salespeopleandfoundthatindividuals engaged in more exploration when performance-based incentives were weakened but this increase wasdriven by the organization’s strongestperformers. Recent study on network oscillation for bankers [82] suggests that switching between exploration and exploitation has positive effects on the employee’s network advantage.

Organization learning, design and adaptation

At the macro level, another important line of research examines exploration and exploitation in the context of organization learning, organization design, and organizational adaptation [58]. This line of work builds on the canonical work by March [57], and suggests that both exploration and exploitation are critical for an organization’s performance, but they are inherently in tension and that this tension must be actively managed [83]. This tension reflects trade-offs between short vs. long-termperformance and stability vs. adaptability [57, 84–87]. Debates in this literature center onseveralfundamental questions: Do exploration and exploitation exist as twoends of a continuum (and so cannot coexist at the same time) or are they orthogonal discrete choices? Can organizations find a balance between exploration and exploitation activities or should they specialize in one or the other? It also explores the antecedents to organizations’decisions topursue exploration or exploitation [59, 88], examining environmental factors (e.g., exogenous shocks, competitive dynamics) as well as organizational factors (e.g., culture, resources, capabilities)thatinfluence that choice. This literaturealsouses the notion of organizational ambidexterity to describe the ability to do both exploration and exploitation simultaneously [89]. Finally, this research examines the performance implications for organizations of adopting different approaches to balancing this enduring trade-off between exploration and exploitation [90]. This line of research is performed using multiple different methodologies including empirical studies using quantitative and qualitative data from organizations, theoretical models [91], and agent-based simulations [59, 92, 93].

Idea formation

At a more micro level, the discussion of exploration and exploitation is particularly relevantto studies on idea formation and innovation process [94–96], which models themechanismofinnovationas randomwalksonthenetworkofideas/landscapeof solutions. In this setting, exploration and exploitation is usually defined as creating new path or reproducing existing ideas. For example, Iacopini et al [94] models the cognitive growthofknowledgeinscienceforover20yearsandvalidateprocesswithconceptnetworks curated from WoS abstracts. Studies have shown that both existing knowledge and novel combinations are essential for producing high-impact scientific papers [97]. The discussion goes beyond science to innovation and technology as well. For example, Youn et al.[98] analyzed technology codes used byUSPTO to quantify innovation strategy,finding a constant rate of exploration and exploitation in patent records. Overall, our results contribute to these three lines of literature in several ways. First, by documenting the relationship between exploration, exploitation and career hot streaks, our results demonstrate broader relevance ofthe concepts of exploration and exploitation, extendingbeyond existing individual ororganizational settings to theunderstanding ofhot streaks and individual creative careers. At root, our results suggest the important role of both exploration and exploitation in individual careers. Curiously, across a wide range of creative domains, a major turning point for individual careers appears most closely linked withneitherexplorationnorexploitationbehaviorinisolation,butratherwiththeparticular sequence of exploration followed by exploitation, which highlights our second contribution. Indeed, extantliteraturehasdocumentedthefundamental roleofexplorationandexploitation in creativity. Yet as creative behaviors, they have traditionally been considered either in isolation [53, 60] or in combination [58, 99] but rarely in succession. Our results suggest a sequential view of creative strategies that balance experimentation and implementation may be particularly powerful for producing long-lasting contributions.