Science of Science: Review Summary
Science of Science: Review Summary
Identifying fundamental drivers of science and developing predictive models to capture its
Networks of scientists, institutions,
evolution are instrumental for the design of policies that can improve the scientific enterprise—
and ideas
for example, through enhanced career paths for scientists, better performance evaluation for Contemporary science is a dynamical system of
organizations hosting research, discovery of novel effective funding vehicles, and even undertakings driven by complex interactions
identification of promising regions along the scientific frontier. The science of science uses among social structures, knowledge representa-
large-scale data on the production of science to search for universal and domain-specific tions, and the natural world. Scientific knowledge
T
other scholarly artifacts, organized into scientific
he deluge of digital data on scholarly out- millions of data points pertaining to scientists disciplines and broader fields. These social, con-
put offers unprecedented opportunities to and their output and capturing research from all ceptual, and material elements are connected
explore patterns characterizing the struc- over the world and all branches of science. Sec- through formal and informal flows of informa-
ture and evolution of science. The science ond, SciSci has benefited from an influx of and tion, ideas, research practices, tools, and samples.
of science (SciSci) places the practice of collaborations among natural, computational, Science can thus be described as a complex, self-
science itself under the microscope, leading to and social scientists who have developed big organizing, and constantly evolving multiscale
a quantitative understanding of the genesis of data–based capabilities and enabled critical network.
scientific discovery, creativity, and practice and tests of generative models that aim to capture Early studies discovered an exponential growth
developing tools and policies aimed at accelerat- the unfolding of science, its institutions, and in the volume of scientific literature (2), a trend
ing scientific progress. its workforce. that continues with an average doubling period
The emergence of SciSci has been driven by One distinctive characteristic of this emerging of 15 years (Fig. 1). Yet, it would be naïve to
two key factors. The first is data availability. In field is how it breaks down disciplinary bounda- equate the growth of the scientific literature with
addition to the proprietary Web of Science (WoS), ries. SciSci integrates findings and theories from the growth of scientific ideas. Changes in the
the historic first citation index (1), multiple data multiple disciplines and uses a wide range of publishing world, both technological and eco-
sources are available today (Scopus, PubMed, data and methods. From scientometrics, it takes nomic, have led to increasing efficiency in the
Google Scholar, Microsoft Academic, the U.S. the idea of measuring science from large-scale production of publications. Moreover, new pub-
Patent and Trademark Office, and others). Some data sources; from the sociology of science, it lications in science tend to cluster in discrete
of these sources are freely accessible, covering adopts theoretical concepts and social processes; areas of knowledge (3). Large-scale text analysis,
Fig. 1. Growth of science. (A) Annual production of scientific articles indexed in the WoS database. (B) Growth of ideas covered by articles indexed in the
WoS. This was determined by counting unique title phrases (concepts) in a fixed number of articles (4).
1
Center for Complex Networks and Systems Research, School of Informatics, Computing, and Engineering, Indiana University, Bloomington, IN 47408, USA. 2Indiana University Network Science
Institute, Indiana University, Bloomington, IN 47408, USA. 3Department of Biology, University of Washington, Seattle, WA 98195-1800, USA. 4Cyberinfrastructure for Network Science Center, School
of Informatics, Computing, and Engineering, Indiana University, Bloomington, IN 47408, USA. 5Department of Sociology, University of Chicago, Chicago, IL 60637, USA. 6Computational Social
Science, ETH Zurich, Zurich, Switzerland. 7Ernest and Julio Gallo Management Program, School of Engineering, University of California, Merced, CA 95343, USA. 8Center for Network Science, Central
European University, Budapest 1052, Hungary. 9Department of Mathematics, Central European University, Budapest 1051, Hungary. 10Institute for Network Science, Northeastern University, Boston,
MA 02115, USA. 11Kellogg School of Management, Northwestern University, Evanston, IL 60208, USA. 12Northwestern Institute on Complex Systems, Northwestern University, Evanston, IL 60208,
USA. 13Laboratory for the Modeling of Biological and Sociotechnical Systems, Northeastern University, Boston, MA 02115, USA. 14ISI Foundation, Turin 10133, Italy. 15Centre for Science and
Technology Studies, Leiden University, Leiden, Netherlands. 16Center for Cancer Systems Biology, Dana-Farber Cancer Institute, Boston, MA 02115, USA.
*Corresponding author. Email: [email protected] (S.F.); [email protected] (A.-L.B.)
using phrases extracted from titles and abstracts the risk of failure to publish at all. Scientific Measurements show that the allocation of bio-
to measure the cognitive extent of the scientific awards and accolades appear to function as medical resources in the United States is more
literature, have found that the conceptual territory primary incentives to resist conservative tend- strongly correlated to previous allocations and
of science expands linearly with time. In other encies and encourage betting on exploration research than to the actual burden of diseases
words, whereas the number of publications grows and surprise (3). Despite the many factors shaping (18), highlighting a systemic misalignment be-
exponentially, the space of ideas expands only what scientists work on next, macroscopic pat- tween biomedical needs and resources. This mis-
linearly (Fig. 1) (4). terns that govern changes in research interests alignment casts doubts on the degree to which
Frequently occurring words and phrases in along scientific careers are highly reproducible, funding agencies, often run by scientists embedded
article titles and abstracts propagate via citation documenting a high degree of regularity under- in established paradigms, are likely to influence
networks, punctuated by bursts corresponding lying scientific research and individual careers (14). the evolution of science without introducing
to the emergence of new paradigms (5). By Scientists’ choice of research problems affects additional oversight, incentives, and feedback.
applying network science methods to citation primarily their individual careers and the careers
networks, researchers are able to identify com- of those reliant on them. Scientists’ collective Novelty
munities as defined by subsets of publications choices, however, determine the direction of Analyses of publications and patents consistently
that frequently cite one another (6). These com- scientific discovery more broadly (Fig. 2). Con- reveal that rare combinations in scientific dis-
munities often correspond to groups of authors servative strategies (15) serve individual careers coveries and inventions tend to garner higher
holding a common position regarding specific well but are less effective for science as a whole. citation rates (3). Interdisciplinary research is
issues (7) or working on the same specialized Such strategies are amplified by the file drawer an emblematic recombinant process (19); hence,
subtopics (8). Recent work focusing on biomedical problem (16): Negative results, at odds with the successful combination of previously discon-
science has illustrated how the growth of the established hypotheses, are rarely published, nected ideas and resources that is fundamental
literature reinforces these communities (9). As leading to a systemic bias in published research to interdisciplinary research often violates expecta-
Problem selection
How do scientists decide which research prob-
lems to work on? Sociologists of science have
long hypothesized that these choices are shaped Fig. 2. Choosing experiments to accelerate collective discovery. (A) The average efficiency rate
by an ongoing tension between productive tradi- for global strategies to discover new, publishable chemical relationships, estimated from all
tion and risky innovation (12, 13). Scientists who MEDLINE-indexed articles published in 2010. This model does not take into account differences in
adhere to a research tradition in their domain the difficulty or expense of particular experiments. The efficiency of a global scientific strategy is
often appear productive by publishing a steady expressed by the average number of experiments performed (vertical axis) relative to the number of
stream of contributions that advance a focused new, published biochemical relationships (horizontal axis), which correspond to new connections
research agenda. But a focused agenda may limit in the published network of biochemicals co-occurring in MEDLINE-indexed articles. Compared
a researcher’s ability to sense and seize oppor- strategies include randomly choosing pairs of biochemicals, the global (“actual”) strategy inferred
tunities for staking out new ideas that are re- from all scientists publishing MEDLINE articles, and optimal strategies for discovering 50 and
quired to grow the field’s knowledge. For example, 100% of the network. Lower values on the vertical axis indicate more efficient strategies, showing
a case study focusing on biomedical scientists that the actual strategy of science is suboptimal for discovering what has been published. The
choosing novel chemicals and chemical relation- actual strategy is best for uncovering 13% of the chemical network, and the 50% optimal strategy is
ships shows that as fields mature, researchers most efficient for discovering 50% of it, but neither are as good as the 100% optimal strategy for
tend to focus increasingly on established knowl- revealing the whole network. (B) The actual, estimated search process illustrated on a hypothetical
edge (3). Although an innovative publication tends network of chemical relationships, averaged from 500 simulated runs of that strategy. The strategy
to result in higher impact than a conservative one, swarms around a few “important,” highly connected chemicals, whereas optimal strategies are much
high-risk innovation strategies are rare, because more even and less likely to “follow the crowd” in their search across the space of scientific
the additional reward does not compensate for possibilities. [Adapted from (15)]
(25–27). Papers of this type are twice as likely to according to their citation scores, which may previous output, markedly boosts the num-
receive high citations (26). In other words, a be rooted in a selection bias that offers better ber of citations collected by that paper in the
balanced mixture of new and established ele- career opportunities to better scientists (43, 44). first years after publication (47). After this
ments is the safest path toward successful re- Moreover, scientists tend to move between in- initial phase, however, impact depends on the
ception of scientific advances. stitutions of similar prestige (45). Nevertheless, reception of the work by the scientific com-
when examining changes in impact associated munity. This finding, along with the work re-
Career dynamics with each move as quantified by citations, no ported in (46), suggests that, for productive
Individual academic careers unfold in the con- systematic increase or decrease was found, not scientific careers, reputation is less of a critical
text of a vast market for knowledge production even when scientists moved to an institution of driver for success than talent, hard work, and
and consumption (28). Consequently, scientific considerably higher or lower rank (46). In other relevance.
careers have been examined not only in terms words, it is not the institution that creates the A policy-relevant question is whether creativity
of individual incentives and marginal productivity impact; it is the individual researchers that make and innovation depend on age or career stage.
(i.e., relative gain versus effort) (29), but also an institution. Decades of research on outstanding researchers
institutional incentives (30, 31) and competition Another potentially important career factor and innovators concluded that major break-
(32). This requires combining large repositories is reputation—and the dilemma that it poses throughs take place relatively early in a career,
of high-resolution individual, geographic, and for manuscript review, proposal evaluation, and with a median age of 35 (48). In contrast, recent
temporal metadata (33) to construct represen- promotion decisions. The reputation of paper work shows that this well-documented propen-
tations of career trajectories that can be ana- authors, measured by the total citations of their sity of early-career discoveries is fully explained
lyzed from different perspectives. For example, by productivity, which is high in the early stages
one study finds that funding schemes that are of a scientist’s career and drops later (49). In
tolerant of early failure, which reward long-term other words, there are no age patterns in in-
The integration of citation-based metrics with 7. U. Shwed, P. S. Bearman, The temporal structure of scientific 31. P. Azoulay, J. S. Graff Zivin, G. Manso, Incentives and
alternative indicators will promote pluralism consensus formation. Am. Sociol. Rev. 75, 817–840 (2010). creativity: Evidence from the academic life sciences.
doi: 10.1177/0003122410388488; pmid: 21886269 Rand J. Econ. 42, 527–554 (2011). doi: 10.1111/
and enable new dimensions of productive special- 8. J. Bruggeman, V. A. Traag, J. Uitermark, Detecting j.1756-2171.2011.00140.x
ization, in which scientists can be successful in communities through network data. Am. Sociol. Rev. 77, 32. R. Freeman, E. Weinstein, E. Marincola, J. Rosenbaum,
different ways. Science is an ecosystem that re- 1050–1063 (2012). doi: 10.1177/0003122412463574 F. Solomon, Competition and careers in biosciences. Science
quires not only publications, but also communi- 9. F. Shi, J. G. Foster, J. A. Evans, Weaving the fabric of science: 294, 2293–2294 (2001). doi: 10.1126/science.1067477;
Dynamic network models of science’s unfolding structure. pmid: 11743184
cators, teachers, and detail-oriented experts. We Soc. Networks 43, 73–85 (2015). doi: 10.1016/ 33. J. A. Evans, J. G. Foster, Metaknowledge. Science
need individuals who can ask novel, field-altering j.socnet.2015.02.006 331, 721–725 (2011). doi: 10.1126/science.1201765;
questions, as well as those who can answer them. 10. L. M. A. Bettencourt, D. I. Kaiser, J. Kaur, Scientific discovery pmid: 21311014
It would benefit science if curiosity, creativity, and topological transitions in collaboration networks. 34. V. Larivière, C. Ni, Y. Gingras, B. Cronin, C. R. Sugimoto,
J. Informetr. 3, 210–221 (2009). doi: 10.1016/ Bibliometrics: Global gender disparities in science.
and intellectual exchange—particularly regard- j.joi.2009.03.001 Nature 504, 211–213 (2013). doi: 10.1038/504211a;
ing the societal implications and applications of 11. X. Sun, J. Kaur, S. Milojević, A. Flammini, F. Menczer, pmid: 24350369
science and technology—are better appreciated Social dynamics of science. Sci. Rep. 3, 1069 (2013). 35. S. F. Way, D. B. Larremore, A. Clauset, in Proceedings of
and incentivized in the future. A more pluralistic doi: 10.1038/srep01069; pmid: 23323212 the 25th International Conference on World Wide Web
12. T. S. Kuhn, The Essential Tension: Selected Studies in (WWW ‘16) (ACM, 2016), pp. 1169–1179.
approach could reduce duplication and make Scientific Tradition and Change (Univ. of Chicago Press, 1977). 36. J. Duch et al., The possible role of resource requirements and
science flourish for society (95). 13. P. Bourdieu, The specificity of the scientific field and academic career-choice risk on gender differences in
An issue that SciSci seeks to address is the the social conditions of the progress of reasons. publication rate and impact. PLOS ONE 7, e51332 (2012).
allocation of science funding. The current peer Soc. Sci. Inf. (Paris) 14, 19–47 (1975). doi: 10.1177/ doi: 10.1371/journal.pone.0051332; pmid: 23251502
053901847501400602 37. J. D. West, J. Jacquet, M. M. King, S. J. Correll, C. T. Bergstrom,
review system is subject to biases and inconsisten- 14. T. Jia, D. Wang, B. K. Szymanski, Quantifying patterns of The role of gender in scholarly authorship. PLOS ONE 8,
cies (96). Several alternatives have been proposed, research-interest evolution. Nat. Hum. Behav. 1, 0078 (2017). e66212 (2013). doi: 10.1371/journal.pone.0066212;
such as the random distribution of funding (97), doi: 10.1038/s41562-017-0078 pmid: 23894278
15. A. Rzhetsky, J. G. Foster, I. T. Foster, J. A. Evans, Choosing 38. X. H. T. Zeng et al., Differences in collaboration patterns
Rev. Econ. Stud. 76, 283–317 (2009). doi: 10.1111/ 74. D. de Solla Price, A general theory of bibliometric and other 93. S. Ravindran, “Getting credit for peer review,” Science, 8
j.1467-937X.2008.00531.x cumulative advantage processes. J. Am. Soc. Inf. Sci. 27, February 2016; www.sciencemag.org/careers/2016/02/
55. S. Milojević, Principles of scientific research team formation 292–306 (1976). doi: 10.1002/asi.4630270505 getting-credit-peer-review.
and evolution. Proc. Natl. Acad. Sci. U.S.A. 111, 3984–3989 75. A.-L. Barabási, R. Albert, Emergence of scaling in random 94. R. Costas, Z. Zahedi, P. Wouters, Do “altmetrics” correlate
(2014). doi: 10.1073/pnas.1309723111; pmid: 24591626 networks. Science 286, 509–512 (1999). doi: 10.1126/ with citations? Extensive comparison of altmetric
56. G. Palla, A.-L. Barabási, T. Vicsek, Quantifying social group science.286.5439.509; pmid: 10521342 indicators with citations from a multidisciplinary perspective.
evolution. Nature 446, 664–667 (2007). doi: 10.1038/ 76. P. D. B. Parolo et al., Attention decay in science. J. Informetr. J. Assoc. Inf. Sci. Technol. 66, 2003–2019 (2015).
nature05670; pmid: 17410175 9, 734–745 (2015). doi: 10.1016/j.joi.2015.07.006 doi: 10.1002/asi.23309
57. G. J. Borjas, K. B. Doran, Which peers matter? The relative 77. D. Wang, C. Song, A.-L. Barabási, Quantifying long-term 95. A. Clauset, D. B. Larremore, R. Sinatra, Data-driven
impacts of collaborators, colleagues, and competitors. scientific impact. Science 342, 127–132 (2013). doi: 10.1126/ predictions in the science of science. Science 355, 477–480
Rev. Econ. Stat. 97, 1104–1117 (2015). doi: 10.1162/ science.1237825; pmid: 24092745 (2017). doi: 10.1126/science.aal4217
REST_a_00472 96. S. Wessely, Peer review of grant applications: What
78. Y.-H. Eom, S. Fortunato, Characterizing and modeling
58. P. Azoulay, J. G. Zivin, J. Wang, Superstar extinction. Q. J. Econ. do we know? Lancet 352, 301–305 (1998). doi: 10.1016/
citation dynamics. PLOS ONE 6, e24926 (2011). doi: 10.1371/
125, 549–589 (2010). doi: 10.1162/qjec.2010.125.2.549 S0140-6736(97)11129-1; pmid: 9690424
journal.pone.0024926; pmid: 21966387
59. A. M. Petersen, Quantifying the impact of weak, strong, and 97. N. Geard, J. Noble, paper presented at the 3rd World
79. M. Golosovsky, S. Solomon, Stochastic dynamical model of a
super ties in scientific careers. Proc. Natl. Acad. Sci. U.S.A. Congress on Social Simulation, Kassel, Germany, 6 to 9
growing citation network based on a self-exciting point
112, E4671–E4680 (2015). doi: 10.1073/pnas.1501444112; September 2010.
process. Phys. Rev. Lett. 109, 098701 (2012). doi: 10.1103/
pmid: 26261301 98. Calm in a crisis. Nature 468, 1002 (2010). doi: 10.1038/
60. R. K. Merton, The Matthew effect in science. Science 159, PhysRevLett.109.098701; pmid: 23002894
80. A. F. J. van Raan, Sleeping Beauties in science. 4681002a; pmid: 21170024
56–63 (1968). doi: 10.1126/science.159.3810.56
Scientometrics 59, 467–472 (2004). doi: 10.1023/B: 99. R. Roy, Funding science: The real defects of peer review and
61. L. Allen, J. Scott, A. Brand, M. Hlava, M. Altman, Publishing:
SCIE.0000018543.82441.f1 an alternative to it. Sci. Technol. Human Values 10, 73–81
Credit where credit is due. Nature 508, 312–313 (2014).
81. Q. Ke, E. Ferrara, F. Radicchi, A. Flammini, Defining and (1985). doi: 10.1177/016224398501000309
doi: 10.1038/508312a; pmid: 24745070
62. H.-W. Shen, A.-L. Barabási, Collective credit allocation in identifying Sleeping Beauties in science. Proc. Natl. Acad. 100. J. Bollen, D. Crandall, D. Junk, Y. Ding, K. Börner, An efficient
science. Proc. Natl. Acad. Sci. U.S.A. 111, 12325–12330 Sci. U.S.A. 112, 7426–7431 (2015). doi: 10.1073/ system to fund science: From proposal review to peer-to-
(2014). doi: 10.1073/pnas.1401992111; pmid: 25114238 pnas.1424329112; pmid: 26015563 peer distributions. Scientometrics 110, 521–528 (2017).
63. L. Waltman, A review of the literature on citation impact 82. I. Tahamtan, A. Safipour Afshar, K. Ahamdzadeh, Factors doi: 10.1007/s11192-016-2110-3
REFERENCES This article cites 91 articles, 25 of which you can access for free
http://science.sciencemag.org/content/359/6379/eaao0185#BIBL
PERMISSIONS http://www.sciencemag.org/help/reprints-and-permissions
Science (print ISSN 0036-8075; online ISSN 1095-9203) is published by the American Association for the Advancement of
Science, 1200 New York Avenue NW, Washington, DC 20005. 2017 © The Authors, some rights reserved; exclusive
licensee American Association for the Advancement of Science. No claim to original U.S. Government Works. The title
Science is a registered trademark of AAAS.