ARTICLE

Received 29 May 2016 | Accepted 12 Dec 2016 | Published 19 Jan 2017 DOI: 10.1057/palcomms.2016.105 OPEN

“Excellence R Us”: university research and the

fetishisation of excellence
Samuel Moore1, Cameron Neylon2, Martin Paul Eve3, Daniel Paul O’Donnell4 and Damian Pattinson5

ABSTRACT The rhetoric of “excellence” is pervasive across the academy. It is used to refer
to research outputs as well as researchers, theory and education, individuals and organiza-
tions, from art history to zoology. But does “excellence” actually mean anything? Does this
pervasive narrative of “excellence” do any good? Drawing on a range of sources we inter-
rogate “excellence” as a concept and find that it has no intrinsic meaning in academia. Rather
it functions as a linguistic interchange mechanism. To investigate whether this linguistic
function is useful we examine how the rhetoric of excellence combines with narratives of
scarcity and competition to show that the hyper-competition that arises from the perfor-
mance of “excellence” is completely at odds with the qualities of good research. We trace the
roots of issues in reproducibility, fraud, and homophily to this rhetoric. But we also show that
this rhetoric is an internal, and not primarily an external, imposition. We conclude by
proposing an alternative rhetoric based on soundness and capacity-building. In the final
analysis, it turns out that that “excellence” is not excellent. Used in its current unqualified
form it is a pernicious and dangerous rhetoric that undermines the very foundations of good
research and scholarship. This article is published as part of a collection on the future of
research assessment.

1 Kings College, London, UK 2 Curtin University, Perth, Australia 3 Birkbeck, University of London, UK 4 University of Lethbridge, Canada 5 Research Square,

London, UK Correspondence: (e-mail: [email protected])

PALGRAVE COMMUNICATIONS | 3:16105 | DOI: 10.1057/palcomms.2016.105 | www.palgrave-journals.com/palcomms 1

ARTICLE PALGRAVE COMMUNICATIONS | DOI: 10.1057/palcomms.2016.105

scholars? Does “excellence” live up to the expectations that

E
Introduction: the ubiquity of excellence rhetoric
“ xcellence” is the gold standard of the university world. academic communities place upon it? Is “excellence” excellent?
Institutional mission statements or advertisements pro- And are we being excellent to each other in using it?
claim, in almost identical language, their “international This article examines the utility of “excellence” as a means for
reputation for [educational] excellence” (for example, Baylor, organizing, funding, and rewarding science and scholarship. It
Imperial College London, Loughborough University, Monash argues that academic research and teaching is not well served by
University, The University of Sheffield), or the extent to which this rhetoric. Nor, we argue, is it well served by the use of
they are guided by principles of “excellence” (University of “excellence” to determine the distribution of resources and
Cambridge, Carnegie Mellon, Gustav Adolphus, University incentives to the world’s researchers, teachers and research
College London, Warwick and so on). University research offices institutions. While the rhetoric of “excellence” may seem in the
and faculties turn this goal into reality through centres and current climate to be a natural method for determining
programmes of “excellence”, which are in turn linked through which researchers, institutions, and projects should receive
networks such as the Canadian “Networks of Centres of scarce resources, we demonstrate that it is not as efficient,
Excellence” or German “Clusters of Excellence” (OECD, 2014; accurate, or necessary as it may seem. As we show, indeed, a focus
Networks of Centres of Excellence of Canada 2015). Funding on “excellence” impedes rather than promotes scientific and
agencies use “excellence to recognize excellence” (Nowotny, scholarly activity: it at the same time discourages both the
2014). intellectual risk-taking required to make the most significant
The academic funding environment, likewise, is saturated with advances in paradigm-shifting research and the careful “Normal
this discourse. A study of the National Endowment for the Science” (Kuhn [1962] 2012) that allows us to consolidate our
Humanities is entitled Excellence and Equity (Miller, 2015). The knowledge in the wake of such advances. It encourages
Wellcome Trust, a large medical funder, has grants for researchers to engage in counterproductive conscious and
“sustaining excellence” (Sustaining Excellence Awards, 2016). unconscious gamesmanship. And it impoverishes science and
The National Institutes of Health (NIH), the largest funder of scholarship by encouraging concentration rather than distribu-
civilian science in the United States, claims to fund “the best tion of effort. The net result is science and scholarship that is less
science by the best scientists” (Nicholson and Ioannidis, 2012) reliable, less accurate, and less durable than research assessed
and regularly supports “centres of excellence”. The University according to other criteria. While we acknowledge that it often
Grants Commission of India recently awarded 15 institutions the seems politically necessary to argue for “excellence”, and while we
title of “University with Potential for Excellence” (University understand that funding and accreditation bodies and agencies
Grants Commission, 2016). In the United Kingdom, the must play a political as well as scientific game, we here present the
“Research Excellence Framework” uses expert assessment of evidence that the internalization of such rhetoric into the research
“excellence” as a means of channelling differential funding to space can be counter-productive.
departments and institutions. In Australia, the national review The article itself falls into three parts. In the first section, we
framework is known as “Excellence in Research for Australia”. In discuss “excellence” as a rhetoric. Drawing on work by Michèle
Germany, the Deutsche Forschungsgemeinschaft supports its Lamont and others, we argue that “excellence” is less a
“Clusters of Excellence” through a long standing “Excellence discoverable quality than a linguistic interchange mechanism by
Initiative” (OECD, 2014). which researchers compare heterogeneous sets of disciplinary
As this range of examples suggests, “excellence”, as used by practices. In the second section, we dig more deeply into the
universities and their funders, is a flexible term that operates in a question of “excellence” as an assessment tool: we show how it
variety of contexts across a range of registers. It can describe alike distorts research practice while failing to provide a reliable means
the activities of the world's top research universities and its of distinguishing among competing projects, institutions, or
smallest liberal arts colleges. It applies to their teaching, research, people. In the final section, we consider what it might take to
and management. It encompasses simultaneously the work of change our thinking on “excellence” and the scarcity it
their Synthetic Biologists and Urban Sociologists, their Anglo- presupposes. We consider alternative narratives for approaching
Saxonists and Concert Pianists. It defines their Centres for the assessment of research activity, practitioners, and institutions
Excellence in Teaching and their Centres of Excellence for and discuss ways of changing the “scarcity-thinking” that has led
Mechanical Systems Innovation (The University of Tokyo Global us to our current use of this fungible and unreliable term. We
Center of Excellence, 2016; “USC Center for Excellence in propose that a narrative built on “soundness” and “capacity”
Teaching”, 2016), their multiculturalism (Office of Excellence offers us the opportunity to focus on practice of productive
and Multicultural Student Success 2016) and their athletic research and on the crucial role that social communication and
training programmes (Excellence Academy, 2016). “Excellence” criticism plays. Where there is more heterogeneity and greater
is used to define success in academic endeavour from Montreal to opportunity for diversity of outcomes and perspectives, we argue,
Mumbai. research improves.
But what does “excellence” mean? Is there a single standard for
identifying this apparently ubiquitous quality? Or is “excellence” What is “excellence”?
defined on a discipline-by-discipline, or case-by-case basis? Can In her book, How Professors Think: Inside the Curious World of
you know “excellence” before you see it? Or is it defined after the Academic Judgment, Michèle Lamont opens by noting that
fact? Does the search for “excellence”, its use to reward and “ ‘excellence’ is the holy grail of academic life” (Lamont, 2009, 1).
punish individual institutions and researchers, and its utility as a Yet, as she quickly moves to highlight, this “excellence is
criterion for the organization of research help or hinder the actual produced and defined in a multitude of sites and by an array of
production of that research and scholarship? Tertiary education actors. It may look different when observed through the lenses of
enrols approximately 32% of world’s student age population, and peer review, books that are read by generations of students,
OECD countries spent on average 1.6% of their GDP on current articles published by ‘top’ journals, elections at national
University-level teaching and research in 2015; the United States academies, or appointments at elite institutions” (3). Or as Jack
alone spent 2.7% or US$484 billion (The Economist, 2015). Is Stilgoe suggests: “ ‘Excellence’ tells us nothing about how
“excellence” really the most efficient metric for distributing the important the science is and everything about who decides”
resources available to the world’s scientists, teachers, and (Stilgoe, 2014).

2 PALGRAVE COMMUNICATIONS | 3:16105 | DOI: 10.1057/palcomms.2016.105 | www.palgrave-journals.com/palcomms

PALGRAVE COMMUNICATIONS | DOI: 10.1057/palcomms.2016.105 ARTICLE

This tallies with the work of others who have considered (large word counts, single authorship, publication or review in
reforms to the review process in recent years. Kathleen popular literary magazines and journals) (O’Donnell, 2015).
Fitzpatrick, for instance, has also situated the crux of evaluation Finally, as we will go on to show, it is clear that evaluative cultures
in the evaluator, not the evaluated. For, as Fitzpatrick notes, are operating without even internal consensus beyond a few
broad categories of performance.
“in using a human filtering system, the most important thing That said, it remains tempting to argue that such concepts of
to have information about is less the data that is being filtered, value, even if they are ungrounded and unshared, can be
than the human filter itself: who is making the decisions, and used pragmatically to foster consensus. This is the point of
why. Thus, in a peer-to-peer review system, the critical activity Wittgenstein’s (2001: section 293) famous “beetle in a box”
is not the review of the texts being published, but the review of metaphor, which he uses to exemplify the “private language
the reviewers.” (Fitzpatrick, 2011, 38) argument”. For Wittgenstein, the question of unique non-
communicable epistemic knowledge (such as pain experience),
The challenge here is that it is not possible to conduct a “review should actually be framed in terms of public, pragmatic
of the reviewers” without some reference to the evaluated language games/contexts. If we each have an object in a box
material. It is possible to query the conduct of reviewers or the that is called a “beetle,” but none of us can see each other’s
process they are (supposed to be) applying against another set of “beetles”, he argues, then the important thing is not what the
disciplinary norms (that is, are the reviewers acting in good faith? objects in our boxes actually are but rather how we negotiate and
Have they provided a useful report? Do they know the field as use the term socially to engender intersubjective understanding or
normatively defined?); but to assess qualitative aspects of action. In such cases, “if we construe the grammar of the
reviewers’ judgment of a specific work requires an external expression of sensation on the model of ‘object and designation’,
evaluation of the work itself—a type of circularity in which a pre- the object drops out of consideration as irrelevant” and
shared evaluative culture must exist in order to pass judgment on designation is all that matters.
the evaluation that is its basis: the “shared standards” of which We might therefore productively ask: even if “excellence” is a
Lamont writes (2009: 4). concept that carries little or no information content, either within
Yet despite the anti-foundational nature of this problem, there communities or across them, might it nonetheless be useful as a
remains a pressing need, in Lamont’s view, to ensure that “peer “beetle”? That is, as a carrier of interpretation or a set of social
review processes [... are] themselves subject to further evaluation” practices functioning as an expert system to convert intrinsic,
(247). Calls for training in peer review practices as well as calls for qualitative, and non-communicable assessment into a form that
greater transparency occur across disciplinary boundaries, but allows performance to be compared across disciplinary or other
generally without addressing the differences in practice that occur boundaries? Might it, indeed, even be useful given the political
on either side of those boundaries. Lamont suggests that current necessity for research communities and institutions to present an
remedies to this problem—which mostly consist of changing the (ostensibly) unified front to government and wider publics as a
degrees of anonymity or the point at which review is conducted means of protecting their autonomy? Could “excellence” be, to
(pre- versus post-filter)—are insufficient and constitute “imper- speak bluntly, a linguistic signifier without any agreed upon
fect safeguards”. Instead, she suggests, it is more important that referent whose value lies in an ability to capture cross-disciplinary
members of peer-review communities should be educated “about value judgements and demonstrate the political desirability of
how peer evaluation works,” avoiding the pitfalls of homophily public investment in research and research institutions?
(in which review processes merely re-inscribe value to work that In actual practice, it is not even useful in this way. Although, as
exhibits similitude to pre-existing examples) by re-framing the its ubiquity suggests, “excellence” is used across disciplines to
debate as a “micro-political process of collective decision making” assert value judgements about otherwise incomparable scientific
that is “genuinely social” (246–247). As with most problems in and scholarly endeavours, the concept itself mostly fails to
scholarly communication, the challenge with peer review is capture the disciplinary qualities it claims to define. Because it
therefore not technical but social. lacks content, “excellence” serves in the broadest sense solely as
As Lamont and others show, then, “excellence” is a pluralized an (aspirational) claim of comparative success: that some thing,
construct that is specific to (and conservative within) each person, activity, or institution can be asserted in a hopefully
disciplinary environment. Yet even the most obvious solution to convincing fashion to be “better” or “more important” than some
this challenge—interdisciplinary diversity of evaluators—only other (often otherwise incomparable) thing, person, activity, or
leads to further problems. For the differences in practice of institution—and, crucially, that it is, as a result, more deserving of
review and perceptions of “excellence” across disciplinary reward. But this emphasis on reward, as Kohn (1999) and others
boundaries, combined with a lack of appreciation that these have demonstrated, is itself often poisonous to the actual qualities
differences exist, makes it difficult to reach consensus within such of the underlying activity.
diverse pools of reviewers. This is because, as Stirling (2007b) has
noted, “it is difficult indeed to contemplate any single general
index of diversity that could aggregate properties [...] in a Is “excellence” good for research?
uniquely robust fashion”. If diversity itself cannot easily be Thus far, we have been arguing that “excellence” is primarily a
collapsed onto a single measurable vector then there is little hope rhetorical signalling device used to claim value across hetero-
of aggregating diverse senses of “excellence” into a coherent and geneous institutions, researchers, disciplines, and projects rather
universal framework. than a measure of intrinsic and objective worth. In some cases,
This suggests that “excellence” resides between different the qualities of these projects can be compared in detail on other
communities and is ill-structured/defined in each context. Local bases; in many—perhaps most—cases, they cannot. As we have
groups and disciplines may have their own more specific (though argued, the claim that a research project, institution, or
sometimes conventional rather than explicit) measures of practitioner is “excellent” is little more than an assertion that
“excellence”: Biologists may treat some aspects of performance that project, institution, or practitioner can be said to succeed
as “excellent” (for example, number of publications, author better on its own terms than some other project, institution, or
position, citations counts), while failing to recognize aspects practitioner can be said to succeed on some other, usually largely
considered equally or more “excellent” by English professors incomparable, set of terms.

PALGRAVE COMMUNICATIONS | 3:16105 | DOI: 10.1057/palcomms.2016.105 | www.palgrave-journals.com/palcomms 3

ARTICLE PALGRAVE COMMUNICATIONS | DOI: 10.1057/palcomms.2016.105

But what about these sets of “own terms”? How easy is it to While these statistics have almost certainly changed in the last
define the “excellence” of a given project, institution, or few years with changes in the demographics of submission
practitioner on an intrinsic basis? Even if we leave aside the and, especially, the development of venues that focus on the
comparative aspect, are there formal criteria that can be used to publication of “sound science” (Public Library of Science, 2016),
identify “excellence” in a single research instance on its own terms the basic sense that journal peer review is a gatekeeper that is
or that of a single discipline? frequently circumvented remains.
Research suggests that this is far harder than one might think. Articles that are initially rejected and then go on to be
Academics, it turns out, appear to be particularly poor at published to great acclaim or even just in journals of a similar or
recognizing a given instance of “excellence” when they see it, or, if higher ranking represent what are in essence false negatives in our
they think they do, getting others to agree with them. Their ability to assess “excellence.” They are also evidence of terrible
continued willingness to debate relative quality in these terms, inefficiency. The rejection of papers that are subsequently
moreover, creates a basis for extreme competition that has serious published with little or no revision at journals of similar rank
negative consequences. increases the costs for everyone involved without any counter-
vailing improvement in quality. In addition to multiplying the
systemic cost of refereeing and editorial management by the
Do researchers recognize excellence when they see it? The short number of resubmissions, such articles also present an opportu-
answer is no. This can be seen most easily when different nity cost to their authors through lost chances to claim priority
potential measures of “excellence” conflict in their assessment of a for discoveries, for example, or, even more commonly, lost
single paper, project, or individual. Adam Eyre-Walker and Nina opportunities for citation and influence (Gans and Shepherd,
Stoletzki, for example, conclude that scientists are poor at esti- 1994; Campanario, 2009; Şekercioğlu, 2013; Brembs, 2015; Psych
mating the merit and impact of scientific work even after it has Filedrawer, 2016).
been published (2013). Post-publication assessment is prone to More worryingly, there is also considerable evidence of false
error and biased by the journal in which the paper is published. positives in the review process—that is to say submissions that are
Predictions of future impact as measured by citation counts are judged to meet the standards of “excellence” required by one
also generally unreliable, both because scientists are not good at funding agency, journal, or institution, but do worse when
assessing merit consistently across multiple metrics and because measured against other or subsequent metrics. In a somewhat
the accumulation of citations is itself a highly stochastic process, controversial work, Peters and Ceci submitted papers in slightly
such that two papers of similar merit measured on other bases disguised form to journals that had previously accepted them for
can accumulate very different numbers of citations just by chance. publication (Peters and Ceci, 1982; see Weller, 2001 for a
Moreover, Wang et al. (2016) show that in terms of citation critique). Only 8% overall of these resubmissions were explicitly
metrics the most novel work is systematically undervalued over detected by the editors or reviewers to which they were assigned.
the time frames that conventional measures use, including, for Of the resubmissions that were not explicitly detected, approxi-
instance, the Journal Impact Factor that Eyre-Walker and Sto- mately 90% were ultimately rejected for methodological and/or
letzki suggest biases expert assessment. other reasons by the same journals that had previously published
This is true even of work that can be shown to be successful by them; they were rejected, in other words, for being insufficiently
other measures. Campanario, Gans and Shepherd, and others, for “excellent” by journals that had decided they were “excellent”
example, have traced the rejection histories of Nobel and other enough to enter the literature previously.
prize winners, including for papers reporting on results for which When it comes to funding, a similar pattern of false positives
they later won their recognition (Gans and Shepherd, 1994; may pertain: a study by Nicholson and Ioannidis (2012) suggests
Campanario, 2009; Azoulay et al., 2011: 527–528). Campanario that highly cited authors are less likely to head major biomedical
and others have also reported on the initial rejection of papers research grants than less-frequently-cited but socially better-
that later went on to become among the more highly cited in their connected authors who are associated with granting agency study
fields or in the journals that ultimately accepted them groups and review panels. Fang, Bowen and Casadevall have
(Campanario, 1993, 1996; Campanario, 1995; Campanario and discovered that “the percentile scores awarded by peer review
Acedo, 2007; Calcagno et al., 2012; Nicholson and Ioannidis, panels” at the NIH correlated “poorly” with “productivity as
2012; Siler et al., 2015). Yet others have found a generally poor measured by citations of grant-supported publications” (Fang
relationship between high ratings in grant competitions and et al., 2016). These suggest a bias towards conformance and social
subsequent “productivity” as measured by publication or citation connectedness over innovation in funding decisions in a world in
counts (Pagano, 2006; Costello, 2010; Lindner and Nakamura, which success rates are as low as 10%. It also provides further
2015; Fang et al., 2016; Meng, 2016). evidence of funding-agency bias against disruptively innovative
As this suggests, academics’ abilities to distinguish the work noted by many researchers over the years (Kuhn [1962]
“excellent” from the “not-excellent” do not correlate well with 2012; Campanario, 1993, 1995, 1996, 2009; Costello, 2010;
one another even within the same disciplinary environment Ioannidis et al., 2014; Siler et al., 2015).
(there tends to be greater agreement at the other end of the scale,
distinguishing the “not acceptable” from the “acceptable,” see
Cicchetti, 1991; Weller, 2001). To earn citations or win prizes for Fraud, error and lies. To the extent that the above are evidence
a rejected manuscript, after all, authors need to begin by of inefficiencies in the system, some might argue that individual
convincing a different journal (and its referees) to accept work problems in determining “excellence” in specific cases are
that others previously have found wanting. resolved in the longer term and over large samples. Of course,
But this is not something that only Nobel prize winners are these examples only show work for which multiple measures of
good at: as Weller reported in the early years of this century, most “excellence” can be compared: given their unreliability, this sug-
(51.4%) rejected manuscripts were ultimately published; in the gests that work that is not measured more than once may be
vast majority of cases (approximately 90%), these previously unjustly suppressed or unjustly published, without us being able
rejected articles were accepted on their second submission and, in to tell the difference. On the other hand, it is presumably possible
the vast majority of these cases (also approximately 90%), that even such extreme examples of differing perceptions of
at a journal of similar prestige and circulation (Weller, 2001). “excellence” represent honest differences of opinion as to the

4 PALGRAVE COMMUNICATIONS | 3:16105 | DOI: 10.1057/palcomms.2016.105 | www.palgrave-journals.com/palcomms

PALGRAVE COMMUNICATIONS | DOI: 10.1057/palcomms.2016.105 ARTICLE

qualitative merit of the research or researchers. The same cannot academic culture and arguably par for the course in applying for
be said, however, of actual fraud and outright errors. competitive research funds” (6). Quoting an interviewee, they
As various studies have concluded, reported instances of both continue, “If you can find me a single academic who hasn’t had to
fraud and error (as measured through retractions) are on the rise bullshit or bluff or lie or embellish to get grants, then I will find
(Claxton, 2005; Dobbs, 2006; Steen, 2011; Fang et al., 2012; you an academic who is in trouble with his [sic] Head of
Grieneisen and Zhang, 2012; Yong, 2012b; Chen et al., 2013; Department” (6; “[sic]” as in Chubb and Watermeyer). Here we
Andrade, 2016). This is particularly true at higher prestige see how a competitive requirement, perceived or real, for
journals (Resnik et al., 2015; Siler et al., 2015; Belluz, 2016). If we “excellence”, in combination with a lack of belief in the ability
add to this list of (potentially) “false positives” studies that cannot of assessors to detect false claims, leads to a conception of
be replicated, the number of papers that meet one measure of “excellence” as pure performance: a concept defined by what you
“excellence” (that is, passing peer review, often at “top” journals) can get away with claiming in order to suggest (rather than
while failing others (that is, being accurate and reproducible, and/ actually accomplish) “excellence”.
or non-fraudulent) rises considerably (Dean, 1989; Burman et al., What is striking about these behaviours, of course, is that they
2010; Lehrer, 2010; Bem, 2011; Goldacre, 2011; Yong, 2012b; are unrelated to (and to a great extent perhaps even incompatible
Rehman, 2013; Resnik and Dinse, 2013; Hill and Pitt, 2014; with or opposed to) the actual qualities funders, governments,
Chang and Li, 2015; Open Science Collaboration, 2015). It is the journal editors and referees, and researchers themselves are
very focus on “excellence”, however, that creates this situation: ostensibly using “excellence” to identify. No agency, ministry,
the desire to demonstrate the rhetorical quality of “excellence” press, or research office intentionally uses “excellence” as
encourages researchers to submit fraudulent, erroneous, and shorthand for “able to embellish results or importance convin-
irreproducible papers, at the same time as it works to prevent the cingly”, even as the researchers being adjudicated under this
publication of reproduction studies that can identify such work. system report such embellishment as a primary criterion for
In other words, erroneous, and especially fraudulent or success. Whether it occurs through fraud, cutting corners, or
irreproducible papers are interesting because they represent a exaggeration, this performance of “excellence” is commonly
failure of both our ability to identify and predict actual qualitative justified as being necessary for survival, suggesting a cognitive and
“excellence” and the incentive system that is used to encourage cultural dissonance between those aspects of their work that the
scientists and scholars to produce the kind of sound and performers feel is essential and those aspects they feel they must
defensible work that should be a sine qua non for quality. As emphasise, overstate, embellish, or fabricate to appear more
Fang, Steen, and Casadevall (2012; cf Steen, 2011 for which the “excellent” than their competitors. The evidence that fraud and
later article represents a correction) have shown, the majority of corner-cutting are a problem at the core of the research process
retracted papers are withdrawn for reasons of misconduct suggests that the pressure for these performances of “excellence”
including fraud, duplicate publication, or plagiarism (67.4%), is not restricted to stages that do not matter. As Kohn argues,
rather than error (21.3%)—although inadvertent error should reward-motivation affects scientific creativity (the ability to
presumably itself be disqualification from “excellence”. But even “break out of the fixed pattern of behaviour that had succeeded
these figures may under-represent the true incidence of in producing rewards… before”) as much as it does evidence-
misconduct. Mistakes and errors made in good faith are a natural gathering or the inflation of results (1999, 44; see also Lerner and
and necessary part of the research process. Yet, as focus groups Wulf, 2006; Azoulay et al., 2011; Tian and Wang, 2011).
and surveys conducted by various researchers have demonstrated,
some forms of error can be misconduct in the form of a (semi-)
deliberate strategy for ensuring quick and/or numerous publica- Competition for scarce resources and the performance of
tions by “ ‘cutting a little corner’ in order to get a paper out before “excellence”. So why do researchers engage in this kind of
others or to get a larger grant,... [or] because... [a researcher] dubious activity? Clearly for both Chubb and Watermeyer’s
needed more publications that year” (Anderson et al., 2007: 457– interviewees, as well as those identified as having committed
458; see also Fanelli, 2009; Tijdink et al., 2014; Chubb and scientific fraud, it is competition for scarce resources, whether
Watermeyer, 2016). funding, positions, or community prestige. Of course this is not a
Thus in one small sample of detailed surveys, Fanelli showed new issue (Smith, 2006). Taking time away from his work on the
that while only a small percentage of scientists (1.97% pooled difference machine, Charles Babbage published an analysis of
weighted average, n = 7) admitted to fabricating, falsifying, or what he saw as the four main kinds of scientific frauds in an 1830
modifying data, a much larger percentage claimed to have seen polemic, Reflections on the Decline of Science in England: And on
others engaging in similarly outright fraudulent activity (14.12%, Some of Its Causes. These included the self-explanatory “hoaxing”
n = 12). Furthermore, even larger percentages had engaged in and “forging,” in addition to “trimming” (“clipping off little bits
(33.7%) or seen others engage in (72%) questionable research here and there from those observations which differ most in
described using less negatively loaded language (Fanelli, 2009; the excess from the mean and in sticking them on to those which are
percentage of scientists admitting to explicit misconduct is too small”) and “cooking” (“an art of various forms, the object of
considerably higher [15%] in Tijdink et al., 2014). As Fanelli which is to give ordinary observations the appearance and
concludes: “Considering that these surveys ask sensitive questions character of those of the highest degree of accuracy”) (Babbage,
and have other limitations, it appears likely that this is a 1831: 178; see Zankl, 2003; and Secord, 2015 for a discussion).
conservative estimate of the true prevalence of scientific The motivation for these frauds, then as now, involves prestige
misconduct” (2009, 9)—a conclusion very strongly supported and competition for resources. Babbage’s typology of fraudulent
by the anecdotal admissions of Anderson et al.’s focus groups. science was but a minor chapter in a book otherwise mostly
The drive for “excellence” in the eyes of assessors is shown even concerned with the internal politics of the Royal Society. He
more starkly in work by Chubb and Watermeyer (2016). In attributed the decline he saw in English science to the lack of
structured interviews, academics in Australia and the United attention and professional opportunities available to potential
Kingdom admitted to outright lies in the claims of broader scientists. He was, as a result, keenly sensitive to questions of
impacts made in research proposals. As the authors note: “Having credit and its importance in determining rank and authority.
to sensationalize and embellish impact claims was seen to have Indeed, as Casadevall and Fang remind us, “Since Newton,
become a normalized and necessary, if regretful, aspect of science has changed a great deal, but this basic fact has not.

PALGRAVE COMMUNICATIONS | 3:16105 | DOI: 10.1057/palcomms.2016.105 | www.palgrave-journals.com/palcomms 5

ARTICLE PALGRAVE COMMUNICATIONS | DOI: 10.1057/palcomms.2016.105

Credit for work done is still the currency of science…. Since the existing studies, fuelled by a focus on novelty in most definitions
earliest days of science, bragging rights to a discovery have gone of “excellence”. As Nosek et al. note
to the person who first reports it” (Casadevall and Fang, 2012:
13). The prestige of first discovery always has been a scarce Publishing norms emphasize novel, positive results. As such,
resource. Now that that prestige is measured also through the disciplinary incentives encourage design, analysis, and report-
scarce resource of authorship in “the right journals” and coupled ing decisions that elicit positive results and ignore negative
ever more strongly to the further scarce resources of career results. Prior reports demonstrate how these incentives inflate
advancement and grant funding, it should not be a surprise that the rate of false effects in published science. When incentives
the competition for those markers has become steadily stronger. favour novelty over replication, false results persist in the
The performance of “excellence” has become more marked literature unchallenged, reducing efficiency in knowledge
as a result. accumulation. (2012)
If scandals such as fraudulent articles were the only way in
which this overwhelming competitive focus on “excellence” hurt This bias against replication is even more remarkable, however,
research, it would be bad enough. But the emphasis on rewarding when it involves studies that invalidate rather than confirm the
the performance of “excellence” also has a more general impact original result, especially when the original result has a high
on research capacity: it is the mechanism by which “the Matthew profile or is potentially field-defining—qualities that one would
effect”—that is, the disproportionate accrual of resources to those assume would increase the novelty and interest of the (non)
researchers and institutions that are already well-rewarded— replication itself (Goldacre, 2011; Wilson, 2011; Nosek et al.,
operates in a hyper-competitive research environment, creating 2012; Yong, 2012a, b; Aldhous, 2011; for a view from the other
distortions throughout the research cycle, even for work that is side of replication, see Bissell, 2013). This is in part, a function of
not fraudulent or the result of misconduct (Bishop, 2013; as its publishing economics: commercial journals earn money from
etymology implies, the “Matthew effect” predates today’s subscription, access, and reprint fees (Lundh et al., 2010); high
hypercompetition, see Merton, 1968, 1988)1: it increases the profile results and a high prestige reflected by a high Impact
stakes of the competition for resources and, as a result, Factor help maintain the demand for these journals and hence
encourages gamesmanship; creates a bias towards (non- ensure both a continuing stream of interesting new material and a
disruptively) novel, positive, and even inflated results on the steady or rising income for the journal as a whole (Lawrence,
part of authors and editors; and discourages the pursuit and 2007; Munafò et al., 2009; Lundh et al., 2010; Marcovitch, 2010).
publication of types of “Normal Science” (such as replication Undercutting (or perhaps even qualifying) the high-profile results
studies) that are crucial to the viability of the research enterprise, that help bring in these subscribers, new articles, and attention
without being glamorous enough to suggest that their authors are attacks the very foundation of this success—a journal that
“excellent”. publishes high profile but incorrect papers is undercutting its case
for subscription and author submissions. One doesn’t need to
Positive bias and the decline effect. Just how destructive this imagine a conspiracy to promote poor science to understand how
need to perform “excellence” is can be illustrated by the well- a conscious or unconscious bias against replication studies might
known bias towards positive results in scientific publication (for arise under such circumstances.
example, Dickersin et al., 1987, 2005; Sterling, 1959; Kennedy, The reluctance of major journals to publish replication studies
2004; Young and Bang, 2004; Bertamini and Munafò, 2012; embeds this bias in the incentive system that guides authors. As
Rothstein, 2014; Psych Filedrawer, 2016). Thus, for example, Wilson notes:
Fanelli (2011) demonstrated a 22% growth between 1990 and
2007 in the “frequency of papers that, having declared to have [M]ajor journals simply won't publish replications. This is a real
‘tested’ a hypothesis, reported a positive support for it”. This is all problem: in this age of Research Excellence Frameworks and
the more remarkable given that the late 1980s were themselves other assessments, the pressure is on people to publish in high
not a halcyon period of unbiased science: in an 1987 study of 271 impact journals. Careful replication of controversial results is
unpublished and 1041 published trials, Dickersin et al. found that therefore good science but bad research strategy under these
14% of unpublished and 55% of published trials favoured the pressures, so these replications are unlikely to ever get run. Even
experimental therapy (1987). As Young et al. suggest, “the general when they do get run, they don't get published, further reducing
paucity in the literature of negative data” is such that “[i]n some the incentive to run these studies next time. The field is left with
fields, almost all published studies show formally significant a series of “exciting” results dangling in mid-air, connected only
results so that statistical significance no longer appears dis- to other studies run in the same lab. (2011)
criminating” (2008, 1419).
Another artifact of this positive bias is the “decline effect,” or As Rothstein (2014) argues “The consequences of this problem
the tendency for the strength of evidence for a particular finding include the danger that readers and reviewers will reach the
to decline over time from that stated on its first publication wrong conclusion about what the evidence shows, leading at
(Schooler, 2011; Gonon et al., 2012; Brembs et al., 2013; Groppe, times to the use of unsafe or ineffective treatments”.
2015; Open Science Collaboration, 2015). While this effect is also
well-known, Brembs et al. have recently shown that its presence is
Homophily. Thus far, we have been discussing the negative
significantly positively correlated with journal prestige as
impact of “excellence” largely in terms of its effect on the practice
measured by Impact Factor: early papers appearing in high
and results of professional researchers. There is, however, another
prestige journals report larger effects than subsequent studies
effect of the drive for “excellence”: a restriction in the range of
using smaller samples (2013, see Figs. 1b and 1c in this reference).
scholars, of the research and scholarship performed by such
scholars, and the impact such research and scholarship has on the
The bias against replication. Finally, there is a bias against the larger population. Although “excellence” is commonly presented
publication of replication studies in disciplines where such pat- as the most fair or efficient way to distribute scarce resources
terns make scientific sense. Indeed, there are currently insufficient (Sewitz, 2014), it in fact can have an impoverishing effect on the
structural incentives to perform work that “merely” revalidates very practices that it seeks to encourage. A funding programme

6 PALGRAVE COMMUNICATIONS | 3:16105 | DOI: 10.1057/palcomms.2016.105 | www.palgrave-journals.com/palcomms

PALGRAVE COMMUNICATIONS | DOI: 10.1057/palcomms.2016.105 ARTICLE

that looks to improve a nation’s research capacity by differentially In other words, the works that—and the people who—are
rewarding “excellence” can have the paradoxical effect of redu- considered “excellent” will always be evaluated, like the canon
cing this capacity by underfunding the very forms of “normal” that shapes the culture that transmits it, on a conservative basis:
work that make science function (Kuhn [1962] 2012) or distract past performance by preferred groups helps establish the norms
attention from national priorities and well-conducted research by which future performances of “excellence” are evaluated.
towards a focus on performance measures of North America and Whether it is viewed as a question of power and justice or simply
Europe (Vessuri et al., 2014). A programme that seeks to reward as an issue of lost opportunities for diversity in the cultural co-
Humanists, similarly, by focussing on output in “high impact” production of knowledge, an emphasis on the performance of
academic journals paradoxically reduces the impact of these same “excellence” as the criterion for the distribution of resources and
disciplines by encouraging researchers to focus on their profes- opportunity will always be backwards looking, the product of an
sional peers rather than broader cultural audiences (Readings, evaluative process by institutions and individuals that is
1996), reducing the domain’s relevance even as its performance of established by those who came before and resists disruptive
“excellence” improves. A programme of concentration on the innovation in terms of people as much as ideas or process.
“best” academics, in other words, can have the effect of focussing
attention on problems and approaches in which “excellence” can
be performed most easily rather than those that could benefit the Alternative narratives: working for change
most (or provide the greatest actual impact) from increased If, as we have argued, “excellence” in all its many forms and
attention. meanings is both unreliable as a measure of actual quality, and
Moreover, a concentration on the performance of “excellence” pernicious in the way it promotes poor behaviour and
can promote homophily among the scientists themselves. Given discourages good, what then are the alternatives? Given the
the strong evidence that there is systemic bias within the political realities that have promoted the use of this rhetoric in
institutions of research against women, under-represented ethnic defence of science and scholarship, are there other, less damaging
groups, non-traditional centres of scholarship, and other ways in which we can evaluate and promote the value of research
disadvantaged groups (for a forthright admission of this bias and its communication?
with regard to non-traditional centres of scholarship, see Because “excellence” is used so ubiquituously across the
Goodrich, 1945), it follows that an emphasis on the performance research space, a complete answer to this question is far beyond
of “excellence”—or, in other words, being able to convince the scope of any single paper: there is no single alternative that
colleagues that one is even more deserving of reward than others can replace the rhetoric of “excellence” in scholarly publishing,
in the same field—will create even stronger pressure to conform research funding, government and university policy, public
to unexamined biases and norms within the disciplinary culture: relations, and promotion and tenure practices. In some areas,
challenging expectations as to what it means to be a scientist is a moreover, technological and economic changes suggest fairly
very difficult way of demonstrating that you are the “best” at obvious directions in which progress is being made—a prime
science; it is much easier if your appearance, work patterns, and example being the change from the physical scarcity that
research goals conform to those of which your adjudicators have characterized print journals, adjudication to the abundance that,
previous experience. In a culture of “excellence” the quality of technically at least, characterizes a web-based publication
work from those who do not work in the expected “normative” infrastructure (for well-known discussions of this, see Shirky,
fashion run a serious risk of being under-estimated and 2010; Nielsen, 2012).
unrecognised (King et al., 2014, 2016; O’Connor and O’Hagan, In many ways, however, the greatest challenge is research
2015; University of Arizona Commission on the Status of funding and infrastructure. The continuing competition for
Women, 2015; this is, in part, an explanation for the systemically government and private funds raises questions of prioritization
underreported and poorly acknowledged and rewarded work of and adjudication that are unlikely to be rapidly answered by
women “assistants” in many of the great scientific discoveries of changes in technology or attitudes. A central test of our critique
the twentieth century). There is a clear case to answer that, absent of rhetorics of “excellence” is therefore to ask whether there are
substantial corrective measures and awareness, a focus on any alternatives in this arena. Since funding applications tend to
“excellence” will continue to maintain rather than work to collect examples of “excellence” from other aspects of the research
overcome social barriers to participation in research by currently enterprise as a form of justification (success in funding is a
underrepresented groups. function of one's ability to demonstrate “excellence” in different
Homophily is in some senses a variant on Merton’s “Matthew types of performance), it also represents the apex of the problem.
effect,” discussed above. It is also a variant on the old argument Perhaps because it is so hard, the tendency in policy, at least in
that existing power structures—those populated by those whom it the traditional North Atlantic centres of research in the last
is assumed already exemplify “excellence”—tend towards con- several decades, has clearly been in a non-distributive direction:
servatism in their processes of evaluation. It underpins the calls to for the concentration of resources on “top” institutions (in earlier
reassess the focus of mainstream scholarship, whether this is periods, such as the early space race, for example, the focus was
“great men” history, the “Dead White Male” in literary “canon”, arguably more distributive). The Research Excellence Framework
or the bias towards the ills of the western male patient in medical in the United Kingdom (REF) and massive new research centres
research. As Barbara Herrnstein Smith says with respect to such as the Crick in London are intended to create a “critical
literary evaluation: mass” of “excellent” or “world-leading” research. In Canada,
which is an outlier internationally in the push towards
…[a work that “endures”] will also also begin to perform stratification (Usher, 2016), it remains the case that the “top”
certain characteristic cultural functions by virtue of the very universities (which have their own independent lobby group),
fact that it has endured...In these ways, the canonical work receive a disproportionate share of research resources when
begins increasingly not merely to survive within but to shape measured, for example, against the percentage of students
and create the culture in which its value is produced and (including Doctoral students) they educate (U15 Group of
transmitted and, for that very reason, to perpetuate the Canadian Research Universities/Regroupement des universités
conditions of its own flourishing. (Herrnstein Smith, 1988 de recherche du Canada, 2016). In the much larger U.S. post
emphasis in the original) secondary system, ten universities received nearly 20% of all

PALGRAVE COMMUNICATIONS | 3:16105 | DOI: 10.1057/palcomms.2016.105 | www.palgrave-journals.com/palcomms 7

ARTICLE PALGRAVE COMMUNICATIONS | DOI: 10.1057/palcomms.2016.105

government research funds; as Weigley and Hess note, while governments who need to account for the distribution of public
these universities are among the richest in the country in terms of funds and may fear the optics of a system built on criteria other
their endowments, public funding still constitutes the largest part than “the best”. The narrative and the need for “excellence” (like
of their R&D funding (2013). that of “international competitiveness”) is important as a shared
Many have questioned the value of such an inequitable language of externally recognizable symbols that justify funding
distribution of funds when a less concentrated, or less unequal, to government and to wider publics.
distribution could achieve greater outcomes. Dorothy Bishop As noted earlier, this serves the interests of those who have
argues, with respect to the REF that there should be less of a already “earned” the label. The local construction of “excellence”
disparity between rewarding research that is perceived to be “the is inherently conservative, and maintaining its structures serves
best” and that which is perceived as merely average. Instead, the interests of those who hold local power. Therefore, narratives
Bishop (2013) argues, all research submitted to the REF should arguing for redistribution need to be more than just interesting
receive some funding and the perceived best research should ideas and more than simply factually correct. They need to be
receive a smaller overall proportionate gain. This would have the politically as well as intellectually compelling.
benefit of decreasing the funding gulf between elite and middle-
tier universities and would encourage diversity in the process. Of
course such an approach may be politically troublesome for the Soundness and capacity over “excellence”. This is where a
academy, as long as the criterion it promotes is relative rhetoric built around “soundness” and “capacity” offers oppor-
“excellence” rather than, say, “capacity”, “breadth”, “soundness”, tunities. The idea that “sound research is good research”, and
“comprehensiveness” or “accessibility”. If funding is allocated on “more research is better than less”—that our focus should be on
a scattered basis, following the logic that predictive approaches to thoroughness, completeness, and appropriate standards of
quality are weak at best, then the authority claims of the description, evidence, and probity rather than flashy claims of
university are substantially devalued as long as the rhetoric superiority—presents an alternative to the existing notions of
used to defend them privileges a “winner-take-all” measure of “excellence”. Such a narrative also addresses deeper concerns
effectiveness. regarding a breakdown in research culture through hypercom-
There is, however, a compelling case to be made for the value petition. These terms resonate with public and funder concerns
of greater redistribution of research funding. Cook et al. (2015) for value, and they align with the need for improved commu-
showed that for UK Bioscience groups an optimal allocation of nications and wider engagement encouraged by many govern-
fixed resources would involve spreading the money between a ments and agencies.
larger number of smaller groups. This was the case whether It might be argued in the case of “soundness” in particular that
number of publications or number of citations were used as the the term is as subjective as “excellence”. Stirling (2007a) has
measure of productivity. A similar conclusion is reached by argued that the implication that expert analysis can be free from
Fortin and Currie who argue that scientific impact is only “weakly subjective values in determining something like “soundness” is
money-limited” and that a more productive strategy would be to itself misleading and exclusionary. Certainly “soundness” or
distribute funds based on “diversity” rather than perceptions of “scientificness” rhetorics have been used to give credibility to
“excellence” (Fortin and Currie, 2013). Gordon and Poulin controversial technologies and to shut a range of perspectives out
argued that, for science funding in Canada through the National of public discourse in ways that are similar to uses of “excellence”
Science and Engineering Research Council (NSERC, the main we have criticized.
STEM funding agency), it would have cost less at a whole system But the evaluation of “soundness” is based in the practice of
level simply to distribute the average award to all eligible scholarship, whereas “excellence” is a characteristic of its objects
applicants than to incur the costs associated with preparing, (outputs and actors). In this sense “soundness” aligns well with
reviewing and selecting proposals (2009; although see Roorda, approaches that locate the value of scholarship and evaluation in
2009 for a critique of their calculation). A rough calculation of the the nature of its processes (that is, “proper practice”) and its
system costs of preparing failed grant applications would suggest social conduct. While disagreeing on what the outputs of research
that they are in the same order of magnitude as research grant can actually mean, scholars from Fleck, through Merton, Kuhn,
funding itself (Herbert et al., 2013). Ravetz and Latour have all focussed on how practice in a social
What this suggests is that “excellence” is not the only policy context in which norms and ethics are sustained and enforced
choice concerning the resourcing of research, nor even, leads to productive scholarship (Fleck [1935] 1979; Ravetz, 1973;
necessarily, the only politically compelling one: from concentrat- Latour and Woolgar, 1986; Latour, 1987). “Soundness” can be
ing resources on the most deserving, allegedly “excellent”, assessed by how it supports socially developed and documentable
institutions and researchers, to distributing them amongst all processes and norms. In contrast assessment of “excellence”
those that meet some minimum criteria—or even some subset, by depends on how convincing the performance of importance and
lottery (Health Research Council of New Zealand, 2016; Fang impact is. Like “excellence” the criteria for “soundness” are
et al., 2016), arguments can be made for a variety of different not universal qualities distinct from pre-existing socially devel-
methods of funding research. In the context of scarce resources oped practice; but in contrast to “excellence”, the qualities of
and a desire to maximize outcomes, indeed, there is even an “soundness” can be benchmarked. They are also more precise:
argument for focussing most attention on the worst institutions; “excellence” in the senses we are discussing is used describe the
those that might most benefit from resources to improve (Bishop, competitive position of an entire performance in relation to
2013), have the greatest scope for improvement, and would go the others; “soundness” focusses on details: statistical or bibliographic
longest way to ensuring an increase in basic capacity. In this case, appropriateness, say, or well-chosen evidence.
rather than “excellence” appraisers would be looking for some Another question about “soundness” involves its cross-
sort of baseline level of qualification, “credibility” (Morgan, 2016), disciplinary application. What is “soundness” in the context of
perhaps, or “soundness”. This would be a shift from focussing on the Humanities? Eve (2014, 144) has suggested that “soundness”
evaluation of outputs to an evaluation of practice. in a humanities paper might involve the ability to “evince an
The challenge with any redistributive scheme is how to engage argument; make reference to the appropriate range of extant
with politics. While proposing interesting and valuable thought scholarly literature; be written in good, standard prose of an
experiments, they do not address the needs of working with appropriate register that demonstrates a coherence of form and

8 PALGRAVE COMMUNICATIONS | 3:16105 | DOI: 10.1057/palcomms.2016.105 | www.palgrave-journals.com/palcomms

PALGRAVE COMMUNICATIONS | DOI: 10.1057/palcomms.2016.105 ARTICLE

content; show a good awareness of the field within which it was journals only the papers they consider most important–avoiding,
situated; pre-empt criticisms of its own methodology or for example, “wasting” anybody’s time by submitting “non-
argument; and be logically consistent”. More recently, Morgan original” work such as replication studies. Over time, success in
(2016) has suggested that “credibility” may be the humanities this venture, its own form of hypercompetition, leads to a
equivalent of “soundness”. Others have focussed on the term differentiated set of ranked journals driven by their own
“quality” in the sense in which it used in quality assurance performative targets, or aspirations to join the top ranks. Authors
(Funtowicz and Ravetz, 1990; Funtowicz and Ravetz, 2003), as and editors engage in a cycle of performance that reduces the
fitness for an explicitly defined purpose. As we have argued above breadth of research journals are willing to publish and authors
all of these appear to capture the sense that productive willing to submit.
scholarship can be defined by allegiance to socially defined PLOS ONE and its competitors also compete, but on quite
research practice as much as performance of success. different terms and in ways that arguably improve rather than
Our argument here is not that expanding our boundary for imperil the research enterprise. Speed of publication, for example,
resourcing from “excellence” to “soundness” and “capacity” is all always features in author surveys, and journals like PLOS ONE
that is necessary to change research culture and improve the often advertise their average turnaround times. They even
distribution of resources; rather, it is that a move from resourcing compete on the basis of journal prestige, reputation and Impact
based on the performance of an ineluctable quality to one based Factor (Solomon, 2014), albeit with a heavier emphasis on
on the demonstration of documentable, socially developed soundness and number of publications (that is, capacity) rather
practice, is the first step to solving the problems our rhetoric of than exclusivity and “excellence”. Even when the criteria for
“excellence” has created. Soundness appears be a plausible basis inclusion is only soundness, membership in the club of authors
on which to build a new narrative, or rather to combine existing still provides a prestige benefit: that the doors of the club are more
threads into a more consistent rhetorical framework. Such a open does not necessarily mean that there is no benefit to
framework will work to refocus our attention on research that is membership (Potts et al., 2016).
sufficiently valuable to be worth pursuing. To drive adoption and But PLOS ONE and similar journals also demonstrate that it is
practice towards making this real, however, will require more not simply enough to create mechanisms that test for soundness
than narrative. It will need resources to be redistributed towards and capacity. Even when offered a distributive narrative,
supporting a broader class of research activities. researchers often still find it difficult to avoid the concentrating
rhetoric of “excellence”. A common complaint from the managers
of journals such as PLOS ONE, indeed, is that their journals’
Do soundness and capacity sell? Although we have been referees, who are usually made up of previous authors, often seek
focussing on funding, the rhetoric of soundness and capacity, to reject papers that they feel do not meet their own perceptions
about the idea that the most important quality of research is that of “excellence,” instead of focussing on the journal’s formal
it be done and done with care, does resonate with other aspects of criterion of “soundness”. Many anecdotes from PLOS ONE
the research enterprise. authors, likewise, involve being surprised by how tough the
Some examples of this are the broad area of reproducibility refereeing process was for their articles—a response that signals
(Burman et al., 2010; Lehrer, 2010; Goldacre, 2011; Yong, 2012b; relative “excellence” that might otherwise not be apparent to the
Rehman, 2013; Chang and Li, 2015; Open Science Collaboration, reader (see especially Curry, 2012 and comments). The
2015), reporting guidelines for animal experiments (Kilkenny performance of “excellence”, the signalling of relative superiority
et al., 2010) and clinical trials (Schulz et al., 2010), and work on through an additional line on the CV, is still more important
registered replication studies in social psychology (Simons et al., from a career perspective than the science itself: nobody gets
2014). All have been areas of substantial professional and popular tenure for publishing to arXiv, no matter how good the quality of
discussion and the emphasis on the need for clarity of description their research. At least that appears to be what most tenure-track
and “doing things properly” is consistent. The idea that research academics believe. And while reader attention or online
must be reproducible, safe, and complete can be at least as conversation are gaining some currency as indicators of qualities
compelling an argument as that it must be simply excellent. valued in an article, the current discourse indicates that authors
Another place where the rhetoric of “soundness” and need to feel that they have cleared a higher bar than they in
“capacity” has booked considerable success is the online journal fact have.
PLOS ONE and the journals that have since begun to follow its In other words, initiatives like PLOS ONE will have truly
approach.2 PLOS ONE was launched with the stated aim of succeeded in changing researchers’ own bias towards (ultimately
publishing any scientific research that was deemed technically undemonstrable) “excellence” only when their rejection rate is
sound, regardless of its perceived novelty or impact. This seen to be less important than the evidence that controls are in
approach was made possible by two developments in academic place to ensure and encourage the recognition of “soundness”.
publishing—the move to fully online publications without the
need for print editions, and the growing acceptance of Article
Processing Charge (APC)-funded Open Access as a viable Caveats and further work
publication model. These enabled the journal to consider and The potential scope of the project of this article is huge, and we
publish any manuscript that met its criteria, with no limitations have only been able to touch on some of its aspects. We have
on page space or fixed subscription revenue. As a result, the focused on narratives and rhetoric and sought to bring evidence
journal grew very quickly, becoming the largest journal in the of how existing rhetorics are damaging. What we have not done,
world within 5 years of launching (MacCallum, 2011). as a variety of both anonymous reviewers and non-anonymous
The PLOS ONE model has been widely emulated, with almost commenters have noted, is address the power politics that
every major scientific publisher now offering a journal with underlie many of the structures that we are critiquing. Nor have
similar editorial criteria. This has created a competitive landscape we analysed the degree to which different actors within the
with interesting properties. Traditional journals compete by system are able to enact change.
seeking to publish the most “excellent” papers that they can Understanding how the changes we propose in narrative
attract and demonstrate this by the number of papers they reject. and indeed culture can be achieved politically and institutionally
This also leads authors to self-select for submission to those is a much larger project, one on which others are already engaged

PALGRAVE COMMUNICATIONS | 3:16105 | DOI: 10.1057/palcomms.2016.105 | www.palgrave-journals.com/palcomms 9

ARTICLE PALGRAVE COMMUNICATIONS | DOI: 10.1057/palcomms.2016.105

and one that is critically important in the current political own merits. Unlike concentration, and the hypercompetition to
climate. Institutional change is challenging and slow. We hope which it leads, which break down our standards and cultures
that alongside the criticism, implicit and explicit of some in systematic, predictable, and negative ways, redistribution
existing institutions, we have offered some routes forward to be enhances capacity and breadth of participation. And thirdly, we
investigated and explored. have shown that top-loading of research funding based upon
We have also not undertaken a historical analysis. While we anti-foundational principles of “excellence” is likely to hurt the
draw on literature from a range of periods we have not incremental advances upon which research implicitly relies.
addressed how and when our current narratives developed. The argument for redistribution is a challenging one to
While we would argue that it has deep roots, we have neither the advance. The rhetorics of scarcity, of concentration and
expertise nor the space to probe the history through which competition are linked to strong cultural and economic
excellence rhetorics became institutionalized in their current narratives, particularly in the United Kingdom and United States.
forms. The differing registers and locations of excellence rhetorics But as a route towards this goal we have argued that it is possible
over time—policing access to the right clubs, publication in the to build upon existing narratives of “soundness”, “credibility” and
right journals, career success and contributions to institutional “capacity”—which is to say on narratives of reproducibility,
funding—is deserving of further study and would additionally transparency, high-quality reporting, and a breadth and diversity
strengthen the political analysis. of activity—to build a case for strong cultural practices that focus
on fundamental standards that define proper scholarly and
scientific practice. This focus on the practice of research,
Closing the loop: planning for cultural change including its communications, rather than the performance of
In this article, we have advanced an argument that “excellence” is success at research can also be aligned with developing narratives
not just unhelpful to realising the goals of research and research of Responsible Research and Innovation and public engagement.
communities but actively pernicious. A narrative of scarcity For instance the approach of Post-Normal Science advocated by
combined with “excellence” as an interchange mechanism leads Funtowicz and Ravetz (2003; 1990), focuses on assessing the
to concentration of resources and thence hypercompetition. quality of the process of research practice, and emphasises the
Hypercompetition in turn leads to greater (we might even say need to effectively communicate the weaknesses of any claims
more shameless, see Anderson et al., 2007; Fanelli, 2009; Tijdink made on the basis of research.
et al., 2014; Chubb and Watermeyer, 2016) attempts to perform In taking this approach we root the discourse in long-standing
this “excellence”, driving a circular conservatism and reification traditions and culture, while also engaging with the newer
of existing power structures while harming rather than improving concerns. It is through showing that we can recognize sound and
the qualities of the underlying activity. credible research and that we can build strong cultures and
We have also argued that, while many commentaries reviewed communities around that recognition, that we lay the ground-
throughout this piece lay the blame for this at the feet of external work for making the case for redistribution. And that would be
actors—institutional administrators captured by neo-liberal excellent.
ideologies, funders over-focussed on delivering measurable
returns rather than positive change, governments obsessed with
economic growth at the cost of social or community value—the
roots of the problem in fact lie in the internal narratives of the
Notes
academy and the nature of “excellence” and “quality” as
1 The name of the Matthew Effect is derived from Matthew 13:12: “For whosoever hath,
supposedly shared concepts that researchers have developed into to him shall be given, and he shall have more abundance: but whosoever hath not,
shields of their autonomy. The solution to such problems lies not from him shall be taken away even that he hath”.
in arguing for more resources for distribution via existing 2 As noted in the disclosure of competing interests, three of the authors of this article
channels as this will simply lead to further concentration and have worked for PLOS previously.
hypercompetition. Instead, we have argued, these internal
narratives of the academy must be reformulated. References
Finally, we have argued for a more pluralistic approach to the Aldhous P (2011) Journal Rejects Studies Contradicting Precognition. New
distribution of resources and credit. Where competition does take Scientist, 11 May, https://www.newscientist.com/article/dn20447-journal-
place it should do so on the basis of the many different qualities, rejects-studies-contradicting-precognition/, accessed 19 February.
plural, that are important to different communities using and Alpher RA, Bethe H and Gamow G (1948) The origin of chemical elements.
Physical Review; 73 (7): 803–804.
creating research. But it should also be recognized that Anderson MS, Ronning EA, De Vries R and Martinson BC (2007) The perverse
competition is not, in this context, an unalloyed good. In the effects of competition on scientists’ work and relationships. Science and
context of assessing the risks of application of research Stirling Engineering Ethics; 13 (4): 437–461.
and others argue for “broadening out and opening up” the Andrade R de O (2016) Sharp Rise in Scientific Paper Retractions. University
technology assessment process (Ely et al., 2014, see also Stilgoe, World News, 8 January http://www.universityworldnews.com/article.php?
story = 20160108194308816.
2014), that is to say increasing both the set of criteria considered Azoulay P, Zivin JSG and Manso G (2011) Incentives and creativity: Evidence from
and the range of people who have a voice in its assessment and the academic life sciences. The Rand Journal of Economics; 42 (3): 527–554.
application. The same approach needs to be applied to research Babbage C (1831) Reflections on the Decline of Science in England: And on Some
assessment itself. of Its Causes, by Charles Babbage (1830). To Which Is Added On the Alleged
This leads to our argument for a focus on redistribution instead Decline of Science in England, by a Foreigner (Gerard Moll) with a Foreword by
Michael Faraday (1831). B. Fellowes: London.
of concentration, which, we suggest, is necessary for three core Belluz J (2016) Do ‘Top’ Journals Attract ‘Too Good to Be True’ Results? Vox. 11
reasons. Firstly because “excellence” cannot be recognized or January, http://www.vox.com/2016/1/11/10749636/science-journals-fraud-
defined consensually, except as a Wittgensteinian “beetle in a retractions.
box” that no-one has ever seen, and even then, unlike Bem D (2011) Feeling the future: Experimental evidence for anomalous retroactive
Wittgenstein’s beetle-owners, by researchers who cannot agree influences on cognition and affect. Journal of Personality and Social Psychology;
100 (3): 407–425.
even within disciplinary communities on which aspects of Bertamini M and Munafò MR (2012) Bite-size science and its undesired side
“excellence” might matter or be useful. Second because, as we effects. Perspectives on Psychological Science: A Journal of the Association for
have argued, there is a case to be made for redistribution on its Psychological Science; 7 (1): 67–71.

10 PALGRAVE COMMUNICATIONS | 3:16105 | DOI: 10.1057/palcomms.2016.105 | www.palgrave-journals.com/palcomms

PALGRAVE COMMUNICATIONS | DOI: 10.1057/palcomms.2016.105 ARTICLE

Bishop D (2013) The Matthew Effect and REF2014. BishopBlog, http://deevybee. Fang FC, Bowen A and Casadevall A (2016) NIH peer review percentile scores are
blogspot.ca/2013/10/the-matthew-effect-and-ref2014.html, accessed 15 Octo- poorly predictive of grant productivity. eLife; 5 (February). doi:10.7554/
ber. eLife.13323.
Bissell M (2013) Reproducibility: The risks of the replication drive. Nature; 503 Fang FC, Steen RG and Casadevall A (2012) Misconduct accounts for the majority
(7476): 333–334. of retracted scientific publications. Proceedings of the National Academy of
Brembs B (2015) The cost of the rejection-resubmission cycle. The Winnower. Sciences of the United States of America; 109 (42): 17028–17033.
doi:10.15200/winn.142497.72083. Fitzpatrick K (2011) Planned Obsolescence. New York University Press: New York.
Brembs B, Button K and Munafò M (2013) Deep impact: Unintended Fleck L (1979) Genesis and Development of a Scientific Fact; Bradley F and Trenn
consequences of journal rank. Frontiers in Human Neuroscience; 7, 291. TJ (trans). Trenn TJ and Merton RK (eds). University of Chicago Press:
Burman LE, Reed WR and Alm J (2010) A call for replication studies. Public Chicago, IL.
Finance Review; 38 (6): 787–793. Fortin J-M and Currie DJ (2013) Big science vs. Little science: How scientific
Calcagno V, Demoinet E, Gollner K, Guidi L, Ruths D and de Mazancourt C (2012) impact scales with funding. PLoS One; 8 (6): e65263.
Flows of research manuscripts among scientific journals reveal hidden Funtowicz SO and Ravetz JR (1990) Uncertainty and Quality in Science for Policy.
submission patterns. Science; 338 (6110): 1065–1069. Theory and Decision Library A-Springer: The Netherlands.
Campanario JM (1993) Consolation for the scientist: Sometimes it is hard to Funtowicz SO and Ravetz JR (2003) Post-Normal Science. In: International Society
publish papers that are later highly cited. Social Studies of Science; 23, 342–362. for Ecological Economics Internet Encyclopaedia of Ecological Economics. http://
Campanario JM (1995) Commentary on influential books and journal articles isecoeco.org/pdf/pstnormsc.pdf.
initially rejected because of negative referees’ evaluations. Science Communica- Gans JS and Shepherd GB (1994) How are the mighty fallen: Rejected classic
tion; 16 (3): 304–325. articles by leading economists. The Journal of Economic Perspectives: A Journal
Campanario JM (1996) Have referees rejected some of the most-cited articles of of the American Economic Association; 8 (1): 165.
all times? Journal of the American Society for Information Science; 47 (4): 302–310. Goldacre B (2011) I Foresee That Nobody Will Do Anything about This Problem.
Campanario JM (2009) Rejecting and resisting Nobel class discoveries: Accounts by Bad Science. 23 April, http://www.badscience.net/2011/04/i-foresee-that-
Nobel Laureates. Scientometrics; 81 (2): 549–565. nobody-will-do-anything-about-this-problem/.
Campanario JM and Acedo E (2007) Rejecting highly cited papers: The views of Gonon F, Konsman J-P, Cohen D and Boraud T (2012) Why most biomedical
scientists who encounter resistance to their discoveries from other scientists. findings echoed by newspapers turn out to be false: The case of attention deficit
Journal of the American Society for Information Science and Technology; 58, hyperactivity disorder. PLoS One; 7 (9): e44275.
734–743. Goodrich DW (1945) An analysis of manuscripts received by the editors of the
Casadevall A and Fang FC (2012) Winner takes all. Scientific American; 307 (2): 13. American Sociological Review from May 1, 1944 to September 1, 1945. American
Chang AC and Li P (2015) Is Economics Research Replicable? Sixty Published Sociological Review; 10 (6): 716–725.
Papers from Thirteen Journals Say ‘Usually Not.’ 2015-083. Finance and Gordon R and Poulin BJ (2009) Cost of the NSERC science grant peer review
Economics Discussion Series. Washington DC: Board of Governors of the system exceeds the cost of giving every qualified researcher a baseline grant.
Federal Reserve System. http://www.federalreserve.gov/econresdata/feds/2015/ Accountability in Research; 16 (1): 13–40.
files/2015083pap.pdf. Grieneisen ML and Zhang M (2012) A comprehensive survey of retracted articles
Chen C, Hu Z, Milbank J and Schultz T (2013) A visual analytic study of retracted from the scholarly literature. PLoS One; 7 (10): e44118.
articles in scientific literature. Journal of the American Society for Information Groppe DM (2015) Combating the scientific decline effect with confidence
Science and Technology; 64 (2): 234–253. (intervals) BioRχiv. doi:10.1101/034074.
Chubb J and Watermeyer R (2016) Artifice or integrity in the marketization of Guedj D (2009) Nicholas Bourbaki, collective mathematician: An interview with
research impact? Investigating the moral economy of (pathways to) impact Claude Chevalley. The Mathematical Intelligencer; 7 (2): 18–22.
statements within research funding proposals in the UK and Australia. Studies Hassell MP and May RM (1974) Aggregation of predators and insect parasites and
in Higher Education; 1–13. its effect on stability. The Journal of Animal Ecology; 43 (2): 567–594.
Cicchetti DV (1991) The reliability of peer review for manuscript and grant Health Research Council of New Zealand (2016) Explorer Grants. Health Research
submissions: A cross-disciplinary investigation. The Behavioral and Brain Council, http://www.hrc.govt.nz/funding-opportunities/researcher-initiated-pro
Sciences; 14 (1): 119–135. posals/explorer-grants, accessed 19 February.
Claxton LD (2005) Scientific authorship. Part 1. A window into scientific fraud? Herbert DL, Barnett AG, Clarke P and Graves N (2013) On the time spent
Mutation Research; 589 (1): 17–30. preparing grant proposals: An observational study of Australian researchers.
Cook I, Grange S and Eyre-Walker A (2015) Research groups: How big should BMJ Open; 3 (5): e002800.
they be? PeerJ; 3 (June): e989. Herrnstein Smith B (1988) Contingencies of Value: Alternative Perspectives for
Costello LC (2010) Perspective: Is NIH funding the ‘best science by the best Critical Theory. Harvard University Press: Cambridge, MA.
scientists’? A critique of the NIH R01 research grant review policies:. Academic Hill H and Pitt J (2014) Failure to replicate: A sign of scientific misconduct?
Medicine: Journal of the Association of American Medical Colleges; 85 (5): Publications; 2 (3): 71–82.
775–779. Hoover WG, Moran B, Holian BL, Posch HA and Bestiale S (1988) Computer
Curry S (2012) PLoS ONE: From the Public Library of Sloppiness? Reciprocal simulation of nonequilibrium processes. In: Schmidt SC and Homes NC (eds)
Space, http://occamstypewriter.org/scurry/2012/04/01/plos1-public-library-of- Shock Waves in Condensed Matter 1987; North-Holland. pp 191–194.
sloppiness/, accessed 1 April. Hoover WG, Posch HA and Bestiale S (1987) Dense‐fluid Lyapunov spectra via
Dean DG (1989) Structural constraints and the publications dilemma: A review constrained molecular dynamics. The Journal of Chemical Physics; 87 (11):
and some proposals. The American Sociologist; 20 (2): 181–187. 6665–6670.
Dickersin K (2005) Publication bias: Recognizing the problem, understanding its Ioannidis JPA, Boyack KW, Small H, Sorensen AA and Klavans R (2014)
origins and scope, and preventing harm. In: Rothstein HR, Sutton AJ and Bibliometrics: Is your most cited work your best? Nature; 514 (7524): 561–562.
Borenstein M (eds.) Publication Bias in Meta-Analysis; John Wiley & Sons: Kennedy D (2004) The old file-drawer problem. Science; 305 (5683): 451.
Chichester, UK, pp 9–33 Kilkenny C, Browne WJ, Cuthill IC, Emerson M and Altman DG (2010) Improving
Dickersin K, Chan S, Chalmersx TC, Sacks HS and Smith H (1987) Publication bias bioscience research reporting: The ARRIVE guidelines for reporting animal
and clinical trials. Controlled Clinical Trials; 8 (4): 343–353. research. PLoS Biology; 8 (6): e1000412.
Dobbs D (2006) Trial and Error. The New York Times 15 January, http://www. King M, West JD, Jacquet J, Correll S and Bergstrom CT (2014) Gender
nytimes.com/2006/01/15/magazine/15wwln_idealab.html?_r=0. Composition of Scholarly Publications. Eigenfactor, http://www.eigenfactor.org/
Ely A, Van Zwanenberg P and Stirling A (2014) Broadening out and opening up gender/self-citation/, accessed 7 January.
technology assessment: Approaches to enhance international development, co- King MM, Bergstrom CT, Correll SJ, Jacquet J and West JD (2016) Men set their
ordination and democratisation. Research Policy; 43 (3): 505–518. own cites high: Gender and self-citation across fields and over time. arXiv
Eve MP (2014) Open Access and the Humanities: Contexts, Controversies and the [physics.soc-ph], http://arxiv.org/abs/1607.00376.
Future. Cambridge University Press: Cambridge, UK. Kohn A (1999) Punished by Rewards: The Trouble with Gold Stars, Incentive Plans,
Excellence Academy. (2016) Indiana University, http://iuhoosiers.com/sports/2015/ A’s, Praise, and Other Bribes. Houghton Mifflin: Boston, MA.
6/25/GEN_0625153134.aspx, accessed 9 September. Kuhn TS ((1962) 2012) The Structure of Scientific Revolutions, Fourth edition,
Eyre-Walker A and Stoletzki N (2013) The assessment of science: The relative University of Chicago Press: Chicago, IL.
merits of post-publication review, the impact factor, and the number of Labbé C (2010) Ike Antkare: One of the great stars in the scientific firmament. ISSI
citations. Edited by Jonathan A. Eisen. PLoS Biology; 11 (10): e1001675. Newsletter; 6 (2): 48–52.
Fanelli D (2009) How many scientists fabricate and falsify research? A systematic Lamont M (2009) How Professors Think: Inside the Curious World of Academic
review and meta-analysis of survey data. PLoS One; 4 (5): e5738. Judgment. Harvard University Press: Cambridge, MA.
Fanelli D (2011) Negative results are disappearing from most disciplines and Latour B (1987) Science in Action: How to Follow Scientists and Engineers Through
countries. Scientometrics; 90 (3): 891–904. Society. Harvard University Press: Cambridge, MA.

PALGRAVE COMMUNICATIONS | 3:16105 | DOI: 10.1057/palcomms.2016.105 | www.palgrave-journals.com/palcomms 11

ARTICLE PALGRAVE COMMUNICATIONS | DOI: 10.1057/palcomms.2016.105

Latour B and Woolgar S (1986) Laboratory Life: The Construction of Scientific Potts J, Hartley J, Montgomery L, Neylon C and Rennie E (2016) A Journal is a
Facts. Princeton University Press: Princeton, NJ. club: A new economic model for scholarly publishing. Social Science Research
Lawrence PA (2007) The mismeasurement of science. Current Biology; 17 (15): Network (April). doi:10.2139/ssrn.2763975.
R583–R585. Psych Filedrawer (2016) The Filedrawer Problem. PsychFileDrawer.org, http://
Lehrer J (2010) The Truth Wears Off. New Yorker 13 December, http://www. www.psychfiledrawer.org/TheFiledrawerProblem.php, accessed 19 February.
newyorker.com/magazine/2010/12/13/the-truth-wears-off. Public Library of Science. (2016) Who we are. PLoS, https://www.plos.org/who-we-
Lerner J and Wulf J (2006) Innovation and Incentives: Evidence from Corporate are, accessed 12 May.
R&D. W11944. National Bureau of Economic Research, http://www.nber.org/ Ravetz JR (1973) Scientific Knowledge and Its Social Problems. Penguin Books:
papers/w11944.pdf. London, UK.
Lindner MD and Nakamura RK (2015) Examining the predictive validity of NIH Readings B (1996) The University in Ruins. Harvard University Press: Cambridge, MA.
peer review scores. PLoS One; 10 (6): e0126938. Rehman J (2013) Cancer Research in Crisis: Are the Drugs We Count on Based on
Lord RG, de Vader CL and Alliger GM (1986) A meta-analysis of the relation Bad Science? Salon, 1 September, http://www.salon.com/2013/09/01/is_cancer_
between personality traits and leadership perceptions: An application of validity research_facing_a_crisis/.
generalization procedures. The Journal of Applied Psychology; 71 (3): 402. Resnik DB and Dinse GE (2013) Scientific retractions and corrections related to
Lundh A, Barbateskovic M, Hróbjartsson A and Gøtzsche PC (2010) Conflicts of misconduct findings. Journal of Medical Ethics; 39 (1): 46–50.
interest at medical journals: The influence of industry-supported randomised Resnik DB, Wager E and Kissling GE (2015) Retraction policies of top scientific
trials on journal impact factors and revenue—Cohort study. PLoS Medicine; 7 journals ranked by impact factor. Journal of the Medical Library Association;
(10): e1000354. 103 (3): 136–139.
MacCallum CJ (2011) Why ONE is more than 5. PLoS Biology; 9 (12): e1001235. Roderick GK and Gillespie RG (1998) Speciation and phylogeography of Hawaiian
Marcovitch H (2010) Editors, publishers, impact factors, and reprint income. PLoS terrestrial arthropods. Molecular Ecology; 7 (4): 519–531.
Medicine; 7 (10): e1000355. Roorda S (2009) The real cost of the NSERC peer review is less than 5% of a
Matzinger P and Mirkwood G (1978) In a fully H-2 incompatible chimera, T cells proposed baseline grant. Accountability in Research; 16 (4): 229–231.
of donor origin can respond to minor histocompatibility antigens in association Rothstein HR (2014) Publication bias. In: Wiley StatsRef: Statistics Reference
with either donor or host H-2 type. The Journal of Experimental Medicine; 148 Online. John Wiley & Sons.
(1): 84–92. Schooler J (2011) Unpublished results hide the decline effect. Nature; 470 (7335):
Meng W (2016) Peer Review: Is NIH Rewarding Talent? Science Transparency, 437.
https://scienceretractions.wordpress.com/2016/01/10/peer-review-is-nih-reward Schulz KF, Altman DG, Moher D and CONSORT Group (2010) CONSORT 2010
ing-talent/, accessed 10 January. statement: Updated guidelines for reporting parallel group randomised trials. BMJ;
Merton RK (1968) The Matthew Effect in science. Science; 159 (3810): 56–63. 340 (March): c332.
Merton RK (1988) The Matthew Effect in science, II: Cumulative advantage and Secord JA (2015) Visions of Science: Books and Readers at the Dawn of the
the symbolism of intellectual property. Isis; An International Review Devoted to Victorian Age. University of Chicago Press: Chicago, IL.
the History of Science and Its Cultural Influences; 79 (4): 606–623. Şekercioğlu ÇH (2013) Citation opportunity cost of the high impact factor
Miller S (2015) Excellence and Equity: The National Endowment for the obsession. Current Biology; 23 (17): R701–R702.
Humanities. University Press of Kentucky: Lexington, KY. Sewitz S (2014) The Excellence Agenda Is a Trojan Horse for Austerity. Research,
Moran B, Hoover WG and Bestiale S (2016) Diffusion in a periodic Lorentz gas. http://www.researchresearch.com/index.php?option=com_news&template=rr_
Journal of Statistical Physics; 48 (3–4): 709–726. 2col&view=article&articleId=1346207, accessed 3 September.
Morgan D (2016) Lessons Learned, and How the Landscape Has Already Changed. Shirky C (2010) Cognitive Surplus: Creativity and Generosity in a Connected Age.
Lecture presented at the Open Access @ UNT/Library Publishing Forum, Penguin Press: New York.
University of North Texas, 19 May, https://openaccess.unt.edu/symposium/ Siler K, Lee K and Bero L (2015) Measuring the effectiveness of scientific
2016/live-streaming-oa-uc-press-lessons-learned-and-how-landscape-has- gatekeeping. Proceedings of the National Academy of Sciences; 112 (2): 360–365.
already-changed. Simons DJ, Holcombe AO and Spellman BA (2014) An introduction to registered
Mrs Kinpaisby (2008) Taking stock of participatory geographies: envisioning the replication reports at Perspectives on Psychological Science. Perspectives on
communiversity. Transactions of the Institute of British Geographers; 33 (3): Psychological Science; 9 (5): 552–555.
292–299. Smith R (2006) Research misconduct: The poisoning of the well. Journal of the
Munafò MR, Stothart G and Flint J (2009) Bias in genetic association studies and Royal Society of Medicine; 99 (5): 232–237.
impact factor. Molecular Psychiatry; 14 (2): 119–120. Solomon DJ (2014) A survey of authors publishing in four megajournals. PeerJ; 2
Networks of Centres of Excellence of Canada, Communications. (2015) Networks (April): e365.
of Centres of Excellence, http://www.nce-rce.gc.ca/index_eng.asp, accessed 13 Steen RG (2011) Retractions in the scientific literature: Is the incidence of research
April. fraud increasing? Journal of Medical Ethics; 37 (4): 249–253.
Nicholson JM and Ioannidis JPA (2012) Research grants: Conform and be funded. Sterling TD (1959) Publication decisions and their possible effects on inferences
Nature; 492 (7427): 34–36. drawn from tests of significance—Or vice versa. Journal of the American
Nielsen MA (2012) Reinventing Discovery: The New Era of Networked Science. Statistical Association; 54 (285): 30–34.
Princeton University Press: Princeton, NJ. Stilgoe J (2014) Against Excellence. The Guardian 19 December, https://www.
Nosek BA, Spies JR and Motyl M (2012) Scientific Utopia: II. Restructuring theguardian.com/science/political-science/2014/dec/19/against-excellence.
incentives and practices to promote truth over publishability. Perspectives on Stirling A (2007a) ‘Opening up’ and ‘Closing down’: Power, participation, and
Psychological Science: A Journal of the Association for Psychological Science; 7 pluralism in the social appraisal of technology. Science, Technology & Human
(6): 615–631. Values; 33 (2): 262–294.
Nowotny H (2014) Excellence Attracts Excellence--and What about the Rest? Stirling A (2007b) A general framework for analysing diversity in science,
Reflections on Excellence and Inclusion. Lecture presented at the EMBO–EMBL technology and society. Journal of the Royal Society, Interface / the Royal Society;
Anniversary Science and Policy Meeting, Heidelberg, July, http://www.helga- 4 (15): 707–719.
nowotny.eu/downloads/helga_nowotny_b160.pdf. Sustaining Excellence Awards.. (2016) Wellcome Trust, http://www.wellcome.ac.uk/
O’Connor P and O’Hagan C (2015) Excellence in university academic staff Funding/Public-engagement/Funding-schemes/Sustaining-Excellence-Awards/
evaluation: A problematic reality? Studies in Higher Education; 41 (11): index.htm, accessed 19 May.
1943–1957. Tartamelia V (2014) The True Story of Stronzo Bestiale (and Other Scientific
O’Donnell DP (2015) Could We Design Comparative Metrics That Would Favour Jokes). Parolacce, http://www.parolacce.org/2014/10/05/the-true-story-of-
the Humanities? Daniel Paul O’Donnell, http://people.uleth.ca/ ~ daniel.odon stronzo-bestiale/, 5 October.
nell/Teaching/could-we-design-comparative-metrics-that-would-favour-the- The Economist. (2015) The World Is Going to University, http://www.economist.
humanities, accessed 29 March. com/news/leaders/21647285-more-and-more-money-being-spent-higher-educa
OECD (2014) Chapter 6. The German Excellence Initiative. In: Promoting Research tion-too-little-known-about-whether-it.
Excellence. OECD Publishing, pp 145–163. The University of Tokyo Global Center of Excellence. (2016) Global Center of
Office of Excellence and Multicultural Student Success (2016) University of Toledo. Excellence for Mechanical Systems Innovation. The University of Tokyo Global
http://www.utoledo.edu/success/excel/, accessed 9 September. COE, http://www.u-tokyo.ac.jp/coe/english/list/category2/base7/summary.html
Open Science Collaboration. (2015) Estimating the reproducibility of psychological accessed 12 May.
science. Science; 349 (6251): aac4716. Tian X and Wang TY (2011) Tolerance for failure and corporate innovation. The
Pagano M (2006) American Idol and NIH grant review. Cell; 126 (4): 637–638. Review of Financial Studies (December). doi:10.1093/rfs/hhr130.
Peters DP and Ceci SJ (1982) Peer-review practices of psychological journals: The Tijdink JK, Verbeke R and Smulders YM (2014) Publication pressure and scientific
fate of published articles, submitted again. The Behavioral and Brain Sciences; 5 misconduct in medical scientists. Journal of Empirical Research on Human
(2): 187–195. Research Ethics; 9 (5): 64–71.

12 PALGRAVE COMMUNICATIONS | 3:16105 | DOI: 10.1057/palcomms.2016.105 | www.palgrave-journals.com/palcomms

PALGRAVE COMMUNICATIONS | DOI: 10.1057/palcomms.2016.105 ARTICLE

U15 Group of Canadian Research Universities/Regroupement des universités de Author Contributions

recherche du Canada. (2016) Our Impact. U15, http://u15.ca/our-impact, The corresponding author is cn. Author contributions, described using the CASRAI
accessed 18 May. CRedIT typology (http://casrai.org/credit), are as follows: conceptualization: me, sm, cn,
University Grants Commission. (2016) Universities (UPE). University Grants dod, dp; methodology: me, sm, cn, dod, dp; investigation: me, sm, cn, dod, dp; resources:
Commission, http://www.ugc.ac.in/page/Universities-(UPE).aspx, accessed 19 me, sm, cn, dod, dp; writing – original draft preparation: me, cn, dod; writing – review
February. and editing: me, sm, cn, dod, dp; funding acquisition: sm.
University of Arizona Commission on the Status of Women. (2015) Avoiding
Gender Bias in Reference Writing. University of Arizona: Tucson, AZ, Acknowledgements
http://www.csw.arizona.edu/sites/default/files/csw_2015-10-20_lorbias_pdf_0.
In keeping with our argument, and following an extensive tradition of subverting tra-
pdf.
ditional scarce markers of prestige, the authors have adopted a redistributive approach to
USC Center for Excellence in Teaching. (2016) http://cet.usc.edu/, accessed 12
the order of their names in the byline. As an international collaboration of uniformly nice
May.
people (cf. Moran et al., 2016; Hoover et al., 1987; see Tartamelia, 2014 for an expla-
Usher A (2016) Massification Causes Stratification. Higher Education Strategy
nation), lacking access to a croquet field (cf. Hassell and May, 1974), writing as indivi-
Associates, http://higheredstrategy.com/massification-causes-stratification/,
duals rather than an academic version of the Borg (see Guedj, 2009), and not identifying
accessed 5 May.
any excellent pun (cf. Alpher et al., 1948; Lord et al., 1986) or “disarmingly quaint nom
Vessuri H, Guedon J-C, Cetto and Mara A (2014) Excellence or quality? Impact of
de guerre” (cf. Mrs Kinpaisby, 2008, 298 [thanks to Oli Duke-Williams for this
the current competition regime on science and scientific publishing in Latin
reference]) to be made from the ordering of our names, we elected to assign index
America and its implications for development. Current Sociology; 62 (5):
numbers to our surnames and randomize these using an online tool. For the avoidance of
647–665.
doubt, while several of the authors have pets, none of them are included as authors (cf.
Wang J, Veugelers R and Stephan PE (2016) Bias against novelty in science: A
Matzinger and Mirkwood, 1978); none of us are approaching a tenure decision (cf.
cautionary tale for users of bibliometric indicators. Social Science Research
Roderick and Gillespie, 1998); and none of us are fictional entities who generate their
Network; (January). doi:10.2139/ssrn.2710572.
papers algorithmically using SciGen (see Labbé, 2010 for the contrasting case of “Ike
Weigley S and Hess AEM (2013) Universities Getting the Most Government
Antkare,” who nevertheless greatly outranked all the authors of this paper on several
Money. 247wallst.com, http://247wallst.com/special-report/2013/04/25/universi
formal measures of excellence before being outed). This article arose from a meeting at
ties-getting-the-most-government-money/, accessed 25 April.
the Triangle Scholarly Communications Institute funded by the Andrew W. Mellon
Weller AC (2001) Editorial Peer Review: Its Strengths and Weaknesses. Information
Foundation. The authors wish to thank Ben Johnson, and commenters on Hacker News
Today: Medford NJ.
for their criticisms and comments on an earlier version of this manuscript.
Wilson A (2011) Failing to Replicate Bem’s Ability to Get Published in a Major
Journal. Notes from Two Scientific Psychologists, http://psychsciencenotes.
blogspot.ca/2011/05/failing-to-replicate-bems-ability-to.html, accessed 7 May. Additional information
Wittgenstein L (2001) Philosophical Investigations: The German Text, with a Competing interests: Moore, Neylon, and Pattinson are all previous employees of PLOS.
Revised English Translation. Blackwell: Oxford, UK. Eve and O'Donnell declare no competing interests.
Yong Ed. (2012a) A Failed Replication Draws a Scathing Personal Attack from a
Psychology Professor. Not Exactly Rocket Science, http://blogs.discovermaga Reprints and permission information is available at http://www.palgrave-journals.com/
zine.com/notrocketscience/2012/03/10/failed-replication-bargh-psychology- pal/authors/rights_and_permissions.html
study-doyen/#.VsZpH0Leezc, accessed 10 March. How to cite this article: Moore S et al. (2017) “Excellence R Us”: university research and
Yong E (2012b) Replication studies: Bad copy. Nature; 485 (7398): 298–300. the fetishisation of excellence. Palgrave Communications. 3:16105 doi: 10.1057/
Young NS, Ioannidis JPA and Al-Ubaydli O (2008) Why current publication palcomms.2016.105.
practices may distort science. PLoS Medicine; 5 (10). doi:10.1371/journal.
pmed.0050201.
Young SS and Bang H (2004) The file-drawer problem, revisited. Science; 306 This work is licensed under a Creative Commons Attribution 4.0
(5699): 1133–1134. International License. The images or other third party material in this
Zankl H (2003) Fälscher, Schwindler, Scharlatane. Erlebnis Wissenschaft. Wiley- article are included in the article’s Creative Commons license, unless indicated otherwise
VCH Verlag: Weinheim. in the credit line; if the material is not included under the Creative Commons license,
users will need to obtain permission from the license holder to reproduce the material.
To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/
Data availability
Data sharing is not applicable as no datasets were analysed or generated during this
study.

PALGRAVE COMMUNICATIONS | 3:16105 | DOI: 10.1057/palcomms.2016.105 | www.palgrave-journals.com/palcomms 13