Biological Conservation 238 (2019) 108199

Contents lists available at ScienceDirect

Biological Conservation
journal homepage: www.elsevier.com/locate/biocon

Building a tool to overcome barriers in research-implementation spaces: The T

conservation evidence database
⁎
William J. Sutherlanda,b, , Nigel G. Taylorc, Douglas MacFarlaned, Tatsuya Amanoe,
Alec P. Christiea, Lynn V. Dicksf, Anaëlle J. Lemassong, Nick A. Littlewooda, Philip A. Martina,b,
Nancy Ockendona, Silviu O. Petrovana, Rebecca J. Robertsonh, Ricardo Rochaa,
Gorm E. Shackelforda,b, Rebecca K. Smitha, Elizabeth H.M. Tylera, Claire F.R. Wordleya
a
Conservation Science Group, Department of Zoology, University of Cambridge, The David Attenborough Building, Pembroke Street, Cambridge CB2 3QZ, UK
b
BioRISC, St. Catharine's College, Cambridge CB2 1RL, UK
c
Tour du Valat, Research Institute for the Conservation of Mediterranean Wetlands, Arles 13200, France
d
School of Psychology, University of Western Australia, Australia
e
School of Biological Sciences, Goddard Building (8), The University of Queensland, Brisbane, 4072, Queensland, Australia
f
School of Biological Sciences, University of East Anglia, Norwich NR4 7TJ, UK
g
Joint Nature Conservation Committee, JNCC, Peterborough PE1 1JY, UK
h
School of Biology, University of Leeds, Leeds LS2 9JT, UK

A R T I C LE I N FO A B S T R A C T

Keywords: Conservation practitioners, policy-makers and researchers work within shared spaces with many shared goals.
Evidence-based conservation Improving the flow of information between conservation researchers, practitioners and policy-makers could lead
Evidence-based policy to dramatic gains in the effectiveness of conservation practice. However, several barriers can hinder this transfer
Evidence-based practice including lack of time, inaccessibility of evidence, the real or perceived irrelevance of scientific research to
Delphi technique
practical questions, and the politically motivated spread of disinformation. Conservation Evidence works to
Subject-wide evidence synthesis
Research-implementation space
overcome these barriers by providing a freely-available database of summarized scientific evidence for the ef-
fects of conservation interventions on biodiversity. The methods used to build this database – a combination of
discipline-wide literature searching and subject-wide evidence synthesis – have been developed over the last
15 years to address the challenges of synthesizing large volumes of evidence of varying quality and measured
outcomes. Here, we describe the methods to enhance understanding of the database and how it should be used.
We discuss how the database can help to expand multi-directional information transfers between research,
practice and policy, which should improve the implementation of evidence-based conservation and, ultimately,
achieve better outcomes for biodiversity.

1. Introduction This problem has been widely conceptualized as a “research-im-

plementation gap” (Anon, 2007; Knight et al., 2008; Westgate et al.,
Despite efforts to conserve it, biodiversity is being lost at an 2018, see Glossary in Supplementary Material). More recently, it has
alarming and increasing rate (Dirzo et al., 2014; Ripple et al., 2017). been reconceptualized as an issue within a series of “research-im-
Research on the effectiveness of conservation interventions is critical to plementation spaces”: arenas in which various stakeholders and interest
ensure conservation efforts are beneficial, efficient, and not creating groups interact, collaborate and learn together (Toomey et al., 2017).
additional harms (Cardinale et al., 2012). The number of publications This concept explicitly recognizes the existing connections between
evaluating the impact of conservation-relevant interventions is growing research and practice rather than implying there are voids between
annually, but the lessons learned are often not employed in manage- research and practice that need to be filled, as well as the broader
ment decisions or policy (Sutherland et al., 2004; Young and Van context in which scientific knowledge is produced and utilized.
Aarde, 2011). Within research-practice and research-policy spaces, several clearly

⁎
Corresponding author at: Conservation Science Group, Department of Zoology, University of Cambridge, The David Attenborough Building, Pembroke Street,
Cambridge CB2 3QZ, UK.
E-mail address: [email protected] (W.J. Sutherland).

https://doi.org/10.1016/j.biocon.2019.108199
Received 28 March 2019; Received in revised form 28 June 2019; Accepted 2 August 2019
0006-3207/ © 2019 Published by Elsevier Ltd.
W.J. Sutherland, et al. Biological Conservation 238 (2019) 108199

Table 1
Some barriers that inhibit interaction between research and practice/policy, and how the Conservation Evidence database helps to overcome or lower these barriers.
Suggested barriers between conservation research and practice/policy Example references How the Conservation Evidence database helps to overcome
or lower this barrier

Research produces fragmented information that often does not address McNie, 2007; Roux et al., 2006; Knight Practitioners/policy-makers suggest interventions to be
questions or problems relevant to conservation practice/policy. et al., 2008; Bainbridge, 2014; Gossa included in the database. Interventions with little evidence
et al., 2015; Rose et al., 2018 are highlighted for researchers. Thus, the database can act as
a source of inspiration for practice/policy-relevant research.
The database includes practice/policy-relevant research e.g.
published in the journal Conservation Evidence or reports
from conservation organizations.
Practitioners/policy-makers need answers more quickly than they can Bainbridge, 2014; Gossa et al., 2015 Database is created proactively, reviewing the evidence for
be produced by research, or even reviews of existing research. all interventions before a specific request from practitioners/
policy-makers. Evidence synthesis prioritizes breadth of
interventions covered over depth of review for each
intervention to provide some synthesized evidence for all
interventions, rather than detailed synthesis for few
interventions.
Locating and accessing relevant primary literature is often too time- Pullin and Knight, 2005; Gossa et al., Database can be queried using search terms or with various
consuming (due to the large volume of published literature, 2015; Westgate et al., 2018 filters (subjects, countries, threats, actions). Evidence within
including much that is not relevant to practitioners/policy-makers) each subject is organized in a consistent way (interventions
grouped under threat and action categories).
Even when primary literature is located, reading papers can be time Pullin and Knight, 2005; Bainbridge, Database contains short, summaries (usually < 200 words)
consuming, as much research is not streamlined for practitioners/ 2014; Westgate et al., 2018 of each study, plus key messages to guide users through the
policy-makers. summary paragraphs.
Much of the primary literature is technical and difficult to interpret for Pullin and Knight, 2005; Roux et al., Content of database is in plain English, avoiding jargon
non-specialists. Research is often not communicated effectively for 2006; Bainbridge, 2014; Rose et al., where possible (and explaining it otherwise).
non-scientists. 2018
Primary literature may be in a foreign language. Arlettaz et al., 2010; Gossa et al., 2015 Summaries are written in English, even for primary articles
not in English. We appreciate this introduces language
barriers for users for whom English is a foreign language.
Financial barriers can be prohibitive (journal articles are often hidden Arlettaz et al., 2010; Gossa et al., 2015 Database and related outputs are free to access.
behind paywalls, which can be too expensive for conservation
practitioners/policy-makers; books can also be too expensive).
Practitioners/policy-makers do not trust that the research, or synthesis, Bainbridge, 2014 Summary paragraphs include key information (e.g. study
is credible. design, raw data, major reported caveats) to allow users to
make some judgement about study quality (internal
validity).
Methods used to produce synthesis are reported alongside
the database.
The uptake of evidence is often undermined by socio-political agendas, Hameleers and van der Meer, 2019; A small contribution: the Conservation Evidence database
whereby practitioners/policy makers tend to accept Ecker and Ang, 2019 serves as an independent fact-checking resource to help
information—or disinformation—that confirms pre-existing debunk disproven or unfounded claims.
worldviews but be critical of evidence in conflict.

defined barriers limit collaboration and coproduction of knowledge Cochrane Reviews; Cochrane Library, 2019) and financial barriers (e.g.
(Roux et al., 2006; van Kerkhoff and Lebel, 2015; Table 1). These in- paywalls, although Environmental Evidence provides open access sys-
clude communication barriers (e.g. length, linguistic and statistical tematic reviews).
complexity of scientific articles), financial barriers (e.g. studies hidden To address these issues, we have developed a method to rapidly
behind paywalls), relevance barriers (research often lacks direct re- synthesize evidence across entire subject areas (comprising tens or
levance to practitioners or policy-makers), synthesis barriers (an over- hundreds of related review questions), whilst being transparent, ob-
whelming volume of unsynthesized scientific literature) and socio-po- jective and minimizing bias. Target end users (i.e. researchers, practi-
litical barriers (e.g. motivated skepticism of information that challenges tioners and policy-makers) are actively involved in the synthesis pro-
existing worldviews). cess. Uniquely, our subject-wide evidence syntheses (Sutherland and
Evidence synthesis is fundamental to overcoming some of these Wordley, 2018) are part of a broader discipline-wide project, pooling
barriers, increasing the flow of ideas within research-implementation resources to increase speed and cost-effectiveness. The ultimate output
spaces, and ultimately helping researchers, practitioners and policy- of this process is the freely accessible, plain-English Conservation Evi-
makers navigate towards the common goal of conserving biodiversity. dence database, which contains evidence for the effects of conservation
Evidence synthesis methods aim to locate, collate, and synthesize re- interventions. The database is complemented by other tools in the
levant information, usually from published literature. They range from Conservation Evidence toolbox (e.g. the journal Conservation Evidence
unsystematic, ad hoc literature reviews, to comprehensive systematic and Evidence Champions). Together, these tools are designed to over-
reviews, and even reviews of reviews (Collins et al., 2015). However, come or lower barriers within research-implementation spaces, in-
these existing approaches have shortfalls. Traditional literature reviews creasing the use of evidence in practical conservation and policy-
can be subjective, liable to bias and methodologically opaque (Collins making, and enabling practice and policy to influence research. Ulti-
et al., 2015; Haddaway et al., 2015). Systematic reviews are designed to mately, we hope this will lead to more targeted conservation research
reduce those issues, but can be expensive and time-consuming (Borah and more effective conservation action.
et al., 2017; Haddaway and Westgate, 2019). Therefore, they are not In this paper, we focus on the Conservation Evidence database,
always possible in conservation, where resources are limited (Soulé, describing the methods used to create it and how it helps to overcome
1985; Gerber, 2016). The intended audience of reviews and systematic barriers between conservation researchers, practitioners and policy-
reviews sometimes face communication barriers (e.g. Cochrane Clinical makers. Although aspects of the methods have been described pre-
Answers are needed as a “readable, digestible” entry point to medical viously (e.g. Dicks et al., 2016; Sutherland and Wordley, 2018), this

2
W.J. Sutherland, et al. Biological Conservation 238 (2019) 108199

paper provides the only complete and detailed overview of the methods inclusion criteria (Section 2.1) are added to a discipline-wide re-
currently used by Conservation Evidence. Through increasing metho- pository, and tagged or filed by subject areas.
dological transparency and communicating what the database is (and is Sources with a broad, discipline-wide scope (e.g. Journal of Applied
not) designed to do, we hope this paper will encourage effective and Ecology, Biological Conservation) are searched annually to keep the lit-
appropriate use of this tool. We also discuss the database in a broader erature repository up to date. Specialist subject sources (e.g. Journal of
context, acknowledging that published evidence is just one of a multi- Mammalogy, British Trust for Ornithology reports) are searched when
tude of factors within research-implementation spaces that affect con- the synthesis project for that subject is carried out (Section 2.3). Ty-
servation decision making. pically, sources are searched from their first publication date until a
specified recent date. The list of sources and years screened is published
2. Building the conservation evidence database alongside evidence synthesis products.
The key advantages of this method are that it does not depend on
2.1. An overview of the conservation evidence database search term choice and can identify novel interventions not suggested a
priori by the authors or advisory board for a synthesis (Sutherland and
The Conservation Evidence database gathers, organizes, and sum- Wordley, 2018). It is also highly repeatable and transparent, notwith-
marizes studies that quantify the effects of conservation interventions standing some inevitable variation in the interpretation of inclusion
(i.e. actions that have been or could be used to conserve biodiversity) criteria. We use Kappa tests (Cohen, 1960) to identify, and then correct,
on any aspect of biodiversity (e.g. abundance of a focal species, survival inconsistency between searchers. Because all relevant publications are
rates of translocated individuals, use of nest boxes, extent of habitat) or added to a discipline-wide repository, each journal issue or block of
human behavior related to biodiversity conservation (e.g. levels of reports only needs screening once. Each new synthesis or synthesis
hunting, or sales of products detrimental to biodiversity). Ultimately, update can draw from (and contribute to) an existing repository rather
the database will present the evidence for interventions across the en- than starting afresh (Fig. 1), substantially increasing cost-effectiveness.
tire discipline of biodiversity conservation. Four key types of informa- The disadvantages of the systematic manual search approach are
tion fall largely outside the scope of the database: qualitative data, that it requires a high initial outlay of time and money, cannot easily
unpublished practitioner experience, traditional or indigenous knowl- incorporate some sources that contain a large number of publications
edge, and detailed information on social or ethical issues (see Section (e.g. mega-journals such as PLoS ONE), and cannot cover sources with a
3.4). At present our focus is restricted to quantitative data that provide likely low yield of relevant publications. Thus, search terms are used
objective information on the size and direction of effects. instead of, or to complement, systematic manual searching in some
The database is split into subject areas, usually along taxonomic specific cases (see Section 2.3.4). In the future, automated processes
lines (e.g. bats, amphibians) with some taxa split by habitat (e.g. forest based on machine learning could reduce the cost of systematic source-
vegetation, shrubland vegetation). Subjects are distinct areas of re- by-source literature screening, whilst increasing coverage (Westgate
search and practice, which we delimit according to (1) what we, and et al., 2018).
our advisory boards (Section 2.3.2), think would produce a useful
synthesis for practitioners; (2) shared conservation challenges and re- 2.2.2. Non-English and grey literature
levance of interventions across the subject; (3) the abundance and A large proportion of the global scientific literature in conservation
distribution of literature, with a subject needing to be covered within a is not published in English (Amano et al., 2016). Conservation Evidence
1–3 year project and (4) aims and budgets of funders. For example, bat is creating a list of priority conservation journals in 20 different lan-
conservation is synthesized separately from conservation of other ter- guages. Of these, 159 journals have been searched by fluent speakers of
restrial mammals because chiropterologists form a distinct research and each language, with more searches underway. Results are being added
practice community, and because initial funding was limited. At the to the discipline-wide literature repository (Fig. 1), with titles and ab-
user interface, subject areas provide a rapid overview of the scope of the stracts translated into English. Papers retrieved during these searches
database, and coarse filters to focus on the most relevant information. are being incorporated into the Conservation Evidence database as staff
In each subject area, the database provides a comprehensive list of language skills permit. The aim is to reduce bias in the database to-
interventions. For each intervention, the database provides: back- wards evidence from English-speaking countries.
ground information such as the logic behind the intervention and how it “Grey literature” refers to documents not controlled by commercial
might be carried out; standardized paragraphs summarizing individual publishers, such as governmental and non-governmental reports,
scientific studies that have quantified the effects of that intervention; newsletters, conference proceedings, and theses (Farace and Schöpfel,
key messages that provide a narrative index to the combined evidence 2010). Including grey literature in evidence syntheses may help to
from all of those studies; and an overall effectiveness category based on counteract the problem of publication bias, where studies reporting
an assessment of the evidence (effectiveness, certainty, and harm) by a negative or non-significant findings are less likely to be written up and
panel of experts. published in journals (McAuley et al., 2000; Dwan et al., 2013). Con-
In the following sections, we outline the two levels of work involved servation Evidence is making a concerted effort to systematically search
in building the database and the steps therein (Fig. 1). At a high level, more grey literature sources (e.g. 687 reports from the British Trust for
discipline-wide literature searches identify publications that fall within Ornithology and 945 from Scottish Natural Heritage were searched by
the scope of the Conservation Evidence project. At a lower level, subject- 2017) and include relevant publications in the database.
wide evidence syntheses collate and summarize studies across subject
areas. 2.3. Subject-wide evidence syntheses

2.2. Discipline-wide literature searches 2.3.1. Defining the subject and its scope
The detailed process of synthesizing evidence for the Conservation
2.2.1. Systematic manual searches Evidence database is broken down into subject-focused work packages,
The main method used by Conservation Evidence to find relevant or subject-wide evidence syntheses (Fig. 1). The precise subject and
literature is discipline-wide, systematic, manual literature searching. scope of each synthesis is decided at an early stage in consultation with
This means identifying literature sources (e.g. academic journals, report the advisory board (see Section 2.3.2). It is essential to define what each
series, organizational websites) that are likely to contain relevant in- synthesis will include and exclude (Pullin and Stewart, 2006).
formation, then manually scanning the title and abstract (or summary) The subject is usually defined taxonomically, then sometimes fur-
of every document in those sources. All documents meeting the general ther refined by habitat type (see Section 2.1). It is occasionally defined

3
W.J. Sutherland, et al. Biological Conservation 238 (2019) 108199

Fig. 1. An overview of the methods used to build the Conservation Evidence database (discipline-wide literature searches and subject-wide evidence synthesis), and
how a range of end users are incorporated into the construction process. Rectangles represent processes, and rhomboids represent outputs. Numbers indicate section of
text where item is explained.

by other areas of interest, such as invasive species management or studies are excluded for most interventions within vegetation-focused
sustainable agriculture. The geographic scope is usually global. Con- syntheses.
servation Evidence syntheses are focused on the effects of conservation
interventions, so the question structure for review typically follows a 2.3.2. Forming an advisory board
PICO format (population, intervention, comparator, outcome). There is The advisory board for each synthesis is a panel of subject experts
a separate review of the evidence for each intervention. Outcome who can help to refine its scope (Section 2.3.1) as well as its structure
measures are usually direct measures of effects on biodiversity, but may and language (Section 2.3.7), identify interventions (Section 2.3.3) and
include less direct or intermediate outcomes (see Section 2.1). A identify key sources of evidence to search (Section 2.2.1). Advisors
synthesis-specific list of focal metrics may be constructed (e.g. abun- contribute to planning each synthesis as well as reviewing a near-final
dance of certain indicator plant taxa) to guide consistent reporting of version. Since 2018, we have formalized the input of the advisory board
results from summarized studies. Synthesis-specific inclusion/exclusion to the planning stage by asking them to review a synthesis protocol.
criteria may also be defined. For example, laboratory and greenhouse These protocols are registered on the Open Science Framework

4
W.J. Sutherland, et al. Biological Conservation 238 (2019) 108199

(https://osf.io/mz5rx/) and published ahead of each synthesis on the invasive species), may use search terms to query databases of scientific
Conservation Evidence website. We have always used and reported literature. In such cases, employing search terms can be a useful com-
standard methodologies that allow for robust evidence synthesis, but plement to, or replacement for, journal searching. If this approach is
we now appreciate the added value of publishing protocols in advance taken, records are kept and presented to show the databases searched,
(Haddaway and Macura, 2018). the terms used and the dates searches were carried out.
Advisory boards are selected to provide expertise in diverse topics
within the subject, represent the geographic range covered by the evi- 2.3.5. Summarizing relevant studies
dence synthesis, and to include a mix of academics, practitioners and Each publication retrieved through literature searches is screened at
policy-makers. Thus, anticipated users of the database contribute to its full-text by the synthesis authors. If the publication contains at least one
development, helping to ensure applicability to practice and to increase study (i.e. conceptually distinct experiment or test of an intervention)
the likelihood of uptake. So far, advisory boards for Conservation that meets the general inclusion criteria as well as any specific criteria
Evidence syntheses have comprised a mean of 11.6 ( ± 6.9 SD, n = 15) defined for that synthesis, then each study is summarized in a stan-
subject experts. Of the 157 individual experts from a total of 28 dardized paragraph. Reviews and meta-analyses are summarized as
countries across six continents, approximately 53% have been from evidence if they provide new or collective data relevant to the synth-
institutes operating more within research domains, and 47% from or- esis.
ganizations oriented more towards policy and practice domains. Summary paragraphs consistently present the same key information
from each study in the same order (see Fig. A1. in Appendix). This
2.3.3. Intervention scanning includes study design; years of study; habitat; location; conservation
Based on initial literature scans and consultation with the advisory intervention; target species or habitat; whether there was a statistically
board, a list of conservation interventions for the subject of the synth- significant effect of the intervention and the direction of any effect;
esis is created. The aim is to produce a comprehensive list of all in- quantitative data on the outcome of the intervention; and a brief
terventions that have been tried or suggested for the subject of the overview of the methods and monitoring approach. Summary para-
synthesis and that could realistically be implemented. The intervention graphs are concise—typically around 150–200 words—and written in
list can be modified, and added to, as the synthesis process proceeds. plain English, avoiding technical terms wherever possible. Although
Including all possible interventions and then populating these with short, the aim is for summary paragraphs to include sufficient detail of
evidence forms the basis for identifying and mapping evidence gaps the study context and methods to allow users to begin to assess its
(see Section 2.3.6). importance and relevance to their own system (e.g. location, length of
Interventions are grouped in a consistent manner across syntheses: monitoring, exactly how the intervention was done) and interpret
primarily according to the IUCN threat category that they address simple context-dependencies in results.
(Salafsky et al., 2008) and, for interventions that tackle multiple Conservation Evidence does not follow a formal process for criti-
threats, secondary categories based on IUCN action types. The naming cally appraising studies: generally, all studies that meet inclusion cri-
and division of interventions are guided by both the existing literature teria are summarized. However, the design and size of each study are
and the advisory board. Where possible, interventions are described at a reported to help the user—and expert assessors (Section 2.3.8)—judge
fine scale (for example “Set longlines at the side of the boat to reduce its importance and reliability (internal validity). As a simple example,
seabird bycatch” is a separate intervention from “Set lines underwater the reader might give more weight to results from reviews, and parti-
to reduce seabird bycatch”; Williams et al., 2013), so that they can be cularly systematic reviews, than to results from individual case studies.
combined in multiple ways by the user to address larger closed ques- Major concerns (from the original authors or synthesis authors) over the
tions (e.g. are longlines at the side of the boat more effective than study design are explicitly highlighted in summary paragraphs. Ex-
setting lines underwater to reduce seabird bycatch?), or open questions ceptionally, studies may not be summarized if they clearly involve in-
(what is the state of knowledge on seabird bycatch reduction valid comparisons, or are missing key information that severely inhibits
methods?). comprehension. These issues are noted in the subject-wide literature
As the evidence synthesis is constructed, background information is repository (Figure 1).
added to each intervention. This briefly explains the logic behind the
intervention, key issues regarding practical implementation, any un- 2.3.6. Key messages: an overview of the summaries
avoidable technical terms used, and potential harms to society or the Summary paragraphs describing studies that test the same inter-
wider environment. This background information is not, for pragmatic vention are grouped together. “Key messages” provide a brief overview
reasons, based on systematic literature searches and is therefore not of the studies testing each intervention: usually some indication of the
taken into account during expert assessment (Section 2.3.8). number of studies, their geographical distribution, and their reported
effects on key outcome metrics. Key messages are intended to provide
2.3.4. Collating subject-relevant literature an index to the evidence, easing the user into summary paragraphs and
The synthesis authors collate a repository of literature that is re- helping them identify the most relevant studies to their situation, and to
levant to their synthesis, based on title and abstract/summary facilitate comparisons of studies.
screening. The authors draw documents from the discipline-wide re- The key messages also highlight knowledge clusters and gaps in
pository—which contains the results from screening sources with a relation to interventions, targets, outcomes, habitats and geographic
discipline-wide focus and sources with a focus on other previously- locations – and thus help identify where further research is needed. For
synthesized subjects (Section 2.2.1)—but also search sources most re- example, no studies were found testing the intervention “Leave un-
levant to their synthesis subject (e.g. herpetological journals for an harvested cereal headlands within arable fields” for bird populations
amphibian-focused synthesis). Relevant sources are identified in colla- (Williams et al., 2013). Furthermore, whilst four studies tested the in-
boration with the advisory board. The documents extracted from these tervention “Leave standing deadwood/snags in forests” for amphibian
synthesis-specific searches in turn feed back into the discipline-wide populations, they were all carried out in the USA (three in Virginia) and
repository. For example, searches of herpetological journals for the all but one focused on salamanders (Smith and Sutherland, 2014). The
amphibian-focused synthesis will also return papers relevant to a key messages across all interventions in a synthesis map the distribution
synthesis for reptiles. of evidence across the subject area. Ultimately, key messages across the
Conservation Evidence syntheses on a new subject area unlikely to entire Conservation Evidence database will provide a “mega-map” of
retrieve many publications from the existing discipline-wide repository, evidence for the whole conservation discipline.
or on a very specific subject (e.g. the control of a particular group of We realize our key messages may be interpreted as an invitation to

5
W.J. Sutherland, et al. Biological Conservation 238 (2019) 108199

vote count (i.e. draw conclusions based on the number of studies reporting different output metrics, is a challenge. Conservation
showing positive vs negative results), which is usually a misleading Evidence uses expert assessment rather than meta-analyses to synthe-
method of synthesis (Stewart and Ward, 2019). This is not the intended size studies testing the same intervention, but reporting very different
use. Key messages include information about study designs to suggest metrics, into an overall effectiveness category. For example, expert
that the value of evidence varies between studies. Online, they link assessment can combine studies reporting the appearance of Sphagnum
directly to the summary paragraphs that contain data to indicate the moss species on bogs following rewetting (a desirable change) and
magnitude of any effects. We have added an explicit warning about vote studies reporting a decrease in tree cover on bogs following rewetting
counting to key messages on the Conservation Evidence website, and (also a desirable change) to give an overall assessment that the inter-
are considering alternative ways to provide an overview of the evidence vention is effective. Meta-analyses tend not to combine different me-
base. trics because the resulting effect size would not be linked to any metric
and would therefore lose some meaning. Some studies that can be
2.3.7. External review of synthesis considered by expert assessment also lack sufficient detail for the cal-
Once the evidence has been summarized, the draft synthesis is re- culations involved in meta-analysis (Haddaway and Verhoeven, 2015).
viewed by the advisory board. They identify problems with language Conservation Evidence highlights (e.g. in synopsis introductions) in-
and structure, and suggest further relevant publications not retrieved terventions or groups of interventions where we think conducting a
through literature searches (Sections 2.2.1 and 2.3.4). The synthesis meta-analyses would be worthwhile as a more robust alternative to
authors then include relevant studies and rectify any problems. expert assessment.

2.3.8. Expert assessment 2.4. Accessing the database

Expert assessment is an important final step in synthesizing and
presenting the evidence (Fig. 1). The aim is to consider studies holi- Outputs from each subject-wide evidence synthesis (interventions,
stically and generate a generalized, overall effectiveness category for summary paragraphs, key messages, expert assessments) are freely
each intervention. This provides users with a supplementary decision- available within the searchable online database, www.
support tool, alongside the key messages and individual study sum- conservationevidence.com. Users can search and filter the database in
maries. multiple ways, including by taxon, habitat, intervention and threat.
For a Conservation Evidence synthesis, the evidence for each in- Synopses capture most of each subject-wide evidence synthesis (inter-
tervention is assessed using a modified Delphi technique (Mukherjee ventions, summarized paragraphs and key messages) in a free-to-
et al., 2015). This involves a panel of experts—academics, practitioners download pdf, and in some cases as a printed book. An annual pub-
and policy-makers from across the geographic range of the synthe- lication, What Works in Conservation (e.g. Sutherland et al., 2018a,
sis—carrying out several rounds of scoring for each intervention. The 2018b), presents the key messages and expert assessment for inter-
experts score the effectiveness, certainty and harm of each intervention, ventions reviewed so far, as a rapid overview and gateway into the
based on the evidence presented in the synthesis. Anonymized scores online database (via hyperlinks).
and comments are shared within the expert panel between rounds of
scoring, to be used as a basis for refining scores. After 2–3 rounds, final 2.5. Updates
median scores are used to assign an overall effectiveness category for
each intervention (Sutherland et al., 2018a, 2018b; Table A1 in Ap- The Conservation Evidence database is designed to allow the regular
pendix). incorporation of new evidence. Updating each subject-wide evidence
Effectiveness is scored by considering whether the intervention synthesis involves searching new volumes, issues, or documents within
produces a desirable outcome in the summarized studies, and the the originally-searched literature sources; searching additional litera-
magnitude of that outcome (0% = not effective, 100% = highly effec- ture sources; and adding new interventions or adjusting existing ones
tive). Certainty is a measure of how confident assessors are that the (e.g. where new literature suggests actions could be divided into mul-
effectiveness score applies across all appropriate contexts. The certainty tiple interventions). Further documents suggested by users since the
score incorporates (1) the strength or reliability of the evidence as a publication of the original synthesis can also be included. Thus, all users
whole, based on the number of studies and their quality (internal va- can contribute studies to evidence syntheses through publishing their
lidity e.g. study design, replication, bias); and (2) how generalizable the own articles and/or highlighting articles published by others. Key
results of these studies are, which will depend on the taxonomic/ha- messages are updated and expert assessments repeated for any inter-
bitat/geographical coverage of studies (external validity). Certainty is ventions where new evidence was added. Conservation Evidence has
scored from 0% (no evidence) to 100% (lots of high quality evidence, started to update existing syntheses and, in the short term, aims to
high generalizability). An intervention could be scored as having high produce updates every few years. In the longer term, we envisage up-
effectiveness if it is supported by many studies showing strong desirable dating the database in near-real time as new evidence is published,
outcomes, but low certainty if those studies use low quality study de- perhaps with the help of artificial intelligence to find publications and/
signs or only consider a specific local context. Harm is scored by rating or extract data (Westgate et al., 2018).
the magnitude of undesirable effects on the subject of the synthesis
from undertaking the intervention (0% = no undesirable effects, 3. Discussion
100% = major undesirable effects). The harm score is important to
distinguish interventions that lack desirable effects from those that have 3.1. Synthesizing complex evidence at scale
undesirable effects: such interventions could receive identical effec-
tiveness (and certainty) scores. The methods developed to build the Conservation Evidence data-
Currently, the Conservation Evidence database presents the overall base allow for the synthesis of complex evidence across broad subjects
effectiveness categories and the percentage scores for their three com- and ultimately across whole disciplines. Using discipline-wide searches
ponents (effectiveness, certainty, and harms). Whilst the percentage and subject-wide syntheses, we can efficiently synthesize evidence for
scores are useful for giving assessors flexibility and to generate medians both major and obscure topics, with a large or limited evidence base,
across all assessors, we realize they could give a false sense of precision respectively. Through a combination of summary paragraphs, key
to database users. Thus, in the future, we may move towards categorical messages and expert assessment, we can present a general overview of
groupings to present scores for the three components. the evidence incorporating a diversity of metrics, whilst allowing users
Combining evidence from disparate locations, of varying rigor, and to drill down to the evidence most relevant to their situation. Key

6
W.J. Sutherland, et al. Biological Conservation 238 (2019) 108199

messages and expert assessment can also highlight knowledge gaps and searching grey and non-English literature.
clusters for subjects and interventions. Finally, by using short summary We acknowledge that even if all the available journals and grey
paragraphs in plain English, we produce a user-friendly end product. literature were screened in multiple languages, a substantial amount of
We believe the truly unique feature of our methods is the combination conservation-relevant knowledge would not be captured and a knowl-
of subject-wide synthesis and discipline-wide searches: we are not edge-accessibility barrier remains. Many reports remain as internal
aware of any other synthesis projects that work across entire subjects documents and are not publicly accessible. Moreover, some data on the
and simultaneously collate literature for future syntheses within the effects of conservation interventions are not formally reported and re-
discipline. We suggest these methods could be used to synthesize evi- mains as case experience in minds and notebooks. Although it may be
dence for other themes within biodiversity conservation, such as threats possible to include case experience in the Conservation Evidence da-
or monitoring methods. tabase, for example through the use of interviews, it can be difficult to
The Conservation Evidence database complements other systematic capture in a systematic fashion and is likely subject to behavioral, social
evidence synthesis outputs. The Conservation Evidence database pro- and cognitive biases that can be difficult for third parties to assess. As
vides syntheses of evidence over a broad range of topics, for which the such, we have decided not to include it in the database at present. Still,
investment in a systematic review is not (yet) justified but something such experiential and tacit knowledge should be used to complement
more than just a map of the evidence would be useful. Systematic re- the Conservation Evidence database when making conservation deci-
views, which favor depth of review over breadth of topics reviewed, are sions (Section 3.3).
desirable for interventions with a large evidence base, where studies The Conservation Evidence database aims to present scientific in-
present contrasting results, for contentious topics, or where the risks formation in a format relevant to practitioners and policy-makers who
posed by an incorrect conclusion are severe (Collaboration for often struggle with the technical language, statistical analyses and
Environmental Evidence, 2013; Collins et al., 2015). Systematic maps length of scientific articles (Pullin and Knight, 2005)—including sys-
provide a rapid and inexpensive overview of the state of evidence in a tematic reviews (Tricco et al., 2016). The database uses short para-
broad subject or topic, without detailing what the evidence finds. They graphs in plain English. The content is also edited/reviewed by prac-
are most useful for identifying knowledge gaps and clusters, which can titioners and policy-makers who sit on the synthesis advisory boards. In
help direct research effort where it is most needed (Haddaway et al., addition, interventions are tagged (grouped) according to the IUCN
2016). All of these systematic outputs—including the Conservation universal classification schemes of threats and actions (Salafsky et al.,
Evidence database—are clearly organized, permanent, searchable and 2008), which were developed with input from practitioners and
designed to minimize several key sources of bias, especially compared therefore reflect their thought processes.
to other communication methods such as traditional literature reviews, The database breaks down some language barriers by summarizing
notes from conference presentations, or word of mouth. some articles originally published in languages other than English,
making them more accessible to English speakers (and at least all in a
3.2. How the conservation evidence database helps to overcome barriers in common language). We take opportunities to translate syntheses into
research-implementation spaces alternative languages where possible (e.g. Hebrew for bee conserva-
tion), and have incorporated Google Translate into our website for “on
The Conservation Evidence database is designed to overcome some the fly” translation. We appreciate that we still have work to do to
of the barriers between conservation research, practice and policy break down language barriers for non-English-speaking users of the
(Table 1), facilitating the flow of information between (and within) Conservation Evidence database.
these domains. For example, the scientific literature is vast and ever- In many cases, the knowledge transfer barriers in the research-im-
expanding, yet only a fraction of it is directly relevant to practitioners plementation space arise upstream of evidence synthesis: there are
or policy-makers (Westgate et al., 2018). The Conservation Evidence often no (or few) scientific studies relevant to practitioners or policy-
database helps to overcome this barrier by presenting a relevant subset makers. For example, scientists may focus on global analyses, complex
of the literature, containing quantitative information about the effects statistics and studies that push the boundaries of fundamental scientific
of conservation interventions—which can be a key factor in making knowledge to generate publications with a high academic impact—but
robust conservation decisions (Adams and Sandbrook, 2013; see also which are of little use to practitioners and policy-makers (McNie, 2007;
Section 3.4). Furthermore, the database is categorized at multiple levels Hulme, 2011; Braunisch et al., 2012). To overcome this barrier, prac-
(subjects, interventions, individual studies), allowing users to quickly titioners and policy-makers contribute to shaping the interventions in-
drill down to relevant information, and combine it within and across cluded in the Conservation Evidence database. Furthermore, the data-
levels to generate a custom evidence synthesis. We are currently de- base highlights knowledge gaps (i.e. which interventions are supported
veloping an online tool that allows users to formally generate custom by no, little or low quality evidence) and clusters (e.g. in certain loca-
evidence syntheses. tions or habitat types). Thus, researchers can see which questions are of
A key feature of the Conservation Evidence database is its breadth, interest to practitioners and policy-makers, and which are lacking evi-
synthesizing evidence for a large number of questions (interventions). dence-based answers. Being able to demonstrate a knowledge gap for a
Since the first Conservation Evidence synthesis began in 2010, we have practice- or policy-relevant question may help researchers justify re-
reviewed over 1800 interventions (Sutherland et al., 2018a, 2018b) for search funding.
15 subjects. The aim is to synthesize the evidence for the effects of all There may also be psychological barriers limiting the flow of in-
interventions, for all taxa and all habitats, everywhere in the world. formation between research, practice and policy. For example, if in-
Clearly there is a trade-off between breadth and depth of the data- stitutionalized methods and relationships do not currently involve in-
base, so we cannot claim to have captured all of the available evidence teractions between research and practice, a certain degree of activation
for each intervention. The assumption is that users benefit from a energy will be needed to change habits (van Kerkhoff and Lebel, 2015).
synthesis of the evidence in the sources we search, as long as that is Further, scientific evidence is often discounted when it challenges
based on an unbiased sample of the available evidence, and users un- people's pre-existing values or worldviews, especially when they are
derstand that the evidence base might be incomplete. A similar as- strongly connected to defined social identities (Roux et al., 2006;
sumption implicitly, or explicitly, supports the use of other forms of Newell et al., 2014). Related to this, there is a growing availability of
rapid evidence assessment (e.g. Collins et al., 2015). As we have already highly visible and accessible, but often unreliable, in-
included studies from over 280 journals and grey literature sources in formation—especially on social media—which can “crowd out” reliable
the database, we think we have captured a substantial proportion of the sources of evidence (Ladle et al., 2005). Whilst many solutions to these
relevant literature. We reduce publication and geographic biases by problems are largely outside of the scope of the Conservation Evidence

7
W.J. Sutherland, et al. Biological Conservation 238 (2019) 108199

project (e.g. detecting and removing disinformation on social media 3.3. Other tools in the Conservation Evidence toolbox that help overcome
platforms), we suggest that the Conservation Evidence database may the barriers in research-implementation spaces
indirectly help to combat the spread of disinformation by increasing the
accessibility and visibility of verifiable research evidence (see also The database is a core part of the wider Conservation Evidence
Section 3.4 on Evidence Champions) and may help to reduce the impact project, which contains other tools to help overcome barriers between
of politically motivated disinformation by providing an open, objective, conservation researchers, practitioners and policy-makers. We briefly
independent fact-checking resource for practitioners (Ecker and Ang, discuss these here.
2019; Hameleers and van der Meer, 2019). The journal Conservation Evidence publishes research, monitoring
In an attempt to normalize use of the Conservation Evidence data- results, and case studies on the effects of conservation interventions.
base, and reduce the psychological barrier of using a new tool, the There is no requirement for novelty, complex statistical analyses, or
database is integrated into an increasing number of practitioner-focused technical discussions. It is designed specifically to encourage practi-
resources and decision-support software tools. It complements existing tioners to submit their quantitative data and make them accessible to
information on the websites of the IUCN Red List (https://www. all. By converting unpublished reports, internal documents and data
iucnredlist.org), the National Biodiversity Network (https://nbn.org. from field notebooks into open access publications, this journal helps
uk), the British Trust for Ornithology (https://www.bto.org), and the overcome the knowledge-accessibility barrier discussed above (Section
UNEP-Agreement on the Conservation of African-Eurasian Migratory 3.2). Providing an outlet for sharing robust, conservation-relevant pri-
Waterbirds (https://www.unep-aewa.org/). The database is embedded mary research could also encourage greater collaboration between re-
in the Conservation Management System software (https://www. searchers and practitioners.
software4conservation.com/cmsi-software) used by Natural England Since 2017 we have worked with a group of designated Evidence
and 10 other organizations to plan land management. The Cool Farm Champions. These are organizations committed to using evidence
Tool (https://coolfarmtool.org/) is used by major grocery retailers to (particularly the Conservation Evidence database) when planning,
help farmers choose practices that reduce greenhouse gas emissions funding, or publishing practical conservation actions, and/or testing a
and, through the integration of the Conservation Evidence database, certain number of interventions each year and publishing the results.
could be beneficial for biodiversity. These techniques are intended to address some of the psychological
Unfortunately, we currently have limited data about the effective- barriers to the use of evidence (Section 3.2) by making a balanced as-
ness of Conservation Evidence at breaking down barriers. Empirical sessment of evidence a routine and expected part of conservation
evidence that the database can improve the effectiveness of conserva- planning. Evidence Champions are supported through training in evi-
tion when used is limited to one study. Walsh et al. (2014) demon- dence interpretation and generation techniques.
strated that information synthesized by Conservation Evidence (on the We also run more general workshops to explain what the
effectiveness of various interventions to control predators for bird Conservation Evidence database is and how it can be used, or how
conservation) changed practitioners' stated choices of management in practitioners can best carry out research to feed into the database.
favor of more effective interventions, and away from interventions that Again, these can help to reduce behavioral or psychological barriers to
were likely to be ineffective or even harmful. Data on whether the the use or production of conservation-relevant evidence.
Conservation Evidence database increases the effectiveness of con-
servation in practice are difficult to collect, but we are seeking research 3.4. Conservation in practice: other factors and actors in research-
funding to do so (and encourage others to take up the challenge too). implementation spaces
We do know that the website is well used: it received an average of
29,000 page views per month between January and May 2019, by an The Conservation Evidence database is built within the collabora-
average of 11,700 visitors per month from over 220 countries and tive spaces occupied by conservation researchers, practitioners and
overseas territories. About 25% of visitors have used the website more policy-makers. When the database is used to make practical or policy
than once. Copies of What Works in Conservation have been read online conservation decisions, other actors (e.g. NGOs, governments, land-
or downloaded almost 39,000 times as of June 2019. owners, farmers, indigenous communities, activists), issues (e.g. spiri-
Evidence Champions (see section 3.4) and others have provided tual and cultural values, financial resources, political), and information
feedback that the database has helped their decision-making. For the (e.g. the basic biology, distribution and status of species and habitats,
AEWA (Agreement on the Conservation of African-Eurasian Migratory the presence and degree of threats, local knowledge and practical ex-
Waterbirds) Secretariat, “it was a very helpful source, as we could use it perience) are introduced to these arenas (Roux et al., 2006; Toomey
as a good reference and depending on the case also as a source for good et al., 2017; Evans et al., 2017). Conservation decisions are not made
examples on the ground.” The Rufford Foundation, who ask grant ap- based on scientific evidence alone; socially acceptable decisions must
plicants to reference Conservation Evidence in their applications, said, balance the needs of nature and people. In particular, the quantitative
“I think that it has been valuable as a way of encouraging those de- data from the Conservation Evidence database will need to be com-
signing projects to look further afield to see if techniques they plan to bined with qualitative data, for example derived from interviews or
use have been tried before and, if yes, with what result. All of this has focus groups, to capture relevant tacit knowledge and values and ulti-
certainly helped our reviewers. Overall, Conservation Evidence has … mately design effective conservation strategies (Roux et al., 2006;
greatly improved the quality of the applications we receive.” Sutherland et al., 2018a, 2018b).
The Conservation Evidence database has been recognized in the Similarly, the Conservation Evidence database cannot tell practi-
political sphere. An example from the Conservation Evidence database, tioners or policy-makers when or how to intervene. This decision will
publicized by Sutherland and Wordley (2017), was used by Lord John be influenced by site-specific issues and information mentioned above,
Krebs to ask the UK government to ensure the government's 25 year as well as assessments of the focal site's history and desired future for all
environment plan would be evidence-based. The database has also been stakeholders. We recognize the potential that a list of interventions—-
referred to in multiple policy briefs and government documents, such as some assessed as beneficial to species or habitats—might encourage
Defra's Consultation on the National Pollinator Strategy (2014), The unnecessary active intervention. In some cases, particularly in rela-
Scottish Government's Consultation on the Scotland Rural Development tively intact sites, interventions may not be required to reach a desired
Programme (SRDP) 2014–2020, and The New Zealand Government's state and might do more harm than good to biodiversity. Thus, we
Improvements to Biodiversity Assets Systems and Processes (2014). caution against assuming that intervening is always better than not
intervening. To this end, we also include some passive interventions in
our syntheses (e.g. Allow shrubland to regenerate without active

8
W.J. Sutherland, et al. Biological Conservation 238 (2019) 108199

management) to highlight that doing nothing is a management option pdf, Accessed date: March 2019.
to consider. Collins, A.M., Coughlin, D., Miller, J., Kirk, S., 2015. The Production of Quick Scoping
Reviews and Rapid Evidence Assessments: A how to Guide. Available at. https://
www.gov.uk/government/publications/the-production-of-quick-scoping-reviews-
3.5. Conclusion and recommendations and-rapid-evidence-assessments, Accessed date: March 2019.
Dicks, L.V., Wright, H.L., Ashpole, J.E., Hutchison, J., McCormack, C.G., Livoreil, B.,
Zulka, K.P., Sutherland, W.J., 2016. What works in conservation? Using expert as-
The Conservation Evidence database is assembled through a sys- sessment of summarised evidence to identify practices that enhance natural pest
tematic, repeatable process, with input from conservation researchers, control in agriculture. Biodivers. Conserv. 25, 1383–1399. https://doi.org/10.1007/
practitioners and policy-makers. It is a powerful and pragmatic tool to s10531-016-1133-7.
Dirzo, R., Young, H.S., Galetti, M., Ceballos, G., Isaac, N.J.B., Collen, B., 2014.
improve the use of scientific evidence by practitioners and policy-ma- Defaunation in the anthropocene. Sci 345, 401–406. https://doi.org/10.1126/
kers, and encourage new research that is guided by practice and policy science.1251817.
needs. The database aims to complement existing evidence synthesis Dwan, K., Gamble, C., Williamson, P.R., Kirkham, J.J., 2013. Systematic review of the
empirical evidence of study publication bias and outcome reporting bias—an updated
methods, and is complemented by other tools within the Conservation
review. PLoS One 8 (7), e66844. https://doi.org/10.1371/journal.pone.0066844.
Evidence toolbox, helping to create interactive spaces where re- Ecker, U., Ang, L.C., 2019. Political attitudes and the processing of misinformation cor-
searchers, practitioners and other key stakeholders can collaboratively rections. Polit. Psychol. 40, 241–260. https://doi.org/10.1111/pops.12494.
pursue evidence-based conservation. Evans, M.C., Davila, F., Toomey, A., Wyborn, C., 2017. Embrace complexity to improve
conservation decision making. Nat. Ecol. Evol. 1, 1588. https://doi.org/10.1038/
Several concrete recommendations arise from our work building the s41559-017-0345-x.
Conservation Evidence database and this article reflecting on the Farace, D.J., Schöpfel, J.S., 2010. Introduction grey literature. In: Farace, D.J., Schöpfel,
methods used to build it. Conservation researchers, practitioners and J.S. (Eds.), Grey Literature in Library and Information Studies. De Gruyter Sauer,
Berlin/New York.
policy-makers should consult the database when making conservation Gerber, L.R., 2016. Conservation triage or injurious neglect in endangered species re-
decisions, to ensure those decisions are informed by evidence alongside covery. Proc. Natl. Acad. Sci. 113, 3563–3566. https://doi.org/10.1073/pnas.
expert opinion, experience, local knowledge and values. Second, con- 1525085113.
Gossa, C., Fisher, M., Milner-Gulland, E.J., 2015. The research–implementation gap: how
servation intervention projects should be monitored and the results practitioners and researchers from developing countries perceive the role of peer-
published, whether or not successful and/or novel, in order to reviewed literature in conservation science. Oryx 49, 80–87. https://doi.org/10.
strengthen the evidence base. Third, conservationists should engage 1017/S0030605313001634.
Haddaway, N.R., Macura, B., 2018. The role of reporting standards in producing robust
with the Conservation Evidence project, offer constructive feedback and literature reviews. Nat. Clim. Chang. 8, 444–453. https://doi.org/10.1038/s41558-
help us to make the database as useful as possible for you. Finally, the 018-0180-3.
database should, and will, be constantly growing and evolving as it Haddaway, N.R., Verhoeven, J.T., 2015. Poor methodological detail precludes experi-
mental repeatability and hampers synthesis in ecology. Ecol. Evol. 5, 4451–4454.
incorporates new evidence, methodological improvements and tech-
https://doi.org/10.1002/ece3.1722.
nological developments. Haddaway, N.R., Westgate, M.J., 2019. Predicting the time needed for environmental
systematic reviews and systematic maps. Conserv. Biol. 33, 434–443. https://doi.org/
Acknowledgements 10.1111/cobi.13231.
Haddaway, N.R., Woodcock, P., Macura, B., Collins, A., 2015. Making literature reviews
more reliable through application of lessons from systematic reviews. Conserv. Biol.
The Conservation Evidence project is or has been supported by A.G. 29, 1596–1605. https://doi.org/10.1111/cobi.12541.
Leventis Foundation, Arcadia, British Ecological Society, Defra, Haddaway, N.R., Bernes, C., Jonsson, B.G., Hedlund, K., 2016. The benefits of systematic
mapping to evidence-based environmental management. Ambio 45, 613–620.
Economic and Social Research Council, Joint Nature Conservation https://doi.org/10.1007/s13280-016-0773-x.
Committee, MAVA Foundation, Natural England, Natural Environment Hameleers, M., van der Meer, T.G.L.A., 2019. Misinformation and polarization in a high-
Research Council, The Nature Conservancy, South West Water, choice media environment: how effective are political fact-checkers? Commun. Res.
https://doi.org/10.1177/0093650218819671.
Synchronicity Earth and Waitrose. Dozens of volunteers and staff have Hulme, P.E., 2011. Practitioner's perspectives: introducing a different voice in applied
contributed to building the Conservation Evidence database. ecology. J. Appl. Ecol. 48, 1–2. https://doi.org/10.1111/j.1365-2664.2010.01938.x.
Knight, A.T., Cowling, R.M., Rouget, M., Balmford, A., Lombard, A.T., Campbell, B.M.,
2008. Knowing but not doing: selecting priority conservation areas and the re-
References
search–implementation gap. Conserv. Biol. 22, 610–617. https://doi.org/10.1111/j.
1523-1739.2008.00914.x.
Adams, W., Sandbrook, C., 2013. Conservation, evidence and policy. Oryx 47 (3), Ladle, R.J., Jepson, P., Whittaker, R.J., 2005. Scientists and the media: the struggle for
329–335. https://doi.org/10.1017/S0030605312001470. legitimacy in climate change and conservation science. Interdiscip. Sci. Rev. 30 (3),
Amano, T., González-Varo, J.P., Sutherland, W.J., 2016. Languages are still a major 231–240. https://doi.org/10.1179/030801805X42036.
barrier to global science. PLoS Biol. 14 (12), e2000933. https://doi.org/10.1371/ McAuley, L., Tugwell, P., Moher, D., 2000. Does the inclusion of grey literature influence
journal.pbio.2000933. estimates of intervention effectiveness reported in meta-analyses? Lancet 356,
Anon, 2007. The great divide. Nature 450, 135–136. https://doi.org/10.1038/450135b. 1228–1231. https://doi.org/10.1016/S0140-6736(00)02786-0.
Arlettaz, R., Schaub, M., Fournier, J., Reichlin, T.S., Sierro, A., Watson, J.E., Braunisch, McNie, E.C., 2007. Reconciling the supply of scientific information with user demands: an
V., 2010. From publications to public actions: when conservation biologists bridge analysis of the problem and review of the literature. Environ. Sci. Pol. 10, 17–38.
the gap between research and implementation. BioSci 60, 835–842. https://doi.org/ https://doi.org/10.1016/j.envsci.2006.10.004.
10.1525/bio.2010.60.10.10. Mukherjee, N., Huge, J., Sutherland, W.J., McNeill, J., van Opstal, M., Dahdouh-Guebas,
Bainbridge, I., 2014. Practitioners perspective: how can ecologists make conservation F., Koedam, N., 2015. The Delphi technique in ecology and biological conservation:
policy more evidence based? Ideas and examples from a devolved perspective. J. applications and guidelines. Methods Ecol. Evol. 6, 1097–1109. https://doi.org/10.
Appl. Ecol. 51, 1153–1158. https://doi.org/10.1111/1365-2664.12294. 1111/2041-210X.12387.
Borah, R., Brown, A.W., Capers, P.L., Kaiser, K.A., 2017. Analysis of the time and workers Newell, B.R., McDonald, R.I., Brewer, M., Hayes, B.K., 2014. The psychology of en-
needed to conduct systematic reviews of medical interventions using data from the vironmental decisions. Annu. Rev. Environ. Resour. 39, 443–467. https://doi.org/10.
PROSPERO registry. BMJ Open 7, e012545. https://doi.org/10.1136/bmjopen-2016- 1146/annurev-environ-010713-094623.
012545. Pullin, A.S., Knight, T.M., 2005. Assessing conservation management's evidence base: a
Braunisch, V., Home, R., Pellet, J., Arlettaz, R., 2012. Conservation science relevant to survey of management-plan compilers in the United Kingdom and Australia. Conserv.
action: a research agenda identified and prioritized by practitioners. Biol. Conserv. Biol. 19, 1989–1996. https://doi.org/10.1111/j.1523-1739.2005.00287.x.
153, 201–210. https://doi.org/10.1016/j.biocon.2012.05.007. Pullin, A.S., Stewart, G.B., 2006. Guidelines for systematic review in conservation and
Cardinale, B.J., Duffy, J.E., Gonzalez, A., Hooper, D.U., Perrings, C., Venail, P., Narwani, environmental management. Conserv. Biol. 20, 1647–1656. https://doi.org/10.
A., Mace, G.M., Tilman, D., Wardle, D.A., Kinzig, A.P., Daily, G.C., Loreau, M., Grace, 1111/j.1523-1739.2006.00485.x.
J.B., Larigauderie, A., Srivastava, D.S., Naeem, S., 2012. Biodiversity loss and its Ripple, W.J., Wolf, C.W., Newsome, T.M., Galetti, M., Alamgir, M., Crist, E., Mahmoud,
impact on humanity. Nature 486, 59–67. https://doi.org/10.1038/nature11148. M.I., Laurance, W.F., 15,364 scientist signatories from 184 countries, 2017. World
Cochrane Library, 2019. About Cochrane Clinical Answers. Available at. https://www. scientists' warning to humanity: a second notice. BioSci. 67, 1026–1028. https://doi.
cochranelibrary.com/cca/about, Accessed date: 23 March 2019. org/10.1093/biosci/bix125.
Cohen, J.A., 1960. Coefficient of agreement for nominal scales. Educ. Psychol. Meas. 20, Rose, D.C., Sutherland, W.J., Amano, T., González-Varo, J.P., Robertson, R.J., Simmons,
37–46. https://doi.org/10.1177/001316446002000104. B.I., Wauchope, H.S., Kovacs, E., Durán, A.P., Vadrot, A.B.M., Wu, W., Dias, M.P., Di
Collaboration for Environmental Evidence, 2013. Guidelines for Systematic Review and Fonzo, M.M.I., Ivory, S., Norris, L., Nunes, M.H., Nyumba, T.O., Steiner, N., Vickery,
Evidence Synthesis in Environmental Management. Version 4.2. Environmental J., Mukherjee, N., 2018. The major barriers to evidence-informed conservation policy
Evidence. www.environmentalevidence.org/Documents/Guidelines/Guidelines4.2. and possible solutions. Conserv. Lett. 11 (5), e12564. https://doi.org/10.1111/conl.

9
W.J. Sutherland, et al. Biological Conservation 238 (2019) 108199

12564. Sutherland, W.J., Dicks, L.V., Ockendon, N., Petrovan, S.O., Smith, R.K., 2018b. What
Roux, D.J., Rogers, K.H., Biggs, H.C., Ashton, P.J., Sergeant, A., 2006. Bridging the sci- Works in Conservation 2018. Open Book Publishers, Cambridge, UK. https://doi.org/
ence-management divide: moving from unidirectional knowledge transfer to 10.11647/OBP.0131.13.
knowledge interfacing and sharing. Ecol. Soc. 11, 4. [online] URL: http://www. Toomey, A.H., Knight, A.T., Barlow, J., 2017. Navigating the space between research and
ecologyandsociety.org/vol11/iss1/art4/. implementation in conservation. Conserv. Lett. 10, 619–625. https://doi.org/10.
Salafsky, N., Salzer, D., Stattersfield, A.J., Hilton-Taylor, C., Neugarten, R., Butchart, S.H., 1111/conl.12315.
Collen, B., Cox, N., Master, L.L., O'Connor, S., Wilkie, D., 2008. A standard lexicon for Tricco, A.C., Cardoso, R., Thomas, S.M., Motiwala, S., Sullivan, S., Kealey, M.R.,
biodiversity conservation: unified classifications of threats and actions. Conserv. Biol. Hemmelgarn, B., Ouimet, M., Hillmer, M.P., Perrier, L., Shepperd, S., Straus, S.E.,
22, 897–911. https://doi.org/10.1111/j.1523-1739.2008.00937.x. 2016. Barriers and facilitators to uptake of systematic reviews by policy makers and
Smith, R.K., Sutherland, W.J., 2014. Amphibian Conservation: Global Evidence for the health care managers: a scoping review. Implement. Sci. 11, 4. https://doi.org/10.
Effects of Interventions. Pelagic Publishing, Exeter, UK. 1186/s13012-016-0370-1.
Soulé, M.E., 1985. What is conservation biology? BioSci 35, 727–734. https://doi.org/10. van Kerkhoff, L.E., Lebel, L., 2015. Coproductive capacities: rethinking science-govern-
2307/1310054. ance relations in a diverse world. Ecol. Soc. 20 (1), 14. https://doi.org/10.5751/ES-
Stewart, G., Ward, J., 2019. Meta-science urgently needed across the environmental 07188-200114.
nexus: a comment on Berger-Tal et al. Behav. Ecol. 30 (1), 9–10. https://doi.org/10. Walsh, J.C., Dicks, L.V., Sutherland, W.J., 2014. The effect of scientific evidence on
1093/beheco/ary155. conservation practitioners' management decisions. Conserv. Biol. 29, 88–98. https://
Sutherland, W.J., Wordley, C.F.R., 2017. Evidence complacency hampers conservation. doi.org/10.1111/cobi.12370.
Nat. Ecol. Evol. 1, 1215–1216. https://doi.org/10.1038/s41559-017-0244-1. Westgate, M.J., Haddaway, N.R., Cheng, S.H., McIntosh, E.J., Marshall, C., Lindenmayer,
Sutherland, W.J., Wordley, C.F.R., 2018. A fresh approach to evidence synthesis. Nature D.B., 2018. Software support for environmental evidence synthesis. Nat. Ecol. Evol. 2,
558, 364–366. https://doi.org/10.1038/d41586-018-05472-8. 588–590. https://doi.org/10.1038/s41559-018-0502-x.
Sutherland, W.J., Pullin, A.S., Dolman, P.M., Knight, T.M., 2004. The need for evidence- Williams, D.R., Pople, R.G., Showler, D.A., Dicks, L.V., Child, M., zu Ermgassen, E.K.H.J.,
based conservation. TREE 19, 305–308. https://doi.org/10.1016/j.tree.2004.03.018. Sutherland, W.J., 2013. Bird Conservation: Global Evidence for the Effects of
Sutherland, W.J., Dicks, L.V., Everard, M., Geneletti, D., 2018a. Qualitative methods for Interventions. Pelagic Publishing, Exeter, UK.
ecologists and conservation scientists. Methods Ecol. Evol. 9 (1), 7–9. https://doi. Young, K.D., Van Aarde, R.J., 2011. Science and elephant management decisions in South
org/10.1111/2041-210X.12956. Africa. Biol. Conserv. 144, 876–885. https://doi.org/10.1016/j.biocon.2010.11.023.

10