Overstated carbon emission reductions from voluntary

REDD+ projects in the Brazilian Amazon

Thales A. P. Westa,b,c,1, Jan Börnerc,d, Erin O. Sillse, and Andreas Kontoleonb,f
a
Land Use Economics and Climate Division, Scion–New Zealand Forest Research Institute, Rotorua 3010, New Zealand; bCentre for Environment, Energy and
Natural Resource Governance, University of Cambridge, Cambridge CB3 9EP, United Kingdom; cCenter for Development Research, University of Bonn, 53113
Bonn, Germany; dInstitute for Food and Resource Economics, University of Bonn, 53115 Bonn, Germany; eDepartment of Forestry and Environmental
Resources, North Carolina State University, Raleigh, NC 27695; and fDepartment of Land Economy, University of Cambridge, Cambridge CB3 9EP, United
Kingdom

Edited by Eric F. Lambin, Stanford University, Stanford, CA, and approved August 12, 2020 (received for review March 6, 2020)

Reducing emissions from deforestation and forest degradation constructed by projecting the forest cover expected in the ab-
(REDD+) has gained international attention over the past decade, sence of REDD+ (9). These baseline scenarios typically assume
as manifested in both United Nations policy discussions and hun- a continuation of historical deforestation trends (10), and thus
dreds of voluntary projects launched to earn carbon-offset credits. eventually become unrealistic counterfactuals as the regional eco-
Downloaded from https://www.pnas.org by JOSIP JURAJ STROSSMAYER UNIVERSITY OF OSIJEK on October 13, 2022 from IP address 193.198.137.10.

There are ongoing discussions about whether and how projects nomic and political context change. Notably, these types of changes
should be integrated into national climate change mitigation efforts were observed in the Brazilian Amazon during 2004–2012, a period
under the Paris Agreement. One consideration is whether these of sharply declining rates of forest loss (11), and also during 2019,
projects have generated additional impacts over and above national when deforestation soared again (12) (Fig. 1). Consequently, credits
policies and other measures. To help inform these discussions, we for reduced deforestation (or lack thereof) claimed by voluntary
compare the crediting baselines established ex-ante by voluntary REDD+ projects in the Brazilian Amazon may have been artifacts
REDD+ projects in the Brazilian Amazon to counterfactuals con- of external factors rather than REDD+ activities. Furthermore,
structed ex-post based on the quasi-experimental synthetic control critics of voluntary REDD+ projects have raised concerns that
method. We find that the crediting baselines assume consistently deforestation baselines might be intentionally inflated by profiteers
higher deforestation than counterfactual forest loss in synthetic seeking to financially benefit from the commercialization of su-
control sites. This gap is partially due to decreased deforestation perfluous credits, or “hot air” (13–15). In addition to the direct cost
in the Brazilian Amazon during the early implementation phase of of not effectively offsetting GHG emissions, the excess credits
the REDD+ projects considered here. This suggests that forest car- generated by these projects impose an indirect cost on legitimate
bon finance must strike a balance between controlling conservation climate change mitigation efforts by undercutting the price of
investment risk and ensuring the environmental integrity of carbon their credits.
emission offsets. Relatedly, our results point to the need to better Early efforts to address these concerns included the estab-
align project- and national-level carbon accounting. lishment of standards and registries for voluntary carbon-offset
projects. These standards were designed to ensure the environ-
| |
impact evaluation synthetic control payment for environmental mental integrity of carbon offsets by requiring projects to use
| |
services carbon credit deforestation approved carbon-accounting methodologies for establishing de-
forestation baselines, monitoring, and reporting, all subject to

C oncerns over global warming have led both the public and
private sectors to promote climate change mitigation through
the reduction of carbon (CO2) emissions from deforestation and
third-party audits. Among those, the verified carbon standard

Significance
forest degradation in tropical countries—a concept known as
REDD+ (1). This strategy gained international attention after There are efforts to integrate the reduced carbon emissions
2005 as a voluntary, performance-based payment mechanism for from avoided deforestation claimed by voluntary REDD+ pro-
reduced carbon emissions (2). While the regulations and capacity jects into national greenhouse gas emission inventories. This
for national REDD+ programs are still under development in requires careful consideration of whether and how much of the
many countries, hundreds of voluntary, subnational REDD+ reduced carbon emissions can be attributed to projects. How-
projects are operational worldwide (3). These projects intend to ever, credible evidence on the effectiveness of such voluntary
preserve forests through a variety of activities, e.g., improved activities is limited. We adopted the quasi-experimental syn-
monitoring and control, promotion of sustainable land uses, and thetic control method to examine the causal effects of 12 vol-
engagement of local communities (4), either as proof of concept untary REDD+ projects in the Brazilian Amazon. We compared
or to profit from the commercialization of “carbon-offset credits” these ex-post estimates of impacts with the reductions in for-
(i.e., Mg CO2 removed from or not emitted to the atmosphere) in est loss claimed by those projects based on ex-ante baselines.
a variety of markets. While these markets do not provide the level Results suggest that the accepted methodologies for quanti-
of funding originally envisioned for national REDD+ programs, fying carbon credits overstate impacts on avoided deforesta-
they are substantial: In 2018 alone, the volume of carbon offsets tion and climate change mitigation.
traded totaled 98.4 million Mg CO2, with a market value of
US$295.7 million; a third of those credits (30.5 million Mg CO2) Author contributions: T.A.P.W., J.B., E.O.S., and A.K. designed research; T.A.P.W. per-
were generated by REDD+ projects (5). The Paris Agreement formed research; T.A.P.W. analyzed data; and T.A.P.W., J.B., E.O.S., and A.K. wrote
the paper.
has raised thorny questions about how the carbon emission re-
ductions claimed by these projects relate to nationally determined The authors declare no competing interest.

contributions (NDCs) and national greenhouse gas (GHG) This article is a PNAS Direct Submission.
emission inventories reported to the United Nations Framework Published under the PNAS license.
Convention on Climate Change (6–8). 1
To whom correspondence may be addressed. Email: [email protected].
Carbon credits from REDD+ [at both the project and na- This article contains supporting information online at https://www.pnas.org/lookup/suppl/
tional levels (1)] are issued based on performance, as defined by doi:10.1073/pnas.2004334117/-/DCSupplemental.
the comparison of realized forest cover to a baseline scenario First published September 14, 2020.

24188–24194 | PNAS | September 29, 2020 | vol. 117 | no. 39 www.pnas.org/cgi/doi/10.1073/pnas.2004334117

 displaced instead (24). Finally, we contrast our counterfactuals to
the crediting baselines adopted by the voluntary projects.

Results
Before assessing the impacts of the REDD+ projects, we ex-
Deforestation (1000 km2)

plored whether the synthetic controls can accurately replicate

deforestation trends in the project areas without REDD+. This
20
“proof of concept” was implemented by dividing the pretreat-
ment period (i.e., before project implementation) into “training”
and “testing” periods. We found that the synthetic control
method was able to replicate pretreatment deforestation trends
reasonably well in 10 of the 12 synthetic controls (SI Appendix,
Fig. S2). Our findings for the other two projects (i.e., Jari/Amapá
and Suruí) must be interpreted with particular caution.
10
Deforestation in the REDD+ Areas. Overall, we find no significant
evidence that voluntary REDD+ projects in the Brazilian Am-
Downloaded from https://www.pnas.org by JOSIP JURAJ STROSSMAYER UNIVERSITY OF OSIJEK on October 13, 2022 from IP address 193.198.137.10.

azon have mitigated forest loss. Deforestation is consistently

lower in the REDD+ project site than in the synthetic control in
only four of the projects (Fig. 3 and SI Appendix, Fig. S3), and
this difference is only outside the confidence interval around
0 zero established by the placebo tests in one project (Maísa; Fig. 4
and SI Appendix, Fig. S4). The only two REDD+ projects from
2000 2003 2006 2009 2012 2015 2018 our sample that were implemented in protected areas, i.e., Suruí

SUSTAINABILITY
Year and Rio Preto-Jacundá, experienced among the largest cumu-
lative losses of forest cover after REDD+ implementation, along
with Jari/Amapá (Fig. 3). This is partly a function of their large

SCIENCE
Fig. 1. Annual deforestation in the Brazilian Amazon from PRODES data
(bars). The blue bars indicate voluntary REDD+ project start dates. The red project areas and the widespread forest fires that occurred in
lines illustrate 10-y deforestation averages prior to project implementation, those protected areas in 2010–2011 and 2015, respectively (see
commonly adopted as projects’ deforestation baselines. SI Appendix for details). For Rio Preto-Jacundá, we find much
higher deforestation than in its synthetic control (which is the
same order of magnitude in size); specifically, the differences
(VCS) (16) has certified the greatest number of voluntary between deforestation (both cumulative and annual) in the
REDD+ projects worldwide (5). Rio Preto-Jacundá area and its synthetic controls were sub-
Despite the growing literature on local REDD+ interventions, stantially greater than the differences between deforestation
there have only been a few evaluations of their impacts on car- in the placebos and their synthetic controls (Fig. 4 and SI
bon emissions using rigorous, counterfactual-based methods Appendix, Fig. S4).
(17–19). This study systematically compares deforestation base- Across all projects, we find substantial differences between the
lines established ex-ante with counterfactual estimates of de- deforestation baseline scenarios adopted ex-ante by the REDD+
projects and the observed forest loss (ex-post) in the synthetic
forestation constructed ex-post. We employ the synthetic control
controls (Fig. 5 and SI Appendix, Fig. S5). The Suruí project,
method to construct deforestation counterfactuals and assess the
implemented in an indigenous territory, is the only case where
reductions in forest loss that can be attributed to voluntary the synthetic control deforestation exceeded the baseline de-
REDD+ projects (20–22). We apply this method to all VCS- forestation adopted by the project proponents. This may reflect
certified REDD+ projects for unplanned deforestation imple- the fact that the baseline for Suruí was developed based on a
mented in the Brazilian Amazon in the last decade (2008–2017; participatory, system dynamics model (25), as opposed to the
Fig. 2 and SI Appendix, Table S1). We focus on this region for assumptions based on historical deforestation trends adopted by
several reasons: its global relevance for conservation and all other projects (see SI Appendix for details).
REDD+; the ongoing discussions in Brazil about “nesting”
voluntary projects into a national REDD+ program (6–8); and Carbon Offset Implications. Credits from the voluntary REDD+
the recent availability of a spatially explicit cadastral database projects are generally issued after a third-party audit (i.e., verifica-
(23) that allows us to define a pool of rural properties similar to tion) every 1 to 5 y. These credits are based on the estimated carbon-
the REDD+ project areas. We construct synthetic controls from emission reductions from the avoided deforestation brought about
donor pools of properties based on weighted combinations of by the projects, calculated as the difference between the carbon
accessibility and biophysical characteristics that result in the best emissions under the baseline scenario minus the observed emissions
matches of historical deforestation trends. Unlike the typical from the project area and leakage.
According to the projects’ ex-ante estimates, up to 24.8 million
approach to crediting baselines, we then construct counterfactual
carbon offsets could potentially have been generated by the
deforestation scenarios based on the actual deforestation ob-
REDD+ interventions by 2017 (Fig. 5 and SI Appendix, Table
served in those synthetic controls during the period when the S1). According to the VCS database, only 5.4 million tradable
REDD+ projects were operational. We evaluate whether the credits from these projects have been certified and made avail-
REDD+ projects caused additional reductions in deforestation able to offset GHG emissions from private and public sources by
compared to the counterfactual deforestation as represented by that year (SI Appendix, Table S1) (26). Using the synthetic
the synthetic controls (i.e., REDD+ additionality) and assess the control method to estimate REDD+ counterfactuals, we find no
robustness of our results with placebo tests (22). We also ex- systematic evidence that the certified carbon offsets claimed by
amine trends in forest loss in buffer zones around the REDD+ the voluntary projects in our sample (with the exception of
project areas after project implementation to assess the plausibility Maísa) are associated with additional reductions in deforestation
that any apparent reductions in deforestation may have been in the REDD+ areas above and beyond the background

West et al. PNAS | September 29, 2020 | vol. 117 | no. 39 | 24189
 R
R oo rr aa ii m
m aa
A
Amm aa pp áá

A
Amm aa zz oo nn aa ss P
P aa rr áá
M
M aa rr aa nn hh ãã oo

A
A cc rr ee

R
R oo nn dd ôô nn ii aa
Downloaded from https://www.pnas.org by JOSIP JURAJ STROSSMAYER UNIVERSITY OF OSIJEK on October 13, 2022 from IP address 193.198.137.10.

Forest in 2017
M
M aa tt oo G
G rr oo ss ss oo
Non-forest in 2017
Water
REDD+ projects

Kilometers
0 125 250 500 750 1,000

Fig. 2. VCS-certified REDD+ projects established during 2008–2017 in the Brazilian Amazon forest biome.

reduction in deforestation achieved in the Brazilian Amazon baseline. The opposite happens when unanticipated forest
over the same period (11). Even for the Maísa case, our results threats, such as fires, emerge at the regional scale.
suggest that nearly 40% of the 50,000 tradable carbon offsets In contrast, the synthetic control methodology uses historical
issued by the project by 2017 (SI Appendix, Table S1) may not be trends to identify appropriate weighted combinations of com-
genuinely additional (Fig. 5). parison areas but then constructs the counterfactual based on the
observed deforestation in those areas. These counterfactuals
Leakage. If REDD+ implementation mitigates forest loss in thus incorporate the effects of contemporaneous drivers of de-
project areas by effectively excluding deforestation agents, it forestation, including agricultural commodity prices, currency
could displace, and hence increase, deforestation next to the exchange rates, and environmental regulations (28–30). As such,
project areas. Shifts in deforestation after project start in 10-km the synthetic control method is less prone to incorrectly attribute
buffer zones surrounding the REDD+ projects suggest that such changes in deforestation to REDD+.
leakage effects could have occurred in three cases (i.e., Maísa, We note some caveats on our analysis. First, we base our
Florestal Santa Maria, and Manoa; SI Appendix, Fig. S6). Fur- evaluation on the project boundaries defined by the polygons
thermore, leakage presupposes a direct conservation impact, and available from the VCS project database, which are somewhat
all three of the projects exhibited lower deforestation than their larger than the areas officially reported by project proponents
synthetic controls, although this estimated effect of REDD+ is (SI Appendix, Table S2). Most of those polygons correspond to
only larger than the placebo tests in the Maísa project (Fig. 4 and Amazonian rural properties registered in the Brazilian Rural
SI Appendix, Fig. S4). It is also worth noting that while defor- Environmental Registry (CAR), whose owners are legally enti-
tled to clear up to 20% of their forest area. Second, our synthetic
estation in the buffer zones of these three projects rose between
controls do not perfectly match the REDD+ project areas in
the project start dates and 2017, postintervention rates were still
terms of size, accessibility, and biophysical characteristics. In
lower on average than in the pre-REDD+ period.
particular, the synthetic control for Agrocortex is only 61% the
Discussion size of the project area (SI Appendix, Tables A1 and A2). While
historical deforestation is similar in the synthetic controls and
Our findings partially support early skepticism about the con-
project areas, clearly there is future potential for more defor-
tribution of voluntary REDD+ projects to climate change miti-
estation in the larger project areas than in their smaller synthetic
gation (15, 27). In particular, they raise questions about the controls. Third, the construction of our synthetic controls may
environmental integrity of offsets calculated using deforestation not have included all relevant structural determinants of defor-
counterfactuals based on the continuation of historical trends estation. Last, the period of analysis may not have been long
(e.g., Fig. 1). In all projects that established crediting baselines enough to observe significant REDD+ impacts in some cases.
using historical trends, we find that the crediting baselines sig- Despite these caveats, the weight of the evidence suggests that
nificantly overstate deforestation in comparison to the counter- these projects caused less reduction in deforestation than claimed
factual estimates based on synthetic controls. This pattern (Fig. 5 and SI Appendix, Fig. S5) and that few projects actually
reflects the confounding effect created by Brazil’s post-2004 ef- achieved emission reductions. Suspicion about the environmental
forts to control Amazonian deforestation that were uniquely integrity of carbon offsets is not restricted to REDD+ or voluntary
successful (11, 28, 29). If carbon credits are expected to reflect interventions. A series of reports on other market-based initiatives
changes in emissions caused by REDD+, then using historical for climate change mitigation, i.e., the Joint Implementation (JI)
baselines leads to excess carbon credits for projects when de- and the Clean Development Mechanism (CDM) of the Kyoto
forestation at the regional level drops below the historical Protocol, also raised concerns about the true climatic contributions

24190 | www.pnas.org/cgi/doi/10.1073/pnas.2004334117 West et al.

Downloaded from https://www.pnas.org by JOSIP JURAJ STROSSMAYER UNIVERSITY OF OSIJEK on October 13, 2022 from IP address 193.198.137.10.

SUSTAINABILITY
SCIENCE
Fig. 3. Cumulative post-2000 deforestation in Amazonian areas with REDD+ projects (red) versus synthetic controls (blue). The dashed black lines are the
project start dates.

from certified carbon offsets. These reports suggest that about projects are eligible to issue large amounts of carbon offsets at
three-quarters of JI credits are unlikely to represent additional the time of certification, retroactively corresponding to the pe-
emission reductions (31) and that 73% of the potential 2013–2020 riod between the certification and the project start date. This can
CDM credits have a low likelihood of environmental integrity (in help to fund project start-ups, but it also implies that projects
contrast to 7% with high likelihood) (32). have not actually had access to carbon revenues during their
The projects that we evaluated may have had little additional early years of operation. Carbon crediting rules may thus partially
impact because they did not adopt the most effective actions to explain why we find limited evidence for avoided deforestation.
achieve their REDD+ objectives, perhaps because of uncer- Our results emphasize the need to reassess approaches to
tainties about the future availability of funds or concerns about measuring project additionality. While ex-post counterfactual
unfairly raising local expectations of carbon payments. Hence, methods such as illustrated here would ensure a high level of
our results do not imply that voluntary REDD+ projects cannot environmental integrity, they would introduce substantial un-
achieve their objectives if designed and implemented effectively. certainty about the credits that can be obtained from a given
There is both quasi-experimental and experimental evidence that reduction in deforestation in project areas. An alternative ap-
conditional payments for environmental services (PES) can ef- proach often suggested in the literature is to require projects to
fectively reduce deforestation (3, 33), and recent literature sug- adopt national or subnational (jurisdictional) baselines that are
gests that REDD+ implemented through well-designed conditional predefined, and periodically updated, by the government (6, 7,
PES can deliver positive conservation outcomes (34–36). 41), as well as default carbon-stock values or a common carbon-
Another possible explanation for the lack of impact is diffi- density map (42). Imposing one common baseline would have
culty with the on-the-ground implementation and execution of the benefits of facilitating the inclusion of carbon emission re-
activities envisioned by project proponents (37, 38). One exam- ductions claimed by decentralized initiatives into national GHG
ple is the Suruí project, which attracted international attention as emission inventories, ensuring consistency in the treatment of
one of the first voluntary REDD+ interventions implemented in leakages, and avoiding double-counting reductions (6, 8, 43),
an indigenous territory (4). The project aimed to use the finan- while still offering relative certainty about carbon credits con-
cial revenues from carbon sales to promote sustainable land-use ditional on project performance. However, national and subna-
practices in the Suruí territory but was not able to prevent the tional baselines are typically based on historical data and thus
illegal invasion of loggers and miners. are not any more likely to capture contemporaneous deforesta-
A third possible explanation for underperformance relates to tion drivers and their dynamism [although it is also possible to
challenges with the commercialization of carbon offsets and apply the synthetic control method to nations (44)]. Thus, they
correspondingly limited revenues available to implement project do not address the main problem identified by our analysis: the
activities (39). One way that voluntary REDD+ projects over- limitations of historical data for baseline development.
come that challenge is by claiming “retroactive credits” (40). Periodic baseline updates based on recent deforestation trends
Often, projects that are certified in a given year claim to have could help mitigate the influence of factors external to voluntary
started much earlier (SI Appendix, Table S1). As a result, those REDD+ projects on the carbon credits that they claim. In fact,

West et al. PNAS | September 29, 2020 | vol. 117 | no. 39 | 24191
Downloaded from https://www.pnas.org by JOSIP JURAJ STROSSMAYER UNIVERSITY OF OSIJEK on October 13, 2022 from IP address 193.198.137.10.

Fig. 4. Placebo tests: cumulative deforestation in REDD+ project areas minus deforestation in their respective synthetic controls (red), and placebos minus
their respective synthetic controls (blue dots). The dashed black lines are the project start dates (assumed the same for placebos). The shaded blue areas
represent 99% confidence intervals around the mean of the placebos. The number of placebos varies by project based on whether synthetic controls with low
MSPE could be constructed for the placebo tests.

current VCS rules already require projects to revise their base- Suruí and Rio Preto-Jacundá , were implemented in an indigenous territory
lines every 10 y (16). Our results suggest that this interval should and a sustainable-use reserve, respectively. Following VCS-approved carbon-
accounting methodologies, historical deforestation rates were the basis of all
be shorter. Baseline updates could be based on control areas that
project deforestation baselines with the exception of the Suruí project (e.g.,
share similar characteristics as the REDD+ projects, as dem- Fig. 1). In the latter, baseline deforestation rates were informed by a partici-
onstrated in this study with the construction of the synthetic patory, and community-specific, system dynamics model (25).
controls. In addition, coupled human–natural system models, Rigorous impact evaluations rely on the establishment of credible coun-
such as was used in the Suruí case, can be used to explore al- terfactuals for what would have happened in the absence of an intervention
ternative baseline scenarios and quantify the potential downside (48, 49), which are unobservable. We construct “synthetic controls” to serve
risks involved in conservation investments under dynamic pat- as counterfactuals for the REDD+ project areas (20, 50). We adopted this
terns of land-use change, although at increased project devel- approach, as opposed to more traditional methods from the impact evalu-
opment costs (25). These models could also shed light on the ation literature (e.g., difference-in-differences estimator), because of our
small number of treated units and likely heterogeneity of the treatment
potential impacts of REDD+ on local livelihoods and biodi-
across them (49, 51, 52). Synthetic controls were constructed as a weighted
versity (45, 46), which we do not consider here but recognize as average of selected donor units through a nested optimization procedure
fundamentally important. that minimizes the differences in pretreatment characteristics between the
We do provide empirical evidence for a phenomenon that was project and the control, with characteristics weighted such that the resulting
anticipated in the early policy debate over REDD+ (47), i.e., de weighted average outcome of the selected donor units most closely matches
facto additionality of REDD+ projects depends on both project the pretreatment outcome in the treated unit (21, 22). Specifically, the it-
implementation and national circumstances. Carbon finance and erative procedure minimizes the mean squared prediction error (MSPE) of
crediting systems must safeguard against both hot air from the outcome, or the sum of squared residuals between the treated unit and
overstated claims of carbon additionality and excessive risks to the synthetic control, over the pretreatment period (50).
Two sets of synthetic controls were constructed as a weighted combination of
private conservation investments associated with desirable gov-
areas selected from “donor pools” (20, 50) composed of Amazonian properties
ernment action to combat deforestation, as observed in Brazil registered in the CAR database (23) that do not overlap with project areas and
from 2005 to 2012. that had ≥90% forest cover in the first year of the analysis. In the first set, we
used cumulative deforestation as the optimization outcome, whereas the sec-
Materials and Methods ond set was based on annual deforestation. We note that the optimization
We examined the impacts of 12 voluntary REDD+ projects implemented in the algorithm selected different donors for the synthetic controls for each outcome,
Brazilian Amazon since 2008 and certified under the VCS before May 2019 to which allows us to use the second set as a robustness check. Donor pools were
curb local unplanned deforestation (Fig. 2 and SI Appendix, Tables S1 and S2). preferably based on properties from the same state as the REDD+ project and
Project areas were defined by the geospatial polygons reported by the project within ±25% the size of the project area. Whenever the resulting synthetic
proponents and available from the VCS project database. Ten of the 12 pro- controls had substantially different land areas or pretreatment annual and
jects were implemented in privately owned properties, whereas the other two, cumulative deforestation (i.e., before project implementation), the donor pools

24192 | www.pnas.org/cgi/doi/10.1073/pnas.2004334117 West et al.

Downloaded from https://www.pnas.org by JOSIP JURAJ STROSSMAYER UNIVERSITY OF OSIJEK on October 13, 2022 from IP address 193.198.137.10.

SUSTAINABILITY
SCIENCE
Fig. 5. Cumulative deforestation from the baseline scenarios adopted by the REDD+ projects (orange) versus observed cumulative deforestation in the
synthetic controls (blue). The dashed black lines are the project start dates.

were expanded to all properties in the Amazon biome (see SI Appendix for MSPEs (SI Appendix, Table S3). This proof of concept differs from standard
details). Last, for the cases of persistent unbalanced synthetic controls, donor model-validation practices because the donors selected as synthetic controls
pools were expanded to properties with ±50% the size of the project area. based on the first half of the pretreatment periods do not necessarily match
Synthetic controls for the REDD+ projects implemented in a sustainable-use the final set of donors when the full pretreatment period is used.
reserve (i.e., Rio Preto-Jacundá) and an indigenous territory (i.e., Suruí) were We examined the robustness of our findings with a series of placebo tests,
constructed based on donor pools composed of other sustainable-use reserves in which we create synthetic controls for all CAR polygons in the donor pool
and indigenous territories, respectively. (i.e., not subject to REDD+ activities) and compute the difference in both
The spatial covariates structurally related to deforestation (30) used for annual and cumulative deforestation between each placebo and its syn-
the construction of the synthetic controls were obtained from official maps thetic control (Fig. 4 and SI Appendix, Fig. S4). Because placebo areas are not
produced by government agencies in Brazil (SI Appendix, Fig. S7 and Table
exposed to REDD+, any differences in forest loss between placebos and their
S4). The covariates represent 1) property size, 2) initial forest cover, 3) slope,
synthetic controls are statistical “noise.” In order to increase the number of
4) soil quality, and distances from 5) state capitals, 6) towns, 7) federal
placebo tests, we use the expanded placebo donor pools of all Amazonian
highways, and 8) local roads, as well as the proportion of 9) primary and 10)
properties within ±50% the project size. In accordance with the previous
secondary forest, 11) pastureland, 12) agriculture, and 13) urban areas in
literature (22), we discarded placebo tests with pretreatment MSPE five
2000, 2004, 2008, and 2012 (for projects implemented after 2012) within
times higher than the pretreatment MSPE of the REDD+ polygon. We used
10-km buffer zones of the project and potential donor areas. In accordance
with the previous literature (21, 50), we also used the pretreatment annual the gaps in deforestation between the placebos and their respective syn-
and cumulative deforestation rates to inform the construction of the two thetic controls to create 99% confidence intervals around the mean placebo
sets of synthetic controls. Temporal land-use information in the buffer zones effect estimate, which is approximately zero in all cases. Analyses were
was obtained from the TerraClass dataset produced by Brazil’s National In- conducted with the Synth package (version 1.1) available for R software
stitute for Space Research. Annual deforestation data for the 2001–2017 (version 3.6.0) (50). Last, we computed the annual deforestation in 10-km
period were processed from the MapBiomas land-use/cover dataset, version buffer zones surrounding the project areas as an indicator of possible
3.1, for the Brazilian Amazon biome (Fig. 2 and SI Appendix, Fig. S1). leakage effects (24), i.e., because increasing deforestation could reflect the
While the construction of our synthetic controls was based on all infor- displacement of deforestation due to the REDD+ activities.
mation available from 2001 to the project start year (i.e., pretreatment pe-
riod), we conducted a separate analysis in which a different set of synthetic Data Availability. All study data are included in the article and SI Appendix.
controls were constructed based on data constrained to the first half of the
pretreatment period (i.e., training period), so they could be tested against the ACKNOWLEDGMENTS. We thank two anonymous reviewers for valuable
second half (i.e., testing period; SI Appendix, Fig. S2). We evaluated the comments and suggestions. J.B. acknowledges support from the German
outcome of this analysis both visually and by comparing training and testing Federal Ministry of Education and Research.

1. A. Angelsen, REDD+ as result-based aid: General lessons and bilateral agreements of 3. J. Börner et al., “National and subnational forest conservation policies—What works,
Norway. Rev. Dev. Econ. 21, 237–264 (2017). what doesn’t” in Transforming REDD+: Lessons and New Directions, A. Angelsen, Ed.
2. UN-REDD, The UN-REDD Programme Strategy 2011–2015, (United Nations Collaborative (Center for International Forestry Research, Bogor, Indonesia, 2018), pp. 105–116.
Programme on Reducing Emissions from Deforestation and Forest Degradation in De- 4. T. A. P. West, Indigenous community benefits from a de-centralized approach to
veloping Countries, Geneva, 2011). REDD+ in Brazil. Clim. Policy 16, 924–939 (2016).

West et al. PNAS | September 29, 2020 | vol. 117 | no. 39 | 24193
 5. S. Donofrio, P. Maguire, W. Merry, S. Zwick, “Financing emissions reductions for the 29. J. Assunção, C. Gandour, R. Rocha, Deforestation slowdown in the Brazilian Amazon:
future: State of the voluntary carbon markets 2019” (Forest Trends’ Ecosystem Mar- Prices or policies? Environ. Dev. Econ. 20, 697–722 (2015).
ketplace, Washington, DC, 2019). 30. J. Busch, K. Ferretti-Gallon, What drives deforestation and what stops it? A meta-
6. D. Lee, P. Llopis, R. Waterworth, G. Roberts, T. Pearson, “Approaches to REDD+ analysis. Rev. Environ. Econ. Policy 11, 3–23 (2017).
nesting: Lessons learned from country experiences” (World Bank, Washington, DC, 31. A. Kollmuss, L. Schneider, V. Zhezherin, “Has joint implementation reduced GHG
2018). emissions? Lessons learned for the design of carbon market mechanisms” (Stockholm
7. Verified Carbon Standard, “Jurisdictional and nested REDD+ (JNR) requirements” Environment Institute, Stockholm, 2015).
(Verified Carbon Standard, Washington, DC, 2017). 32. M. Cames et al., “How additional is the Clean Development Mechanism?” (Institute
8. Food and Agriculture Organization, “From reference levels to results reporting: for Applied Ecology, Berlin, 2016).
REDD+ under the United Nations Framework Convention on Climate Change. 2019 33. E. O. Sills, K. Jones, “Causal inference in environmental conservation: The role of
update” (Food and Agriculture Organization, Rome, 2019).
institutions” in Handbook of Environmental Economics, K.-G. Maler, J. R. Vincent, Eds.
9. T. A. P. West et al., A hybrid optimization-agent-based model of REDD+ payments to
(Elsevier, Amsterdam, 2018), pp. 395–437.
households on an old deforestation frontier in the Brazilian Amazon. Environ. Model.
34. S. Jayachandran et al., Cash for carbon: A randomized trial of payments for ecosystem
Softw. 100, 159–174 (2018).
services to reduce deforestation. Science 357, 267–273 (2017).
10. C. Dezécache, J.-M. Salles, B. Hérault, Questioning emissions-based approaches for the
35. G. Simonet, J. Subervie, D. Ezzine‐de‐Blas, M. Cromberg, A. E. Duchelle, Effectiveness
definition of REDD+ deforestation baselines in high forest cover/low deforestation
of a REDD+ project in reducing deforestation in the Brazilian Amazon. Am. J. Agric.
countries. Carbon Balance Manag. 13, 21 (2018).
Econ. 101, 211–229 (2019).
11. T. A. P. West, J. Börner, P. M. Fearnside, Climatic benefits from the 2006–2017 avoided
36. P. Cuenca, J. Robalino, R. Arriagada, C. Echeverría, Are government incentives ef-
deforestation in Amazonian Brazil. Front. For. Glob. Chang. 2, 52 (2019).
12. L. Ferrante, P. M. Fearnside, Brazil’s new president and “ruralists” threaten Ama- fective for avoided deforestation in the tropical Andean forest? PLoS One 13,
zonia’s environment, traditional peoples and the global climate. Environ. Conserv. 46, e0203545 (2018).
261–263 (2019). 37. G. Simonet et al., “Forests and carbon: The impacts of local REDD+ initiatives” in
Downloaded from https://www.pnas.org by JOSIP JURAJ STROSSMAYER UNIVERSITY OF OSIJEK on October 13, 2022 from IP address 193.198.137.10.

13. O. Mertz et al., Uncertainty in establishing forest reference levels and predicting Transforming REDD+: Lessons and New Directions, A. Angelsen, Ed. et al. (Center for
future forest-based carbon stocks for REDD+. J. Land Use Sci. 13, 1–15 (2018). International Forestry Research, Bogor, Indonesia, 2018), pp. 117–130.
14. S. W. Rifai, T. A. P. West, F. E. Putz, “Carbon cowboys” could inflate REDD+ payments 38. A. E. Duchelle, C. De Sassi, E. O. Sills, S. Wunder, “People and communities” in
through positive measurement bias. Carbon Manag. 6, 151–158 (2015). Transforming REDD+: Lessons And New Directions, A. Angelsen, Ed. et al. (Center for
15. C. Seyller et al., The “virtual economy” of REDD+ projects: Does private certification International Forestry Research, Bogor, Indonesia, 2018), pp. 131–141.
of REDD+ projects ensure their environmental integrity? Int. Rev. 18, 231–246 (2016). 39. T. Laing, L. Taschini, C. Palmer, Understanding the demand for REDD+ credits. Envi-
16. Verra, VCS Standard (Version 4.0, Verra, Washington, DC, 2019). ron. Conserv. 43, 389–396 (2016).
17. E. O. Sills et al., Building the evidence base for REDD+: Study design and methods for 40. N. Linacre, R. O’Sullivan, D. Ross, L. Durschinger, “REDD+ supply and demand
evaluating the impacts of conservation interventions on local well-being. Glob. En- 2015–2025” (US Agency for International Development Forest Carbon, Markets and
viron. Change 43, 148–160 (2017). Communities Program, Washington, DC, 2015).
18. A. E. Duchelle, G. Simonet, W. D. Sunderlin, S. Wunder, What is REDD+ achieving on 41. L. Pedroni, M. Dutschke, C. Streck, M. E. Porrúa, Creating incentives for avoiding
the ground? Curr. Opin. Environ. Sustain. 32, 134–140 (2018). further deforestation: The nested approach. Clim. Policy 9, 207–220 (2009).
19. A. B. Bos et al., Comparing methods for assessing the effectiveness of subnational 42. G. P. Asner et al., High-resolution forest carbon stocks and emissions in the Amazon.
REDD plus initiatives. Environ. Res. Lett. 12 (2017). Proc. Natl. Acad. Sci. U.S.A. 107, 16738–16742 (2010).
20. E. O. Sills et al., Estimating the impacts of local policy innovation: The synthetic 43. L. Schneider et al., Double counting and the Paris Agreement rulebook. Science 366,
control method applied to tropical deforestation. PLoS One 10, e0132590 (2015). 180–183 (2019).
21. A. Abadie, J. Gardeazabal, The economic costs of conflict: A case study of the Basque
44. A. Roopsind, B. Sohngen, J. Brandt, Evidence that a national REDD+ program reduces
country. Am. Econ. Rev. 93, 113–132 (2003).
tree cover loss and carbon emissions in a high forest cover, low deforestation country.
22. A. Abadie, A. Diamond, J. Hainmueller, Synthetic control methods for comparative
Proc. Natl. Acad. Sci. U.S.A. 116, 24492–24499 (2019).
case studies: Estimating the effect of California’s tobacco control program. J. Am.
45. T. A. P. West et al., Impacts of REDD+ payments on a coupled human-natural system
Stat. Assoc. 105, 493–505 (2010).
in Amazonia. Ecosyst. Serv. 33, 68–76 (2018).
23. A. A. Azevedo et al., Limits of Brazil’s Forest Code as a means to end illegal defor-
46. T. Iwamura, E. F. Lambin, K. M. Silvius, J. B. Luzar, J. M. V. Fragoso, Socio-
estation. Proc. Natl. Acad. Sci. U.S.A. 114, 7653–7658 (2017).
24. L. Aukland, P. M. Costa, S. Brown, A conceptual framework and its application for environmental sustainability of indigenous lands: Simulating coupled human-
addressing leakage: The case of avoided deforestation. Clim. Policy 3, 123–136 (2003). natural systems in the Amazon. Front. Ecol. Environ. 14, 77–83 (2016).
25. C. S. M. N. Vitel et al., Land-use change modeling in a Brazilian indigenous reserve: 47. A. Angelsen, Moving Ahead with REDD: Issues, Options and Implications, (Center for
Construction of a reference scenario for the Suruí REDD Project. Hum. Ecol. 41, International Forestry Research, Bogor, Indonesia, 2008).
807–826 (2013). 48. P. W. Holland, Statistics and causal inference. J. Am. Stat. Assoc. 81, 945–960 (1986).
26. K. Hamrick, M. Gallant, “Fertile ground: State of forest carbon finance 2017” (Forest 49. D. B. Rubin, Estimating causal effects of treatments in randomized and non-
Trends’ Ecosystem Marketplace, Washington, DC, 2017). randomized studies. J. Educ. Psychol. 66, 688–701 (1974).
27. Riksrevisjonen, “The Office of the Auditor General of Norway’s investigation of 50. A. Abadie, A. Diamond, J. Hainmueller, Synth: An R package for synthetic control
Norway’s International Climate and Forest Initiative” (Document 3:10, Riksrevisjonen, methods in comparative case studies. J. Stat. Softw. 42, 1–17 (2011).
Bergen, Norway, 2018). 51. P. R. Rosenbaum, D. B. Rubin, The central role of the propensity score in observational
28. J. Börner, K. Kis-Katos, J. Hargrave, K. König, Post-crackdown effectiveness of field- studies for causal effects. Biometrika 70, 41–55 (1983).
based forest law enforcement in the Brazilian Amazon. PLoS One 10, e0121544 52. A. Abadie, A. Diamond, J. Hainmueller, Comparative politics and the synthetic control
(2015). method. Am. J. Pol. Sci. 59, 495–510 (2015).

24194 | www.pnas.org/cgi/doi/10.1073/pnas.2004334117 West et al.