Professional Documents
Culture Documents
Development and Testing of A Community Flood Resilience Measurement Tool
Development and Testing of A Community Flood Resilience Measurement Tool
Received: 24 May 2016 – Published in Nat. Hazards Earth Syst. Sci. Discuss.: 31 May 2016
Revised: 8 December 2016 – Accepted: 19 December 2016 – Published: 26 January 2017
ability of “resilience strengthening” initiatives. As identi- A central critique of resilience thinking is that it is a nor-
fied by National Academies of Sciences (NRC, 2012) and mative approach that accepts the system(s) as a given and
Levine (2014) among many others, there is an urgent need works within it, crowding out space for questioning the un-
for the continued development of theoretically anchored and derlying problems. Berkes and Ross (2013) identify lack of
practically applicable disaster resilience measurement frame- attention to power and agency as key critiques of resilience
works and tools so that the field may (a) deepen understand- in DRM. That is, by focusing on existing community capac-
ing of the key components of “disaster resilience”, (b) en- ities, resilience thinking might miss important institutional
hance our ability to benchmark and measure disaster re- arrangements that are limiting community capacity. Further
silience over time, and (c) compare how resilience changes to this is the critique that resilience is attractive to the “small
as a result of different capacities, actions, interventions, and government” discourse and is being used to justify shifting
hazard events. risk from government onto citizens (Welsh, 2013). Our ap-
The primary purpose of this paper is to present the Zurich proach to resilience, outlined below, starts with the current
Alliance community flood resilience measurement frame- system before connecting with development and vulnerabil-
work and associated tool, developed by an alliance of NGOs, ity theory to put people at the centre of decisions regarding
academic institutions, and the private sector1 . Each member their risk and wellbeing.
brought a unique perspective and experience on the theo- One of the early tasks of the alliance was to identify the re-
retical and practical considerations for designing and using search gaps in community flood resilience. As part of this re-
a community flood resilience measurement framework and search we conducted a review of resilience definitions, which
tool. At the time of writing the tool has undergone baseline is summarized in Keating et al. (2014, 2017). We then came
implementation in 75 communities worldwide. The unique to our definition of disaster resilience which underlies the
contribution of this framework and approach is that it is measurement framework: the ability of a system, commu-
building a standardized database of pre-event characteristics nity, or society to pursue its social, ecological, and economic
and post-event outcomes, which will then be explored to gen- development and growth objectives, while managing its dis-
erate evidence empirical evidence for resilience-building ini- aster risk over time in a mutually reinforcing way. Central to
tiatives (discussed below). We trace the process of frame- this conceptualization are the key community capitals – so-
work and tool development and rollout, outlining key suc- cial, human, physical, financial, and natural – which holis-
cesses as well as challenges. We note that this paper is not tically make up the socio-economic system (DFID, 1999;
designed to build theory but rather to offer a practical appli- Keating et al., 2014, 2017). This conceptualization is cen-
cation of theory. As a secondary purpose of this paper, we be- tred on enhancing wellbeing as the goal of resilience, rather
lieve that by sharing our process others, such as researchers, than disaster risk management, which can be a means to
policymakers, and practitioners, entering the fledging field of resilience. This puts focus on the interplay between disas-
resilience measurement may learn from our experience. ter risk management and development trajectories, such that
Before delving into resilience measurement, we must first if one undermines the other then disaster resilience is not
identify what it is we are attempting to measure; for disas- achieved. A key example of undesirable interplay is that of
ter resilience this is not straightforward. “Resilience” has a erosive coping – where actions taken to respond and recover
long history, being adopted in different disciplines includ- from floods erode long-term wellbeing, potentially even trap-
ing engineering, psychology, ecological systems theory, and ping people in a poverty cycle (Heltberg et al., 2012). Alter-
economics. In the disasters field, the central interest of this natively, actions which are designed to enhance wellbeing,
paper, resilience has a long and multidisciplinary history (for but lead to an unbalanced increase in disaster risk, are also
a review see Zhou et al., 2010). As the concept took hold undesirable.
in the disaster literature and practice, efforts to define it be- Reviews and analyses of the state-of-the-art of disas-
came a priority. Currently there exists a plethora of defini- ter resilience measurement (Schipper and Langston, 2015;
tions of “disaster resilience” (see for example Adger et al., Winderl, 2014; Levine, 2014; Mitchell, 2013; Constas and
2005; NRC, 2012; ABD, 2013; DFID, 2011; IPCC, 2012; Barrett, 2013) catalogue a plethora of offerings focused from
Twigg, 2009). The field is characterized by lack of concep- the household to the national scale, from single hazards to
tual clarity and considerable debate (Zhou et al., 2010). multiple hazards to general resilience, and designed for dif-
ferent purposes. We, along with many others (Schipper and
1 The alliance members who designed and are managing the im- Langston, 2015; Levine, 2014), emphasize that there is no
one-size-fits-all “resilience” measure, nor should there be.
plementation of the tool are the International Federation of Red
Cross and Red Crescent Societies (IFRC), the International Insti-
The development of various and varied resilience measure-
tute for Applied Systems Analysis (IIASA), the Wharton Business ment frameworks should be seen as a positive step towards
School’s Risk Management and Decision Processes Center (Whar- understanding resilience and operationalizing the concept. At
ton), the international development non-governmental organization the same time, many scholars and practitioners would like to
Practical Action, and Zurich Insurance Group who are also funding know if there are any widely applicable capacities which pro-
the endeavour.
vide for disaster resilience or if they are solely contextually The initial programme will allow for preliminary empirical
and temporally specific. testing and feedback from implementers.
Empirical evidence for the existence of such capacities The paper proceeds as follows: first we review current
is a critical gap in the field. Despite the proliferation of thinking on measuring resilience and draw out pertinent chal-
frameworks, Winderl’s (2014) review concurs with many lenges, particularly relating to empirical validation. We then
others that “no general measurement framework for disas- describe the measurement framework development process.
ter resilience has been empirically verified yet” (p. 19). This Next we present the content of the framework and then de-
highlights a key challenge for any resilience-building ef- scribe the operationalized tool. In the conclusion we present
forts: if resilience cannot be empirically verified, how can our path forward for undertaking much needed empirical
we know we are measuring actual resilience? When most analysis to understand community flood resilience. By docu-
discuss “measuring resilience” they are really aiming to mea- menting our process, we hope that others looking to measure
sure resilient capacity before an event. Stakeholders need to resilience, or develop a resilience measurement framework
know before a risk event occurs whether they have effectively of their own, can learn from our experience.
balanced risk and opportunity in building capacity to with-
stand and recover from the event. Related to this is the fact
that civil groups, authorities, and NGOs would like to know 2 Review: the challenge of measuring resilience
whether activities they have implemented with the purpose of
building resilience have achieved their goal. Winderl (2014) Several national and international aid agencies have pro-
outlines the problem of circular reasoning when it comes to posed versions of resilience indicators (Alinovi et al., 2009;
measuring ex ante: if we determine a priori which charac- USAID, 2013) and a number of regional disaster resilience
teristics make a system resilient, then design interventions to indicators have also been developed (Cutter et al., 2010;
enhance these, then measure again – we will necessarily find Resilience Capacity Index, 2017). Twigg’s (2009) Charac-
that resilience has been increased. However, with no empir- teristics of Disaster Resilience Community is designed for,
ical evidence to justify the selection of those characteristics and in cooperation with, NGO and civil society organiza-
in the first instance, we have no real evidence that resilience tions; it systematically and extensively explores many fac-
has actually been increased. As a latent property disaster re- tors which may contribute to disaster resilience. More re-
silience is only visible, or “revealed”, after a disaster has oc- cently the BRACED project has proposed a composite in-
curred (Schipper and Langston, 2015; Frankenberger et al., dex to measure resilience based around tracking adaptation,
2014), yet measuring impacts or outcomes in isolation tells anticipation, absorption, and transformation (Bahadur et al.,
us very little of what contributed to actual resilience. This is a 2015).
critical distinction which is not always fully acknowledged in The last few years have seen a number of reviews of
disaster resilience definitions and measurement frameworks. disaster resilience measurement frameworks (Schipper and
However, this is precisely the distinction that is critical for Langston, 2015; Winderl, 2014; Ostadtaghizadeh et al, 2015;
empirically testing resilience measures. That is, to empiri- Oddsdottir et al., 2013). Schipper and Langston (2015) re-
cally test and validate a measure of resilience, it is required view 17 sets of indicators from the household to the national
that (a) implementation of the measurement occurs across level. We do not aim to reproduce such a review here, but
different contexts and is monitored consistently over time instead draw on these and other critical analyses to highlight
and (b) understanding and capturing both pre-event char- some of the challenges associated with measuring disaster
acteristics (what we call sources of resilience) and a set of resilience. There are many substantial differences between
pre-determined post-shock outcome measures occur, in or- national versus local-level disaster resilience and associated
der to learn which sources are most effective for the realiza- measurement frameworks. Because the framework presented
tion of resilient outcomes. We know of no disaster resilience here is at the community level, our review focuses on is-
measurement frameworks or programs which do this. It is sues and challenges at this scale. There are few examples of
this gap that the Zurich Alliance community flood resilience comprehensive community disaster resilience measurement
measurement framework and associated tool presented be- frameworks available, only a handful of which have been im-
low aims to fill in the context of community flood resilience. plemented in the field and none that are empirically validated
The content of our measurement framework has been drawn (Winderl, 2014); this is because measuring resilience is not
from the existing literature (providing theory and evidence) straightforward for two main reasons: (1) it is a latent quality
and alliance members and peer group expertise. It is an that is not revealed until tested and (2) the characteristics that
indicator-based approach consisting of 88 measures of poten- influence this latent quality are often a complex set of holistic
tial sources of resilience (listed in Appendix B) and 29 post- and qualitative characteristics.
flood outcome measures (also listed in Appendix B). At the Measuring latent and qualitative characteristics is not a
time of writing the tool is being tested by six NGOs, in ap- challenge unique to resilience measurement. Gathering and
proximately 75 communities across eight country programs. synthesizing subjective (opinion) and/or qualitative (narra-
tive) information is a mainstay of modern development mon-
itoring and evaluation (M & E) practice. The Resilience Mea- are present: an increased score in some areas might not
surement Technical Working Group has undertaken a com- actually afford a benefit until it reaches a certain threshold
prehensive review of, and developed a position on, qualita- or unless it is combined with some other characteristic or
tive data and subjective indicators for resilience measure- capacity. A further challenge is that resilience can change
ment (Constas et al., 2015). They find that qualitative and over time depending on changes both within and outside
subjective measures are essential to resilience analysis, par- the system (Holling, 2001). The extremely scale-, place-,
ticularly in relation to social factors. They suggest mixed and system-specific nature of capacity profiles also creates
methods approaches and that solid planning (including re- difficulties when attempting to generalize a set of key factors
sourcing) are required to undertake effective qualitative data which enhance resilience (Tol and Yohe, 2007; Vincent,
collection. Below we describe how our framework and tool 2007).
tackles this challenge. Lastly, Béné et al. (2012) and Levine (2014) put forward
Thus community disaster resilience measurement frame- the critique that a resilience-focused perspective runs the
works or indicators – including ours – share many common risk of diverting attention away from the most vulnerable or
theoretical and practical challenges. First, in doing this work, marginalized groups in a community, in favour of a more av-
initial questions immediately confronted are (1) defining an erage community-wide perspective. We must be cognizant of
appropriate scale of analysis both geographically and tem- the fact that building the resilience of the majority does not
porally – specifying boundaries such as “resilience of what necessarily meet the development needs of the poorest. Re-
to what?” (Carpenter et al., 2001) – and (2) identifying the silience is not a pro-poor concept, in that it does not exclu-
potential end users (“indicators for whom?”) and potential sively apply to, or benefit, the poor. As such, resilience build-
purposes (“indicators for what?”). This helps bring clarity ing does not replace poverty reduction (Béné et al., 2012) but
into the complex process of measurement framework devel- can rather be considered as one element in protecting current
opment (de Sherbinin et al., 2013). Without these specifics and future development gains.
it becomes increasingly difficult to distinguish disaster re-
silience measurement frameworks from general development
assessments. A key challenge, however, is balancing the need 3 Method: development of the measurement
for specific indicators (to a particular hazard in a particular framework
place for a particular institution) and the need for wide ap-
The Zurich Alliance community flood resilience measure-
plicability. A related further challenge is that resilience to
ment framework has been pilot tested, reviewed, and revised,
one hazard does not necessarily translate into resilience to
and now a beta version has been adapted into a web-based
another. In fact, it is entirely possible that measuring and en-
platform and integrated mobile data collection application,
hancing resilience to one hazard may inadvertently reduce
via which implementers collect data and assign grades for
resilience to another (Schipper and Langston, 2015).
each of the indicators (pre-event “sources of resilience” and
Particularly pertinent to our community-level framework
post-event impact assessment).
is the observation by several authors (Béné et al., 2012;
The conceptualization of disaster resilience underlying the
Frankenberger et al., 2014) that resilience at the community
framework is set out in the introduction. Narrowing this
level is dependent upon changes at lower “levels”, i.e. indi-
down, for the purposes of this framework we are specifically
viduals and groups within communities, as well as changes
interested in the resilience of communities to flooding. While
at higher municipality, national, or even global levels, which
acknowledging that national and global drivers play a signif-
are outside the scope of direct NGO community program-
icant part in flood resilience, the community level is where
ming initiatives. NGOs are often limited in their capacity
flood impacts are felt most viscerally, where much action on
to activate change at these scales. “Community” itself is a
flood resilience needs to be taken and are the focus of many
particularly elusive concept without clear definition. It is a
flood and development activities. The NGO practitioners on
highly contested term and has been critiqued from multiple
the design team put forward the definition of community
perspectives. Particularly relevant to disaster resilience is the
used here. A “community” could be defined geographically
inherent tension between discourses of empowerment, which
(perhaps in rural contexts) or by administrative boundaries
dominate the rhetoric, which have “concealed a much more
(which may work in more urban situations). However, no sin-
conformist and conservative reality” (Shaw, 2007, p. 26)
gle community will “feel” like another and there are cultural
borne of the paternalistic history of the development field.
aspects to consider, too. As a result we have concluded that
Our definition of community is outlined in Sect. 3.
when it comes to ground reality a community largely defines
As communities are dynamic complex systems,
itself.
Levine (2014) argues that the modular approach to
measuring resilience, such as measuring a set of discrete
characteristics, “assumes that improvements in any com-
ponent of resilience score are of equal importance” (p. 8).
As he suggests, this is deeply problematic when thresholds
3.1 Foundations of the measurement framework drawn on in the development phase of the framework. Of
particular influence here was the that this framework (and
The measurement framework was conceived by integrating our’s) places people and their agency at the literal centre of
the underlying disaster resilience perspectives and frame- thinking on disaster resilience. The “qualities” listed in the
works of the alliance members. Constas et al. (2014) iden- panel on the left were used to cross-check indicator grade
tify resilience as a “multidimensional capacity” and while definitions, especially in relation to equity. The top panel on
their suggested dimensions differ slightly from the ones pre- external resources informed the framework to separate out
sented here, we are in full agreement that community flood “community-level” indicators and “enabling condition” indi-
resilience is about the combination of capacities across dif- cators.
ferent dimensions. Following the focus on overall wellbe- Also informing our thinking was Practical Action’s (Pas-
ing and development as the overarching goal of disaster re- teur, 2011) Vulnerability to Resilience (V2R) framework
silience in our definition, the sustainable livelihoods (SL) (Fig. 2). This is an analytical approach that can be used at
framework (DFID, 1999) was drawn upon to capture com- community level to combine current and future risks with
munity assets and capacities. The SL framework is an asset- the capacities and assets that the community have available
based framework, representing the core capacities (or asset to drive their development. Sections of this framework di-
base) that enables the overall community system to provide rectly informed indicator development: bullet points in the
wellbeing, opportunity, and risk management. From a prac- cycle were expanded and further specified so that each step
tical perspective, it has been widely used in community de- in the cycle is captured in our framework.
velopment as a conceptual device (Knutsson and Ostwalk, The NGOs assessments and data gathering processes used
2006) and as such is well understood within the community to inform their work within these frameworks were emu-
development field. It is applicable for developing and devel- lated to assess the sources of resilience (for example fo-
oped countries, at multiple scales, qualitatively and quantita- cus groups and household surveys). Secondly, our measure-
tively2 . ment approach is, uniquely, informed by risk engineering ex-
However, capital levels and combinations in and of them- pertise, in this case from Zurich Insurance. Risk engineer-
selves do not tell us explicitly how well a community may ing is a technical assessment approach to identify, assess
perform in the face of the uncertain risks and opportuni- and improve risk to specific perils. Risk engineering often
ties. This led to the question of whether we could identify works with technical risk grading standards (TRGSs), tech-
some general properties or principles to look for in commu- nical documents which offer a standardized view of risk and
nities that are thought to enhance resilience over time and highlight priority actions that could be taken to reduce risks.
in various contexts. This led to consideration of the four TRGSs are tailored to different perils, taking account of the
“resilience properties” (4Rs) defined by MCEER (Bruneau, different factors that make up risk associated with that peril.
2006; Cimellaro et al., 2010): redundancy, resourcefulness, Each TRGS includes risk factors (indicators) and defines the
rapidity, and robustness (defined below). evidence needed to earn that factor a grade of A, B, C, or D.
The next step was to make this conceptual framework op- Grades are assigned as follows:
erational. For this we first drew on the practical and program-
matically focused expertise of our NGOs and risk engineers 1. A for best practice for managing the risk;
within the measurement framework design team. In partic-
ular, the IFRC’s (2012) framework depicted in Fig. 1 was 2. B for good industry standard, with no immediate need
for improvement;
2 These points were articulated by participants at our peer review
of the framework held on 29 June–1 July 2015. 3. C for deficiencies, with room for visible improvement;
5 Figure
Figure 2: Vulnerability
2. Vulnerability to Resilience
to Resilience framework Framework.
(source: Pasteur, Source:
2011). Pasteur, 2011
4. D for significantly below good standard, with potential silience). However, as we review above, much work has been
for imminent loss. done to establish specific measures of disaster resilience.
Our measurement framework drew on the insights from this
Engineers compare data, often gathered from location- body of work1 via a comprehensive review of what sources
specific site visits, with the definitions in the TRGSs to make of resilience have been qualitatively shown or are widely
a judgment on risk quality and conduct conversations with considered to be important for community-level resilience
the site or company about how to manage the risks they are to flooding. The articles and reports which finally con-
facing. Clearly, a community is profoundly different from tributed most substantially to the content of the measurement
a firm. Similarly, risk is not the same as (or the inverse framework are (in alphabetical order) Bahadur et al. (2015),
of) resilience. The insight drawn from the risk engineering Cutter et al. (2010), DFID (2009, 2011), IFRC (2011,
approach for the community flood resilience measurement 2014), MCEER (2007), NRC (2012), OECD (2014), Ox-
framework was regarding the utility and benefits of having a fam GB (2013), Pasteur (2011), and Twigg (2009). A further
trained assessor make a qualified judgment to assign a grade 46 documents were reviewed and informed the content of the
of A to D for each of the different factors. We discuss the framework; these are listed in Appendix A. It should be noted
benefits of this approach below. that the development, testing, and reviewing process that the
framework has gone through has seen it evolve substantially,
3.2 Measurement framework content development and it would no longer be possible to assign citations to spe-
cific sources of resilience.
The SL 5C framework provides a holistic perspective of The first version of the measurement framework – Ver-
wellbeing, as well as community assets and capacities. The sion 1 – was piloted in 24 communities in Mexico, Nepal, and
IFRC (2012) and Pasteur (2011) frameworks provide a Peru. This pilot testing was undertaken by the design team in
highly generalized approach to features for improving disas- collaboration with NGO staff familiar with the communities
ter resilience, while the 4Rs inform the quality of the commu- being assessed. The framework was then revised in response
nity characteristics (which became known as sources of re-
to the lessons learned from this testing. One of the first point out, an approach to resilience measurement which
lessons learned was that effective use of such a framework measures ex ante but does not test whether what was
requires that all sources be discrete concepts, since multiple measured was relevant once the event occurred cannot make
parts within each source often resulted in an average, and any claims about having measured resilience. By tracking
hence meaningless, grade. Furthermore, some sources were the sources of resilience both pre- and post-event, together
assigned an interdependency condition; i.e. they cannot be with outcomes in the event of a flood, we can observe how
graded above a certain threshold unless a certain grade is at- development, disasters, and DRM activities occurring in the
tained or exceeded on another, interdependent, source (inter- community are eroding or supporting wellbeing. Over time
dependency is a point highlighted by Levine, 2014). and studies in different contexts, this testing can build an
Version 1 of the measurement tool was dominated by dis- evidence base for what actually makes the difference for
aster preparedness and response capacities, as opposed to community-level resilience to flooding.
risk reduction and more systemic development considera- The 88 sources of resilience have been developed with
tions. We consider that this was due to a number of factors, the underlying frameworks of the five capitals (5Cs) and the
including because they (a) continue to dominate the disasters 4Rs. These were then cross checked by categorizing them
field, (b) are front and centre of other disaster resilience mea- within three other operational frameworks commonly used
surement frameworks, and (c) are relatively easy to measure. by NGOs. These provide a cross check not only that we are
By reflecting on our definition of disaster resilience, we saw measuring a holistic set of sources but also that the suite
the need to balance preparedness and response with other im- of categorizations attached to each source provide for an-
portant elements of disaster resilience. Many of the sources alytical depth by allowing for multiple perspectives on the
now are concerned with the capacity of households and the results. These also increase the ability to communicate re-
community to consider disaster risk in their broader planning sults to many different stakeholder audiences in the contexts
and to take action to reduce risk rather than just prepare for within which they are working. We provide examples of each
it. of these categorizations below.
Version 2 of the measurement framework was developed Before exploring dimensions of the tool in detail, we pro-
by the design team in response to lessons learned from the pi- vide Fig. 3 as a reference point. The figure shows the steps
lot testing. This version saw each source also assigned a num- that are undertaken when using the framework and where
ber of qualities or categorizations in addition to the 5C-4Rs; different aspects fit. The first step is data collection (see
these are presented below. A peer-review workshop was then Sect. 4.3), which can be done using various methods. Af-
convened with 10 preeminent disaster resilience and mea- ter data have been collected, experts assign grades to each of
surement experts. The workshop led to a number of changes the 88 sources of resilience, according to the source rationale
to Version 3 of the framework and also crystallized a number and the A–D grade definitions. Each source has already been
of issues regarding usage and feeding back information to assigned (tagged) to the five categorizations (Sects. 4.1.1
communities. The importance of power dynamics in social to 4.1.5). Once all 88 sources are graded, results are gen-
capital sources was emphasized, expanding the social cap- erated according to the different categorizations.
ital section to include what might be termed “institutional”
or “political” capital. The review also underlined the chal- 4.1 How sources are organized within the framework
lenge of providing suitable flexibility in how the tool could
be applied to enable it to be appropriate to context, while still The 88 sources of resilience are each categorized by the 5Cs,
maintaining enough standardization to generate comparable by the 4Rs, by 10 themes, by the two perspectives of the
data. The testing version of the tool is such that each source system level (community and enabling environment), and by
is graded to produce standardized data (A–D grading), yet the five phases of the DRM cycle (Appendix B lists sources
is sufficiently general to apply across contexts; implementers with associated categorizations).
are able to choose one or more data sources as they see fit,
from at least two and up to five options, each with associated 4.1.1 Five capitals
indicators for each source.
The most prominent categorization of the sources of re-
silience is along the 5Cs of the SL framework. This framing
4 The community flood resilience measurement was an inherent part of the measurement framework from
framework inception, and strongly emphasizes the multidimensional na-
ture of resilience. For example, it is not enough to have a
This measurement framework applies specifically to school facility which is robust and accessible in times of
community-level resilience to flooding and is designed to flooding (physical capital) – a community also needs a strat-
(a) help guide NGO community development and flood- egy to maintain or quickly resume schooling in the event of
based programs and (b) to provide a platform for empirical a flood (social capital) and an intrinsic value placed on edu-
analysis of resilience. As Frankenberger and Nelson (2013) cation by parents (human capital). We follow the definition
of the five capitals by Nelson et al. (2007) and provide an frastructure, equipment, and improvements in genetic
example source within each capital group. resources, e.g. crops, livestock. The source of resilience
“access to health care facilities” assesses the adequacy
– Human capital is the education, skills, and health of
of the infrastructure to support community health and
household members. The source of resilience “flood
how it stands up in flood situations.
protective behaviour and knowledge” assesses people’s
knowledge/skills about how to behave during a flood – Financial capital is the level, variability, and diversity of
event, in order to prevent death and injury. income sources and access to other financial resources
(credit, savings, cattle) that together contribute to wealth
– Social capital is the reciprocal claims on others by virtue
(Nelson et al., 2007). The source of resilience “house-
of social relationships and networks, the close social
hold financial savings that protect long-term assets” as-
bonds that aid cooperative action and the social bridg-
sesses the availability of liquid assets to cover expected
ing, and linking via which ideas and resources are ac-
flood losses.
cessed. The source of resilience “community represen-
tative bodies/structures for flood management coordina- 4.1.2 Four properties of a resilient system
tion” assesses the degree of formal organization of the
community as a whole around flood risk management. The 4Rs help to evaluate where there may be weaknesses
– Natural capital is the natural resource base, e.g. pro- to one or more of these four areas and thus where the com-
ductivity of land, and actions to sustain productivity, as munities’ systems may be vulnerable should a disaster event
well as the water and biological resources from which occur. We hypothesize that the 4Rs may be able to shed light
livelihoods are derived. The source of resilience “basin on why some communities fare better than others in the same
health” assesses how changes in natural habitats are ex- disaster, despite identical capital endowments. An examina-
acerbating or reducing the flood risk in the target com- tion of the financial capital profiles of the two communities
munity. might reveal that Community A has a diversified income base
whereas Community B is dependent on a single industry.
– Physical capital is capital items produced by economic This redundancy has been demonstrated to be a source of
activity from other types of capital that can include in- quicker recovery after a disaster. The 4Rs are defined below
on ex post crisis response and a neglect of ex ante risk re- sources of resilience) as well as post-event outcomes which
duction (Kellet and Caravani, 2013). The reasons for this are informed the outcome measures. We also reviewed standard
many and complex; the purpose here is to shine a light on practice in flood impact assessment in order to identify which
strengths and weaknesses at the community level. Within the flood impacts tend to be most important for communities. Fi-
framework we define four stages of the DRM cycle and pro- nally, these outcome measures were validated against prac-
vide an example source for each: titioner experience at two NGO workshops in 2016, where
participants were asked to identify the most salient impacts
– Prospective risk reduction are activities that address and of floods; outputs from this exercise aligned extremely well
seek to avoid the development of new or increased dis- with the outcome measures in the framework.
aster risks. For example, flood exposure perception (hu- Outcomes are built around the 10 themes because this is
man capital) assesses the accuracy of community per- where they most logically aligned, with at least one outcome
ception/expectation of flood inundation areas. variable for each theme. They are also assigned a system
– Corrective risk reduction are activities that address and level. We provide an example of each type of outcome mea-
seek to correct or reduce disaster risks which are al- sure.
ready present. For example, sustainable use of natural – Control variables record the severity and extent of the
resources (natural capital) assesses whether livelihood flooding. These are required so that impacts can be anal-
activities impact on natural capital are managed to min- ysed with control for how severe the flood was. For ex-
imize their negative impact on these habitats. ample, flood frequency assesses the return period of the
flood event.
– Crisis preparedness is action carried out before an event
to build capacities needed to effectively manage the – Impact variables record the level of loss or damage of
flood emergency situation and achieve orderly transi- the flood and the time to recover on the different di-
tions from response to recovery and reconstruction. For mensions of community wellbeing. For example, preva-
example, measurement and forecasting (physical capi- lence of post-flood illness assesses the frequency of wa-
tal) assess the quality of locally accessible flood fore- ter and vector born disease in the post-flood period, as
casting network. compared to normal times.
– Coping is the ability of a community to utilize avail- – Action variables record what actions the community
able skills and resources to manage the adverse con- took during and following the flood, which may impact
ditions brought on by the flood. For example, business wellbeing. For example, selling assets assesses to what
credit access (financial capital) assesses the availability extent both productive and non-productive assets were
of credit (both formal and semi-formal, and can include sold in order to cope with flood impacts.
vendor credit) for members of the community during
and directly following a flood. Our approach aims to collect outcome measurement data
within eight weeks of the event. Collecting in the first weeks
The cyclical nature of the DRM cycle is a critical feature; following an event was considered to be disruptive to relief
the stages of the cycle naturally blend into one another. In efforts, yet waiting much longer than this runs the risk of lack
particular, the coping and recovery phase following a disaster of priority and missing critical aspects of a resilient outcome
provides an opportunity for prospective risk reduction in the (e.g. recovery time).
next cycle. These designations will likely be refined as the
analysis proceeds. 4.3 Data collection methods
4.2 Post-flood outcome measurement Assessors are provided with multiple options for gathering
the data required to grade each sources of resilience and out-
Within this framework outcomes refer to the way in which come measures. Data to inform grading of many sources of
a flood has impacted a community’s wellbeing and devel- resilience can be collected via any of the available methods.
opment potential. We have identified 29 outcome measures If a data collection method is available for a source, the tool
to be collected following flood events which may occur in provides pre-defined questions such as a household survey
program communities during the testing period. These con- question or a question to ask an expert in an interview. In
sist of 19 flood impact measures, seven during-flood “ac- the set-up phase, assessors select one or several of the data
tion” measures, and three flood severity control measures collection methods they intend to use for each source. Each
(see Appendix B for full list). This information is required data collection method has strengths and weaknesses; asses-
to empirically explore the effectiveness of the sources. The sors make their selection depending on the local context and
measures included here were drawn from the literature re- appropriateness, the resources available to them to gather the
view described in Sect. 3.2, During this review process we data, availability of existing information, and internal NGO
identified both pre-event characteristics (which informed the objectives. The five options for data collection are as follows:
– Household surveys are used to collect data on recipi- manual, and other associated guidance documents. The as-
ent communities, as is standard practice for many NGO sessor (or assessment team as is often the case) is presented
practitioners. In this case the measurement tool web in- with the data collected in the data collection phase and con-
terface automatically loads surveys onto mobile phones, siders it in reference to the A–D grade definitions. The ben-
which are then taken to the field by enumerators. Train- efit of using an A–D grading approach, rather than collect-
ing is provided to implementation teams on appropri- ing and trying to “objectively” assess raw data, is that it al-
ate sample size, as well as good best practice sampling lows for semi-quantitative assessment of qualitative proper-
techniques. ties. It also allows for multiple information sources to be con-
sidered, and local understanding and context to be incorpo-
– Community consultations or community discussions are rated. Finally, it allows assessors to use their expert judge-
another mainstay of contemporary development prac- ment rather than forcing them to creatively generate data
tice, as such practitioners are typically very skilled at which are unavailable. The grading stage is finalized using a
setting up and facilitating these processes. There are peer-review approach where a colleague assessor sufficiently
many existing community consultation processes which familiar with the local community context will jointly dis-
data gathering associated with this resilience measure- cuss the grades with the main assessor. Grades will then be
ment endeavour can be integrated with, including par- finalized and “frozen”, i.e. they cannot be further changed,
ticipatory rural appraisal and vulnerability–capacity as- before the results are displayed. This is to avoid “engineer-
sessment. ing” certain aspects into desired results by changing grades
retrospectively.
– Key informant interviews are another important source A fully specified source of resilience, named “access to
of information for assessing sources of resilience. In this school facilities” from the physical capital group, is set out
case practitioners use their local knowledge and con- in Appendix C.
tacts to identify people who hold information critical to The output is the list of 88 grades, plus a series of av-
understanding resilience in the community. For exam- erage grades along the 5Cs, 4Rs, themes, system level, and
ple, they may speak with the local school principal to DRM cycle breakdowns. Each group of sources is weighted
ascertain the state of flood contingency planning for the equally, i.e. for the capital categorization, which has five
local school. capitals; 20 % (100/5) of the final grade goes to each cap-
ital group. Similarly, within each group sources are also
– Interest group discussions are similar to community weighted equally. It has been structured this way because
group discussions in that they are common practice some groups have more sources than others. There is cur-
and are run in a similar fashion. Interest groups who rently no empirical evidence to support a larger weight
might have information pertinent to community flood for any sources over others, although part of this study is
resilience include, for example, a local weavers group, to explore this question. Depending on the results of the
local small business owners, or the local disaster pre- analysis of all resilience measurement data being collected
paredness committee. through the testing process, different weights or weighting
approaches may be incorporated in the future.
– Third-party sources are secondary (as opposed to pri-
In the event of a flood, a similar process is undertaken to
mary) sources of information which practitioners can
collect the data for the 29 outcome indicators. Again the data
use to gather information on the sources of resilience,
can be collected using any of the five data collection meth-
which have been produced by a third party. These might
ods (for example, household survey). However, in this case
include NGO reports, government reports or documen-
assigning an A–D grade is too premature. For example, there
tation, newspaper reports, or even census data.
is no way to tell how many deaths are acceptable to warrant
For many sources of resilience more than one data collection an A vs. C grade. Instead the raw data will be collected and
option is available, and more than one can be selected when the assessor will be asked on a scale of A–D what is his or
the study is being set up. Preliminary feedback from baseline her professional perception of the level of resilience, where
implementation indicates that while selecting more than one A is perceived to be an excellent or resilient outcome and D is
data collection method occasionally results in a contradic- a bad or not resilient outcome. Over time these perceptions
tion, more often than not it enhances confidence in the final matched with the raw data may allow for deeper insight into
grade assigned. what actual flood resilience looks like for communities.
4.4 Grading and weighting 4.5 What the measurement framework does well and
what it does not do
Once all data has been collected, the 88 sources of resilience
are individually graded from A to D. Grading is done by a Frankenberger et al. (2014) argue that “[a]lthough ample lip
trained assessor drawing on their experience, training, a user service is often given to conducting comprehensive analy-
sis, many NGOs rely heavily on participatory rural appraisal 5 From framework to tool
methods. Such reliance on qualitative data means they miss
out on capturing important contextual information that is of-
ten available through secondary sources”. They further point To test the framework at scale and collect the data required
out that NGOs tend to focus data collection on areas most for undertaking empirical analysis of community flood re-
directly related to their theory of change. One key benefit of silience, this framework was implemented into a commu-
our framework and tool is that it encourages NGOs to look nity flood resilience measurement tool – an integrated, hy-
beyond the aspects they have traditionally focused on and to brid web-based and mobile device system for creating ques-
search out a broader variety of information sources. tionnaires based on the flexible combination of data col-
Frankenberger et al. (2014) also note that NGO program- lection methods for each source, assigning data collection
ming is often constrained by the rigidity of donor require- work, collecting data, undertaking grading, generating out-
ments. There exists a strong preference for “visible” pro- puts, and storing data on a (protected) central database. The
gramming such as flood protection infrastructure or humani- six NGOs implementing the tool have been trained in un-
tarian food distribution; less visible, or socially focused, pro- derstanding the framework and using the tool. Over 2 years
gramming is less desirable because it is more difficult to they will collect baseline, end line, and outcome measures
quantify. At the same time, disaster resilience programming (if a flood occurs) in 75 communities in eight countries.
is challenging because building disaster resilience requires A train-the-trainer workshop was conducted with approxi-
an integrated approach which crosses many traditional sec- mately 20 NGO staff from 9 to 13 November 2015. During
toral or thematic boundaries such as education, market ac- this week the staff were trained to (a) implement the tool and
cess, and environmental stewardship. We propose that one of (b) train their colleagues to implement the tool.
the key benefits of a measurement framework such as this Implementing the tool includes being confident in ones un-
one is that once validated it may assist NGOs to analyse derstanding of the rationale behind the tool, being able to
and then show how different aspects of their community de- consistently grade sources, interpret outputs, and use the in-
velopment work collectively contribute to building flood re- ternet and mobile applications. It also critically requires a
silience. It may also allow for long-term incremental change sound understanding of how the tool outputs should, and
to be shown within a short funding cycle time frame. should not, be utilized. The measurement tool does not re-
By making a distinction between community level and place existing processes, but it does foster an open dialogue
enabling environment sources of resilience, using the mea- around findings. It is designed to help identify potential ar-
surement tool has the potential to crystallize the focus of eas for intervention, but choosing an intervention is a much
advocacy work. Sources of resilience which are designated more complex process which must consider multiple factors
community level are possibly more effectively targeted by and perspectives.
community-level action. Enabling environment designated As with all information, the implementing NGO is obliged
sources, in contrast, may be the target of higher-level stake- to feed information back to the community in an accessible
holder engagement or advocacy towards authorities. and responsible manner. While the final decision rests with
As Levine (2014) argues, thresholds pose a significant the NGO, we strongly advise against feeding back informa-
problem to resilience measurement frameworks such as this tion on quantitative scoring because of the specialist train-
one. For many sources of resilience, it may be that they are ing required to accurately interpret grades. Grades on their
not at all useful until they have reached a certain level, which own are largely meaningless; what is important for analy-
corrupts the grading approach. Our approach to this valid cri- sis and communities is how different sources interact and
tique is to emphasize that we are not purporting to be present- how resilience is changing over time in that community. Pilot
ing the definitive community flood resilience measure; rather, testing to date has shown that the information gleaned from
we are collecting information which will enable us to em- the measurement process, presented in a qualitative way, can
pirically analyse community flood resilience. An exploration provide a very useful starting point for discussions with com-
of whether thresholds exist in sources of resilience, when it munities about flood resilience.
comes to whether they impact outcomes, will be a key focus Internal communication must also be carefully considered.
of that analysis. There is the risk that the generation of a grade may result in
Many scholars (Levine, 2014; Mitchell, 2013; Béné et competition between project managers or field workers. It is
al., 2012) have highlighted the at-times uneasy relationship up to the core assessment team to contextualize the measure-
between resilience and vulnerability. Measuring resilience ment process and resulting grades within the wider context of
should in no way replace a vulnerability analysis. Our frame- their community development work to ensure that the grades
work is designed to work alongside rather than replace vul- are understood. There may be a temptation to gravitate to-
nerability and capacity assessments or any other analysis and wards interventions which one believes will increase the re-
engagement processes. Indeed, we encourage the use of ex- silience grade with the least amount of resources or effort.
isting data gathering processes to collect data for measuring While one of the benefits of the tool is that it might iden-
the sources and outcomes. tify relatively cost-effective “quick wins”, it should never be
the case that the objective of the intervention becomes the quantitative information is then complemented by anony-
increase of the resilience grade per se. mous structured feedback and anonymous in-depth semi-
For the purpose of the empirical research, all data which structured interviews and focus group discussions with im-
go into the tool system are effectively anonymized. All re- plementers throughout the process. This feedback process
spondents provide informed consent before information is will also explore how the process and results of the measure-
collected, and no individual can be identified from the data ment tool are informing decision-making on interventions.
(either in its raw form or aggregated). Preliminary implementer feedback at the time of writing
has indicated that the process of training staff to use the tool,
and implementing the tool, is already producing positive out-
6 Conclusion and way forward comes. In particular, implementing NGOs have reported that
the holistic view of the community system is building local
Schipper and Langston’s (2015) review and expert consulta- staff capacity to think systemically about their work. For ex-
tion report argue that measurement frameworks are “based ample, in addition to the traditional physical infrastructure,
on assumptions about how systems work, albeit informed as- the human and social elements required to make an early
sumptions” (p. 19). Measurement frameworks make assump- warning system operational are being considered early. The
tions about the way in which the presence or absence of each data collection effort holds many co-benefits for supporting
indicator (often measured ex ante) will impact resilience ex other NGO work beyond a flood focus. The data collection
post. If this project is to achieve its aim of empirically ex- technology is considered superior in its accuracy and effi-
ploring the critical sources of resilience, we need to do more ciency to traditional paper-based approaches.
than just measure sources in a number of communities; we NGOs implementing the measurement tool have expressed
need to test those measures against outcomes. This is why we a desire for a more explicit handling of power, indicating
are collecting post-flood outcome data – so that the original a preference for a sixth “political capital” to be added to
sources can be compared to outcomes in order to understand the five capitals framework. Similarly they have indicated
what really made a difference in the flood event and recovery the need for more explicit incorporation of social inclu-
(i.e. actual resilience). sion/exclusion in the measurement framework. These are all
The implementation of the measurement tool de- recommendations which should be considered carefully by
scribed above will generate an unprecedented database of the field as it move forward.
community-level information and experience with flooding, Via this process we hope to be able to generate evidence
collected in a consistent way. Data will be augmented by in- for which of the sources of resilience, if any, make a differ-
formation from implementing NGOs regarding the commu- ence across diverse contexts. It is possible that community
nity context, major events, or changes in the community dur- flood resilience is entirely contextually specific, or it boils
ing the study period, and interventions implemented. Analy- down to a few core sources of resilience meeting a certain
sis of these data will lead to insights about community flood threshold; this is what we will explore. The extensive test-
resilience generally: what typical community profiles, pat- ing phase is also providing a platform for the formation of
terns in strengths and weaknesses across the different cat- a peer group of practitioners working on community flood
egorizations, correlations between different capacities, and resilience and using the measurement tool. The insights this
effectiveness of different intervention types are. peer group generates together provide a critical feedback to
Our analysis of the validity of the sources of resilience resilience measurement and practice, as well as builds practi-
within the framework – our hypothesis from research to-date tioner capacity. In the future reflection on this extensive pro-
– will take a multi-pronged approach. Firstly, we will explore cess will allow critical reflection on many of the challenges
the empirical relationship between the sources (recorded in associated with measuring resilience.
the baseline) and outcomes (recorded in the post-flood out- Winderl (2014) concludes his comprehensive review of
come measurement). A suite of quantitative data analysis measurement frameworks with a set of recommendations or
methods will be used to undertake this endeavour. Further- lessons, presumably for the development of future frame-
more, each time a source is graded (75 communities at base- works or iterations of existing ones. These recommendations
line and end line), assessors are asked two question about the are theoretically sound but would be impractical to imple-
source: ment within one framework. When working in such an ap-
plied field, and relying on collaboration from communities,
1. Were you confident grading this source? (Y/N)
NGOs and other non-research institutions, there is little space
2. Is this source relevant to assess resilience to flooding? for idealism in research design. Our endeavour to date has
(Y/N) highlighted that capacity and willingness to implement com-
plex theoretical frameworks are very limited at the grassroots
In this way we are collecting quantitative information on NGO level. There is a need to appreciate that staff resources
the perspective of the practitioners working with flood prone and budgets are very limited, and implementing idealistic
communities on each of the 88 sources of resilience. This frameworks in a time-consuming manner will distract from
Appendix A: Additional literature which informed the and emergency management, CARRI research report 1,
development of the Zurich Flood Resilience Community and Regional Resilience Institute, Oak Ridge,
Measurement Framework 2008a.
Cutter, L. S., Barnes, L., Berry, M., Burton, C., Evans, E.,
In addition to the documents cited in the article, the working Tate, E., and Webb, J.: A place-based model for under-
group also reviewed and drew insight from the following standing community resilience to natural disasters, Global
sources: Environ. Change, 18, 598–606, 2008b.
Hills, T., Pramova, E., Neufeldt, H., Ericksen, P., Thornton,
Adger, W., Huges, T., Folke, C., Carpenter, S., and P., Noble, A., Weight, E., Campbell, B., and McCartney, M.:
Rockstöm, J.: Social-Ecological Resilience to Coastal A Monitoring Instrument for Resilience, CCAFS Working
Disasters, Science, 309, 1036–1039, 2005. Paper no. 96, CGIAR Research Program on Climate Change,
Aerts, J. C. J. H., Botzen, W. J. W., Emanuel, K., Lin, N., Agriculture and Food Security (CCAFS), Copenhagen, Den-
de Moel, H., and Michel-Kerjan, E. O.: Evaluating flood mark, 2015.
resilience strategies for coastal megacities, Science, 344, ICE: Flooding: Engineering Resilience, report from the
473–475, doi:10.1126/science.1248222, 2014. Institution of Civil Engineers, 15 pp., http://test2.ice.org.uk/
Ainuddin, S. and Routray, J. K.: Earthquake hazards and getattachment/6fb6807f-6e90-4780-b463-1ad88a2d3837/
community resilience in Baluchistan, Nat. Hazards, 63, SEE-SoN\T1\textemdashFlooding.aspx (last access: 24 Jan-
909–937, 2012. uary 2017), 2008.
American Red Cross: Community resilience Assess- IFRC: Of networks, norms, and trust. The role of social
ment Tool, Household and committee surveys for mea- capital in reinforcing community resilience, Geneva, 2012.
suring overall community resilience and for tracking IFRC: Asia-Pacific Input Document for the post-
changes following Red Cross integrated interventions (“Ri- 2015 Framework for Disaster Risk Reduction (HFA2)
taline”), http://preparecenter.org/sites/default/files/ritaline_ – Key Area 1 – Building Community Resilience,
presentation_v2014_02.pdf (last access: January 2017), http://web.archive.org/web/20160329061902/http:
2013. //6thamcdrr-thailand.net/6thamcdrr/Portals/0/KA1_Final_
Australian Bureau of Statistics: Measuring Social Capital. Hi_301014-2.pdf (last access: 24 January 2017), 2014.
An Australian Framework and Indicators, Information Jones, L., Ludi, E., and Levine, S.: Towards a characterisa-
paper 1378.0, Canberra, 2004. tion of adaptive capacity: a framework analysing adaptive
Australian Red Cross:Relationships matter: the application capacity at the local level, ODI Background Notes, Decem-
of social capital to disaster resilience, National Disaster ber 2010, Overseas Development Institute, London, 2010.
Resilience Roundtable report, Melbourne, Australia, 2013. Jonkman, S. and Kelman, I.: An analysis of the causes and
Briguglio, L., Cordina, G., Farrugia, N., and Vella, S.: circumstances of flood disaster deaths, Disaster, 29, 75–97,
Economic vulnerability and resilience concepts and mea- 2005.
surements, Research Paper/UNU-WIDER, No. 2008.55, Kurosaki, T. and Khan, H.: Floods, Relief Aid, and House-
UNU-WIDER, Helsinki, 2005. hold Resilience in Rural Pakistan: Findings from a Pilot
Bruneau, M., Chang, S. E., Eguchi, R. T., Lee, G. C., Survey in Khyber Pakhtunkhwa, Rev. Agrar. Stud., 2,
Rourke, T. D., Reinhorn, A. M., Shinozuka, M., Tierney, 79–107, 2011.
K., Wallace, W., and Von Winterfeldt, D.: A framework to Low, B., Ostrom, E., Simon, C., and Wilson, J.: Redundancy
quantitatively assess and enhance the seismic resilience of and diversity: do they influence optimal management?, in:
communities, Earthq. Spectra, 19, 733–752, 2003. Navigating social-ecological systems: building resilience
Burton, C. G.: The development of metrics for community for complexity and change, edited by: Berkes, F., Colding,
resilience to natural disasters., PhD 3523093, University of J., and Folke, C., Cambridge University Press, Cambridge,
South Carolina, South Carolina, 2012. 83–114, 2003.
Cabell, J. and Oelofse, M.: An indicator framework for Low, S. and Kim, L.:The Role of Social Capital in Strength-
assessing agroecosystem resilience, Ecol. Soc., 17, 18, 2012. ening Disaster Resilience in Thailand, prepared for the
Cannon, T. and Muller-Mahn, D.:Vulnerability, resilience Southeast Asia Regional Delegation (SEARD), International
and development discourses in context of climate change, Federation of Red Cross and Red Crescent Societies (IFRC)
Nat. Hazards, 55, 621–635, 2010. and the Thailand Red Cross Society (TRCS), Bangkok,
Cumming, G., Barnes, G., Perz, S., Schmink, M., Sieving, 2014.
K., Southworth, J., Binford, M., Holt, R., Stickler, C., and Mayunga, J. S.: Understanding and applying the concept of
Van Holt, T.: An exploratory framework for the empirical community disaster resilience: a capital-based approach, A
measurement of resilience, Ecosystems, 8, 975–987, 2005. draft working paper prepared for social vulnerability and
Cutter, L. S., Barnes, L., Berry, M., Burton, C., Evans, E., resilience building, Munich, Germany, 1–16, 2007.
Tate, E., and Webb, J.: Community and regional resilience Norris, F. H., Stevens, S. P., Pfefferbaum, B., Wyche, K.
to natural disasters: perspective from hazards, disasters F., and Pfefferbaum, R. L.: Community resilience as a
metaphor, theory, set of capacities and strategy for disaster Twigger-Ross, C., Kashefi, E., Weldon, S., Brooks, K.,
readiness, Commun. Psychol., 41, 127–150, 2008. Deeming, H., Forrest, S., Fielding, J., Gomersall, A., Har-
Olsson, P., Folke, C., and Berkes, F.: Adaptive co- ries, T., McCarthy, S., Orr, P., Parker, D., and Tapsell, S.:
management for building resilience in social-ecological Flood Resilience Community Pathfinder Evaluation: Rapid
systems, Environ. Manage., 34, 75–90, 2004. Evidence Assessment, Defra, London, 2014.
Rose, A.: Economic Resilience to natural and man-made UN: Targets and Indicators For Addressing Disaster Risk
disasters: Multidisciplinary origins and contextual dimen- Management in the Post-2015 Development Agenda, UNDP
sions, Environ. Hazards, 7, 383–398, 2007. Learning Resources Center, 18–19 July 2013, New York,
Simonovic, S. P. and Peck, A.: Dynamic Resilience to 2013.
Climate Change Caused Natural Disasters in Coastal Megac- van Kien, N.: Social Capital, Livelihood Diversification and
ities: Quantification Framework, British J. Environ. Clim. Household Resilience to Annual Flood Events in Vietnamese
Change, 3, 378–401, 2013. Mekong River Delta, EEPSEA Research Report No. 2011-
Torrens Resilience Institute: Developing a model and tool to RR10, Economy and Environment Program for Southeast
measure community disaster resilience, Community Disaster Asia (EEPSEA), Singapore, 2011.
Resilience Scorecard Toolkit, Adelaide, 2012. WaterAid: Disasters framework, WaterAid, London, UK,
2013.
Table B1. The 88 sources of resilience grouped by capital, showing other categorization tags.
Table B2. The 29 ex post outcome measures, showing variable type and other categorization tags.
Appendix C: Example source of resilience Data may be collected via household survey ques-
tions, community consultation discussion topics/questions,
As an illustrative example, we present one of the 88 sources key informant interviews, interest group discussion top-
of resilience, fully specified. ics/questions, or third-party sources, as appropriate to con-
The source is named “access to school facilities”. text determined by the implementing NGO. For the example
The categorizations are as follows: source the data collection options are shown in the table be-
low3 .
– capital group: physical
The “access to school facilities” source of resilience is
– 4Rs: robustness graded A to D with the following guidance:
– level: community level – A: school facility (or location where formal school set-
ting takes place) is built robust, located away from flood
– DRM cycle: prospective risk reduction zone, and accessible through safe and protected ways
even during and after floods; schooling continues to take
– theme: education.
place.
The source is explained to the assessor with the following
– B: school facility is impacted by flooding but maintains
description and instructions: “This aspect of the education
sufficient basic staffing and equipment to provide care,
theme considers the adequacy of the infrastructure to sup-
or school may be impacted but informal schooling is
port provision of education and how it stands up in flood
planned to go on in a safe place during and after floods.
situations – schooling is an important aspect of daily life”.
Both the interruption itself and the lost education time lead – C: school facility is impacted and cannot avoid signifi-
to problems (children at home instead of daily rhythm at cant lost school curriculum or, while informal schooling
school). Schooling during floods should obviously be con- may be available, it is unplanned or inconvenient and
ducted only where and when it is safe to do so depending leads to significant lost school curriculum.
on the flood scenario. For flash flood situations, rapidity and
robustness are key and schooling should resume as soon as – D: no schooling facility, or school prone to damage ren-
possible. For long-standing, large-scale standing water flood dering it in-operational during flood, or school not ac-
situations, it is important that schooling can continue, such cessible during flood for either teachers or students.
as in alternate locations or safe locations.
Table C1. Data collection questions and answer options for source
“access to school facilities”.
Edited by: S. Fuchs tion Network, Yale and Columbia Universities, New Haven and
Reviewed by: two anonymous referees New York, 2013.
DFID: Sustainable Livelihoods Guidance Sheets, Department of In-
ternational Development, UK, http://www.eldis.org/vfile/upload/
References 1/document/0901/section2.pdf (last access: 24 January 2017),
1999.
ADB – Asian Development Bank: Investing in resilience: Ensur- DFID: Defining Disaster Resilience: A DFID Approach Paper, UK
ing a disaster-resistant future, Asian Development Bank, Manila, Department for International Development, London, 2011.
2013. Frankenberger, T. and Nelson, S.: Background Paper for the Expert
Adger, N., Hughes, T., Folke, C., Carpenter, S., and Rockström, J.: Consultation on Resilience Measurement for Food Security, Pa-
Social-Ecological Resilience to Coastal Disasters, Science, 309, per presented at Expert Consultation on Resilience Measurement
1036–1039, 2005. Related to Food Security, 19–21 February 2013, Rome, 2013.
Alinovi, L., Mane, E., and Romano, D.: Measuring House- Frankenberger, T., Constas, M. A., Nelson, S., and Starr, L.:
hold Resilience to Food Insecurity: Application to Pales- Resilience programing among nongovernmental organizations:
tinian Households, EC-FAO Food Security Programme, Working Lessons for policy makers, Food Policy Report, International
Paper, http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1. Food Policy Research Institute, Washington, D.C., 2014.
1.175.7411&rep=rep1&type=pdf (last access: 24 January 2017), Heltberg, R., Hossain, N., and Reva, A. (Eds.): Living Through
2009. Crises: How the Food, Fuel, and Financial Shocks Affect the
Bahadur, A., Peters, K., Wilkinson, E., Pichon, F., Gray, K., Poor, World Bank, Washington, D.C., 2012.
and Tanner, T.: The 3As: Tracking Resilience Across Braced, Holling, C.: Understanding the Complexity of Economic, Ecologi-
BRACED Knowledge Manager working paper, Overseas Devel- cal, and Social Systems, Ecosystems, 4, 390–405, 2001.
opment Institute, London, 2015. IFRC: Characteristics of a Safe and Resilient Community, Com-
Béné, C., Godfrey Wood, R., Newsham, A., and Davies, M.: Re- munity Based Disaster Risk Reduction Study, ARUP Interna-
silience: new utopia or new tyranny? Reflection about the poten- tional Development, International Federation of Red Cross and
tials and limits of the concept of resilience in relation to vulner- Red Crescent Societies, Geneva, http://www.ifrc.org/PageFiles/
ability reduction programmes, Institute of Development Studies, 96986/Final_Characteristics_Report.pdf (last access: 24 Jan-
Brighton, 2012. uary 2017), 2011.
Berkes, F. and Ross, H.: Community resilience: toward an inte- IFRC: Understanding community resilience and program fac-
grated approach, Soc. Nat. Resour., 26, 5–20, 2013. tors that strengthen them: A comprehensive study of Red
Bruneau, M.: Enhancing the Resilience of Communities Against Cross Red Crescent Societies tsunami operation, Interna-
Extreme Events from an Earthquake Engineering Perspective, J. tional Federation of Red Cross and Red Crescent Soci-
Secur. Educ., 1, 159–167, doi:10.1300/J460v01n04_14, 2006. eties, June 2012, https://www.ifrc.org/PageFiles/96984/Final_
Carpenter, S., Walker, B., Anderies, J. M., and Abel, N.: From Synthesis_Characteristics_Lessons_Tsunami.pdf (last access:
Metaphor to Measurement: Resilience of What to What?, 24 January 2017), 2012.
Ecosystems, 4, 765–781, 2001. IFRC: IFRC Framework for Community Resilience, International
Carpenter, S., Westley, F., and Turner, M.: Surrogates for Resilience Federation of Red Cross and Red Crescent Societies, Geneva,
of Social-Ecological Systems, Ecosystems, 8, 941–944, 2005. http://www.ifrc.org/Global/Documents/Secretariat/201501/
Cimellaro, G., Reinhyorn, A., and Bruneau, M.: Seismic resilience 1284000-FrameworkforCommunityResilience-EN-LR.pdf (last
of a hospital system, Struct. Infrastruct. Eng., 6, 127–144, 2010. access: 24 January 2017), 2014.
Constas, M. and Barrett, C. B.: Principles of Resilience Measure- IPCC: Managing the Risks of Extreme Events and Disasters to Ad-
ment for Food Insecurity: Metrics, Mechanisms, and Imple- vance Climate Change Adaptation, in: A Special Report of Work-
mentation Plans, presented at Expert Consultation on Resilience ing Groups I and II of the Intergovernmental Panel on Climate
Measurement Related to Food Security sponsored by Food and Change, edited by: Field, C. B., Barros, V., Stocker, T. F., Qin, D.,
Agricultural Organization and World Food Programme, 19– Dokken, D. J., Ebi, K. L., Mastrandrea, M. D., Mach, K. J., Plat-
21 February 2013, Rome, 2013. tner, G.-K., Allen, S. K., Tignor, M., and Midgley, P. M., 2012.
Constas, M., Frankenberger, T., and Hoddinott, J.: Resilience Mea- Keating, A., Campbell, K., Mechler, R., Michel-Kerjan, E.,
surement Principles: Toward an Agenda for Measurement De- Mochizuki, J., Kunreuther, H., Bayer, J., Hanger, S., Mc-
sign, Resilience Measurement Technical Working Group Tech- Callum, I., See, L., Williges, K., Atreya, A., Botzen, W.,
nical Series 1, Food Security Information Network, Rome, 2014. Collier, B., Czajkowski, J., Hochrainer, S., and Egan, C.:
Constas, M., Frankenberger, T., Klaus, D., and Mock, M.: Qual- Operationalizing Resilience Against Natural Disaster Risk:
itative Data and Subjective Indicators for Resilience Measure- Opportunities, Barriers and A Way Forward, Zurich Flood
ment, Resilience Measurement Technical Working Group, Tech- Resilience Alliance, http://opim.wharton.upenn.edu/risk/library/
nical Series No. 4, Food Security Information Network, Rome, zurichfloodresiliencealliance_ResilienceWhitePaper_2014.pdf
2015. (last access: 24 January 2017), 2014.
Cutter, S., Burton, C., and Emrich, C.: Disaster resilience indicators Keating, A., Campbell, K., Mechler, R., Magnuszewski, P.,
for benchmarking baseline conditions, J. Homeland Secur., 7, 51, Mochizuki, J., Liu, W., Szoenyi, M., and McQuistan, C.: Dis-
doi:10.2202/1547-7355.1732, 2010. aster resilience: What it is and how it can engender a meaningful
de Sherbinin, A., Reuben, A. Levy, M., and Johnson, L.: How En- change in development policy, Dev. Policy Rev., 35, 65–91, 2017.
vironmental Indicators Are Being Used in Policy and Manage-
ment Context, Center for International Earth Science Informa-
Kellett, J. and Caravani, A.: Financing disaster risk reduction: A 20- Resilience Capacity Index: Resilience Capacity Index, University
year story of international aid, ODI and the Global Facility for of California, Berkeley, http://brr.berkeley.edu/rci/, last access:
Disaster Reduction and Recovery at the World Bank, London, 24 January 2017.
Washington, 2013. Schipper, E. L. F. and Langston, L.: A Comparative Overview
Knutsson, P. and Ostwalk, M.: A process-oriented sustainable liveli- of Resilience Measurement Frameworks: Analysing Indicators
hoods approach – A tool for increased understanding of vul- and Approaches, Overseas Development Institute Working Pa-
nerability, adaptation and resilience, Mitig. Adapt. Strat. Global per, Overseas Development Institute, London, 2015.
Change, doi:10.1007/s11027-006-4421-9, 2006. Schipper, L. and Pelling, M.: Disaster risk, climate change and inter-
Levine, S.: Assessing Resilience: Why Quantification Misses the national development: scope for, and challenges to, integration,
Point, Humanitarian Policy Group Working Paper, ODI, London, Disasters, 30, 19–38, 2006.
2014. Shaw, M.: Community development and the politics of community,
MCEER: Engineering resilience solutions from earthquake engi- Commun. Dev. J., 43, 24–36, 2007.
neering to extreme events, Multidisciplinary Center for Earth- Tol, R. and Yohe, G.: Infinite Uncertainty, Forgotten Feedbacks, and
quake Engineering Research, Buffalo, USA, 2007. Cost-Benefit Analysis of Climate Change, Climate Change, 83,
Mechler, R.: Reviewing estimates of the economic efficiency of dis- 429–442, 2007.
aster risk management: opportunities and limitations of using Twigg, J.: Characteristics of a Disaster Resilient Community, http://
risk-based cost–benefit analysis, Nat. Hazards, 81, 2121–2147, discovery.ucl.ac.uk/1346086/1/1346086.pdf (last access: 24 Jan-
2016. uary 2017), 2009.
Mitchell, A.: Risk and Resilience: From Good Idea to Good Prac- UNISDR: Terminology, http://www.unisdr.org/we/inform/
tice – A Scoping Study for the Experts Group on Risk and Re- terminology (last access: 24 January 2017), 2009.
silience, Organization for Economic Co-operation and Develop- UNISDR: From Shared Risk to Shared Value – The Business Case
ment, Paris, 2013. for Disaster Risk Reduction, United Nations Office for Disaster
Nelson, D., Adger, W., and Brown, K.: Adaptation to environmen- Risk Reduction, Geneva, 2013.
tal change: contributions of a resilience framework, Annu. Rev. UNISDR: Global Assessment Report on Disaster Risk Reduction
Environ. Resour., 32, 395–419, 2007. – Making Development Sustainable: The Future of Disaster Risk
NRC – National Research Council: Disaster Resilience: A National Management, United Nations Office for Disaster Risk Reduction,
Imperative, The National Academies Press, Washington, D.C., Geneva, http://www.preventionweb.net/english/hyogo/gar/2015/
2012 en/gar-pdf/GAR2015_EN.pdf (last access: 24 January 2017),
Oddsdottir, F., Lucas, B., and Combaz, É.: Measuring Disaster 2015.
Resilience, GSDRC Helpdesk Research Report 1045, GSDRC, USAID: The Resilience Agenda: Measuring Resilience in US-
University of Birmingham, Birmingham, UK, 2013. AID, http://www.usaid.gov/sites/default/files/documents/1866/
OECD: Guidelines for Resilience Systems Analysis. How to TechnicalNote_MeasuringResilienceinUSAID_June2013.pdf
analyze risk and build a roadmap to resilience, https: (last access: 24 January 2017), 2013.
//www.oecd.org/dac/ResilienceSystemsAnalysisFINAL.pdf (last Vincent, K.: Uncertainty in adaptive capacity and the importance of
access: 24 January 2017), 2014. scale, Global Environ. Change, 17, 12–24, 2007.
Ostadtaghizadeh, A., Ardalan, A., Paton, D., Jabbari, H., and Welsh, M.: Resilience and responsibility: governing uncertainty in a
Khankeh, H. R.: Community Disaster Resilience: a Systematic complex world, Geogr. J., 180, 15–26, doi:10.1111/geoj.12012,
Review on Assessment Models and Tools, PLOS Curr. Disast., 7, 2013.
doi:10.1371/currents.dis.f224ef8efbdfcf1d508dd0de4d8210ed, Winderl, T.: Disaster Resilience Measurements: Stocktaking
2015. of ongoing efforts in developing systems for measuring
Oxfam GB: A Multidimensional Approach for Mea- resilience, United Nations Development Programme –
suring Resilience, Oxfam GB Working Paper, http: UNDP, http://www.fao.org/fileadmin/user_upload/drought/
//oxfamilibrary.openrepository.com/oxfam/bitstream/10546/ docs/DisasterResilienceMeasurements.pdf (last access: 20 Jan-
302641/4/dp-measuring-resilience-010813-en.pdf (last access: uary 2017), 2014.
24 January 2017), 2013. Zhou, H., Wang, J., Wan, J., and Jia, H.: Resilience to natural haz-
Pasteur, K.: From Vulnerability to Resilience, Practical Action Pub- ards: a geographic perspective, Nat. Hazards, 53, 21–41, 2010.
lishing, Rugby, 2011.