CENTRAL EUROPEAN REVIEW

OF ECONOMICS AND MANAGEMENT

ISSN 2543-9472; eISSN 2544-0365

www.cerem-review.eu
Vol. 3, No. 3, 19-44, September 2019 www.ojs.wsb.wroclaw.pl

Assessment of sustainability competencies: a

literature review and future pathways for ESD
research and practice
Gisela CEBRIÁN
Camilo José Cela University, Spain
Jordi SEGALÀS, Àngels HERNÁNDEZ
Universitat Politècnica de Catalunya – Barcelona Tech, Spain

Abstract:

Aim: This paper aims at reviewing existing theoretical frameworks in sustainability competencies and
identifying suitable evaluation strategies and instruments for sustainability competencies assessment in
the context of Education for Sustainable Development.

Design / Research methods: To gain a comprehensive view of the evaluation and assessment
processes of sustainability competencies a systematic literature review was conducted using a set of
keywords. After a refining phase and selection of articles centred in evaluation processes a final sample
of 43 articles was analysed.

Conclusions / findings: Little evidence exists on the development, outcomes and impact that courses
introducing students to sustainability competencies have. Further empirical research is needed on the
development and implementation of assessment tools for sustainability competencies.

Originality / value of the article: This paper outlines the state of the art of evaluation and assessment
tools for sustainability competencies in higher education and suggests pathways for further research
and practice based on a systematic literature review.

Keywords: sustainability competencies, sustainability, higher education, assessment, evaluation

JEL: I20, I23, J24, Q01, Q56

Correspondence address: Gisela Cebrian, Camilo José Cela University, Spain. E-mail: [email protected]
Received: 02.02.2018, Revised: 11.07.2018, Accepted: 06.12.2018
doi: http://dx.doi.org/10.29015/cerem.664

© 2019 WSB UNIVERSITY IN WROCŁAW

 Gisela CEBRIÁN, Jordi SEGALÀS, Àngels HERNÁNDEZ.

1. Introduction

The past two decades have witnessed increasing recognition and political
agreement over the role of education as a major agent to transform current society
into a more sustainable, equitable and socially just one (UNESCO 2005; United
Nations 2012). This has been reflected in international and national strategy and
policy development, for example the Declaration of the United Nations Decade of
Education for Sustainable Development (abbr. UNDESD, 2005-2014) in 2005, the
United Nations Economic Commission for Europe (UNECE) Strategy for Education
for Sustainable Development (ESD) in 2011 and the 17 Sustainable Development
Goals (SDGs) of the 2030 Agenda for Sustainable Development, adopted by world
leaders at the United Nations Sustainable Development Summit in 2015. The
Declaration of the UNDESD in 2005 acted as a catalyst to the processes of
integrating the principles of education for sustainable development (ESD) into all
levels of education (UNESCO 2005). According to UNESCO (2009: 2) ESD is
based on “values of justice, equity, tolerance, sufficiency and responsibility,” with
respect as its core. The existence of diverse views of sustainability and diverse ways
to embed ESD are acknowledged as a positive element to ensure that new
developments are culturally and locally relevant but with “consensus around a range
of key principles covering the scope, purpose and practice” (Wals 2009: 25). In an
expert review, Tilbury (2011) highlighted important ESD learning processes, such as
collaboration, systems thinking, innovation, and active and participatory learning.
Higher education (HE) is a principal agent for addressing the current sustainability
challenge that society is facing, because of its key mission of knowledge generation
and transfer through research and teaching (UNESCO 2005; United Nations 2012).
Sustainability in HE calls for interdisciplinary and innovative practice to
promote sustainability in all its activities (Cotton, Winter 2010). Many academics in
the field of sustainability in HE claim a paradigm shift – an epistemological change
– is needed towards sustainability that is based on holism, critical subjectivity and
systems-thinking (Sterling 2004). The curriculum, pedagogy, structure, organisation
and ethos are shaping dimensions of education; therefore embedding sustainability
implies a cultural change rather than an add-on to existing curricula and structures.

20
ASSESSMENT OF SUSTAINABILITY COMPETENCIES: A LITERATURE REVIEW …

ESD can foster a sustainable social transformation, through the clarification and
reassessment of values; it should be creative, innovative and constructive, culturally
appropriate and action-orientated (Tilbury, Wortman 2004).
To date a number of universities worldwide have signed international
declarations and have publicly committed to embed sustainability within their
campus, outreach, education and research (Wright 2010). However, despite the
declaration of good intentions and the development of policies and strategies at the
national and international level, little has been achieved in terms of embedding
sustainability holistically in the HE curriculum (Cebrián et al. 2015). Most of the
research in the field has focussed on: environmental management and greening of
university estates and operations; descriptive case studies and examples of good
practice of universities; embedding sustainability in specific courses such as
environmental sciences, business and engineering; theoretical developments on
teaching and learning approaches towards sustainability; university and policy
analysis (Barth, Rieckmann 2016; Cotton et al. 2009; Fien 2002; Wright 2010). The
lack of theorisation of research conducted in the field has been criticised for often
leading to descriptive and non-theoretical accounts (Corcoran et al. 2004; Fien
2002). It could be argued that the focus has been on explaining part of the stories of
transformation, as papers have concentrated on the achievements and positive
experiences without paying sufficient attention to the barriers to progress and the
process of change per se (Velazquez et al. 2005). The environmental management
and greening of campus operations and estates has seen much more progress than
curriculum development (Verhulst, Lambrechts 2015).
Emergent research in the field of sustainability in HE has explored the learning
outcomes and competencies that educational programmes need to seek to develop in
students for them to become change agents towards sustainability (Cebrián, Junyent
2015; Wiek et al. 2011). Despite the divergence in the usage of different concepts
such as abilities, learning outcomes and competencies, and the existence of some
criticisms around the usage of these terms, there is a need to define competencies in
sustainability in order to foster curriculum developments and innovations.
Developing sustainability competencies amongst graduates is particularly critical to
the development of sustainability literacy and students becoming positive change

21
 Gisela CEBRIÁN, Jordi SEGALÀS, Àngels HERNÁNDEZ.

agents in their workplace and personal lives (Sipos et al. 2008). However, little
evidence exists on the development, outcomes and impact that courses introducing
students to these competencies have (Wiek et al. 2011). Further empirical research is
needed on the development and implementation of assessment tools for
sustainability competencies.

2. The Edinsost project

This paper presents a systematic literature review to identify the state of the art
of evaluation and assessment tools for sustainability competencies in higher
education, which served as the basis to guide the EDINSOST research project,
“Education and social innovation for sustainability. Training in Spanish Universities
of change agent graduates to meet challenges in society.” In this project, funded by
the Spanish Government, ten universities are working together with the goal of
creating synergies and common frameworks and criteria to integrate sustainability
competencies, learning processes and assessment tools. Project objectives are to: 1)
Define the map of sustainability competencies of university degrees covered by the
project and establish the framework to facilitate their integration in a holistic
manner; 2) Validate teaching strategies for the acquisition of sustainability
competencies, from a constructivist and community oriented pedagogical approach
(Simulacion, Case studies, Service Learning, Problem Based Learning and Project
oriented learning; 3) Diagnose the state of faculty sustainability training needs and
develop and pilot training proposals; and 4) Diagnose the state of learning of
sustainability competencies of higher education students and prepare and pilot
training proposals. This paper outlines existing theoretical frameworks in
sustainability competencies, presents a comprehensive systematic literature review
of recent literature in sustainability competencies’ assessment, and suggests suitable
assessment strategies and tools, and pathways for further research and practice.

22
ASSESSMENT OF SUSTAINABILITY COMPETENCIES: A LITERATURE REVIEW …

3. Research process

Systematic literature reviews are a commonly used in social and educational

sciences to map the state of the art of specific fields of study. The aim is to conduct
a systematic, replicable and transparent search and analysis process (Fink 2009).
Recent studies have reported on systematic literature reviews in the area of
sustainability in HE, which include quantitative and qualitative approaches (Barth,
Rieckmann 2016). For example, Lozano et al. (2017) have used hermeneutics and
grounded theory to create a framework to connect sustainability competencies and
pedagogical approaches. Jim Wu and Shen (2016) used a mix-method approach to
outline research topics that emerged during the UNDESD. Likewise, Figueiró and
Rauffle (2015) conducted a systematic review to map and evaluate the status of
sustainability in management education. In a Mindt and Rieckmann (2017)
systematic literature review the state of the art concerning teaching-learning
approaches and methods for sustainability-driven entrepreneurship in higher
education was outlined. Finally, Barth and Rieckmann (2016) outline a bibliometric
overview, which combines quantitative analysis with a qualitative analysis of
content areas and research methodologies in the field of HE for sustainability.
Gaining a comprehensive overview of the evaluation and assessment processes
of sustainability competencies in HE is essential to tackle the second objective of the
Edinsost project, which is focused on validating teaching strategies for the
development of sustainability competencies, from a constructivist and community
oriented pedagogical approach. For this reason, a systematic review of existing
research and practice focused on the design and development of sustainability
competencies assessment strategies and tools has been conducted.
The data collection process consisted of a search in the 2 main databases: Web
of Science and Scopus. These are the more relevant and comprehensive databases
covering social and educational sciences. The literature search was conducted using
the following keywords: (“higher education” OR “university” OR “universities” OR
“tertiary education” OR “college”) AND (“education for sustainability” OR
“education for sustainable development”) AND (“assessment” OR “evaluation”).
This search produced a total of 80 hits in Web of Science and 121 in Scopus. A

23
 Gisela CEBRIÁN, Jordi SEGALÀS, Àngels HERNÁNDEZ.

second phase consisted of refining the sample, removing duplicates and selecting
only articles that are peer-reviewed. In a third phase, articles focusing explicitly in
the assessment process, including the design of assessment or evaluation tools and
studies focusing on identifying students’ knowledge, attitudes and/or competencies
development were selected. This led to a final sample of 43 articles (figure 1). Going
through the steps of (1) data collection, (2) data processing and coding and (3) data
analysis, we produced an overview that combines quantitative and qualitative
analysis of content areas and evaluation strategies and instruments used.

Figure 1. Diagram of the research process

Source: Authors’ own elaboration.

4. Results of the systematic literature review

In this section the results of the review are provided. First, existing theoretical
frameworks of sustainability competencies. Second, the results in relation to
evaluation and assessment processes of sustainability competencies, providing a
general overview of publications by year and journal, are outlined. Third, the

24
ASSESSMENT OF SUSTAINABILITY COMPETENCIES: A LITERATURE REVIEW …

content of the articles is characterised by the object and focus of the assessment, and
the type of evaluation and assessment tools used. Forth, previous studies on
students’ perceptions, attitudes and behaviours in sustainability. Finally, assessment
approaches of sustainability in university programmes and curriculum.

5. Sustainability competencies frameworks

Emergent research in the field of sustainability in HE has explored the learning

outcomes and competencies that educational programmes need to seek to develop in
students for them to become change agents towards sustainability (Mochizuki,
Fadeeva 2010; Sipos et al. 2008; Svanström et al. Rowe 2008; Wiek et al. 2011).
However, it is not possible to describe a mandatory set of competencies for
sustainability because of the variety of the definitions of the terms sustainability and
competence in educational settings (Mochizuki, Fadeeva 2010). Despite the
divergence in the usage of different concepts such as abilities, learning outcomes
and competencies, and the existence of some criticisms around the usage of these
terms, there is a need to define competencies in sustainability in order to foster
curriculum developments on ESD (Wiek et al. 2011). De Haan (2010) introduces the
elements of the sustainability competence or Gestaltungskompetenz. It expresses the
abilities and competencies of students in contexts of sustainability and can be
defined as the ability to shape future scenarios by active participation in modelling
and transforming society towards sustainable practices (Barth et al. 2007).
According to De Haan (2010) the elements of sustainability competence are:
 Competence to think in a forward-looking manner, to deal with uncertainty,
and with predictions, expectations and plans for the future.
 Competence to work in an interdisciplinary manner.
 Competence to achieve open-minded perception, transcultural
understanding and cooperation.
 Participatory competence.
 Planning and implementation competence.
 Ability to feel empathy, sympathy and solidarity.

25
 Gisela CEBRIÁN, Jordi SEGALÀS, Àngels HERNÁNDEZ.

 Competence to motivate oneself and others.

 Competence to reflect in a distanced manner on individual and cultural
concepts.
Rieckmann (2012) conducted a Delphi study in which sustainability key
competencies were defined by experts from Europe and Latin American, where
systemic thinking, anticipartory and critical thinking emerged as the most relevant
ones. Moreover, in a recently conducted literature review and framework proposal
(Lozano et al. 2017) a set of twelve sustainability competencies have been
identified: systems thinking; interdisciplinary work; anticipatory thinking; justice
responsibility and ethics; critical thinking and analysis; interpersonal relations and
collaboration; empathy and change of perspective; communication and use of media;
strategic action; personal involvement; assessment and evaluation; and tolerance for
ambiguity and uncertainty.
Developing these competencies amongst graduates is particularly critical to the
development of sustainability literacy (Stibbe 2009) and students becoming positive
change agents in their workplace and personal lives (Sipos et al. 2008). The use of
certain type of pedagogies, and teaching and learning approaches and strategies such
as project-based learning, service learning and action learning (Bessant et al. 2013;
Thomas 2009), foster the competencies or skills necessary to deal with
sustainability, such as critical and creative thinking, problem-solving skills, action
competence, collaboration, and futures thinking, therefore creating empowered and
globally responsible citizens and professionals who can become active change
agents (Wals 2010).
In terms of learning outcomes, Sipos et al. (2008) developed the transformative
sustainability learning (TSL) framework and conducted three case studies on courses
related to sustainability and citizenship. They concluded that courses that were
engaging students in a cognitive, psychomotor and effective sphere enhanced TSL
(Sipos et al. 2008). Wiek et al. (2011) conducted a literature review on existing
studies and frameworks on competencies on sustainability and developed an
integrative framework on key sustainability research and problem solving
competencies, namely “systems-thinking competence, anticipatory competence,
normative competence, strategic competence, and interpersonal competence” (Wiek

26
ASSESSMENT OF SUSTAINABILITY COMPETENCIES: A LITERATURE REVIEW …

et al. 2011: 205). Other research in the area has also developed competence
frameworks for specific subject areas: engineering (Mulder et al. 2012), teacher
education (Cebrián, Junyent 2015; Sleurs 2008), and educators at all levels of
education (UNECE 2012). Research has also looked at the inclusion of sustainability
competencies in the programme descriptors of undergraduate degrees (Cortés et al.
2010; Lambrechts et al. 2013; Segalàs et al. 2009). Thus the relevance of developing
key competencies on sustainability has been acknowledged by international agencies
such as UNESCO (2005; 2017), UNECE (2009) and for accreditation agencies
(ABET 2017; Engineering Council 2013). UNESCO has recently published a set of
learning objectives for each of the 17 Sustainable Development Goals of the 2030
Agenda for Sustainable Development (UNESCO 2017). Also, UNECE
commissioned a group of ESD experts to develop a framework on ESD
competencies for educators (UNECE 2012). The UNECE framework is based on
Delors’ four pillars of education (Delors 1996). Table 1 summarises the UNECE
ESD competencies for educators framework.

Table 1. UNECE framework on ESD competencies for educators

Domains Brief description
Knowledge Learning to know refers to understanding the challenges facing
society both locally and globally and the potential role of educators
and learners (The educator understands...);
Interpersonal Learning to live together contributes to the development of
competency partnerships and an appreciation of interdependence, pluralism,
mutual understanding and peace (The educator works with others in
ways that...);
Ethics and Learning to be addresses the development of one’s personal attributes
values and ability to act with greater autonomy, judgement and personal
responsibility in relation to sustainable development (The educator is
someone who...).
Practical skills Learning to do refers to developing practical skills and action
competence in relation to education for sustainable development (The
educator is able to...);
Source: Adapted from UNECE (2012: 13-14).

However, as this is a relatively new and emerging area of research, little

evidence exists on the development, outcomes and impact that courses introducing

27
 Gisela CEBRIÁN, Jordi SEGALÀS, Àngels HERNÁNDEZ.

students to these competencies have (Lozano et al. 2017; Wiek et al. 2011). Further
empirical research is needed on the development and implementation of assessment
tools for sustainability competencies (Cebrián, Junyent 2015; Sleurs 2008). There is
still further research to be conducted to implement innovative and transformative
teaching and learning approaches and transformative institutional strategies that lead
to sustainability competencies (Barth, Rieckmann 2016; Sterling et al. 2017).
Therefore, as stated by Wiek et al. (2016) the research agenda in the following years
needs to focus on operationalising sustainability competencies, framing the different
levels of competence and measuring and evaluating students’ competencies
development.
A tendency exists to focus on developing competencies’ frameworks without
paying sufficient attention to the individual and cultural context, and the
organisational change processes required to achieve embedding ESD (Mochizuki,
Fadeeva 2010). Developing innovative courses that consider sustainability
competencies can foster transformative learning amongst students but also engage
stakeholders and the community, and in turn contribute to generate organisational
change in the context of HE by opening up innovative programme designs
(Mochizuki, Fadeeva 2010).

6. General overview of publications focused on evaluation and assessment

processes of sustainability competencies

The review shows that the 43 research articles identified, with focus on
evaluation and assessment processes of sustainability competencies, were found
between the period of 2005 and 2017. After 2005 there is an increasing tendency
due to the catalyst or lever effect of the UN Decade on Education for Sustainable
Development (UNESCO 2005), with a pick in 2015 coinciding with the end of the
decade and with the hold of special issues in ESD of non-specific sustainability
journals (figure 2). As stated in the previous sections, this a relatively new and
emerging research area, so it is expected an increasing body of literature focused on
the design and development of innovative teaching and learning methods, and
assessment tools for sustainability competencies in the next years (Wiek et al. 2016).

28
ASSESSMENT OF SUSTAINABILITY COMPETENCIES: A LITERATURE REVIEW …

Figure 2. Distribution of the research articles focusing on evaluation processes

of sustainability competencies

Source: Authors’ own elaboration.

Regarding the distribution by journal (Figure 3), the “Journal of Cleaner

Production and International Journal of Sustainability in Higher Education” emerged
as the most numerous sources, with a 21% of the total publications each. The
following largest source is Environmental Education Research with a 9% of the total
of publications both. The scope of all these journals are to help advance
understanding of environmental issues, sustainability and ESD through focusing on
papers reporting research in the area. Also, the journal Assessment & Evaluation in
Higher Education has a 9% of total publications, due to a Special Issue in
assessment and evaluation of sustainable development in HE in 2015.

29
 Gisela CEBRIÁN, Jordi SEGALÀS, Àngels HERNÁNDEZ.

Figure 3. Distribution of the research articles by journal

Source: Authors’ own elaboration.

A word cloud of the keywords of the 43 papers was conducted, where the
common keywords education, Education for Sustainability, Education for
Sustainable Development, Sustainability and Sustainable Development were
removed in order to provide a more detailed overview of relevant keywords. The
word cloud shows a set of predominant keywords such as learning, assessing,
curriculum, curricula, competences, campus and environmental (Figure 4). It also
provides some insights into other commonly used words such as attitudes, values,
outcomes, system, engineering and teacher amongst others. This shows the focus on
engineering and teacher education studies, and attitudes, values and learning
outcomes.

30
ASSESSMENT OF SUSTAINABILITY COMPETENCIES: A LITERATURE REVIEW …

Figure 4. Word cloud of keywords

Source: Authors’ own elaboration.

7. Tools for the evaluation and assessment of sustainability performance

amongst students

The articles were analysed in relation to the object of the evaluation or

assessment (Figure 5), 33% of the articles focus on evaluation and assessment of
sustainability competencies, skills and outcomes, 23% explore the perceptions,
understandings, attitudes and behaviours of students and 21% have the goal of
designing assessment tools for students’ learning, programmes of study or the
sustainability performance of universities. Finally, 14% of the papers found centered
in the assessment of the inclusion of sustainability in specific university programmes
or courses and only 9% in assessing students’ knowledge and understandings of
sustainability.

31
 Gisela CEBRIÁN, Jordi SEGALÀS, Àngels HERNÁNDEZ.

Figure 5. Object of the evaluation and assessment publication

Source: Authors’ own elaboration.

From the set of articles assessing competencies, skills or learning outcomes a

diversity of contexts, subject areas and tools are found. For example, Cebrián and
Junyent (2015) created an open-ended questionnaire to explore teacher students’
perceived ESD competencies. Nikel (2007) used survey questionnaires, narrative
tasks and an interview to study the perspectives of 30 student teachers about ESD
competencies. Segalàs, Ferrer-Balas and Mulder (2008; 2010) undertook a 5-year
research project to analyse how sustainability competencies were introduced into
technological universities. Conceptual maps were used as assessment tools of
sustainability competencies. Habron, Goralnik and Thorp (2012) assessed
undergraduates’ systems thinking competency through a short answer exam, online
interactive small group dialogue exam, homework assignments, completion of an
online community engagement tutorial, and completion of a final reflexive project
(either in a group or individual). Mercer et al. (2017) used educational game design
to foster design thinking and communication skills amongst students and assessed
students’ development using questionnaires and qualitative feedback. Moreover,
Warr et al. (2017) designed and assessed the impact of a cross-disciplinary place-

32
ASSESSMENT OF SUSTAINABILITY COMPETENCIES: A LITERATURE REVIEW …

based learning initiative on both the operational and student learning outcomes.
Hegarty et al. (2011) evaluated student-learning outcomes in a stand-alone course on
sustainability through critical analysis of articles, ecological footprint calculator and
field-specific problem analysis (PBL). Rose, Ryan and Desha (2015) undertook a
curriculum renewal to embed sustainability into a first year engineering curriculum
and used “before and after surveys” to evaluate learning outcomes. Furthermore,
Shephard et al. (2015) used a longitudinal mixed-effects repeat-measures statistical
model to assess the development of affective outcomes related to sustainability.
Pretorius, Lombard and Khotoo (2016) used evidence-based reflection to provide a
narrative assessment of the experience gained with Inquiry-based in two
undergraduate sustainability-focussed modules in open and distance learning at the
University of South Africa. In recently published research, García, Junyent and
Fonolleda (2017) have developed a rubric to assess professional competencies in
ESD. Likewise, Sandri, Holdsworth and Thomas (2018) propose an assessment tool,
based on a scenario/vignette question design, to capture data on sustainability
graduate attributes in context and has the potential to be used across universities to
enable comparative research.
Regarding the type of evaluation, there are only 25 cases that specify what type
of evaluation is conducted. Of these, 19 correspond to summative evaluation and 6
to formative evaluation. Regarding the involvement of students in their own
evaluation, only 6 cases out of 25 use self-assessment tools. When analysing the
assessments tools, the most commonly used is the survey and questionnaire (used in
20 cases), followed by reflexive diary (used in 5 cases) and interviews (used in 4
cases) (Figure 6). One of the main reasons why questionnaires are used, is because it
is less time-consuming, easy to distribute amongst a larger number of students and
in turn it provides a larger amount of information. The challenge is that
questionnaires do not allow obtaining other type of information regarding the
process of learning itself.

33
 Gisela CEBRIÁN, Jordi SEGALÀS, Àngels HERNÁNDEZ.

Figure 6. Type of assessment tools used

Source: Authors’ own elaboration.

As the findings of the literature review show, a variety of tools are suggested
and used to assess competencies and learning outcomes. Therefore, using a range of
assessment tools can be positive to gain a more comprehensive overview of the
development of sustainability competencies. However, this also mirrors the lack of a
common framework of sustainability competencies and effective teaching and
learning approaches that help students develop these competencies (Sterling et al.
2017). Further research is needed to design and validate instruments for assessing
and monitoring students’ sustainability performance (Barth, Rieckmann 2016).

8. Studies on students’ perceptions, attitudes and behaviours in sustainability

Most of the studies exploring university students’ perceptions, understanding,

attitudes and behaviour have used questionnaires or surveys (Azapagic et al. 2005;
Kagawa 2007; Murga-Mentoyo 2008). For example, Biasutti and Frate (2017)
developed and validated a quantitative 20-item scale that measured Italian university

34
ASSESSMENT OF SUSTAINABILITY COMPETENCIES: A LITERATURE REVIEW …

students’ attitudes towards sustainable development. Mosher and Desrochers (2014)

conducted a pretest posttest study to evaluate the impact of a sustainability
workshop in changing students’ behaviour. Moreover, Solís-Espallargas and
Valderrama-Hernández (2015) explored the change of perception of teacher students
due to a specific module on ESD through action-research and pre- post-
questionnaires. In a study conducted in Australia a values approach was outlined to
incorporate sustainability concepts into business courses (Sidiropoulos 2014).
Student feedback showed how a values approach to ESD effectively produces
changes in values, attitudes and behaviour over time, building graduate capability in
sustainability.
Furthermore, other papers appeared in the search that focus on university
educators’ perceptions, knowledge or sustainability competencies. Aznar, Ull,
Piñero and Martínez-Agut (2017) used an evaluative research based on a
quantitative approach to assess the impact of the inclusion of sustainability within
the teacher education curriculum. They used questionnaires and in-depth interviews
with faculty to evaluate their knowledge, perception and attitude towards
sustainability directly impacting on the training of future teachers. Cebrián (2015;
2017) conducted a collaborative action research with academic staff to foster critical
reflection and action towards embedding ESD in teaching practice. Findings showed
how action research enabled a change of vision and understanding of ESD and
endorsed new teaching practices. Roberts and Roberts (2008) hosted a staff
development event to provide a space for exchanging and sharing innovative
practice in ESD in the university context.

9. Assessment of sustainability in university programmes and curriculum

Also research focused on the assessment of the inclusion of sustainability in

university programmes and curriculum in different universities appeared in the
review (Watson et al. 2013). Specific tools have been designed for this purpose,
such as Sustainability Tool for Assessing University’s Curricula Holistically
(STAUNCH) (Glover et al. 2011). This was used across the Welsh higher education
sector and emerged as a valuable tool for recognising what is being offered in the

35
 Gisela CEBRIÁN, Jordi SEGALÀS, Àngels HERNÁNDEZ.

curriculum about sustainability. However, it does not reflect the quality or

effectiveness of the curriculum content (Glover et al. 2011). Other studies
(Lambrechts et al. 2013) have evaluated the presence and integration of
sustainability competencies in different programmes and curricula. Makrakis and
Kostoulas-Makrakis (2016) conducted a sequential mixed methods evaluation in
RUCAS programme “Reorientation of University Curricula to Address
Sustainability (RUCAS): A European Commission Tempus-funded Programme.”
Using both qualitative and quantitative approaches in combination in an iterative
evaluation process was found useful to enrich and produce more robust results.
Sustainability assessment tools for sustainability programmes have been
characterised, which include indicators and criteria for university performance in
management, operations, estates, curriculum and outreach. For example, the
INDICARE-model assesses participatory processes in HE’s sustainability initiatives
(Disterheft et al. 2016). As pointed out by Fischer, Jenssen and Tappeser (2015) in a
comparative analysis of 12 assessment tools for sustainable universities, these have
become more than instrumental facilitators of change. They have also established
normative standards by framing what fields and issues should universities engage
with.

10. Conclusions

As the findings of this review indicate a variety of frameworks of sustainability

competencies and learning outcomes have been suggested (Wiek et al. 2011). This
mirrors the lack of common definitions and frameworks, and the importance of
defining common frameworks of sustainability competencies. This is a previous
necessary step to create innovative teaching and learning, and transformative
institutional approaches that can lead to sustainability competencies (Barth,
Rieckmann 2016). So far little evidence exists on the development, outcomes and
impact that courses introducing students to these competencies have (Lozano et al.
2017). The research efforts in the next years need to be put on operationalizing

36
ASSESSMENT OF SUSTAINABILITY COMPETENCIES: A LITERATURE REVIEW …

sustainability competencies and developing tools to measure and evaluate students’

competencies development (Cebrián, Junyent 2015).
From the articles analysed in this review, a divergence in the object and focus of
the evaluation and assessment is observed, which includes competencies, skills,
outcomes, perceptions, attitudes and behaviours of students. Also, different
assessment approaches and tools are used, such as questionnaires, reflexive diaries,
interviews, narrative tasks, scenario/vignette question design, conceptual maps and
pre-post-test amongst others. The results indicate that most of the papers published
centred in summative evaluation rather than formative or self-assessment. Using a
range of assessment tools can be positive to gain a more comprehensive overview of
the development of sustainability competencies. However, it is critical to develop
effective teaching and learning approaches that help students develop these
competencies (Sterling et al. 2017), jointly with the design and implementation of
summative, formative and self-assessment tools.
Over the last years, there has been a rapid increase on the number of
publications regarding the assessment of sustainability competencies. Also it is
plausible an emergent diversification of the assessment tools used. Questionnaires
have been commonly used to assess or explore students’ knowledge, attitudes and
behaviours towards sustainability. Other tools have been identified as suitable to
assess sustainability competencies such as reflexive diaries, interviews, conceptual
maps, rubrics and scenario/vignette visioning (Sandri et al. 2018). There is an
emergent literature on qualitative assessment tools such as interviews and reflexive
diaries or portfolios that facilitate the assessment of more normative sustainability
competencies. However, still a lot of the articles focus on summative evaluation
rather than formative or self-evaluation assessment. Thus developing formative and
self-evaluation tools is needed in order to get a more comprehensive overview of
students’ learning process and sustainability competencies.
However the emergence of studies on the design of assessment tools, it is still
necessary to conduct further developments and research in this area. Based on the
review conducted, authors suggest the following pathways for future research and
practice that will improve the assessment methodology and tools of sustainability
competencies:

37
 Gisela CEBRIÁN, Jordi SEGALÀS, Àngels HERNÁNDEZ.

 Conduct longitudinal studies using summative, formative and self-

assessment tools within HE and in the professional life of graduates, which
can provide evidence of the development of sustainability competencies
through higher education studies and in the posterior professional life.
 Carry out comparative analysis of different assessment tools against
sustainability competencies. This would provide evidence on the
effectiveness of different assessment tools.
 Design and test assessment tools in line with ESD principles such as critical
thinking, collaboration, teamwork and systems thinking.
 Develop specific rubrics for each sustainability competence and adapt them
to different programmes and contexts.

Acknowledgments

The authors gratefully acknowledge the financial support received from the
Spanish Ministry of Economy and Competitiveness to conduct the project
EDINSOST “Education and social innovation for sustainability. Training in Spanish
Universities of change agent graduates to meet social challenges” (Ref. EDU2015-
65574-R).

References

ABET (2017), Criteria for accrediting engineering programs, Engineering Accreditation Commission,
USA.

Azapagic A., Perdan S., Shallcross D. (2005), How much do engineering students know about
sustainable development? The findings of an international survey and possible implications for the
engineering curriculum, „European Journal of Engineering Education”, vol. 30 no. 1, pp. 1-19.

Aznar Minguet P., Ull M.A., Piñero A., Martínez-Agut M.P. (2017), La evaluación de la formación de
formadores. Un catalizador en el proceso de cambio curricular hacia la sostenibilidad. „Revista
Iberoamericana de Educación”, vol. 73, pp. 225-252.

Barth M., Rieckmann M. (2016), State of the art in research on higher education for sustainable
development, in: Routledge handbook of higher education for sustainable development, eds. Barth M.,
Michelsen G., Rieckmann M, Thomas I., Routledge, London, pp. 100-113.

38
ASSESSMENT OF SUSTAINABILITY COMPETENCIES: A LITERATURE REVIEW …

Barth M., Godemann J., Rieckmann M., Stoltenberg U. (2007), Developing key competencies for
sustainable development in higher education, „International Journal of Sustainability in Higher
Education”, vol. 8 no. 4, pp. 416-430.

Bessant S., Bailey P., Robinson Z., Ormerod M., Tomkinson C.B., Tomkinson R., Boast R. (2013),
Problem-based learning. A case study of sustainability education. A toolkit for university educators,
http://www.keele.ac.uk/media/keeleuniversity/group/hybridpbl/PBL_ESD_Case%20Study_Bessant,%2
0et%20al.%202013.pdf [10.10.2013].

Biasutti M., Frate S. (2017), A validity and reliability study of the attitudes toward sustainable
development scale, „Environmental Education Research”, vol. 23 no. 2, pp. 214-230.

Cebrián G. (2017), A collaborative action research project towards embedding ESD within the higher
education curriculum, „International Journal of Sustainability in Higher Education”, vol. 18 no. 6, pp.
857-876.

Cebrián G., Junyent M. (2015), Competencies in education for sustainable development. Exploring the
student teachers’ views, „Sustainability”, vol. 7 no. 3, pp. 2768-2786.

Cebrián G., Grace M., Humphris D. (2015), Academic staff engagement in education for sustainable
development, „Journal of Cleaner Production”, vol. 106, pp. 79-86.

Corcoran P.B., Walker K.E., Wals A.E.J. (2004), Case studies, make-your-case studies, and case
stories. A critique of case-study methodology in sustainability in higher education. „Environmental
Education Research”, vol. 10 no. 1, pp. 7-21.

Cortés A.C., Segalàs J., Cebrián G., Junyent M., Tilló T., Marquilles P., Mora M. (2010), Sustainability
competences in Catalan university degrees. Paper presented at The 14th European Roundtable on
Sustainable Production and Consumption (ERSCP) – The 6th Environmental Management for
Sustainable Universities (EMSU), Delft, The Netherlands, October 25-29th, 2010.

Cotton D., Bailey I., Warren M., Bissell S. (2009), Revolutions and second-best solutions. Education
for sustainable development in higher education, „Studies in Higher Education”, vol. 34 no. 7, pp. 719-
733.

Cotton D., Winter J. (2010), It’s not just bits of paper and light bulbs. A review of sustainability
pedagogies and their potential for use in higher education, in: Sustainability education. Perspectives
and practice across higher education, eds. Jones P., Selby D., Sterling S., Earthscan, London, pp. 39-54.

De Haan G. (2010), The development of ESD-related competencies in supportive institutional

frameworks, „International Review of Education”, vol. 56 no. 2-3, pp. 315–328.

Delors J. (1996), Learning. The treasure within, UNESCO, Paris,

http://www.unesco.org/education/pdf/15_62.pdf [23.01.2011].

Disterheft A., Caeiro S.S., Filho W.L., Azeiteiro U.M. (2016), The INDICARE-model – measuring and
caring about participation in higher education’s sustainability assessment, „Ecological Indicators”, vol.
63, pp. 172-186.

Engineering Council (2013), Guidance on sustainability for the engineering profession, Engineering
Council, London.

39
 Gisela CEBRIÁN, Jordi SEGALÀS, Àngels HERNÁNDEZ.

Fien J. (2002), Advancing sustainability in higher education. Issues and opportunities for research,
„International Journal of Sustainability in Higher Education”, vol. 3 no. 3, pp. 243-253.

Figueiró P.S., Rauffle E. (2015), Sustainability in higher education. A systematic review with focus on
management education, „Journal of Cleaner Production”, vol. 106 no. 1, pp. 22-33.

Fink A. (2009), Conducting research literature reviews. From the Internet to paper, Sage, Los Angeles.

Fischer D., Jenssen S., Tappeser V. (2015), Getting an empirical hold of the sustainable university. A
comparative analysis of evaluation frameworks across 12 contemporary sustainability assessment tools,
„Assessment & Evaluation in Higher Education”, vol. 40 no. 6, pp. 785-800.

García M.R, Junyent M., Fonolleda M. (2017), How to assess professional competencies in Education
for Sustainability? An approach from a perspective of complexity, „International Journal of
Sustainability in Higher Education”, vol. 18 no. 5, pp. 772-797.

Glover A., Peters C., Haslett S.K. (2011), Education for sustainable development and global
citizenship. An evaluation of the validity of the STAUNCH auditing tool, „International Journal of
Sustainability in Higher Education”, vol. 12 no. 2, pp. 125-144.

Habron G., Goralnik L., Thorp L. (2012), Embracing the learning paradigm to foster systems thinking,
„International Journal of Sustainability in Higher Education”, vol. 13 no. 4, pp. 378–393.

Hegarty K., Thomas I., Kriewaldt C., Holdsworth S., Bekessy S.A. (2011), Insights into the value of a
“stand-alone” course for sustainability education, „Environmental Education Research”, vol. 17 no. 4,
pp. 451-469.

Jim Wu, Y.C., Shen J.P. (2016), Higher education for sustainable development. A systematic review.
„International Journal of Sustainability in Higher Education”, vol. 17 no. 5, pp. 633-651.

Kagawa F. (2007), Dissonance in students’ perceptions of sustainable development and sustainability.

Implications for curriculum change, „International Journal of Sustainability in Higher Education”, vol.
8 no. 3, pp. 317-338.

Lambrechts W., Mulà I., Ceulemans K., Molderez I., Gaeremynck V. (2013), The integration of
competences for sustainable development in higher education. An analysis of bachelor programs in
management, „Journal of Cleaner Production”, vol. 48, pp. 65-73.

Lozano R., Merrill M.Y., Sammalisto K., Ceulemans K., Lozano F.J. (2017), Connecting competences
and pedagogical approaches for sustainable development in higher education. A literature review and
framework proposal, „Sustainability”, vol. 9 no. 1889, pp. 1-15.

Makrakis V., Kostoulas-Makrakis N. (2016), Bridging the qualitative-quantitative divide. Experiences

from conducting a mixed methods evaluation in the RUCAS programme, „Evaluation and Program
Planning”, vol. 54, pp. 144-151.

Mercer T.G., Kythreotis A.P., Robinson Z.P., Stolte T., George S.M., Haywood S.K. (2017), The use
of educational game design and play in higher education to influence sustainable behaviour,
„International Journal of Sustainability in Higher Education”, vol. 18 no. 3, pp. 359-384.

40
ASSESSMENT OF SUSTAINABILITY COMPETENCIES: A LITERATURE REVIEW …

Mindt L., Rieckmann M. (2017), Developing competencies for sustainability-driven entrepreneurship

in higher education. A literature review on teaching and learning methods, „Teoría de la Educación.
Revista Interuniversitaria”, vol. 29 no. 1, pp. 129-159.

Mochizuki Y., Fadeeva Z. (2010), Competences for sustainable development and sustainability:
Significance and challenges for ESD, „International Journal of Sustainability in Higher Education”,
vol. 11 no. 4, pp. 391-403.

Mosher H.R., Desrochers M. (2014), The effects of information regarding sustainability issues and
behavioral self-management instruction on college students’ energy conservation, „International
Journal of Sustainability in Higher Education”, vol. 15 no. 3, pp. 359-370.

Muler K.F., Segalàs J., Ferrer-Balas D. (2012), How to educate engineers for/in sustainable
development. Ten years of discussion, remaining challenges, „International Journal of Sustainability in
Higher Education”, vol. 13 no. 3, pp. 211-218.

Murga-Mentoyo M.A. (2008), Percepciones, valores y actitudes ante el desarrollo sostenible. Detección
de necesidades educativas en estudiantes universitarios, „Revista Española de Pedagogía”, vol. 66 no.
240, pp. 327-343.

Nikel J. (2007), Making sense of education “responsibly”. Findings from a study of student teachers’
understanding(s) of education, sustainable development and Education for Sustainable Development,
„Environmental Education Research”, vol. 13 no. 5, pp. 545-564.

Pretorius R., Lombard A., Khotoo A. (2016), Adding value to education for sustainability in Africa
with inquiry-based approaches in open and distance learning, „International Journal of Sustainability in
Higher Education”, vol. 17 no. 2, pp. 167-187.

Rieckmann M. (2012), Future-oriented higher education. Which key competencies should be fostered
through university teaching and learning?, „Futures”, vol. 44 no. 2, pp. 127-135.

Roberts C., Roberts J. (2008), Starting with the staff. How swapshops can develop ESD and empower
practitioners, „Environmental Education Research”, vol. 14 no. 4, pp. 423-434.

Rose G., Ryan K., Desha C. (2015), Implementing a holistic process for embedding sustainability. A
case study in first year engineering, Monash University, Australia, „Journal of Cleaner Production”,
vol. 106 no. 1, pp. 229-238.

Sandri O., Holdsworth S., Thomas I. (2018), Vignette question design for the assessment of graduate
sustainability learning outcomes, „Environmental Education Research”, vol. 24 no. 3, pp. 406-426.

Segalàs J., Ferrer-Balas D., Svanström M., Lundqvist U., Mulder K.F. (2009), What has to be learnt for
sustainability? A comparison of bachelor engineering education competences at three European
universities, „Sustainability Science”, vol. 4 no. 1, pp. 17-27.

Segalàs J., Ferrer-Balas D., Mulder K.F. (2008), Conceptual maps. Measuring learning processes of
engineering students concerning sustainable development, „European Journal of Engineering
Education”, vol. 33 no. 3, pp. 297-306.

41
 Gisela CEBRIÁN, Jordi SEGALÀS, Àngels HERNÁNDEZ.

Segalàs J., Ferrer-Balas D., Mulder K.F. (2010), What do engineering students learn in sustainability
courses? The effect of the pedagogical approach, „Journal of Cleaner Production”, vol. 18 no. 3, pp.
275-284.

Shephard K., Harraway J., Lovelock B., Mirosa M., Skeaff S., Slooten L., Strack M., Furnari M.,
Jowett T., Deaker L. (2015), Seeking learning outcomes appropriate for “education for sustainable
development” and for higher education, „Assessment & Evaluation in Higher Education”, vol. 40 no. 6,
pp. 855-866.

Sidiropoulos E. (2014), Education for sustainability in business education programs. A question of

value, „Journal of Cleaner Production”, vol. 85, pp. 472-487.

Sipos Y., Battisti B., Grimm K. (2008), Achieving transformative sustainability learning. Engaging
head, hands and heart, „International Journal of Sustainability in Higher Education”, vol. 9 no. 1, pp.
68-86.

Sleurs W. (2008), Competencies for ESD (Education for Sustainable Development) teachers. A
framework to integrate ESD in the curriculum of teacher training institutes – Comenius 2.1 project
118277-CP-1-2004-BE-Comenius-C2.1.,
http://www.unece.org/fileadmin/DAM/env/esd/inf.meeting.docs/EGonInd/8mtg/CSCT%20Handbook_
Extract.pdf [15.03.2011].

Solís-Espallargas C., Valderrama-Hernández R. (2015), La educación para la sostenibilidad en la

formación de profesorado. ¿Qué estamos haciendo?, „Foro de Educación”, vol. 19, pp. 165-192.

Sterling S. (2004), Higher education, sustainability, and the role of systemic learning, in: Higher
education and the challenge of sustainability. Problematics, promise and practice, eds. Corcoran P.B.,
Wals A.E.J., Kluwer Academic Publishers, Dordrecht, pp. 49-70.

Sterling S., Glasser H., Rieckmann M., Warwick P. (2017), „More than scaling up”. A critical and
practical inquiry into operationalizing sustainability competencies, in: Envisioning futures for
environmental and sustainability education, eds. Corcoran P.B., Weakland J.P., Wals A.E.J.,
Wageningen Academic Publishers, The Netherlands, pp. 153-168.

Stibbe A. (ed.) (2009), The handbook of sustainability literacy. Skills for a changing world, Green
Books, Devon.

Svanström M., Lozano-Garcia F., Rowe D. (2008), Learning outcomes for sustainable development in
higher education, „International Journal of Sustainability in Higher Education”, vol. 9 no. 3, pp. 339-
351.

Thomas I. (2009), Critical thinking, transformative learning, sustainable education, and problem-based
learning in universities, „Journal of Transformative Education”, vol. 7 no. 3, pp. 245-264.

Tilbury D. (2011), Education for sustainable development. An expert review of processes and learning,
UNESCO, Paris, http://unesdoc.unesco.org/images/0019/001914/191442e.pdf [16.09.2011].

Tilbury D., Wortman D. (2004), Engaging people in sustainability, IUCN, Gland.

42
ASSESSMENT OF SUSTAINABILITY COMPETENCIES: A LITERATURE REVIEW …

UNECE (2009), Learning from each other. The UNECE strategy for education for sustainable
development, UNECE, Geneva, http://sustainabledevelopment.un.org/content/documents/798ece5.pdf
[21.02.2011].

UNECE (2012), Learning for the future. Competences in education for sustainable development,
UNECE, Geneva,
http://www.unece.org/fileadmin/DAM/env/esd/ESD_Publications/Competences_Publication.pdf
[13.02.2013].

UNESCO (2005), United Nations Decade of Education for Sustainable Development (2005-2014).
Draft International Implementation Scheme, UNESCO, Paris,
http://portal.unesco.org/education/en/file_download.php/e13265d9b948898339314b001d91fd01draftFi
nal+IIS.pdf [10.11.2011].

UNESCO (2009), UNESCO World Conference on Education for Sustainable Development. Bonn
Declaration, UNESCO, Paris, http://www.esd-world-conference-
2009.org/fileadmin/download/ESD2009_BonnDeclaration080409.pdf [15.11.2011].

UNESCO (2017), Education for sustainable development goals. Learning objectives, UNESCO, Paris,:
http://unesdoc.unesco.org/images/0024/002474/247444e.pdf [15.06.2018].

United Nations (2012), The future we want. Outcome document adopted at Rio+20,
http://www.uncsd2012.org/content/documents/727The%20Future%20We%20Want%2019%20June%2
01230pm.pdf [15.09.2012].

Velazquez L., Munguia N., Sanchez M. (2005), Deterring sustainability in higher education
institutions. An appraisal of the factors which influence sustainability in higher education institutions,
„International Journal of Sustainability in Higher Education”, vol. 6 no. 4, pp. 383-391.

Verhulst E., Lambrechts W. (2015), Fostering the incorporation of sustainable development in higher
education. Lessons learned from a change management perspective, „Journal of Cleaner Production”,
vol. 106, pp. 189-204.

Wals A.E.J. (2009), Review of contexts and structures for education for sustainable development 2009,
UNESCO, Paris, http://unesdoc.unesco.org/images/0018/001849/184944e.pdf [03.06.2012].

Wals A.E.J. (2010), Mirroring, Gestaltswitching and transformative social learning. Stepping stones for
developing sustainability competence, „International Journal of Sustainability in Higher Education”,
vol. 11 no. 4, pp. 380-390.

Warr Pedersen K., Pharo E., Peterson C., Clark G.A. (2017), Wheels of change in higher education. A
collaborative, multi-stakeholder project as a vehicle for sustainability education, „International Journal
of Sustainability in Higher Education”, vol. 18 no. 2, pp. 171-184.

Watson M.K., Lozano R., Noyes C., Rodgers M. (2013), Assessing curricula contribution to
sustainability more holistically. Experiences from the integration of curricula assessment and students’
perceptions at the Georgia Institute of Technology, „Journal of Cleaner Production”, vol. 61, pp. 106-
116.

43
 Gisela CEBRIÁN, Jordi SEGALÀS, Àngels HERNÁNDEZ.

Wiek A., Bernstein M.J., Foley R.W., Cohen M., Forrest N., Kuzdas C. et al. (2016), Operationalising
competencies in higher education for sustainable development, in: Routledge handbook of higher
education for sustainable development, eds. Barth M., Michelsen G., Thomas I., Rieckmann M.,
Routledge, London, pp. 241-260.

Wiek A., Withycombe L., Redman C.L. (2011), Key competencies in sustainability. A reference
framework for academic program development, „Sustainability Science”, vol. 6 no. 2, pp. 203-218.

Wright T.S.A. (2010), University presidents’ conceptualizations of sustainability in higher education,

„International Journal of Sustainability in Higher Education”, vol. 11 no. 1, pp. 61-73.

44