Barrett LLAKES
Barrett LLAKES
Martyn Barrett
University of Surrey, UK
Overview of the talk
Published Published
in 2016 in 2018
Published
in 2018
• The cognitive test consisted of a set of scenarios, with each scenario being
followed by a series of questions
• Each scenario:
– Depicted a real-life situation
– Focused on issues and situations where different perspectives exist
– Gave voice to these different perspectives
• The questions asked after each scenario require the student to engage in
cognitive processes such as:
– Evaluating information, formulating an argument, and explaining a
complex situation or problem
– Identifying biases or gaps in information
– Managing conflicting arguments
– Identifying and analysing multiple perspectives or world views
– Explaining differences in communication
– Recognising the importance of socially appropriate communication
conventions
– Evaluating actions and consequences by identifying and comparing
different courses of action
• The scenarios were pre-tested in field trials in nine countries
• The students’ responses to the test questions were scored using rubrics
• The rubrics were also pre-tested to ensure that they could be used by
trained scorers in a reliable and cross-culturally comparable way
• In the main data-collection in 2018, the responses yielded by the different
scenarios were found to form a single unidimensional scale
Example scenario
General conclusions
Summary take-away messages for early career
researchers and PhD students
• Ensure that you have a well-articulated conceptual framework
underpinning your activities (see the RFCDC and the PISA GC framework
for examples)
• Your framework needs to have clear specifications of the component
concepts on which your activities are focused
• Build on existing frameworks – do not build your own from scratch
• Use assessment methods that map directly onto, or are appropriate for,
your conceptual framework
• The choice of assessment methods should also be driven by the specific
goals of your work, for example:
– Educational assessment to support the development of learners’
democratic proficiency (RFCDC)
– Evaluation of the effectiveness of an educational system, programme
or intervention (PISA GC assessment)
• You also need to factor in the practicality of conducting the assessment
Further information about the RFCDC is available from:
www.coe.int/competences