Key Risk Indicators

Operational Risk Sound

Practice Guidance

An IRM Group Company

Foreword
The Institute of Operational Risk (IOR) was created in January 2004 and became part of
the Institute of Risk Management in 2019. The IOR’s mission is to promote the development
of operational risk as a profession and to develop and disseminate sound practice for the
management of operational risk.
The need for effective operational risk management is more acute than ever. Events such as
the global financial crisis or the COVID-19 pandemic highlight the far-reaching impacts of
operational risk and the consequences of management failure. In the light of these and numerous
other events organisations have to ensure that their policies, procedures, and processes for the
management of operational risk meet the needs of their stakeholders.
This guidance is designed to complement existing standards and codes for risk management
(e.g. ISO31000). The aim is to provide guidance that is both focused on the management of
operational risk and practical in its application. In so doing, this is a guide for operational risk
management professionals, to help them improve the practice of operational risk in organisations.
Readers looking for a general understanding of the fundamentals of operational risk management
should start with the IOR’s Certificate in Operational Risk.
Not all the guidance in this document will be relevant for every organisation or sector. However,
it has been written with the widest possible range of organisations and sectors in mind. Readers
should decide for themselves what is relevant for their current situation. What matters is gradual,
but continuous improvement.

The Institute of Operational Risk Sound Practice

Guidance
Although there is no one-size-fits-all approach to the management of operational risk,
organisations must benchmark and improve their practice regularly. This is one of a series of
papers, which provides practical guidance on a range of important topics that span the discipline
of operational risk management. The objectives of these papers are to:
• Explain how to design and implement a ‘sound’ (robust and effective) operational risk
management framework
• Demonstrate the value of operational risk management
• Reflect the experiences of risk professionals, including the challenges involved in developing
operational risk management frameworks

2
Contents
1. Introduction 4
2. Definitions 5
2.1 Risk indicators 5
2.2 Control indicators 6
2.3 Performance indicators 6
2.4 Different metrics have different uses in different contexts 6
2.5 Key indicators 7
3. Using indicators 8
3.1 Risk monitoring 8
3.2 Implementing risk appetite 8
3.3 Governance and assurance 8
3.4. Risk assessment and modelling 8
4. The Desirable Features of Operational Risk Indicators 10
4.1 Relevant 10
4.2 Measurable 10
4.3 Forward looking (Leading) 11
4.4 Easy to collect and monitor 12
4.5 Comparable 13
4.6 Auditable 13
5. Selecting Indicators and Setting Thresholds and Limits 15
5.1 Selecting indicators: top down or bottom up? 15
5.2 Deciding frequency 16
5.3 Triggers for escalation 17
6. Managing and Reporting Risk Indicators 18
6.1 Adding or changing indicators 18
6.2 Continual improvement 18
6.3 Taking action to resolve threshold or limit breaches 19
6.4 Reporting 19
6.4.1 Reporting to different audiences 19
6.4.2 Frequency 20
6.4.3 Data visualisation 21
7. Conclusion 23

3
Section 1 - Introduction
Risk indicators, commonly known as ‘Key Risk Indicators’ or ‘KRIs’ are an important operational
risk management tool. As with any type of risk, operational risk exposures are dynamic and
change frequently. Operational risk indicators offer a cost-effective means to keep track of
potential changes in exposure.

All organisations use operational risk indicators in some form or another. Management relies on a
range of indicators to help them do their jobs and make effective decisions. This includes metrics
relating to the performance of people, processes and systems, along with the impact of external
events, four elements that define the scope of operational risk. These indicators are monitored
by management at different levels within an organisation, right up to the executive and board of
directors.

The IOR’s view is that the monitoring of operational risk indicators forms an important part of an
operational risk management framework and that the (operational) risk function should play a
central role in the design and implementation of risk indicators as a managerial decision tool. A
well organised operational risk indicator process can support the assessment, monitoring and
control of operational risk exposures. Helping to improve risk awareness and facilitating well-
informed operational risk management decisions.

4
Section 2 - Definitions
Operational risk indicators are measurable metrics that provide a proxy for operational risk
exposure. A change in the value of a metric signals that a particular risk exposure may be
changing, that it may be increasing or decreasing in probability or impact, or that a risk event
may be about to occur very soon. In this regard an indicator may signal:
• A potential change in inherent/gross exposure (the underlying probability and or impact of risk
events) to one or more categories of operational risk
• Control weaknesses and hence a change in residual exposure
• A decline in the performance of the organisation due to the impact of operational risk
exposures
• A decline in the performance of the operational risk management framework
2.1 Risk indicators
A risk indicator acts as a proxy for risk exposure. A change in the value of a risk indicator signals
a change in probability and or impact. In this regard, risk indicators may relate to the causes
or effects of operational risk events. Table 1 contains some examples of causal and effect
indicators:

Causal Indicators Effect Indicators

Number and type of causes identified in The direct financial cost of operational loss events
loss event or near miss data collection (asset write downs, provisions for liability claims)
Staff turnover as a % of staff The indirect costs of operational loss events (e.g.
lost market share, goodwill payments to customers,
fines, etc)
Staff morale (collected from staff surveys) Duration of staff absence due to health and safety
incidents
Number of IT patches not implemented Customer satisfaction scores
Number of attempted IT hacking attacks Number and duration of disruptions to operational
processes and systems
Number of overdue internal audit actions Number of negative press reports following a loss
event
Number of manual interventions to Number of negative social media posts following a
correct automated process failures loss event

Table 1: Example risk indicators for cause and effect

Usually, risk indicators signal a change in an organisation’s underlying exposure to risk, also
known as inherent or gross risk. This means that action may be required to enhance the control
environment to maintain the current level of residual or net risk (exposure net of controls). If action
is not taken, then an increase in inherent or gross risk may translate into an increase in residual
risk.

5
2.2 Control indicators
Control effectiveness indicators, usually referred to as Key Control Indicators (KCIs), are metrics
that provide information on the extent to which a given operational risk control is meeting its
intended objectives. KCIs indicate the effectiveness of particular controls at a particular point
in time. Examples of KCIs include the results of formal control testing, along with loss and near-
miss information which relates to the success or failure of controls about specific operational risk
events.

For further guidance on the assessment and monitoring of control effectiveness please refer to
the IOR’s Sound Practice Guidance Paper on Risk and Control Self Assessment.
2.3 Performance indicators
Performance indicators usually referred to as Key Performance Indicators (KPIs), measure
performance or the progress made towards the achievement of targets. This might include
financial performance, or the performance of processes and systems, along with progress
towards the achievement of financial targets, business plans and project plans.

KPIs are used widely in finance, accounting and general business management, but are equally
applicable in the context of operational risk. Some financial, accounting or business management
performance indicators may function as risk indicators (e.g. a high level of business growth can
put pressure on governance systems and internal controls, increasing the potential for fraud,
human error, etc. Indicators that relate to the performance of organisational systems, processes
and human resources may also signal a change in operational risk. Including the performance of
the systems, processes and tools that comprise the operational risk management framework.

Examples of operational risk-relevant performance indicators include:

• Metrics on the efficiency of IT systems, including systems availability, security, or recovery
times
• Metrics on the efficiency of operational processes (e.g. machine breakdowns, product faults,
etc)
• Metrics on the reliability and performance of products and services (breakdowns, complaints,
etc)
• Metrics on service centre performance (e.g. time taken to respond to or resolve a service
request)
• Metrics on the performance of outsourcing service providers
• Metrics on the performance of (operational) risk management processes and procedures
2.4 Different metrics have different uses in different contexts
Specific metrics may be used as risk, control or performance indicators in different contexts.
For example, asset growth is a common indicator of business performance, but may also be
an indicator of operational risk. A large increase in asset growth could signal an increase in
operational risk exposures, because of the strain this might put on governance systems and
internal controls. Equally poor IT or customer services performance may increase the potential for
hacking attacks, mis-spelling, customer complaints, etc.

6
2.5 Key indicators
Indicators are not automatically key indicators in every context. An indicator becomes ‘key’ when
it tracks an especially important risk exposure (a key risk), or it does so especially well (a key
indicator in the sense that it is a good proxy for the risk in question).
The IOR’s perspective is that a Key Risk Indicator should typically be treated as an indicator that
is applied to key operational risks. A key operational risk may be one that an organisation has a
high exposure to, which threatens the achievement of organisational objectives, or is outside of
appetite.
Assigning risk, control or performance indicators to key risks will allow an organisation and its
management to monitor these risks and control them effectively. Indicators might also be used to
signal effective internal control (where they are within stated thresholds or limits) and to provide
assurance to the board that these risks are being managed effectively.
In this context, the first step for an organisation implementing KRIs for operational risk should
be to identity its key risks. The results of a group-wide or executive-level risk and control self
assessment (RCSA) process could be used as the basis for this exercise (please refer to the
IOR’s Sound Practice Guidance on RCSAs). An organisation’s key risks should be those that have
the largest inherent and or residual risk exposure scores. These being the risks that represent the
greatest threat to the achievement of an organisation’s objectives.

7
Section 3 - Using indicators
Indicators have a range of uses. First and foremost, they are a business intelligence tool that
provides information to support risk management decisions. In this regard, they can support
risk monitoring and assessment, the implementation of a risk appetite framework and corporate
governance.

Section 3.1 - Risk monitoring

Indicators can be used by organisations to track changes in their exposure to operational risk,
especially when they are forward-looking, leading indicators (see section 4.3). Leading indicators
help detect future changes in probability and or impact. As proxy variables, indicators do not
reveal the actual/precise change in probability or impact, but they can help to signal that a
change is occurring or will occur shortly. Such a signal should then prompt action to investigate
the situation and implement further controls where necessary.

A further benefit of using indicators to monitor operational risks is that collecting and reporting
data on indicators is usually less time consuming and resource-intensive than a full-blown RCSA.
This means that indicators can be used to track changes in inherent risk, control effectiveness
and residual risk between each RCSA update. For example, RCSAs might be updated quarterly,
while indicators might be updated monthly or potentially even weekly, daily, hourly or real-time
(e.g. the monitoring of financial crime is often real-time in larger banks).
Section 3.2 - Implementing risk appetite

As explained in the IOR’s Sound Practice Guidance Paper on Risk Appetite, procedures should
be put in place to ensure that an organisation remains within its chosen appetite and tolerances
for operational risk.
The monitoring of risk, control and performance indicators is a common method used in
organisations to help ensure they remain within their appetite and or tolerance for risk. Thresholds
and limits may be set, which are aligned with the chosen appetite/tolerance. Then when these
thresholds or limits are breached, or trending towards a breach, targeted action can be taken
to address the situation and bring the organisation’s exposure to operational risk back within its
appetite/tolerance.
Section 3.3 - Governance and assurance

Many governance codes, including the UK Governance Code, expect boards to monitor
an organisation’s significant risk exposures and to keep these exposures within appropriate
limits. Boards are also expected to assure themselves that the organisations they govern have
appropriate risk management and internal control systems.

Operational risk, control and performance indicators can be used to help support these
governance responsibilities. High level, organisation-wide indicator reports can be used to help
them monitor significant operational risk exposures (the key risks) and to assure themselves
that the associated controls are effective. Metrics on the performance of the operational risk
management framework may also be used to help assure the board that this framework is
effective.

Section 3.4 - Risk assessment and modelling

Risk and control indicators may be used to support RCSAs and risk modelling where appropriate.

8
About RCSAs a significant rise or fall in particular risk or control indicators should prompt a
review of probability and or impact scores. It may be that risk and control owners decide that
these scores should remain unchanged. But this should be a conscious decision that follows an
appropriate discussion of the situation.

Concerning risk modelling, risk and control indicators may be used as variables in statistical
models. Alternatively, they may be used to help validate these models. Questions should be
asked when there is a significant change in the predictions of a risk model, but there is no
change in the related risk and control indicators or vice-versa. Such a situation may indicate that
either the risk model is imperfect or that the wrong indicators have been identified.

9
Section 4 - The Desirable Features of Operational Risk
Indicators
The selection and use of too many operational risk indicators can be as detrimental as too few.
This is because decision-makers will struggle to take in all of the information (failing to see the
wood for the trees) and because of the costs involved in collecting and reporting the information.
Organisations are advised to establish very specific characteristics for the data they use as
indicators, separating broad data from specific metrics used to indicate changes in exposure
levels (which may include metrics on inherent/gross risk exposure, control effectiveness or risk
management performance).

The characteristics that organisations should consider when selecting effective operational risk
indicators are outlined below.
Section 4.1 - Relevant
Operational risk indicators must provide reliable and accurate information on an organisation’s
operational risk exposures. This should include providing management with information on both
current and future exposures.

Relevance can change over time, as new operational risk exposures emerge, and existing
exposures are either mitigated or modified. Linking periodic reviews of the selected suite of
operational risk indicators with the completion of risk and control self-assessments is an effective
way to maintain relevance. As is drawing on the experience, knowledge and understanding of
risk and control owners to help select the initial set of indicators and to suggest changes, as
necessary (both removals and additions).

The following questions are useful to consider when assessing the relevance of existing
operational risk indicators, or when considering adopting new ones:
• Does the metric help quantify or measure the risk?
• Does the metric help monitor the exposure?
• Does the metric help manage the exposure and its consequences?
Section 4.2 - Measurable
Indicators should be measurable in a consistent and reliable manner. This is to allow the
construction of trends to facilitate comparisons over time. It also enables the use of targets, limits
and thresholds.

This feature requires that indicators should be one of the following:

• Numbers or counts (number of days, employees, etc)
• Monetary values
• Percentages and ratios
• Time durations
• A value from some pre-defined rating set (such as that used by a credit rating agency)
Indicators that are described by text are prone to being subjective, can easily be misinterpreted
and are subject to manipulation through the structure of the text employed. Hence they are not
recommended. Though they are in theory possible.
10
Measurable indicators should reflect the following characteristics:
• They must be quantifiable as either a cardinal (absolute) or ordinal (relative) value. Cardinal
values are ‘real’ numbers that imply that 2 is twice as large numerically than 1 and so on.
Ordinal values simply provide an indication of scale (e.g. we can say that 2 is greater than 1,
but not by how much greater). RCSA probability and impact scores are often ordinal values.
Statistical models provide cardinal values (real probabilities and precise financial impacts)
• Indicators must have values which are precise and not prone to excessive subjectivity
(cardinal values are more precise than ordinal ones)
• Values must be comparable over time
• Indicators should be based on primary source data (e.g. data direct from the original source
and not subject to the interpretation of modification by a third party) and be meaningful
without interpretation to some more subjective measure or benchmark (e.g. a subjective
maturity framework)
Good indicators are those that quickly convey the required message, without the need for
comparison or reference to other information. In this regard, percentages and ratios – presented
against and appropriate benchmark - are typically far more useful than the actual underlying
information.
Section 4.3 - Forward looking (Leading)
The use of so-called ‘lagging’ indicators, which provide information on past events or issues is
not recommended, unless past trends can be relied upon to indicate the future. Rarely is this
the case in the modern world. Much better to use ‘leading’ or ‘preventive’ indicators that provide
management with sufficient lead time to correct the situation before operational risk events
happen.

Critical to this preventative feature of operational risk indicators is the capacity to capture the
causes of the risks rather than counting the number of realised events. This is where loss data
collection (both internal and external) can be invaluable, where data is collected on the causes
of events. Forward-looking scenario analysis can also help identify causes not currently reflected
in the loss data. For more guidance please refer to the IOR’s papers on Loss Data and Scenario
Analysis.

Table 2 compares some examples of leading and lagging indicators

Leading Indicators Lagging Indicators

Pay gap and job satisfaction metrics Staff turnover as a measure of staff morale
to capture the causes of staff
resignations
Metrics on product/service quality Customer complaints (counting the number of unhappy
control customers having already told their friends and
relatives how much they dislike your product or service)
Frequency of policy and procedure Losses from breaches
breaches
Frequency of machine/vehicle Cost of machine/vehicle breakdowns
servicing

Table 2: Comparing leading and lagging indicators

11
Major operational risk events are often the result of a chain of causes and effects. For example,
bad IT, leading to poor information, leading to a wrong decision, leading to poor customer
service, leading to complaints, is an example of one such a cause-and-effect chain. Therefore,
a lagging indicator for one risk can be leading for another: for example, a lagging indicator of
bad IT (such as IT breakdown) can be a leading indicator of customer dissatisfaction; since poor
customer service can be caused by IT disruption. Similarly, consider the number of unresolved
customer complaints – such complaints relate to issues that have already occurred (the lagging
aspect), but which still need to be addressed (the current aspect).

Lagging and current indicators can also have a leading element to them that may need to be
considered. For example, in the case of unresolved customer complaints, an organisation’s
failure to address these could give rise to a costly lawsuit at some point in the future and/or bad
publicity, leading to reduced sales.

Truly leading indicators are rare and are usually related to causal drivers within the business
environment within which the organisation operates – they tend to be measures of the state
of people, process, technology and the market that affects the level of risk in a particular
organisation. A leading or preventive indicator can be something as simple as the number of limit
breaches on market or credit risk exposures, or cash movements, or the average length of delays
in executing particular activities. In themselves, such occurrences may not be loss events in their
own right, but if their value starts to increase this may point to the potential for a higher frequency
or severity of operational loss events.

In addition to causal indicators, indicators of exposure, stress or failure may also provide more
leading information for managers.

Exposure indicators relate to the nature of the business environment and its critical
dependencies. The business environment may be volatile or stable, growing or mature, regulated
or free. Critical dependencies in the external environment include the solvency and conduct of
service providers, suppliers and vendors, or large clients. They may also include external events
that impact on essential systems or staff. Examples events include the spread of a pandemic,
regulatory change or trends in cybercrime.
Exposure indicators are typically one-off, alerting management when change occurs. They do not
necessarily fit a quarterly red-amber-green classic reporting, but they allow an alert to be raised
whenever there is a change in a key stakeholder to the organisation.

Failure indicators typically warn of failures in important operational risk controls or wider
processes procedures and systems. In this regard control indicators are a common source of
failure indicator (see section 2.2).

Stress indicators reflect the stretch in business resources, systems and processes, whether
human, financial or physical. Tiredness is a well-documented cause for accidents, mistakes and
slips, whether in medical services or road safety. Many human resources departments capture
the number of hours of overtime per staff member. In equipment, overloaded IT hardware or
software are likely to lead to downtime or crashes.
Section 4.4 - Easy to collect and monitor
Ease of monitoring means that the value of the data collected for a metric must exceed the costs
of collection: this cost-benefit rule is as applicable to investments in risk reporting and monitoring
(except for mandatory reporting for regulatory purposes), as it is to any other type of business
investment.

12
An indicator’s cost of collection and ease of monitoring should be at the centre of the selection
decision. One way to achieve this is through recycling: reusing what already exists. As explained
above organisations make use of a wide range of metrics to support their operations, these
metrics offer a good starting point for the selection of operational risk indicators.

The use of automated risk indicator systems often provided as part of IT packages for operational
risk assessment and reporting, can further reduce the costs of collection and facilitate easy
monitoring. However, they are only recommended for organisations with mature risk indicator
frameworks. The IOR recommends starting small, selecting a limited set of indicators and
collecting the data manually. This facilitates a good understanding of where the data is coming
from, what it indicates, and how it can be used. Once an indicator or set of indicators have
proven themselves useful, then consider technology solutions to reduce the manual workload, but
in a manner which allows the easy replacement and addition of new indicators.

An important aspect relating to the collection process is quality assurance. The collection cycle
needs to incorporate specific deadlines for submission and should be auditable in terms of data
sources and collection channels. There should also be an independent quality control process to
ensure that erroneous or misleading data is not sent to management.
Section 4.5 - Comparable
Indicators must provide data that is comparable with some form of benchmark. Comparability
allows management to understand the relative ‘scale’ of the indicator. This helps them to
determine when action is required to address the value of the indicator or the risks or controls
that it relates to.

Relevant benchmarks are either over time or across comparable internal departments or
business units and external organisations. An organisation can track its own evolution through
time, provided that the type of indicator and information collected is stable over a long period.
Cross department/unit or external organisational comparisons are also very useful. They provide
a wider context and are not prone to inconsistent historical trends.
Some industries share data in less sensitive areas like staff sickness absence or health and safety
incidents, for example. Where data is shared in this way it should be used as a benchmark. For
example, along with the ‘raw’ metric an organisation’s position relative to the industry distribution
could be provided (4th to 1st quartile). Comparisons between internal departments and units
could be made in the same way and used to help facilitate friendly competition to improve the
value of indicators and by extension the related operational risk exposures.
Section 4.6 - Auditable
Auditable means that the data used to produce a metric is:
• Comprehensive and accurate, and that this remains consistent over time
• Comes from a documented course
• Is constructed using a clear and consistent formula
• Is reported in a clear and timely manner
For good governance, independent validation of the indicator selection process (including how
data is sourced, aggregated and delivered to management) should be undertaken reasonably
early in the lifecycle of the organisation’s risk indicator programme. The organisation’s internal
audit function should normally perform such a validation.

13
Periodically, further quality assurance checks should be made to ensure that indicators remain
relevant and that the data used is timely, accurate and complete. This may also be conducted by
internal audit or the (operational) risk function.

The results of any risk indicator audits should be reported to the operational risk committee or
equivalent, as well as the statutory audit committee.

14
Section 5 - Selecting Indicators and Setting
Thresholds and Limits
As explained in the section above indicators should be selected with care and used effectively.
This section contains guidance on the processes that can be used to select a set of indicators
and for setting appropriate thresholds and limits.
Section 5.1 - Selecting indicators: top down or bottom up?
There are two main approaches that organisations can use to select the indicators they wish to
monitor: top-down or bottom-up. The top-down approach starts with senior management and/
or directors who select the indicators that are to be monitored across the business, while the
bottom-up approach allows business entity/area level managers to select and monitor their own
sets of indicators. In both cases, the aim is to cover the most significant information requirements
that each level of the organisation requires to achieve their objectives.

Neither approach is automatically better than the other; both can, or should, co-exist. A top-down
approach can facilitate aggregation and senior management understanding, while a bottom-up
approach ensures that local managers can select and monitor those indicators that are most
relevant to their particular situation. In practice, many organisations employ a combination of
the two and this is generally considered to be the best approach.The selection process for top-
down indicators could be conducted vertically (by business line) or horizontally (by department)
depending on the organisation structure of the company. Top-down indicators should meet the
following criteria:
• Reflect the operational risk profile of the division, business line, country or region or of the
overall organisation, depending upon the level at which selected
• Facilitate aggregation across relevant business entities, product or service areas, countries
or business lines, resulting in a meaningful and understandable metric at the relevant level of
management
• Should apply to all parts of the organisation structure below the level where they are being
applied
• Are usually imposed by management and must be reported on, without choice
Typically, the selection process for bottom-up indicators should consider:
• The results of Risk Control Self Assessments (RCSA), ensuring that indicators are identified to
facilitate the ongoing monitoring of identified risks and controls
• The results of any regulatory examinations or audit findings to help facilitate the rectification of
any control or monitoring deficiencies that may have been identified
• Identified during the new product review process (mainly short term) to monitor and manage
the operational risk during the implementation phase
• The views of the appropriate risk owners (e.g. the relevant department managers or business
line managers) or that of the local Operational Risk Manager, both during and between RCSA
exercises
• Any insights that may have been provided by recent loss events (for example in terms of the
identification of significant new indicators)
• Changes in the economic environment, which might mean that certain indicators become
more important (e.g. indicators of fraud risk may become more important in a recession, etc)

15
Section 5.2 - Deciding frequency
The frequency of measurement will typically follow the cycle of the activity: from real-time (usually
measured using automated systems) to monthly, quarterly or semi-annually. Frequency of
monitoring (data capture) is not the same as the frequency of reporting (see section 6.5.2).
Section 5.3 - Thresholds and limits
Implementing a set of indicators without any guidelines on how to interpret the data and what
actions are required will not deliver many benefits to the organisation. The organisation needs to
establish, for each relevant indicator being monitored, a set of threshold values or limits where,
if the indicator’s value breaches the threshold or limit, the organisation knows it needs to act.
Equally the establishment of thresholds and limits for specific indicators is an important part of an
effective operational risk appetite framework.

However, the establishment of thresholds and limits in isolation of an informed understanding of

the indicator and its values over at least a minimum period of time is equally likely to deliver little
value. It is strongly recommended that the organisation implement its indicator set, collect data
for 6 months at the very least, but preferably 1 year, then assess the data and its trends over that
time to establish the initial thresholds and limits. If possible, draw upon any publicly available
information or benchmarks to assist in establishing the starting points for an organisation’s
thresholds and limits.

Thresholds may take several forms, including (1) a cap or upper boundary, where as soon as the
indicator value exceeds the threshold value, the escalation process kicks in; (2) a floor or lower
boundary, where as long as the indicator value is above the threshold value, nothing happens,
but when it drops below that level, the escalation process starts; and (3) a collar or combination
of a cap and floor/upper and lower boundary, where essentially the indicator values are expected
to remain within the pre-defined range.

Also, a more sophisticated indicator monitoring programme could include a variety of threshold
types, including percentage based; absolute number or value; deviations from a predefined
value; etc. It should be expected that over a period of time, as the organisation becomes more
risk-aware and the benefits of proactive risk management deliver value, indicator thresholds
should be tightened. This implies that the organisation should periodically review not just the
indicators it is using, but the thresholds applied to those indicators. However, if the thresholds
are too narrow, they will result in false alerts and then over time, people ignoring the alerts
altogether. Too broad, on the other hand, and the organisation learns too late that a major issue
has suddenly emerged, with potentially significant adverse consequences.

To establish the initial threshold values, decide first on whether a cap, floor or collar is required,
then establish whether the threshold is an absolute number or value, a percentage, ratio or other
derived value or some form of deviation or variance. Next, review historical data for the indicator
in question and establish its ranges over time. Assess existing budgets or targets, relevant public
information and the organisation’s risk appetite and apply this information to the historical ranges.
Then, decide where the first level of slight discomfort within the data range lies and use this as
the basis for establishing your first threshold. Monitor the next few data submissions against the
threshold and adjust if necessary.

It is common to set limits and thresholds using a RAG (Red, Amber, Green) approach. Indicators
that are within their amber zone should normally be given greater priority than those that are
green, with even greater priority being given to red indicators.

16
Table 3 illustrates the normal significance and response criteria that are assigned to red, amber
or green indicators. Note that for indicators that are assigned a single limit (indicating zero
tolerance for values above or below this limit) there may be a case to omit the amber threshold
and present such indicators as being either red or green.

Red • The value of this indicator is far too high/low suggesting that the organisation
may be exposed to significant risk.
• Immediate action is required on the part of management to manage the risk(s)
in question
Amber • The value of this indicator is higher/lower than normal suggesting that the
organisation may be exposed to an elevated and potentially significant level of
risk.
• Management attention is required to determine whether action needs to be
taken soon.
Green • The value of the indicator is within normal parameters, suggesting that the
organisation is not exposed to significant risk.
• No action is required – the indicator and its associated risks are under adequate
control.

Table 3: RAG thresholds for operational risk metrics

It must be stressed that as indicators are proxies, the aim is not to manage the indicator, but
rather the associated operational risk exposures. A breach of an indicator is a signal of potential
threats ahead. Getting the indicator back into the amber or green zone does not necessarily
mean that these threats have been averted.

Limits and thresholds should reflect the implementation of the risk appetite statement cascaded
down to the organisation. Please refer to the IOR’s Sound Practice Guidance Paper on
Operational Risk Appetite for more information.
Section 5.3 - Triggers for escalation
Having set one or more thresholds, the final step is to determine the response required when
a threshold has been breached. This is commonly referred to as a trigger condition, which
determines what action is to be taken and by whom.

Where an organisation has implemented an escalating series of thresholds, it is likely that

each threshold will result in some form of triggered notification to increasingly senior levels of
management.

In the same manner, as different boundary thresholds can be applied to different indicators,
different trigger conditions can be established. The most basic is a “touch” trigger, where as soon
as the boundary is reached, the trigger is initiated and alerts generated as appropriate. Other
trigger conditions include “repetitive touch” where when the boundary is first reached nothing
happens but if in the next data submission period the boundary is still in breach, then the alert is
triggered.
As with the associated thresholds and limits, triggers should be linked to an organisation’s
operational risk appetite. Triggers should also be linked to the degree of sophistication required
in the warning system and must consider the resource overhead (people, systems and cost)
necessary to implement more sophisticated structures.

17
Section 6 - Managing and Reporting Risk Indicators
Once an initial set of indicators are determined and thresholds and or limits are determined the
focus shifts to review and reporting. All the initial effort will be wasted without time and resources
devoted to the effective reporting of the selected indicators. Plus, selected risk indicators and
their thresholds/limits should be reviewed on a regular basis to ensure that they remain relevant.
At a minimum, it is recommended that selected indicators and their thresholds/limits should be
reviewed on an annual basis. Though the appropriate frequency will vary depending on the
nature, scale and complexity of an organisation and the dynamism of its operating environment.

Emphasis should also be given to documentation. Documents should be drafted on the following:
• Procedures for adding or changing indicators and their associated thresholds
• Documentation on each selected metric, which notes the data source, any formulae used to
calculate the metric, frequency of collection and the rationale for selecting the indicator and
any assigned thresholds or limits
• Procedures for the reporting of risk indicators
The (operational) risk function is normally responsible for the design, implementation, and
documentation of management processes for operational risk indicators.
Section 6.1 - Adding or changing indicators
Organisations should review their selected risk indicators whenever there are changes in their
key risks (e.g. the addition or removal of key risks) or when the relevance of the chosen indicators
changes.

A procedure for adding or changing operational risk indicators should be put in place which
explains:
• The frequency with which the chosen indicators should be reviewed
• Who has the authority to approve the addition, change or removal of particular risk indicators,
bearing in mind that different individuals may be responsible for different areas of risk
• Whether changes can be made on a top-down and/or bottom-up basis
• When removing a previously monitored indicator, what will happen to the data that has been
collected (will it be retained or deleted?)
• When replacing an existing indicator with another, similar indicator, whether past data should
be recalculated or amended and applied to the new indicator
• The introduction of indicators relating to a new product or new business activities. Including
how long such indicators should be monitored for post-implementation
• The introduction of indicators following recommendations by department manager(s),
regulators and/or auditors (both internal and external)
Section 6.2 - Changing thresholds and limits
Changes to thresholds and limits are a common occurrence, especially for new indicators where,
initially, relatively little data is available (see section 5.3.).

A clearly defined procedure and governance process are required to control the setting and
changing of limit/threshold levels. This procedure should explain who has the authority to make
changes and who may approve these changes.
18
From a bottom, up perspective, local management may be authorised to change thresholds
and limits, with approval provided by the (operational) risk function. From a top-down basis,
the authority will normally rest with senior management, with approval by the board or board
delegated risk committee or equivalent.
Section 6.3 - Taking action to resolve threshold or limit breaches
When an amber or red threshold is exceeded action is required (see section 5.3.). Actions
should be documented and assigned to an owner (e.g. the risk or control owner). In addition,
it is recommended that actions are Specific, Measurable, Actionable, Realistic and Timebound
(SMART) and that progress is reviewed to ensure on-time completion. Further guidance on action
planning is provided in the IOR Sound Practice Guidance Papers on RCSAs and Loss Events.
Section 6.4 - Reporting
There is little point collecting data on operational risk indicators is this is not reported to the
appropriate level of management in a timely and usable fashion. However, organisations that
have just begun to collect data on new operational risk indicators may decide to wait 6 months
to a year before producing reports. This is to ensure that sufficient data is collected to facilitate
trend analysis and the setting of thresholds or limits. Where pre-existing data is used there is no
need to delay the commencement of reporting. This is a further benefit of recycling existing data
into operational risk indicators.
Section 6.4.1 - Reporting to different audiences
Different audiences within an organisation will require different reports on risk indicators. Figure 1
illustrates this according to the management hierarchy.

Figure 1: Levels of operational risk indicator reports

Where possible operational risk indicator reports should be developed in conjunction with the
there intended audience to ensure maximum comprehension and usability. However central
co-ordination can help to ensure that a consistent view of information is delivered so that reports
can be compared across business lines and functions and or aggregated for senior management.
In larger organisations, documented procedures for indicator reporting may also be necessary to
ensure consistency.

19
Some features of a sound indicator report/reporting process include:
• Short – care must be taken to avoid producing overly detailed reports with large numbers of
indicators. Management will not have the time or attention required to process large amounts
of information. One way to achieve this is through exception reporting, only reporting on
indicators that have breached thresholds or limits, or which are trending adversely, indicating
that a future breach is likely;
• Simple – reports should not be overly complex and contain jargon terms, large tables of data
or complex mathematical formulae. Where possible the simplest possible graphs and charts
should be used
• Timeliness – reports should be produced promptly so that they can be acted upon whilst the
data they contain is still relevant
• Accuracy – inaccurate metrics will provide a false picture of an organisations exposure
to operational risk and may mean that it ends up over-exposed or investments too much
reducing certain risks. Processes should be in place to check the accuracy of reported
metrics on an ongoing basis
• Trending – reports should make clear the historical trends of the chosen indicators to provide
some indication of their volatility and or where they may be heading
• Clear escalation procedures – so that the recipients of a report know when to escalate areas of
concern to more senior management
• Compliance – with any regulations that may exist, where appropriate
Section 6.4.2 - Frequency
There is no right answer to the frequency of reporting. It will depend on the nature of the risks,
indicators and environment. Reporting should be linked to the timeliness of decision-making and
action formulation and reports of different frequency will be required to suit specific audiences.
Table 4 outlines some of the more common frequencies that can be used for indicator reporting.

Interval Benefits Suitable For Audience Drawbacks

Daily Potential risk or Dealing with routine Local Too frequent to
control issues issues (e.g. running a call management allow detailed
identified centre or IT function) analysis
immediately
Weekly Track common As above Local May not capture
issues on a regular management the full extent of
basis an ongoing issue
or concern
Monthly Aligns with monthly Monitoring operational Local and senior May not be
management performance management sufficiently timely
committees
Quarterly Aligns with quarterly Monitoring threats to Senior Lacks sufficient
reporting and organisation objectives management detail
many audit/risk
committees Board Not timely
committees

20
Yearly Concurrent with year Reviewing the high-level • Senior • Lacks
end financial results operational risk profile and management sufficient
and reports impact on going concern detail
• Board
committees • Not timely

Table 4: Frequencies for operational risk indicator reports

Section 6.4.3 Data visualisation

An indicator report should be presented in a user-friendly manner with appropriate visual aids
and use clear and simple language. The presentation can be in the form of:
• Country or regional view reports
• Organisation wide reports
• Business-specific reports
• Special theme reports (i.e. focus on a specific control topic e.g. fraud, information security,
etc)
Judgement on the part of the (operational) risk function on what to include or exclude from a
report may be necessary to help report users to reach the right conclusions. However, users and
auditors should be able to access data on all available indicators, on request, so that they can
satisfy themselves that the most appropriate indicators have been presented.

The provision of suitably detailed narrative to support the figures is critical to ensure that
information consumer is able to interpret the reports that they receive and use them to support
decision-making. In particular, brief and relevant commentary should be provided to explain
abnormal items and data trends. Many operational risk indicator reports are presented in a very
simple format. For example:

Figure 2 Example simple risk indicator report

21
More sophisticated data visualisations are also possible and are used in some organisations. This
can include heatmaps, interactive geographical maps, such as the Kaspersky Cyber Threat Real
Time Map, and network diagrams that highlight relationships between different operational risk
indicators.

22
Section 7 - Conclusion
The IOR’s view is that a sound framework for operational risk management should include the
selection, monitoring and reporting of risk, control and performance indicators for key operational
risks. Though the implementation of indicator monitoring and reporting can be time-consuming,
the benefits are considerable. Management is effectively blind without access to the appropriate
risk metrics. It is impossible to drive a car without access to metrics on factors like speed or
temperature. Similarly, management requires metrics to support effective decision making and
to ensure that they steer the organisation away from threats to its strategic and operational
objectives.

Rarely are complex approaches required, however. The focus should be on providing the right
information at the right time and in a format that can be easily understood.

23
 www.theirm.org

Developing risk professionals