Professional Documents
Culture Documents
Quality Assessment Manual Chapter 2
Quality Assessment Manual Chapter 2
Establishing a Quality
Assurance and
Improvement Program
O v e rv i e w
IIA Practice Guide, Quality Assurance and Improvement Program, states that Quality
should be built in to, and not on to, the way the activity conducts its businessthrough
its internal audit methodology, policies and procedures, and human resource practices.
Imbedded in this process of building quality is the concept of value as defined by stakeholders
and the commitment of the internal audit activity to deliver that value to its stakeholders.
The practice guide further states, Delivering quality requires a systematic and disciplined
approach as professionalsquality does not just happen. It is the combination of the right
people, the right systems, and a commitment to excellence. Building an effective Quality
Assurance and Improvement Program (QAIP) is similar to establishing a total quality management program where products and services are analyzed to verify they meet stakeholder
expectations, operations are evaluated to determine their efficiency and effectiveness, and
practices are assessed to confirm their conformance to standards. Maintaining an effective
QAIP also requires leaders who are responsible for setting the proper tone in support of quality.
17
Using key concepts of quality as a foundation in establishing a QAIP, the internal audit
activity should consider:
18
For the internal audit profession, it is important to assure that internal audit activities globally maintain the highest possible standards of service delivery to the organizations they
support. The Institute of Internal Auditors (IIA) established the International Professional
Practices Framework (IPPF) to guide the internal audit profession, and the mandatory guidance portion of the IPPF is the foundation for developing an internal audit activitys QAIP.
>
T h e QAI P F r a m e w o r k
Standard 1300: Quality Assurance and Improvement Program states that the chief audit
executive (CAE) must develop and maintain a QAIP that covers all aspects of the internal
audit activity.
Common elements of all QAIPs include:
A framework is the complete environment for developing and implementing the QAIP. IIA
Practice Guide, Quality Assurance and Improvement Program, has documented a programbased framework consisting of Governance, Professional Practice, and Communication. See
figure 2-1. This framework is intended as guidance only. CAEs may develop their own QAIP
structure in conformance with the Standards.
Continuous Improvement
of IA Processes
Reporting & Follow-up
External Assessment
Periodic
Self-assessment
Ongoing Monitoring
Communication
Professional Practice
Governance
Quality Assessments
Quality Assurance Over
Entire IA Activity
19
organization. IIA Practice Advisory 1310-1: Requirements of the Quality Assurance and
Improvement Program provides more detail.
Internal Assessments
Two key elements of the quality assessment process comprise the internal assessment portion
of the internal audit activitys QAIP: ongoing monitoring and periodic self-assessments.
Ongoing Monitoring
What is important to remember is that a QAIP must be built into the procedures of the
internal audit activity and not onto the way the activity conducts its business. As noted in
IIARF publication Implementing the IPPF, 3rd edition, The most obvious internal method
for continuously assessing quality is management oversight of internal audit work. Adequate
supervision from the beginning through the end of engagements is the most fundamental
element of a quality assurance program
20
As noted in the practice guide Quality Assurance and Improvement Program, the Deming
Cycle (or Plan-Do-Check-Act cycle) provides a possible structure in establishing the QAIP.
Applying the Deming Cycle to the ongoing monitoring portion of the QAIP might look
like figure 2-2 on page 21.
The steps in the Deming Cycle are as follows:
Do is essentially executing the process and collecting data for analysis and
follow-up in the Check and Act steps of the cycle.
Check is the step where actual results are compared to expected outcomes and
differences are analyzed.
Plan
Act
Do
Check
The ongoing monitoring element of the QAIP would primarily address conformance with
the following Standards:
2200: Engagement Planning
2300: Performing the Engagement
2400: Communicating Results
2500: Monitoring Progress
To this end, ongoing monitoring applies to all assurance and consulting assignments and
should achieve objectives described in Standard 2340 that engagements must be properly
21
supervised to ensure objectives are achieved, quality is assured, and staff is developed. The
standard also requires that Appropriate evidence of supervision is documented and retained.
This documentation provides assurance that ongoing monitoring is incorporated into the
routine policies and practices used to manage the internal audit activity. In other words, a
quality review must be performed for each engagement. This review provides an opportunity
for ongoing evaluation, coaching, and feedback for each auditor assigned to the engagement.
Practice Advisory 2340-1 provides guidance for engagement supervision to assure ongoing
monitoring uses processes, tools, and information considered necessary to evaluate conformance with the mandatory guidance of the IPPF. Integrating suggestions from the Practice
Advisory into the ongoing monitoring process provides opportunities for training and
development of staff as well as performance evaluation.
As noted in the practice guide Quality Assurance and Improvement Program, ongoing monitoring mechanisms may include:
22
Any weaknesses or areas for improvement should be addressed as they are identified during
the ongoing monitoring process.
Developing appropriate metrics to measure the performance of audit activities is also
important and these metrics can be reviewed and revised as needed. Results of ongoing
monitoring must be reported to the board or the audit committee at least annually. The
adequacy and effectiveness of the ongoing monitoring portion of the QAIP should also be
evaluated as part of periodic self-assessments described in the next section.
Periodic Self-assessments
Periodic self-assessments are designed to assess conformance with the internal audit charter,
The IIAs Definition of Internal Auditing, Standards, Code of Ethics, regulatory requirements, and to assess the efficiency and effectiveness of internal audit in meeting the needs
of its various stakeholders. The QAIP should document and define a systematic and disciplined approach to the periodic self-assessment process which could use tools provided in
the appendices of this Quality Assessment Manual.
The periodic self-assessment must produce a documented review of conformance with each
area of the mandatory guidance of the IPPF. It should assess the adequacy and effectiveness
of the ongoing monitoring portion of the QAIP and lead to a conclusion regarding whether
the internal audit activity is adding value to the organization.
IIA Practice Guide, Assisting Small Internal Audit Activities in Implementing the
International Standards for the Professional Practice of Internal Auditing, recommends that
the small internal audit activity perform an internal self-assessment at least annually. IIA
Practice Guide, Quality Assurance and Improvement Program, recommends the QAIP be
reviewed annually and sections of the program updated throughout the year as needed. This
annual periodic self-assessment process provides the audit committee and the board assurance that the internal audit activity maintains the standard of performance that is required
by The IIA. Recommendations for improvement should be tracked by a follow-up report,
the results of which are listed at each board meeting.
The periodic self-assessment element of the QAIP would primarily address conformance
with the following Standards in addition to those addressed by ongoing monitoring:
1000: Purpose, Authority, and Responsibility
1100: Independence and Objectivity
1200: Proficiency and Due Professional Care
1300: Quality Assurance and Improvement Program
2000: Managing the Internal Audit Activity
23
Do
Act
24
Check
Metrics
and the
QAI P
As noted in IIA Practice Guide, Quality Assurance and Improvement Program, Ultimately,
the QAIP should measure whether internal audit is meeting its own objectives, as well as those
of the broader organization. IIA Practice Guide, Measuring Internal Audit Effectiveness and
Efficiency, states that Establishing performance measures is critical in determining if an audit
activity is meeting its goals and objectives, consistent with the highest quality practices and
standards. Performance measures are therefore an integral part of the internal audit activitys
QAIP, enabling it to address key quality considerations outlined earlier in this chapter:
IIA Practice Guide, Measuring Internal Audit Effectiveness and Efficiency, outlines a process
to identify key performance measures for activities that stakeholders believe add value and
improve the organizations operations. These measures become a critical part of the internal
audit activitys QAIP in assessing the value-added portion of the IPPF mandatory guidance.
Effectiveness can be defined as doing things right the first time. Analyzing the costs of
nonconformance with operational standards, or doing things wrong the first time, will help
identify appropriate effectiveness metrics. For the internal audit activity, such costs might
include unnecessary audit procedures performed, portions of engagements reworked, audit
objectives not achieved, engagement reports delayed, or key engagement risks overlooked.
Reporting and analyzing performance measures will usually lead to identification of areas
for improvement by the internal audit activity, and these improvement plans become part
of the QAIP. Progress in completing these plans should be reported to key stakeholders at
least annually.
25
A robust QAIP will include metrics that present the internal audit activitys focus on adding
value (stakeholder satisfaction), productivity (effectiveness and efficiency), and quality
(continual improvement).
A s s e s s m e n t , E va l u a t i o n ,
R e po r t i n g
and
26
What rating scale will be used to support a conclusion regarding the QAIP
and the internal audit activitys conformance with the mandatory guidance
of the IPPF?
Answering the first question will depend on the design of the internal audit activitys QAIP
and the level of resources devoted to the internal assessment process. As noted previously,
relevant practice guides recommend annual internal self-assessments; the Standards does not
specifically state a frequency. Some CAEs may view internal self-assessments as action taken
during years when an external assessment is not performed. Certain parts of the QAIP may
be evaluated every year, while other portions may be evaluated less frequently. The Planning
Guides described in appendix A and the Programs described in appendix D can be used to plan
and perform an internal assessment and evaluation of the QAIP and the internal audit activity.
The second question is not specifically addressed in the Standards as it does not prescribe an
assessment scale; however, the Standards does require the degree of conformance with the
mandatory guidance of the IPPF be assessed. Appendix E has an evaluation summary master
framework that contains conformance criteria linked with mandatory guidance of the IIPF,
which CAEs can use to assess the conformance of their internal audit activity and their
QAIP. Appendix E describes an assessment scale, and several assessment scales are also reviewed
in IIA Practice Guide, Quality Assurance and Improvement Program. CAEs may choose an
appropriate assessment scale to perform their internal assessment and evaluation.
This discussion of rating scales leads back to the concept of a maturity model introduced in
chapter 1. Internal audit activities in the early stages of establishing their QAIP could use
a maturity model to help them achieve general conformance with the mandatory guidance
of the IPPFconfirmed by their internal self-assessment process and eventually validated
by an external assessor. Internal audit activities with mature QAIPs, where multiple internal
and external assessments have been completed, could use a maturity model to help them go
beyond conformance with the mandatory guidance of the IPPF to a higher level of quality.
Standard 1320 states, The chief audit executive must communicate the results of the quality
assurance and improvement program to senior management and the board. Therefore,
conclusions arising from the internal assessments of the internal audit activitys conformance
with the mandatory guidance of the IPPF should be provided to key stakeholders such as
described by the standard. As noted earlier, the relevant practice guides suggest this reporting
should occur annually. Failing to report any results of the internal assessment component
ongoing or periodicof the QAIP would indicate lack of conformance with the quality
standards of the IPPF.
Continuous Improvement
IIA Practice Guide, Quality Assurance and Improvement Program, states, The primary
objective of a QAIP is to promote continuous improvement. Adoption of the Deming Cycle
in establishing a QAIP, as suggested in the practice guide, leads to continuous improvement
efforts. Organizations that establish total quality management programs cite several positive
outcomes from their focus on continuous improvement. These benefits can also be realized
by the internal audit activity as they continuously improve their QAIP:
Reduced error rates leading to increased stakeholder satisfaction and recognition of value produced by the internal audit activity
27
Improved internal audit staff morale resulting from a focus on process improvements where all ideas are welcome
Linkage
to
External Assessments
Chapter 2 outlined steps to building an effective QAIP focusing on the internal assessment processongoing monitoring and periodic self-assessment. External assessments are
an element of the QAIP as prescribed by the Standards; however, the Standards only requires
an external assessment to occur at least once every five years. The remaining components
of the QAIP should be continuously active between external assessments, establishing the
foundation of a successful internal audit activity.
The primary link between a QAIP and an external assessment is the reporting process originating from the QAIP. For a QAIP to be deemed effective, CAEs should expect that external
reviewers will confirm what the CAE is measuring through the periodic self-assessment
process and reporting to key stakeholders. The CAEs report of the periodic self-assessment
may be used as the basis for assessment by an external reviewer.
A secondary link from a QAIP and an external assessment is the documentation maintained
by the CAE as evidence of an effective QAIP. This includes charters, policies, procedures,
metrics, audit reports, annual audit plans, engagement workpapers, audit committee
minutes, staff training records, etc. External reviewers will want to examine relevant documentation that describes key elements of the QAIP (see Tool A0: Background Information
and Document Request Checklist in appendix A).
The decision to schedule an external assessment results from the CAEs evaluation of the
internal audit activitys QAIP. Key factors facilitating this evaluation include:
Does the CAE believe that the internal audit activity generally conforms with
the mandatory guidance of the IPPF?
Has feedback from key stakeholders been incorporated into the QAIP?
As noted in Practice Advisory 1312-1, CAEs can choose from two methodologies for
external assessments. The first approach is a full external assessment conducted by a qualified, independent external assessor or assessment team comprised of competent professionals
under the leadership of an experienced professional project manager. The second approach
involves an independent validation of the CAEs self-assessment of the internal audit activity
performed by a qualified, independent external assessor or assessment team. Several factors
may influence the CAEs decision in selecting an appropriate external assessment method to
review the internal audit activitys QAIP.
The manual reviews tools, techniques, and methods used by external assessors to conduct a
full external assessment (see chapter 4) or an independent validation of the CAEs periodic
self-assessment (see chapter 3).
29
O v e rv i e w
Guides
of
R e l e va n t P r a c t i c e
The following IIA Practice Guides provide many recommendations and suggestions that
internal audit activities will find useful in establishing their QAIP.
Published in April 2011, this document specifies a working definition of the term small
internal audit activity. It includes specific examples and leading practices, relevant to the
CAE and audit management of small internal audit activities, on how best to approach
implementation of the Standards. As noted in the guide, the degree of challenge for a small
internal audit activity in establishing a QAIP is high. However, the guide presents specific
suggestions that address conformance with each of the standards for a small internal audit
activity to consider.
The IIA is constantly producing new Practice Guides. Readers of this manual should
check the Guidance section of The IIAs website for relevant guides not listed here as
well as updates to the guides noted above.
31