Putting in Place The Necessary Capacities and Conditions: Section

Putting in Place
the Necessary
Capacities and
Conditions
Section
M
a
n
a
g
i
n
g

f
o
r

I
m
p
a
c
t

i
n

R
u
r
a
l

D
e
v
e
l
o
p
m
e
n
t
A

G
u
i
d
e

f
o
r

P
r
o
j
e
c
t

M
&
E
7
7-2
Table of Contents of Section 7
7.1 An Overview of Putting in Place the Necessary Capacities and Conditions 3
7.1.1 Capacity for People and Their Organisations 3
7.1.2 Paying Attention to Incentives 4
7.1.3 Getting an Optimal Structure for M&E Responsibilities 5
7.1.4 Thinking through the Information System 6
7.1.5 Finances and Resources to Do the Job 6
7.2 Human and Institutional Capacity 8
7.2.1 Essential Capacities for M&E 8
7.2.2 Acquiring the Right People 9
7.2.3 Ensuring Capacity of Good Quality 10
7.2.4 Building Capacity for Participatory M&E 11
7.2.5 Developing an M&E Training Plan 16
7.3 Incentives for M&E 19
7.3.1 Motivating Staff (Project and Implementing Partners) 20
7.3.2 Encouraging Primary Stakeholders 23
7.4 Organising M&E Structures and Responsibilities 24
7.4.1 Ensuring Clarity of M&E Functions and Responsibilities 24
7.4.2 How to Make the Most of Consultants 28
7.5 Organising the Projects System for Managing Information 31
7.5.1 Why an Information System Is Vital 31
7.5.2 Setting Up a Computerised Information System 31
7.6 Finances and Resources to Operate the M&E System 35
7.6.1 Budget Items to Consider 35
7.6.2 Levels of Allocation 37
7.6.3 Costs and Resources for Participatory M&E and Selected M&E Activities 38
Further Reading 40
Building capacity for M&E involves external
and on-the-job training not only for project staff
but also implementing partners and primary stake-
holders. Joint development of M&E is critical for capacity-building.
For people to do a good job, they need to be motivated. So the
right incentives are critical and come in many shapes and forms.
Make the most of inputs of staff, partners, primary stakeholders
and consultants by ensuring everyone is clear about her/his
responsibilities. This helps you avoid gaps and duplicate or
contradictory efforts.
Pay attention to where M&E functions and information flows fit
in the project so that the insights gained can feed into timely
decision making.
People can fulfil their M&E functions better with an appropriate
and flexible information system that provides easy access for data
entry, analysis and retrieval.
All of the above needs to be supported by thoughtful use of the
existing budget.
In all M&E efforts, work towards creating learning processes that
support local development. A decentralised learning process
between stakeholders requires local capacity-building, with local
resources and structures.
This Section is useful for:
Managers to ensure that M&E staff have suffi-
cient resources, capacities and structures to
develop and implement the M&E system;
M&E staff to check that proposed resources,
capacities, support and structures are sufficient
for M&E to be effective;
Consultants to make sure the M&E
component has been designed comprehen-
sively enough and with sufficient resources,
capacities and flexibility and to ensure that
sufficient and appropriate resources, capacities
and support structures exist for carrying out
M&E responsibilities.
Key Messages
7.1 An Overview of Putting in Place the Necessary Capacities
and Conditions
7.1.1 Capacity for People and Their Organisations
When asked why a project M&E system is not working, a common response is poor or
insufficient capacity. Capacity is the ability of individuals and organisations to perform
functions effectively, efficiently and in a sustainable manner
1
. According to most people,
capacity means the human ability knowledge and skills to do a given task.
A most common answer to inadequate capacity is lets send the M&E officer on a training
course where new knowledge can be heard and new skills can be practised. Although a
training course can provide valuable input, every course has limitations. In practice, much
capacity is built on the job through concrete experience.
For an effective M&E system you need skilled people who can, between them, fulfil the M&E
functions and tasks. Key tasks include: designing the general outline of the M&E system (see
Section 3); setting up and operating supportive computerised systems (see 7.5); facilitating
learning in reflective events (see Section 6); and managing the communication of M&E
findings (see Section 6). Meeting capacity needs will require that you:
1. Acquire the right people by:
hiring already trained people;
training your staff (internally or via external courses);
hiring external consultants for focused inputs.
2. Ensure capacity of good quality by:
removing disincentives and introducing incentives for learning;
being clear about what you expect;
keeping track of staff performance through regular evaluations;
outsourcing data verification;
striving for continuity of staff;
finding a highly qualified person to coordinate M&E.
A GUIDE FOR PROJECT M&E SECTION 7
7-3
1
UNDP. 1998. Capacity assessment and development in a systems and strategic management context. Management Development and
Governance Division, technical advisory paper No.3. New York, N.Y.: United Nations Development Program.
3. Build capacity for M&E. Start by developing an M&E training plan for all stakeholders and
with them. This entails agreeing on who is expected to do what and assessing if they have
the necessary skills and conditions. You can undertake training using a combination of these
three options:
external courses;
internal courses, tailor-made for stakeholders and linked to the development of the M&E
plan itself;
on-the-job training/mentoring.
4. Invest in capacity for participatory M&E (PM&E). Work closely with project staff, imple-
menting partner staff and primary stakeholders to identify what is needed to make PM&E
work and to develop plans to fill capacity gaps. When working with consultants on PM&E,
clearly define her/his responsibilities, hire the same consultant(s) to ensure consistency in
approach and build relationships among stakeholders; also, include PM&E in the terms of
reference (TOR) and discuss with each potential candidate how she/he sees PM&E.
7.1.2 Paying Attention to Incentives
Putting in place incentives for M&E means offering stimuli that encourage project managers,
M&E officers and primary stakeholders to perceive the usefulness of M&E, not as a bureau-
cratic task, but as an opportunity to discuss problems openly, reflect critically and criticise
constructively in order to learn what changes are needed to enhance impact. It involves imple-
menting encouragements and removing disincentives.
When thinking about incentives, consider those you can put in place within the boundaries of
the organisation, that is, without rocking the boat, and also those that might require structural
changes to the way the project is organised and operated. Also consider whom they are meant
to stimulate so that they engage with learning-oriented and participatory M&E. This will allow
you to fine-tune incentives for particular groups.
Incentive systems should be equitable, applied in a timely manner, be compatible with the
projects principles and strategies, and be recognised as part of a projects policy. Incentives
need to be context specific and aimed at supporting sustainability of efforts. This is why
financial incentives are undesirable in many contexts, as sustaining them beyond the life of
the project would be unfeasible.
Good incentives for M&E are closely linked to general management efforts to improve overall
project performance. Examples of common incentives include:
clarity of M&E responsibility in job descriptions and work plans;
appropriate salaries and other rewards, such as housing and vehicle use;
support to carry out required project activities, such as making financial and other resources
easily available;
professional development for career advancement.
You do not have as much influence over incentives for implementing partners and primary
stakeholders as you have for project staff. Yet it is crucial for them to be as motivated as project
staff, when it comes to participatory learning-oriented M&E. See 7.3 for more ideas.
7-4
Incentives will change during the life of the project. Keep motivation high by changing incen-
tives. They may vary per project phase as the M&E tasks and issues change and, in some cases,
actions taken early on may prove to be incentives at a later stage. For example, in Ghana,
potential applicants for project positions had to go through an intense selection process. On
top of this, ministry staff that applied also had to be nominated by the head of their depart-
ment. Because they knew they had passed a tough recruitment process, project staff, including
those in M&E, held a high respect for each others professional skills and abilities.
7.1.3 Getting an Optimal Structure for M&E Responsibilities
Getting the basic structure for a projects M&E functions and responsibilities right can avoid
major communication bottlenecks, conflicts of power and interest, forgotten or duplicated
tasks, and wasted efforts. This saves resources and headaches. Organising responsibilities
means considering the most appropriate contribution for project staff, partner organisation
staff and primary stakeholders and how to link these.
M&E is part of every single persons job, from the messenger to the project director.
Monitoring is a daily and spontaneous activity. Yet it is important that M&E functions also
have a clear position in the project structure, whether among project staff, with partners or
among primary stakeholders. High visibility and clear positions of authority for those with
M&E responsibilities can help link information to its use in decision-making.
To ensure clarity of M&E functions and tasks:
define the M&E responsibilities of implementing partners and primary stakeholders;
consider what staffing levels are appropriate for the set of M&E tasks and functions you need
to fulfil;
allocate clear levels of authority to M&E-related staff;
ensure overlap between project management and M&E;
use detailed job descriptions for each staff member to coordinate inputs.
All projects use consultants in some form, local or foreign, short-term or long-term, extending
big responsibilities or small tasks. Ensure that:
you are using them strategically for M&E development in ways that build local capacity and
build on existing M&E forms;
when contracting them, you are completely clear about what you expect them to add to the
existing systems and expertise, by when and in what manner (particularly vis--vis primary
stakeholders) they are expected to work;
you are working with as much continuity of consultants as possible to minimise the need
to reconcile conflicting advice.
7-5
7.1.4 Thinking through the Information System
In IFAD-supported projects, the quantity of information that is collected and needs to be
shared justifies well thought-out information systems that store data and make data accessible
to others. This is also vital for a participatory process. Documentation provides the foundation
for interactive communication, transparency, consensus-building and continuity.
Storage of two types of information is needed impact-related information to guide the
project strategy and progress-related information to track operations. To store this range of
information, from survey data to copies of contracts and correspondence, will probably
require different information storage systems.
Computers can make a critical contribution to tracking and using data but are no panacea.
Achieving impact certainly does not depend on computerising data. Information that needs to
be shared can also be photocopied and circulated, with each recipient using a common filing
system.
To set up a computerised information system, follow these steps:
1. Define what you want to store in the information system and for what purpose.
2. Define your basic network structure by analysing how, when and by whom the database will
be used (see Figure 7-1 as an example of a network structure).
3. Identify how you plan to process the information, who will do it and what forms this will
require.
4. Compare options for software and hardware (the network) and decide whether to invest in
existing software or contract a specialist for tailor-made software.
5. With your preferred option in mind, undertake a more focused data management analysis.
6. Establish the formats needed for database entry.
7. Provide user training on the system, otherwise it might never get used optimally.
8. Adjust the system regularly by evaluating its use with the users.
7.1.5 Finances and Resources to Do the Job
Solid and systematic learning costs money. Financial resources are needed for the time people
spend, for supporting information management systems, training, transport, and so forth.
Key items to include in the budget are:
contracts for consultants/external expertise (fees and travel expenses);
physical non-contractual investment costs;
recurrent labour costs (permanent staff salaries, temporary support staff);
focused labour inputs, such as technical assistance, TA (short or long term, national or inter-
national);
training and study tours for M&E-related capacity-building;
non-labour operational costs (e.g., stationery, meetings, allowances for primary stakeholders
and project implementers, and external data such as maps).
7-6
While there are no fixed rules for this, M&E budgets range from 2% to 15% of all costs. In
projects where stakeholders are exploring new ways of working with partners, M&E budgets
are likely to be proportionally higher as more time is needed to reflect on what works. Note
that each project clusters its M&E costs differently, according to the adopted approach.
Irrespective of how the M&E budget is calculated, it will always overlap to some degree with
other project activities. Therefore, do not excessively detail the M&E budget. Much learning
occurs through the normal interactions of project implementation. What is most important is
to budget for the events, procedures and staff time that support project learning and reflec-
tion.
Participatory learning processes are more time intensive than those in which only a few
people are involved. More time is needed to organise meetings with larger numbers of people
and more diverse groups and to reach agreement on how to proceed with M&E or on what
data mean. Consider these budget items for participatory M&E:
specific training for staff in participatory techniques and participatory M&E;
extra meetings with stakeholders for designing M&E;
additional meetings for local-level analysis;
short training workshops on key steps in designing M&E and specific elements such as indi-
cators and methods (including using the logframe matrix).
7-7
7.2 Human and Institutional Capacity
7.2.1 Essential Capacities for M&E
Two sets of skills are critical for M&E to be effective: that for dealing with diverse data and
that for dealing with diverse people. But many more skills come into play. Table 7-1 lists key
areas of M&E knowledge and skills to which each project should have access. As M&E officer
or project director, you can use this as a checklist to know if you have the right mix of skills
and understanding available among the project stakeholders.
Depending on the projects size and resources, functions can be combined or be assigned to
implementing partners, sub-contractors, primary stakeholders or project staff. No matter who
has the competency, they must be available to the project and not duplicate efforts and not
work in opposite directions (see Box 7-1).
Table 7-1. Capacities needed among project stakeholders for good M&E
Box 7-1. Eliminating duplication of efforts
7-8
Knowledge/Skill Needed Possible Responsibillity
Overview perspective of project M&E system (basic procedures
and core communication flows) to make integrated design
possible
Project director, and M&E staff and managers (from project or
implementing partners)
Good understanding of gender, participation and poverty issues
to ensure focused and appropriate M&E feedback to donors
Sector specialists or project component coordinators
Understanding of how to develop a joint learning system in a
participatory manner
M&E unit staff or consultants with experience in
participatory M&E
Ideas for learning methods M&E staff or field staff, with help from external consultant
Facilitation skills for reflective sessions Everyone who is responsible for some aspect of joint analysis
(from community-based to project team)
Understanding of field tools to collect data Data collectors, particularly field agents but also managers who
do relationship/staff monitoring
Ability to check data quality External organisation, maybe cooperating institution
Ability to aggregate and statistically analyse data Statistician or economist
Ability to assess implications of data for each project component Managers of project components (which could be implementing
partners) and primary stakeholders
Understanding of local conditions, changes and impacts Primary stakeholders
Independent opinions on project impact Externally contracted evaluators
In an Indonesian project, fieldworkers collect much data about local credit groups. But so does the bank. Key data on savings and loans is
collected from farmers groups by fieldworkers for the project and by the groups themselves for the bank. Once a farmers groups
request for a loan is approved, the bank starts keeping a computerised record of the groups loan and savings. The group also keeps its
own hand-written records on savings and loans to ensure transparency among the members. The groups information is collected each
month from all 55,000 members. This is processed, typed and aggregated before being sent via the sub-districts and ending up at the
ministry in Jakarta. Meanwhile, computer printouts from the bank with the same data also reach the ministry every month. While both
farmers groups and the bank need to monitor the credit process for accountability reasons, duplication of efforts could be reduced if
farmers were to use the banks records to check against their own and if the ministry were to accept the banks records as sufficient
evidence of farmer credit operations.
7.2.2 Acquiring the Right People
As project director or coordinator of an M&E unit, you have three options to ensure you have
enough of the right kind of capacity on hand.
1. Hire already trained people. This is ideal but very difficult for most projects to achieve. Few
people are skilled in conventional M&E, let alone the type of participatory learning process-
es encouraged in this Guide, which asks for people to be creative, conceptually clear and
good communicators. Hiring already trained people means being able to provide enough
appropriate incentives to make and keep the job attractive (see 7.3).
2. Train the people you need. Training, on-the-job or through external courses, will always be
necessary. Even the most trained M&E professional will need to upgrade skills and under-
standing. Primary stakeholders will need capacity-building to undertake their own M&E and
contribute to the projects. Field staff will require continual skill building as information
needs in a project shift and new methods of data collection and analysis are required.
3. Hire external consultants for focused inputs. Consultants are a common source of M&E exper-
tise. This is a relatively expensive option and does not contribute as much to local capacity-
building but it is often the only alternative when local expertise is not available and time is
short. Consultants are particularly necessary at project start-up when staff may not yet be
hired and relationships with implementing partners are weak. The initial workload involves
establishing management and M&E processes, staff recruitment, finalising the AWPB, defin-
ing reporting procedures, setting up information systems, and providing on-the-job train-
ing. To make the most of consultants requires clarity about their expected contribution
(see 7.4).
Your personnel strategy will be a mix of these three options. If, as project director, you want to
create a learning environment, then you are likely to:
try to hire the best possible person to guide M&E efforts or seek such a person from among
the implementing partners to take on that responsibility;
seek focused inputs by consultants on specific issues where time and/or skills are lacking, for
example, to develop a participatory M&E approach to stimulate self-evaluation among pri-
mary stakeholder groups;
draw up a plan of ongoing M&E training for all stakeholders contributing to M&E.
7-9
7.2.3 Ensuring Capacity of Good Quality
Capacity is not only a question of sufficient numbers. Good quality is fundamental. Being
able to recognise good quality M&E will help keep your learning efforts on track. Box 7-2
shows what good quality M&E meant for one project in India.
Box 7-2. Do you know how to recognise good M&E?
Remember that what is essential for one level of staff, with its specific responsibilities, may not
be necessary at another level. For example, while you expect extension staff to be excellent
communicators with primary stakeholders, the project statistician must be excellent at
working with numbers. However, in projects where extension staff input field data, they must
have several qualities.
Recognising good quality is one thing, but what can you do to ensure good quality M&E
among the project and implementing partner staff? One Tanzanian project attempted this by
changing its staff selection procedure to advertise positions outside the government, rather
than only within. This offered more chance of finding someone with the right qualifications.
In some Latin American countries, contestants for a management job in an IFAD-supported
job participate in a training and selection process that includes a two to three day workshop.
There they are observed discussing issues, how they deal with groups, make decisions, etc. The
selection panel includes government staff, primary stakeholders, and experts. Section 7.3 deals
with incentives as a way to ensure good quality.
Below are other suggestions for ensuring good quality.
1. Remove disincentives and introduce incentives for learning. Encourage project and partner organ-
isation staff to be curious and open about learning by providing a range of incentives. Limit
those disincentives that may keep them from sharing their mistakes and learning from
them. See 7.3 for more on incentives.
2. Be clear about what you expect. Clarify the standards of the M&E capacity you expect and put
procedures in place that make sure these standards are reached and maintained. Staff job
descriptions (see Annex E) and performance reviews are key mechanisms.
3. Keep track of staff performance through regular evaluations. In most project documents, staff
performance is assessed by the degree to which they implement project activities. But also
assess what they have been learning from primary stakeholders and colleagues, what learn-
ing innovations they are initiating and how they are using any information they have col-
lected.
7-10
Feedback on M&E from one project in India reported, The project has gone into intensive and continuous training of staff for M&E and
the managers were quite satisfied with the quality and timeliness of reports. There are two agricultural officers dedicated specifically to
M&E who have been with the project almost since inception and appeared to be very well versed in the basic principles and practices of M
& E. They analysethe monthly reports regularly, provide feedback for action to appropriate authorities and issues are taken up to the highest
level if necessary.
Quote sections in italics are criteria for good M&E capacity. In this case, good quality M&E staff are those who can:
command a good understanding of the project context and stakeholders information needs;
understand basic principles and practices of M&E;
analyse data regularly;
provide action-oriented feedback to the correct level in the organisation;
deliver required reports that are up to good quality standards and on time;
raise critical issues based on M&E findings at the highest possible level.
4. Outsourcing verification of data. Any project can benefit from an external view on what is
happening. One way to do this is to sub-contract an organisation periodically to check the
quality of data and of data use. This will give you confidence in the methods being used to
gather data and can provide additional guidance to project stakeholders on data quality. But
avoid making it a policing exercise.
5. Ensuring continuity of staff. Continuity of staff is both very valuable and very difficult. By lim-
iting the number of people who come and go during the project life, you can build a more
consistent and less fragmented body of experience. A large part of keeping staff is offering
the right incentives.
6. Find a highly qualified person to coordinate M&E. This can give them higher status vis--vis the
rest of the team and makes it more likely that you have the kind of capacity you need at that
level.
7.2.4 Building Capacity for Participatory M&E
Few appraisal reports detail how primary stakeholders should engage with M&E. Therefore, for
projects working with participatory M&E (PM&E) there is also little clarity on the capacities
needed by staff, by primary stakeholders and by consultants. For many projects that are still
getting to grips with basic M&E, the idea of undertaking PM&E may appear overwhelming.
While it does require careful thought (see Box 7-3), many small changes can be made that
contribute to more interactive forms of learning.
Box 7-3. When and where to start with capacity-building for participatory M&E
Getting to Grips with the I mplications of PM&E
Making M&E a learning process that extends beyond your project team means making M&E
participatory. You will need to think through the many implications of including primary
stakeholders actively in reflections on progress and impact, as well as in data collection,
analysis, selection and updating of indicators, etc. (see Table 7-2).
For PM&E to be worthwhile, stakeholders staff, implementing partners and primary stake-
holders must be able to participate meaningfully. This means that project and partner-organi-
sation staff need skills in participatory facilitation techniques plus an appreciation of the
importance of seeking other peoples views. Staff need to be committed to making participa-
tion happen. Good PM&E also means that primary stakeholders must have the conditions and
understanding to make a significant contribution. To do PM&E well will inevitably require
capacity-building for everyone.
7-11
When the ADIP project in Bangladesh started discussing PM&E, the project did not know how to undertake it. The implementing
partners also were unclear on how to proceed with PM&E. While some partners were implementing elements of participatory moni-
toring, they had not been selected for their experience with M&E nor PM&E. The project was unable to provide the necessary guidance
as it had no policy or strategy on participation and did not possess the necessary experience, capacity or financial resources. Project
management has always relied on external consultants and so had no internal skills. Local government departments were in the same
situation. To rectify the situation, the project sought several months of an M&E consultants input on participatory impact monitoring.
Project managers also needed training. For both elements, the project sought extra funds not provided for in the budget. The M&E
consultant was hired and then trained project stakeholders in PM&E. The consultant also worked with project management on how to
use PM&E for better management and increased impact. The lessons learned were: (1) plan for PM&E in project design, (2) budget for
PM&E, (3) select partners for their PM&E skills, and (4) implement PM&E training programmes early on.
Table 7-2. How participatory M&E differs from conventional M&E
2
Primary stakeholders can participate in diverse forms and to various degrees of intensity in
project M&E (see 2.7). The project team will need to decide with the implementing partners
and intended beneficiaries what level of participation is feasible and appropriate. This forms
the basis for understanding what capacities are needed. The following questions need to be
considered:
1. Why are we seeking primary stakeholders active involvement in M&E? What do we expect
will happen? What benefits do we expect for them and for the project (see Box 7-4)?
2. To make the process inclusive, whose participation is vital in M&E remembering that a pri-
mary stakeholder group is far from homogenous?
3. What role should each stakeholder group ideally play? Do we see a role for primary stake-
holders, for example, only in checking indicators suggested by field staff or also in co-
designing the learning process from indicators to methods to feedback?
4. What obstacles do primary stakeholders and staff experience in involving primary stake-
holders?
5. What capacities do we lack to make PM&E happen?
The answers will give you ideas about whose capacities will need strengthening in which way.
For example, if representatives from primary stakeholders are to be involved in designing a
participatory impact assessment, then they will need to be trained in the idea of impact
assessment and will need to understand interviewing skills, the notion of indicators, and
various types of analysis. Project staff will need to have the capacity either to facilitate such
training for group representatives or to facilitate a hands-on learning process with them in the
field.
Box 7-4. Degrees of local participation in M&E require capacity
7-12
Facet of M&E Participatory M&E Conventional M&E
Who plans and manages
the process
Primary stakeholders, project staff, managers and other stake-
holders, often helped by a facilitator
Senior managers and outside experts
Role of primary stake-
holders
Designing and adapting the methodology, collecting and
analysing data, sharing findings, identifying lessons learned and
linking them to action
Information provision only
How success is measured Internally-defined indicators, including more qualitative judge-
ments and stories of personal change
Externally defined, mainly quantita-
tive indicators
Approach Indicative and adaptive Predetermined and fixed
Do you want to involve local women and men in:
Defining what is meant by impact, M&E and learning?
Designing purpose, process and methods for M&E?
Defining themes to monitor/evaluate?
Defining indicators?
Giving their opinion of project history and the changes in the context?
Giving their views on the degree to which project objectives have been met?
Helping analyse and draw conclusions from the data/results?
Sharing feedback with the primary stakeholders?
Presenting and communicating the findings?
If so, what capacities will they and you need?
2
Adapted from Guijt, I. and Gaventa, J.1998. Participatory Monitoring and Evaluation: Learning from Change.
IDS Policy Briefing 12. Brighton: IDS.
Capacities for Staff (Project and I mplementing Partner)
Once everyone has agreed on how primary stakeholders will participate in M&E, then together
you can build a clearer picture of the capacities that project and partner staff will need. A
capacity-building programme for staff of the project and partner organisations must deal with
three issues.
1. Knowing why primary stakeholder views matter. This discussion on why local views on progress
and impact matter requires an understanding of the importance of citizen participation, not
just as an instrument for the project, but as an empowering activity itself that strengthens
local self-reliant development. Where implementing partner staff are reluctant or hesitant
to engage in more participatory forms of M&E such as an annual project review with pri-
mary stakeholders project staff will need additional skills to promote the idea and offer
training. As project director or M&E coordinator, you will also need the skills to negotiate
participatory forms as part of the partnership, in addition to understanding the issues deeply
enough to argue the case effectively.
2. Building the facilitation skills to make it happen. The art of facilitation needs to be mastered,
particularly by those who will interact most with primary stakeholders. This means under-
standing and practising techniques but also having the skills to design jointly rather than
impose ideas of what should happen (see Box 7-5).
3. Being committed to seeing it through. No matter how experienced someone is with participa-
tory data gathering or analysis, if he or she does not have an attitude of genuine interest and
humbleness, then results will be compromised. Designing and implementing PM&E requires
a self-critical look by all staff at their own attitudes and behaviours vis--vis primary stake-
holders. This may require training. For the WUPAP programme in Nepal, M&E training was
formulated as follows: Based on the agreed design of the system, CBOs, PNGOs and pro-
gramme staff will be trained in applying appreciative inquiry, reflection, focus group dis-
cussions, mapping, self-assessment and similar tools and techniques to strengthen their
capacity to analyse, learn and act. This training will help them establish two-way commu-
nication and learning dialogue within and between them and other service providers.
Box 7-5. Negotiations in PM&E on socially sensitive comparisons
3
7-13
In an M&E design workshop in Brazil, local farmers, NGO staff, farmer union representatives and university academics were deciding
which method could assess the impact on milk production of a local mineral livestock salt. Discussion sought ways to compare the use to
allow for more reliable analysis of impacts. The academics and some NGO staff wanted to compare milk production between cows fed
with and without the mineral salt. However, all farmers who feed the salt to their cows are convinced of its merits. For a comparative
study, farmers who were not involved or interested in the mineral salt would have to be included. However, the farmers at the workshop
who would be doing the data collection, collation and analysis were reluctant to include such farmers. They said it would be too difficult
socially to discuss the non-use of salt with their neighbours. Without the comparison, the indicator milk production was no longer felt
to be feasible and another indicator and method were selected.
3
Abbot, J. and Guijt, I. 1998. Changing Views on Change: Participatory Approaches to Monitoring the Environment.
SARL Programme Discussion Paper No. 2. London: IIED.
Capacities for Primary Stakeholders
Giving primary stakeholders the opportunity to participate does not necessarily mean they
will be able to use this opportunity. Building their capacity to participate is critical. Building
local capacity will often mean simply going through each step of the M&E process with them.
In this way, developing the M&E system and training primary stakeholders happens hand-in-
hand. Some focused training sessions may be possible and useful but always in combination
with the actual design of the M&E system, otherwise it becomes theoretical knowledge. Each
new step can begin with a short training type session to encourage a meaningful input.
For example, if primary stakeholders are to be involved in selecting indicators, then a session
on what an indicator is, its use and the advantages and disadvantages of various examples of
indicators will be needed. One IFAD-supported project found inconsistency and lack of clarity
in indicators chosen by primary stakeholders. Also all indicators were considered equal, so that
a simpler more quantitative indicator, such as regular meetings of the general assembly, was
given the same weight as a more complex qualitative indicator of organisational development,
such as ownership and management of project infrastructure. A session with the primary
stakeholders on how to select good-quality indicators might have helped.
PM&E with primary stakeholders can also be built into the overall participatory approach of
the project and might not need a specific focus on building their M&E capacities (see Box 7-6).
Building capacity for PM&E can contribute to building overall capacity and vice versa and, at
the same time, it encourages project ownership and success.
Box 7-6. Incorporating M&E into a Moroccan projects participatory approach
7-14
To build capacity for pastoralist involvement in a Moroccan project, cooperatives were created as independent project partners that
could continue with activities after the project would finish. These cooperatives received financial and technical support. The structure
was simple. Pastoralists made up a general assembly and elected a president, secretary and treasurer. There was also at least one paid
staff member: an administrator responsible for the cooperatives office duties. Membership was open to everyone, including people
without livestock and several cooperative presidents were from among the poorest. Many members struggled with basic organisa-
tional issues at the community level, such as accurate accountability and efficient communication, but they gradually became a strong
force. Building internal evaluation processes strengthened the groups.
One project team member started a self-evaluation process with the cooperatives administrative staff to discuss issues they faced. At
these meetings, a facilitator assisted staff on issues they were unclear about, such as designing monitoring forms. The meetings were a
chance for staff to analyse specific problems and offer solutions. Meetings became more regular and extra ones were called for specific
needs or pressing problems. This self-evaluation process led to practical management changes early on in the project. These were:
purchasing computers to assist in financial accounting, training the cooperatives administrative staff in bookkeeping and local laws
governing cooperatives, and improving staff contracts.
Another internal evaluation process is the projects system of classifying cooperatives progress. The provincial agricultural department
had set key objectives and key indicators related to professionalism in the cooperatives. Each cooperative was scored, according to
these indicators, at large meetings in the presence of cooperative members. Members were able to give input in the scoring of other
cooperatives, which encouraged inter-cooperative competition and motivation to improve performance. This also stimulated commu-
nication between cooperatives, project staff and the government department.
7-15
Working with Consultants or Sub-Contractors
Consultants are commonly hired for their capacity to develop computerised databases,
identify useful indicators or establish information needs for operational management. Less
common, but on the increase, is hiring consultants for their capacity in participatory M&E or
sub-contracting this work. This raises questions. For example, an IFAD-supported project on
community development, FODESA in Mali, is sub-contracting its work for annual participatory
evaluations. What must it include in the terms of reference (TOR) to ensure that the work is
high quality? What role will project staff need to play to check on quality? How can the
consultant ensure that the local annual evaluations complement the existing project-driven
M&E procedures?
How to work well with consultants is discussed in 7.4. Here, the focus is on three issues that
are particularly critical for PM&E: focusing consultant input, ensuring high quality work and
integrating consultant outputs with conventional M&E.
By its very nature, involving primary stakeholders will require ongoing testing of methods and
processes and adjusting of M&E plans. So one limitation of hiring consultants for PM&E is
that they are usually only available for short periods, not continually during project life. To
make the most of consultants:
Make clear what she/he will be responsible for and what will be the responsibility of project and
partner staff. Most consultants will only have time to develop detailed ideas and to test them
out with staff and primary stakeholders before handing over the refining to the M&E unit.
Hire the same consultant for the series of inputs needed in developing the participatory work.
The more you work with different consultants, with their different perspectives on participa-
tion, the more time project staff will need to invest in understanding and integrating the
different outputs, and the more often the M&E direction will change.
Screen CVs of potential consultants and seek recommendations from people whose work with
local communities you respect before you decide whom to hire. But remember that only by
seeing a consultant in action, can you know for sure how good she/he is.
Include in the terms of referencehow you expect her/him to work with PM&E (see Annex E)
expected concepts/approaches and timeframes, field trials of methods with primary stake-
holders to establish feasibility and relevance, etc. Also request that the consultants recommen-
dations on participatory forms of M&E take into account feasibility within the budget and
considering the projects staffing resources, and that any recommendations be clearly linked to
the rest of the project M&E system.
Discuss how the potential candidate sees PM&E. Ask her/him to define and explain M, E and
PM&E and how she/he views the link between M&E for accountability and M&E for
learning. Get agreement on this before making the choice. If there is too much difference
between their perspectives and the rest of the projects M&E, then ensuring a good fit will be
difficult.
7.2.5 Developing an M&E Training Plan
Assessing Training Needs
A training plan emerges by comparing the needs for certain skills with existing capacities and
then outlining steps to fill the gap. An M&E training plan should consider two basic skill sets:
skills to facilitatethe design of the M&E plan and those to implement the plan.
Clearly, your first priority will be to get a plan in place. This might require a first round of
training key stakeholders.
Once the basic M&E plan is in place, only then does it make sense to analyse training needs
with more precision, as only then you will know the type of M&E and responsibilities
involved. This step can be quite detailed. For each level of the objective hierarchy, you will
have specified the type of information you require and the data-gathering methods. For each
of these, you must check whether the right people have the right skills.
Remember that this includes staff from the project and implementing partners and also
primary stakeholders. Just as it is unlikely that you will have a fully trained team at project
start-up, implementing partners are also likely to need and ask for some form of training. It is
also quite likely that there will be considerable differences among the partners, all of which
will contribute to the projects M&E system. Provide for in-house training of key stakeholders
on fundamental aspects of M&E. But dont forget that jointly developing the M&E system will
give a huge boost to M&E capacity.
The gaps you identify will form the basis for a training plan. Table 7-3 shows the elements to
include in a training plan. Table 7-4 shows an initial training plan of a project in Nepal. In a
project in Zimbabwe, the M&E training plan was merged with project management training
needs, due to the large degree of overlap in skills and audience. Three levels of staff were to be
trained in separate workshops: (1) senior management; (2) middle level staff; and (3) field staff
below district level and community workers. Each workshop had an audience-specific training
content and duration. Additional training was planned to include evaluation skills for senior
staff; participatory analysis for field staff; and for all staff, problem solving and conflict resolu-
tion.
Table 7-3. Elements to include in an M&E training plan
7-16
Identified Skill Gap Who Should Have the Skill
(Person and Organisation)
When It Is Needed
(Month/Project Year)
Most Appropriate Training
Option (Cost/Benefit/Time)
Table 7-4. Indicative training plan without analysis of timing, from WUPAP, Nepal
(acronyms relate to project stakeholders)
Assessing Training Options
Once you know what skills are needed, you can decide what the most appropriate training
option would be. You have three training options to fill the skill gap. Your training plan will
probably include all three options in some form.
1. External courses. The most common courses are those provided in Europe or North America.
Increasingly, however, regionally based courses are available. The advantages of external
courses are the exchange with participants from other countries, the more focused/intensive
use of time, the lower cost to send staff to an existing course rather than design your own
training event, and the credibility overseas accreditation can add to a projects M&E and
incentives. The disadvantages are that external courses are not focused around project
needs, the project M&E system cannot be created as one output of the training course,
course material may not always be relevant, and a one-off training is never enough to build
sustained understanding.
2. Internal courses. You can hire a consultant to deliver a training course for any of the stake-
holder groups. The advantages are that internal courses: are relatively cost-effective if larger
numbers of staff are trained, target project needs and skill levels, serve to build a common
language and provide understanding for all involved. Also, if the course is interactive, then
you jointly develop elements of the M&E system en route. Disadvantages include: highly
dependent on the quality of the consultant, more expensive and more time consuming if
the training is integrated with the actual development of the M&E plan.
3. On-the-job training/mentoring. Most learning occurs through informal sharing of problems
and solutions. You can formalise this approach to capacity-building by allocating time for
key M&E stakeholders to consult with external M&E mentors. This can work as an incentive
for staff, offers the option of timely and problem-focused advice, and allows for ideas that
are fine-tuned to the project M&E system. However, finding these mentors may be difficult
and they are unlikely to have ideas for all queries. Making such mentors available to prima-
ry stakeholders is equally important for their capacity development but may be even more
difficult to organise. An alternative is to encourage and arrange for staff and stakeholders to
interact with other projects that are more advanced in PM&E processes. The exchange-visit
7-17
Designation Institution/Person Content of Training
Three-day training seminar by a resource
person
PM, TLs, project M&EO, district M&EOs,
PNGOs, SSMs, SMs
PRA, appreciative inquiry, focus group
meetings, socio-economic and poverty
profiles, facilitation
Network administration, database
management
Project M&EO One- to two-week training in a computer
college at Kathmandu
Database management, data collection District M&EOs, PNGOs, SSMs, SMs Two-day training workshop by project
M&EO
Participatory self-assessment, reflective
sessions, facilitation, data collection and
reporting
CBOs chairmen and secretaries One-day training workshop at district level
by SSMs, SMs
Visits to NGO/CBO project, SAPAP, MEDEP
for hands-on experience
Direct observation of the M&E products
and processes
One-day training workshop on sustain-
able-livelihoods approach to M&E by a
resource person
Sustainable livelihoods approach to M&E
approach can be particularly successful for training primary stakeholders. Some on-the-job
strategies can be simple. A project manager in Indonesia maintained a strict procedure of
monitoring at monthly intervals as well as quarterly and annually. He used this as a means
to train staff in systematic and standardised implementation. As soon as he was confident
in their capacity, he would reduce the monitoring frequency to quarterly and half-yearly
intervals, being aware that it did not really make sense to monitor at such high frequency.
The final training plan for M&E capacity-building may and should overlap with project
management and development training needs. Box 7-7 below provides an example of a
capacity-building and institutional-support training plan. Such targets can be part of the
project logframe matrix. In Nepal, the WUPAP logframe matrix includes a one-page summary
of the M&E component as part of project management. It clearly outlines what is expected
from the project in terms of M&E structures and quality.
Box 7-7. Capacity-building and institutional support component, output targets of the SFPDP project, Malawi
(M&E components are italicised)
7-18
National level capacity-building
Staff obtaining MSc level
Graduates with technical diploma
In-service technical courses
Extension staff upgrading
Institutional support
Training for transformation courses (this is a specific
approach to facilitating leadership and empowerment)
Training for government department members
Staff training in accounts
Government department management training courses
External review reports completion
Completion of stakeholders workshop
Completion report production
Project specific capacity-building
Production of participatory approaches manual
Production of training materials on scheme design and imple-
mentation
Production of farmers organisation manual
Production of general scheme organisation and management
manual
Production of specific scheme organisation and management
manuals
Staff PRA training courses held
Staff training course in farmer organisation
Staff training course in scheme design
Staff training course in scheme organisation and management
Government department staff computer training course
Farmer training in scheme organisation and management
National study tours for farmers/staff
PRA training of trainers course
AWPB and M&E courses
M&E study tours to neighbouring countries
4
Lusthaus et al. 1999. See Further Reading.
7.3. Incentives for M&E
Putting in place incentives for M&E means offering stimuli that encourage project managers,
M&E officers and primary stakeholders to perceive the usefulness of M&E not as a bureaucratic
task but as an opportunity to discuss problems openly, reflect critically and criticise construc-
tively in order to learn what changes are needed to enhance impact. Giving incentives
involves implementing encouragements and removing disincentives. Changing incentives
touches the very heart of project culture and norms.
Not all stakeholders will automatically be motivated to, for example, learn new facilitation
skills or have the patience to develop a joint learning process. Getting the factors right that
motivate project stakeholders will enable them to experiment with changes. Basic motiva-
tional factors,
4
such as the following, relate to good management (see Section 2):
Everyone accepts and shares the mission statement.
The organisational culture and institutional atmosphere encourages good performance.
The history and traditions of the project and related organisations are respected and known
for a focus on learning and improvement.
There is a supportive, fair and inclusive leadership and management style.
There is an attractive incentive and reward system, including performance-related incentive
plans.
There is a shared idea of promoting teamwork towards organisational goals.
When thinking about incentives, consider those you can put in place within the boundaries of
the organisation that is, without rocking the boat and those that might require structural
changes to the way the project is organised and operated.
Examples of structure-conforming incentives for learning-oriented project M&E are:
prize for most innovative fieldworker of the month/year;
study trips and training courses for staff;
avoidance of staff retrenchment.
Structure-challenging incentives can include:
review of project directors performance by project staff;
downward accountability, that is reporting to primary stakeholders on progress with
implementation and including them in decision making on strategic planning;
citizens jury about project strategy (see Section 8).
Incentive systems should be equitable, applied in a timely manner, compatible with the
projects principles and strategies, and recognised as part of a projects policy. Incentives need
to be context specific (see Box 7-8). Rewards should not be so great as to be unfeasible in the
long term or to create disharmony either internal or external to the project. This has been
happening, for example, in Andhra Pradesh (India), where a project found it difficult to get a
good M&E team together because of the stiff competition between development programmes
for the limited number of trained personnel. Generally there was frequent movement of
government and NGO staff due to frequent transfers, low salaries and difficult work condi-
tions. Project officers lamented that another project stole their community organisers by
offering a higher salary.
7-19
Box 7-8. The region and the project-specific nature of incentives
7.3.1 Motivating Staff (Project and Implementing Partners)
The impact of not considering staff incentives sufficiently can be far reaching. Box 7-9 shows
two contrasting situations in which incentives made a noticeable difference to the workplace
and to M&E.
Box 7-9. Incentives for a positive working environment in Ghana
Good incentives for M&E are closely linked to general management efforts to improve overall
project performance. For example, project staff in India felt that one reason M&E had been
able to reorient the project strategy was that the primary stakeholder groups who best adopted
skills were recognised and publicised in the project newsletter. This public recognition gave
M&E staff a positive image. In another example, a grading of primary stakeholder groups had
been carried out by project management and presented back to the groups as a self-assessment
exercise. The criteria for grading had been progressively refined based on feedback from the
groups and other implementing partners. The desire by groups for upgrading acted as a
7-20
In Morocco, government salaries and other benefits such as housing and holidays are already relatively good, so project staff generally
attach less value to additional personal remuneration for M&E. They are motivated by having the right equipment and support, such as
funds to hire enumerators, fuel for vehicles and essential equipment and supplies such as computers for local offices and paper for
surveys.
In Yemen, the M&E post was held by the same person since the post had been created several years earlier. This contrasted sharply with
the high staff turnover elsewhere. The M&E unit received good support and recognition from the project director and was included in
the decision-making processes. Further incentives included the pooled use of vehicles, external training on M&E and performance-related
salaries. Even though the unit had inadequate resources to undertake all planned activities, this did not detract from job satisfaction.
Team members in a project in Ghana worked well and without much competitiveness between each other. For example, there was a lack
of possessiveness about budgetary allocations. One outcome of the good working environment was that district staff were more willing
to drop by the project offices, which increased informal information exchange and helped in activities such as the annual planning
process. Reasons for the positive work environment were identified as follows:
Most staff at headquarters and in the districts had been seconded from the same ministry. Many team members knew each other
even before that, having attended the same schools and universities. Thus friendships were old.
The project technical content was focusing on specific staple crops, which had previously received little or no attention by the agricul-
tural development sector. Project staff were aware of the crops critical importance to the survival of many poor people in the country.
Thus, staff felt they were really contributing to improving livelihoods.
Salaries were higher than regular ministry staff, also giving additional allowances for travel and fieldwork. Even with the longer work
hours and overtime required, this was very motivating.
Professional training skills were offered to staff, thus making them more marketable once the project came to an end.
The strict job selection procedure lent prestige to anyone working for the project.
In an IFAD-supported project elsewhere in Africa, the M&E unit had three managers during the first four years of project life. The first
manager helped design the M&E system but left before implementation started. The second left after four months. Both left due to more
attractive incentives being offered elsewhere. The current manager, promoted from a deputy position, described his own dissatisfaction,
We get no recognition by management and little support or resources. Salaries were low, the local M&E manager received $100 per
month compared to the $2,500 for technical assistance. The only incentives were provision of a house and use of an old car. The M&E
manager explained, The qualifications and experience of the 18 staff in my department are high all headquarters staff and district
heads are university graduates, with several specialised in programme evaluation. Most of my staff have long experience in extension and
M&E. They have a very high level of work frustration. This situation has solutions. While salaries and mobility are the responsibility of
local government, financing of specific tasks are project decisions. The project could provide several interesting non-monetary incentives
(see Tables 7-5 and 7-6).
powerful incentive for improved performance, including their M&E activities. After more
feedback and participatory discussions, a grading system was then also developed for the
implementing partners, which were local NGOs. This development was welcomed by the
NGOs themselves.
Tables 7-5 and 7-6 are checklists of incentives and disincentives to help you assess whether
you have done everything possible to establish motivating conditions. Sometimes very simple
incentives can be effective. For example, in projects that are moving from a control-oriented
to a learning-oriented M&E system, providing training to staff and other stakeholders is proof
that they are trusted and are being encouraging to participate more freely in M&E. By
investing in staff, the transition of project style becomes real. Sometimes very simple disincen-
tives are in place that can obstruct learning. For example, in China, project M&E runs parallel
to a state system of data collection that monitors province and district performance. As
decisions at provincial and district level are made using the state data, there is little incentive
for project staff to assess their own data critically.
Table 7-5. Checklist for staff incentives that encourage learning-oriented, participatory M&E
Table 7-6. Checklist for staff disincentives that hinder learning-oriented, participatory M&E
7-21
Have you got in place the following incentives?
Clarity of M&E responsibility: clear job descriptions, work plans, partner contracts
Financial and other physical rewards: appropriate salaries and other rewards, such as housing and vehicle use
Activity support: support, such as financial and other resources, for carrying out required project activities
Professional development for career advancement: training/external-learning opportunities, attending congresses to listen to and
present M&E experiences, incorporating M&E experience into post-graduate studies/thesis
Personnel and partner strategy: hiring staff who have an open attitude to learning, signing on partners who are willing to try out
more participatory forms of M&E
Recognition: listening to staff and acting on their recommendations, publicly recognising staff via competitions on best M&E prac-
titioner or encouraging staff to present M&E experiences in public
Project culture: compliments and encouragement for those who ask questions and innovate, giving relatively high status to M&E
among staff
Professional support groups: encouraging and funding staff to attend regional professional meetings of, for example, PREVAL in
Latin America or the African Evaluation Association
Performance appraisal processes: equal focus on staff capacity to learn and innovate, rather than only on if they have reached their
quantitative targets
Showing the use of M&E data: making the data explicit and interesting by displaying them on public boards and in newsletters
Feedback: telling data collectors, information providers and others involved in the process how their data was used (analysed),
what it contributed to improve the project
Have you removed the following disincentives from your project?
Using the M&E unit as the place to park demoted or unqualified staff
Not making clear how data will be or was used
Confused or incomplete terms of reference for staff on M&E
Repeated complaints to staff about their incompetence in M&E
Chastising those who innovate within the project boundaries or those who make mistakes
Focusing performance appraisals only on activities undertaken
Salaries that are low and not paid on time
Frequent rotating of seconded staff to different posts
Staff feeling isolated or helpless in terms of their contribution being recognised towards achieving the project goal
Unconstructive attitudes towards what constitutes participation and/or towards the primary stakeholder groups
Staff evaluation is particularly important. Performance-related incentives are generally consid-
ered important, but only if they go beyond quantitative achievements of targets. In several
projects, the rigid monitoring of target achievement rates has encouraged the false reporting
of achievements. In such cases, the supposed performance-enhancing incentive becomes an
incentive for dishonesty and a disincentive for critical learning. Box 7-10 shows how staff
appraisals can work as helpful reflections.
Box 7-10. Staff performance appraisal in KAEMP, Tanzania
I ncentives with Seconded Staff and I mplementing Partners
Some of the (dis-)incentives in Tables 7-5 and 7-6 are also relevant for seconded project staff
and for partner organisation staff. But staff who are seconded from the government are already
working within a specific incentive system, as are those working with implementing partner
organisations. The project will not usually have the capacity to offer many incentives to those
outside its direct authority but it can take into account external but related incentive struc-
tures when designing its own incentive system.
The projects influence over incentives for implementing partners is often much reduced. Yet it
is crucial for implementing partners to be as motivated for participatory learning-oriented
M&E as project staff. To work smoothly with partners, here are some ideas.
1. Negotiate what the expectations are regarding M&E styles and responsibilities in contracts
with implementing partners to avoid problems at a later stage (see Box 7-11).
Box 7-11. Clarifying expectations midway in Mali
2. If partner organisations hesitate about PM&E, then it might be possible to split the M&E
tasks. Project staff can facilitate the more participatory part while they fulfil the less inter-
active M&E tasks. Simultaneously, however, you can organise events to raise awareness
about PM&E and identify obstacles with the partners.
7-22
In the past, the staff appraisal system of the government of Tanzania was confidential. The staff members were appraised by their supervi-
sors and this appraisal was put in their personnel file without their knowledge of what was said or why. Now the government has shifted
to a new system of appraising staff that is based on annual work plans. This new system is not in place yet in every department it
depends on the department heads commitment to the new system. The new staff appraisal process is as follows:
The employee fills in a form to assess her/his performance (government performance appraisal form).
After filling in the form, there is a conference between the employee and her/his supervisor. The supervisor may challenge some of the
assessments made by the employee and ask her/him to support them.
The form is then forwarded to the appropriate authority such as the district executive director (DED), who is the head of administra-
tion in the district. If there is a promotion or disciplinary action to be considered, then it is acted upon here.
Mr. Kiberanga, the district M&E officer, has been through the annual process twice already. He has found it to be very helpful, as he is
able to get feedback on his work and learn more about his strengths and weaknesses. It is a new method, but it is like a mirror. You can
understand how people perceive you.
In PDR-San, a Mali project into its fifth year, the project director must agree on the extent of M&E responsibilities with a critical imple-
menting partner. Responsibilities need urgently to be renegotiated as the partner is only reporting with tables of raw data. The partner
does not consider itself responsible for providing accompanying analysis or explanations. Yet the project director rightly says that he or
his team cannot be expected to interpret the data, as they are not in the field implementing the activities. When questioned, the director
of the implementing partner says that M&E is the same as statistics, whereas the project director sees M&E as including interpretation
of data. The different expectations of what constitutes M&E are caused by ambiguity in the start-up documents and by non-participatory
planning in the early phase of the project.
3. Provide opportunities for joint training events which all field staff or all managers attend,
irrespective of whether they are directly contracted, seconded or come from implementing
partners.
4. Create and maintain a positive spirit of collaboration. An M&E consultant remarked that
many projects she had visited held a condescending attitude toward implementing partners.
This attitude negatively affected partner motivation and transparency of reporting, and so
the overall project monitoring.
7.3.2 Encouraging Primary Stakeholders
For projects which have not yet developed a fuller participatory approach, a first step in
encouraging wider participation is to understand the opportunity costs for primary stake-
holders to engage with project M&E (see Table 7-7). People can only be expected to invest
valuable time when the returns are of value to them.
Discuss with primary stakeholders what they will forgo when you ask them to participate in
project M&E. Although their main benefit is in the form of an improved project, other
benefits are needed if they are to sustain the effort you are asking of them. Basic financial
compensation must be discussed although it is not always appropriate. Consider paying for
meals during M&E sessions and reimbursing accommodation and transport costs. When
primary stakeholders start to take on a key role in M&E, additional compensation must be
considered as might be the case of someone sitting on a steering committee or acting as a
local community monitor to such an extent that they become a de facto project team member
and lose time on their primary livelihoods.
Table 7-7. Primary stakeholders opportunity costs for getting involved in participatory evaluation
5
Public recognition of primary stakeholders input is important to sustain interest. Best of all, of
course, is when primary stakeholders see that their voices have made a difference to the
project. This can be ensured by regular feedback from the project management on progress
and processes and particularly on developments and impacts at policy and institutional levels.
Feedback of results works when the project has been operating for sufficient time to see
results. For this reason, it is critical to have the primary stakeholders fully behind what the
project is trying to achieve, and to share the vision and possibilities that might (or might not)
occur. In a Peruvian project, monthly and annual presentations are given by field workers to
community organisations, which has brought the community closer to the project and
reduced the earlier hierarchical and paternalistic tendencies.
7-23
How Primary Stakeholders May Spend Their Time for the Project What They May Lose as a Result
Contributing to the management and implementation of the
project by:
Interacting in interviews, focus groups, committee meetings,
workshops/seminars and community meetings
Analysing data, opinions and problems during mapping,
transect walks, computerised data analysis, etc.
Engaging in project-related cultural activities such as theatre,
dance, storytelling and video/radio productions
Travelling to and from meetings, making study tours in other
communities
Waiting for other local people, outsiders, decisions and funds
Time for their other activities, which may include productive,
reproductive, developmental or political
Cash contributions or extra spending on food, transportation
and accommodation
Social position in relation to their non-project friends, peer
group or rival groups
Satisfaction with their situation and lifestyle, at home, work
etc.
5
Adapted from Jackson, E. 1999. "The Strategic Choices of Stakeholders: Examining the Front-End Costs and Downstream Benefits of
Participatory Evaluation". Presented to the World Bank Conference on Evaluation and Poverty Reduction. Washington, D.C.: World Bank.
Many projects aim to influence government processes and policies, from the national to the
village level. Primary stakeholders can be very motivated by working with structures and
mechanisms that enable their voices to be heard at policy levels, knowing that it might be
making a difference. There are many examples where projects have influenced national and
regional policies. For example, in Nepal, recommendations from project evaluation workshops
were passed to the government that then officially endorsed, in one case, that stunting among
children should be instituted as an important indicator of project impact with respect to
gender equity. In the same country, government regulations on forestry were amended to take
into account leasehold forestry groups that had previously been sidelined. In India, policy
changes negotiated during a microfinance project enabled direct lending to implementing
NGO banks.
In another case in India, villagers turned around a form of corruption in village councils. It is
normal practice in India to use village land records as testimonies of titles and possessions of a
piece of land. It is also known that certain local village functionaries would expect a small
present (bribe) for getting a copy. Members of one womens self-help group in an IFAD-
supported project were not spared from this practice. Most of the groups had to pay up.
However, they started recording in the minutes of their meetings the names of the official
who had asked for the bribe and the amount taken. These minutes were then circulated, as
normal. When the other village functionaries found out about the forced bribery, they imme-
diately returned the money.
7.4 Organising M&E Structures and Responsibilities
7.4.1 Ensuring Clarity of M&E Functions and Responsibilities
Should there be an M&E unit? If so, should it fit into the project structure? If so, where? If
not, where will the M&E functions be housed? Who is answerable to whom in terms of
project impact, progress, lessons learned and problems? Appraisal reports often define the
higher level of M&E functions and responsibilities by stipulating how the project will be
governed in terms of steering, coordination and management committees. But many other
details of the M&E functions need to be considered at start-up.
A central coordination or management unit services some projects. For instance, it might
monitor the physical and financial performance of the project and project parties or liase
with central ministries and agencies and IFAD. Other projects have no centralised M&E unit
but instead share M&E tasks among the implementing partners and with primary stakeholder
organisations.
Box 7-12 provides a detailed example of one projects strategy for its M&E functions and
responsibilities. Box 7-13 provides three contrasting alternative structures. None of these is
intended to serve as a model. They highlight two critical aspects:
considering where to locate the M&E functions in a project structure (with primary stake-
holders and implementing partners);
being clear about M&E responsibilities.
7-24
Box 7-12. Functions and responsibilities of the M&E unit in the ADIP project, Bangladesh
7-25
The major functions of the M&E system and unit in the ADIP project have been defined as follows:
Develop monitoring instruments and revise/modify these after field-testing.
Develop guidelines and provide training to concerned staff of the project implementing agencies.
Collect and record data by project component.
Process and analyse data to provide information for reviews and reports.
Prepare the annual work plan and budget of components/activities.
Prepare and submit reports, both routine and special.
Organise formal and informal discussions, meetings, workshops for reviewing and implementation for reflection.
Establish a feedback loop by providing and receiving feedback to and from all stakeholders concerned, and follow up.
To implement the functions, the following broad strategy was devised:
on-site monitoring by NGO and M&E staff specifically, field visits with feedback and follow-up and collection of on-farm and field
trials data which are then verified by the department agricultural extension and feedback to primary stakeholders by extension staff;
off-site monitoring by the M&E staff and NGOs, reporting progress regularly and compiling reports manually and with computer;
special surveys and evaluation studies initiated and undertaken by internal and external project actors;
reviews, through formal and informal discussions and workshops at all levels.
This project has three clearly defined lines of monitoring: (1) conducted and managed by the NGOs themselves for their own use; (2)
jointly implemented by NGOs and project staff and meant for all concerned including project management; and (3) conducted exclu-
sively by project staff for extension, research and other project activities. The NGOs perform M&E activities for their own needs, specifi-
cally M&E on credit operations, maintenance of group discipline and quality of groups. To maintain quality control, the project staff
particularly those from the M&E unit regularly visit the NGOs and their groups to assess performance and provide feedback.
Project implementation reports and AWPBs specify in detail the ongoing responsibilities and tasks of each staff member in contributing to
the overall output of the M&E system.
Due to concerns from IFAD and the cooperating institution about M&E staffing, the M&E unit has been expanded and is now staffed
with 19 professionals. There are also four statistical assistants at the district level and one research officer at headquarters whose inputs
are sometimes used for M&E purposes. The staffing arrangement in terms of distribution between government staff and technical assis-
tance is as follows:
Designation Staff Position Government Technical Assistance
Headquarters - Senior M&E officer 1
- Research officer 1
- M&E specialist 1
District level - M&E officer 4
- Field monitoring officer 4
Local level - Monitoring associate 8
This new staffing arrangement brings with it major challenges in the form of bringing everyone on board to a common understanding of
their roles and responsibilities, and also in ensuring that capacity is improved in terms of quality as well as quantity.
Box 7-13. Examples of M&E structures
Finding the Right Place for M&E Functions
It cannot be overemphasised that M&E is part of every single persons job, from the messenger
to the project director. The director of the MARENASS project in Peru said, We did not have a
formal M&E unit for the first three years. This did not stop us from monitoring and evaluating
our work. Everyone contributed, even the office driver had an M&E function. Everyone, in
their own way, keeps track of the operational and sometimes the more strategic aspects of
their work whether there is enough petrol in the tank or whether the team is working well
and impacts are emerging. Monitoring is a daily and spontaneous activity. Decentralisation to
encourage ownership of learning processes further emphasises this.
Nevertheless, experience shows that the location of those responsible for M&E is critical for
performance. Linking it directly to project management helps ensure that M&E findings are
used to inform decisions. This can be interpreted in different ways. The M&E unit of a
Guatemalan project was planned as part of the executive board that supported project
management, while in China the M&E unit was located within the responsible ministry. Try
to make sure that M&E functions are represented at a high strategic and resource management
level, and that they are also incorporated into the approaches and activities of all project
implementers.
Simply giving M&E a high visual status in the project is not enough, as was found by one
IFADsupported project in Africa. It had been decided that to improve M&E, the M&E unit
would be elevated to the status of a project department, similar to a field department. Despite
this, the M&E coordinator explained that M&E still:
made a limited contribution to major decisions in discussions on technical issues, while
management decisions were made without M&E participation;
made a limited contribution to monitoring field implementation, as this was undertaken by
technical assistance experts directly under the project manager;
had insufficient resources allocated to carry out its functions, in particular, inadequate bud-
get support, transport and computer support.
Changes would be required in terms of attitude and in terms of organisational structures and
clarity of responsibilities.
7-26
One project set up the M&E section as a cell within the project management unit. This cell was staffed by a monitoring officer and
one statistical assistant, with occasional support from hired local consultants. Meanwhile, data collection was the responsibility of
project implementation units present in each district where the project was operating. Each project implementation unit was headed
by a project officer responsible to the project coordinator and was provided at least one computer for M&E. The information collected
in the districts was thus computerised and fed back to the projects central M&E unit.
The M&E unit of a 67-village income-generating project was staffed and run entirely by two officers. Decentralised management units,
which were supported by NGOs, collected and analysed the primary data. Rather than dealing directly with the project components,
the M&E officer communicated through the project director.
Another M&E unit was located in the same organisational unit as project management. It had one highly qualified M&E manager,
strongly supported by a chief technical advisor and assisted in the field by staff of other departments. M&E-related staff reported to
the project director and the various heads of departments and were included in all project meetings. Reports by technical staff were
also sent to the M&E unit, which sent them on to the project director. Field visits were made on a monthly basis, and all reporting
was distributed to non-beneficiary stakeholders. At the local level, a project coordination committee, headed by the district commis-
sioner and comprising of local stakeholders, met with other district committees on a monthly basis when special requests from parlia-
ment or from project departments were aired.
Setting up M&E-conducive organisational structures should have been addressed at project
formulation (see Section 3). Responsibilities and power can be set up in such a way as to
hamper or encourage learning among project stakeholders (see Box 7-14). They are often inad-
equately addressed but also affected by changes in the operating context that can require
negotiations about organisational structures with responsible departments, IFAD or cooper-
ating institutions, and implementing partners.
Box 7-14. When responsibilities and structures obstruct good M&E: Yemen, Morocco and Colombia
Clarity of M&E Responsibilities
Clarity of M&E functions and tasks (see Figure 4-2, Section 4 and Annex E) is essential. In a
Colombian project, where monitoring had been separated from evaluation, confusion arose.
Evaluation was the responsibility of the evaluation unit. Although monitoring staff did not
have evaluation in its mandate, it still needed evaluation information for its own reporting.
The evaluation unit had only undertaken extremely limited field visits since project start-up,
so the monitoring staff ended up having to take on some of the evaluation activities. Keep
monitoring and evaluation tasks and functions connected, rather than splitting them among
different people as is common particularly when evaluation is sub-contracted.
Several key lessons have emerged from project experiences in terms of ensuring clarity of
functions and tasks.
1. Clarify the M&E responsibilities of implementing partners. In Colombia, a project reallocated
the M&E responsibility of reporting on impacts to project management. It had been in the
hands of an implementing partner but this had not worked well. The project had learned an
important lesson: always arbitrate services after making it clear in the agreement the com-
mitments and responsibilities of each party and, above all, stipulate the information that
should be reported in terms of effects and impacts, establishing concrete and clear mecha-
nisms to guarantee this.
2. Clarify the M&E responsibilities of primary stakeholders. Clarity of functions with primary stake-
holders can be laid down in memoranda of understanding and also in contracts. Although
7-27
The M&E functions of a Yemeni project were carried out by the M&E department of a government agency responsible for M&E in
several projects, using national guidelines. This agency had much experience and was able to commence project M&E activities at an
early stage. However, the agency did not have direct access to the projects M&E resources and had limited funds. Obtaining authori-
sation for activities and resources was a lengthy procedure, having to pass through a hierarchy of project management and govern-
ment staff. Furthermore, as the relationship was sometimes tense, this affected M&E budgeting, incentives and adoption of M&E
recommendations by the project. Project staff would often check data themselves without involving the agency, especially for informa-
tion critical for management decisions. Being that the project M&E system had been based on the existing government system, it had
limited relevance for the project. Also, the government agency did not prioritise M&E for this project as they also had several other
projects to attend to. Overall, this organisational structure was hindering effective M&E.
Two projects in Morocco were built directly into the provincial-level department of agriculture and hence shared many of the same
staff, including the person responsible for monitoring agricultural activities for the entire province. While incorporating project activi-
ties within government structures may encourage sustainability, staff found that both projects had to compete with ongoing govern-
ment work and political pressures. This resulted in a conflict between ensuring good management of the project and maintaining the
normal work of the department.
The model of project execution in a Colombian project appraisal report would have led to scarce interaction and knowledge of field
issues by the project management. Information was to be sent from the implementing partner to the project in the form of a general
progress report, the project itself having no direct contact with the partner NGOs developing the fieldwork or with the rural micro-
enterprises of the primary stakeholders. Furthermore, the appraisal report stipulated that the project pay for an external evaluation
unit to undertake evaluation functions. There was a danger that this would lead to duplicating activities, with the project M&E officer
assuming monitoring functions and the implementing partner guiding the evaluation.
community members may start with a simple monitoring role, other roles, such as stake-
holder responsibility for managing contractors (see Box 2-20, Section 2), can increase effi-
ciency and effectiveness in implementation, and local ownership.
3. Consider what staffing levels are appropriate for the set of M&E tasks and functions that you need
to fulfil. M&E performance is affected by staffing that is top-heavy, too light or distributed
in a way that obstructs good communication and coordination. The M&E system for a very
extensive income-generating project in Benin was designed with as light an overall structure
as possible. One person was responsible for synthesising all periodic reports, checking with
the field, entering basic data, dealing with multiple requests from project coordinators and
so on. Much monitoring was carried out on field activities and impacts, but this was too
much for one person to analyse, and there was insufficient time for partner performance
assessments and impact studies. The project director described the situation as being like a
city having only one point of contact to the outside world, only to be cut off every time a
problem comes up on the bridge.
4. Allocate clear levels of authority to M&E-related staff. M&E-related staff need sufficient recog-
nition to undertake functions that others in the project may perceive as intrusive. For exam-
ple, if one person needs to coordinate progress reports, then she/he must have the publicly
given status to ensure that other staff communicate the necessary data. Where M&E func-
tions are decentralised and shared among different organisations, clear lines of communi-
cation and authority are equally pertinent but may require more negotiation.
5. Ensure overlap between project management and M&E. This will encourage the much-needed
interchange to guarantee that M&E findings inform decisions. For example, in the RADP
project in Yemen, the M&E coordinator would act as interim project director in the latters
absence. Linking M&E responsibilities and activities with technical project departments also
increases the overall capacity for learning.
6. Job descriptions for each staff member are crucial management tools. They help clarify expecta-
tions. See Box 7-15 for items to consider when elaborating job descriptions
(also see Annex E).
Box 7-15. M&E items to consider when drafting job descriptions
7.4.2. How to Make the Most of Consultants
All projects contract consultants at some point in some form local or foreign, short-term or
long-term, extending large responsibilities or small tasks.
Strategic Use of Consultants for M&E Development
Hiring a consultant for a particular part of the M&E process usually means either you need
extra expertise to cover for project staffs lack of time or you need specific expertise or a partic-
ular view (e.g., on component activities or methodology) that project or partner staff may not
have.
Consultants can be hired at any point of the project life. They can be hired as one-offs or
hired for a longer-term input over several years. Working with consultants has both advan-
7-28
Which M&E responsibilities does the position include?
What is the responsibility: design, data gathering, data inputting, reporting, facilitating, use of M&E data in decision-making?
What type of issues will the position need to track progress with activities, quality of process, etc.?
What are the minimal reporting requirements you need from the position (e.g., raw data, analysed data, lessons learned, actions to
be undertaken)?
What quality standard do you expect the position to fulfil (e.g., timeliness or type, product/service)?
What are the deadlines for the products/services?
tages and disadvantages (see Table 7-8). With these in mind, consider well what you would
like them to contribute, particularly in terms of building project capacity. Project staff in
Benin saw that review missions were making very positive inputs through their systematic
analysis and identification of pressing issues. The staff realised that the M&E system itself
should be producing such an output but it had only ever posted generalists without such
capacity to coordinate M&E work. To increase internal professionalism, they took to hiring
experienced consultants for specialist expertise.
Table 7-8. Advantages and disadvantages of consultants
Clarity When Contracting Consultants
A project in Benin contracted a consultant to design its M&E system but did not include
follow-up in the contract. The consultants TOR stopped at design. He helped review initial
indicators and ideas for the baseline survey, the project team was not yet sufficiently settled to
learn optimally from his input. As the consultant was not involved during the crucial phase of
setting up the M&E system he had designed, additional technical know-how on data collec-
tion and analysis was lost. Clarity is vital: first in terms of the projects needs and then in the
terms of reference about the expected contribution to M&E. Consider the items below.
1. What contribution do you expect them to make, what specific outputs should they deliver,
and what are the limits of responsibilities? Team leader of external evaluation, database
designer, developer of participatory annual review process, etc.
2. When do you expect them to deliver the first draft and the final report? All in one intense
period or in a series of shorter inputs? Before start-up or after?
3. To whomis the consultant responsible and with whomis she/he expected to work?
4. How is the consultant expected to work (alone, with project staff, in a more or less partici-
patory style, minimum number and types of stakeholders to consult, etc.)? Will there be an
opportunity for staff/primary stakeholders to respond to the consultants report?
5. Who owns the final report? Who will use it and how?
6. How much is this going to cost, also in terms of other support that the consultant may need
from the project (e.g., transport or translation)? This will enable you to assess whether you
are getting your moneys worth.
7-29
Advantages of Working with Consultants Disadvantages of Working with Consultants
Give quicker output (perhaps).
Able to stand back from the project and ask questions
that staff arent able to see.
Bring interesting ideas from other projects.
Can raise sensitive issues that project staff fear raising.
Can provide on-the-job training if they work closely
with project staff.
May be trusted by funding agencies for the impar-
tiality of their views.
Can increase the professional level of M&E.
Can provide focused inputs that are only needed
once.
Dependency on outside expertise diverts budget and attention away
from investing in local staff. (This is reduced if working with local consul-
tants.)
If working with different consultants, then project staff may need to
reconcile contradictory or different views and ideas on how to tackle M&E.
If not well recruited, they may only provide a piece of the missing
expertise.
If they are not committed long term to the project, then this may make
their suggestions unfeasible and fit poorly in the project.
If they do not have local experience, then advice or ideas may be inap-
propriate.
If they work in isolation, then local capacity wont be built.
If not living locally, then they cannot help adjust and adapt their recom-
mendations as they are being implemented.
The project coordinator may not know the exact areas of technical expertise needed, so may
find it difficult to write out the terms of reference. Asking the opinions of those who will be
directly affected by the consultants input is good to avoid miscommunication about purpose
and expected outputs. This will also help ensure a close fit between the gap in staff skills that
the consultant is expected to fill and the needs of staff (project or implementing partner).
Annex E provides some examples of TORs for consultants input.
Reconciling Conflicting Advice
The importance of continuity of perspective for M&E should not be under-estimated. The
more different consultants have a say in project M&E, the more problematic this may become,
as each consultant has a theory of M&E, a set of methods, different priorities, etc. The more
these are mixed, the more confusing this can be for a project. Many projects have faced the
situation of working with different consultants over the years due to an inability of the
consultant to commit him/herself to the project in an ongoing manner. A common effect is
that the project is left picking up the pieces of partial and sometimes contradictory advice (see
Box 7-16).
Box 7-16. Conflicting concepts from consultants in Mali
To limit conflicting advice, ask the consultant to:
read all key material produced by previous consultant(s);
spend time with project and partner staff to clarify what was found useful from the previ-
ous consultant(s) and what is now needed;
make sure that the consultants contribution builds on what has been found useful so far
and that she/he explicitly states how the existing elements of M&E fit with her/his contri-
bution.
But it may already be too late. Before you know it, you are looking at vastly different sets of
advice. First and foremost, use your own judgement on what will work for you in your
context. You might find it helpful to come up with a cost estimate and technical capacity
assessment for the more complicated recommendations consultants make. Second, refer to
relevant sections in this Guide for guidance on what is considered good practice.
Project Sustainability and Technical Assistance
Working with technical assistance (TA), even more than with consultants, requires careful
thought to avoid TA leaving a lack of sustainability. TA is often appointed for several years to
implement and be responsible for the work plan. In one project, M&E staff were experiencing
problems integrating M&E into the project. A TA team arrived and was able to connect the
M&E staff with project management effectively. But the TA team also took on many M&E
tasks and reports written without much input from local M&E staff. Applying a long-term
perspective, the TA input might have been organised more effectively to build local capacity.
7-30
The PDR-San project in Mali has worked with two different consultants on developing its M&E framework. Both have recommended a
different way of dealing with the idea of impact. One has focused on a list of indicators, another has abolished that and focused on
household case studies. To make matters more confusing, the cooperating institution has a different opinion yet again. Just after devel-
oping an elaborate questionnaire for tracking impact on several case study households, the cooperating institution told the project that
the project should not invest in assessing impact. This should be done only by an impartial external evaluation team or organisation. It is
no wonder that project management is left wondering whose ideas to follow.
7.5 Organising the Projects System for Managing
Information
7.5.1. Why an Information System Is Vital
Typical IFAD-supported projects are large initiatives, in terms of numbers of primary stake-
holders to be reached, geographic coverage, types of activities, budgets, time frame and
sometimes number of staff. Keeping track of relevant information means that memory and
handwritten notes alone will never be enough for those responsible for management or M&E.
Furthermore, several people/partners may wish to use the data at different moments for
different aspects of the work. The quantity of information that is collected and shared justifies
some form of information system that stores data and makes it accessible to others. A partici-
patory process is also vital. Documentation provides the foundation for interactive communi-
cation, transparency, consensus-building and continuity.
Information storage is needed at two levels impact-related to guide the project strategy and
progress-related to track operations. The focus of most M&E-related information systems is on
registering indicator-related information to assess progress with implementing the logframe. In
addition, AWPBs often include formats to track information on operational aspects, such as
personnel issues, vehicle use and accounts (see Box 7-17). To store this range of information,
from survey data to copies of contracts and correspondence, will probably require different
information storage systems (see Box 17-18).
Box 17-17. Information for clarity of responsibility
Box 17-18. Aspects to include in the management information system of WUPAP, Nepal (extract from project document)
7.5.2. Setting Up a Computerised Information System
The information system of your project will consist of paper-based archives as well as comput-
erised databases. As the latter are often not as well structured, this section focuses on how to
go about setting up a computerised information system.
Computers can make a critical contribution to tracking and using data but are no panacea (see
Table 7-9). Much time and effort is wasted in many projects on computerising data that is
then never used. In Jordan and Ecuador, projects spent substantial amounts of resources and
efforts to develop a computerised database system but relatively little effort in ensuring that it
became functional. On the other hand, as a project in Tanzania learned, handling information
coming in from several project components may be usefully computerised to extract higher-
level findings on, say, cost effectiveness or efficiency of an intervention.
7-31
The PDR-San project in Mali keeps track of all correspondence in which the project requests something of its implementing partners in
order to monitor the decision-making process and its timing. This has been useful in analysing where bottlenecks were occurring and
correcting them. It has also been very helpful during supervision missions to show that in several significant cases delays were not caused
by project management but by one of the partners. This included a delay of more than one year in contracting the M&E consultant that
the project badly wanted.
Within the first six months, the programme will establish a comprehensive MIS (management information system) to provide:
intranet with email facility and a database application for storing M&E data, reports and records;
a filing and documentation system for efficient tracking of documents and a library to store programme documents and written and
audio-visual records for reference;
software and hardware support.
Be clear about what will be stored on paper. For example, what will you do with interview
notes? With copies of maps or matrix rankings (if you use participatory diagramming
methods)? With geographic maps? These cannot always be placed on computers, although
software options for this are growing.
The larger the number of people who need to use the same data for different kinds of analysis,
the more useful it is to computerise it on a network. But this is only true if they have easy
access to the network and the information on it. In deciding when, where and what informa-
tion needs to be computerised, this should be located as close to the field as possible to avoid
mishaps with or loss of raw data, to enable data aggregation but above all to facilitate access
for data inputting and use.
Achieving impact certainly does not depend on computerising data. Information that needs to
be shared can also be photocopied and circulated, with each recipient using a common filing
system. Further, some projects may find themselves in a culture that doesnt keep written
records of activities, such as the KAEMP project in Tanzania. The project team there had to
start by training participants in keeping farm records and writing notes, rather than investing
in systems and training for computerising data. Other projects have resisted computerisation
as it means having to become more transparent and undergo more checking. Changing this
requires sensitivity.
When developing computerised system, it is paramount that this is not divorced from
management information needs. This can occur if software is used with which management
staff are not comfortable and which relies on more junior level expertise.
Table 7-9. Benefits and drawbacks of incorporating computer systems as part of project M&E
6
The process of setting up the information system has eight steps. These steps will be iterative,
particularly in situations where M&E is being set up with project stakeholders. A system needs
to be adapted as information needs in particular become clearer for the stakeholders, which is
rarely possible in one go to the degree needed for a computerised database.
1. Define what you want to store in the information system and for which purpose. Section 5 dis-
cusses in detail how to decide what to monitor and evaluate.
2. Define your basic network structure (see Figure 7-1 as an example), analysing how and when the
database will be used and by whom. This will strongly determine the design and timing of data
inputting and analysis. Ask all project and partner staff: Who needs to have access to what
information and who needs to input data?
7-32
6
Furman, R. 2001. Computerised Monitoring and Evaluation Systems: Design and Implementation Guidelines.
In: A. Ocampo (ed.). Reflections on Project Monitoring and Evaluation Methodology. Rome: IFAD.
Benefits Drawbacks
Makes it necessary to define M&E indicators and variables in
highly precise terms.
Provides precise information with different levels of aggrega-
tion.
Reduces the amount of time usually required to process data,
so increases time available for analysis.
Facilitates timely access to information by various groups and
creates conditions necessary for improving those groups analyt-
ical capabilities.
Systems are inflexible to changes in indicators and units of
analysis; methods are not straightforward to integrate.
Quantitative measures may dominate over qualitative
measures.
Analysis may be very mechanical if it is limited to producing
reports via standardised computer programmes, limiting reading
to these existing report formats.
Figure 7-1. Example of defining the network and its connections (from WUPAP, Nepal)
3. Identify how you plan to process the information, who will do it and what forms you need for this.
As previously mentioned, locate this at the lowest possible level to make it easier for people
collecting the data to analyse them. This also limits distortions in data analysis.
4. Compare and decide on options of existing software and hardware (network). Keep in mind that
data inputting should be relatively easy to do for those responsible, that data access must be
in the right place at the right time in the right form for the users, and that the system needs
to be within the financial and technical means of project and implementing partners. This
means that a decision must be arrived at with all involved. You have two options, each with
advantages and disadvantages (see Table 7-10): (1) basic existing spreadsheets and database
structures, such as Excel and Access (both part of Word Office) and (2) custom-designed soft-
ware.
Table 7-10. Comparing existing and custom-made software
7-33
Email
Data
Email
Data
Email, Data
LAN
SERVER
Policy Unit
Kathmandu
File and Print Sharing
i n t e r n e t
P
M
U
,

N
e
p
a
l
g
a
n
g
DTs and PNGOs
Email
Data
Existing Software Custom-Designed Software
Pro:
Relatively cheap (purchase of licences, purchase of hardware,
training, supplies)
Easily available
Easily adaptable by the project and often relatively easy to
master
Interchangeable/Compatible with others
More well-known
Con:
Less flexible and less suited to specific project needs
Pro:
Specific to the data needs of the project
Can be made accessible to all levels, e.g., by working with
icons rather than words
Con:
More expensive (design personnel, purchase of licences,
purchase of hardware, specialist training sessions, supplies)
Needs time and expertise to develop, test, refine
The more specific, the more difficult to adapt at later stages
Largely due to budgetary restrictions, the PADEMER project in Colombia does not use an
automated information system or a beneficiary database. Instead, it simply extracts informa-
tion from implementation and impact reports and stores this on Excel spreadsheets. A project
in India established a simple data-handling system without a central network. Instead, district
units were equipped well enough to provide the monthly progress reports on diskette or by
email. This information was used to track project progress throughout the region.
If you decide to contract a consultant to custom make your system, be sure the purchased or
designed software can be used flexibly during the project life, since information needs are
guaranteed to change. One project in Mali is planning its computer system around the
assumption that data to be stored will stay the same for many years. This will pose increas-
ingly serious problems for M&E officers as time proceeds and information needs change. Box
7-19 describes one software programme to aid with monitoring annual work plans.
Box 7-19. ASISTENTE - a programme from Latin America for annual work plan monitoring
7-34
ASISTENTE is software that monitors activities and results set out in the logframe and annual work plan (AWP). It developed out of a
project in Guatemala that needed to handle: a large, isolated and ethnically diverse region; servicing many stakeholder groups; diverse
stakeholder involvement in project implementation and receipt of information; and decentralised services to farmer organisations. The
current version of ASISTENTE has been adapted for use in all Latin American projects. Specific characteristics of ASISTENTE are:
Versatile: it can be used in diverse analysis contexts of the project AWP, including project components, results of operational plans,
beneficiary services and users of the monitoring system.
Multi-purpose: it operates on various local/satellite computers where monitoring information is inputted and local reports produced. A
central computer consolidates information sent by the satellite computers to produce consolidated reports. Information is available for
all stakeholders.
User-friendly: information is inputted once to be processed to obtain diverse reports. This requires no specialised training and is rela-
tively quick, since the programme uses icons and legends that are activated with a mouse. Reports produced are numbered and with
graphics and so are accessible to many people. Reports can first be checked on screen prior to printing or filing and they can be refor-
matted for insertion in other documents or reports.
Safe: restricted access profiles defined by the supervisor avoid mishandling of inputted data. Each user is assigned a name, password
and access profile, and is only able to save and modify her/his own information and generate her/his own reports. Users do not have
access to information saved by others nor can they generate reports by other users without authorisation. The system supervisor has
the right to configure terminals and users, input and modify information, generate all reports and assign passwords.
Complementary software, also emerging from the Guatemala project and based on the same elements as above, is called DIRECTORIO
(Programme for the Registration and Monitoring of Primary Stakeholders, Social Organisations and Implementing Organisations). It is for
information on identification of and impact on primary stakeholders, their local organisations and implementing organisations. The
majority of projects held this information, but usually not organised in a useful way. DIRECTORIO stores information related to:
number of primary stakeholders and their productive activities;
number of men and women;
place of residence and age;
services offered and number of primary stakeholders receiving each service;
implementing organisations offering services;
effects and impact achieved.
DIRECTORIO can hold a complete register of the primary stakeholders, rural organisations and implementing organisations, incorporate a
gender focus and the concept of stakeholder services, input and monitor indicators on effect and impact on the primary and other stake-
holders, generate various types of numerical and graphic reports, and classify data by project and by region.
5. With your preferred option in mind, undertake a more focused data management analysis, talking
with everyone from the secretaries to the responsible ministry about information needs and
uses and what peoples roles are in data management. This communication is needed to
ensure that the system you have in mind is capable of dealing with the precise information
and reporting needs.
6. Establish the formats needed for database entryand keep them in line with those for gathering
information. Think about how you will store qualitative and quantitative information. Do
not develop precise data formats until you are completely clear on what needs to be com-
puterised.
7. Provide user trainingon the system, otherwise it might never get used optimally. The ongo-
ing storing, updating and accessing of computerised information take skills that require user
training and follow-up support from an accessible professional. Training includes those per-
sonnel involved in designing field data collection materials. In a project in India, for exam-
ple, training for data entry was given at the start of field implementation. However, staff
responsible for monitoring at the overall-project level were not included in the training.
This, together with problems in data quality and data checking, meant the impact studies
requested by the supervision missions still werent available by the time of the mid-term
review. In another project, in Malawi, many M&E functions were carried out at the nation-
al level, such as an agricultural survey and household survey, the results of which were com-
puterised. However, when it came to analysing the results, the M&E sections at the divi-
sional level did not have the documentation on the software used, and thus analysis was
delayed.
8. Adjust the system regularly by evaluating critically with the users what information is being
used by whom, what is not, what problems exist, and whether other people need to or
would like to have access to information.
7.6 Finances and Resources to Operate the M&E System
7.6.1. Budget Items to Consider
To be effective, the M&E functions need to be supported with a realistic and clear budget.
Knowing what to include in M&E budgets is not always as clear cut as in other areas of the
project, as many M&E functions and activities overlap with implementation and management
activities. For example, management meetings where M&E information is discussed and
analysed is this a management cost or an M&E cost?
While it does not matter where such costs are allocated, it is critical to include them, make
clear that they are for M&E purposes and be clear about who can decide how to use the M&E
budget. However, do avoid putting all M&E under the heading project management as this
makes it very unclear what is available for the M&E. The M&E budget wherever it might be
allocated needs to cover the items listed in Table 7-11.
7-35
Table 7-11. Items to consider in an M&E budget
7-36
Categories Specific Items
Contracts: consultants/external expertise
(fees and travel expenses)
Developing a detailed M&E plan
Establishing monitoring mechanisms and formats
Establishing information management systems
Facilitating review workshops
Conducting specific survey or monitoring work
Assisting with capturing and documenting lessons learned
Training and capacity-building (workshops, courses)
Impact assessment studies (e.g., baseline studies)
Mid-term and final external evaluations
Physical non-contractual investment costs Equipment for monitoring
Communications and presentation equipment
Establishment of M&E offices (e.g., furniture, equipment)
Publication materials
Computers and software
Vehicles
Training and study tours for M&E-related
capacity-building
External and on-site training courses:
Training of primary stakeholders to build capacity in M&E
Training of selected implementing partners concerned with community development
on introducing and supporting participatory monitoring
Training selected M&E staff of service providers (government and NGO) on relevant
M&E aspects
Training M&E officers and key management staff on M&E, including computer
training
Financial management training, as appropriate
Promoting exchange of experiences with other projects, among the different stake-
holder groups
Course fees
Labour costs:
Recurrent permanent staff salaries,
temporary support staff
Investment technical assistance
(short- or long-term, national, interna-
tional)
Planning and developing the M&E system
Conducting regular monitoring
Report writing and analysis
Participating in review processes and events
Information management
Capturing and documenting lessons learned
Disseminating M&E findings
Supporting community based/participatory M&E processes
Non-labour operational costs Vehicle fuel and maintenance and other transport
Office running costs (overheads, maintenance)
Stationery
Meetings
Allowances for primary stakeholders and project implementers
External data, such as maps
Communication and publication costs printing/copying documents, editing, layout
and publication of key documents
Specific evaluation events (M&E planning workshops, review workshops, field survey
work, stakeholder consultations, specific monitoring activities, mid-term and final
external evaluations):
Venue costs
Advertising materials
Accommodation
Attendance fees and course fees
7.6.2 Levels of Allocation
While there are no fixed rules for this, M&E budgets range from 2% to 15% of all costs. In
projects, such as MARENASS in Peru and FODESA in Mali, where the stakeholders are
exploring new ways of working together, M&E budgets are likely to be proportionally higher
since more time is needed for reflection on what works. Table 7-12 shows the budget line for
five Latin American projects, including MARENASS.
Note that each project clusters its M&E costs in different ways due to the different approaches
adopted. M&E in MARENASS is decentralised and sub-contracted. Therefore the project coordi-
nation itself does not need vehicles or materials for M&E. However, clustering all M&E costs
under studies makes it necessary for the budget to be accompanied by a detailed M&E plan
to ensure clarity about how the funds are to be allocated.
Regardless of how the M&E budget is calculated, it will always overlap to some degree with
other project activities. For example, about 78% of total annual funds for the MARENASS
project goes directly to farmers management of their own community development plans,
with the remainder going to two components: project management and M&E. Yet, even
within these two components, many costs were related to community development such as
festivals, communication bulletins and so on, raising this figure to 82-85%. Therefore, do not
detail the M&E budget excessively. Much learning occurs through the normal interactions of
project implementation. What is most important is to ensure inclusion in the budget of the
events, procedures and staff time that support project learning and reflection.
Many projects experience funding delays at start-up. To deal with this, financial advances are
available from IFAD through the Special Operations Facility (SOF). These grants have proven
to be extremely useful in expediting project start-up and implementation. The Philippine
government, for example, administered such a grant to facilitate staff interview and recruit-
ment, the start-up workshop and purchase of equipment. Other uses of SOF grants include
engaging a project expediter to assist with fulfilment of conditions for loan effectiveness and
financing training in management systems. Many projects use the SOF to cover initial M&E
costs, such as M&E training for staff and partners and preparing baseline studies.
Table 7-12. M&E budgets for five projects in Latin America, compared to total project budget, from appraisal reports
(in United States dollars)
7-37
Item Country and project
PROSALAFA PRODECOP SARAGURO MARENASS PRODECOP IV
Investments: jeep 4x4, computer equipment, furniture
and office equipment
20,800 28,830 27,950 24,329
Running costs: salaries, travel, materials, maintenance,
insurance
144,900 346,485 213,010 227,532
Studies: baseline, evaluation studies, documentation,
monitoring, publications, taxes
205,000 540,000 119,916 330,815 1,770,000
Total M&E 370,700 915,315 360,876 582,676 1,770,000
Total project 26,742,700 24,365,000 16,753,781 34,491,969 19,142,800
Share of M&E 1.4% 3.8% 2.2% 1.7% 9.2%
Venezuela Equador Peru Chile
7.6.3 Costs and Resources for Participatory M&E and Selected M&E Activities
From a budget perspective, participatory learning processes are more time intensive than those
in which only a few people are involved. Time is needed to organise meetings with larger
numbers of people and more diverse groups and to reach agreement on how to proceed with
M&E or on what data mean. Perhaps it will be necessary to invest in training both project and
implementing partner staff in facilitation skills and also in training primary stakeholders in,
for example, impact assessment methods and developing local indicators. Some projects have
found that capacity-building on participatory M&E tasks can be undertaken on a cost-sharing
basis, but usually the project bears the full cost.
Specific budget items to consider for more participatory forms of M&E are:
specific training for staff in participatory techniques and participatory M&E;
extra meetings with stakeholders for designing M&E;
additional meetings for local-level analysis;
short training workshops on key steps in designing M&E and specific elements such as indi-
cators and methods (including using the logframe matrix).
Participatory projects also require ongoing planning processes to determine what will be
implemented for whom and with whom. In cases where project activities emerge, the M&E
budget needs similar flexibility. Therefore the budget could include a contingency line for
significant but unanticipated opportunities. Often, the items in an M&E budget are suffi-
ciently broad that they can be used for diverse types of M&E activities. For example, including
a budget line participatory impact assessment leaves the project with much flexibility about
when and how to undertake this.
Boxes 7-20, 7-21 and 7-22 show examples of how to go about calculating costs for specific
M&E activities: a baseline study, participatory impact monitoring and the management infor-
mation system. Note that these costs are not intended to be indicative. Your costs will depend
on context-specific factors, such as the size of the geographic area, local labour costs, sample
size, type of methodology, use of external expertise, etc.
Box 7-20. Provincial baseline study in China (2000)
7-38
Total cost =USD 13,173
Activity
1,200 household interviews in 37 project areas in seven counties; data entry cost =USD 10.98 per interview
Personnel
For interviews: two full-time staff per project area =74 staff
For data management: two full-time staff per county =14 staff
For managing/reporting: one province-level consultant
Time schedule
Interviews: two weeks
Data entry: ten days
Data checking, aggregating, analysing: two weeks
Reporting: two weeks
Box 7-21. Introducing participatory impact monitoring (PIM) in Vietnam (2000)
Box 7-22. Costs for setting up the management information system in Nepal (2001)
7-39
Total cost of introducing PIM per project =USD 15,000. This includes: consultants fees, training workshop materials and follow-up assis-
tance.
This cost can be reduced by:
a) training and employing local experts to act as facilitators and consultants;
b) integrating PIM activities into ongoing planning, extension and M&E activities;
c) cleaning up existing data cemeteries and substituting more expressive impact-related information for fewer columns of figures;
d) accepting reliable statements on trends rather than insisting on statistical accuracy.
Total cost of setting up the management information system: NPR (Nepalese rupees) 1,719,239, spread over four years, with 43% of this
for the first year, when procurement costs are high. Total expenditure, per item is:
Server software 255,500
Email server software 73,000
Proxy server software 43,800
Network hub and cables 21,900
Installation 51,100
Internet dial-up charges 169,859
Dial-up networking phone charges from districts and project unit 636,970
Purchase of publications and media for library 212,323
Network maintenance contract 254,788
At the time of writing the document, the exchange rate was 1 USD per 75 NPR (April 2001).
Further Reading
Gubbels, P. and Koss, C. 2000. From the Roots Up: Strengthening Organizational Capacity through Guided Self-Assessment. (second edition).
Oklahoma City: World Neighbors. Order via: http://www.wn.org/wnstore.
ISNAR. 2001. Framework and Methodology on Training Needs and Organizational Constraints Assessment in Kenya.
Download via: http://www.isnar.cgiar.org/activities/training/request.htm.
Lamy, E. and Lessard, P. 2001. Capacity Building: A Manual for NGOs and Field Workers. Quebec: CECI.
Also available in French and Spanish. Order via: http://www.pactpub.com
Lusthaus, C., M. Adrien, G. Anderson and F. Carden. 1999. Enhancing Organizational Performance. A Toolbox for Self-assessment. Ottawa:
IDRC. Also available in French and Spanish. Order via: [email protected]
Lynton, R. and Pareek, U. 1990. Training for Development (second edition). Connecticut: Kumarian Press.
Order via: http://www.kpbooks.com.
7-40
Section 1. Introducing the M&E Guide
Section 2. Using M&E to Manage for Impact
Section 3. Linking Project Design, Annual Planning and M&E
Section 4. Setting up the M&E System
Section 5. Deciding What to Monitor and Evaluate
Section 6. Gathering, Managing and Communicating Information
Section 7. Putting in Place the Necessary Capacities and
Conditions
Section 8. Reflecting Critically to Improve Action
Annex A. Glossary of M&E Concepts and Terms
Annex B. Annotated Example of a Project Logframe Matrix and
Logframe Explanation (relates to Section 3)
Annex C. Annotated Example of an M&E Matrix (relates to
Section 5)
Annex D. Methods for Monitoring and Evaluation (relates to
Sections 3, 6 and 8)
Annex E. Sample Job Descriptions and Terms of Reference for
Key M&E Tasks (relates to Section 7)
List of Booklets in the Guide

Putting in Place The Necessary Capacities and Conditions: Section

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Putting in Place The Necessary Capacities and Conditions: Section

Uploaded by

Copyright:

Available Formats

Putting in Place

You might also like