by Richard Bond, Senior Consultant, IMA International

I have been working with IMA international on development of a new online course, so what is it about?  Well, it is about a unique system and set of tools which has emerged from my 40 years of developing learning systems in large, complex projects many of which followed a learning process. It’s Less about frameworks, indicators and terminal evaluations; more about monitoring internally to learn and change during the investment period. A different sort of animal.

So, why might you want to learn about establishing a genuine learning process system in your large and complex project? Well, four reasons;

Firstly, conventional M&E tends to be ritualised and essentially for accountability purposes to central government or donors. Fair enough, they do provide the funds for investment, but does it get the best results? For that there must be priority for learning over accountability.

​Secondly, the policy learning needs of government and donors are poorly met by summative ‘snapshot’ evaluations. Even if the findings are robust – a big ‘IF’ – then the chances of any future project having the same context and following the same process that might benefit from the lessons are remote. Learning should be applied during existing investments.

Thirdly, existing framework-based M&E systems are not managerially relevant for project information needs, especially at the operational level.

And finally, unless your assumed theory is absolutely rock-solid, an unlikely scenario in complex, diverse, resource and information-poor environments, where change in human behaviour is needed; then you will have to learn and change as you go along.

OK, but in what ways would it be different from a conventional M&E system?

Perhaps the most radical difference would be that it would be designed and operated from the field level not the centre. The centre would concentrate on training support and reflection. 

The monitoring is highly analytical and using combined qualitative and quantitative systems provides information on all stages of emerging results for multiple management levels in the organisation, all done in real-time. This allows true process monitoring and when supported by targeted mini-reviews explains weaknesses for high-level reflection. The whole is based on a modified ‘Action-Learning’ cycle.

That is a tall order, so what methods enable such a system?

There are five innovative tool sets; ‘Cognitive Mapping’ taps into the real client field situation and these results help formulate a non-linear ‘Theory of Change’ where all hypotheses are made explicit. During implementation, field operatives monitor outputs in a managerially relevant way using a system called ‘ATOM’ that not only keeps an eye on the big picture of implementation but provides diagnosis of weaknesses. There are also a few key techniques, called WEDEX and CAJUS for periodic checks on the emergence of outcomes and impact, also capable of disaggregation, that help target internally driven mini-reviews when failure of hypothesis is suspected. These in turn feed back into the reflective Theory of Change model to give evidence-based adjustment.

All these methods have been used in field situations over the last 40 years, are done by existing field staff, can be developed on conventional computer systems by local staff and function as a coherent real-time system.