Monitoring and evaluation (M&E) has long been considered a necessary companion to WASH interventions but the relationship between ‘doing’ and ‘observing’ continues to be a tricky one. Over three sessions during the UNC Water and Health conference last week, Professor Barbara Evans and Dr Jamie Bartram took participants through a highly interactive investigation of where M&E are currently at in the WASH world, which fuelled conversations both in and outside the sessions.
What’s M&E for anyway? An old question – but from the discussion this question sparked, clearly still relevant. Tracking the progress of a particular intervention; product control; utility management; overseeing the work of others; and, comparison between countries (or communities) were all mentioned as reasons to do M&E.
However, one of the most discussed reasons was ‘to satisfy the requirements of donors’ which offers insight into some of the difficulties surrounding M&E, in particular for implementers. Data is collected and collated into reports for donors but rarely otherwise used by implementers, let alone shared with beneficiaries. In an interesting and perhaps deliberate parallel process, Evans and Bartram incorporated their own monitoring into the three sessions: collecting self-administered forms from participants about the discussion points and then reporting the findings from the previous day back to the group each morning. Whilst this no doubt entailed a mammoth effort behind the scenes the message was clear: as a participant both collecting and providing data, knowing that the findings would be shared with us and that the next day’s work depended upon this new information, the imperative to participate in the process was high. The process actually felt valuable.
The issue of M&E being valuable to implementers is critical; but is something we’re clearly still not getting quite right. Ms Jyoti Shukla – Senior Manager for the Water and Sanitation Program at Water Global Practice – alluded to this during the Monday plenary when she made the crucial remark that:
‘we should be collecting evidence not for the sake of evidence, but it’s important that findings from evidence get implemented’.
As someone who sits on the proverbial fence of WASH, i.e. not as a ‘doer’ but as an ‘observer’, I’m often confronted with this dilemma. I have the privilege of observing and reporting on what others do, and can only hope that implementers might find it useful because otherwise it is just collecting evidence for the sake of evidence. On the other hand, my experience of implementers (not just in the WASH sphere) is that the sharing of evidence is often experienced as a punitive rather than a useful process. This is perhaps where we are failing in terms of ensuring that M&E is valuable for implementers. In many of my conversations with other ‘fence-sitters’ during the week, it was clear that this is a widely occurring phenomenon: findings which should be seen as opportunities for improving practice are experienced instead as bad reviews which could potentially be interpreted by donors as reasons to cut funding.
It’s difficult to see how M&E could become a non-punitive experience in a field where funding continues to be mostly donor-dependent and allocated in project time-scales. The perception of M&E being a risky endeavour was highlighted by Bartram in his opening address to the conference where he congratulated World Vision on their courage to team up with the UNC Water and Health Institute for an upcoming research partnership. In addition, all too often we hear that M&E is expensive. Whilst it does come at a cost, if M&E were perceived as an opportunity for being more effective perhaps it would seem less of a financial – and reputational – burden.
How then, can M&E become a valuable rather than a burdensome experience for implementers? Including M&E as part of funding requirements is a great first step, in particular when resources are allocated accordingly. For implementers who are not beholden to donors in quite the same way, such as smaller CSOs or utilities, the story is somewhat different. For this reason, M&E should be fit for purpose – which means it might be conducted and used in different ways by implementers.
The last of the 3 sessions focussed on the myriad ways in which M&E is conducted within the WASH sphere – both in terms of what indicators are used as well as how indicators are measured. The question of whether standardising M&E within WASH might make the process easier for implementers was posed and met with predictable debate. For those who need to compare data at the broadest level, such as the JMP, there is a strong imperative to standardise M&E processes. However, the risk with standardisation is that M&E might become even less fit for purpose than it is currently.
Unsurprisingly, these sessions and the conversations they sparked ended inconclusively but they served their purpose as a jump-off point for an ongoing discussion about how M&E processes might be shared and discussed in a more centralised, but not necessarily standardised, format. In turn this might be part of the continuing challenge to ensure M&E is valuable for implementers and not just food for the data graveyard.
Naomi Francis is a PhD Candidate at the Nossal Institute for Public Health at the University of Melbourne.
