Four Tools for Thoughtful Monitoring: Part One

I recently found myself reflecting on more than 30 years in international development, almost all of them focused on monitoring and evaluation, and increasingly driven by a passion for what I call Thoughtful Monitoring.

I thought about the tools I have used and adapted across very different programmes and contexts: with rice farmers in Bangladesh; sorghum farmers in Malawi; policymakers in Nigeria and Lesotho; programme managers from Vietnam; communities living in and around Uganda’s Bwindi Impenetrable rainforest; and market traders in Tanzania and Bougainville, a reluctant part of Papua New Guinea.

At first glance, these experiences may have little in common. Yet, in each case, my colleagues and I adapted different combinations of four methods that have made a genuine difference to the quality and relevance of support for those most in need:

1. Robert Chambers’ Direct Matrix Ranking (DMR);
2. Lawrence Salmen’s Beneficiary Assessment (BA);
3. David Snowden’s Anecdote Circles (AC); and
4. Robert Kaplan and Robin Cooper’s Activity-Based Costing (ABC).

These are not new tools, and that is partly the point. Together, they offer a coherent alternative to technocratic monitoring models that privilege externally defined objectives, indicator frameworks, dashboards and optimisation over locally meaningful judgements, institutional realities and political feasibility.

The evidence they produce is not only credible. It is usable, particularly in development contexts that are complex, dynamic and politically sensitive.

Better monitoring requires better judgement

Rigorous monitoring, inextricably linked to adaptive management, cannot be detached from human experience. Some of the most powerful evidence emerges precisely where structured judgement meets lived reality.

Bringing these four methods together creates a sequenced approach to learning and decision-making. It is participatory without being impressionistic and disciplined without becoming technocratic. It can also provide a practical architecture for Collaboration, Learning and Adaptation (CLA) and Thinking and Working Politically (TWP).

Instead of treating learning as a reporting requirement or a post hoc evaluation exercise, the approach embeds it in everyday programme decisions. It connects beneficiary experience and definitions of success with frontline judgement, explicit trade-offs and operational feasibility.

This challenges a stubborn assumption: that better monitoring is primarily about better data, a stronger focus on results, more sophisticated dashboards or smarter algorithms.

Good monitoring is fundamentally about good judgement—judgement that is socially grounded, politically aware and collectively constructed. Human interaction matters here not simply as a moral preference, but as an epistemic necessity. In complex social systems, meaning, power and behaviour cannot be reduced to proxies and, for the most part, defy prediction.

The four methods introduce different voices into that judgement. Beneficiary Assessment and Direct Matrix Ranking create space for those programmes intend to support. Anecdote Circles bring in the people who deliver that support. Activity-Based Costing connects their insights with those making financial and operational decisions.

Together, they can also help correct a familiar public-sector malaise: the separation of financial from non-financial monitoring.

Part One considers the first two methods: listening to lived experience and making people’s values and trade-offs explicit.

Beneficiary Assessment: grounding judgement in lived experience

Lawrence Salmen’s Beneficiary Assessment moves monitoring away from abstract notions of impact and towards the way development actually works—or fails to work—on the ground.

It treats beneficiaries as subjects in a conversation about issues that matter to them, not as objects of a survey that matters to an interviewer (personal communication, Lawrence Salmen). In other words, people are not data points. They are social actors embedded in power relations, market systems and informal institutions.

Conversational interviews are usually conducted with individuals and groups, using simple prompts or a short checklist of questions, as shown in Box 1 below.

Through systematic listening and observation, BA uncovers more than satisfaction or dissatisfaction. It explores the reasons for both.

This can reveal:

• what aspects of a programme matter most to people;
• where access or delivery breaks down in practice;
• how institutional behaviour is experienced at the point of contact;
• which assumptions about participation and response do not hold; and
• how experiences differ among women, young people, older people and marginalised groups.

That last point is crucial. Equity and inclusion become visible rather than assumed.

Beneficiary Assessment therefore provides a diagnostic frame. It helps us understand what matters, where systems fail and which institutional behaviours enable or obstruct results. However, listening alone does not resolve the choices programmes must make. That is where Direct Matrix Ranking becomes particularly useful.

Direct Matrix Ranking: making values and trade-offs explicit

Robert Chambers’ Direct Matrix Ranking remains a quietly powerful participatory monitoring tool. At its best, it does something conventional monitoring instruments routinely fail to do: it makes participants’ own criteria explicit and forces analysts to confront trade-offs as the people affected understand them.

DMR:

• makes local priorities explicit;
• is participatory and visual;
• connects quantitative choices with qualitative explanations; and
• supports planning, targeting and user-centred delivery.

Instead of asking people to assess support against criteria imposed by logframes or pre-coded indicators, DMR invites them to define what matters. They might then rank interventions, delivery methods or design choices against those criteria.

This surfaces local value systems, tacit knowledge and contextual reasoning that surveys and dashboards often flatten. Its elegance lies in its discipline: it is structured enough to support comparison and analysis, but open enough for participants to think aloud, negotiate and debate meaning in real time.

How it works

Consider an exercise in which farmers compare rice varieties:

1. List the varieties across the top of a matrix.
2. Ask farmers which criteria matter most when choosing a variety—for example, yield, tolerance, cost or the time until it can be consumed.
3. If useful, discuss and agree an importance weighting for each criterion, such as a scale from one to five.
4. Score each variety against each criterion—for example, one for poor and five for excellent.
5. If weights are being used, multiply each score by its weight and calculate the total for each variety.
6. Discuss why the highest-ranked variety is preferred under those criteria and what qualifications or disagreements remain.

The same exercise can assess the people or organisations providing support. A programme might list different forms of assistance and ask participants to define the criteria by which field staff, community services or advisers to a government ministry should be assessed.

The calculation is not the most important part. The surrounding conversation is.

Narratives prevent rankings from becoming decontextualised scores. Rankings prevent narratives from remaining impressionistic. Together, they show not only what people prefer, but why, under which conditions and at what cost.

From listening to structured choice

Used together, Beneficiary Assessment and Direct Matrix Ranking create a productive sequence.

BA offers narrative depth and institutional critique. It identifies what matters to people, how support is experienced and where systems break down. DMR then asks participants to compare and prioritise possible responses against those lived concerns.

The result is decision-useful evidence. It explains preferences, tolerances and non-negotiables instead of merely recording opinion.

Yet even this is not enough. Programmes also need to understand what those closest to delivery are seeing, where rules collide with reality and whether a preferred change is operationally and financially possible.

That is the focus of Part Two: how Anecdote Circles and Activity-Based Costing bring frontline experience and feasibility into the same monitoring architecture.

Continue with Part Two: From Frontline Stories to Feasible Action.