Using AI in Monitoring and Evaluation: Workflows, Limitations, and Validation

Monitoring and evaluation (M&E) teams frequently manage extensive narrative records, community feedback, and qualitative surveys. Authoritative bodies across the development sector—such as the UK civil-society network Bond in its 2026 guidance on AI across the evaluation lifecycle, and ALNAP in its May 2025 Explain Brief on artificial intelligence—note that automated language tools can reduce routine administrative work when deployed carefully.

Where AI supports M&E workflows

In everyday evaluation cycles, generative models and natural language processing tools offer practical efficiency in five specific functions:

• Transcription: Converting recorded audio from field interviews, focus groups, and community meetings into written text. This accelerates documentation while avoiding days of manual typing.
• Coding open-ended survey answers: Sorting free-text survey responses into established thematic categories or tagging feedback from community scorecards against a predefined codebook.
• Summarising qualitative material: Distilling lengthy key informant interview transcripts and workshop notes into concise summaries, thematic briefs, or draft sections for internal review.
• Drafting data-cleaning scripts: Writing initial code snippets in languages such as R or Python to standardise variable names, flag missing values, and identify duplicate survey entries.
• First-pass analysis: Scanning qualitative datasets to identify dominant phrases, emerging themes, and broad patterns before evaluators begin deeper, contextual analysis.

Practical limits and ethical risks

Despite operational gains, sector guidance published by ALNAP, The MERL Tech Initiative, and BetterEvaluation highlights significant structural weaknesses:

• Hallucinated findings: Large language models generate responses by calculating word probabilities rather than verifying truth. When analysing field records, an AI system can invent nonexistent quotes, misinterpret evidence, or invent findings that misrepresent community realities.
• Missing languages and dialects: Most commercial models are trained on dominant international languages. As ALNAP evaluators note, automated systems frequently misunderstand regional dialects, vernacular terms, and local languages used by crisis-affected populations, leading to skewed interpretations.
• Algorithmic and cultural bias: Training data mirrors historical and systemic prejudices. Automated systems can overlook minority perspectives, misinterpret sensitive social dynamics, or amplify majority viewpoints at the expense of vulnerable respondents.
• Consent and data protection: Pasting field interview notes or survey databases into public AI interfaces transfers information to external servers. This practice risks exposing personally identifiable information (PII). Unless an explicit enterprise agreement guarantees private data handling without retaining prompts for model training, doing so breaches participant consent and standard "do no harm" humanitarian standards.

Four-step validation protocol

To reconcile efficiency with evaluation rigour, the UK Evaluation Society's good practice guidelines (published in November 2025) underline the necessity of human control, transparency, and quality assurance. Evaluators should follow this sequence before integrating AI-processed data:

1. De-identify all inputs: Strip names, physical locations, phone numbers, and identifying programme references from qualitative records before entering data into any digital tool. If working with high-risk protection data, avoid third-party cloud processing entirely.
2. Benchmark against raw field data: Perform manual spot-checks on a randomised sample of AI outputs—such as 15 to 20 per cent of coded open-ended responses or summarised interviews—comparing them directly against primary notes or audio recordings.
3. Check for omitted and nuanced voices: Qualitative evaluation requires detecting silence, hesitation, and indirect answers that automated summaries often discard. Examine whether non-dominant perspectives or critical community remarks were flattened or omitted by the software.
4. Record tools and prompts transparently: Maintain an evaluation audit log detailing which platforms were used, the prompt text supplied, the version numbers of software, and the exact steps taken by human evaluators to verify the final text.

Checklist

Before any AI-generated analysis, code, or summary enters a formal monitoring report or evaluation deliverable, confirm that:

• Participant consent protocols explicitly permitted automated processing of their interview or survey contributions.
• All personally identifiable data and location indicators were verified as removed prior to analysis.
• The organisation confirmed that the AI platform does not retain, share, or train on submitted evaluation inputs.
• A human evaluator manually audited a sample of AI-coded responses against raw survey answers to ensure accuracy.
• Field teams fluent in local languages and cultural nuances reviewed all automated translations and theme assignments.
• The final report includes a methodology note stating which AI tools were used, for what purposes, and how outputs were checked.