Abstract Suppose an evaluation team has a corpus of interviews and progress reports, plus (at least) two candidate theories of change (ToCs): an original one and a revised one. A practical question is…

2026 01 08 The partner{.banner} We partnered with INTRAC on a project for the SCC (Strengthening Civil Society) programme. This programme is led by a consortium of four organisations: PAX, ABAAD, Defe…

For evaluators who use AI on interviews, reports and other text, and for the people who commission them. Reconstructible reporting means handing over, with your report, the data and enough detail for…

Marshalling causal evidence at scale for Contribution Analysis and beyond. {.banner} start multi column: PowerTool {.panel} Causal Mapping is not primarily an evaluation method; it is a way of identif…

Contribution analysis is about testing and refining a theory of change to build a credible case for contribution. It's focused on a specific theory of change and on the contributions to outcomes. Thes…

To evaluate a program, the evaluator can use Contribution Analysis (CA) [@mayneMakingCausalClaims2012]. We start with a program logic or Theory of Change (ToC), consisting of possible pathways from in…

For evaluators and applied social researchers who use AI in qualitative analysis, or use or assess work that does. Summary A frontier AI can turn a terms of reference and a pile of transcripts into a…

010 Testing rival theories ((theory fit))|The theory fit paper is about rival published theories and which of them the material supports. 020 Realist mechanisms ((realist mechanisms))|The realist pape…

!100 Task 3 Answering questions Individual questions/img/screenshot 110 does the evidence support your theory of change qq testing theory.png One way to do this is to use the elements from your theory…

Is there already a standard for this? Before publishing 003 An open format for reconstructible reporting ((open format))|an open format for reconstructible reporting, we looked for an existing open st…

Outcome harvesting is the most participatory of the methods in this chapter, and the one where a machine has least business doing the interesting part. In this paper we ask which of the questions comm…

For evaluators, commissioners and anyone else working out what evaluation analysis should look like now that AI can do much of it. In our experience, a frontier model such as Claude Opus 5.5, given th…

For evaluators who use AI to analyse interviews, reports and other text, and for the people who commission their work. These are the principles we work to when AI does some or all of an analysis of te…

Work in progress. This is a design sketch. We have written the workflows and validated them against the engine, but we have not run them, so there are no results to report. Parts of the literature sea…

Work in progress. This page sets out the design for a study. We have run one of the five theories through the workflow on the real corpus. The results are under A first run . We have not yet run the s…

Work in progress. Which steps an analysis may be built from if somebody else is to rebuild and check it, and how far to break it down. Part of 000 Pilots, not passengers reconstructible reporting in e…

Theory based evaluation is the family that the other method papers in this chapter belong to. Contribution analysis, realist evaluation and process tracing each state a theory in advance and then test…

[@powellDoesOurTheory2023] Source: book chapter draft in . Purpose Show how QuIP style causal mapping can compare an official programme ToC (“their theory”) with empirically coded beneficiary narrativ…