SOAS Research Online

A Free Database of the Latest Research by SOAS Academics and PhD Students

[skip to content]

Juden, Matthew Tom (2022) Towards a multi-dimensional model of impact evaluation quality: assessing development impact evaluation methods with respect to context. PhD thesis. SOAS University of London. DOI: https://doi.org/10.25501/SOAS.00036909

[img]
Preview
Text - Submitted Version
Available under License Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 (CC BY-NC-ND 4.0).

Download (1MB) | Preview

Abstract

In this thesis, I give an account of the quality of (quasi-)experimental impact evaluation methods for development interventions that goes beyond internal validity considerations to also incorporate the transferability of findings to new contexts. To investigate how well different (quasi-)experimental impact evaluation methods facilitate transferability, I adapt tools from realist synthesis to extract and synthesise the programme theories that underpin a set of evaluations of the same intervention-outcome pair. From this synthesis of programme theory, I derive the markers of intervention causation in context (MICCs) that a (quasi-)experimental impact evaluation of an intervention of that type would have to report to facilitate the transferability of findings. I systematically build a complete set of (quasi-)experimental impact evaluations for two intervention-outcome pairings, and apply my method to these two cases. This generates case-specific insights such as identifying evidence gaps where minimal further data generation offers large gains in understanding. Further, the analysis generates cross-case insights such as the tendency to better report causally significant features of intervention implementation than causally significant features of context. Most importantly, the analysis suggests there is no association between method choice and the facilitation of transferability, in theory or in practice. I argue that we can nonetheless improve on ‘there is no gold standard’ by showing how generating a middle-range theory of intervention causation capable of underpinning the list of MICCs for a type of intervention provides a guide to evaluation method choice and to transferring results between contexts. In parallel, to render my main results more useful to the relevant experts and therefore more likely to influence practice, I conduct semi-structured interviews with development intervention evaluation experts. I identify a broad discursive trend in favour of theory-based evaluation and a nascent interest in the use of middle-range theories to underpin transferability. I therefore frame my main results in these terms.

Item Type: Theses (PhD)
SOAS Departments & Centres: Departments and Subunits > Department of Development Studies
SOAS Research Theses
Supervisors Name: Peter Mollinga
DOI (Digital Object Identifier): https://doi.org/10.25501/SOAS.00036909
Date Deposited: 25 Mar 2022 11:36
URI: https://eprints.soas.ac.uk/id/eprint/36909
Funders: Economic and Social Research Council

Altmetric Data

Statistics

Download activity - last 12 monthsShow export options
Downloads since deposit
6 month trend
105Downloads
6 month trend
120Hits
Accesses by country - last 12 monthsShow export options
Accesses by referrer - last 12 monthsShow export options

Repository staff only

Edit Item Edit Item