⛶
🌻 !MENASP Network, University of Bath#
13 Oct 2023

2023-10-13

cannot be publicised i think

An impact evaluation of the MENASP Network, an academic-and-policy network hosted at the Institute for Policy Research, University of Bath, that brings together policy makers and academics working on social policy in the Middle East and North Africa. The evaluation asked what the routes to impact are for the network and the affiliated MENASP-CP project, according to its stakeholders, and what unintended or unexpected consequences they have noticed.

Evaluating a research network is hard because influences are diffuse and travel a long way down the "river of impact". So we collected stories about the most significant changes in MENA social policy research and practice to which the network had contributed, then coded each story as a set of causal links and aggregated them into a map.

Approach: We combined Most Significant Change interviews with causal mapping. Thirty-two people were interviewed by the researcher, ten more by an automated QualiaInterviews bot to widen the reach to stakeholders further down the river of impact, plus ten written testimonies. Thirty-seven distinct respondents in total: international researchers based in or working on MENA, NGO staff, international-agency representatives, and the network's founder.

Coding: Transcripts were uploaded into Causal Map and coded using a hybrid human and AI approach. The analyst built an initial codebook by reading transcripts, running an AI summary, and running an exploratory zero-shot AI coding pass. The Causal Map AI then coded the full set against this codebook, iterating until labels stabilised. Factor labels use a two-part form ("MENASP's positive characteristics; regional focus") so high-level themes can be aggregated while keeping specific detail.

The eight most-mentioned factors covered: MENASP's positive characteristics (diverse, friendly, UK-MENA centric); what MENASP provides (webinars, conferences, mailing list); sharing and interacting; involvement with MENASP; researchers benefit; Dr Jawad personally; improved social policy approaches in MENA; and improved social policy knowledge in MENA.

The map showed a self-reinforcing process more than a one-way pipeline. The strongest claim across respondents was that the network helps define the identity of social policy as a field in MENA, especially for early-career researchers, and that this professional identity is itself a prerequisite for taking a seat at the policy table. Connections, ideas, publications and a welcoming space were named as outputs in their own right.

"MENASP has been instrumental in my career development. Through their funding opportunities and networking events, I have been able to conduct research, publish my work, and connect with other professionals in the field."

"We produced a report, many points are being taken up with ministry counterparts. It is data driven so it opens the appetite for evidence based policy."

We also broke the map down by respondent group. NGO respondents emphasised the benefits to NGOs from MENASP's characteristics and services; researcher respondents emphasised sharing, interacting and benefits to researchers. A diagnostic chart of "average links" showed which groups stayed close to the network and which traced impact further downstream.

The same evidence base surfaced critical comments: that the network's impact is diffuse and hard to visualise, that engagement with ministries could be stronger, and that the network is too English-centric for some regional stakeholders. The map made these tensions visible alongside the positive pathways rather than burying them.