Coding with AI

20 pages in this theme.

A workflow for causal coding with and without AI
Summary You have a stack of documents or interviews and you want to answer research or evaluation questions rigorously. This is one workflow for getting there: eight steps, from planning through codin…
AI coding experiments synthesis
Coding texts with LLMs: some transferable learnings from a recent experiment series Causal Map is software for a particular kind of qualitative research: you feed it interviews or documents, and it he…
AI-assisted causal mapping – Summary (validation)
[@powellAIassistedCausalMappingForthcoming] This page outlines the paper point by point. For the story of the study, see the case study Can AI accurately map causal claims A validation study ((cs vali…
AI-assisted causal mapping. Uncovering causal pathways with INTRAC
2026 01 08 The partner{.banner} We partnered with INTRAC on a project for the SCC (Strengthening Civil Society) programme. This programme is led by a consortium of four organisations: PAX, ABAAD, Defe…
An M&E time machine. Using AI to measure changes in a system across a time period
2023 09 19 Summarised from "An M&E time machine: Using AI to measure changes in a system across a time period on features which only emerge during it" by Steve Powell, Gabriele Caldas Cabral and Hanna…
Autocoding with AI
There is a whole chapter on this:
Can AI accurately map causal claims - A validation study
2026 03 03 Summarised from "AI assisted causal mapping: a validation study" by Steve Powell and Gabriele Caldas Cabral. Extracting causal claims from qualitative interviews is a notoriously slow proce…
Causal mapping is an interesting QDA approach which is very suitable for scaling with AI
Abstract This paper argues that AI can be used inside a qualitative workflow without becoming the analyst, provided it is based on a method whose coding task is narrow enough to be locally checkable.…
Code Like Me – teaching the AI your coding style
You have coded some of your interviews by hand and you want the rest coded the same way: with your labels, and with the same sense of which claims count. Code Like Me is how the Causal Map app does th…
Coding QuIP data with AI and MapCat
A QuIP corpus is the kind of data AI coding was built for: many interviews, each dense with statements about what led to what, all needing the same treatment. Hand coding sixty transcripts takes weeks…
Comparing a Fine-Tuned Model to an Engineered Prompt in the Context of Causal Connections in a Passage of Text
2023 09 01 Summary{.banner} Samuel Goddard, a student at the MSc in Data Science at the University of Bath, compared the performance of a fine tuned OpenAI model and an engineered prompt in identifyin…
Consolidating overlapping labels into a shorter list
Coding in people's own words, or letting a model name what it finds, leaves many labels that overlap in meaning, and this page says how to turn them into a shorter list without losing what they meant.…
Different kinds of coding and recoding
Assuming you're not coded with a fixed codebook, and assuming you're coding multiple sources or you've got texts which are longer than your selected chunk size, so you are breaking them up into multip…
Evaluating AI causal coding – prompts, models and the metrics that lied
Evaluating AI assisted causal coding: prompts, models, and the metrics that lied. A day long systematic evaluation, conducted by an AI agent under human direction, of the prompts and models behind Cau…
Grouping the second level
Grouping the second level when only the first level has a codebook A common way to code is to fix the first level of the label with a codebook and leave the second level free. The coder, or the AI, pi…
Intro – auto-coding with AI
Causal coding is fascinating but can take a lot of time. Using AI to help you is pretty easy, but there are still quite a few things to think about, as we will explain in this chapter. Two case studie…
Just add rigour Three do’s and don’ts
!500 AI in qualitative social science/img/map 050 just add rigour three do s and don ts add rigour.png Three do’s and don’ts when using AI for text analysis. A lot of evaluation work is a kind of text…
Research on the ability of LLMs to detect causal claims
Summary This note explains why modern Large Language Models (LLMs) especially since instruction tuned chat models often seem to have a “native” ordinary language grasp of causation: they can spot, gen…
Trust the algorithm, not the AI
I often hear concerns about algorithms and AI, in everyday life as well as in evaluation, taking over our lives or making us submit to decisions made by machines. The worry about losing control to mac…
Using generative AI for text analysis, rigorously
How to guard against some important threats. Steve Powell, Causal Map Ltd, 2023 10 13 (With thanks to Rick Davies for some comments) Summary As evaluators and researchers we often give instructions to…