AbstractPeople involved in project monitoring and evaluation of complex projects are familiar with what we call the ”time machine” problem: the things we want to measure (drivers, outcomes, intervening factors) may emerge and change unpredictably during a project’s lifespan and so cannot be fully specified until project end: but we need to know about them at baseline so we can design appropriate measurement instruments for tracking change.We demonstrate a novel workflow to help solve this problem which uses an AI-controlled chatbot to interview respondents, and then uses AI to code the transcripts and identify ”causal links” where stakeholders said that one thing influences another.We analyse the resulting causal information for differences across time: tracking evidence for emerging trends on emerging variables.The approach is reproducible, scalable and cost-effective. Further work is needed, especially to address the bias in the language models which drive the AI’s responses.

Evaluators are interested in capturing how things causally influence one another. They are also interested in capturing how stakeholders think things causally influence one another. Causal mapping, the collection, coding and visualisation of interconnected causal claims, has been used widely for several decades across many disciplines for this purpose. It makes the provenance or source of such claims explicit and provides tools for gathering and dealing with this kind of data, and for managing its Janus-like double-life: on the one hand providing information about what people believe causes what and on the other hand preparing this information for possible evaluative judgements about what actually causes what. Specific reference to causal mapping in the evaluation literature is sparse, which we aim to redress here. In particular we address the Janus dilemma by suggesting that causal maps can be understood neither as models of beliefs about causal pathways nor as models of causal pathways per se but as repositories of evidence for those pathways.