Talking To Machines – Upcoming Synthetic Replication Games
Raymond Duch
In the context of a large replication effort of experimental work using LLMs as synthetic participants, the Talking to Machine team is organizing a series of one-day events dedicated to adapting the
original material of experiments published in top-tier journals to a standardised format which allows replicating them with a synthetic sample. The objective of the project is to systematically evaluate whether, and under which conditions, LLMs can be leveraged
to augment or replace human samples in experimental work. More details on the project and the first iteration conducted during IMEBESS 2025 can be found here.
We are inviting researchers from diverse backgrounds and at all career stages to participate. During the event, participants will form small teams of 3-4 members, based on their research interests. Each team will be assigned a study from a leading journal.
Participants will be recognized as co-authors on the paper that will present the aggregated results of multiple synthetic replications using a variety of LLMs.
Upcoming Games
- Friday, 24 Oct 2025 - Hybrid - Nuffield College, University of Oxford - Focused on field experiments
- Tuesday, 18 Nov 2025 - Virtual - Focused on experiments in behavioural economics
- Tuesday, 16 Dec 2025 - Virtual - Focused on experiments in political sciences
Games Structure
Researchers participating in the event will join a small team of 3-4 members with shared research interests. Each team will engage in the synthetic replication of one or more experiments published in a top-tier journal. Each synthetic replications will involve:
- The computational replication of the original results using the provided analysis script.
- Converting the original data collection instrument into a set of standardised experimental prompts.
- Introducing – and documenting – the prompt adaptations required to conduct the experiment with a synthetic sample.
- Running a pilot synthetic replication and benchmarking the LLM predictions with the results of the original study.
Registration
To register and find more information on each event, please head to the Talking
to Machines website.
We hope to see many of you!
Best wishes,
Talking to Machines
Raymond Duch
Senior Research Fellow
Nuffield College
OX1 1NF Oxford
mobile: +44 7 760 167 008