LLM Persuasion in the Wild: Debunking Conspiracies on Truth Social
Building on the success of various survey experiments leveraging LLMs as a tool to durably debunk conspiracy theories and shift political, consumer, and health attitudes, we validate this approach on a naturalistic social media environment (Truth Social). The goal of this experiment is to demonstrate the feasibility of real-time, automated, scalable, and effective debunking on a target audience. We present the experimental pipeline and infrastructure, early field pilot results, and findings from a companion survey experiment. We also plan to discuss the challenge of measuring post-treatment outcomes in field experiments and introduce a method for predicting belief change from conversation data. We welcome input on outcome measures and results framing, as additional field pilots are currently underway.