The Promise of AI-Generated Video Designs for Survey Experiments in International Relations: An Experimental Test

Generative AI tools promise to make IR survey experiments more immersive by delivering treatments through realistic videos. But would these novel designs offer real methodological gains over traditional text-based vignettes? To address this question, we conducted a preregistered experiment testing the relative effectiveness of four “communication modalities”: an AI-generated video, long text with AI-generated images, plain long text, and a short-text vignette. Contrary to our expectations, the AI-generated video did not elicit the strongest affective responses nor the largest shifts in policy preferences; its effects were statistically indistinguishable from plain long-form text and even marginally weaker than the text-with-images treatment. All three long-form modalities outperformed the short vignette across all measures except attentiveness. The findings suggest that we should be cautious about the promise of AI-generated videos for IR experiments, whereas designs combining engaging text-based descriptions with AI-generated images may offer a cost-effective alternative to the traditional short plain-text vignettes.