Methodology

Methodology

Increasing Precision in Survey Experiments Without Introducing Bias

Scott Clifford University of Houston
,
Geoffrey Sheagley University of Georgia
,
Spencer Piston Boston University
Abstract
The use of survey experiments has surged in political science as a method for estimating causal effects. By far, the most common design is the between-subjects design in which the outcome is only measured posttreatment. This design relies heavily on recruiting a large number of subjects to achieve adequate statistical power. Alternative designs that involve repeated measurement of the dependent variable promise greater precision, but are rarely used out of fears that these designs will bias treatment effects (e.g., due to consistency pressures). Across six studies, we assess this conventional wisdom by testing experimental designs against each other. Our results demonstrate that repeated measures designs substantially increase precision, while introducing little to no bias. These designs also offer new insights into the nature of treatment effects. We conclude by encouraging researchers to adopt repeated measures designs and providing guidelines for when and how to use them.
Content
Thumbnail image of content item
Supplementary material
Thumbnail image of content item
Appendix
Appendix for “Increasing Precision in Survey Experiments Without Introducing Bias”
Comments
Log in using your APSA account or Register to comment
Comments are not moderated before they are posted, but they can be removed by the site moderators if they are found to be in contravention of our Commenting Policy – please read this policy before you post. Comments should be used for scholarly discussion of the content in question. You can find more information about how to use the commenting feature here .
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.