Survey item placement has little effect on data quality or treatment-effect estimates

Low data quality threatens the scientific merit of survey-based research. Researchers routinely screen out inattentive respondents at the start of a survey, but this does not guarantee that respondents remain attentive throughout. Researchers contributing items to omnibus surveys or deciding where to place key components, such as experiments, need to know whether an item’s position within a survey affects data quality. Existing evidence, however, is limited and inconsistent. Here we report five high-powered surveys (N = 15,159) that randomly assigned a range of data-quality measures, and a diverse set of survey experiments, to early, middle, or late positions. Position effects on data-quality measures were generally modest, mixed in direction and inconsistent across item types. Treatment-effect estimates remained stable across positions. Taken together, these findings indicate that where researchers place items and experiments within a survey is unlikely to matter substantially for data quality or for the treatment effects they estimate.