Are We The Baddies? We May Get The Data We Deserve

Much scholarship has debated the merits of samples from Amazon’s Mechanical Turk (MTurk) and similar online marketplaces. Prior research has highlighted important, valid concerns about participant attention, ‘cheating’, fraudulent data, trolling and the use of VPNs which challenge the validity of work using these samples. However, while researchers must take these concerns seriously, we may be simply getting the data we deserve. In this paper, I argue that it is important for researchers to also focus on ourselves, and we should adopt more ethical practices that minimize data quality concerns, including signaling the value of the data to participants with adequate pay. Towards this end, I provide recommendations about how to address some ethical and practical implications of low compensation for participants.