PlumX Metrics
Embed PlumX Metrics

Do imposters threaten data quality? An examination of worker misrepresentation and downstream consequences in Amazon's Mechanical Turk workforce

Computers in Human Behavior, ISSN: 0747-5632, Vol: 83, Page: 243-253
2018
  • 44
    Citations
  • 0
    Usage
  • 86
    Captures
  • 0
    Mentions
  • 0
    Social Media
Metric Options:   Counts1 Year3 Year

Metrics Details

  • Citations
    44
    • Citation Indexes
      44
  • Captures
    86

Article Description

Amazon's Mechanical Turk (MTurk) is fast becoming the most popular online research platform, and as such, it is crucial for researchers to recognize its advantages and shortcomings. Here, we focused on the issue of worker deception and examined the downstream consequences of demographic misrepresentation in MTurk. In two studies, we asked: “Are we testing who we think we are testing?” and “Does demographic deception ultimately have an impact on data quality?” We found that in the presence of explicit eligibility requirements, an alarmingly high proportion of our samples misrepresented themselves in order to qualify for the studies (55.8% in Study 1 and 21.8% in Study 2). We also found that the nature of the downstream consequences of demographic deception varied across studies. Specifically, the scope of the impact may rest with the relationship between the demographic variable of interest and the outcome measure. In sum, the impact of demographic deception on data quality is multi-faceted, and a fruitful avenue of future research is to identify additional motivating factors that may underlie such deception.

Provide Feedback

Have ideas for a new metric? Would you like to see something else here?Let us know