Allie K. Miller
Amazon is shutting down Mechanical Turk in September. In 2023, a study from academic researchers in Switzerland found that up to 46% of AMT workers were using AI to complete their tasks. I can’t imagine that number got any lower, and I wouldn’t be surprised if 2026 numbers were 80%+ if the same study were to run again. I remember using AMT for data labeling a decade ago. We would have several labelers label/count each thing, and then look at consensus, variability, and risk of each thing before providing final labels. Unless these data labeling companies control the entire environment (ex: Scale AI opens an office that has human labelers come in every day), it’s a ripe opportunity for cheating. Just like asking kids to do their homework without any help from ChatGPT. We should assume that, even if they are asked not to do so or even if they sign a contract that they says they shouldn’t, humans outside of controlled environments will use AI to get something done if it gives them an advantage. For those of us who have been in AI 10+ years, this feels like a big deal. I feel nostalgic for easier AI times sometimes, and then I open up my Monday triage emails and Slacks from Claude Code and Codex and jump right back in.
