It’s kind of crazy they’re shutting this down just when this Service probably has the most possibilities ever. You have an agent with Multiple people doing actual physical tasks in the real world seems like something that could be really powerful.
It’s kind of crazy they’re shutting this down just when this Service probably has the most possibilities ever. You have an agent with Multiple people doing actual physical tasks in the real world seems like something that could be really powerful.
In spite of the name I think the overwhelming majority of the tasks were things that could be done by LLMs and they're probably getting flooded by people using bots to do exactly that. Even before the age of LLMs Mechanical Turk data was pretty bad because you'd have a bunch of people racing to answer questions as quickly as possible to get their $0.25 or whatever. So it was essentially a test of 'can you input random answers as quickly as possible while paying enough attention to notice the attention question that says to mark d.'
Any remotely open platform for this sort of stuff is going to wrecked by LLMs. But making some sort of high trust, high verification market is going to end up sending prices high enough that it becomes an unattractive proposition for use. And even in that case it's just going to be a cat and mouse game of people figuring out how to game the system enough to get trusted before handing it off to the LLM.
Seems like whole thing is destined to fail. Buyers of services are there to exploit and pay minimum they can get away with. And other side is ready to cheat if they can to get most out of it.
And same dynamic will apply to most cases. Unless you actually bring it in house with strict oversight. Which is thing to avoid originally...
The concept behind it isn't going away. There are plenty of companies that hire people in bulk to do manual data labeling, transcription, RLHF, moderation and lots more. It's just that they are now catering to large AI companies, not regular people looking to get some repetitive work done (since that can now mostly be done by AI).
Rentahuman is still around: https://news.ycombinator.com/item?id=46868675
Right? Not to mention the training/tuning possibilities.
Mercor has a market value of $20B basically doing the same thing but desperately trying to find workers.
that was my first thought too, but i guess llms don’t need an api to hire humans
[dead]