I am not going to argue this on the basis of LLM's suck at fiction, because even if it's true, it's not really that relevant. The problem is that what LLM's are good at is producing mediocre fiction particular to the tastes of the individual reading at. What people will keep reading is fiction that an LLM is writing because they personally asked it to write it.
I don't want to read fiction generated from someone else's ideas. I want to read LLM fiction generated from my weird quirks and personal taste.
There's more quality fiction out there than you or I will ever have time to read. I don't see a purpose in flooding the world with more mediocre to unreadable fiction.
Realistically, I don't think anyone will be spending hours here instead of reading real fiction anytime soon (I personally wouldn't). There's just so much nuanced complexity when it comes to creative writing as a domain (long-form outputs, creativity, etc.) that coming up with better annotation methods has massive applications in other research, like in scientific discovery. "AI Wattpad" just happens to be a convenient form factor for crowdsourcing from an HCI perspective. I hope you give it a chance.
OK, so you already recognize these stories aren't something that people are going to spend time sorting through. How could you possibly then get any usable preference data out of this?
If you look at similar live benchmarks like LMArena or Design Arena, there's an extremely large number of unique annotators, with a low number of annotations per person - which is normal. However, since this platform is designed to generate fiction catered to individual interests, my hypothesis is that it'll be an added boost of novelty that will help aggregate enough usable data over time.
I tried reading the two top rated stories. They're both unreadable gunk. Why would I (or anyone else) go for another? Why would I tell anyone I know to spend their time reading this?
Thanks for the feedback. What would you need to see to change your mind?
I am not going to argue this on the basis of LLM's suck at fiction, because even if it's true, it's not really that relevant. The problem is that what LLM's are good at is producing mediocre fiction particular to the tastes of the individual reading at. What people will keep reading is fiction that an LLM is writing because they personally asked it to write it.
I don't want to read fiction generated from someone else's ideas. I want to read LLM fiction generated from my weird quirks and personal taste.
There's more quality fiction out there than you or I will ever have time to read. I don't see a purpose in flooding the world with more mediocre to unreadable fiction.
Realistically, I don't think anyone will be spending hours here instead of reading real fiction anytime soon (I personally wouldn't). There's just so much nuanced complexity when it comes to creative writing as a domain (long-form outputs, creativity, etc.) that coming up with better annotation methods has massive applications in other research, like in scientific discovery. "AI Wattpad" just happens to be a convenient form factor for crowdsourcing from an HCI perspective. I hope you give it a chance.
OK, so you already recognize these stories aren't something that people are going to spend time sorting through. How could you possibly then get any usable preference data out of this?
If you look at similar live benchmarks like LMArena or Design Arena, there's an extremely large number of unique annotators, with a low number of annotations per person - which is normal. However, since this platform is designed to generate fiction catered to individual interests, my hypothesis is that it'll be an added boost of novelty that will help aggregate enough usable data over time.
I tried reading the two top rated stories. They're both unreadable gunk. Why would I (or anyone else) go for another? Why would I tell anyone I know to spend their time reading this?