LLMs are bad at creative work and I don’t see them improving any time soon. Try asking an agent to write a story about raccoons. It will almost certainly involve either stealing food or raiding trash with a 50% chance of having a character named Pip. If a location is mentioned, it’ll be Elm Street.
I guess this is the average story and, similarly, the average game is boring and predictable
Ironically, they're getting worse at creative work because all the RL is collapsing their distributions.
Worse in a very specific, bland, uninteresting way, too.
Early generative AI at least had the virtue of relentless, unsettling weirdness, in the same way that generative art from the late 90s and early 2000s did. A handful of people made creative use of that spooky weirdness.
Now it turns out "Airspace" art.
Heh, tried it and got a story about raiding trash starring Pip. But yep, I'm well aware of this problem as well. /r/sillytavernAI is a sub dedicated to trying to coax LLMs into good creative writing but it's kinda impossible to do consistently.
I don't think this is bad. I want tools that deliver what I envision. The fact that given the same input, I get similar output is something I would rather consider a positive. The imagination should come from the human at the wheel.
While they are bad at creative work*, this reasoning isn't going to show it.
If you took the best, most creative, human writer in the world, and for thought experiment reasons they had amnesia (to mimic AI blank context windows) specifically while you asked them for a story idea 100 times in a row, my expectation is that this human would also give you the same idea at least 80 times out of that 100.
* still better than the mean human, but even the top 0.1% of humans aren't all professional authors.
Better than the mean human? The mean human creates far better stories while daydreaming. AI enthusiasts have such a distorted view of human capability, it’s bizarre.
The qualification "better" is doing some hefty lifting in your assumption.
What you think is better is not what I think is better. Imagination is not storytelling. You ask the "mean human" to write, it's going to be worse than an LLM in spelling and grammar, if you get anything at all.
Daydreaming is absolutely storytelling. The average person is able to craft elaborate stories on a whim. Writing is used, in part, for conveying stories, but writing is a separate skill. The good news is the average person can learn to write well, but LLMs, so far, have not demonstrated the ability to craft interesting stories and it’s pretty unlikely they’ll be able to learn, as this is an unverifiable domain.
The mean human gets a middling score in creative writing tasks at the end of their mandatory education, and most then leave school and forget what little they ever learned outside whatever their career path happened to be.
Most humans never do a creative writing course after school, and the longest fiction most people will write is their resume description of what their previous jobs involved, or perhaps their dating profile.
Don't mistake what you see published (or what your friends are like) for the average human: the average Hacker News comment easily above the writing grade (and creativity) of e.g. many of the one-shots stories I've seen attempted on some creative writing subreddits. "Mean" is not a high bar.
This is so absurdly condescending and simultaneously incorrect that it astounds me you’re able to function as part of a society.
Seriously, what is the deal with Pip! I was experimenting with childrens' fiction more than a year ago and 9 times out of 10 you'd get a character named Pip unless you were specific in the prompt not to do so.
I recently saw a study on this actually. https://arxiv.org/abs/2605.26492
I saw that, tested it in Qwen. Elias Thorne, on the first run, as a clockmaker too.
I think the trick is to use LLMs like scalpels instead of hammers. That said, you can place a scalpel in the hands of a deterministic (or not) robot.
Elias Thorne!