It doesn't have to be anything so extreme as novel work. The frontier of models still struggle when faced with moderately complex semantics. They've gotten quite good at gluing dependencies together, but it was a rather disappointing nothingburger watching Claude choke on a large xterm project I tried to give him. Spent a month getting absolutely nowhere, just building stuff out until it was so broken the codebase had to be reset and he'd start over from square 1. We've come a long way in certain aspects, but honestly we're just as far away from the silver bullet as we were 3 years ago (for the shit I care about). I'm already bundling up for the next winter.