In some ways this is similar to those game demos people get the LLMs to build. When you say "build me a cool cyberpunk FPS" you get the FPS it can build, not the FPS the author wanted, or the FPS that is desired by players. It looks impressive but that doesn't make it a good game, or the game anybody actually asked for. It's demo porn.
In the same way if you tell an LLM to go and find an unsolved cipher it can solve, of course it finds the one it can solve out of the set of all possible ciphers. Of course it finds one that uses a one time pad that is public and referenced nearby in the text.
It's the same trick used by those people who film themselves throwing a basketball backwards into the hoop. You do it enough times and don't show the misses. You pick the best one to show. It makes it look like you're a basketball genius when you aren't.
It is of course, still a cool trick. Those videos are fun to watch, and so is an LLM solving a cipher. It is absolutely incredible to live in the timeline where you can tell a computer in plain language to go and find a puzzle on the internet and solve it, and it does exactly that. It's truly a mind boggling miracle.
The first principle is that we must not fool ourself, and ourselves are the easiest people to fool. (Ht Feynman)
Sometimes it hits all three.
>It looks impressive but that doesn't make it a good game, or the game anybody actually asked for.
The game I wrote manually hits 0/3.
Did you get anything out of the attempt?
Yeah, that's a great question. I should probably write about it properly, if only to structure my own thoughts.
The main thing I gained was the wisdom that comes from the pain of doing it wrong.
The most painful lesson was finding out I was more attached to the idea that I had a validated product, than to actually having a validated product. (And treating marketing as an afterthought... lmao)
Also if I had realized how long it was going to take, I would have taken a very different approach. I accidentally took my R&D mindset into production, which... might have worked, if I were immortal and retired!
In short, aim to ship 10x sooner, because everything takes 10x longer than you think. (Reality has 100x more detail than it ought to.)
On the bright side, I also learned to put one damn foot in front of the other, persist despite the horror, and finish the damn thing.
the game written by a human being specifically trained to write games hits all those targets and many more. it did take many years to train that human though, and that human did charge a fee for their game which took many days of labor to create. if nontargeted gratification was the goal, ai produced the better result more efficiently. hard drugs also more efficiently produce a widespread neural spike as compared to the effect of regular human activity. society only gives hard drugs to people who aren't efficiently productive though. what happens when everyone is given cognitive hard drugs?
One of man's deepest needs is for progress. Video games supply this need much more readily than reality does, thereby depriving the individual of necessary drive.
Hitting the target more effectively may be more harmful. We already had that problem before AI, though.
True, but also the AI can also be specifically trained. Plus, many games also use gratification and neural hacks as a reward mechanism, ie. loot boxes.
It can build a C compiler or browser if you don't know or care that it's a buggy collage of the dozen or so existing open-source C compilers or browsers, with no bearing on its ability to solve novel problems for your business.
In the months since those original stunts were pulled, the same playbook has been applied to the math and infosec spaces with some degree of success, and to the medical, legal, etc. etc. with less success. Nothing about the technology has fundamentally changed, but the labs hope that they can squeeze the same juice out of other markets that they did out of software so they don't have to delay their IPOs again.
That's true, what happened to the vibecoded browser?
The fact that Rust-Bun is in production does somewhat show it is possible, but I guess that has a lot of human effort in it too.