I worked in this field since long before LLMs. Nobody outside of the field really cared about GPT2, and even insiders knew the "too dangerous" part was a PR gag at best and the first dig of the moat at worst. After all, they released smaller versions of it along with detailed instructions on training it in the paper, so anyone with a lot of compute and a bunch of internet scrapers could try to recreate it. But basically noone did, even though it would have only cost ~50k back then (and less than 3k today). A few normal users started to take notice with GPT 3, but even then it was super limited. Even instructGPT didn't cause real shockwaves, despite being very close to the final product. Only ChatGPT/3.5 finally lit the fuse and people suddenly cared about having this too.
Since we’re doing anecdotes I definitely agree GPT2 lit the fuse. It woke up a sizable chunk of people paying attention. GPT3 is when I and many others got into a full blown existential crisis - it was the bang after the fuse. Then we got a long tail of laggards and people without vision. Even today you can find a significant chunk of folks in denial still.
Actually many of us do see that, and have been saying so for some time now.
I worked in this field since long before LLMs. Nobody outside of the field really cared about GPT2, and even insiders knew the "too dangerous" part was a PR gag at best and the first dig of the moat at worst. After all, they released smaller versions of it along with detailed instructions on training it in the paper, so anyone with a lot of compute and a bunch of internet scrapers could try to recreate it. But basically noone did, even though it would have only cost ~50k back then (and less than 3k today). A few normal users started to take notice with GPT 3, but even then it was super limited. Even instructGPT didn't cause real shockwaves, despite being very close to the final product. Only ChatGPT/3.5 finally lit the fuse and people suddenly cared about having this too.
Since we’re doing anecdotes I definitely agree GPT2 lit the fuse. It woke up a sizable chunk of people paying attention. GPT3 is when I and many others got into a full blown existential crisis - it was the bang after the fuse. Then we got a long tail of laggards and people without vision. Even today you can find a significant chunk of folks in denial still.
fair point