Aka - it’s a stochastic parrot with a good memory, for anyone still struggling to understand this. It should be obvious, imo, but some people seem to have trouble with the concept.
Aka - it’s a stochastic parrot with a good memory, for anyone still struggling to understand this. It should be obvious, imo, but some people seem to have trouble with the concept.
I don’t understand why this is such big news. OpenAI has essentially made a bunch of marketing copy by gussying up an algorithm being given a near unlimited budget to stochastically permute through its lossy memory.
I think the real scandal is that we are almost 3-4 years into this (I think the release of GPT 3.5 is a good marker of when this public frenzy started) and all we’ve seen is OpenAI and the other major AI frontier companies constantly retracting their preposterous claims every time. We appear to have reach a local maxima in that it has some value in places that tend to be a little easier to scope and limit (computer programming, mathematical proofs). So, given the actual useful economic value this has provided, does this justify the investments? I think we are approaching 1 trillion in CapEx for AI [0]. For context, I believe the annual GDP of Norway is $600 billion.
[0] https://www.fool.com/research/ai-companies-spending-on-data-...
It's crazy, you see someone spin up 50 instances of chatGPT or gemini or whatever to tackle a math problem, it succeeds by brute forcing through a ton of existing theories to find one that extends the problem, and the takeaway is that this technology is magic and going to solve all of our problems.
Whereas I see that and say - if we properly funded the sciences we could have had a bunch of grad students tackling that problem and found this application 20-30 years ago. Sure it's 'nice' that LLMs can fill in for people in brute force work like that but people are perfectly capable of doing that work and if we focused on properly staffing our research institutions we would achieve a lot more a lot faster. Instead this is obviously going to be used to replace staff and further reduce headcounts.