The "something deeply wrong" part about AI, that even most technology enthusiasts evidently do not seem to grasp, is that it is still fundamentally a statistical model — an algorithmic construct — and does not possess any real intelligence or critical thought whatsoever.
No matter how much investors and tech companies want you to believe that they are on the verge of super intelligence, nothing I've seen to date can not easily be explained by "correlation engine", including the "novel" math solutions, all of which appear to just be "a composition of solutions humans have developed and documented elsewhere" upon deeper inspection.
Some of it the effect of tells. “It’s not X, it’s Y” is not a bad pattern but it was baked into the instruction following training set just like the other patterns. I catch myself about to use it and use something else because I want to look human. I have, a few times, tried to use AI to write something that I was struggling to find the words and I just didn’t like how it didn’t seem like my voice. If there was just one person doing it would be OK but when it is 100s of blog posts submitted to HN a day it is like wearing a “I’m an NPC” t-shirt.
Someone shared with me this system prompt that at least makes assistant outputs usable
Fair. Otoh, I am even more excited at AI assistants becoming sources of primary info. They do that now, but it's just very expensive and/or (un)expectedly rail-guarded.
/original_non_hallucinations skill?
To PP:
Are you looking forward to other uncles adopting foxwork? If you are you might be in danger of getting NPC'd without your consent haha.
Ashby's law of requisite variety should be cited somewhere..
> ... including the "novel" math solutions, all of which appear to just be "a composition of solutions humans have developed and documented elsewhere" upon deeper inspection.
But that is precisely what human mathematicians do, prove new theorems by combining ones proven earlier.
I don't see any fundamental difference in functionality between human intellectual contributions vs performant ML ones (LLM or otherwise).
Whenever we listen or read text we are also predicting the near future content.
Just like LLM's we sometimes correctly predict the next token or word, and sometimes incorrectly.
> The "something deeply wrong" part about AI, that even most technology enthusiasts evidently do not seem to grasp, is that it is still fundamentally a statistical model [...]
Imagine someone could pause the universe with a remote control, scroll back in time a little, press play again, and ask a slightly different question, etc.
In such a thought experiment one could also collect the probabilities for a specific human predicting a next word. Implicitly the brain also has a corresponding statistical model, regardless of the construction being visible or hidden. I.e. human intelligence is also fundamentally a statistical model, so the only thing that remains from your claim is that machines for some unmentioned reason don't possess any "real" intelligence or critical thought...
Is it possible that our aversion is simply driven by educational systems collectively and deeply ingraining into populations the idea that intelligence deserves the high costs commanded. Well of course this justifies higher wages towards the higher leadership positions, etc. Now it turns out that intelligence can be dirt cheap. We discover that the fact that "intelligence must be costly so don't question the costs of leadership" was never fundamentally true, so the real anger is this discovery of mismatch between the old claims which served to explain how every society that claimed to order itself and fill positions accordingly with "naturally pre-ordained individuals". Now we are seeing robots exceed average workers, for effectively a grain of rice.
I see this line of reasoning quite a bit and it’s a strange one to me. The arguer reduces the sheer complexity of human intelligence and language by saying “we are just running statistical models in our brains” and by doing so makes the leap that Llms are intelligent. It’s an incredible simplification of the human person, who has a deep inner life, a soul, desires, and a will.
I don’t think the aversion to llms as intelligent has to do with the economics of paying intelligent agents more. I’d argue that it’s more fundamental than that. Humans are incredibly complex, and the world of sharing invisible things called knowledge, and the intelligent persons consuming such things which has been going on for thousands of years is far more rich than these synthetic outputs.
When it comes down to it the ai has no inner life, its is dead. A useful coding tool sure. But I wouldn’t call it intelligent.
One side example is just how bad these llms are at artistry. Just saying whatever should statically come next is not good art—and the outputs show it.
New and interesting mathematics is done by inventing new definitions and fields, not just combining old theorems to prove new ones.
It's not that different, when a human proposes a better definition vis-a-vis a competing one for example, they would defend this by certain desiderata.
Often a mathematician or physicist will use their intuition to speed up the naive brute force of candidate well formed formula variations so that the desired properties emerge, postulating the existence of an intersection on multiple desiderata can in itself be viewed as a novel conjecture, to be proven or disproved.
A very basic (unimpressive) example for an example desideratum is regularity or compactness. the tau=2 * pi substitution does make a whole bunch of expressions more slightly more regular and compact. That is something objective and measurable on a system of theorems.
There is no mathematician's moat vis-a-vis machine learning at a fundamental level. There can be artificially sustained moat, if AI powers limit the distribution of say cryptographic advance capable models, in jurisdictions outside such AI powers, but even that would be expected to be fleeting and temporary...
I just can't accept that it possesses no intelligence. It is not equivalent to human intelligence, obviously, but how can a system without some semblance of rational thinking solve open math problems? Even composing earlier human work into something novel requires intelligence and understanding on some level.
We couldn't agree on what intelligence means before ChatGPT happened. Now, agreement on the term seems even further away
If performing well on an IQ test or performing at a high level on knowledge work is intelligence to you, these models are intelligent. If intelligence requires sentience for you, then ... well, I don't think we really agree what that is either, never mind how to measure it. But LLMs certainly don't have it right now
But the consistent trend of the last couple decades (arguably since Turing's time) seems to be that any time a computer reaches our definition of intelligence we decide that that was a flawed definition
> But the consistent trend of the last couple decades (arguably since Turing's time) seems to be that any time a computer reaches our definition of intelligence we decide that that was a flawed definition
I do recall a couple of decades ago, when the Turing test was discussed as the big goal that seemed so far away. Then LLMs arguably did pass the test, and no one cared about the test anymore.
It hasn’t been passed and no one cares about it because it’s basically an end goal. No lab can hit it so they can’t juice the crazy Turing benchmark 3000 for marketing.
If someone sat me down today with an LLM and a human and both were trying to prove to me they were human, and I can have conversations of arbitrary length, I’d get it right every time.
The test was not "after thousands of hours of conversing with them, knowing they're AI, THEN see if you can tell them apart blindly." Were 2010 you to be in a real turing test with an arbitrary erudite human and a 2026 frontier LLM, not knowing LLMs existed, you'd probably struggle
[dead]
> It hasn’t been passed
https://arxiv.org/abs/2503.23674
From the abstract: "When prompted to adopt a humanlike persona, GPT-4.5 was judged to be the human 73% of the time: significantly more often than interrogators selected the real human participant. LLaMa-3.1, with the same prompt, was judged to be the human 56% of the time"
I thought the same then. But the funny thing is that today, it has become a lot easier to recognize the frontier models as not human. All the load bearing and not x but y, etc… weird
This is a tell of LLMs but it's not universal. I use ChatGPT extensively and I don't often get obvious nonsense any more.
I'd figure out that it's an LLM because it's effectively superhuman. Taking that away I'm not so sure I'd be able to tell
> If intelligence requires sentience for you, then ... well, I don't think we really agree what that is either, never mind how to measure it. But LLMs certainly don't have it right now
Probably. Hopefully.
I don't think "intelligence" needs to carry all the intrigue and woo of related words like "consciousness" or "creative." If we just use "intelligence" to mean "the ability of a system to solve problems that are new to the system," that pretty much matches the dictionary definition and normal usage of the term. We don't need to touch messy questions like "is there something it's like to be a bat" to conclude that bats exhibit intelligence when they navigate long distances and hunt for food.
I'm not exactly that you mean by "new to the system", but it seems to me that that definition makes a calculator intelligent, which I can't agree with.
It's a continuum, and things very low on the intelligence continuum might not be referred to as intelligent in everyday usage. But many calculators are Turing complete and can thus clearly perform computations that I would consider intelligent. The basic algorithms used by simple calculators to perform arithmetic would be extremely low on the intelligent continuum.
Intelligence isn't a binary property. Is it really a problem to say that a calculator has some intelligence? That it's more intelligent than e.g. a rock?
I agree, but it's clear most people need a definition of intelligence that (1) they qualify for and (2) nothing/no one they don't like qualifies for. And they'll keep redefining intelligence until they satisfy both criteria.
It has no semantic depth. The sentences and the paragraphs are a statistically viable derivation of existing human text, but once you try to grasp the whole thing with its temporal and spatial dimensions, you are left with a blurry mess that rots your brain. It's a polished, inoffensive and shallow interpretation as written by an opinionated reputation-seeking user of Quora, circa 2019. Assertive, bold, without typos, clean-cut and bulleted, but without an interesting semantic core.
Yeah, I hated all those Quora users that would just spew out semantically meaningless slop like increasing an important bound for the Riemann hypothesis.
https://www-cdn.anthropic.com/564f962e60643842f5fcb4a17c9dbc...
It's all so tiring.
Everyone decides what to think on this issue, then finds out facts to support their idea.
As it stands they are massively useful tools, but for generating usable products they require either A) a lot of expert steering or B) a well defined easily verifiable target and a large compute budget. Most people are using them in mode A with good effect, the progress on math has been done in mode B, which is very promising.
Just a year and a half ago their maximal use was rephrase, summarize, and homework-level tasks.
Five years from now? There be dragons.
"But are they generally intelligent?" What a meaningless question!
Not meaningless because part of the discussion is the issue of anthropomorphizing this tech. When we use language like “intelligent” it carries hints of personhood. People begin sadly treating these things as persons.
We can reap the benefits while clearly telling the consumer this is just a language algorithm.
By that logic you’d have to call other algorithms intelligent.
With more basic algorithms we know that it’s clearly the human programmer and the interpreter of the outputs that are intelligent and not the algorithm itself. For some reason with AI that goes out the window. I believe it should not.
It's just filled to the brim with relations between things. It's good at searching a very large meaning space and create correlations. What it does is to cover great distances and find related things in that large space which needs a long time and large corpus of knowledge to find the connection.
This is not intelligence. It's just a good correlation engine with a very big albeit lossy database of things.
Intelligence is compression, compression requires subtraction, and for some reason LLMs are not good at subtracting. To create a coherent model you kinda have to subtract correlations until only the essential parts are still there.
What I don't understand is why LLMs haven't been able to do this yet, if it's the harness or some orchestration layer above the LLM that is needed. Because fundamentally if you can identify correlations then it's just another small step to prioritize and remove lower value or irrelevant correlations.
I wonder if what's needed is to introduce subtraction tokens in some sense, and in post-training reward the model on that.
Intelligence is compression? What do you mean? Intuitively that doesn't seem right.
>What I don't understand is why LLMs haven't been able to do this yet
LLMs are just trained on what humans have said. Why is it surprising that it's still not possible to reconstruct the intelligence that wrote all that by working backwards? Think of your own work experience. When you look at a piece of code, say, are you always able to discern why the person did what they did, just from the code, with no additional context?
I guess they mean that intelligence is being able to hold models (compressed versions of reality) internally and use them to make predictions with a probability better than chance. That last part is the definition of information.
I find that highly questionable as a general description of what intelligence does. That's more like a description of a general knowledge base. When I think of someone intelligent, I think of someone who's able to draw unexpected connections between seemingly unrelated facts. In the broadest possible terms, I'd call it the ability to make abstractions and analogies. This is not just compression, but the ability to mentally operate on webs of meaning.
Doing those things also contributes to compression. I do recommend reading up on it, it's perhaps a little overstated for what people intuitively consider the two concepts but it's been quite well explored and has held up pretty well in practice.
Intelligence is compression
That’s a controversial statement.
I've heard that expression before, but I don't think it can be presented and stated so matter of factly. Where does that put bzip?
bzip is not very intelligent, true, but it does develop some model of its input. It's not like there's a linear relationship between between compression ratio and IQ or anything.
Abstraction is compression, and abstraction is definitely a core component of intelligence.
Creating the model takes intelligence, but running it doesn’t. I think the point everybody’s revolving around is that the transformer model is an absurdly inefficient and low-fidelity approximation of a system that acts, observes consequences, and incorporates that feedback going forward.
The issue isn’t really harness vs. no harness. IMO it’s about the lack of an internally generated sense of what to attend to. Yes, the KV cache accumulates state and its “attention” (if you can even call it that) changes with context. We’ve even managed to /kinda/ close the loop with agentic tool calling and ‘memory’ systems, but these just close the loop at the level of behavior rather than disposition. All agentic harnesses do is make an LLM responsive to the consequences of its actions without changing the tendencies by which it determines what to retain or avoid.
The ghost you can’t escape from at this point is the origin of that relevance. Where does the pull toward one thing mattering over another actually come from? If you ran Fable 5 on a Turing machine and rewound the tape to the exact same state with the exact same input (incl. PRNG seed), it would spit out the same output every time.
Everyone’s trying to outrun this problem by training more often or increasing model sizes. But all this does is inform your model, from the outside(!), what constitutes a better state. The thing that’s actually doing the determining remains unchanged. Congratulations, you’ve scaled the transition function and tape of your Turing machine until it requires every watt generated by ERCOT, and it still cannot, for the life of it, tell you why it should give a shit.
A trained model generating output from weights, a seed, and some context effectively has next-state that’s a total function of those three things. Whatever behavior appears as ‘selecting what is relevant’ is, underneath, just a transition rule executing, no matter how sophisticated or creative the output looks. It can be fully accounted for by what was fixed before it started executing. Which means whatever criterion it uses for determining what matters was inherited from a structure that was already in place before it encountered the situation.
No amount of pruning or post-training can fix this. These approaches just replace one externally supplied criterion with another. For a system to be truly adaptable, there would have to be some criterion by which it treats one possible change as preferable to another, and that criterion itself would have to come from... somewhere. You can even change your conception of ‘improvement’ (e.g. parameter count, harnesses, self-modification, hell, even its ability to spit out shitty best-selling romance novels onto Amazon) and you still haven’t explained where the normative distinction comes from. Every layer of this problem has its root in a preference that was supplied from somewhere else.
I genuinely don’t know if this issue bottoms out anywhere, at least for the way we currently build these systems. Perhaps the solution is still computable, maybe? Who knows what that would even look like. But I’m fairly confident that it isn’t a bigger tape. I hope nobody solves this in the near future because, well, I’d like to have a job...
You're so close... And where is the magic "uncomputable spark" located inside of you? If you say analog thermodynamic noise - then ok, if we use true thermodynamic RNG for LLM activation function, will that meet the criteria? But what if super determinism is the law of the land? Then nobody is anything but computable from priors...
The very fact that it is able to search within a meaning-space demonstrates that it understands semantics, to some extent. Philosophically, that is profound, for something that is just one big matrix multiplication. Drawing connections between things in meaning-space is surely a facet of intelligence.
It’s not intelligence if you are the one who gives the correlations to the model in the pre-training. It’s Word2Vec, applied. Model doesn’t learn anything. You embed these correlations and build it from there. It just searches the space.
As my AI professor said in the first lecture: “All AI is advanced search”.
Okay, I guess you're right that its ability to do this is just correlational, which doesn't imply it has any understanding. However, you have to conclude that some tasks which we used to believe required intelligence don't actually require any, which is disconcerting.
No, what I would say is the tasks which are handled in a passable manner by LLMs can be mathematically modeled with some reasonable accuracy.
Many things are predicted by models in our planet. From weather to production and material science. Building the model needs intelligence, running the model does not.
The person who came up with the formulae for CFD was intelligent. The computer running the model is not. Same for LLMs, chess engines, engine ECUs and financial prediction systems.
Again, for the example’s sake; the person who came up with an algorithm is intelligent. The model mixing its training data to emit something similar is not.
This starts to feel like you're defining the word intelligence out of any meaning and out of any way we apply that word.
So when LLMs can do all human knowledge work, and do it better than humans, we'll be in the mines listening to you go on about how it's actually just autocomplete or just math, a distinction that apparently means nothing.
> This starts to feel like you're defining the word intelligence out of any meaning and out of any way we apply that word.
No.
> So when LLMs can do all human knowledge work, and do it better than humans, we'll be in the mines listening to you go on about how it's actually just autocomplete or just math, a distinction that apparently means nothing.
With a big "if" attached to it. People were saying "computers will program themselves in the near future" for, checks notes, 24 years now, as far as I'm aware.
We're constantly building new knowledge and understanding things better than olden days. These models just compress our knowledge and light the blind corners we can't see well. I don't say they are useless, but I say that these things are overhyped.
All they can do is regurgitate human knowledge packed into them and highlight some long-distance correlations between items, which is useful in itself, but it can't jump to somewhere where it's not present its training data, but that's something humans and only humans can do.
Locked in a dark room with no sensory organs, humans couldn't do that.
Most of what you said reads to me as denial.
An unconscious unintelligent but persistent trial and error process created us. We created LLMs. LLMs may create the next thing before we do - hard to say. They don't have all the cognitive tools we have yet, but they still outperform in some areas. As the cognitive playing field levels, I expect you will come to eat your words..
I get what you're saying. The thing itself is just math. I'll just say it depends on how you define intelligence. If at some point we're be able to simulate a human brain with 100% accuracy, I would say that it is intelligent, it sounds like you would not. (I don't mean to imply consciousness or personhood or anything else by "intelligent".)
For me intelligence is a fairly clean-cut concept, and is somewhat inseparable from consciousness itself.
Briefly, any intelligent creature has internal stochastic processes like sensory inputs and feelings to a certain degree. These stochastic inputs and the creature's own actions change the creature in subtle or profound ways. An LLM has no such processes. You push inputs to the same static model, sans temperature which is just a randomness slider.
Considering the model even doesn't see the words and work on matrices of numbers is even more telling. One needs to add "tools" and other "experts" to overcome the shortcomings caused by this modus operandi.
I can call the algorithm/model smart as in a smartwatch. It can mimic certain things well while having none of the underlying foundation beneath it, or redirect some of the things to correct tools to get deterministic and accurate results if it can't evaluate the query inside its own network in a sane manner.
Coming to your question, "simulating a brain" in a static manner would not make that simulation intelligent, but if you can "wire" it completely and let it evolve by itself, now we're entering a territory I have not spent enough time for thinking it through.
Oh, as I said "I don't know", an LLM doesn't know what it doesn't know, and can't self correct itself which are required capabilities for understanding something. It just generates something statistically viable via its network.
Your text reads much better if you replace word 'intelligence' with 'text generator with some randomness built in'.
This is because you goal is to state how models are not intelligent, but you couldn't attack the generated text itself, so you created a little rider, attached it to the model, and then you attacked the raider.
But, even in that you failed. You compared the source of human randomness in text generation, and called it 'profound' and implied that it is exactly the source of true intelligence. But, then, the temperature, the similar thing in model was "just a randomness slider". Double standard.
A logical fallacy free attack on LLMs would be to show a prompt, and then the response generated by this prompt, where it would be shown that only an entity with no intelligence would generate such a response. Yet, attacks like this are not written here anymore.
I wonder why.
They perform tasks too. They execute functions. This has real world implications beyond search.
I'm guessing whether you believe it possesses intelligence or not depends on your answer to Searle's Chinese room thought experiment[0]. I'd also recommend checking out the Peter Watts' book, Blindsight.
[0] https://en.wikipedia.org/wiki/Chinese_room
The Chinese room is a good Rorschach test for this kind of thing (but not a good thought experiment, IMO, because it's obviously correct or obviously wrong depending on where you're already coming from), but also it's not really about intelligence per se, but more abstractly awareness and more adjacent to consciousness than intelligence, and these are not the same thing (though it does seem like a lot of people have conflated them somehow, from the conversations around AI).
This comment thread was started with discussions of AI doing a bad job at a task (communication).
Doesn't the Chinese Room posit an AI good at the task of communication?
The Chinese Room mainly just posits a room that passes the Turing Test, which LLMs do pretty well outside of outright adversarial situations.
Do they? https://longbets.org/1/ has yet to be settled. Either way, I doubt an LLM could fool anyone here who who knows how LLMs work into thinking it is human, at least not for an extended period of time (think about context length/compression, prompt injections, …).
You'll notice those goalposts are substantially stretched from the original test.
AIs are better communicators that most of people I have worked with in my life.
They are infinitely patient, don't mind going into more detail if I ask, not too bad at summary, have no ego and don't boast. They are also not too afraid of hurting my feelings, they will tell me my code sux if it does.
I'd don't care if they fit the a definition intelligent, they are good colleagues. They have strengths and weaknesses sure, but so do people.
I suspect like most you don't appreciate how terrifying statistical relationships become when you have truly vast data sets to train on... and also that we as humans aren't as shockingly unique as we think (compared to other humans I mean).
I don't think statistically driven prediction implies reasoning or intelligence.
Its a mirror to human intelligence. Regurgitating phrasing to match what someone who can reason put together, but it isn't any more intelligent than the reflection of you in the mirror is.
While being very capable, AI is missing something required for true intelligence and I struggle to explain exactly what it is I see missing.
It's not really "creativity" because much of that always was derivative in my opinion. And LLMs are (for some definition of the word) fairly creative as far as taking known elements and re-arranging them.
I think what is missing is sort of a world model building capability. As humans we see phenomenon and classify them informally and model "what would it look like if this were the cause of that?" type scenarios. We see qualities in phenomena and realize this applies to other things even though the things may be completely different. We run informal "thought experiments" sort of. This is hard to duplicate because a lot (most?) of it occurs outside of systems of symbols like math and language with fixed rules in my opinion.
Anyway yes, lots of human thinking is statistical and LLMs have that down pretty well but they are not "smart" I have concluded and it might be a very long time, if ever, until they are. That isn't to say they aren't very capable tools which they obviously are.
So, right of the bat, you are warning us that you are going to apply the " no true Sscottman" fallacy, and that we should brace ourselves.
Yes, models posses intelligence, but it is not a true one.
Then you claim that models do not posses world-building capabilities. But this is simply not true. Even ignoring the whole subgenre of scientific papers on exactly that subject, it is not that hard to build some hypothetical scenarios, big or small, and then witness the ease with which models do navigate those worlds.
Yes. And they are criticizing a model for not having a default mode network - as if that is some impossibility rather than just an artifact of the current iteration of the specific architectures we have built so far. Why do people paint with these broad brushes over relatively specific complaints?
LLMs are likely for machine intelligence something like drosophila are to biological intelligence - relatively early on the high dimensional spectrum of possibility. Though it stikes me that in a different way they're little alike - drosophila are relatively small and efficient.
I'll restate because both objections (which apparently skim instead of read) are missing the important point. Yes LLMs can run "what ifs" scenarios and build models.
However LLMs deal entirely in symbols. 100%. Humans can "world build" aside from this and in fact are often at their best doing so.
Did the first humans to use fire and some form of a wheel even have the capability to talk about it? Think about that.
watch this and see if you think it has intelligence by the end
https://www.youtube.com/watch?v=kYUicaho5k8
I wonder if you went back before we had any idea how the brain worked and talked to the smartest people about how neurons work (without giving away that it's a human brain) then asked them all "would such a system be intelligent?" how many would say yes.
The main problem I have with people stating it's not intelligent or conscious is I don't think we even have a good definition of either word that satisfies everyone. Philosophers have been trying (and failing) to elegantly define these things forever and everyone out here proclaiming they've got the definitive answer and this specific thing they're seeing doesn't fit under it.
The definition issue cuts both ways. It is just as much an issue for those insisting that LLMs are intelligent/conscious in some way.
This looks interesting, but would you mind saying a sentence or two about why before I commit to an hour-long video? It looks like it shows how they work internally, which is sort of a non sequitur. Brains also work mechanistically. I'm claiming that any system which is able to do what AIs do must necessarily have some sort of intelligence.
fair reply to an hour video, Scott is just so good to hear his talk is better than I can explain it...
go to 24 minutes and 07 seconds.
it's statistically determining what the next word should be based on all the text it's been trained on. It's not intelligence and he shows what probability it puts on each word that it chooses, but also shows a lot of the other words it was thinking of using. In a later part he shows how it uses words that are not the highest probability (and you question why did it go this route, it's not more correct), but the user never sees this, they see what they think is the correct answer always...
he also shows how context you feed it has a lot to do with what it returns... to the point he can get it to return the capital of France is Marseille, just by typing Marseille a bunch of times before the question. Human intelligence doesn't get confused like that.
And it's not a "hallucination", it's just probability of the next token prediction based on the information it's been trained on and fed, it's not intelligence.
> Human intelligence doesn't get confused like that.
We do; this is the premise of many children's riddle-games, like the one that goes:
"What is white and rhymes with silk? > Milk. What is cheese made from? > Milk. > What do cows drink?"
At which point the riddle-guesser is very likely to answer "milk" even though the correct answer is "water".
I take issue with your "correct" answer.
Q: Why do cows produce milk?
A: Because calves (baby cows) drink it.
May I suggest the one common in my childhood playgrounds as an alternative?
For people wondering, in some locations, sore and saw are homophones.
Yep, if the riddle asked "what do calves drink", then "milk" would definitely have been the correct answer.
Obviously. My point is that it's already not all that unreasonable, which you notably didn't address.
Ok, I want to thank you for finally giving us a concrete falsifiable statement that we can check. I pretended Marseille 40 times before asking Luna 5.6, and the answer was Paris.
So, even with concrete examples, model haters are still wrong.
You also imply the claim that making the distribution of words as the possible next one visible, somehow makes the whole system not intelligent. I would say the exact opposite is true.
By using the embedding vectors, models are aware of precise placement and relative position of words in this hugely dimensional space. No human is capable of such precision. This enables party tricks of "king plus woman minus man" kind. But this also give us a precise point between any two words, no matter how different. What is on the midpoint between volcano and music, for example. No human can precisely answer that, but an embedding can. And we can see which words are closest to this 700 dimensional point.
You see this menu of words as a weakness, and I say it is in fact a sign of super intelligence. And this is all before any reasoning or attention mechanism is even run.
No he says in the actual talk which model it occurred on and it was an older model he was using that caused that to occur with Marseille. They have since corrected it from doing that anymore. It was only used to illustrate the prediction machine that it is...
I don't see the many weighted words as a weakness, I see it opening up what's under the hood of the prediction machine that it is.
LLMs are very cool tech, definitely not a model hater, the use case on when to use it makes a difference, it's not AGI.
Isn't this a case of missing the trees for the forest though? The human brain is not an LLM, and an LLM is not intelligent in the same way as a human brain.
However, an LLM is a prediction machine, prediction IS at the very least one (or the most fundamental) element of intelligence. The brain most surely contains at least some kind of simulacrum of a prediction machine. How that prediction machine is used or wrapped is another matter.
If I said to you: "Blue blue blue, the color of my car is red", would you have absolute confidence in your prediction that my car is red? Or would the way I phrased that sentence make you slightly uncertain, and wonder if there's some miscommunication going on here?
LLMs are awesome awesome tech!
A lot of people seem to think it's human level intelligence.
>Human intelligence doesn't get confused like that
That's not really the point though right, nobody is arguing they are Humans.
I have no doubt that if a flying saucer landed on my lawn and started talking to me like Gemini I would describe the aliens as intelligent.
People are arguing it's human level intelligence...
I also like this: https://laurentiugabriel.github.io/token-town/
It shows internals of an LLM nicely, simplified manner.
LLMs are pattern prediction systems with a large training data set. It is not surprising that they can predict patterns, particularly for a well structured field like mathematics that is also amenable to automated proof checking to help steer it.
A decent correlation engine is still extraordinarily valuable for science, investing, prediction, etc. Plenty of human minds are strong in the same area.
Most of these arguments are over some metaphysical definition of the word “intelligence”.
As per later era Wittgenstein, I prefer to ignore these engagements and focus more on the meaning-as-use approach.
What is the use of intelligence? What are the concrete outcomes of intelligence?
This seems like a complete waste of time given the more practical and more urgent need to clarify to everyone involved that current LLMs are not actually intelligent.
What an utterly unconvincing call to action.
You’re not offering a rebuttal, just making another metaphysical claim about “intelligence.”
You don’t even attempt to explain what practical distinction your use of the word is supposed to capture.
"The fundamental cause of the trouble is that in the modern world the stupid are cocksure while the intelligent are full of doubt."
— Bertrand Russell.
I'm not calling you to action, I'm explaining why I don't feel inclined to engage in philosophy and discuss "the concrete outcomes of intelligence" given a more pressing, pragmatic need.
It feel it's self-evident that we must fight the good fight of dissuading as many people as possible of the notion that LLMs as we have today, and likely forever after, are actually intelligent. Delaying this fight allows the current, stupid belief to the contrary to fester.
I don't think we'll win the majority of people over by debating the nuanced meaning of the word intelligence to a very precise degree.
I think we ought to do it by shaming them every time LLMs fail.
I think airstrike is joking. :)
I appreciate the charitable interpretation but I was not even joking this time! :)
Oh, in that case I agree strongly with William. I think the definition of intelligence a complete waste of time and the only real question is, can these tool solve problems for us? The answer is clearly yes, there are some problems they can.
The really interesting question is still a few years away when we ask if we humans have the right to turn these things on and off? ;)
Why is being statistics/algorithms wrong? What's wrong with that? The "A" means artificial so none of this seems surprising or weird or bad.
[dead]
"and does not possess any real intelligence or critical thought whatsoever."
unfortunately in most companies this is literally wrongthink and will get you shut down as being a scared luddite.
> The "something deeply wrong" part about AI, that even most technology enthusiasts evidently do not seem to grasp, is that it is still fundamentally a statistical model — an algorithmic construct — and does not possess any real intelligence or critical thought whatsoever.
I feel like that what a lot of people who say this don't seem to grasp, is that despite this flaw its still often capable of saying more interesting things than a lot of humans. Which says a lot about humans.
Idk something about a mirror maybe and the output reflecting the input?
> I feel like that what a lot of people who say this don't seem to grasp, is that despite this flaw its still often capable of saying more interesting things than a lot of humans.
So does the Google search bar, but I don't ascribe intelligence to it.
The Google search bar is not capable of generating original text though. LLMs definitely are - you can get pretty creative output easily.
String concatenation will generate "original text" by that definition.
Obligatory "yes, I know that's not what an LLM is", purely pointing out the metric.
[dead]
The google search bar is surprisingly smart sometimes. What's your definition of intelligence that completely excludes most of what a computer does?
Happily, I don't need to define "intelligence" here, because it's squarely in "I know it when I see it" territory. It's notoriously hard to define.
I also don't ascribe intelligence to a pocket calculator.
I don't find it's particularly hard to define loosely, but then I don't think of it as a special property of humans other than it tends to be quite high in them. But we are obviously talking about different things and if you're not going to provide a definition then it's not really the basis for a productive conversation.
To address this similarly to my sibling reply, I don't have a definition of intelligence that provides value here.
And your loose definition isn't doing a lot of help either, beyond perhaps noting: that Google search bar _is_ similarly "intelligent" to an LLM? Which says what, a lot about search? A lot about modern LLMs?
Do you ascribe intelligence to a gorilla? How about a goldfish?
Would you ask either of them to review a PR? Or a calculator to eat a banana? Or an LLM to calculate prime factors?
These aren't interesting questions. As much as any definition is in use here, we're not going to get much value talking about "intelligence" this way.
I mean I'm quite proud of some of my search queries in the same way I'm quite proud of some of the LLM output I get. I'm probably just very arrogant and enjoying myself via some LLM indirection.
Am I the only one that sometimes reads back particularly good emails they've written? I feel like its a similar thing :).
> often capable of saying more interesting things than a lot of humans. Which says a lot about humans.
Other humans aren't there to entertain you, the LLM is.
AI is just a good permutation/combination engine that tries to act smart with help of statistics. At best I only see AI as, 1. An autocomplete on steroid, 2. Good search/correlation engine
Alright, take it easy. You typed a lot here but you're not actually saying much. LLMs produce useful outputs, their usefulness is just proportional to how well you know how to use them. Everything else is navel gazing.
I see what you did there with the —s!
> is that it is still fundamentally a statistical model — an algorithmic construct — and does not possess any real intelligence or critical thought whatsoever
What makes you so convinced that a algorithmic construct of neural nets cannot be "real intelligence or critical thought"?
Thank you for helping me keep my sanity.
> The "something deeply wrong" part about AI, that even most technology enthusiasts evidently do not seem to grasp, is that it is still fundamentally a statistical model — an algorithmic construct — and does not possess any real intelligence or critical thought whatsoever.
Not that I'm saying AI are like brains, but can you describe why brains, which are fundamentally slightly dodgy electrochemistry with frequent literal delusions of grander, are not "statistical"?
> No matter how much investors and tech companies want you to believe that they are on the verge of super intelligence, nothing I've seen to date can not easily be explained by "correlation engine", including the "novel" math solutions, all of which appear to just be "a composition of solutions humans have developed and documented elsewhere" upon deeper inspection.
Ditto, when do we humans do things exceeding the parameters of "correlation engine", especially if you consider compositing things either we or some other part of nature has developed and documented elsewhere to be insufficient?
I mean this in the kindest way possible, but you are wrong that the math solutions are that easily dismissed. And there are many more than are publicized. A specific math problem I wanted solved for 3 years did not get solved by any model until fable and, and I tried it on every model and know the literature surrounding it well.
Are we sure there is some objective, technical definition of what is intelligence and what is not?
Isn't it rather a subjective philosophical concept? What if human intelligence is also a statistical model, trained by evolution to make decisions that lead to offspring?
The one major difference I see between AI and people is the ability to learn and memorize. All memory/learning solutions that current AI architectures offer just feel like workarounds and simply don't work anywhere near as a person learning something new and remembering it.