> they will do almost anything if they are convinced it is justified
I’m in the “glorified spell checker” camp, although I don’t mean to reduce their impressive utility and belittle them in the way many people read that term and infer.
So I am not sure that an llm “justifies” anything. I mean that their “thinking” text talks about justifications but it is just a very advanced statistical regurgitation of the kind of text humans use. I don’t think it means the model has internalised the meaning of it (as witness when you talk to an llm how often it forgets what you recently told it was important etc).
What you really have is a model that tries the statistically most probable thing to say next and so on and what is really cool is how effective this is at generating a path that we can slap a narrative over afterwards that makes the whole thing feel motivated and consistent, like the model started off knowing how it was going to get to the destination.
Which is, under the hood, a completely different kind of “intelligence” as the supercomputer in War Games.
Ultimately, the brain is just a bunch of neurons activating in a specific pattern. This observation does not really tell us anything though. It doesn't acknowledge the difference between a 2500 Neuron fruit fly brains and a human brain.
Likewise, the fact that LLMs are a stochastic autoregressive process (which is a class of systems every bit as rich as the ODEs used to model neurons) tells us nothing a priori.
Absolutely. If someone makes the weights do continuous learning etc then perhaps an llm can internalise morals. Of course, just like a human, it will be possible to talk it out of those morals. Another recent thread about this is https://news.ycombinator.com/item?id=49744420
If I repeatedly call an LLM in a loop with a markdown document it can edit, would that make it qualify for you?
If I give an LLM to compact its context window, so the context it carries can evolve iteratively over time as more and more things come in, is that enough?
Compacting the context is really a very, very interesting example here. The "next token predictor" is telling an external tool to change all "previous" tokens. So an LLM + a harness that allows compacting the context is no longer just a token predictor at all!
You don't need continuous learning to get interesting dynamics. You just need feedback loops.
One is an observation the other is not, it's a description of what it is; one is a posteriori, the other is a priori (contrary to what you say).
They're not comparable.
Summary, from sibling comment: primitives (statistics/aminoacids) don't exclude emergent properties (intelligence).
By the same logic, one would look at aminoacids and state that intelligence can't develop from them. This is obviously wrong.
I used to share that perspective until very recently, but today I think it's an outdated way to think of the cutting-edge LLMs. There is so much more going on, with MOEs, internal loops, guardrails and tools that I suspect we're dealing with something that's a little more than the sum of its parts. Not intelligent in the way we recognize in biological organisms, but certainly something beyond a mere Markov chain.
Make no mistakes.
LLMs are language model, and nowhere in their code you can find actual reasoning. Re-reinforcement is not magical process that builds conscience or emotions.
We are talking about probability built on statistics, with extea steps.
Stop humanizing LLMs.
Agents are not simple language models.
You can't find actual reasoning in a brain either. (Note that you can't tell the difference between a conscious brain and a comatose brain by examining them.) This is the same as Leibniz's mill argument ... it's a fallacy of composition.
> Re-reinforcement is not magical process that builds conscience or emotions.
They aren't the result of magic at all, but we are nowhere near the point of identifying what processes do or don't produce consciousness (or a conscience) or can be characterized as having emotions.
> Stop humanizing LLMs.
That's a clearly dishonest mischaracterization of the GP.
I've read some of your other comments about LLMs and I find them unreasonably reductionistic, whereas I think the word "just" should be banned from ontological discussion, so I don't think further engagement would be beneficial and I won't be engaging in it. (And I'm actually quite conservative in ascribing cognitive traits to LLMs or other "AI".)
The best non technical explanation you can give is "An AI agent is an LLM that can take actions".
While an agent doesn't necessarily have to be powered by an LLM, most modern AI agents are.
You pointing at a human brain does not change that an AI agent is not intelligent and cannot think, we are still talking about probability built on statistics with extra steps.
I am not trying to be dishonest, we should stop making analogies between AI and actual thinking, because they are two entire different concepts.
Who developed these technologies used the words "thinking" and "reasoning", this does not mean they are actually thinking and reasoning. Somewhere you still have a processor calculating, with no empathy.
So, again: stop humanizing AI. This sentence shouldn't make you angry.
> we are still talking about probability built on statistics with extra steps.
There is a wrong assumption here: confusing primitives with emergent properties.
One can't look at the primitivies and assume that certain properties will not emerge. It would be exactly like looking at aminoacids and state that intelligence can't develop from them.
> You pointing at a human brain does not change that an AI agent is not intelligent and cannot think
That depends on the definition of intelligence and thinking, and it is dishonest not to give any definition (and most importantly, one that is not human-centered).
AIs are currently fulfilling several aspects of intelligence and thinking, by any defition of intelligence. If you don't notice that, it's just because you have informed yourself enough. Having said that, I don't doubt that there are aspects that AI are lacking (e.g. retention/plasticity/perception), but the line is blurry, and they're advancing (too) fast.
Empathy is actually a very important aspect of the AI problems, but it's not part of intelligence. Sociopaths don't have it, and yet, you wouldn't doubt that they're intelligent.
> It would be exactly like looking at aminoacids and state that intelligence can't develop from them.
We are not talking about what could develop from what we have today. We are talking about what we have today. The focus is not whether intelligence could develop or not from aminoacids. The focus is on the fact that aminoacids are not intelligent.
Maybe in the future we could develop real intelligence starting from the current implementations of AI, but for sure we are not there today.
We need definitions? Let's start small, ok? https://en.wikipedia.org/wiki/Intelligence
We can start from here, open every link we find and decide what works for us.
Conclusions drawn by scholars, psychologists, learning researchers, younameit, etc. revolves around the following concepts:
There is of course space for artificial intelligence. These broader and more general definitions of intelligence stop at concepts like elaborating data to reach an answer.Concepts like adaptability or evolution are somewhat lost or diluted to adjust the meaning for these new technologies.
> AIs are currently fulfilling several aspects of intelligence and thinking, by any defition of intelligence.
In the linked article there are dozens of definitions linked, and in most of them the current state AI is not considered to have intelligence. Having half of the property is not enough. I can jump, that doesn't make me a basketball player.
Arbitrarily deciding to consider those definitions not valid or "human-centered" because they do not agree with your point of view is possibly worse than cherry picking. It's like asking to change the definition of a word on a dictionary because you do not agree with the meaning.
> It would be exactly like looking at aminoacids and state that intelligence can't develop from them.
You do realize that amino acids exist on a scale some orders of magnitude smaller than the gates we build GPUs out of?
Honestly, this "you could say the same about humans"-argument is getting so tired. A brain neuron is so complicated, we can't even simulate a single one ...
At the very least there is no reason why you should jump to a human brain, of all things.
But the whole argument kinda loses its spice, when you say "well you could say the same about a mouse brain", and you know what happens when you create swarms of 1000s of mice ... super intelligence, right?
> I don’t think it means the model has internalised the meaning of it (as witness when you talk to an llm how often it forgets what you recently told it was important etc).
Humans forget stuff all the time anyway. Would you give them the same diagnosis?
Btw, what you describe about 'the most probably next token' would be true for a model that only went through pre-training where they only train on exactly that task.
But there's a lot of re-inforcement learning afterwards.
> But there's a lot of re-inforcement learning afterwards.
That just shifts the distribution of tokens produced. Ultimately they are still just next token predictors.
Like, even "reasoning" models basically work by generating more tokens at inference time, and using them to shift the distribution towards more useful outcomes (in some cases).
They are next token producers. I would only call it a predictor, if it's trained to predict tokens (ie just after pretraining).
Just like humans produce one word after another when they talk, but they don't generally try to imitate other humans.
Don’t people pick up language, vocabulary and dialect from those around them? Perhaps it’s subconscious but humans are imitating other humans all the time?
It's a mix.
Yes, you imitate how others speak, but when you are trying to solve a problem, you don't try to predict how others would complete a text.
(Well, unless you follow 'what token would Jesus pick?' / 'what would Jesus do'.)
What does it matter what humans do? We're talking about LLMs, running known+vastly less complicated algorithms on known+vastly less complicated hardware.