The really exciting thing in this article is that this research led them to a new technique for growing brain stem cells in vitro, which has been very difficult until now. If that isn't being oversold, it's big news. That's going to make future research into diseases like ALS a lot easier.
Agreed. This isn't being oversold.
And actually, the fact that the brain is actually two "developmental" systems combined into one seems like big news to me. We might not know its other uses, but probably for cognitive science and even AI research this might be super useful. In AI, maybe two models should be developed and trained in conjunction that talk to each other, each with different functions and roles, because brains work better this way.
And from what I read, evolution did this type of combined evolution somewhat often. The parts of the ear evolved from different "Gill slits" from when we were fish, meaning the ear isn't evolved from one group of cells in the embryo, but from multiple groups of cells that evolved into one organ system, the ear. Yeah, the middle ear evolved from a different "gill slit" from the outer and inner ear.
"The brain does it" is not in itself a reason to do the same in AI.
Sometimes, what the brain does is genuinely a good solution to a given task - one that's good regardless of whether your neural network is of artificial or biological variety. But sometimes, what the brain does is an evolutionary kludge, or a hack that works around one of the "being made of flesh" issues - of which there are a great many.
We have known examples of both - and a few known features that might go either way.
The brain does have some known useful features that we are yet to plunder - usually because we know they're there somewhere but not how they work. We don't know how the brain stabilizes online learning, for example. Or what low k-complexity priors and data augmentation processes does it use to enable its sample efficiency.
But a two-system split? Useless by itself. Splitting a network in two is easy - but if we don't know what that split does, what it buys us, what useful bias does it impart? We're just adding complexity. See: the investigation into HRMs, and how the "hierarchical" part proved to be a lot less meaningful than anticipated.
Eh, think you might not have read my comment well enough, because actually I partially agree with you in that two models may not be the best way to train an LLM. Yeah, I did say "In AI, maybe two models should be developed and trained in conjunction," meaning this is just some random idea based on the two system idea that might be useful.
Was just implying we don't know all the useful ideas (not just in tech but in medicine, genetics, evolutionary bioligy, mathematics...) that might be developed because we now know that the brain is two separate systems. It's all good though.
> "The brain does it" is not in itself a reason to do the same in AI.
What are the Tensors in LLMs modeled after?
That's the thing: we ripped the entire idea of learning connectionist networks straight out of the brain - but our implementation of that idea is very different.
The asynchronous nature of BNNs? The brain does that because its neurons are slow and synchronization is unnatural. The predictive Hebbian learning hybrid that the brain uses? Loses to backprop if you have what it takes to implement backprop.
There are a lot of mediocre or unapplicable ideas in the brain for every good one. Figuring out how the brain does the things it does is hard enough - but making use of that is even harder. It's why ANNs and BNNs diverge so much in form even as they converge in function.
> And actually, the fact that the brain is actually two "developmental" systems combined into one seems like big news to me.
It was known and discussed in evolutionary-developmental neuroscience, but not much outside that.
> maybe two models should be developed and trained in conjunction that talk to each other, each with different functions and roles, because brains work better this way.
I mean, is that not essentially what a Mixture of Experts model is?
Ah, didn't know the term for this, but vaguely remembered articles multiple models being trained together. Good to know:)
Not sure why this is such big news, it's been known for centuries that guys have two brains, and often do the thinking with the little one.
The article didn’t make it clear: the breakthrough is the discovery of a new third brain, beyond the two previously known.
Which one contains the soul?
Umm, as a man myself, I've got to say that I am always well behaved and have never gratified my sexual needs, ever! Sheesh, you neanderthal (haha:)
[flagged]
Could you clarify what you are trying to achieve with this comment? I work in a related field and talk to stem cell/organoid growers from time to time.
It is in fact hard to grow brain organoids from stem cells; this is still very much a thing that requires expert hands and minds to do effectively.
Unfortunately I'm not able to validate the second point (that the results of the research made a new technique for growing brain cells) because the paper is very technical in ways I'm not an expert.
No one but you said "non important."
How do you have such stunning confidence that you're replying to someone who knows 'nothing about' the subject matter? Maybe he's a neuroscientist.
I think YOU are the one showing stunning confidence in something you know nothing about (op's area of expertise. Unless you know them personally. Then ignore me).
easy there:)