I disagree, because you can represent the constituent parts of any AI model as code and data. We can reliably build AI with this knowledge, but not brains.
Understanding does have layers, and that's why "poorly understood" is a meaningless goalpost. A book can be well understood without researching the gematria behind character's the names when you write them in reverse. An LLM can be well-understood even if you don't comprehensively test each quantization for miraculous unexpected behavior at the FFN level.
A book can be poorly understood if you know it's made out of paper and ink, but can't read. The contents would be recognizable but are meaningless symbols and numbers. You might note that some patterns of symbols repeat, but be completely clueless why or what it represents.
Notably, you could still print them all day.
Being able to put a PC together doesn't mean you understand computers. If I ask you to make me a computer that runs a wristwatch, you'll likely be lost unless you have very specialized understanding.
I think we actually all know what "poorly understood" means. There's no need to play tedious semantic games.
The semantic game occurred when someone was equivocating "LLMs are poorly understood" with "brains are poorly understood", which are worlds apart in the extent to which they are "not understood".
“equivocating” is not the word you meant to use here. Probably “equating” I suspect. But yes, I stand behind my claim and I deny that it’s a semantic game. The two areas are not identical in human understanding but they’re on the same order.
If I ask you what makes a good fantasy novel and you explain to me the English alphabet and grammar, you actually haven’t explained what I asked.
It’s likely that at some point we will also be able to represent a scan of the human brain digitally as code and data. Assuming we don’t separately make a huge number of leaps in neuroscience which are far from assured, it will be very likely that we can get the appearance of an operational human brain simulated digitally well before we gain much if any understanding of why it does what it does.
All that is to say, being able to build something is not not not the same thing as understanding it.
Your DNA is merely data, and humans can reliably make more of it too.