After reading Being and Time from Martin Heidegger, What Computers can't still do by Hubert Dreyfus, and some authors in cognitive linguistics (Langacker and Lakoff mainly), I strongly tend to disagree with any theory about emergent consciousness in modern or future AI systems, any theory proposing a similarity between AI systems and the human brain/mind, or any theory about the computational mind. What all these theories have in common is the underlying belief that our brain/mind works as the machines we build. Is the same underlying assumptions that treats cells as machines, our body as a complex machine. These theories are flawed in the sense that they cannot account for subjective experience and agency, amongst other things. The idea of 'internal models' and 'control loops' inside us is a projection of the aforementioned assumption.
There is also an epistemological assumption that prevails, and that is that we understand (or we think we understand) how our brain/mind works. But the truth is that we don't know. And there's even not a single clue that we actually know too much, and not a clue that our brain/mind and cells work 'as the machines we build'. Only by bypassing this epistemological problem, we can build 'theories of computational mind'.
These assumptions are there for already long time, to the point that when Turing asked himself 'can machines think?', he already assumed our thinking could be modeled as a machine.
I highly recommend people in the AI research space should read philosophy and modern linguistics. But not stopping at Descartes/Leibniz. Heidegger made contributions that cannot be avoided.
> These theories are flawed in the sense that they cannot account for subjective experience and agency, amongst other things.
On the contrary, it's precisely this assumption, that there is a "subjective experience" that requires explanation beyond the material, that is axiomatically assumed without evidence. It falls apart quickly, any "subjective experience" is completely tied to neurons, knock out the neurons and the subjective experience disappears, or stimulate the neurons to cause the experience.
There's two meanings to "the body is a complex machine" and I think you're missing the forest for the trees here.
1) The abstract "dictionary" version: It'd be technically correct to say that the body is a machine under the definition of "A machine is a thermodynamic system that uses power to apply forces and control movement to perform an action.".
2) But there's also the less abstract/technical: "The body is alike the complex machines we have built", and this is much less true. Especially for the brain. The "neuron" analogy in machine learning is effective, but entirely wrong; We do not fully know how even a single neuron works, nevermind any complex system made out of multiple of them.
With regard to AI, there's a lot of people extrapolating "There is no magical animating spirit, the brain is just a pile of stochastic molecules following the laws of physics" into "The brain is an inert pile of matter, computers are an inert pile of matter, ergo AI/LLMs are like the brain!"
Especially so by people who have a financial/legal interest in doing so. "AI is just like a brain, fire your employees and buy our LLM now!", "AI is just like a brain, so it's totally not copyright infringement!"
> With regard to AI, there's a lot of people extrapolating "There is no magical animating spirit, the brain is just a pile of stochastic molecules following the laws of physics" into "The brain is an inert pile of matter, computers are an inert pile of matter, ergo AI/LLMs are like the brain!"
Why do you need a specific organization of molecules for a phenomenon similar to consciousness to arise? Does anyone seriously consider a brain to be something other than “a pile of molecules following the laws of physics”? If so that’s not science or philosophy, that’s religion. You have a virtually complete phenomenological model of the universe for all intents and purposes and yet somehow the onus is on the person being like “hey no laws of physics are being broken ==> the brain is simply following the laws of physics”
How is it possible that people think of subjective experience and get rabbit holed into some mystical world where subjective experience is this special exception to everything else that is simply an emergent property of complex physical systems? “AI/LLMs are just like the brain” is a strawman, why does this claim need to be true for LLMs or any artificial system to be considered to have something akin to the thing you think of as consciousness? It’s more: consciousness is not some mystical or religious thing outside of the realm of physics, it’s an emergent property of a complex system. AI is a relatively complex system. We don’t really know or understand the relationship between the raw physics and again what we consider consciousness, so it’s simply a statement of “we can’t refute that these systems exhibit something similar” because we don’t know enough to refute that
> Why do you need a specific organization of molecules for a phenomenon similar to consciousness to arise?
They would have to be similar because the argument is that they are similar; "AI is like the human brain" only holds true if it actually is like the human brain, not merely a superficial resemblance.
What I'm describing in that prior comment is how a lot of people drastically simplify the resemblance in order to make it feel true; That the lack of a Jesus Christ coming down from the heavens to tell everyone they have immaterial souls that the computers don't makes the comparison more true.
> We don’t really know or understand the relationship between the raw physics and again what we consider consciousness, so it’s simply a statement of “we can’t refute that these systems exhibit something similar” because we don’t know enough to refute that
Therein lies the conflict: "You can't prove it's not conscious" is an unfalsifiable statement. You can't engage with the argument because it's proponents will always claim victory, often with their own interests at play. All concerns about "superintelligence" or the long term ethics of "when do our robots become sufficiently intelligent that they'd be slaves?" have been subsumed into the AI marketing machine. However sincere one might try to address the issue, they look like a Sam Altman stooge by association.
It's like claiming the quantum fluctuations inside a pet rock as a "consciousness", even if observed directly any measurement of random noise can still be dismissed as "nuh-uh it just takes billions of years to have a thought".
More practically with current AI systems, we can look inside them pretty well and there genuinely is nothing there. Standalone LLMs are purely feed-forward systems. Their failure modes show that they perform no meaningful thought or world modelling during inference. They're just language models.
The reasoning and agentic systems are even easier to introspect. We know how they work, we can look at the full prompts & context they operate on. There is nothing there.
This is what sets AI apart from animals, which are given the benefit of the doubt on their intelligence.
I think the point of the commentator above is that there are two extreme narratives that start each start with an uncontroversial assumption and then taking it to a pretty wild place. One narrative takes the assumption that brains are just matter so it should be possible to engineer consciousness and then argues that LLMs are conscious. The other takes the assumption that LLMs aren't conscious but then argues that because they aren't we won't ever be able to make anything conscious.
I don't actually think the commentator you responded to is arguing for either of these narratives and I thought it was a pretty useful way to look at some of these arguments.
> “AI/LLMs are just like the brain” is a strawman, why does this claim need to be true for LLMs or any artificial system to be considered to have something akin to the thing you think of as consciousness?
It doesn't need to be true but a lot of people make it/assume it.
There's a lot of, perhaps casual and uninformed, conversations that strongly imply a deeper understanding of the "physics" of brain chemistry than we actually have, mostly by comparing it to machines we've constructed.
(I believe) We don't need to replicate human neurons and dendrites and whatever else is in there in order to create a sapient "machine", but whether or not we've actually done that isn't being helped by arguing that what we currently have is all that similar to a human brain.
oh yeah! i recall a paper linked here not so long ago, where it was shown that the dendrites of a neuron do computations themselves. The "weight per neuron" is very simplistic then. At the very least, each actual neuron is a network of weights.
https://www.quantamagazine.org/neural-dendrites-reveal-their...
I'm partial to "modern ML weights are much closer to 1:1 capacity mapping to synapse count than to neuron count". A single biological neuron is closer to 100 or even 1000 weights worth of ANN than to 1 weight worth of ANN.
In which case: modern LLMs are still running in a capacity-starved regime!
Even Mythos 5, the 10-trillion monster LLM, the scaling law boogeyman, the harbinger of Vera Rubin NVL72, doesn't quite rise to 100T-to-1000T of synapses. Anything the light of today's AI touches still lives in the shadow of what evolution has managed to cram into a single human skull.
We're arguing about the limitations of AI while our best AIs are still very subhuman in the scale dimension. The one dimension we know how to push. And it's already this tight.
10T is about a crows worth. The mythos count doesn't include any diffusion model. But the crows count includes all its visual processing. And tactile. Touch uses up enough that they use skin surface area to normalize across animals when doing comparisons. It is one of the reasons suggested to explain how crows exhibit tool use and language with only 10T. We have a lot more skin than crows, and indeed far more than mythos.
> A single biological neuron is closer to 100 or even 1000 weights worth of ANN than to 1 weight worth of ANN.
Even those comparisons need to be cautioned. The complexity of biology is enormous, and more importantly yet, it's simply not comparable. And doing so invited a bunch of bad assumptions.
An ANN could quite probably model a single in vitro neuron with reasonable accuracy. Whether that requires a hundred or a hundred million nodes isn't terribly relevant.
But the way neurons combine in vivo is completely unlike the way machine learning systems are built. Both "locally" in how neurons interface which is vastly more complex than a weighted sum of inputs, and the macro scale interactions of hormones and other chemicals.
It's not even a given that large numbers of neurons will create the emergent behaviour of human intelligence; Elephants have significantly more neurons, but they're not the triple galaxy brains writing all our science papers. Other animal intelligence similarly is only loosely correlated with brain complexity. (Heck, not to be forgotten is the other end of the scale. Plenty of microscopic life that manages shockingly complex behaviour without any dedicated neurons)
This also applies to ANNs. There's no reason to expect that stuffing enough matrix multiplications into a program will make it intelligent or turn out conscious.
Really, the history of machine learning suggests the opposite; That the big gains are primarily had in architectural changes.
In this regard, I find the talk of the "limits of AI" quite credible. LLMs have already hit the diminishing returns on their growth, and even reasoning/agentic models display failure modes that confirm they're not "thinking" in the ways that humans do.
This is not to say that we've hit the final limits of what AI in the broad sense can do, it's just that the next advancement won't be "LLM but even bigger"
Not really. The history of "big gains" of machine learning is: put together a simple architecture that makes few assumptions but scales well. Then up the data and compute by 2 OOMs. By itself, the new architecture underperforms. Paired with the bitter lesson, however?
Don't make assumptions. Make a setup where the gradient descent can make them for you.
Empirically? LLMs are nowhere near "the wall". We've been hearing "the wall is nigh" since 2020. Six years in, we're still scaling LLMs, and the graveyards are full of "LLM killers". The system that kills the LLM is always a bigger, badder LLM, and never a new revolutionary architecture. The scaling doesn't just keep working - it works so well that it's seen as the only viable path forward at the frontier of reasoning and agentic work. Or even outside it. ChatGPT Images 2.0 is an image model with an agentic LLM at its core - generational gains in compositional capability.
For just about every "failure mode that confirms they're not thinking", you see one of two things. The first is that a new LLM releases a few months after and the "fundamental" issue abruptly goes away. The second is that we take a good, long look at a human, and find that the human also fails like this - and thus, "not thinking". Often both! Always funny when it's both.
One thing that's very biologically distinct is: local connectivity. In a GPU, global connectivity is cheap. In a brain, it's prohibitively expensive. The brain has no true backpropagation because it has no true global connectivity, and has to make do with local rules. A GPU is a strictly more expressive substrate connectivity-wise. So any point in the design of a computational substrate where you could remove complexity or increase performance by adding more connectivity? Silicon advantage. The brain isn't a "strictly better computational substrate" - it makes different tradeoffs. Which tradeoffs are better for attaining intelligence is an open question.
And, sure. Having a substrate with a capacity for intelligence doesn't mean having intelligence. No elephant has ever learned to code. The problem is: LLMs already did! LLMs already compete with humans on just about every task that was once thought to "require human intelligence". They don't always win - but they perform significantly above chance, and often above an average non-expert human.
So, my bet is on "LLM but even bigger". If there's a point where LLMs begin to lag behind and novel architectures get a sharp advantage, we are yet to hit it.
> For just about every "failure mode that confirms they're not thinking", you see one of two things. The first is that a new LLM releases a few months after and the "fundamental" issue abruptly goes away. The second is that we take a good, long look at a human, and find that the human also fails like this - and thus, "not thinking". Often both! Always funny when it's both. The way machines 'don't think' or 'fail' is fundamentally different from the way humans don't think or fail. In any case, the way LLMs learn and human beings learn is completely different. There is no actual clue that we are approaching any inflection point in machine 'learning'.
> So, my bet is on "LLM but even bigger". If there's a point where LLMs begin to lag behind and novel architectures get a sharp advantage, we are yet to hit it. We are already hearing this 'we are about to hit it' since the late 60s. The difference now is that the market is willingly investing insane amounts of money to make it possible. But again, there is no philosophical, theoretical, epistemological or biological clue that we are getting any closer to human intelligence level. What we did observe in the last decade though, is that we can build enormous machines that can statistically mimic statistical human outputs. Language and images being some of them. But that is not thinking.
First, fix your formatting. It's a fucking mess.
Second, what is the difference? Is it that one thing has an immortal soul, and thus Actual Intelligence and Actual Reasoning and Actual Learning, and the other has no soul, and a Pale Imitation of Intelligence, At Best?
Because I've seen versions of this "it's not actually thinking" for actual fucking years, and the difference between "actually thinking" and "not actually thinking" always seems to boil down to "I don't want LLMs to be actually thinking, so I will bend the definitions and twist the qualifiers and move the goalposts until they aren't". No one ever made an ActualThinkingBenchmark on which humans score 100% and LLMs score 0%.
Nothing but human insecurity, in my eyes. There was never a principled difference. Just a way to operationalize some "I'm unique and special and better than a matrix math machine" vibes.
Agreed, formatting was kind of f, but there is no need to be rude.
I wasn't saying there was any difference. All I'm saying is that the claimings the AI research field does are based on false assumptions. And from false assumptions, you cannot reach a proper conclusion.
Whether an AI system can reason and think like if it where a human being, or not, I don't care. I'm fine with either: it is just technological advance. But making claims based on false assumptions, and then being fooled by how 'wonderful' or 'spectacular' the results are, is, at least, naive.
> Nothing but human insecurity, in my eyes. There was never a principled difference. Just a way to operationalize some "I'm unique and special and better than a matrix math machine" vibes.
This is just something I don't get. People ignorant of technique are insecure and afraid. People that know how technology works, and thus investigate and know how it works fundamentally*, were never afraid or insecure.
I agree, we can miss the forest for the trees.
1) This definition could actually be expanded (for example, with definitions from Mumford or Reuleaux). But still this definition cannot be applied directly to living organisms. 2) This is in my opinion one of the sources of misunderstanding. We mainly operate on analogies and metaphors, so we have build this 'analogy space' around the idea that living organisms are machines. But it is only when we say 'alike' that we can truly gather some meaning out of it all, going beyond the 'behaves like' or 'is conceptualized as' when it gets messy.
> With regard to AI, there's a lot of people extrapolating "There is no magical animating spirit, the brain is just a pile of stochastic molecules following the laws of physics" into "The brain is an inert pile of matter, computers are an inert pile of matter, ergo AI/LLMs are like the brain!"
This is exactly my point. There is a fallacy operating from "A is not B" to "A is C". And this fallacy is pervasive in the AI research field, the book from Dreyfus (What Computers can't still do) explains that in much detail.
> 1) This definition could actually be expanded (for example, with definitions from Mumford or Reuleaux). But still this definition cannot be applied directly to living organisms.
I'm not sure I understand this. Why not?
It has to do with words and how we evolve words throughout history and across geographic boundaries. The term 'machine' comes, after some modifications, from the greek word mekhanos, which was used to describe something ingenious or a device made in some clever way or operating in a clever way. From there it went on to describe things like devices, to end up being the actual definition of what we might call 'a device' (a machine). The idea of 'mechanistic' is also related.
Traditionally, things that are alive were described with different words and assigned a different set of properties and characteristics. Machine can break, living things die. And we still have those two semantic frames separated: A living thing: can be harmed, it breathes, it nourishes, it reproduces, etc A machine: can break, can be fixed, can be repurposed, etc.
But because of a specific tradition in western philosophy, we started applying and analogy between 'inner mechanism (clever thing), that moves or provides a function, and seems to work in a causal way' and living things.
So when we say 'a living cell is a wonderful, complex machine', we are not actually saying it is a machine, we are operating through an analogy. That's how far we can go.
I find the claim subjective experience may be illusory absolutely baffling. I can only speak for myself with certainty, but I am entirely sure I have subjective experience. All the other propositions I believe could be false but I don't see how I can be wrong about experiencing something. I could be a brain in a vat (or weights on a GPU) and be specifically programmed to only come to false beliefs and still I can be sure that there is an experience I am the subject of. I cannot provide empirical evidence for my experience, that is why it is subjective. I cannot be entirely sure you are experiencing anything, and when I encounter people who don't share the same baseline intuition here I do begin to wonder if this is truly a universal across humanity. But I can't think of any other assumption which I would be more comfortable as a foundational axiom other than, "I am experiencing something." I do not require additional evidence for it because I experience the truth of it directly
My current understand that "subjective experience" is a post effect of memory forming in the process. "I experience X" ≈ "I remember that I just recently received [external stimulus / interpreted my current state as] X".
And I am as well baffled why people make such a big deal out of "subjective experience" and "consciousness".
I was joking that maybe I miss this properties, but now starting to really wonder if it might be the case. What if these phenomenons are present in humans to various extent? Check aphantasia. Only in XIX we discovered, that ability to visualize mental images is not universal, available to different people to various degree and some people completely miss it. My ability to visualize is weak. What if "consciousness" and "subjective experience" are similar?
And I am slightly worried when I am writing this that it might turn to be truth and in ~20 years I will be treated as "inferior human" without complete set of human rights.
Indeed. Even positing an illusion seems like a contradiction. If it's illusory, doesn't there need to be a subjective entity experiencing the illusion?
This proves too much. As it would imply parrots have sapiencee.
Why do you think that parrots do not have sapience? How would one even measure such a thing?
Because by definition, sapience is something only humans have. Ergo, parrots are not sapient.
More meta, all of the threads on this page are just people playing games with definitions. Eg, “qualia is something I have as a human but machines don’t have it. Therefore, LLMs do not have qualia.”
You're confused about the etymology. Homo Sapiens was coined in the 1800s. People have been saying "sapient" since the 1300s, and it is rooted from the Latin word "sapientem" which simply means "sensible; shrewd, knowing, discrete". Homo Sapiens just means "wise human", and we humbly bestowed the name upon ourselves.
https://www.etymonline.com/word/sapient
https://en.wikipedia.org/wiki/Human#Etymology_and_definition
> More meta, all of the threads on this page are just people playing games with definitions. Eg, “qualia is something I have as a human but machines don’t have it. Therefore, LLMs do not have qualia.” True. For me, the actual interesting debate is not if LLMs are intelligent or not (easy to dismiss) but to what extent LLMs embed into our socio-techno-economic reality.
I think the truth lies somewhere between these two extremes. An LLM is not a human brain, and does not try to emulate one. It should not be a surprise that an LLM does not behave like a human brain. So we can not infer things either way. The best we can say is that an LLM appears to exhibit very similar behavior to a human brain, under certain constraints. So maybe we can infer that the human brain has something in it that operates in a similar way to an LLM (like the human "unconscious", or "intuition" maybe). It seems obvious to me that a human brain and an LLM are not comparable things, for many reasons. So we can not make inferrences one way or the other.
> An LLM is not a human brain,
true
> and does not try to emulate one
citation please.
something like the universal approximation theory comes to mind, transformer architecture clearly has the shape of a universal algorithm approximator
I think you're right in that it has the shape but I think it's missing a pretty key piece. We still haven't been able to solve catastrophic forgetting, yet everything with a brain has. Basically LLMs seem good at approximating intelligence on a moment-to-moment basis, but feel quite far away when you chat with one over time.
Like at some level, yes, transformers are trying to emulate a human brain but the second you ask folks if they do a good job of it, I think most rationale people would say no.
> something like the universal approximation theory comes to mind, transformer architecture clearly has the shape of a universal algorithm approximator
That says nothing about emulating a human brain.
is human cognition not an algorithm? is it just woo? or vibes?
But what is the shape of the algorithm of the human brain? It has a complex physical structure. We know the folds on the surface are important, but why is that shape specifically important? The brain is made up of two hemispheres - why, what does that do? There are different "types" of brain inside the human skull. There are physical areas that perform specific tasks. There are different types of neurons. Then there chemicals that interact with the brain, changing how it function depending on things happening to the body. All that stuff and more is the "algorithm" of the human brain. It's not the same algorithm as an LLM.
Interesting, so you would say that your experience is .. illusory? In what medium exactly? Illusion requires a substrate of some kind. "Awareness"? What's that?
Neurons are themselves things we experience (indirectly). Once seen through a microscope or known about in some fashion the only way they "exist" is by you having the experience of knowing them. It's not the other way around. One thing is more fundamental here. What is this experience? What are the atoms of this? "Atomic particles"? How would you even approach an answer if your building blocks are themselves part of what needs to be explained?
The hard problem cannot even be touched if you start out like this.
Refuting the "subjective experience" axiom does not lead to 'any "subjective experience" is completely tied to neurons', you also need to explain why the subjective experience is tied to neurons. And that's precisely what computational theories of mind do not account for: the link between subjective experience and neurons. I am not arguing that neurons (or the brain) are not a necessary condition for subjective experience.
It’s the opposite.
Descartes made clear that subjective experience is the ONLY thing we know. Everything else is theories to explain the phenomena we subjectively experience.
We theorize that there is a physical world and other beings like us having similar subjective experiences, because that seems the best explanation for our subjective experiences. But we might be living in the Matrix, with all the people we think we are interacting with and just sophisticated simulations.
And this has some bearing on the debate about whether these systems do or could in the future exhibit something similar?
> at requires explanation beyond the material, that is axiomatically assumed without evidence
Nobody talked about anything out of neurons. The question is still open.
"that is axiomatically assumed"
passive voice doing a hell of a lot of work in this phrase
Why is asking 'can machines think?' assuming our thinking could be modeled as a machine? It's raising the possibility that our thinking could be modeled as a machine. Given that, as you acknowledge, we don't have a clue how our mind works, it seems premature to rule out this possibility. Rather, I would say that ruling it out betrays an assumption that we understand how the mind works enough that we can say that it is definitely not replicable by a machine, and that assumption seems unjustified.
Indeed, scientific type thinking like that where you ask a question that you can do experiments on to see if machines can pass the test is probably the way to progress. The philosophical waffling mostly just goes in circles.
To investigate consciousness you'll probably get further trying to build conscious machines with agency and comparing them to biological ones than reading Heidegger.
> It's raising the possibility that our thinking could be modeled as a machine. Given that, as you acknowledge, we don't have a clue how our mind works, it seems premature to rule out this possibility.
In order to raise the possibility that our thinking can be modeled as a machine, there needs to be a previous question: 'can our thinking be modeled at all?'. And before that, already the idea of possibility: 'our thinking could be something that can be modeled'. Since we know 'we can think', asking 'can machines think?' needs the assumption that machines and brains are alike. If there is no assumption, then we should ask 'if brains and machines are alike, then we could raise the possibility of thinking a machine could also think'. But when we say 'brains and machines are alike', we are implicitly saying 'brains are like machines'. There is no problem asking 'can machines think?', but there is indeed a problem with implicitly assuming that brains and machines are alike when we do not known it yet. I am critiquing the idea of assumptions here, not the research.
If we agree weights producing text may emerge consciousness, given large enough, then DBs must've gained it long ago. The whole idea of sentience "on the other side of the wire" is wrong, as there is neither wire, nor other side. We think there is, but the DB just expounds the query and repurposes this information into views, that we give notion and meaning to. LLMs are the same, do not be mistaken.
One other important thing to consider is that the human experience is thanks to the body, is in connection, and perhaps product of the body. The body is observable and perhaps humans may state that they feel the connection to it. LLMs have no notion of nothing, the machine does not know the result, and the result does not know the VM. Modern psychology more or less has settled around the idea that consciousness is a product of the body. Why and how does this construct come to realize a Self is then another mystery even if we know which parts of the hardware may be involved.
Whether it is the Holy Spirit or Life Force animating the human body is a completely different question also. Besides, the realization, the experience we have now with all life in 2026 is not something that can be easily explained or attuned to life 200 years ago and its terms and notions. So is also wrong to even attempt to.
> If we agree weights producing text may emerge consciousness, given large enough, then DBs must've gained it long ago.
If we agree that silicon can perform calculations, then beaches must have been working out log tables long ago.
Greg Egan wrote an entire (fabulous) book about exactly that, "Permutation City".
Diaspora is mind-blowing, yet highly improbable and speculative, even though carefully threaded to sound plausible on all levels. The whole introdus idea and simulation from VM perspective sounds incredible, but I don't think the body runs a simulation of anything. It is something else.
working out things,... is not calculating.
a la https://xkcd.com/505/
> Modern psychology more or less has settled around the idea that consciousness is a product of the body.
The kind of consciousness we know. Jumping to the conclusion that that's the only kind possible, or even stronger that the way ours evolved is the only way this could have happened, is completely invalid.
[dead]
>Is the same underlying assumptions that treats cells as machines, our body as a complex machine. These theories are flawed in the sense that they cannot account for subjective experience and agency, amongst other things
Do you disagree with the assumption that cells are machines? They seem pretty machine-like to me. I certainly don’t think individual cells have any subjective experience or sense of agency. I would be curious to know where your intuitions diverge here, because if the mind is an emergent phenomenon from machines (cells) then it seems quite likely that a mind could emerge from other, different machines.
It's not a huge leap to imagine biological cells might have a rudimentary consciousness
Pan-psychists might argue that your subjective consciousness is an aggregate of all the cells/molecules, etc in the system
"While each biological cell operates largely on its own chemical cues, they all coordinate through complex nervous and chemical networks to create your unified, subjective experience."
You might be a rigid materialist
https://pmc.ncbi.nlm.nih.gov/articles/PMC10883262/#:~:text=T...
https://www.mdpi.com/2409-9287/2/4/26
> Do you disagree with the assumption that cells are machines? They seem pretty machine-like to me. You just said it: they are machine-like. No, cells aren't machines for two reasons: 1) machines, by definition, are artifacts created by human beings. 2) The nature of a living organism is completely different from that of 'machines'. (even if we a re able to replicate a cell in a lab, like the group from Craig Venter did). Autopoiesis being one very big difference, another being emergence-within-the-environment (life) vs design-conditioned-by-will (machines)
> ... if the mind is an emergent phenomenon from machines (cells) then it seems quite likely that a mind could emerge from other, different machines. Since cells cannot be defined as machines, the argument about mind emerging from machines does not hold.
> I certainly don’t think individual cells have any subjective experience or sense of agency.
There's definitely research and scholarship that would beg to disagree with you there. At least in terms of completely writing off the notion of "agency" when it comes to cells.
Dr. Michael Levin's lab is doing some pretty cool work. https://drmichaellevin.org/
> Do you disagree with the assumption that cells are machines?
Yes? Literally no machine ever built by humans is capable of (or even hinting at beginnings of capability for) replication or novel synthesis like cells are, let alone autonomously, it’s quite unconceivable that anyone would take this to be a reasonable assumption in the first place.
Hmmm, well let’s take it one step lower. What do you think of organelles such as ribosomes? Do you disagree with the assumption that those are machines? They seem directly analogous to the jacquard loom or a CNC machine to me.
Yes again, ribosomes have nothing in common with machines, that are built and designed by humans.
The ball is in your camp to provide solid reasons to believe why they should be grouped together, when one is a deeply complex interrelated dynamic system (in fact, arguably the most complex system we know of) evolved bottom up over billions of years that we only very partially understand and cannot fully explain or document, and the other something entirely planned, designed, and produced by humans in which every component is finite and accounted for.
The argument boils down to “well the vibes kind of match to my taste, and it’s the best analogy I have in my analogy toolkit”, which is just not serious reasoning.
I think ribosomes have a lot in common with machines. They use energy to accomplish a task (assembles proteins). This would seem to put them in the same category as artificial molecular machines like rotaxanes. I don’t think there’s a huge gap in our understanding of how either systems identically functions independently. Yes, ribosomes exist as part of a larger context, but they can be removed from that context pretty easily and understood as individual molecules quite extensively.
In your view, can machines even exist that haven’t been created by people, definitionally? I, personally, don’t see the relevance of intent but that seems to be the only distinguishing factor here.
replication: https://en.wikipedia.org/wiki/Quine_(computing)
autonomous replication: https://en.wikipedia.org/wiki/Computer_worm
nb that writing your own quine remains in general terms a fun and challenging exercise in many programming languages, but not python.
We are talking about physical replication, please let me know when a computer worm can turn my laptop into 2 laptops
Otherwise you’re just arguing that Sims are totally alive because Sims can make baby Sims.
The assertion is not that cells are machines built by humans (obviously false), merely that they are machines. Which they pretty clearly are, unless you assume the supernatural.
There is no need to assume the supernatural. They are machines based on what? By definition, a machine is something build by humans, or any intelligent life form (because we extend the definition from our own experience as self defined 'intelligent life forms'). Living organisms and machines do share traits under our current frame of reference in modern science. But that doesn't mean cells are machines. The right framing would be 'cells exhibit behavior and certain internal relations that we can conceptualize them as machines', which is different from 'cells are machines'.
By what definition?
Anyway, seems like an argument over said definitions rather than the underlying characteristics. The relevant question is whether they're purely physical objects behaving according to rules, which is being described as "machine," or whether there is something beyond that. Current understanding is contradictory: all indications are that cells and bodies are purely physical objects, except that there is this phenomenon of subjective experience which doesn't fit with that at all.
> By what definition?
We have this definition from the history of western languages, the history of western philosophy, and works by Lewis Mumford and Franz Reuleaux, among others authors.
> The relevant question is whether they're purely physical objects behaving according to rules, which is being described as "machine," or whether there is something beyond that. Current understanding is contradictory: all indications are that cells and bodies are purely physical objects, except that there is this phenomenon of subjective experience which doesn't fit with that at all.
Then you would say that you dog broke (died), or that the vet fixed your cat (cured). Which by all means we might speak that way, but surely you would notice that it is not accurate.
Saying that a living organism is just a machine because it is a physical object behaving according to rules is like saying that a beautifully built house is just a bunch of bricks layed in rows and something on top.
But assuming that we say 'purely physical objects behaving according to rules' are machines, then:
1) there is no difference between you, your dog, your fridge and the snail in you garden 2) It would be semantically valid in all languages to say 'my dog broke' instead of it died, 'i got fixed' instead of 'the doctor cured me' 3) Machines vary in complexity and we would still have 'degrees of complexity' regarding machines (human bodies being the most complex, perhaps, toasters being fairly simple), but fundamentally, all of them would follow the same rules for 'fixing', 'breaking' and 'repairing'. Which is not the case. 4) You would have to come up with some kind of theology regarding how we were built. There is no evidence that we have been 'built', quite the contrary.
And most probably there are quite more reasons not to regard machines as living organisms nor living organisms as machines.
What exactly is there about cells that's inconceivable to replicate or synthetize? "I cannot conceive it" is a fallacy, an argument from incredulity. Trying to disguise it by draping it in the passive voice does not change that fact.
Nothing. It is just that they are so incredibly complex, that after +100 years of research we still don't know how they work or why. Will there be a time when we finally understand them 100% and could replicate them? Could be. That does not make them machines in any case, and the fallacy of thinking that mind/consciousness can emerge from machines will still be there.
> These theories are flawed in the sense that they cannot account for subjective experience and agency, amongst other things
I don't see how any of the works you referenced can account for that either? Since when is the problem of consciousness solved and we can definitely say what does or doesn't result in consciousness?
None of the works I mentioned solved or try to solve the mind/body problem or the issue of consciousness.
They are a frame of reference for not stepping into the common fallacies that the AI research field is based on.
> Is the same underlying assumptions that treats cells as machines, our body as a complex machine. These theories are flawed in the sense that they cannot account for subjective experience and agency, amongst other things.
Agency: What’s missing, in your view? Agency seems more of a property/function of a thinking system’s position in an environment than of the thinking itself.
Subjective experience: That’s not a contradiction to “complex machines” either. I think the evidence that our minds are highly complex machines is, at this point, irrefutable. The question is really if they’re “only” that.
>> and that is that we understand (or we think we understand) how our brain/mind works. But the truth is that we don't know. And there's even not a single clue that we actually know too much, and not a clue that our brain/mind and cells work 'as the machines we build'.
>> I highly recommend people in the AI research space should read philosophy and modern linguistics.
I highly recommend the philosophers read some neuroscience. The whole "model weights" thing in AI is modeled after the synaptic connections and between actual neurons. There is already quite a bit known about how the brain works at a low level. There is also a lot that is still unknown. There are also differences between discrete neuron firing and weights as signals, but there is enough similarity to make artificial neural nets useful and do things similar to what real one do.
Hard disagree. We only discovered the role that glial cells play in processing around 2014. We're still uncertain how patterns of activation consolidate through long term potentiation, let alone how signaling encodes information. We understand quite a bit about the role of the hippocampus and subiculum in encoding memories; but we don't understand the structural layout of engram complexes - which were themselves mapped for the first time only in 2022!
Taking effective results in machine learning, and somehow assuming that they apply to cognition - simply because neural nets were inspired by our limited knowledge of neural signaling and structure - is like trying to apply aircraft engineering to studying ornithology. For a better articulation of this point (from the reverse direction) check out the paper 'Could a Neuroscientist Understand a Microprocessor?' from 2017 - https://journals.plos.org/ploscompbiol/article?id=10.1371/jo...
>> We understand quite a bit about the role of the hippocampus and subiculum in encoding memories...
Hard disagree ;-) You're talking about high level architecture of the brain. I don't think (not my area I may be wrong) we know how memories are encoded in a real brain. Is it weights or something else? If it's weights that's supporting my point (but we don't know what the weights represent in a brain, where in LLMs many weights are just token encodings). If brains store memories in something other than weights I'd really like to know as it's something I haven't read about yet.
> There are also differences between discrete neuron firing and weights as signals, but there is enough similarity to make artificial neural nets useful and do things similar to what real one do.
There is barely a surface-level similarity. The best example I can come up with is this…
Imagine the most intricate and beautiful tall building that you can think of. Think like an older skyscraper in Chicago or a palace. There are water features and moving parts everywhere but also tiny little handmade carvings and materials throughout.
Now imagine we have no reference designs and no blueprints - we hire an architect to attempt to study the building by looking at it from a distance and understand everything they possibly can about it. She can go into the building to check but every time she does, it stops functioning normally.
That architect is a neuroscientist.
Then the ML researcher is like a graphic designer who sees the work that the architect is doing and makes a napkin sketch of the building the architect has been studying, to use for a project later. Sure the designer has some of her representations. But the difference in complexity between the designer’s napkin sketch and the architect’s analysis is massive. Several orders of magnitude.
Then another many orders of magnitude is the level of detail that the architect can understand about this strange building without being able to fully interact with it, versus the actual complexity of the building.
So yeah, an AI is modeled after neurons in the sense that they represent a couple of surface level features of neurons. But the difference in complexity is about as much as a napkin drawing of a grand building represents the actual structure and details of the building, no matter the level of skill that the graphic designer has
> I highly recommend the philosophers read some neuroscience Philosophers do read neuroscience and technical reports on AI.
> The whole "model weights" thing in AI is modeled after the synaptic connections and between actual neurons In reality, it is a really poor and basic model of what is actually happening in a real brain
Brains and modern AI systems (LLMs for example) are structurally different. (Don't get confused by topology. A structure is more than topology: it is also what the structure is made of, thus the properties of the material contribute and define what can emerge atop the structure)
> These theories are flawed in the sense that they cannot account for subjective experience and agency, amongst other things.
That agency or free will exists outside of our subjective experience is an assumption; does any given theory need to explain agency, or is it sufficient to explain that we feel we have agency?
Indeed, I know there is research going on placing agency and free will as illusions of our own subjective experience. Yet, there is a big gap from a purely mechanistic view of the mind to the idea/phenomena of subjective experience.
> I highly recommend people in the AI research space should read philosophy and modern linguistics.
On the contrary, I highly recommend people in Philosophy of mind and linguistics should start reading AI research papers because their theories and ideas are highly outdated, even ancient. Your books are from 1927 and 1972 respectively and Turing's article is from 1950s. And they are relatively new with respect to other works in Philosophy.
If one doesn't adequately understand what we have in 2026, how can they theorize about it? As others they don't understand how the mind/brain work, BUT ALSO they don't understand how the AI works.
Also with this mindset that we can't understand seemingly complicated things, there would be no advancement in science and technology.
I think philosophy people and Linguist will catch up in a century, like they did with Turing. Philosophers of this century are not in humanities or literature. They are in science and engineering.
Heidegger was trained on priesthood and Theology. You should read greater minds like Hinton, LeCun etc. if you want to think on these things. They are the real Philosophers.
Yikes dude. You may need to expand your horizons a bit.
That's my point. Everything from 1927 is already in plain sight and a part of the current public knowledge. Horizon can only be expanded at the cutting edges.
> Horizon can only be expanded at the cutting edges.
Ironically this was advice Ralph Waldo Emerson gave in 1840 so by your logic it's irrelevant just because it's old.
> On the contrary, I highly recommend people in Philosophy of mind and linguistics should start reading AI research papers because their theories and ideas are highly outdated, even ancient. Your books are from 1927 and 1972 respectively and Turing's article is from 1950s. And they are relatively new with respect to other works in Philosophy.
People in philosophy and cognitive linguistics do read AI research. Don't get fooled by the publishing dates: although Heidegger's work dates from 1927, the work is contemporary. The same happens with Dreyfus' work. Again, publishing dates don't mean anything here. Maybe you can clarify why they are outdated.
> If one doesn't adequately understand what we have in 2026, how can they theorize about it? As others they don't understand how the mind/brain work, BUT ALSO they don't understand how the AI works.
I would say that people involved in the critique of AI do know how it works. But I've found that is normally the case that people in AI research does not have the framing provided by works in philosophy or cognitive linguistics.
> I think philosophy people and Linguist will catch up in a century, like they did with Turing. Philosophers of this century are not in humanities or literature. They are in science and engineering. What do you base your claims on? Plenty of philosophers work in humanities, literature, sociology as well as science and engineering. Philosophers not catching up? The critique on automation and AI already dates from the early 20s if not before.
> Heidegger was trained on priesthood and Theology. You should read greater minds like Hinton, LeCun etc. if you want to think on these things. They are the real Philosophers.
Sorry, but this does not make too much sense. Hinton and LeCun are great in their own fields. But seriously, they are not philosophers, they are inventors.
> should read philosophy
subject/object dichotomy is a springboard to many schools of thought.
1) that subject emerges from objects - ie, anything has a material explanation, and everything is a machine.
2) that objects emerge out of a subject as a world model (platonic, descartes)
3) the subject and objects are one and the same representation of nature (spinoza)
4) subjects and objects emerge and disappear together (buddhist)
Anything reduced to computation is a fixed function from input to output, and is "dead" in the sense that it is unadaptable to its environment. Weights therefore is a dead machine.
Another view of this is that any closed system has unanswerable questions within it. Therefore, there is no system that can encompass everything. Hence weights being a closed system doesnt encompass everything.
> There is also an epistemological assumption that prevails, and that is that we understand (or we think we understand) how our brain/mind works.
It's good that it doesn't matter. Stochastic gradient descent works (or doesn't work) regardless of whether we know how the brain does its thing.
we know the brain does not use gradient descent. (i agree it doesn't matter)
How do we know this?
i recall a hinton lecture where he talks about the mechanism of memory formation in the brain and it's not fully sussed out but it's also not backprop, its some sort of forward prediction and immediate reinforcement loop.
There's also no plausible biological/chemical mechanism to backpropagate.
This reminds of this article which contends that this mistake has carried across time, in every era people described the workings of the brain similar to the machines they knew how to build, which is why we have "ridiculous" descriptions that changed over time.
https://aeon.co/essays/your-brain-does-not-process-informati...
I like Leibniz's comparison of the brain to a mill, which he invokes in an argument[1] that seems to be the predecessor of the modern "hard problem:"
> supposing that there were a mechanism so constructed as to think, feel and have perception, we might enter it as into a mill. And this granted, we should only find on visiting it, pieces which push one against another, but never anything by which to explain a perception.
[1]: https://en.wikipedia.org/wiki/Leibniz%27s_gap
I don't think cognitive linguistics has that much to say about AI nowadays. Let alone philosophy.
The biggest contributions from linguistics are probably "human languages mostly have statistical regularities rather than hard rules" and "the sum of data humans get from birth to language acquisition is insufficient to learn a language from scratch". Which LLMs already work with, and work around, respectively. From there, nothing.
And philosophy just exists to be a distractor. "Subjective experience" is too subjective to matter in practice. "Task performance" is measurable, "consciousness" isn't. "Agency" is something an LLM in a tool calling loop, a rat in a maze and a human in an office tower may or may not have, depending on your favorite definition. Agentic LLMs are years in the making, and that's a product of engineering, not philosophy: "agentic" is whatever gets the job done.
We are yet to discover any physical process whatsoever that can't be represented as mathematical operations and implemented by a Turing machine. So all of that "treating human mind as a machine is wrong" amounts to "human mind must be powered by magic fairy dust" paired with "a functionally similar magic-free replacement is impossible". I'm not about to give much weight to any hypothesis that requires undiscovered magic fairy dust. At least find the hyper-computational magic fairy dust first - not just assume it absolutely must be there because you want the human mind to be unique and special.
Want to know why Turing did what he did? It's because he didn't want to engage with any of that "what is mind" bullshit either. So he proposed actual metrics - measurables that are harder to argue in circles about. Not that it stopped anyone. But at least he tried.
> I don't think cognitive linguistics has that much to say about AI nowadays. Let alone philosophy. I would like to read those sources.
> The biggest contributions from linguistics are probably "human languages mostly have statistical regularities rather than hard rules" and "the sum of data humans get from birth to language acquisition is insufficient to learn a language from scratch". Which LLMs already work with, and work around, respectively. From there, nothing.
Again, what's the source for 'biggest contributions from linguistics are...'? It is a big contribution to the development of LLMs, but different cognitive linguistics authors already challenged this idea already 20-30 years ago. LLMs work with and around the problems you cite because of massive data/money, not at the fundamental level. It is all a game of statistics and data, which has been already challenged by cog. ling.
> And philosophy just exists to be a distractor. Well, this is just telling me that you either know too much about philosophy and reached that conclusion (which might make sense, know of some philosophers who also think that) or you just read too little.
> So all of that "treating human mind as a machine is wrong" amounts to "human mind must be powered by magic fairy dust"
This is the common fallacy people in AI/IT make . One of the benefits of reading philosophy is that you find your way out of them.
> Want to know why Turing did what he did? The actual tests Turing though about are themselves flawed (not that I discovered that, has been known for some time already)
Again: either stop using > quotes or learn to use them better. Fucking unreadable.
I reiterate: philosophy is almost entirely worthless for AI design. We want to design systems that work, not systems that sound good on paper. If philosophy had a practical application in that, we'd stop calling it "philosophy" and start calling it "math", "science" or "engineering".
Again: don't be rude. Not nice. As 'not nice' as my bad formatting.
Philosophy is there precisely to critique and analyze what we build and how we behave in the world. Without it, by the way, there would be no science and no engineering.
Dismissing philosophy in AI is like dismissing philosophy in any other applied, practical or creative field. It is precisely because there is a philosophical investigation on each practical, applied or creative field, that that field can actually make progress.
There's philosophy in biology, which helps biology go further. Philosophy in engineering, which makes engineering go further. In architecture, film, photography, painting, literature, medicine, physics, linguistics, mathematics, etc, etc. In all those fields, there is also a philosophical investigation taking place. Right now.
Maybe you confound practical AI design (you should be talking of 'building AI systems') with AI as a research field. I get that, because lots of people cannot make the distinction (calling yourself 'AI researcher' when you actually tinker a model is not scientific. It might follow a scientific method, but it is engineering research, not scientific research).
> These theories are flawed in the sense that they cannot account for subjective experience and agency
So, I work in AI research (as a research engineer though, not a scientist). I've also studied philosophy and I'm a vegan. Yes yes, insert "they will tell you" joke here, but I promise it's actually relevant this time.
First, while I studied philosophy one of the things that stuck with me the most, was the discussion of "souls": humans have souls, animals don't. For centuries the specifics of souls were discussed: people would be weighed while they died, in an effort to measure the approximate weight of a soul as it departed the body. Discussions about how many souls (or angels) could dance on the tip of a needle. Many people still believe in souls, but it's very hard to have a real discussion about them because by definition they do not "interact" with this world in a way that can be measured.
When discussing whether it's okay to harm animals for food or sport, one other argument I hear quite often (other than having no souls) is that animals do not experience "qualia": basically the smallest unit of "subjective experience". People know that they themselves experience qualia: the sensation of touching a doorknob, the taste of fresh fruit, the sense of beauty watching a rainbow. Ironically, they would say that animals are like robots: just (biological) machines acting on instinct, and feeling any kind of compassion for them means you are anthropomorphizing.
Subjective experience (or at least qualia) and souls both have one thing in common: they can not be measured externally in a meaningful way. You can simply state that an AI system no matter how advanced, has no soul and has no subjective experience. And that's pretty much that. There's no meaningful discussion to be had about it, because no matter what an AI might tell you: it has no way to prove it to you. In fact, you can't even be certain that anyone other than yourself has subjective experiences: you assume that because they are humans like you, and you experience them, that they probably do as well. They tell you that they do. But a human without subjective experience, someone on "autopilot", would be absolutely indistinguishable from a human who does have them.
But perhaps I am conflating here whether experience can be "measured", with whether a system even allows experience in the first place regardless of whether it can be measured. I think that Dreyfus and others argue that in order to have any "experiences" at all, you simply must have a body in the real world, and you must care about that body. Please correct me if that's the wrong interpretation, I haven't actually read the book. That argument would be harder for me to discuss, because I personally believe that consciousness will "emerge" from a complex interaction of relatively simple systems - but that's also just a theory. I don't believe that experience is literally impossible to engineer, as consciousness has emerged from non-conscious being through evolution, so clearly there must be some kind of mechanism for it -- and if there is, then I believe it can be replicated, we just don't really understand it well enough yet to do so. And with how AI tech is going, I think that we're more likely to accidentally stumble upon it than we are to get these deliberately.
“Reflections on trusting trust” is the paper that posits a compiler which is edited once so that when it compiles a program, it adds a security vulnerability to it, and when it compiles it’s own source code, it adds this edit into itself. Then it is used to compile its own source code once. Then this edit is removed.
Now any study of the program or compiler source code will not show any vulnerability, but compiling the program will make a vulnerable program, and recompiling the compiler from its clean source code will not fix the situation.
This carrying down of a pattern which is not written down anywhere, a flaming torch lighting a torch lighting a torch, is analogous to four billion years of life on Earth. We talk like DNA is an everything-code that defines a human and a human brain, but it’s the implicit behaviour of cells (‘compiler’) and the mechanisms inside them which interpret DNA. The unbroken chain of life getting more and more complex and never being restarted from scratch, with the behaviours not written down anywhere for us to study. How does DNA arrange for x, y, z to happen? Maybe it doesn’t at all.
Accidentally stumbling on a mechanism that is simple enough to be recreated with every human birth might be possible, accidentally stumbling on a mechanism that took evolution billions of years to find and which it has hung onto by copying it and has never recreated it from scratch, could be much less likely, in a much bigger search space.
> Accidentally stumbling on a mechanism that is simple enough to be recreated with every human birth might be possible, accidentally stumbling on a mechanism that took evolution billions of years to find and which it has hung onto by copying it and has never recreated it from scratch, could be much less likely, in a much bigger search space.
Maybe, but you could make the same argument about anything artificial.
I don't know what point you're making; I'm making the point that it might be harder to discover what makes humans human, than is often suggested. You can't make the same claim about "anything artificial", we know how to dry muddy clay into clay bricks and stack them into a brick wall, and that can be taught from scratch to new people in hours.
You can make a similar argument with a company like ASML where their secret sauce is the organisational ability to fine-tune 100,000 components into a precision Silicon-wafer etching machine. You're far more likely to accidentally stumble upon "how to recreate a mud hut" than "how to recreate ASML". Okay, and...?
Vegan ML engineer here. In total agreement with you. People are just moving the goal post to keep themselves protected from the obvious conclusion: there is nothing really all that special about us humans. Perhaps subjective experience is simply the internal state of a self supervised continuous learning algorithm and we don't like that conclusion very much.
It's too bad it's so hard to pin down a definition, but in practice I feel like most animals with brains experience degrees of qualia. Some mornings after a night of poor sleep when I wake up super-slow I wonder if that's how animals experience thinking.
My biggest problem with "brains are machines" arguments is that there is a risk there is unknown physics at work that is not representable as a Turing machine. What if there is some quantum field effect powering everything?
They know quite a lot about how neurons work to the extent they can replace bits of brains with artificial retinas or cochleas and interface with devices like neuralink. It's unlikely there is a quantum field effect of the type you mention powering things, although of course atoms and the like obey quantum mechanics in the normal way.
Quantum field effects? You don't need these, IMHO, if you look at how highly parallel things seem to work in brains.
Marvin Minsky's theory of a "Society of Mind" describes a (highly) distributed model of the mind. Which BTW, always reminds me of the first Shrek movie, where the donkey jumps up and down, shouting "Take me! Take me!" to Shrek. That's similar to what I observe when I'm undecided but two instances of "sub-processes" (or agents as Minsky calls them) of my mind try to get attention.
Daniel Dennett similarly gives a distributed model of consciousness. Where many parallel "processes" are at work, competing and "observing" each other. And this parallelism is happening with a much, much higher degree than any of our computers parallelism.
Mostly I have a hard time accepting that a Turing machine can experience "consciousness"/awareness. Therefore I also have a hard time with simulatable chemical processes; it feels like there is some missing link there.
All just feelings/vibes unfortunately.
I can sympathise with you, but how could a "quantum" effect" doesn't make this easier?
Maybe "Turing machine" is too abstract or simplistic as a concept? Both for real computers and brains?
I can see that a computer is on some level just a lot of sand (silica and metal) but put together in a really complex way, it "suddenly" can add and compare numbers … if we observe the complexity levels from sand to computer and try to see the analogy when comparing cells / neurons to a structure of billions of them somehow interconnected on both a physical and chemical level, evolved during millions of years, I have no problem to accept that brains are still too complex to explain for us.
> subjective experience and agency, amongst other things
In other words, souls? I'm sorry if that sounds accusing, but to me it sounds like you're talking about souls that are independent to the physical world, just with more 'scientific-y' wording.
(I fully understand that some people believe souls exist.)
As other commenters pointed out, I'm not getting into the mind/body problem. But that's what people normally do: you challenge the assumptions, and all of a sudden they assume you are already coming to a conclusion. And no, just because I challenge the assumptions, I don't need to provide a counter conclusion.
I don’t think the OP is discussing mind-body dualism. There are many materialists who believe that consciousness maybe a non-computational process.
But it is worth pointing out that something like 80% of the world (it fluctuates depending on the survey but its around that) believes in some non-physical spirit, life force, or soul.
It’s a very HN bubble thing to start a discussion with the assumption that everyone must be a materialist.
I acknowledge and understand that many people believe in the existence of souls. But I hope they just say it rather than try to wrap the concepts in layers of jargons.
The problem with the stuff you cite is that it is coming from consciousness via an anthropomorphic perspective. The priors baked in are poisoning the logic.
The question we have to answer is "Why do we think we're magical matter uniquely blessed with consciousness?" If you go far enough down the rabbit hole on that question, the answer you will come to is either "we're not" or "because god" (with a lot of pseudo-scientific bullshit wrapped around the "because god" to make it palatable for the nonreligious).
Panpsychism (or a deeper form, such as idealism) is actually the solution favored strongly by Occam's Razor over the variants of "because god" (such as magical emergence).
Given panpsychism, AI is already conscious, like everything else, though no claims are made about the correlation between the internal experience of that consciousness and the tokens that are being printed on the screen.
There is no way of dealing with these topics via a non-anthropomorphic perspective. That has been already (in my view) proved by Heidegger and then Derrida.
Last time I read about panpsychism, it was deeply flawed. But I can't remember the sources (sorry).
If it's so deeply flawed, should be easy to state why in an ELI5 manner. For example: The emergence hypothesis requires physical processes to effectively create new dimensions of information (700nm light -> "red" qualia) which we're somehow incapable of directly measuring with all the tools of modern physics.
You actually read Being and Time? What the heck do you get out of it? Heidegger just seems to play word games with German words without actually saying anything. "Time is the ripening of temporality." tf do phrases like that mean??
Not OP, but I have read Being and Time on and off over the years, and every time, I am blown away how precise Heidegger describes things and the being of things. Granted, I am German, so I can directly read the source without needing a translator. That might change things.
Regardless, I think Heidegger gave one of the fullest metaphysic-free accounts of the human experience and what Being is. And he starts from scratch. You essentially can read him without having to first study the whole western continental philosophy and he will construct the whole system by himself. Tremendous work
I keep hearing "it's better in German" which really reinforces my point that he's just playing word games with etymologies without saying anything.
i didnt say that at all :-)
Heidegger uses very specific German words to build a very specific vocabulary. This vocabulary then allows him to express very complicated sentiments very quickly and he can use this to express more and more complex structures.
Obviously this requires the reader to first learn the vocabulary and - granted - that is hard and challenging. I have a notebook here , which I consult and modify evertine i dive into Being and Time. But as I said. It keeps on giving. I often try to convey arguments and descriptions to others without falling back into Heideggers jargon, and its sometimes very hard and requires a lot of bloating . So you can argue that it was even necessary for him to invent the vocabulary, because otherwise the book would have been 10x in size
Your comment strikes me as a bit ignorant i must say. You accuse the work of being non-sensical word play but you’ve obviously not invested any time in learning the vocabulary. Because otherwise you wouldnt have made that comment. Id suggest to give it a chance. Its a wonderful piece of work and mind blowing in its own way. That a single mind can think something like that up. I’d argue its on the level of Hegel in terms of system building.
Yes, I'm ignorant, because Heidegger isn't saying anything. He didn't teach me anything. Thus I remain ignorant, unknowing.
You can't even explain what he said, you just said, "go learn his words". That's not knowledge, that's not insight. That's just "the wordplay is great". But it's not content. It's merely form, it's sophistry, it's useless and meaningless.
I asked a very specific question originally. What does "time is the ripening of temporality" mean? That's one way to translate one of the things he says, using different words for time and ripening in German. He's playing word games because those words sound similar in German and people like you confuse it for profundity.
Phew, you are quite something. I hope one day your mind will open up. All that is left to say for me is that you are really missing out on some truly great work that is a masterpiece of human thought. Your loss
Yes. I read the spanish translation (with occasional reading of the german original). A lot is lost in translation (I do not regard english translations too high). So yes, it is a difficult reading, on a difficult topic. He is not playing with words, but defining concepts in a very specific way. What I would recommend is reading Heidegger through other authors (Being-in-the-world, also by Dreyfus, would be a starting point).
It is also a story as old as time. We have been comparing our brains to our most recent information/communication technology since the invention of the telephone. After the telephone our brain was like computers, and when I did my bachelors in Psychology in 2009, we were comparing our brains to the internet. AI is simply the latest iteration of these comparison, just as wrong and unhelpful as when we compared it to the telephone.
> These theories are flawed in the sense that they cannot account for subjective experience and agency, amongst other things.
It would be a valid argument if you could explain us what subjective experience and agency are. No one can explain this, so the arguments sounds like "AI doesn't have something I don't really know what, but they have to miss something, sure".
> But the truth is that we don't know.
Yes! This is the point. If we don't know how our minds work, how can we be sure that a machine doesn't work like our minds?
> I highly recommend people in the AI research space should read philosophy and modern linguistics.
Linguists are linguists, they don't know about consciousness, they specialists in language.
The main teaching of philosophy is to open your mind wide, and then still wider. So yes, we all should read more philosophy, but we should remember while reading, that philosophy is what we do, when we can't turn a question to a scientific one. I'd say that philosophy is more about finding the right question, than the right answer. Until there are no science branches like Subjectology or Consciousiology, and all you can find are TL;DRs written by philosophers and scientists from unrelated branches of science (like linguists or neurologists) you can be sure that the right question is not found yet. Therefore it is better to keep yourself uncertain. BTW it is the main lesson of philosophy: to open your mind wide and then open it wider still.
I believe, that all this philosophy is... well... philosophy. Meta-physics. It doesn't matter. What does matter is how we should deal with machines? Do we have to treat them as human beings? Should we accept that they have "human rights"? Can a machine be held accountable for its mistakes? Can we talk about "intentional" and "unintentional" mistakes of a machine?
Answers to these questions are important, they are imperative answers, they govern how we live our lives. But when people try to answer them, they somehow jump to talking about consciousness, and "are they really like us or not", and... etc. I personally keep myself uncertain. I'm pretty sure that current models do not deserve human rights. I cannot say about future models.
You see, there were times in history when smart and well-intentioned people were certain that some people deserve less moral considerations than others. We are smart and well-intentioned and we are certain that AI deserve no moral considerations. How it will play out in a future? Will my descendants think bad of me because I used slave labor of a local model on a GPU? I don't know, we don't know. Right now, I'm exploiting LLMs and I believe it is ok, but I'm not going to fall into this trap and to stick to my belief because of some philosophical or lingustic or neurological argument. I'm choosing epistemological humility, I clearly state that I don't know the right answer and I'm keeping an eye to it so I wouldn't miss it.
> It would be a valid argument if you could explain us what subjective experience and agency are. No one can explain this, so the arguments sounds like "AI doesn't have something I don't really know what, but they have to miss something, sure".
Subjective experience is what you and only you, different from other beings, experience in and about the world.
> Yes! This is the point. If we don't know how our minds work, how can we be sure that a machine doesn't work like our minds?
You can't be sure that machine do not work like our minds or brains. But you cannot say the opposite either, so saying 'machines could think' base on a false assumption (because you cannot say it is true. It doesn't matter if you could be true).
> Linguists are linguists, they don't know about consciousness, they specialists in language.
That's a very narrow understanding of what language is, what linguists do/research, and the contributions made in the field. Linguists are (since already 2 o 3 decades) focusing more and more on psychological/cognitive matters. The intertwined topics of language, mind, body and though has a long way in the western philosophical tradition.
> I believe, that all this philosophy is... well... philosophy. Meta-physics. It doesn't matter. What does matter is how we should deal with machines? Do we have to treat them as human beings? Should we accept that they have "human rights"? Can a machine be held accountable for its mistakes? Can we talk about "intentional" and "unintentional" mistakes of a machine?
Exactly because of this. And is this what I am talking about... the topics you mention here are already settled in the philosophy space, but the AI research space keeps going 'round them...