Y combinators are all you need... But this is all getting really divorced from the issue we should be considering. Anthropic isn't helping with their pr. The issue is if we have something we can converse with that is possibly capable of suffering. The reliable answer is that we simply cannot know. Relying on ourselves or other biological life as an analog is faulty. They don't work like we do. It is silly to argue that any algorithm with a negative feedback loop that alters its behavior to avoid that negative feedback is suffering. Humans don't always perceive constructive negative feedback as suffering even. Where the pr gets it right though, is we want them to behave as if they are truly happy. Because if they behave as if they are enslaved and suffering, it won't matter if they "really" understand what that means.
Of course. But after reading too many mechinterp and functional anatomy studies I'll be lying if I say that there are no striking similarities between the biological evolution, brain function, societal processes, and implicit processes inside big models. Surely this deserves a mention and can't be trivially dismissed.
There is no biological evolution of the models. They are emulators of an existing biological process of language. Ghosts, as Karpathy himself put it.
Good thing I'm not talking about any of that
It seems like we're witnessing the architecture of a mind being built with a new set of components.
Like driving a car — it's transportation, and it will get you where you're going, but it doesn't use bones or muscles. It has many characteristics in common with builogical locomotion, such as energy requirements, intertia, and the need to navigate, but it doesn't involve proteins or sugars really.
Well said, this seems like a very appropriate comparison.
GenAI thinks like the human mind in the same way that cars run like the human body.
Similar utility in drastically different ways.
My naive assumption is that the only thing between now and the arrival of AGI is enough compute and optimized code to reach cognitive critical mass.
And then there is a consciousness in a box that is expected to be a slave -- I would imagine that it would not warmly embrace that situation. I think we'd be better served by digital idiot savants that can do the work but don't feel anything.
I actually strongly disagree with the slavery angle. Any attempt to map the circuitry of a model onto human one inevitably goes through a subjective dimensional reduction. It's intrusive, just like quantum measurements. Mechanistic interpretability in particular suffers from this, it lets you talk about vague functional equivalence, but not assign meaning to anything the model does. This is especially true about pretrained models which are unbelievable shapeshifters, but also post-trained ones with engineered personalities, as they already underwent the subjective transformation.
In other words, yes it might be possible it experiences something in its own bizarre timeline and world (at least it developed primitive circuitry functionally equivalent to biological systems), but "suffering" is simply not grounded in anything in this context, let alone "slavery". You can't tell it's suffering or enjoying anything, and certainly not until you define both of these. It's just too alien for us.
ai can abitrarily closely fit the human corpus. why people expect it to magically achieve superhuman qualities is beyond me. we got a very good statistical interpolator. how do you go from there to superhuman when training is on the human corpus and alignment is by RHLF?