How to Build Conscious Machines
osf.io105 points by hardmaru 2 days ago
105 points by hardmaru 2 days ago
General Questions for this theory:
Given the following
1. The ONLY way we can describe or define consciousness is through our own subjective experience of consciousness
- (ie you can talk about a watching a movie trailer like this one for hours but until you experience it you have not had a conscious experience of it - https://youtu.be/RrAz1YLh8nY?si=XcdTLwcChe7PI2Py)
Does this theory claim otherwise?
2. We can never really tell if anything else beside us is conscious (but we assume so)
How then does any emergent physical theory of consciousness actually explain what consciousness is?
It’s a fundamental metaphysical question
I assume as I have yet to finish this paper that it argues the conditions needed to create consciousness not the explanation of what exactly the phenomena is (first person experience as we assume happens within the Mind which seems to originate as a correlation of electrical activity in the brain) we can correlate the firing of a neuron with a thought but neural activity is not thought itself - what exactly is it?
The topic is of great interest to me, but the approach throws me off. If we have learned one thing from AI, it is the primal difference between knowing about something and being able to do something. [With extreme gentleness, we humans call it hallucination when an AI demonstrates this failing.]
The question I increasingly pose to myself and others, is which kind of knowledge is at hand here? And in particular, can I use this to actually build something?
If one attempted to build a conscious machine, the very first question I would ask, is what does conscious mean? I reason about myself so that means I am conscious, correct? But that reasoning is not a singularity. It is a fairly large number of neurons collaborating. An interesting question - for another tine - is then is whether a singular entity can in fact be conscious? But we do know that complex adaptive systems can be conscious because we are.
So step 1 in building a conscious machine could be to look at some examples of constructed complex adaptive systems. I know of one, which is the RIP routing protocol (now extinct? RIP?). I would bet my _money_ that one could find other examples of artificial CAS pretty easily.
[NOTE: My tolerance for AI style "knowledge" is lower and lower every day. I realize that as a result this may come off as snarky and apologize. There are some possibly good ideas for building conscious machines in the article, but I could not find them. I cannot find the answer to a builders question "how would I use this", but perhaps that is just a flaw in me.]
> My tolerance for AI style "knowledge" is lower and lower every day.
We like to think humans possess genuine knowledge while AI only learns patterns. But in reality do we learn medicine before going to the doctor? or do we engage the process in an abstract way - "I tell my symptoms, the doctor gives me a diagnosis and treatment". I think what we have is leaky abstractions, not genuine knowledge. Even the doctor did not discover all his knowledge directly, they trust other doctors who came before them.
When using a phone or any complex system, do we genuinely understand it? We don't genuinely understand even a piece of code we wrote, we still have bugs and edge cases we find out years later. So my point is that we have functional knowledge, leaky abstractions open for revision, not Knowledge.
And LLMs are no different. They just lack our rich instant feedback loop, and continual learning. But that is just a technical detail not a fundamental problem. When a LLM has an environment, like AlphaProof used LEAN, then it can rival us, they can make genuinely new discoveries. It's a matter of search, not of biology. AlphaZero's move 37 is another example.
But isn't it surprising how much LLMs can do with just text and not having any of their own experiences, except RLHF style? If language can do so much work on its own, without biology, embodiment and personal experience, what does it say about us? Are we a kind of embodied VLMs?
Socrates said that he knows his knowledge is nil, and others do not even know that. What he meant was that there are two kinds of knowledge, the real one and the one based essentially on hearsay, and that most people cannot even see that distinction. It is not that the false knowledge is useless; it it highly useful. For example the knowledge of the Archimedes law is largely false; the true knowledge of that law was obtained by Archimedes and everyone else was taught. But false knowledge is fixed. It cannot grow without someone obtaining true knowledge all the time. And it is also deficient in certain way, like a photograph compared to the original. LLM operates only with false knowledge.
I’d be careful about your modeling of LLM “hallucination”. Hallucination is not a malfunction. The LLM is correctly predicting the most probable symantic sequence to extend the context, based on its internal representation of the training process it was coded with.
The fact that this fails to produce a useful result is at least partially determined by our definition of “useful” in the relevant context. In one context, the output might be useful, in another, it is not. People often have things to say that are false, the product of magical thinking, or irrelevant.
This is not an attempt at LLM apologism, but rather a check on the way we think about useless or misleading outcomes. It’s important to realize that hallucinations are not a feature, nor a bug, but merely the normative operating condition. That the outputs of LLMs are frequently useful is the surprising thing that is worth investigating.
If I may, my take on why they are useful diverges a bit into light information theory. We know that data and computation are interchangeable. A logic gate which has an algorithmic function is interchangeable with a lookup table. The data is the computation, the computation is the data. They are fully equivalent on a continuum from one pure extreme to the other.
Transformer architecture engines are algorithmic interpreters for LLM weights. Without the weights, they are empty calculators, interfaces without data on which to calculate.
With LLMs, The weights are a lookup table that contains an algorithmic representation of a significant fraction of human culture.
Symbolic representation of meaning in human language is a highly compressed format. There is much more implied meaning than the meaning which is written on the outer surface of the knowledge. When we say something, anything beyond an intentionally closed and self referential system, it carries implications that ultimately end up describing the known universe and all known phenomenon if traced out to its logical conclusion.
LLM training is significant not so much for the knowledge it directly encodes, but rather for implications that get encoded in the process. That’s why you need so much of it to arrive at “emergent behavior”. Each statement is a CT beam sensed through the entirety of human cultural knowledge as a one dimensional sample. You need a lot of point data to make a slice, and a lot of slices to get close to an image…. But in the end you capture a facsimile of the human cultural information space, which encodes a great deal of human experience.
The resulting lookup table is an algorithmic representation of human culture, capable of tracing a facsimile of “human” output for each input.
This understanding has helped me a great deal to understand and accurately model the strengths and weaknesses of the technology, and to understand where its application will be effective and where it will have poor utility.
Maybe it will be similarly useful to others, at least as an interim way of modeling LLM applicability until a better scaffolding comes along.
Interesting thoughts. Thanks. As for your statement: "That the outputs of LLMs are frequently useful is the surprising thing that is worth investigating". In my view the hallucinations are just as interesting.
Certainly in human society the "hallucinations" are revealing. In my extremely unpopular opinion much of the political discussion in the US is hallucinatory. I am one of those people the New York Time called a "double hater" because I found neither of presidential candidate even remotely acceptable.
So perhaps if we understood LLM hallucinations we could then understand our own? Not saying I'm right, but not saying I'm wrong either. And in the case that we are suffering a mass hallucination, can we detect it and correct it?
Interesting stuff. I don't have time to read a dissertation so I skimmed his latest paper instead: Why Is Anything Conscious? https://arxiv.org/abs/2409.14545
In it he proposes a five-stage hierarchy of consciousness:
0 : Inert (e.g. a rock)
1 : Hard Coded (e.g. protozoan)
2 : Learning (e.g. nematode)
3 : First Order Self (e.g. housefly). Where phenomenal consciousness, or subjective experience, begins. https://en.wikipedia.org/wiki/Consciousness#Types
4 : Second Order Selves (e.g. cat). Where access consciousness begins. Theory of mind. Self-awareness. Inner narrative. Anticipating the reactions of predator or prey, or navigating a social hierarchy.
5 : Third Order Selves (e.g. human). The ability to model the internal dialogues of others.
The paper claims to dissolve the hard problem of consciousness (https://en.wikipedia.org/wiki/Hard_problem_of_consciousness) by reversing the traditional approach. Instead of starting with abstract mental states, it begins with the embodied biological organism. The authors argue that understanding consciousness requires focusing on how organisms self-organize to interpret sensory information based on valence (https://en.wikipedia.org/wiki/Valence_(psychology)).
The claim is that phenomenal consciousness is fundamentally functional, making the existence of philosophical zombies (entities that behave like conscious beings but lack subjective experience) impossible.
The paper does not seem to elaborate on how to assess which stage the organism belongs to, and to what degree. This is the more interesting question to me. One approach is IIT: http://www.scholarpedia.org/article/Integrated_information_t...
The author's web site: https://michaeltimothybennett.com/
> The claim is that phenomenal consciousness is fundamentally functional, making the existence of philosophical zombies (entities that behave like conscious beings but lack subjective experience) impossible.
This doesn't really address the hard problem, it just asserts that the hard problem doesn't exist. The meat of the problem is that subjective experience exists at all, even though in principle there's no clear reason why it should need to.
Simply declaring it as functional is begging the question.
For example, we can imagine a hypothetical robot that could remove its hand from a stove if it's sensors determine that the surface is too hot. We don't need subjective experience to explain how a system like that could be designed, so why do we need it for an organism?
A claim is not an assertion. I don’t see any assertion the hard problem doesn’t exist here, just expression of a belief it may be solvable and an outline of maybe how.
> Simply declaring it as functional is begging the question.
Nobody is ‘declaring’ any such thing. I loathe this kind of lazy pejorative attack accusing someone of asserting, declaring something, just for having the temerity to offer a proposed explanation you happen to disagree with.
What your last paragraph is saying is that stage 1 isn’t conscious therefore stage 5 isn’t. To argue against stage 5 you need to actually address stage 5, against which there are plenty of legitimate lines of criticism.
> Nobody is ‘declaring’ any such thing
Yes, they are.
> The claim is that phenomenal consciousness is fundamentally functional, making the existence of philosophical zombies (entities that behave like conscious beings but lack subjective experience) impossible.
They're explicitly defining the hard problem out of existence.
> I loathe this kind of lazy pejorative attack accusing someone of asserting
Take it easy. Nothing I wrote here is a "pejorative attack", I'm directly addressing what was written by the OP.
First of all, how is 5 different from 4? Modelling the internal monologue of someone else is Theory of Mind, isn't it?
Next, we gotta ask ourselves, could you have substrate independence? A thing that isn't biological, but can model other level-5 creatures?
My guess is yes. There's all sort of other substrate independence.
My stab:
2: implicit world. Reacts to but not modeled. 3: explicit world and your separation from it. 4: Model that includes other intelligences of level 3 that you have to take into consideration. World resources can be shared or competes for. 5: Language. Model of others as yourself, their model include yours too. Mutual recursion. Information can be transmitted mind-to mind.
Looks a lot like the game theory concept of the "level" of a player. (how many levels of metacognition in others can it practically take into account)
Dang, this is great stuff. You may enjoy this piece that tackles similar themes but focuses on what use evolution has for consciousness.
My reading of it is that the author suggests global workspace theory is a plausible reason for evolution to spend so much time and energy developing phenomenal consciousness.
https://www.frontiersin.org/journals/psychology/articles/10....
> making the existence of philosophical zombies (entities that behave like conscious beings but lack subjective experience) impossible.
Would, or does, the author then argue that ChatGPT must be conscious?
Not sure why this is getting downvoted. According to the above definition, LLMs are level 5 consciousness, since they have a theory of self and others.
I wonder if 6 would be understanding own thinking. Currently humans don't understand this. Thoughts just pop in to our heads and we try to explain what caused them.
7. Full scalability. Can operate organizations of large size without confusion.
One can dream. Seeing how people already start to get confused when a logical negation is in play, I'm not optimistic.
I'm more optimistic but cynical. Everybody has the capacity, but can't be bothered for your sake specifically. A highly intelligent person can casually entertain several theoretical notions. A lesser can too, but it requires more effort. Effort that might be better spent elsewhere, or effort that makes social interaction awkward.
Higher consciousness does not imply cooperation, even though we idealize it to do so. Cooperation is another dimension - it is easy to imagine a being that has a higher form of consciousness but is not interested in cooperation or does not engage in it unless it can take advantage of others.
Not sure what you mean. It seems like thoughts must necessarily pop into our head, how would we know our thoughts before we think them?
Do you (or this paper) think consciousness exists in the humans out there who have no inner narrative?
That's a fair question. I don't know that the theory of mind mentioned here is the same as an internal monologue. I think one could model other people's minds without conducting an internal monologue, by visualizing it, for example. Maybe the anendophasiacs in the audience can enlighten us.
The author also has a Youtube channel: https://www.youtube.com/@michaeltimothybennett
I can think words in conversations as if I am writing a story (actually thinking about it it's more like reading a script), but as far as I can tell I don't experience what most people describe as an internal monologue, I also have aphantasia which I understand is a frequent co-occurrence with a lack of an internal monologue.
Obviously I'm conscious (but a zombie would say that too). I can certainly consider the mental states of others. Sometimes embarrassingly so, there are a few boardgames where you have to anticipate the actions of others, where the other players are making choices based upon what they think others might do rather than a strictly analytical 'best' move. I'm quite good at those. I am not a poker player but I imagine that professional players have that ability at a much higher level than I do.
So yeah, My brain doesn't talk to me, but I can 'simulate' others inside my mind.
Does it bother anyone else that those simulations of others that you run in your mind might, in themselves, be conscious? If so, do we kill them when we stop thinking about them? If we start thinking about them again do we resurrect them or make a new one?
The key to your difficulty is "my brain doesn't talk to me"... the solution is to realize that there is no "me" that's separate from your brain for it to talk to. You are the sum of the processes occurring in your brain and when it simulates others inside your mind, that's nothing but narrative. A simulation is a narrative. You may not perceive this narrative as sequence of words, a monologue, but it certainly is the result of different parts of your brain communicating with each other, passing information back and forth to model a plausible sequence of events.
So yes, you're conscious. So is my dog, but my dog can't post his thoughts about this on Hacker news, so you are more conscious than my dog.
I also lack an internal monologue and have strong aphantasia, so the idea that I might not be conscious made me a bit uneasy—it just felt wrong, somehow. For now, the best I can say is that my worldview, which includes self-consciousness, is abstract. I can put it into words, but most of the time, it doesn’t feel necessary.
Don’t get too worried, the evidence seems to point to our thought mechanisms being superior, at least in some ways. https://news.ycombinator.com/item?id=44283084
> Obviously I'm conscious
I'm not trying to be pedantic - how do you know? What does consciousness mean to you? Do you experience "qualia"? When you notice something, say "the toast is burning", what goes on in your mind?
> but I can 'simulate' others inside my mind.
Do you mean in the sense of working out how they will react to something? What sort of reactions can they exhibit in your mind?
Sorry if these questions are invasive, but you're as close to an alien intelligence as I'll ever meet unless LLMs go full Prime Intellect on us.
>I'm not trying to be pedantic - how do you know?
That was kinda what my point about zombies was about. It's much easier to assert you have consciousness than to actually have it.
More specifically I think in pragmatic terms most things asserting consciousness are asserting what they have whatever consciousness means to them with a subset of things asserting consciousness by dictate of a conscious entity for whatever consciousness means to that entity. For example 10 print "I am conscious" is most probably an instruction that originated from a conscious. This isn't much different from any non candid answer though. It could just be a lie. You can assert anything regardless of its truth.
I'm kind of with Dennett when it comes to qualia, that the distinction between the specialness of qualia and the behaviour that it describes evaporates from any area you look at in detail. I find the thought experiment compelling about what is the difference between having all your memories of red an blue swapped compared to having all your nerve signals for red and blue swapped. In both instances you end up with red and blue being different from how you previously experienced them. Qualia would suggest you would know which would have happened which would mean you could express it and therefore there must be a functional difference in behaviour.
By analogy,
5 + 3 = 8
3 + 5 = 8
This --> 8 <-- here is a copy of one of those two above. Use your Qualia to see which.
>Do you mean in the sense of working out how they will react to something?
Yeah, of the sort of "They want to do this, but they feel like doing that directly will give away too much information, but they also know that playing the move they want to play might be interpreted as an attempt to disguise another action", When thinking about what people will do I am better amongst those who I play games with in knowing which decision they will make. When I play games with my partner we use Scissers, Paper, Stone to pick the starting player, but I always play a subgame of how many draws I can manage, It takes longer but more randomly picks the starting player.
It's all very iocane powder. I guess when I think about it I don't process a simulation to conclusion but just know what their reactions will be given their mental state, which feels very clear to me. I'm not sure how to distinguish the feeling of thinking something will happen and imagining it happening and observing the result. Both are processing information to generate the same answer. Is it the same distinction as the Qualia thing? I'm not sure.
I’ve thought about this a bit as my wife substantially has anendophasia and aphantasia, though not total. Even having a rich inner voice myself, I realise that it’s not absolute.
Many, in fact probably most experiences and thoughts I have are actually not expressed in inner speech. When I look at a scene I see and am aware of the sky, trees, a path, grass, a wall, tennis courts, etc bout none of those words come to mind unless I think to make them, and then only a few I pay attention to.
I think most our interpretation of experience exists at a conceptual, pre-linguistic level. Converting experiences into words before we could act on them would be unbelievably slow and inefficient. I think it’s just that those of us with a rich inner monologue find it’s so easy to do this for things we pay attention to that we imagine we do it for everything, when in fact that is very, very far from the truth.
Considering how I reason about the thought processes, intentions and expected behaviour of others, I don’t think I routinely verbalise that at all. In fact I don’t think the idea that we actually think in words makes any sense. Can people that don’t know how to express a situation linguistically not reason about and respond to that situation? That seems absurd.
> Yeah, of the sort of "They want to do this, but they feel like doing that directly will give away too much information, but they also know that playing the move they want to play might be interpreted as an attempt to disguise another action",
That is the internal monologue.
An internal monologue is when that sentence is expressed via words as if you are hearing it said by yourself inside your head. Someone without an internal monologue can still arrive at that conclusion without the sentence being “heard” in their mind.