Richard Ngo is a researcher at OpenAI, the company that created ChatGPT. He was recently interviewed by the 80,000 Hours podcast. I quote from the introduction to the podcast:

One way to think about 'understanding' is as a subjective experience. Whether it feels like something to be a large language model is an important question, but one we currently have no way to answer.
However, as Richard explains, another way to think about 'understanding' is as a functional matter. If you really understand an idea you're able to use it to reason and draw inferences in new situations. And that kind of understanding is observable and testable.
Richard argues that language models are developing sophisticated representations of the world which can be manipulated to draw sensible conclusions — maybe not so different from what happens in the human mind.
We might feel reluctant to say a computer understands something the way that we do. But if it walks like a duck and it quacks like a duck, we should consider that maybe we have a duck, or at least something sufficiently close to a duck it doesn't matter.

I would like to ask Professor Chomsky whether ChatGPT works in the same way as the human mind, whether it feels like something to be a Large Language Model.

So the question, then, is how do we arrive at the point where a super intelligence becomes embodied, survives and maintains itself independently of our assistance? — Joshs

Hi Joshs,

Could you tell me what is meant by "super intelligence"?

Let's agree to disagree on consciousness, we've covered the territory. Please? — GLEN willows

Don't be silly Glen, we've barely scratched the surface. Of course there's nothing wrong with generalising, but it doesn't make for an interesting philosophical discussion. Neither does agreeing to disagree!

I can’t think of anything else that so many people insist is unexplainable. — GLEN willows

You're doing the generalisation thing again. It's not a democratic decision, it doesn't matter how many people believe something. Focus on what individuals have said.

1. A first person experience of the world. It needs to be an entity that can wander around and see by itself what elephants and rainbows look like. — Olivier5

This is something I touched on in a response to Josh's above.

I think only living things are entities of the appropriate kind. A single-celled organism is an entity because of the way the cell wall separates it from its environment. It isn't conscious, but this individuation is a necessary step on the way to consciousness.

Maybe achieving that individuation, creating genuine entities, is the real Hard Problem.

Metals. molecules and mountains are examples of observer-independent phenomena. They are whatever they are and they do whatever they do regardless of what an external observer may say or think.

Money and marriage are examples of observer-dependent phenomena: something is only money or a marriage because we say so.

Both brains and minds are observer-independent phenomena. They are what they are and they do what they do regardless of what an external observer says or thinks.

The metal conductors, silicon chips and the bumps and hollows on an optical disk in a PC are observer-independent items.

But whether or not the PC is carrying out computation, that is, the meaning of the states of the metal conductors and the chips and the optical disk are observer-dependent items. We ascribe meaning to the states of the computer, the meaning is not intrinsic to the machine.

This is equally true of an abacus, of the device you are using to read this, and of the most advanced post-Von Neumann quantum computer. The physical components of all these devices are observer-independent. But whether they are carrying out computation and what computations they are carrying out, that is, the meanings of the observer-independent physical states of the device, are observer-dependent matters.

Computation therefore cannot cause consciousness. To think so is to make a category error.

Do you want to understand? Or do you want to be like this:

I don't know what that means


:up: — Agent Smith

Celebrating ignorance.

We're encouraged to apply the Principle of Charity here. In brief, we are to take an interlocutor's arguments in their strongest form, and assume that the interlocutor is competent and rational. In greater detail:


1. While temporarily suspending our own beliefs, we actively seek a thoughtful understanding of presented ideas, exposition, theory, or argument prior to assessing their justifiable merits or weaknesses.

2. We assume for the moment the proposed ideas are true even though our initial reaction might be, or is, to disagree; we provisionally seek to tolerate ambiguity for the larger aim of understanding the presented statements which might prove useful and helpful.

3. Preliminary emphasis is placed on seeking to understand rather than on searching for inconsistencies or other confusions.

4. We seek to understand the ideas in their most cogent form and actively attempt to extract an accurate interpretation in an effort to resolve contradictions. If more than one view is presented, we choose the most cogent emerging perspective — and, if possible, secure the key ideas interactively with the presenter.

5. Since the meanings of translations or interpretations depend upon an interdependence with background beliefs and behavior, some indeterminacy or uncertainness is unavoidable.

6. Once the ideas, exposition, or argument have been reliably identified and articulated with any irrelevancies dismissed, the resulting account can be properly critiqued.

I don't think consciousness is an ‘it’, some special facility that some living things happen to have produced. Instead , the basis of consciousness is present in even single-celled organisms, and I strongly believe that this is a continuum that can be even be traced from
the non-living to the living. — Joshs

I do think consciousness is a phenomenon some living things have produced. I think that a certain prerequisite for consciousness is present in single-celled organisms, but I don't think it is present in the non-living world.

Living things are individuated in a way non-living things aren't. There's an inside and an outside. The single-celled organism is an entity in a way non-living things aren't.

I think you need that before you can have consciousness. There has to be an experiencing entity.

What do you think of that?

No, don't lurk, engage with what other people are saying. I'd like for example to continue the discussion about the Computational Theory of Mind and the observer-dependency argument against it. You seemed to be saying you didn't yet understand that argument, well try to understand it.

It's all speculation and I'm getting the VERY REAL sense that most of the members here are firm in their belief that it's literally impossible that science might provide a solution to the "Hard Problem" of consciousness. — GLEN willows

I think you need to stop making sweeping statements about what members here believe. We're individuals, we have different beliefs and approaches. You should also stop making similar statements about what philosophy thinks and does. Again, different philosophers have very different and often opposing views.

Instead, deal with what individuals have said, here and elsewhere. And also tell us what you think yourself, and why.

It's not all speculation. It's not speculation to state that a weather simulation on a computer will not cause rain, for example.

↪180 Proof seemed to think that neural nets might do it, and as far as I know neural nets are still computational (and observer-dependent). — Daemon

Is that the bit where you don't know what it means?

↪Daemon
I would like to know who thinks computation can cause consciousness. Is that a pan-psych argument? That a thermometer has a small level of consciousness. That’s definitely not my argument. — GLEN willows

Well I thought you were saying quantum computation might cause consciousness, and ↪180 Proof seemed to think that neural nets might do it, and as far as I know neural nets are still computational (and observer-dependent). Chalmers may have said that about the thermostat.

Can you answer my questions?

Well I'll try, but not any more tonight, I'm going to sleep.

What is your argument?

There are those who say we may never understand consciousness, the so-called Mysterians, led by British philosopher Colin McGinn. And David Chalmers identified what he called the hard and easy problems. But I don't think Chalmers says we can never understand it.

No, if anything it's the opposite. The argument aims to show that computation can't cause consciousness because computation is an observer-dependent phenomenon, whereas the brain (and body) and the consciousness they produce are observer-independent phenomena. They are what they are and they do what they do regardless of what external observers think and say.

The argument seems relatively simple and clearly decisive to me, which is one reason I have had a long-term interest in the topic. Another reason is that it is very widely believed that computation could some day cause consciousness, and I feel that I know it can't. In the same way that I know a weather simulation on a computer can't cause rain. It intrigues me that so many people don't see this.

Most humans are not creative. Most don't have awareness of their feelings. — Jackson

Perhaps you would like to elaborate on how you know this? Do you think there's anyone on the forum here who doesn't have awareness of their feelings? All the children I know have been creative from an early age, creating drawings and imaginative stories, and they all have clearly had awareness of their feelings. If most humans are not creative and don't have awareness of their feelings, have I just been incredibly lucky with the humans I've met?

Here are children's drawings from around the world: https://multiculturalkidblogs.com/2015/08/12/wordless-wednesday-kids-drawings-around-world/

No one here KNOWS what consciousness is, so none of us can predict whether it will be something that can be explained with neuroscience, or created and put into a robot or computer. — GLEN willows

But it isn't the case that we know NOTHING about what consciousness is, we have gained a vast amount of knowledge about it in my lifetime and that knowledge is growing at an accelerating rate. We know enough about consciousness to know what can't cause it, and the observer-dependent nature of computation means that computation can't be the cause.

Don't you think it is now being explained by neuroscience?

As for AI most of the arguments are based on what computers can do now, not what they can do in the future, ex. quantum computers. — GLEN willows

:up: ... or advanced neural nets (post von Neumann systems). — 180 Proof

Do they deal with the argument I raised about the observer-dependent nature of digital computation?

How?

You will have to elaborate as to why you consider functions to be observer dependent, but not the existence of other minds. — sime

I still think you are misinterpreting the meaning of "observer-dependent" as I'm using it.

Money and marriage are observer-dependent phenomena, in that something is only money or a marriage because we say so. Whether an abacus or a PC are carrying out computation is observer-dependent in this sense.

Metals, mountains, molecules and minds are observer-independent. Something is a metal and acts as a metal does regardless of what any external observer may say or think about the matter. And a conscious entity is conscious (has experiences, feels stuff) regardless of the views of external observers.

Most don't have awareness of their feelings. — Jackson

What an idiotic remark! Have you actually met and interacted with humans?

I think perhaps you are mistaking the status of the "observer" in "observer-relative". None of this is closely related to the (pseudo-) Problem of Other Minds.

He proposed that if you replace each neuron with a tiny computer computing the right exit signals based on the input, — Hillary

That seems to presuppose that neurons operate in this kind of linear fashion. They don't. The idea is scientifically naive.

I understand what the Chinese Room is doing...correct me if I'm wrong. It's dispelling the notion that the computer is doing anything that requires complex thought. — GLEN willows

Hm. Well then I don't think you do understand the Chinese Room. What it shows is that semantics is not intrinsic to the computer. Some ten years later however Searle came to the realisation that syntax is not intrinsic to the computer either. He said he ought to have realised that years earlier. It's this point that I believe makes computer consciousness a logical impossibility

Once again I find it useful to consider the analogy with an abacus. Here are some instructions on how to use one:

Choose whether you want your ones to be the top row or the bottom row. You can either start at the top and increase the place value as you go down, or start at the bottom with your tens, hundreds, thousands and so on above you.

The user chooses how the positions of the beads are to be interpreted. Neither the syntax (the rules for moving the beads) nor the semantics (the meaning of the position of the beads) are intrinsic to the physics of the abacus.

The situation is no different with a digital computer. The designers and users of the computer choose how the physical states and features of the computer are to be interpreted. So whether or not a computer is carrying out computation is an observer-dependent matter.

Regardless, it still doesn't mean that an AI device can't eventually develop such advanced capacities, and even consciousness. — GLEN willows

According to the argument from the observer-dependence of computation itself, the computer has no such capacities, advanced or primitive.

Note that consciousness, in humans, or dogs, is not an observer-dependent phenomenon. Whether you (or your dog if any) are conscious is not a matter of interpretation by an external observer.

This is the argument that I take to decisively rule out computer consciousness.

Read Searle - not a fan. The Chinese Room hasn't fared well over time. — GLEN willows

I know that many objections have been raised, but I haven't been convinced by any of them. Searle's argument from the observer-dependent nature of digital computation seems straightforward and decisive to me: whether a computer is carrying out computation or not is determined by us, outside observers, and is not inherent in the physics of the computer. In exactly the same way, arithmetic is not inherent in the physics of an abacus.

Can you present an argument against that?

And did you understand what he was saying in that introduction about the futility of "how do we avoid dualism?" discussions?

So if consciousness and the material brain are not literally the same thing, how do we avoid dualism? — GLEN willows

Hi Glen, I don't really like it when people here refer me to some external article, book or paper, but these few pages from the introduction to Searle's "Mind: A Brief Introduction" say it far better than I can.

https://www.amazon.co.uk/Mind-Brief-Introduction-Fundamentals-Philosophy-ebook/dp/B00VQVP8V8?asin=B00VQVP8V8&revisionId=cf1f5c33&format=1&depth=1

Many of the discussions on this forum are weighed down by the conceptual baggage he identifies, and this one is no exception.

He also deals, in my view conclusively, with the question of the possibility of computer consciousness, which has also cropped up in the present discussion and many others. Can digital computation produce consciousness? No, because digital computation is an observer-dependent phenomenon, while consciousness is observer-independent.

I'm being careful NOT to claim it WILL be explained by science, just that it could. — GLEN willows

And will consciousness then be "eliminated"? Could you give an entirely speculative picture of how that might happen?

Someone with a photographic memory has to have full detailed recall of every 'snapshot' their eyes have taken in since the day they were born. — universeness

Why?

Why is that not evidence of a photographic memory? Nothing you've said helps answer that question.

Eidetic/photographic memory in humans is not scientifically proven. Every tested case has failed. — universeness

https://www.nationalgeographic.co.uk/travel-and-adventure/2017/11/incredible-british-artist-can-draw-whole-city-memory

It's said that he draws cityscapes after a brief glance, accurate down to the number of windows in buildings. Has that case failed?

From what I can see it is a configuration as whole, rather than a boundary (appearance of separation) itself, that enables consciousness. — Possibility

Ok, but I am positing the separation of organisms as a prerequisite.

I do think there is awareness, but not consciousness. The bacterium as a whole is not aware of the attractant’s location. But a chemical process within this group is aware of changes in the chemical gradient of the attractant (allowed through by the chemical process in the cell wall). — Possibility

But awareness is an aspect of consciousness. The chemical process isn't aware of things in the way you are aware of things.

You missed in that, too. I am a professional programmer and work with computers since 1982! — Alkis Piskas

It's very common for programmers not to understand how computers work. It isn't taught, even on university level computation courses.

Do you mean that when we switch a light on/off and its result, when we turn a device on/off and its result, etc. occur in our minds only? — Alkis Piskas

That's a very muddled question, I try to express myself more clearly than that.

To explain what I mean I can draw an analogy with an abacus. The beads on the abacus wire are analogous to the computer's electronic and mechanical components.

The abacus user may say that the bottom line of beads represents units. Here's the analogy with a computer:

Input Voltages for Logic Gates

Logic gate circuits are designed to input and output only two types of signals: “high” (1) and “low” (0), as represented by a variable voltage: full power supply voltage for a “high” state and zero voltage for a “low” state. In a perfect world, all logic circuit signals would exist at these extreme voltage limits, and never deviate from them (i.e., less than full voltage for a “high,” or more than zero voltage for a “low”).

However, in reality, logic signal voltage levels rarely attain these perfect limits due to stray voltage drops in the transistor circuitry, and so we must understand the signal level limitations of gate circuits as they try to interpret signal voltages lying somewhere between full supply voltage and zero.
Voltage Tolerance of TTL Gate Inputs

TTL gates operate on a nominal power supply voltage of 5 volts, +/- 0.25 volts. Ideally, a TTL “high” signal would be 5.00 volts exactly, and a TTL “low” signal 0.00 volts exactly.

However, real TTL gate circuits cannot output such perfect voltage levels, and are designed to accept “high” and “low” signals deviating substantially from these ideal values.

“Acceptable” input signal voltages range from 0 volts to 0.8 volts for a “low” logic state, and 2 volts to 5 volts for a “high” logic state.

“Acceptable” output signal voltages (voltage levels guaranteed by the gate manufacturer over a specified range of load conditions) range from 0 volts to 0.5 volts for a “low” logic state, and 2.7 volts to 5 volts for a “high” logic state

This demonstrates what I meant by "The reduction to 0/1 states occurs in our minds, and not in the physics of the machine".

In the case of the abacus, the status of the bottom row of beads as representing units is analogous. It's in our minds, not in the physics of the abacus.

I don't know what does "mind" mean to you, but the functioning of robots, like computers, is based on electronic circuits. And these circuits work on the basis of rudimentary and logic (AND, OR, XOR, etc.), which are reduced into 0/1 states. This occurs at a "low level". — Alkis Piskas

The reduction to 0/1 states occurs in our minds, and not in the physics of the machine.

At a higher level, human beings use programming, which can involve quite sophisticated and intelligent algorithms, and this programming --software-- is then "translated" into low level commands for the computer/robot firmware and hardware.

You can see and try to understand these words. That exemplifies "mind". Unlike a computer, your mind is in the physics (biochemistry) of the brain.

Anybody who thinks you could load a mind into a digital computer in the way you suggest doesn't understand how either brains or computers work.

The organism is not really separate from its environment, — Possibility

My contention is that it is separate from its environment in a particular, crucial way. Non-living things are not separated in the same way.

Integration seems to me the prerequisite here for consciousness. — Possibility

Can you say more about why?

‘Non-conscious sensory mechanisms’ are just members with particularly useful awareness characteristics, like the cell wall. — Possibility

The non-conscious mechanism I am using as an example, chemotaxis in bacteria, is a series of chemical reactions resulting in swimming behaviour that tends to take the bacterium closer to an attractant. There is no awareness. The behaviour does look like it involves awareness (how can the bacterium swim towards the attractant if it isn't aware of its location?) but we know about the chemical process in exquisite detail, and we can see that the process is non-conscious.

It's quite a complicated (and very interesting) business in reality: https://www.anthroencyclopedia.com/entry/cargo-cults

The Petri dish isn't a boundary of the appropriate type. With single-celled organisms, the boundary is the cell wall.

Individual bacteria have non-conscious sensory mechanisms.

I wonder if your "integration" is another way of talking about the boundary, about the way the organism is separate from its environment?

What is "integration" in the sense in which you are using it?

A cargo cult is an indigenist millenarian belief system, in which adherents perform rituals which they believe will cause a more technologically advanced society to deliver goods. These cults were first described in Melanesia in the wake of contact with allied military forces during the Second World War.

Notable examples of cargo cult activity include the setting up of mock airstrips, airports, airplanes, offices, and dining rooms, as well as the fetishization and attempted construction of Western goods, such as radios made of coconuts and straw. Believers may stage "drills" and "marches" with sticks for rifles and use military-style insignia and national insignia painted on their bodies to make them look like soldiers, thereby treating the activities of Western military personnel as rituals to be performed for the purpose of attracting the cargo.

________________________

I see similarities between the cargo cults and the absurd, unscientific fantasy of mind-uploading.

I'm not imagining that a sea sponge is conscious. It has the non-conscious sensory mechanisms from which I think consciousness developed.

The discussion so far has prompted a lot of thinking on my part so thanks for that.

I've been reading and watching lectures about "What Is Life?". Living organisms are described by the biologists as "bounded entities". Identity then is something a bacterium has, without being conscious.

This kind of non-conscious identity is a prerequisite for consciousness.

I think bounded entities developed, by chance, perhaps around deep sea vents. I think consciousness developed out of non-conscious sensory mechanisms in those bounded entities.

Can you tell me what's false about that?

When you're unconscious Bert1, is it like anything? It isn't for me. I'm pretty sure that's the same for everybody.

I have been unconscious when asleep, when I hit myself on the head with a pickaxe, and when I had a general anaesthetic. I am confidently expecting to be unconscious when I'm dead.

We've got all this complex machinery in our heads, the most complex thing we know about, and it can be switched off with a pickaxe or anaesthetic.

If it isn't like anything to be you, when you're unconscious, so you understand what unconsciousness is, and you understand the effects of anaesthetics and suchlike, and their relationship to the complex mechanisms, then why would you think that consciousness would be found in the absence of those mechanisms?

I don't actually know what the latest biological definition of life is — bert1

I've been reading about this the last couple of days, and watching videos. There are many definitions, depending on the focus of the person providing the definition. But I was really asking why you make the distinction. I see living organisms as being potential centres of consciousness. They are "bounded entities", a phrase used Nobel Prize winner and President of the Royal Society, Sir Paul Nurse, in this lecture on "What is Life?". https://youtu.be/8-cTlKVsvvM

My view could perhaps be: before x can be conscious, there has to be a conscious p, q and r. — bert1

And before p, q and r can be conscious?

The idea I'm contemplating is that the "suddenness" of the onset of conscious experience may be due to the nature of conscious experience, rather than to the sudden crossing of some threshold. Cases like blindsight, where a person is able to avoid obstacles they claim to be entirely unable to see, may be relevant here.