How LLM-based chatbots work: their minds and cognition

Pierre-Normand

This thread is devoted to discussing what lessons can be drawn from the philosophy of mind in order to improve our understanding of the (at least apparent) cognition of LLM-based chatbots, and, from the opposite direction, what their performances, and our knowledge of their architecture, entail for various theses in the philosophy of mind (e.g. regarding what it means to think, reason, understand, intend, etc.)

The purpose, therefore, is to compare and contrast the "minds" (or cognition) of LLMs and humans in a philosophically and technically informed way. AI-skeptics and AI-enthusiasts alike are welcome to participate in this thread. By "AI-skeptic" I mean to refer to people who think chatbots are being anthropomorphised too much. By "AI-enthusiasts," I don't mean to refer to people who think AIs will have mostly positive impacts on society but rather to people who think quick dismissals of their capabilities often are expressions of anthropocentrism.

I am myself both a skeptic and an enthusiast. I often read claims in other AI threads that I wish to respond to but that would detract from the original topic. So, when appropriate, I will redirect and respond here. But I don't wish to make this thread my personal playground either. Anyone is free to share ideas (or raise issues) here that are relevant to understanding how LLMs work and what the philosophical significance of their performances are.

On edit: Some of the topics that I'd like to discuss here, I've already begun to explore in my two older AI threads Exploring the artificially intelligent mind of GPT4 and Exploring the artificially intelligent mind of Claude 3 Opus. Those threads, however, were created to report on experiments and discussions with the chatbots. In this new thread, I aim more at encouraging discussions between TPF users. If posters wish to illustrate their arguments with snippets of their conversation with AIs, I would encourage them to put those behind spoilers.

Showmee

Regardless of how “human” large language models may appear, they remain far from genuine artificial intelligence. More precisely, LLMs represent a dead end in the pursuit of artificial consciousness. Their responses are the outcome of probabilistic computations over linguistic data rather than genuine understanding. When posed with a question, models such as ChatGPT merely predict the most probable next word, whereas a human truly comprehends the meaning of what she is saying.

Pierre-Normand

Regardless of how “human” large language models may appear, they remain far from genuine artificial intelligence. More precisely, LLMs represent a dead end in the pursuit of artificial consciousness. Their responses are the outcome of probabilistic computations over linguistic data rather than genuine understanding. When posed with a question, models such as ChatGPT merely predict the most probable next word, whereas a human truly comprehends the meaning of what she is saying. — Showmee

An argument has been made, though, by researchers like Ilya Sutskever and Geoffrey Hinton, that in order to do so much as predict the word that is most likely to follow at some point in a novel or mathematics textbook, merely relying on surface statistics would yield much poorer results than modern LLMs display. The example provided by Sutskever is the prediction of the name of the murderer at the moment when it is revealed in a detective story. In order for the model to produce this name as the most probable next word, it has to be sensitive to relevant elements in the plot structure, distinguish apparent from real clues, infer the states of minds of the depicted characters, etc. Sutskever's example is hypothetical but can be adapted to any case where LLMs successfully produce a response that can't be accounted for by mere reliance on superficial and/or short range linguistic patterns.

Crucially, even occasional success on such tasks (say, correctly identifying the murderer in 10-20% of genuinely novel detective stories while providing a plausible rationale for their choice) would be difficult to explain through surface statistics alone. If LLMs can sometimes succeed where success seemingly requires understanding narrative structure, character psychology, and causal reasoning, this suggests at least some form of genuine understanding rather than the pure illusion of such.

Additionally, modern chatbots like ChatGPT undergo post-training that fine-tunes them for following instructions, moving beyond pure next-token prediction. This post-training shifts the probability landscape to favor responses that are not merely plausible-sounding but accurate and relevant however unlikely they'd be to figure in the training data.

frank

↪Pierre-Normand

My brother is a software engineer and has long conveyed to me in animated terms the ways that C++ mimics the way humans think. I have some background in hardware, like using a compiler to create and download machine language to a PROM. I know basically how a microprocessor works, so I'm interested in the hardware/software divide and how that might figure in human consciousness. I think a big factor in the LLM consciousness debate is not so much about an anthropocentric view, but that we know in detail how an LLM works. We don't know how the human mind works. Is there something special about the human hardware, something quantum for instance, that is key to consciousness? Or is it all in the organic "software"?

So how do we examine the question with a large chunk of information missing? How do you look at it?

Pierre-Normand

Superficially, one might think that the difference between an AI is exactly that we do have private, hidden intent; and the AI doesn't. Something like this might be thought to sit behind the argument in the Chinese Room. There are plenty here who would think such a position defensible.

In a Wittgensteinain account, we ought avoid the private, hidden intention; what counts is what one does.

We can't deduce that the AI does not have private sensations, any more than we can deduce this of our human counterparts. Rather, we seem to presume it. — Banno

This is redirected from this post in the thread Banning AI Altogether.

Regarding the issue of hidden (private) intents, and them being presupposed in order to account for what is seen (public), what encourages the Cartesian picture also is the correct considerations that intentions, like beliefs, are subject to first person authority. You don't need to observe your own behavior to know what it is that you believe or intend to do. But others may indeed need to presuppose such mental states in order to make sense of your behavior.

In order to fully dislodge the Cartesian picture, that Searle's internalist/introspective account of intentionally contentful mental states (i.e. states that have intrinsic intentionality) indeed seem not to have fully relinquished, an account of first person authority must be provided that is consistent with Wittgenstein's (and Ryle and Davidson's) primary reliance on public criteria.

On first-person authority, I’m drawing on Rödl’s Kantian distinction between knowledge from receptivity and knowledge from spontaneity. Empirical knowledge is receptive: we find facts by observation. But avowals like "I believe…" or "I intend…" are paradigms of spontaneous knowledge. We settle what to believe or do, and in settling it we know it not by peeking at a private inner state but by making up our mind (with optional episodes of theoretical of practical deliberation). That fits a Wittgenstein/Ryle/Davidson picture grounded in public criteria. The authority of avowal is practical, not introspective. So when an LLM avows an intention ("I’ll argue for P, then address Q"), its authority, such as it is, would come not from access to a hidden realm, but from undertaking a commitment that is immediately manifest in the structure of its linguistic performance.

Pierre-Normand

We don't know how the human mind works. Is there something special about the human hardware, something quantum for instance, that is key to consciousness? Or is it all in the organic "software"?

So how do we examine the question with a large chunk of information missing? How do you look at it? — frank

My own view is that what's overlooked by many who contemplate the mystery of human consciousness is precisely the piece LLMs miss. But this overlooked/missing piece isn't hidden inside. It is outside, in plain view, in the case of humans, and genuinely missing in the case of LLMs. It simply a living body embedded in a natural and social niche. In Aristotelian terms, the rational, sensitive and nutritive souls are distinct faculties that each presuppose the next one. What's queer about LLMs is that they manifest sapience, the capabilities we identify with the rational soul, and that they distill through a form a acculturation during the process of pre-training on a massive amount of human texts, but this "soul" floats free of any sensitive or nutritive soul.

The process of pre-training really does induct an LLM into many forms of linguistic life: norms of giving and asking for reasons, discourse roles, genre conventions. But this second nature "floats" because it lacks the first-nature ground (nutritive and sensitive powers) that, for us, gives rational life its stakes: human needs, perception-action loops, personal/social commitments and motivations.

frank

It simply a living body embedded in a natural and social niche. — Pierre-Normand

Even with our embeddedness taken into consideration, we still don't have a working theory of consciousness which we could use to assess AI's. Do we forge ahead using philosophical attitudes instead?

Second question: analog-to-digital technology is relatively advanced at this time. If a system included both LLM, sight, hearing, pressure sensing, some robotic capability, and someone to talk to, do the you think it would then be more likely to develop human-like sapience?

Metaphysician Undercover

In order for the model to produce this name as the most probable next word, it has to be sensitive to relevant elements in the plot structure, distinguish apparent from real clues, infer the states of minds of the depicted characters, etc. Sutskever's example is hypothetical but can be adapted to any case where LLMs successfully produce a response that can't be accounted for by mere reliance on superficial and/or short range linguistic patterns. — Pierre-Normand

This is not true. To predict the name of the murderer in the novel, does not require that the LLM does any of that. It requires only that the LLM is able to predict the habits of the author. It needs to "know" the author. You ought to recognize that an LLM does not at all "think" like a human being does, it "thinks" like a computer does. That's two completely different things. One might ask, which form of "thinking" is better, but we can't really say that one is doing what the other does, as you suggest here. We use the same word, "think", but that's only because if the LLM companies said that LLMs "process", and human being "think", it wouldn't be as effective for their marketing.

How LLM-based chatbots work: their minds and cognition

Welcome to The Philosophy Forum!

Categories

More Discussions