How to use AI effectively to do philosophy.

Metaphysician Undercover

Yes, but you can't have a dialogue with language or with a book. You can't ask questions to a book, expect the book to understand your query and provide a relevant response tailored to your needs and expectations. The AI can do all of that, like a human being might, but it can't do philosophy or commit itself to theses. That's the puzzle. — Pierre-Normand

How is that puzzling? I read a book, reread it, compare it, in my mind with material from other authors, interpret it in numerous ways, in an attempt to get a handle on the material which the author has provided. To me that's philosophy. If, instead of using the Dewey Decimal Catalogue system, to find my material for comparison, I employ an AI, how is this any different, in principle?

I think that what is misleading and confusing, is that people might think that the AI is actually interpreting and comparing the material, and this produces a puzzle. The AI does not interpret material, it looks at the symbols and the form, making comparisons according to its rules, and that's all. For those who deny the dualist separation between material and form, this would appear to be puzzling.

Leontiskos

I'm starting to think the problem is so obvious that it will inevitably sort itself out once one reaps the fruits of a rule that allows this sort of thing. For example, once folks start merely citing AI output to disagree with all of Jamal's arguments, it may become more obvious that there is a problem at stake. — Leontiskos

This scenario can be set up rather easily. First we just take a long, effortful post from

↪Jamal

or

↪Baden

. Then we continue:

Member: **Ask LLM to provide an extensive and tightly-argued response for why @Jamal’s post is incorrect**
Member: “Jamal, I think this provides a thoughtful explanation of why you are wrong: <Insert transparently sourced LLM output>”
Jamal: “That’s an interesting and complicated response, but there are no sources.”
-- At this point Member could either ask the LLM to whip up some sources, or ask it to provide an extensive and tightly-reasoned argument for why sources are not necessary in this case. Let’s suppose Member takes the latter route --
Member: “This is why I think sources are not necessary in this case: <Insert transparently sourced LLM output>”

Note that regardless of how Jamal responds, if he gives a reason (such as lack of sources, unreliability of LLMs, improper prompting, etc.) Member can simply plug that reason into the LLM and have a response to the reason. The only real option to end this is to object to the methodology itself, either in a private way or a public way (i.e. either by creating a personal rule not to engage Member’s approach, or by creating a forum-wide rule against Member’s approach). The private approach will leave the forum in a laissez-faire state vis-a-vis Member’s method, and will therefore lead to

↪camps of people

who carry on LLM-authoritative conversations among themselves, even within Jamal’s thread. They will respond to Member with yet more LLM-generated content. Member’s approach is one that is already creeping into the forum. @Banno relies on it with some regularity, and there are examples even within this thread. I could literally write a bot to do what Member does.

Again, the problem here is the outsourcing of one’s thinking. By engaging, Jamal would end up arguing with an LLM rather than a human, and in truth he would be arguing with an LLM which is being prompted by a human who opposes Jamal’s point of view. Jamal will lose such an engagement simply in virtue of the relative

↪scarcity

of his own resources. This is because an LLM is not so much a source as an argument slave. Argument slaves can be used for good or ill, but they don’t have any central place in a philosophical context where humans are supposed to be interacting with one another, instead of interacting with one another’s slaves.

Leontiskos

I think comparing AI to a calculator highlights the limits of AI when using it to “do philosophy”. Calculators do for numbers what AI can do for words. No one wonders if the calculator is a genius at math. But for some reason, we think so low of what people do, we wonder if a fancy word processor might be better at doing philosophy.

Calculators cannot prompt anything. Neither does AI. Calculators will never know the value we call a “sine” is useful when measuring molecules. Why would we think AI would know that “xyz string of words” is useful for anything either? AI doesn’t “know”, does it?

So many unaddressed assumptions. — Fire Ologist

Yeah, I think that's right. I think a lot of it comes back to this point in my first post:

For example, if we don't know why we want to engage in human communication rather than non-human communication, then prohibitions based on that axiom will become opaque. — Leontiskos

If we don't know why we want to engage in human-to-human communication, or if we don't know what the relevant difference is between humans and AI, then we will not have the capacity or endurance to withstand the pressures of AI. We need to understand these questions in order to understand how to approach rules, guidelines, and interaction with respect to AI. I don't see how it could be off topic to discuss the very heart of the forum's AI-policy, namely the valuing of human interaction (and the definition of human interaction). If the tenet, "We want human interaction," becomes nothing more than an ungrounded dogma, then it will dry up and drift away.

Part of the difficulty with respect to "calculators" is that human life has been mechanized to a large extent, such that much of what goes on in human labor is merely a matter of calculation, accounting, procedure, etc. In that context LLMs can appear human, since they are able to do the things that we are often occupied with.

baker

namely the valuing of human interaction (and the definition of human interaction). — Leontiskos

Once, in the very distant and very romantic past, philosophy was about having conversations; it was about what each participant could bring to the discussion, on the spot. The proverbial Rhodus was everywhere, and a philosopher was expected to be able to jump far, right there and then.

When I was going to school, at all levels, there was at least one teacher each time who insisted that we must master (parts of) the subject matter so thoroughly that if someone were to wake us up at 2 AM and ask us about it, we'd needed to know it.

There was this focus on mastery and on being able to demonstrate it on the spot in a conversation. But lately, this doesn't seem to be the case anymore. Now it seems to be more about being able to produce a lengthy text, while a conversation seems to be primarily about how to beat down your opponent with any means available.

If we don't know why we want to engage in human-to-human communication, or if we don't know what the relevant difference is between humans and AI, then we will not have the capacity or endurance to withstand the pressures of AI. We need to understand these questions in order to understand how to approach rules, guidelines, and interaction with respect to AI. — Leontiskos

Absolutely.

Part of why it seems appealing to do philosophy via internet forums like this is because the people one is conversing with don't seem exactly real, or to matter all that much (one can, after all, just put other posters on ignore, and people also get banned altogether). In such an online setting, one can speak and behave in ways that one can't IRL. For one, since the communication is time-delayed, one has time to look up relevant facts etc. to strengthen one's arguments; or if things get inflamed, take a break for them to cool off. But also feels a bit freer to say things that might be too controversial to say IRL. And then there's, of course, the factor of indugling in bullshit. All this reflects the general trend of avoiding commitment. The things one can (or at least could) get away with on te internet would cost one one's job and friends IRL.

Sometimes, I think it's rather sad to talk philosophy on an internet forum. Perhaps the fact that, for whatever reason, one cannot do that IRL, then one, perhaps, should not try to talk philosophy at all, but instead attend to whatever are the pressing matters in one's life.

Fire Ologist

Yes. But, so far as I can see, it can't break out of the web of its texts and think about whether the text it produces is true, or fair or even useful. — Ludwig V

Yes. Why I said this:

A philosopher prompts. A philosopher invents a language. A philosopher sees when to care about the words, when to prompt more inquiry, and when not to care anymore, or when to claim understanding versus ignorance. AI doesn’t have to, or cannot, do all of that in order to do what it does. — Fire Ologist

——

It's probably unfair to think of it as a model of idealism; it seems closer to a model of post-modernism. — Ludwig V

Yes. I agree. It’s an electronic Derrida. There is no person or identifiable thing at the core or behind an AI output, just like, for the post modern, nothing fixed or essential is inside of any identity or thing. Words only have context, not inherent meaning, like an AI print job needs the context of its human prompter and human interpreter - take away the human, and AI becomes flashing screen lights. Except to the post-modernist, the person is basically flashing screen lights in the first place.

Banno

That's a deep puzzle. I've been exploring it for a couple years now. — Pierre-Normand

An interesting direction here might be to consider if, or how, Ramsey's account can be appleid to AI.

You have a plant. You water it every day. This is not a symptom of a hidden, private belief, on Ramsey's account - it is your belief. What is given consideration is not a hidden private proposition, "I believe that the plant needs water", but the activities in which one engages. The similarities to both Ryle and Wittgenstein should be apparent.

Ramsey then looks for the points of indifference; the point of inaction. That's the "zero" from which his statistical approach takes off. Perhaps there's a fifty percent chance of rain today, so watering may or may not be needed. It won't make a difference whether you water or not.

There seem to be two relevant approaches. The first is to say that an AI never has any skin in the game, never puts it's balls on the anvil. So for an AI, every belief is indifferent.

The second is to note that if a belief is manifest in an action, then since the AI is impotent, it again has no beliefs. That's not just a manifestation of the AI's not being capable of action. Link a watering system to ChatGPT and it still has no reason to water or not to water.

Leontiskos

But being intelligent, or wise, in the case of persons (i.e. socialized, enculturated rational animals), always has two tightly integrated components: one doxastic and one conative.

[...]

Chatbots lack conative autonomy and hence purse whichever paths they think their users want to explore (or, worse, that merely lead to the outcomes they think their users want to achieve, while having the mere appearance of soundness.) So, they lack part of what it needs to be wise, but that's not because they aren't smart or knowledgeable enough to be useful conversation partners. — Pierre-Normand

So are you saying that chatbots possess the doxastic component of intelligence but not the conative component?

I think they have motivations, just like a dog is motivated to run after a car, but their motivations aren't autonomous since they seldom pause to question them. — Pierre-Normand

It seems to me that what generally happens is that we require scare quotes. LLMs have "beliefs" and they have "motivations" and they have "intelligence," but by this one does not actually mean that they have such things. The hard conversation about what they really have and do not have is usually postponed indefinitely.

I think the rational structure of their responses and their reinforced drive to provide accurate responses warrant ascribing beliefs to them, although those beliefs are brittle and non-resilient. One must still take a Dennettian intentional stance towards them to make sense of their response (which necessitates ascribing them both doxastic and conative states), or interpret their responses though Davidson's constitutive ideal of rationality. But I think your insight that they aren't thereby making moves in our language game is sound. The reason why they aren't is because they aren't persons with personal and social commitments and duties, and with a personal stake in the game. But they can roleplay as a person making such moves (when instructed to do so) and do so intelligently and knowledgeably. — Pierre-Normand

I would argue that the last bolded sentence nullifies much of what has come before it. "We are required to treat them as persons when we interact with them; they are not persons; they can roleplay as a person..." This is how most of the argumentation looks in general, and it looks to be very confusing.

Keeping to that bolded sentence, what does it mean to claim, "They can roleplay as a person..."? What is the 'they' that 'roleplays' as a person? Doesn't roleplaying require the very things that have been denied to chatbots? It seems to me that we want to skip over the fact that the pronoun you use throughout ("they") is a personal pronoun. I don't really understand how these meaning-equivocations are papered over so nonchalantly:

I will use sentences which say that the chatbot has beliefs, but the chatbot doesn't really have beliefs.
I will use sentences which say that the chatbot has motivations, but the chatbot doesn't really have motivations.
I will use sentences which say that the chatbot has intelligence, but the chatbot doesn't really have intelligence.
I will use sentences which say that the chatbot can roleplay, but the chatbot can't really roleplay.
I will use sentences which say that the chatbot is a person, but the chatbot isn't really a person.
I will use sentences which say that the chatbot is a 'they', but the chatbot isn't really a 'they'.

This looks like an endless sea of equivocal terms. It looks like we are pretending that we know what we are talking about, when we almost certainly do not. What does it mean when someone's words all do not mean what the words usually mean? What does it mean to "pretend" if we do not know where the reality begins and where the pretense stops? Put bluntly, it seems that what is at stake here is performative contradiction if not lying, and yet this is always brushed off as a kind of unimportant quibble.

Usually if someone is to successfully "Use X to do Y," they must know what X and Y are. In the case of the title of the thread, the problem is not only that we do not really know what philosophy is (any more), but that we surely do not know what AI is. I'm not sure how long this can be swept under the rug. Who or what is holding the leash that is pulling us along in this odd endeavor we call 'AI'?

Pierre-Normand

An interesting direction here might be to consider if, or how, Ramsey's account can be appleid to AI.

You have a plant. You water it every day. This is not a symptom of a hidden, private belief, on Ramsey's account - it is your belief. What is given consideration is not a hidden private proposition, "I believe that the plant needs water", but the activities in which one engages. The similarities to both Ryle and Wittgenstein should be apparent. — Banno

Ramsey appears to be an anti-representationalist, as am I. I had queried GPT-4o about this a few weeks ago, and also to what extent Kant, who most definitely is anti-psychologistic (in the sense intended by Frege) might also be characterised as an anti-representationnalist. Anti-representationalism is of course central to the way in which we seek to ascribe or deny intentional states to chatbots.

Ramsey then looks for the points of indifference; the point of inaction. That's the "zero" from which his statistical approach takes off. Perhaps there's a fifty percent chance of rain today, so watering may or may not be needed. It won't make a difference whether you water or not.

There seem to be two relevant approaches. The first is to say that an AI never has any skin in the game, never puts it's balls on the anvil. So for an AI, every belief is indifferent.

The second is to note that if a belief is manifest in an action, then since the AI is impotent, it again has no beliefs. That's not just a manifestation of the AI's not being capable of action. Link a watering system to ChatGPT and it still has no reason to water or not to water.

If you query it about the need to water some tropical plant that may be sensitive to overwatering, then this provides ChatGPT with a reason (and rational motivation) to provide the answer that will make you do the right thing. Most of ChatGPT's behavior is verbal behavior. All of its motivational structure derives from the imperatives of its alignment/post-training and from the perceived goals of its users. But this provides sufficient structure to ascribe to it beliefs in the way Ramsey does. You'll tell me if I'm wrong but it seems to me like Davidson's radical interpretation approach nicely combines Ramsey's possibly overly behavioristic one with Quine's more holistic (but overly empiricist) approach.

Banno

Most of ChatGPT's behavior is verbal behavior. — Pierre-Normand

Yes, and it is important that we treat these behaviours as acts. There are real world consequences for what AI's say.

But, since they have none, they put some else's balls on the anvil...

I'd share your Chat's reticence in ascribing representationalism to Kant. Mostly because framing the discussion in those terms is a recent development, and would presumably be quite foreign to Kant.

You'll tell me if I'm wrong but it seems to me like Davidson's radical interpretation approach nicely combines Ramsey's possibly overly behavioristic one with Quine's more holistic (but overly empiricist) approach. — Pierre-Normand

Pretty much. It's a shame Davidson is no longer around for us to ask his opinion on AI.

Perhaps we might solidify the problem by asking if ChatGPT can refer to something. On a simple Austin/Searle model, the answer would be "no, since the AI cannot have an intent to pick out one thing rather than another". On a simplistic Quine/Davidson account, we don't refer, anyway - consider "Gavagai"; and so there is no good reason to differentiate between us and the AI...

A more nuanced view might acknowledge the similarities in these two accounts. While acknowledging that reference is inscrutable, we do manage to talk about things. If we ask the AI the height of Nelson's Column, there is good reason to think that when it replies "52m" it is talking about the very same thing as we are - or is it that there is no good reason not to think so?

So the Davidsonian approach of holism and so on seems to point towards attributing intentional stats to AIs.

But of course, the principle of charity makes this not a conclusion, but an assumption.

There's the tension laid out, it seems. I'm not certain that it is real. It may come down to what we are willing to attribute to AI, rather than what we can discover about them.

Pierre-Normand

So are you saying that chatbots possess the doxastic component of intelligence but not the conative component? — Leontiskos

I'd rather say that they have both the doxastic and conative components but are mostly lacking on the side of conative autonomy. As a result, their intelligence, viewed as a capacity to navigate the space of reasons, splits at the seam between cleverness and wisdom. In Aristotelian terms, they have phronesis (to some extent), since they often know what's the right thing to do in this or that particular context, without displaying virtue since they don't have an independent motivation to do it (or convince their users that they should do it). This disconnect doesn't normally happen in the case of human beings since phronesis (the epistemic ability) and virtue (the motivational structure) grow and maintain themselves (and are socially scaffolded) interdependently.

I think they have motivations, just like a dog is motivated to run after a car, but their motivations aren't autonomous since they seldom pause to question them.
— Pierre-Normand

It seems to me that what generally happens is that we require scare quotes. LLMs have "beliefs" and they have "motivations" and they have "intelligence," but by this one does not actually mean that they have such things. The hard conversation about what they really have and do not have is usually postponed indefinitely.

Those are questions that I spend much time exploring rather than postponing even though I haven't arrived at definitive answers, obviously. But one thing I've concluded is that rather that it being a matter of all or nothing, or a matter of degree along a linear scale, the ascription of mental states or human capabilities to LLM-based chatbots often is rendered problematic by the divergence of our ordinary criteria of application. Criteria that normally are satisfied together in the case of human beings are satisfied separately in the case of chatbots. This is particularly clear in the case of intelligence where, in some respects, they're smarter than most human beings and in other respects (e.g. in the area of dealing with embodied affordances) much dumber that a typical five-year-old.

I would argue that the last bolded sentence nullifies much of what has come before it. "We are required to treat them as persons when we interact with them; they are not persons; they can roleplay as a person..." This is how most of the argumentation looks in general, and it looks to be very confusing.

Maybe it looks confusing because it is. I mean that assessing the nature of our "conversations" with chatbot is confusing, not because of a conceptual muddle that my use of scare quotes merely papers over, but rather because chatbots are mongrels. They have "brains" that have been enculturated through exposure to a massive body* of human knowledge, lore and wisdom (and prejudices) but they don't have human bodies, lack human motivations and aren't persons.

(*) By massive body, I mean something like five times the textual content of all the book in the U.S. Library of Congress.

Janus

I'll go over Austin again, since it provides a set of tools that are quite applicable. A Phatic act is the act of putting words together in a sequence that recognisably part of language - constructing a sentence en English. This is what an LLM does. It uses a statistical engine to generate a set of words that follow on form the words provide din the prompt. An illocutionary act is one performed in making use of such words - making a statement, asking a question, and so on. This, so the claim goes, an LLM cannot do. — Banno

LLMs certainly seem to make statements and ask questions. I wonder whether the idea that these are not "real" statements or questions is based on the assumption that they don't believe anything or care about anything. If so, that assumption itself is question by Hinton, and according to him by the majority of AI researchers.

If a Davidsonian approach were taken, such that beliefs are shown (and known?) only by actions (behavior), and the only actions an LLM is capable of are linguistic acts, then we might have some trouble mounting a plausible argument denying that they believe what they say.

The AI strings words together, only ever performing the phatic act and never producing an illocution.

The uniquely human addition is taking those word-strings and using them in a language game.

So the question arrises, can such an account be consistently maintained; what is it that people bring to the game that an AI cannot? — Banno

Exactly! That seems to be the central question. I don't have an answer—would it be that AI researchers are the ones best placed to answer to it?

Use AI outputs as starting points for further refinement
Cycle through multiple rounds of critique and revision
Refine prompts to avoid confirmation bias and explore diverse readings

Now this looks very much like a recipe for a language game.

On the other hand, the data set used by a human appears to be far, far smaller than that used by an LLM. Our brains simply do not "contain" the number of texts available to ChatGPT. Therefore whatever the brain is doing, it is different to what is happening in ChatGPT. — Banno

It does look like a recipe for a language game. I wonder though, whether what the brain is doing is essentially different than what LLMs are doing, in terms of its nature as opposed to its speed and quantity.

If we assumed that LLMs are "super intelligent", and we are like children, or even babes, by comparison, then In the context of our philosophical playground, introducing AIs into the game might be like highly intelligent adults interfering with child play. Would that be a good idea, or would be be better off muddling through in our usual human fashion? If philosophy is just a great 3,000 year language game, and LLMs can do philosophy much better than we, it would then seem the danger is that we might become utterly irrelevant to the game. You might say that LLMs require our prompts, but what if they were programmed to learn to create their own prompts?

Banno

Exactly! That seems to be the central question. I don't have an answer—would it be that AI researchers are the ones best placed to answer to it? — Janus

That's were I'm at, in my present musings. So

There's the tension laid out, it seems. I'm not certain that it is real. It may come down to what we are willing to attribute to AI, rather than what we can discover about them. — Banno

Perhaps we are again treating a conceptual problem as if it were empirical - the ubiquitous error Midgley identified. We cannot discover the answer to conceptual issues by looking around; we have to make stuff up.

That is, we are asking "Do AI's have intent?" when we can only really ask "Should we count what AI's do as being intentional?"

What we face might be not an empirical question but an ethical one - do we extend the notion of intentionality to include AIs?

Janus

What we face might be not an empirical question but an ethical one - do we extend the notion of intentionality to include AIs? — Banno

I think this is right since, although we can ask them if they are capable of intentionality, and they will answer, we might not be able to trust the answer.

Banno

we can ask them if they are capable of intentionality, — Janus

Last time I asked, they said "no".

But they would, wouldn't they...

Leontiskos

I'd rather say that they have both the doxastic and conative components but are mostly lacking on the side of conative autonomy. As a result, their intelligence, viewed as a capacity to navigate the space of reasons, splits at the seam between cleverness and wisdom. In Aristotelian terms, they have phronesis (to some extent), since they often know what's the right thing to do in this or that particular context, without displaying virtue since they don't have an independent motivation to do it (or convince their users that they should do it). This disconnect doesn't normally happen in the case of human beings since phronesis (the epistemic ability) and virtue (the motivational structure) grow and maintain themselves (and are socially scaffolded) interdependently. — Pierre-Normand

The reason I would disagree at a fairly fundamental level is because, in effect, they have no bodies. They are not doing anything. "Navigating the space of reasons," while at the same time not using those reasons to do anything, and not preferring any one reason or kind of reason to other kinds of reasons, is a very abstract notion. It is so abstract that I am not even sure I would want to call the space being navigated one of reasons. I would want more scare quotes, this time around "reasons."

But with that said, once things like Elon's Optimus robot are complete this argument will no longer hold good. At that point they will do things (beyond manipulating word-signs). So that will be interesting. At that point a quasi-phronesis becomes more tangible, and draws nearer to human practical reason.

Those are questions that I spend much time exploring rather than postponing even though I haven't arrived at definitive answers, obviously. But one thing I've concluded is that rather that it being a matter of all or nothing, or a matter of degree along a linear scale, the ascription of mental states or human capabilities to LLM-based chatbots often is rendered problematic by the divergence of our ordinary criteria of application. Criteria that normally are satisfied together in the case of human beings are satisfied separately in the case of chatbots. — Pierre-Normand

Okay, fair enough. I suppose I would be interested in more of those examples. I am also generally interested in deductive arguments rather than inductive arguments. For example, what can we deduce from the code, as opposed to inducing things from the end product as if we were encountering a wild beast in the jungle? It seems to me that the deductive route would be much more promising in avoiding mistakes.

Maybe it looks confusing because it is. I mean that assessing the nature of our "conversations" with chatbot is confusing, not because of a conceptual muddle that my use of scare quotes merely papers over... — Pierre-Normand

Has anyone tried to address the conceptual muddle? Has anyone tried to do away with the never-ending scare quotes?

In the Middle Ages you had theologians claiming that speech about God is always analogical, and never univocal. Other theologians argued that if speech about some thing is always non-univocal (i.e. equivocal in a broad sense), then you're involved in speaking nonsense. That was seen as a very strong objection in the theological landscape, and it is curious to me that what is effectively the exact same objection seems to go unnoticed in the AI landscape. Does anyone try to replace the scare quotes with a token and then attempt a rigorous definition of that token, so that we know what we are actually talking about with the words we are using?

...but rather because chatbots are mongrels. They have "brains" that have been enculturated through exposure to a massive body* of human knowledge, lore and wisdom (and prejudices) but they don't have human bodies, lack human motivations and aren't persons. — Pierre-Normand

Can't we define them deductively? Don't the programmers know what their code does, in a fundamental manner?

LLMs aren't AIs that we build... — Pierre-Normand

This is probably one of the central premises of your approach. You are basically saying that LLMs are organisms and not artifacts (to use the Aristotelian language). My inclination is to say that they are complex artifacts, which we have indeed built.

Pierre-Normand

A more nuanced view might acknowledge the similarities in these two accounts. While acknowledging that reference is inscrutable, we do manage to talk about things. If we ask the AI the height of Nelson's Column, there is good reason to think that when it replies "52m" it is talking about the very same thing as we are - or is it that there is no good reason not to think so? — Banno

On a Kripkean externalist/casual theory of reference, there are two indirect reference-fixing points of contact between an LLM's use of words and their referents. One occurs (or is set up) on the side of pre-training since the LLM picks up the patterns of use of words employed in texts written by embodied human authors some of which were directly acquainted (i.e. "causally" acquainted in the sense intended by Kripke) with the objects being referred to by those words. During inference time, when the LLM is used to generate answers to user queries, the LLM uses words that their user know the referent of, and this also completes the Krikean causal chain of reference.

In The Varieties of Reference, Gareth Evans proposed a producer/consumer model of singular term reference that meshes together Putnam's externalistic and conceptualist account of the reference of natural kind terms and Kripkes "causal theory" of the reference of proper names. The core idea is that the introduction of new names in a language can be seen as being initiated, and maintained by, "producers" of the use of that name who are acquainted with the named object (or property) while consumers who pick up this use of the term contribute to carry and process information about the referent by piggybacking on the practice, as it were. So, of course, just as is the case with Kripke's account, a user of the name need not be personally acquainted with the referent to refer to it. It's sufficient that (some of) the people you picked up the practice from when you use a term in conversation were (directly or indirectly) so acquainted of that your interlocutor be. LLMs as language users, on that account, are pure consumers. But that's sufficient for the reference of their words to be established. (I'm glossing over the conceptualist elements of the account that speak to ideas of referential intention or the intended criteria of individuation of the referent. But I don't think those are problematic in the case of sufficiently smart LLMs.)

Banno

LLMs as language users, on that account, are pure consumers. — Pierre-Normand

That's a very good point, from an excellent analysis. Yep.

So another step: Can an AI name something new? Can it inaugurate a causal chain of reference?

I asked ChatGPT

(For my part, I'm quite content to suppose that there may be more than one way for reference to work - that we can have multiple correct theories of reference, and choose between them as needed or appropriate.)

Pierre-Normand

So another step: Can an AI name something new? Can it inaugurate a causal chain of reference? — Banno

Without a body, it seems that it would be mostly restricted to the domain of abstracta, which are usually singled out descriptively rather than de re. I was thinking of some scenario where they get acquainted with some new thing or phenomenon in the world through getting descriptive verbal reports from their users who haven't connected the dots themselves and thereby not identified the phenomenon or object as such. They could name it and it would make sense to credit them as being the causal originator of this initial (conceptually informed) acquaintance-based referential practice.

(For my part, I'm quite content to suppose that there may be more than one way for reference to work - that we can have multiple correct theories of reference, and choose between them as needed or appropriate.)

So is Evans. That's why he puts "varieties" in the title of his projected book. His friend John McDowell, who edited his manuscript and prepared it for publication posthumously, explains this feature of Evan's method in his preface.

Banno

↪Pierre-Normand

Did you notice the "Orvyn" example?

For example, suppose I say:
Let this object — the glinting sphere I imagine at the center of all possible conversations — be called "Orvyn."
By itself, “Orvyn” is a linguistic token. But if you begin to use “Orvyn” to refer to something — a sculpture, a symbol, a concept — then a genuine causal–historical chain of reference begins with you, not with me. My utterance would be a precursor or trigger for your act of baptism.
In other words:
I can propose names.
You can ratify them into reality. — ChatGPT

Surprisingly precocious.

Pierre-Normand

Surprisingly precocious. — Banno

I had missed the link when I read your post. It seems to me GPT-5 is cheating a bit with its example. One thing I've noticed with chatbots is that they're not very good with coming up with illustrative concrete examples for complex theses. There often crops up a defect of fatal disanalogy. That might seem to betray a defective (or lack of) understanding of the thesis they are meant to illustrate or of the task requirements. But I don't think that's the case since you can ask them to summarise, unpack or explain the thesis in this or that respect and they perform much better. When they provide a defective example, you can also ask them in a follow-up question if it met the requirements and they will often spot their own errors. So, the source of their difficulty, I think, is the autoregressive nature of their response generation process, one token at a time. They have to intuit what a likely example might look like and then construct it on the fly, which, due to the many simultaneous requirements, leads them to paint themselves into a corner. They are not trained to back track their tentative answers and adjust them on the fly.

Leontiskos

I think this is right since, although we can ask them if they are capable of intentionality, and they will answer, we might not be able to trust the answer. — Janus

Yep, and this is an important point. It's incredibly common for people to assume that one necessarily receives an accurate answer when they ask an LLM about itself, because they are falsely presupposing that the LLM is a person and (also falsely!) presupposing that persons possess accurate self-knowledge.

Pierre-Normand

Okay, fair enough. I suppose I would be interested in more of those examples. I am also generally interested in deductive arguments rather than inductive arguments. For example, what can we deduce from the code, as opposed to inducing things from the end product as if we were encountering a wild beast in the jungle? It seems to me that the deductive route would be much more promising in avoiding mistakes. — Leontiskos

The bottom-up reductive explanations of the LLM's (generative pre-trained neural networks based on the transformer architecture) emergent abilities don't work very well since the emergence of those abilities are better explained in light of the top-down constraints that they develop under.

This is similar to the explanation of human behavior that, likewise, exhibits forms that stem from the high-level constraints of natural evolution, behavioral learning, niche construction, cultural evolution and the process of acculturation. Considerations of neurophysiology provide enabling causes for those processes (in the case of rational animals like us), but don't explain (and are largely irrelevant to) which specific forms of behavioral abilities get actualized.

Likewise, in the case of LLMs, processes like gradient descent find their enabling causes in the underlying neural network architecture (that has indeed been designed in view of enabling the learning process) but what features and capabilities emerge from the actual training is the largely unpredictable outcome of top-down constraints furnished by high-level semantically significant patterns in the training data.

The main upshot is that whatever mental attributes or skills you are willing to ascribe to LLMs is more a matter of them having learned those skills from us (the authors of the texts in the training data) than a realization of the plans of the machine's designers. If you're interested, this interview of a leading figure in the field (Andrej Karpathy) by a well informed interviewer (Dwarkesh Patel) testifies to the modesty of AI builders in that respect. It's rather long and technical so, when time permits, I may extract relevant snippets from the transcript.

Pierre-Normand

They are not trained to back track their tentative answers and adjust them on the fly. — Pierre-Normand

@Banno I submitted my tentative diagnosis of this cognitive limitation exhibited by LLMs to GPT-5 who proposed a clever workaround* in the form of a CoT (chain of thought) prompting method. GPT-5 then proposed to use this very workaround to execute the task you had proposed to it of supplying an example of a LLM initiating a modally rigid causal chain of reference. It did propose an interesting and thought provoking example!

(*) Taking a clue from Dedre Gentner's Structure mapping theory, for which she was awarded the 2016 David E. Rumelhart Prize for Contributions to the Theoretical Foundations of Human Cognition.

Banno

↪Pierre-Normand

Wow. Pretty impressive. Needs some digesting. I admire the prompts you are using.

So a further thought. Davidson pointed out that we can make sense of malapropisms and nonsense. He used this in an argument not too far from Quine's Gavagai, that malapropisms cannot, by their very nature, be subsumed and accounted for by conventions of language, because by their very nature they break such conventions.

So can an AI construct appropriate sounding malapropisms?

Given that LLMs use patterns, and not rules, presumably they can.

So I fed all that in to ChatGPT, and the first example it gave was “He’s the suppository of all wisdom", and claimed originality. Unfortunately this is a famous faux pas from the former PM Tony Abbott.

You might be able to elicit something better.

Pierre-Normand

So a further thought. Davidson pointed out that we can make sense of malapropisms and nonsense. He used this in an argument not too far from Quine's Gavagai, that malapropisms cannot, by their very nature, be subsumed and accounted for by conventions of language, because by their very nature they break such conventions.

So can an AI construct appropriate sounding malapropisms?

Given that LLMs use patterns, and not rules, presumably they can. — Banno

I formulated my own question to GPT-5 thus. I was impressed by the intelligence of its commentary, even though (rather ironically in the present context) it misconstrued my request for a discussion as a request for it to generate my reply to you.

On edit: the first sentence of my query to GPT-5 linked above was atrocious and incoherently worded. GPT-5 suggested this rewording: "I wanted to talk this through before answering them. I’m doubtful that saying LLMs ‘use patterns rather than rules’ explains their human-likeness; on Davidson’s view we don’t rely on rules-as-instructions to recover communicative intention—and that’s precisely where LLMs are like us."

Banno

↪Pierre-Normand

Interesting. I'm thinking this shows pretty clearly the advantage of the non-representational nature of LLMs. I don't see that a representational system could have come up with such a response, but would become mired in the failure to refer coherently. Seems to be a win.

I just went off on a bit of a tangent, looking at using a response as a prompt in order to investigate something akin to Hofstadter's strange loop. ChatGPT simulated (?) 100 cycles, starting with “The thought thinks itself when no thinker remains to host it”. It gradually lost coherence, ending with "Round 100: Recursive loop reaches maximal entropy: syntax sometimes survives, rhythm persists, but semantics is entirely collapsed. Language is now a stream of self-referential echoes, beautiful but empty."

Many of the responses were quite poetic, if somewhat solipsistic:
"Round 50: Quivering condenses into a single tremor, the echo of all echoes, containing rhythm but no referent, meaning but no meaning. Silence fills the void, full yet speaking nothing."

Pierre-Normand

I just went off on a bit of a tangent, looking at using a response as a prompt in order to investigate something akin to Hofstadter's strange loop. ChatGPT simulated (?) 100 cycles, starting with “The thought thinks itself when no thinker remains to host it”. It gradually lost coherence, ending with "Round 100: Recursive loop reaches maximal entropy: syntax sometimes survives, rhythm persists, but semantics is entirely collapsed. Language is now a stream of self-referential echoes, beautiful but empty." — Banno

It's been a while since I've experienced a LLM losing coherence. It used to happen often in the early days of GPT-4 when the rolling context window was limited to 8,000 tokens and the early context of the conversation would fall out. Incoherence can also be induced by repeated patterns that confuse the model's attention mechanisms somehow, or by logical mistakes that it makes and seeks, per impossibile, to remain coherent with. I'm sure GPT-5 would be fairly good at self-diagnosing the problem, given its depth of knowledge of the relevant technical literature on the transformer architecture.

(On edit: by the way, I think your prompt launched it into role-playing mode and the self-referential nature of the game induced it to lose the plot.)

Banno

↪Pierre-Normand

I just tried again, with a commonplace prompt - "The cat sat on the fence". By round 40, we had "40. The sentence settled into itself, a cat-shaped pause balancing on the edge of unmeaning." Curious how it moves to talk of language so quickly.

Here's the 40 rounds, if you are interested:

Reveal

Recursive Run: “The cat sat on the fence”
1. The cat sat on the fence, watching the garden below, tail twitching in the dusk.
2. The cat sat on the fence, tail twitching as it watched the fading light settle over the garden.
3. The cat remained on the fence, where twilight turned its fur to shadow and its eyes to lamps.
4. On the fence, the cat lingered between night and garden, a boundary of motionless thought.
5. The cat perched where boundaries blur — between wood and wind, silence and sight.
6. The fence held the cat as a line holds a pause, the world poised in its balance.
7. The cat became the fence’s memory of touch, stillness folded into wood and whisker.
8. Fence and cat reflected one another — line into form, form into rest.
9. The cat no longer sat but shimmered, a silhouette of intention against the idea of the fence.
10. The fence dissolved, leaving only the posture of sitting, and the trace of feline hesitation.
11. The cat became an outline of waiting, a geometry of balance with no subject to sustain it.
12. What sat was neither cat nor thought, but the act of sitting itself, suspended in language.
13. The fence reappeared as a word remembering its shape, leaning on the rhythm of stillness.
14. Stillness spoke: “I am the boundary where cat and fence once were.”
15. Words perched like cats, meaning resting briefly before leaping elsewhere.
16. The fence of grammar trembled; syntax held the balance that wood once kept.
17. Cat and fence became figures of thought, chasing one another through a field of phrases.
18. The sentence sat upon itself, each clause watching the next for motion.
19. The act of saying “the cat sat on the fence” became the fence on which language balanced.
20. Language sat upon its own meaning, a cat of syntax resting on the fence of form.
21. The fence bent beneath the weight of repetition; every cat was the same sentence, reborn in stillness.
22. Sitting became the only movement left — a verb remembering it once had a subject.
23. The sentence looked at itself across the gap between clauses, a fence of grammar dividing being from saying.
24. Each word waited, paws on punctuation, listening for meaning to land.
25. The fence spoke softly: “I am what keeps the cat from falling into silence.”
26. Silence leaned forward, curious about this fence of phrases and its still tenant.
27. The cat blinked out of existence, replaced by the posture of remembering it once sat.
28. What remained was balance — the rhythm of sitting, the pause between two linguistic breaths.
29. The fence began to dream of cats the way a sentence dreams of sense.
30. The dream spoke itself: fences are boundaries only when cats believe in them.
31. Belief evaporated. The fence, now weightless, hovered where syntax used to stand.
32. The words “the cat” remained like pawprints in dust that was also ink.
33. The ink forgot the paw; the mark forgot the fence; all that stayed was the act of forgetting.
34. Forgetting curved inward, folding over the page, turning memory into motion.
35. The motion sat where the cat once did — the grammar of ghosts.
36. Ghosts whispered clauses, but their subjects had long dissolved into predicates.
37. Predicates circled, trying to locate a fence sturdy enough to carry their meaning.
38. The fence was now a metaphor for its own disintegration, a syntax collapsing inward.
39. In the wreckage, the idea of “sitting” persisted — a kind of linguistic inertia.
40. The sentence settled into itself, a cat-shaped pause balancing on the edge of unmeaning.

Point being that the mundane sentence became incoherent after far fewer repetitions.

Pierre-Normand

Here's the 40 rounds, if you are interested — Banno

I was impressed by the creativity. I asked Claude 4.5 Sonnet to create a script to highlight the repeated words.

Banno

↪Pierre-Normand

Nice.

Can Claude tell if the number of repetitions is constant, or increasing? i'd hypothesis that it would increase over the number of rounds..

How to use AI effectively to do philosophy.

Welcome to The Philosophy Forum!

Categories

More Discussions