I suppose Ned Block wants to say that just because our best thinkers think a certain AI thinks, it doesn't mean that particular AI actually thinks. However, Block goes on to say, that if our best thinkers think that AI thinks, it's reasonable to assume that AI thinks. The difference between these claims rests on actual thinking (by AI) vs inferring thinking (by our best thinkers).

The intriguing aspect of the Turing test is that we employ it daily, you're using it on other people and other people are using it on you and both parties (you and other people) seem not to be bothered about the deep flaw in the Turing test which Ned Block's alluding to - there's a difference between an entity (human/AI/whatever) passing the Turing test and that entity actually capable of thinking.

I suppose Ned Block is talking solipsism, the view that the only certain truth is one's own existence and consciosuness, here for he doesn't accept the Turing test's ability to demonstrate actual thinking just like solipsists maintain that it's unknown that anything but themselves are thinking. Alan Turing, on the other hand, seems to be quite content with the Turing test's adequacy to identify human-level cognition (AI).

The problem that Block considers is how to define the mind in a non-circular way, which would mean avoiding mentalistic terms and concepts as part of the definition. (This is a notoriously difficult, perhaps intractable challenge.) What about behaviorism, i.e.diagnosing the presence of a mind in some entity by judging its behavior? This is what the Turing Test is supposed to accomplish. But can we describe the test without recourse to any mentalistic concepts, such as thinking? (Remember, thinking is what we want to define, so we cannot appeal to it as part of the definition, on pain of circularity.) So that's the problem that Block is grappling with here:

For example, one might specify that the judge be moderately knowledgeable about computers and good at thinking, or better, good at thinking about thinking. But including a specification of the mental qualities of the judge in the description of the test will ruin the test as a way of defining the concept of intelligence in non-mentalistic terms. — Ned Block

what problems does Block see with specifying that the judges be good at thinking about thinking? — PhilosophyStudent

The problem with specifying that the judges be good at thinking about thinking is that the judges, based on their own ability and understanding of thinking, would potentially bring bias to the test. The best the judge could do is, as mentioned, determine which respondent is "thinking". That being said, "human level cognition" has a massive bell curve attached to it. AI that falls within the curve may or may not be impressive. Most people are not.

The definition of "thinking" is unclear. An amoeba reacts to stimulus (food), envelopes it, and carries on. Can this be defined as thinking? Plants are known to react to external stimuli by secreting chemicals and communicating to other nearby plants, so...thinking plants? My dog thinks, my cat thinks, in that they respond to external stimuli, have internal motivations and preferences, demonstrate external responses to internal feelings. Both demonstrate fear, anger, frustration and affection. "Thinking" is only valued if it can be effectively communicated to the judge or observer. A lack of ability to communicate will reduce the most fantastic thinker to imbecilic levels, or lower.

defining the concept of intelligence in non-mentalistic terms — Ned Block

Isn't this impossible? The essence of thinking is mentalistic if I understand the term correctly. To then attempt to define thinking in a non-mentalistic way would be self-defeating, like trying to define metal non-metalically. Perhaps Ned Block realizes that defining thinking in mentalistic terms is pointless since there's no way to check it directly. Thus, having to rely on indirect evidence, behavior, a non-mentalistic definition is the first order of business. The catch is we've got to deal with the possibility of the features that constitute the non-mentalistic definition being replicable without the, to us, necessary mentalistic phenomena accompanying it. :chin:

defining the concept of intelligence in non-mentalistic terms
— Ned Block

Isn't this impossible? The essence of thinking is mentalistic if I understand the term correctly. — TheMadFool

I guess it depends on one"s assumption that intelligence and thinking are not necessarily mental processes.

It seems to me that Block is simply assuming everything is "physical" and that there is no "mental" component to reality.