Humanist Discussion Group, Vol. 39, No. 97.
        Department of Digital Humanities, University of Cologne
                      Hosted by DH-Cologne
                       www.dhhumanist.org
                Submit to: humanist@dhhumanist.org




        Date: 2025-07-24 18:55:00+00:00
        From: Gabriel Egan <mail@gabrielegan.com>
        Subject: Re: [Humanist] 39.96: repetition vs intelligence: on LLMs

Dear Humanists

I am pleased to be corrected by
Tim Smithers that he does not,
in fact, hold that machines cannot
wield meanings (as I said he did).
I concede that he never claimed
that they could not, only that
with the advent of Large Language
Models (LLMs) they have not achieved
that feat. I apologize for
misunderstanding him on this point.

Smithers's setting me straight on
this has the benefit of clarifying
the difference between our positions.
His objection to my claim that
LLMs are dealing in meanings is
based on his account of the inner
workings of actual LLMs, not on
what it might be possible for
machines to do some day.

This reduces our difference of
opinion to one that has a long
history in the philosophy of
science, most clearly conveyed
in what is called the Chinese
Room Thought Experiment invented
by John Searle. I'll let Wikipedia
explain:

<<
In the thought experiment,
Searle imagines a person who
does not understand Chinese
isolated in a room with a book
containing detailed instructions
for manipulating Chinese symbols.
When Chinese text is passed into
the room, the person follows the
book's instructions to produce
Chinese symbols that, to fluent
Chinese speakers outside the
room, appear to be appropriate
responses. According to Searle,
the person is just following
syntactic rules without semantic
comprehension, and neither the
human nor the room as a whole
understands Chinese. He contends
that when computers execute
programs, they are similarly
just applying syntactic rules
without any real understanding
or thinking.
 >>

If I understand Smithers's
position, it is essentially
the same as Searle's in that
what matters is that processes
inside the room (= inside the
LLM) are merely mechanical
and without semantic
comprehension. The room (or
LLM) is not dealing in meanings.

Searle intended his thought experiment
to show that on principle no machine
could understand language. I accept
that Smithers is not making that
claim. But isn't he using the same
(mistaken) principle: that because
what goes on inside the room/LLM
doesn't seem to be what humans
do in understanding language, the
room/LLM is not understanding
language?

I'd be interested to know if Smithers
considers that a fair account of
his position.

If it is, my response would
be essentially the one argued
against Searle's view by
Daniel Dennett and others,
which is that if the output
of the room/LLM appears to
fluent Chinese speakers as
meaningful and appropriate
responses, then the room/LLM
is in fact dealing in meanings.

If you look at individual
neurons in the brain, none
of them seems to be doing
anything that corresponds
to understanding and generating
meanings. But collectively the
behaviour exhibited by brains
is that of understanding and
generating meanings. Sure, the
human inside Searle's Chinese
Room doesn't understand Chinese.
But the system as a whole -- the
person, the rulebook, the room --
does understand Chinese.

Looking at what an LLM does with
manipulating weights in a neural
network is like looking at what
neurons do in the brain: it is
looking for the comprehension of
meaning at the wrong level. The
thing we call representation is
emergent in the behaviour of the
system as a whole, and it matters
not what the bits inside the system
are doing to create that emergent
behaviour.

To boil this down even further,
it seems that Smithers would argue
that there is a difference between
a machine seeming to understand
meanings and a machine actually
understanding meanings. And I
would reply that there is no real
difference there.

To my knowledge, Smithers and I
have never met. We are each in
this exchange treating the other
as a meaning-making entity. What
makes him think I understand English?
(See <https://www.visuallanguagelab.com/chinese-room>)

Regards

Gabriel Egan

PS My posting yesteday that discussed
nearly orthogonal vectors in high-dimensional
space was indebted to a talk by Grant
Sanderson on how Attention works in LLMS
(<https://www.youtube.com/watch?v=KJtZARuO3JY>).
In editing my posting I accidentally
cut out the credit to Sanderson.


_______________________________________________
Unsubscribe at: http://dhhumanist.org/Restricted
List posts to: humanist@dhhumanist.org
List info and archives at at: http://dhhumanist.org
Listmember interface at: http://dhhumanist.org/Restricted/
Subscribe at: http://dhhumanist.org/membership_form.php