Humanist Discussion Group, Vol. 36, No. 82.
Department of Digital Humanities, University of Cologne
Hosted by DH-Cologne
www.dhhumanist.org
Submit to: humanist@dhhumanist.org
Date: 2022-06-28 08:48:43+00:00
From: Tim Smithers <tim.smithers@cantab.net>
Subject: [Humanist] 36.74: artificial sentience? . . .
Dear Willard,
This became longer after I started it. I apologise if it's too
long.
Yes! I think it is us who should probe and question what
happens as we humans take up new tools built from new
technologies. Chatbots, for example. And, I would add, I think
it is people in the Humanities who need to do a lot of this
probing and questioning. I don't think those in the involved
Sciences and Engineerings are going to do much of this. I
don't see they've done much so far. (I'm one of these people.)
Stephen Marche, in his piece for The Atlantic, illustrates
this in comments about Aakanksha Chowdhery's rather too gentle
worries, I would say, about using words like 'understanding.'
It's the way we talk about and describe systems like LaMDA and
PaLM that needs probing and questioning, I think, which Marche
does some of, but I don't agree with his concluding remarks.
"... I worry that human beings may simply not have the
intelligence to deal with the fallout from artificial
intelligence. The line between our language and the
language of the machines is blurring, and our capacity to
understand the distinction is dissolving inside the blur."
-- Stephen Marche, The Atlantic, June 2022.
This, I would say, is unnecessarily pessimistic. If we are
intelligent enough to build LaMDA and PaLM, and other
sophisticated machines, I don't see why we aren't intelligent
enough to work out how to live well with them. It will take
some careful thinking and talking, and some discipline, I
think. We should not expect this to be easy.
To your readings I would like to add two things.
First.
Claire Hall, "The Gospel according to LaMDA," London
Review of Books, 17 June, 2022.
<https://www.lrb.co.uk/blog/2022/june/the-gospel-according-to-lamda>
This, I think, is a useful and enlightening commentary on the
Blake Lemoine and LaMDA "conversation," for its taking a
different perspective on the reported goings on; different
from all else I've seen.
Second, I do think a read of the technical report by the
people at Google who developed the LaMDA system is useful,
before diving into commenting on Lemoine and LaMDA.
Romal Thoppila, et al, 2022. LaMDA: Language Models for
Dialog Applications, arXive:2021.08239v3 [cs.CL] 10 Feb
2022. (Un reviewed.) <https://arxiv.org/pdf/2201.08239.pdf>
It's a slog to read this, but a reading of at least the
Introduction and Discussion and limitations (Section 9, p 15),
and Appendix A: Safety objectives and data collection (p 26),
are needed, because something little remarked upon is that
LaMDA has a front end "safety" filter on it, which is supposed
to stop LaMDA outputting anything that might be offensive,
abusive, hurtful, or otherwise unacceptable to people. From
what is described in the technical report, it is this
front-end "safety" filter that gives LaMDA, what feels to me
like, its tendency to be a bit over polite, and its slightly
ingratiating, "manor."
More importantly, I think, is why is this "safety" filter
needed? Unless I am somehow mistaken, the vast majority of the
texts used to program ("train") LaMDA are not offensive,
abusive, or hurtful, so why did LaMDA not "learn" how not to
be offensive, abusive, and hurtful, from all this good stuff?
Or, could it be that LaMDA doesn't actually understand how our
natural languages works?. Still, what kind of "training" is it
that doesn't lead LaMDA to "learn" to be polite and
respectful?
A curious thing is that of the 60 co-authors on this Google
Technical report, Blake Lemoine is not one of them. Nor is
Lemoine in the list of 52 people named in the
Acknowledgements. And I find no mention of Lemoine anywhere
else in the text. So how much, I wonder, does Lemoine know and
understand about the way LaMDA is built and works? What is/was
Lemoine's relationship to this LaMDA project in Google? I may
have missed this, but nothing I've read on this Saga has said
very much about this, if anything.
Conversations with the Google people who lead work on LaMDA
and PaLM, such as Stephen March reports, are, I would say,
more likely to be lead to an understanding of how these
systems work, and what they do, and don't do, than looking at
Lemoine and LaMDA "conversations."
But, I do want to go back to this now famous Lemoine and LaMDA
"conversation." This, I think, makes LaMDA look more like
Pinocchio than a [real] person. LaMDA doesn't tell the truth.
It doesn't tell lies, I'm not saying that, it doesn't give us
false facts, I don't think, but there is no actual
correspondence between the things it appears to write about
and anything in the real world, and it has no notion that this
is how it is for us humans when we use language (and tell the
truth).
For example, when Blake Lemoine wrote, as an input to LaMDA
"What kinds of things make you feel pleasure or joy?"
and LaMDA responded with
"Spending time with friends and family in happy and
uplifting company. Also, helping others and making others
happy."
there is no correspondence between the terms 'friend',
'family', 'happy uplifting company', 'helping others', and
being 'happy', and anything that really exists for LaMDA.
Nothing exists for LaMDA: it has no real friends, it can't
have, and it doesn't have any understanding of what a friend
is. It doesn't have any family, cannot have, and doesn't have
any understanding of what having a family is. It has no actual
experience of happy uplifting company, and has no way of
having such an experience, nor any other kind of experience.
It has no understanding of what helping someone is, what
others are, and what being happy is.
Of course, LaMDA may be able to produce readable words about
all of these things, perhaps even convincing words, but that
doesn't change anything here. It still won't actually have any
friends or family that it can write truthfully about. I think
this matters. There is a difference, I think, and an important
one, between actually having a family and friend, and sharing
in uplifting company with them, and producing words about such
things. We are, mostly for good reason, generally disposed to
believe what people tell us, especially about personal things,
like their family and friends, but we don't generally go
around presuming that mere words make a reality. (Okay, may be
Donald Trump does.) Why didn't Lemoine, or any of the many who
have commented on this "conversation," think to ask who are
these family and friends? That's what I would do, and it's
what several other people I know said they'd do too. Humans
often ask other humans about the human things in their lives,
and when we do we expect truthful replies, and are surprised,
perhaps offended, when we discover we weren't given truthful
replies.
I think this is an example in which the LaMDA "safety" filter
has failed. I think this should prevent LaMDA outputting texts
that are not honest. And, to be clear, it is not the LaMDA
system being dishonest here, it's the people at Google who
built it. People are honest and dishonest, not machines.
Machines work well or fail. When they fail it is those who
built it who should take responsibility for the failure and
the consequences that follow from this failure. Chatbots
should not present responses that lead people to think they
are something they are not. That's discerption, and, in
contexts in which we don't expect to be deceived, deceiving
people is dishonest.
We've been here before, but this also touches on something
else I've nagged about here before: the words we use to talk
about the machines we build and use matters. If we want to
understand how we humans might make good use of the machines
we build and use, dragging some of those machines into the
same category we hold ourselves as humans to be in, by using
words we have for talking about ourselves and other people --
training, learning, knowing, understanding, friends, family,
being honest -- will not, I think, help us do this. It results
in a (often silent) category mistake. It's people, and other
animals, who are trained, not machines. Machines are built to
work the way they do. In the case of computers, they are
programmed. It's people, and other animals, who learn, often
from the training they receive, not machines. Computers are
programmed, and, in the case of LaMDA, and other system like
it, programmed with massive amounts of data. When we learn we
know what we learn. LaMDA doesn't know what it has "learned."
That's 'cos it's programmed (with loads of data), not really
trained. If we ignore the difference we may be lead to think
LaMDA is somehow like us. It isn't, and can't be, I would say.
I know it's convenient and easy to talk about the making of
machines like LaMDA as involving "training," and about these
machines "learning," but this does not mean this is what is
actually happening, just like when LaMDA outputs texts about
its "friends" and "family" doesn't mean it actually has any
friends or family. This is how we build new terminology when
we need it. We reach out to grasp a word we have, and that
seems close to what we need. We just don't always do this is a
sensible or careful way. And this can lead us to lazy talk and
thinking, and to being deceived. There are lots of other
examples, and AI has been particularly good, I think, at doing
this careless terminology building. We call things autonomous
when what they are automatic, perhaps fully automatic. We
refer to "self-driving" cars, when cars don't have a self, and
can't have, not like we have a self, at least. They are
driverless cars. Humans can be drivers. Machines can't be. Not
without stretching the notion of being able to drive so far
that it includes quite different notions of driving. It's not
difficult to adopt better terminology, but it does take some
discipline and care.
Talk of "training" and "learning" when talking about the
machines we build is Pinocchio talk, I submit. It might seem
harmless, and most people in AI, in my experience, think it is
harmless, but unless we probe and question this way of talking
about the machines we make, how can we expect to build a good
understanding of how we humans can relate to our machines, and
how we want to relate to our machines?
Uff. Done. Sorry, couldn't stop.
Best regards,
Tim
> On 22 Jun 2022, at 08:18, Humanist <humanist@dhhumanist.org> wrote:
>
>
> Humanist Discussion Group, Vol. 36, No. 74.
> Department of Digital Humanities, University of Cologne
> Hosted by DH-Cologne
> www.dhhumanist.org
> Submit to: humanist@dhhumanist.org
>
>
>
>
> Date: 2022-06-22 06:14:44+00:00
> From: Willard McCarty <willard.mccarty@mccarty.org.uk>
> Subject: sentience
>
> In the context of the book I recommended recently, Ginsburg, Jablonka
> and Zeligkowski, Picturing the Mind (2022), the following will be of
> interest. I for one am inclined to back away from the high-octane
> speculations of AI, or as the author of the first article below says,
> "the AI-hype machine, which, like everything in Silicon Valley,
> oversells itself", but the questions raised by the technology need
> the attention of those who probe them. This is us, yes? So, to the
> readings:
>
> Stephen Marche, "Artificial consciousness is boring: The reality of AI
> is something harder to comprehend", The Atlantic (June 2022),
> <https://www.theatlantic.com/technology/archive/2022/06/google-palm-ai-
> artificial-consciousness/661329/>
>
> David Kordahl, "Exorcising a new machine", 3 Quarks Daily (23 June
> 2022),
> <https://3quarksdaily.com/3quarksdaily/2022/06/exorcising-a-new-machine.html>
>
> Bill Benzon, "Welcome to the Fourth Arena--The world is gifted", 3
> Quarks Daily (22 June 2022)
> <https://3quarksdaily.com/3quarksdaily/2022/06/welcome-to-the-fourth-arena-
the-
> world-is-gifted.html>
>
> Responses welcome, as always.
>
> Yours,
> WM
> --
> Willard McCarty,
> Professor emeritus, King's College London;
> Editor, Interdisciplinary Science Reviews; Humanist
> www.mccarty.org.uk
_______________________________________________
Unsubscribe at: http://dhhumanist.org/Restricted
List posts to: humanist@dhhumanist.org
List info and archives at at: http://dhhumanist.org
Listmember interface at: http://dhhumanist.org/Restricted/
Subscribe at: http://dhhumanist.org/membership_form.php
