Humanist Discussion Group, Vol. 37, No. 571.
Department of Digital Humanities, University of Cologne
Hosted by DH-Cologne
www.dhhumanist.org
Submit to: humanist@dhhumanist.org
[1] From: Simon Rae <simon.rae@gmail.com>
Subject: Re: [Humanist] 37.569: human error & the infallible computer (74)
[2] From: maurizio lana <maurizio.lana@uniupo.it>
Subject: Re: [Humanist] 37.570: human error & the infallible computer (38)
[3] From: Willard McCarty <willard.mccarty@mccarty.org.uk>
Subject: the belief in infallability (35)
--[1]------------------------------------------------------------------------
Date: 2024-04-28 19:21:45+00:00
From: Simon Rae <simon.rae@gmail.com>
Subject: Re: [Humanist] 37.569: human error & the infallible computer
Way back in 1975 I had just started in my post as a Trainee Computer Advisor at
The City University in London. As I was new to the job it was suggested that I
sit in on a series of lectures that James Haag, a visiting American professor of
Computing Science, was giving to the computing students so that I could get the
feel for ‘academic computing’ rather than the commercial computing environment
that I had come from. At the beginning of his first lecture he announced that,
to the student who kept the best set of notes on his lectures, he would give
their pick of the computer books that were published by his publisher. A
pedagogic ploy to raise interest and engagement that perhaps worked when he was
lecturing in America, but in the UK? Suffice to say I was the only one to keep
any notes at all and, after some deliberation as to whether I should be
disqualified by my staff rather than student status, I was duly awarded my
choice of 10 books, one or two of which I still have.
But with respect to the topic of this particular Humanist thread… one of the
stories that Professor Haag told, possibly during a discussion of computer
fallibility, was about how an error in some American software had nearly caused
WWIII. Apparently this software was designed to keep an eye open for missiles
coming over the horizon into the US. And, on its first time going live, it duly
detected something coming over the horizon! Only the realisation that they had
forgotten to program in the rising of the moon saved a call to the President.
It’s a story that I’ve told several times in the intervening nearly 50 years, to
varying degrees of scepticism, so it was nice to see Humanist [37.569] give me
some sense of closure.
When planning or designing anything we would do well to ask whether we have
allowed for ‘the moon rising’…
Cheers
Simon
Simon Rae
> On 27 Apr 2024, at 08:53, Humanist <humanist@dhhumanist.org> wrote:
>
> Humanist Discussion Group, Vol. 37, No. 569.
> Department of Digital Humanities, University of Cologne
>
> Date: 2024-04-27 07:28:01+00:00
> From: Willard McCarty <willard.mccarty@mccarty.org.uk>
> Subject: the infallible computer: the British Post Office scandal and
beyond
>
> …
>
> In 1985 Brian Cantwell Smith, with the near miss of 5 October 1960[1] in
> mind, made the point to a conference on unintended nuclear warfare:
>
> The 'near miss' of 1960 was due to a computer error in the North
> American Aerospace Defense Command (NORAD) software, which
> mistook a moon rise for Soviet nuclear missiles coming over the
> horizon. Cantwell Smith wraps up his point thus:
>
>> But perhaps this is an overly abstract way to put it . Perhaps,
>> instead, we should just remember that there will always be another
>> moon-rise .
>
> On how many occasions, when the opportunity is right, do we remind our
> audiences that smart machines act according to models of the world,
> not the reality that is modelled? …
>
> Comments welcome/
>
> Yours,
> WM
>
> -----
> [1] See John G. Hubble, "'You are under attack!' The Strange incident of
> October 5". Reader's Digest, April 1961. According to Donald K.
> MacKenzie, ""Hubble's article... remains the best available account of
> the incident." Mechanizing Proof: Computing, Risk, and Trust Inside
> Technology (MIT Press, 2001, n. 4, p. 340).
--[2]------------------------------------------------------------------------
Date: 2024-04-28 10:34:42+00:00
From: maurizio lana <maurizio.lana@uniupo.it>
Subject: Re: [Humanist] 37.570: human error & the infallible computer
mumble mumble...
i think that this thread has a sub-text of great relevance for the main
text (human error).
the Reader's Digest explains that a nuclear war was avoided thank to one
man, NORAD’s deputy commander-in-chief, Canada’s Air Marshal C. Roy
Slemon who didn't act according to the rules.
so here we have not a human error but the human correction of a computer
error through an apparently erroneous decision.
the same is true for one Russian man in nearly identical situation:
Stanislav Yevgrafovich Petrov, lieutenant colonel of the Soviet Air
Defence Forces (for the whole story see
https://en.wikipedia.org/wiki/Stanislav_Petrov).
i mean that we have the correction of an error through a deliberate
disobedience to rules, orders, etc.
if rules, orders, etc. are meant to keep the world 'in order', here we
have the world which remains 'in order' because of an act apparently
opposite, an act of disorder.
then there is also Vasily Aleksandrovich Arkhipov, a Soviet Naval
officer in the Cuba crisis: it was not a matter of ICBMs but once more
we have an act o disorder which keeps the world 'in order'. and we don't
know how many other similar situations.
Maurizio
my body is the light
my body is the way
l. cohen, the gipsy wife
------------------------------------------------------------------------
Maurizio Lana
Università del Piemonte Orientale
Dipartimento di Studi Umanistici
Piazza Roma 36 - 13100 Vercelli
--[3]------------------------------------------------------------------------
Date: 2024-04-29 07:33:12+00:00
From: Willard McCarty <willard.mccarty@mccarty.org.uk>
Subject: the belief in infallability
I wouldn't be at all surprised if technology and theology correlate. Be
that as it may, I take Henry Schaffer's point that 'the computer' (by
which he meant hardware) is not the villain, rather software. I take his
important point with some reservations, however. Is it not the case that
designers of hardware make decisions, technical ones to be sure but
decisions nonetheless. Can we be absolutely sure that these decisions
have nothing whatever of ethics in them as well as economics, aesthetics
and other human concerns? Better, perhaps, to say 'computing systems'.
Again, the problem I pointed to, with reference to the British Post
Office scandal, was belief in the infallibility of 'the computer'. This
term is a convenient fiction, used at the moment to talk about digital
machines, as components (of refrigerators, automobiles, data centres &c)
and as dedicated appliances (laptops, desktops). In all of these is
no distinction commonly makde between hardware and firmware and
software. Common usage makes this an all-purpose term to embrace
many different things. Very convenient but not silly, I think.
How else do we think in words about the whole bundle? When that
Post Office manager referred to 'the computer' as infallible, I wager he
(as happens) was referring to a very large bundle of things that had been
imposed on his workplace and colleagues. But, I wager, his belief is an
ambient and widely shared one in our society, and it needs urgently to be
corrected. Brian Cantwell Smith made a run at this in 1985. It still lurks,
ready to bite.
Comments?
Yours,
WM
--
Willard McCarty,
Professor emeritus, King's College London;
Editor, Interdisciplinary Science Reviews; Humanist
www.mccarty.org.uk
_______________________________________________
Unsubscribe at: http://dhhumanist.org/Restricted
List posts to: humanist@dhhumanist.org
List info and archives at at: http://dhhumanist.org
Listmember interface at: http://dhhumanist.org/Restricted/
Subscribe at: http://dhhumanist.org/membership_form.php
