Open the Pod Bay Doors · Episode 3 of 10 · 24 min
Norbert Wiener's 1960 Warning About What We Ask Machines to Do
Before any machine could do it, Norbert Wiener described the real failure mode of artificial intelligence in 1960: not malice but literalness, a machine pursuing the purpose you specified instead of the one you meant. The episode sets his warning beside Turing's talk on machines taking control and I. J. Good's intelligence explosion, correcting how both are usually quoted.
Who's who
- Norbert Wiener+1 more episodeMIT mathematician and founder of cybernetics; wrote the 1960 Science warning
- Alan TuringMathematician who asked whether machines can think
- Irving John Good+1 more episodeBletchley Park cryptanalyst who described an intelligence explosion
- Arthur L. SamuelIBM checkers-program author who published a refutation of Wiener
- John McCarthy+1 more episodeOrganizer of the Dartmouth project who chose the name artificial intelligence
- Marvin Minsky+1 more episodeDartmouth co-proposer; later consulted for Kubrick
- Sara TuringAlan Turing's mother, whose 1959 book first printed his Manchester talk
- Stanley Kubrick+2 more episodesDirector who hired both Minsky and Good
Where it happens
- Chicago, Illinois+15 more episodesWhere Wiener lectured on December 27, 1959
- Bletchley Park, EnglandWartime codebreaking site where Good worked in Hut 8
- Manchester, England+1 more episodeWhere Turing gave his unpublished 'heretical theory' talk
- Hanover, New HampshireSite of the 1956 Dartmouth summer workshop
Timeline
- 1941Wiener joins the fire-control committee working on anti-aircraft aiming
- October 1950Turing publishes 'Computing Machinery and Intelligence' in Mind
- August 31, 1955Dartmouth proposal coins the phrase artificial intelligence
- May 6, 1960Wiener's paper on moral and technical consequences of automation appears in Science
- September 16, 1960Arthur Samuel's published refutation appears in Science
- 1965Good publishes his paper on the first ultraintelligent machine
Still debated
Historians dispute whether Wiener was heeded or merely admired, and whether the Dartmouth optimism was a sincere forecast or a funding pitch. Whether his warning is specific enough to mean anything today, or only fits in hindsight, is also argued.
← PreviousSamuel Butler Predicted the AI Problem in 1863Next →How Kubrick Built HAL 9000 — and Why HAL Actually Refuses
