Machine Analytique de Charles Babbage, exposée au Science Museum de Londres (Mai 2009). Bruno Barral (ByB). WIKIMEDIA.
I am neither a mathematician nor an engineer, so my references in this article to the technical aspects of Wiener’s contributions in those disciplines will be minimal, and, I hope, safely general.
What I am is a journalist who for more than two decades has studied the history and philosophy of technology, and who recently completed a book based on those studies . In the final section of that book I pay homage to Norbert Wiener’s outspoken ethical stands in regard to the dangers of the technologies he was so instrumental in bringing into being. It is Wiener’s ethics and the ideas behind them that I’d like to talk about here.
Wiener was far ahead of his time in recognizing that our habits of technology-fueled consumption pose significant threats to health and the environment. He predicted that within the foreseeable future we would be facing growing coal and gas shortages, growing scarcity of water with which to supply our cities, growing rates of infection due to increased air travel and antibiotic resistance, growing problems related to processed and synthetic foods, growing risks of nuclear power accidents, and growing risks of nuclear war.
Wiener was far ahead of his time in recognizing that our habits of technology-fueled consumption pose significant threats to health and the environment.
One of the technologies that worried Wiener most was the one he was most directly responsible for creating, automation. Today we are hearing a striking amount of public discussion and debate about the impact automation is likely to have on employment. That intelligent machines might be used by owners and managers to eliminate or demean jobs on a massive scale was a prospect that Wiener spoke of often. “Those who suffer from a power complex,” he wrote, “find the mechanization of man a simple way to realize their ambitions” .
Wiener did more than worry about these things. He spoke out about them, in best-selling books, in magazine articles, and in speeches. He also declined to participate in military and corporate projects that would make those threats more likely to materialize, a stance that cost him dearly in terms of his career, his pocketbook, and his reputation.
In his personal life, Wiener was in many respects a difficult, unpleasant man, but his willingness to stand up for what he believed in, and his willingness to pay the price for those beliefs, make him, in my view, a genuinely heroic figure.
Perhaps the perception of Wiener’s that most impresses me, and that I think is most crucial for us to understand, remember, and act upon today, is his appreciation of uncertainty, both in human understanding and as human understanding is manifested in the technologies humans create. The evidence is overwhelming that uncertainty is a universal, inescapable quality of the human condition, and that our endeavors to affect change in the world need, therefore, to be accompanied by a substantial measure of caution and humility. That so many scientists and technicians seem not to appreciate this simple truth would be astonishing were it not for the fact that their pursuits are to a large degree driven by a quest for certainty. Wiener was rare among his colleagues in recognizing that contradiction, which is one reason his colleagues found him annoying.
For several years, the physicist and historian Steve Heims asked mathematicians and scientists what they thought of Wiener’s social concerns and his preoccupation with the uses of technology. “The typical answer,” Heims said, “went something like this: ‘Wiener was a great mathematician, but he was also an eccentric. When he began talking about society and responsibility of scientists, a topic outside of his area of expertise, well, I just couldn’t take him seriously.’” 
In this article I will review Wiener’s positions on uncertainty and then attempt to place him in historical perspective as a significant contributor to what the philosopher of technology Langdon Winner calls “the counter-tradition to the dream of mastery.” Winner’s description recognizes the point I just made: that the desire to achieve mastery – over nature, over events, over other people, over uncertainty itself – is to a large degree the motivation that drives scientific and technological projects. The counter-tradition to the dream of mastery recognizes that the dream of mastery is just that: a dream, one that we long to achieve but never will.
The Uncertainty of Norbert Wiener
In my opinion, evidence that Wiener really was a genius can be found in “The Theory of Ignorance,” a paper he wrote at the age of ten. According to Dark Hero of the Information Age, the superb biography by Flo Conway and Jim Siegelman, Wiener spoke in the paper about “the impossibility of being certain of anything” and disputed “man’s presumption in declaring that his knowledge has no limits.” All human knowledge, he added, “is based on an approximation” . (Wiener refers briefly to this paper in the second volume of his autobiography. See , [p. 324].)
That intelligent machines might be used by owners and managers to eliminate or demean jobs on a massive scale was a prospect that Wiener spoke of often.
Uncertainty would become a central focus of Wiener’s scientific career, and of the ethical stands that accompanied that career. His breakthrough paper on Brownian motion was inspired by gazing at the waters of the Charles River from his office at M.I.T., wondering how he could find some sort of quantifiable pattern within the seemingly random ripples and waves on the river’s surface. This was a problem that would lead to the dynamic at the heart of cybernetics, and that can also be seen philosophically as addressing one of the central riddles of existence. As Wiener himself later put it, “the highest destiny of mathematics” is nothing less than the discovery of order within disorder , [p. 33].
Wiener’s solution for that problem emerged from the techniques of statistical mechanics developed by Willard Gibbs, an obscure theorist who Wiener considered “America’s greatest scientist.” Gibbs’ applications of probability theory demonstrated, Wiener believed, that chance is part of the “warp and weft” of physics, and thus affirmed his recognition of a sort of “irrationality in the world” , [pp. 68–69], , [p. 34].
Here is how Wiener described Gibbs’ ideas in the second volume of his autobiography, I Am a Mathematician:
“The great physical tradition of Newton had necessarily been one of determinism, where a perfect knowledge of the universe at one instant is understood to involve a perfect knowledge of its history throughout all time. It would have been Newton’s assumption that to give the present positions and speeds of the particles in a wave moving across the surface of the Charles would allow us to plot the moment of all this wave forever. Unfortunately, no perfect knowledge of the present is available to us with our limited measuring instruments, and the problem that faces the working physicist is to find out how far he can go with the imperfect knowledge available to him.
For this he must work not with a single, fixed universe but with many different universes simultaneously, each having some preassigned probability. He cannot tell you what will always happen but what may happen at certain times, given certain conditions” . [p. 34].
Even before this, Wiener had the chutzpah to challenge the certainty of Bertrand Russell, with whom Wiener briefly studied after earning his degree in mathematical philosophy at Harvard. Russell would point Wiener in directions that would shape his later career, including advising him to study a paper of Einstein’s on Brownian motion. Nonetheless, Wiener was repulsed by Russell’s entire approach to mathematics. Conway and Siegelman quote a letter Wiener wrote to his father, expressing “a great dislike” for the great man. “His mind impresses one as a keen, cold, narrow logical machine,” Wiener wrote, “that cuts the universe into neat little packets, that measure, as it were, just three inches each way” [4, [p. 30].
Wiener soon went public with his reservations regarding Russell’s too-neat mathematics, publishing a paper attacking the central argument in Russell’s and Alfred North Whitehead’s magisterial Principia Mathematica. Contrary to what Russell and Whitehead claim, Wiener wrote, “it is highly probable that we can get no certainty that is absolute in the propositions of logic and mathematics, at any rate in those that derived their validity from the postulates of logic” , [p. 142], , [p. 33].
Twenty years later Wiener would note with pleasure that Gödel’s incompleteness theorems affirmed that judgment. As he put it in I Am a Mathematician,
“To me, logic and learning and all mental activity have always been incomprehensible as a complete and closed picture and have been understandable only as a process by which man puts himself in rapport with his environment…We are swimming upstream against a great torrent of disorganization, which tends to reduce everything to the heat-death of equilibrium and sameness described in the second law of thermodynamics” , [p. 324].
That comment amounts to a summary statement of the essential argument of cybernetics, which posits information as the antidote to entropy. That comment also suggests Wiener’s determination to pursue a mathematics that is not lost in abstraction, but that has practical application in the real world, a determination that in turn reflects his natural inclination to holism. This was another quality that distinguished Wiener from his contemporaries in science, one that made him highly critical of the growing tendency toward specialization.
Just as I find it astonishing that the ten-year-old Wiener would have the insight to appreciate the futility of certainty, so I find it remarkable that an unknown mathematician in his early twenties would openly challenge so formidable a figure as Bertrand Russell. Where did this willingness – eagerness, even – to take on the powers that be come from? It’s hard to avoid the conclusion that the answer lies in Wiener’s troubled childhood.
Wiener once described himself as “an extravagantly bent twig” , [p. 274]. His father’s relentless, brutal pressures to mold him into a child prodigy combined with Wiener’s nearsightedness, his clumsiness, his Jewishness, his brilliance, his eclecticism, his ambition, and his utter lack of social graces all served to make him perennially insecure, and thus perennially alienated from anything that smacked of complacency. One of his closest friends described Wiener as “a foreigner wherever he was” , [p. 377].
Like many wounded people, Wiener both hated his outsider status and reveled in it. He also used it. “…I knew very well that I was competitive beyond the run of younger mathematicians, and I knew equally that this was not a pretty attitude,” he wrote in I Am a Mathematician. “However, it was not an attitude which I was free to assume or to reject. I was quite aware that I was an out among ins and that I would get no shred of recognition that I did not force. If I was not to be welcomed, well then, let me be too dangerous to be ignored” , [p. 87].
This attitude animated Wiener’s ethical convictions as well as his achievements in science. Standing at a remove from conventional wisdom, he was able to recognize its failures and criticize them without risking his position in the club. Steve Heims quotes an essay Wiener wrote criticizing Rudyard Kipling for his “fear-rooted conformity.” Clearly thinking of himself, Wiener added, “The marginal man is always relatively the more civilized human being” , [p. 377].
The Counter-Tradition to the Dream of Mastery
Langdon Winner’s discussion of the counter-tradition to the dream of mastery can be found in his seminal book, Autonomous Technology: Technics-out-of-Control as a Theme in Political Thought. That title suggests the degree to which the ideas Winner so brilliantly explores reflect and affirm Wiener’s belief in uncertainty. Indeed, at one point Winner quotes a long confessional passage from Wiener’s Cybernetics, in which Wiener acknowledges his responsibility for unleashing powerful new techniques into “the world of Belsen and Hiroshima,” knowing that those who would inherit access to those techniques included “the most irresponsible and most venal of our engineers” .
As Wiener well knew, a technology can go awry even when deployed by the most responsible and least venal of engineers, simply because once unleashed, it interacts with its environment in ways that are impossible to fully predict or control. This lack of control is what the often misunderstood term “technological autonomy” refers to. A related term describes the surprising results our technologies so often produce: “unintended consequences.”
The dream of mastery can be interrupted by any number of factors, among them human and technical fallibility and chance. Winner provides a number of quotations that demonstrate how long and how often these conditions have been recognized. One is a poem from Solon of Athens, dating from approximately the sixth century BCE.
“Danger, for all, lies in all action, and there is no telling which way the end will be after a thing is begun” , [p. 93].
It’s likely that one of the classic narratives in the counter-tradition to the dream of mastery, the Tower of Babel story in the Book of Genesis, was composed within a century or so of Solon’s poem.
Winner points out that one of the chief reasons mastery is unattainable is the interconnectedness of the world. Because of that interconnectedness, repercussions expand promiscuously; consequences fall like dominoes. The Roman emperor Marcus Aurelius seemed to anticipate particle physics when he said, “All things are implicated with one another, and the bond is holy; there is hardly anything unconnected with another thing” , [p. 94].
Wiener well knew that technology can go awry even when deployed by the most responsible of engineers, simply because once unleashed, technology interacts with its environment in ways that are impossible to fully predict or control.
Hannah Arendt, in her book The Human Condition, reflected on the implications of interconnection. “Action,” she wrote, “no matter what its specific content, always establishes relationships and therefore has an inherent tendency to force open all limitations and cut across all boundaries.”
- “These consequences are boundless, because action, though it may proceed from nowhere, so to speak, acts into a medium where every reaction becomes a chain reaction and where every process is the cause of new processes. Since action acts upon beings who are capable of their own actions, reaction, apart from being a response, is always a new action that strikes out on its own and affects others. Thus action and reaction among men never move in a closed circle and can never be reliably confined to two partners” , [p. 95]. (I have expanded on Winner’s quotation from Arendt. See .
With his understanding that information and feedback are the lingua franca of both organisms and machines, Wiener recognized that the chain reactions to which Arendt referred apply not only to the relationships between human beings, but also to the relationships between human beings and machines, and to the relationships between machines and machines. For that reason he also recognized that, thanks to the growing power, ubiquity, and interconnectedness of our machines, the dream of mastery has become one of the most dangerous illusions of our time.
In his final book, God and Golem, Inc., published posthumously in 1964, Wiener cited The Sorcerer’s Apprentice as a literary object lesson in how quickly technology can escape our control. For much of history, he wrote, the ramifications of misplaced intentions were relatively limited, simply because the scope of our reach was relatively limited. Human impotence, he wrote, “hitherto shielded us from the full destructive impact of human folly.” Modern technologies were in the process of changing that. “The penalties for errors of foresight, great as they are now,” Wiener wrote, “will be enormously increased as automatization comes into full use” .
Speed, Wiener frequently argued, is the most basic reason this is true. In a 1960 essay for Science magazine, “Some Moral and Technical Consequences of Automation,” he wrote that although we may in theory be able to control our machines, such control means little if we can’t impose it until it’s too late. “By the very slowness of our human actions, our effective control of our machines may be nullified,” Wiener said. “By the time we are able to react to information conveyed by our senses and stop the car we are driving, it may already have run head on into a wall.”
I thought of that quote as I read the accounts of Wall Street’s “Flash Crash” in May of 2010, when averages on American stock exchanges mysteriously plunged nearly 1000 points in a matter of minutes, a chain reaction later blamed on computerized trading programs. Meanwhile millions of gallons of oil were still gushing into the Gulf of Mexico after the explosion two weeks earlier of the Deepwater Horizon offshore oil rig, and no one was yet sure they knew how to stop it. A year after that a tsunami flooded the Fukushima nuclear reactors in Japan, precipitating a series of events that made me think of a comment from Wiener’s previous book, The Human Use of Human Beings: “It is not necessary to bring in the consideration of war,” he said, “for us to see how much more naked we lie to disaster than at any time before” , [p. 41].
It’s not only speed that causes us to lose control of our technologies as they mature. Wiener recognized that while information is the ballast that enables machines and organisms to maintain homeostasis and resist entropy, gluts of information and the corruption of information can have the opposite effect. In Cybernetics he identified the control of media by corporate executives, politicians, and “hucksters” as the most “anti-homeostatic” factor in modern society (quoted in , [p. 180].
A Shadow Wantonly Evoked
The counter-tradition to the dream of mastery suggests that the technological project as a whole may have become anti-homeostatic. As out of control as our technologies might be, the survival of civilization as we know it literally depends on them. We can’t bring ourselves to surrender the technologies that are destroying the planet because the disruption required to do so would destroy us. Langdon Winner calls these intractable commitments “the technological imperative.” In my book I call them evidence of “de facto technological autonomy.”
The most obvious autonomous technologies are the artificial intelligence systems Wiener pioneered, which are daily becoming ever-more-capable of running themselves. Wiener was keenly aware that by crossing the boundary between machine and organism, his work increasingly came to resemble the building of the golem. Like Frankenstein, the golem is a creature capable of acting for reasons of its own, and it doesn’t always follow directions.
“It may be seen,” Wiener wrote in the Science article, “that the result of a programming technique of automatization is to remove from the mind of the designer and operator an effective understanding of many of the stages by which the machine comes to its conclusions and of what the real tactical intentions of many of its operations may be…To avoid a disastrous consequence, it is not enough that some action on our part should be sufficient to change the course of the machine, because it is quite possible that we lack information on which to base consideration of such an action.”
This brings to mind a comment by the Scottish essayist Thomas Carlyle, another eccentric who represented with distinction the counter-tradition to the dream of mastery:
“The shadow we have wantonly evoked stands terrible before us, and will not depart at our bidding” .
Doug Hill is an independent scholar who lives in Philadelphia, PA, U.S.A. He blogs about technology at http://thequestionconcerningtechnology.blogspot.com and can be followed on Twitter at @DougHill25. His book, Not So Fast: Thinking Twice About Technology, is currently being reviewed for publication by a university press in the United States. He can be reached by email at email@example.com.