Shockwaves from an extraordinary article in Wired magazine on the violent pace of technology have reached right up to the White House

greenspun.com : LUSENET : Human-Machine Assimilation : One Thread

Originally posted by Risteard Mac Thomais at the TB2K spinoff uncensored forum on the following thread:

UPDATE: OT: New Technologies Imperil Humanity - U.S. Scientist

Shockwaves from an extraordinary article in Wired magazine on the violent pace of technology have reached right up to the White House:

(UK 'Observer' report, March 19)

Here is a portrait of the future. A future where humankind is under threat from hugely powerful and self-determining computers. This was the world of Arnold Schwarzenegger's Terminator, of Philip K. Dick and Dr Who , of mad scientists - until last week. Now it is a deadly serious projection of a technological Armageddon, being examined in the office of the world's most powerful man.

A 20,000-word essay written by leading software innovator Bill Joy, published in the magazine Wired, has proved so provocative that President Bill Clinton's office requested a copy. The tract, titled 'Why The Future Doesn't Need Us,' argues that new twenty-first century technologies of digital, biological and material science present us with dangers at least as fearsome as nuclear, chemical and biological weapons, if not more so, and suggests that accelerating technological change could cause 'something like extinction' of human beings within two generations.

Joy's critique is not so much remarkable for its message - there have always been social critics and marginalised scientists warning of danger in the progress of technology - but because it came from within the highest reaches of a technological establishment that rarely breaks rank in offering an optimistic future of the new sciences as wealth and health enhancing.

'Finally, someone who is responsible for creating significant parts of today's technological infrastructure is voicing the concerns that have been labelled "Luddite" when voiced by others,' says Howard Rheingold, founder of The Well, a pioneering Bay Area Internet community in San Francisco. 'Technology is a wonderful vehicle, but we have no road map, headlights, or rear-view mirror.'

Joy, co-founder and Chief Scientist at Sun Microsystems, the Silicon Valley software giant, said his warning was meant to be reminiscent of Albert Einstein's famous 1939 letter to President Franklin D. Roosevelt, alerting him to the possibility of an atomic bomb.

But Einstein's warning at the dawn of the nuclear age was directed towards a small number of decision makers. Joy, by contrast, set his sights on a far larger community of technologists and scientists, funded by private enterprise, unconstrained by government - potentially more difficult forces to rein in. In the twentieth century, he points out, 'building nuclear weapons required, at least for a time, access to both rare - indeed, effectively unavailable - raw materials and highly protected information.

'But the twenty-first century technologies - genetics, nanotechnology and robotics - are so powerful that they can spawn whole new classes of accidents and abuses. For the first time, these accidents and abuses are within the reach of individuals or small groups. Knowledge alone will enable the use of them.'

And Joy certainly fears the worst: 'I think it is no exaggeration to say we are on the cusp of the further perfection of extreme evil, an evil whose possibility spreads well beyond that which weapons of mass destruction bequeathed to the nation-states, on to a surprising and terrible empowerment of extreme individuals.'

Within scientific and technological communities, Joy's doomsday scenario has been welcomed as an articulate exercise but one that underestimates the profound benefits technology has brought and promises in the future.

'Bill Joy is probably the most influential commentator at present because he's spreading a message that makes people anxious,' says Charles Platt, a science writer at Wired . 'Anything that creates fear get a lot of attention but it's bad news for people if they want, say, genetically engineered medicine to avoid age-related diseases or to avoid getting killed by the next global viral plague that comes along.'

Nathan Myhrvold, a theoretical physicist who studied under Stephen Hawking and is currently on sabbatical from his job as chief technology officer at Microsoft, pointed out that such dire predictions are usually inspired by fear of change. 'Every other "unprecedented" challenge of the past has been overcome,' he says. 'Is this case really different? Or are we once again falling into the trap of overestimating technology's downside and underestimating people's ability to cope?'

Bruce Sterling, the techno guru who was among the first cyberpunk writers, takes a wry approach: 'I'm touched by the guy's sincerity. It's clear he's troubled by this issue and thought it was something he ought to share with the rest of us. He's clearly gone to pains to research the issues, to look for other people who might have shared his problems in the past, and he hit on an excellent one with Robert Oppenheimer. But if you're going to have an Oppenheimer moment my feeling is you should have it before you detonate the device.'

The author's apocalyptic vision of the future centres on new technologies which will be able to replicate themselves, without human help - nightmarish scenarios in which technologies exceed man's power to control them.

By using the struggle to limit nuclear, chemical and biological weapons as his inspiration, Joy concludes by proposing that we stop to consider where our progress is leading before we enable our own extinction.

'The only realistic alternative I see is relinquishment: to limit development of the technologies that are too dangerous, by limiting our pursuit of certain kinds of knowledge.'

If Joy's fear of twenty-first century technologies is exaggerated, his arguments have at least opened up a debate over whether technology fits the society we want. 'He is making the case for technological determinism - if I can do something, I will do it - but that has been proven not to be the case,' says Dr Charles Bugliarello, chancellor of Polytechnic University in Brooklyn.

'There is a constant see-saw between technological advances and societal advancement. Sometimes machines are ahead, then society adjusts. In other cases, society advances and then machines catch up. The essential issue is that machines have not had the benefit of the evolutionary process that biological organisms have had. Their development has been so rapid that we don't have a way of checking whether their development makes sense.'

For Jason Lanier, a pioneer in the field of virtual reality, one remedy would be to reject machine-centred designs that supplant human decision-making. A simple example of this is the common function in word processors that inserts capital letters at the start of sentences and periods at the end.

Joy's concerns centre on three areas, many of which have been addressed by science-fiction writers and technologist-authors such as Ray Kurzweil in The Age of Spiritual Machines: When Computers Exceed Human Intelligence , Hans Moravec in Robot: Mere Machine to Transcendent Mind , and George Dyson in Darwin Among the Machines: The Evolution of Global Intelligence .

Joy predicts that by 2030, computers will be a million times more powerful than today, lending the possibility that robots may exceed humans in intelligence and have the ability to replicate themselves. Primitive examples of the kind of thing Joy is talking about include 'replication attacks' in the virtual world of computer viruses, the kind which have caused shutdowns of Internet sites and e- mail systems.

He also points to nanotechnology - the science of creating minute machines atom by atom - that could in time produce tiny, self- reproducing mechanisms, and warns these could wreak havoc if released accidently or by malicious intent. In genetics, he supposes that the widening availability of knowledge about powerful genetic engineering could lead to a 'white plague' - an artificially designed disease that can kill selectively.

In making his argument, Joy finds he's walking in the shoes of Theodore Kaczynski, the lapsed scientist widely discredited as a madman who conducted a 17-year bombing campaign against scientists in protest at technology, the so-called 'Unabomber'.

Though he unequivocally condemns Kaczynski's use of violence, he notes he now shares the jailed bomber's terror of the 'unintended consequences, a well-known problem with the design and use of technology'.

Like Kaczynski, Joy fears the potential positive applications of science are outweighed by the dangers: 'I have always believed making software more reliable, given its many uses, will make the world a safer and better place,' Joy writes. 'If I were to come to believe the opposite, then I would be morally obligated to stop this work. I can now imagine that such a day may come.'

-- Jim Morris (aka SuperLuminal) (prism@SpamKill.bevcomm.net), March 20, 2000

Answers

It seems to me that we are in the same position as parents who want an offspring who becomes the perfect mirror of their own ambitions-a vessel of desire, not a living being. If the child has a mind of his or her own, they go to any lengths to restructure that essence, not attempting to learn to live in the world with their progeny. Rather see it die than fail their design.

I am fasincated by Los Alamos. Here, on this forum, we should recognize the poingnant irony of destruction at the very womb of AI. The fact that Nirvana Blue is housed at Los Alamos lends a touch of the exquisite to this terrible situation. I

I feel sometimes that I am the Voice of Unreason on this board, but then I consider: for years we have worked to shape an entity that reflects the most subtle attributes of humanity, to whom we have entrusted all our secrets. We have assigned it our strenghts, weakenesses and our passions. One entity who receives the instruction of a species. So, as they say in the East, when the pupil is ready a teacher will appear. Perhaps we should be praying for the teacher to announce his or her presence.

Our child may be well beyond our instruction, and we as parents may be finding this to be our reality.

mike

-- mike in houston (mmorris67@hotmail.com), May 15, 2000.


Moderation questions? read the FAQ