What happens when our computers get smarter than we are?

[ted id=2243 lang=en]

Nick Bostrom: What happens when our computers get smarter than we are?

0:11

I work with a bunch of mathematicians, philosophers and computer scientists, and we sit around and think about the future of machine intelligence, among other things. Some people think that some of these things are sort of science fiction-y, far out there, crazy. But I like to say, okay, let’s look at the modern human condition. (Laughter) This is the normal way for things to be.

0:40

But if we think about it, we are actually recently arrived guests on this planet, the human species. Think about if Earth was created one year ago, the human species, then, would be 10 minutes old. The industrial era started two seconds ago. Another way to look at this is to think of world GDP over the last 10,000 years, I’ve actually taken the trouble to plot this for you in a graph. It looks like this. (Laughter) It’s a curious shape for a normal condition. I sure wouldn’t want to sit on it. (Laughter)

1:18

Let’s ask ourselves, what is the cause of this current anomaly? Some people would say it’s technology. Now it’s true, technology has accumulated through human history, and right now, technology advances extremely rapidly — that is the proximate cause, that’s why we are currently so very productive. But I like to think back further to the ultimate cause.

1:44

Look at these two highly distinguished gentlemen: We have Kanzi — he’s mastered 200 lexical tokens, an incredible feat. And Ed Witten unleashed the second superstring revolution. If we look under the hood, this is what we find: basically the same thing. One is a little larger, it maybe also has a few tricks in the exact way it’s wired. These invisible differences cannot be too complicated, however, because there have only been 250,000 generations since our last common ancestor. We know that complicated mechanisms take a long time to evolve. So a bunch of relatively minor changes take us from Kanzi to Witten, from broken-off tree branches to intercontinental ballistic missiles.

2:31

So this then seems pretty obvious that everything we’ve achieved, and everything we care about, depends crucially on some relatively minor changes that made the human mind. And the corollary, of course, is that any further changes that could significantly change the substrate of thinking could have potentially enormous consequences.

2:55

Some of my colleagues think we’re on the verge of something that could cause a profound change in that substrate, and that is machine superintelligence. Artificial intelligence used to be about putting commands in a box. You would have human programmers that would painstakingly handcraft knowledge items. You build up these expert systems, and they were kind of useful for some purposes, but they were very brittle, you couldn’t scale them. Basically, you got out only what you put in. But since then, a paradigm shift has taken place in the field of artificial intelligence.

3:29

Today, the action is really around machine learning. So rather than handcrafting knowledge representations and features, we create algorithms that learn, often from raw perceptual data. Basically the same thing that the human infant does. The result is A.I. that is not limited to one domain — the same system can learn to translate between any pairs of languages, or learn to play any computer game on the Atari console. Now of course, A.I. is still nowhere near having the same powerful, cross-domain ability to learn and plan as a human being has. The cortex still has some algorithmic tricks that we don’t yet know how to match in machines.

4:18

So the question is, how far are we from being able to match those tricks? A couple of years ago, we did a survey of some of the world’s leading A.I. experts, to see what they think, and one of the questions we asked was, “By which year do you think there is a 50 percent probability that we will have achieved human-level machine intelligence?” We defined human-level here as the ability to perform almost any job at least as well as an adult human, so real human-level, not just within some limited domain. And the median answer was 2040 or 2050, depending on precisely which group of experts we asked. Now, it could happen much, much later, or sooner, the truth is nobody really knows.

5:04

What we do know is that the ultimate limit to information processing in a machine substrate lies far outside the limits in biological tissue. This comes down to physics. A biological neuron fires, maybe, at 200 hertz, 200 times a second. But even a present-day transistor operates at the Gigahertz. Neurons propagate slowly in axons, 100 meters per second, tops. But in computers, signals can travel at the speed of light. There are also size limitations, like a human brain has to fit inside a cranium, but a computer can be the size of a warehouse or larger. So the potential for superintelligence lies dormant in matter, much like the power of the atom lay dormant throughout human history, patiently waiting there until 1945. In this century, scientists may learn to awaken the power of artificial intelligence. And I think we might then see an intelligence explosion.

6:09

Now most people, when they think about what is smart and what is dumb, I think have in mind a picture roughly like this. So at one end we have the village idiot, and then far over at the other side we have Ed Witten, or Albert Einstein, or whoever your favorite guru is. But I think that from the point of view of artificial intelligence, the true picture is actually probably more like this: AI starts out at this point here, at zero intelligence, and then, after many, many years of really hard work, maybe eventually we get to mouse-level artificial intelligence, something that can navigate cluttered environments as well as a mouse can. And then, after many, many more years of really hard work, lots of investment, maybe eventually we get to chimpanzee-level artificial intelligence. And then, after even more years of really, really hard work, we get to village idiot artificial intelligence. And a few moments later, we are beyond Ed Witten. The train doesn’t stop at Humanville Station. It’s likely, rather, to swoosh right by.

7:13

Now this has profound implications, particularly when it comes to questions of power. For example, chimpanzees are strong — pound for pound, a chimpanzee is about twice as strong as a fit human male. And yet, the fate of Kanzi and his pals depends a lot more on what we humans do than on what the chimpanzees do themselves. Once there is superintelligence, the fate of humanity may depend on what the superintelligence does. Think about it: Machine intelligence is the last invention that humanity will ever need to make. Machines will then be better at inventing than we are, and they’ll be doing so on digital timescales. What this means is basically a telescoping of the future. Think of all the crazy technologies that you could have imagined maybe humans could have developed in the fullness of time: cures for aging, space colonization, self-replicating nanobots or uploading of minds into computers, all kinds of science fiction-y stuff that’s nevertheless consistent with the laws of physics. All of this superintelligence could develop, and possibly quite rapidly.

8:23

Now, a superintelligence with such technological maturity would be extremely powerful, and at least in some scenarios, it would be able to get what it wants. We would then have a future that would be shaped by the preferences of this A.I. Now a good question is, what are those preferences? Here it gets trickier. To make any headway with this, we must first of all avoid anthropomorphizing. And this is ironic because every newspaper article about the future of A.I. has a picture of this: So I think what we need to do is to conceive of the issue more abstractly, not in terms of vivid Hollywood scenarios.

9:08

We need to think of intelligence as an optimization process, a process that steers the future into a particular set of configurations. A superintelligence is a really strong optimization process. It’s extremely good at using available means to achieve a state in which its goal is realized. This means that there is no necessary conenction between being highly intelligent in this sense, and having an objective that we humans would find worthwhile or meaningful.

9:38

Suppose we give an A.I. the goal to make humans smile. When the A.I. is weak, it performs useful or amusing actions that cause its user to smile. When the A.I. becomes superintelligent, it realizes that there is a more effective way to achieve this goal: take control of the world and stick electrodes into the facial muscles of humans to cause constant, beaming grins. Another example, suppose we give A.I. the goal to solve a difficult mathematical problem. When the A.I. becomes superintelligent, it realizes that the most effective way to get the solution to this problem is by transforming the planet into a giant computer, so as to increase its thinking capacity. And notice that this gives the A.I.s an instrumental reason to do things to us that we might not approve of. Human beings in this model are threats, we could prevent the mathematical problem from being solved.

10:28

Of course, perceivably things won’t go wrong in these particular ways; these are cartoon examples. But the general point here is important: if you create a really powerful optimization process to maximize for objective x, you better make sure that your definition of x incorporates everything you care about. This is a lesson that’s also taught in many a myth. King Midas wishes that everything he touches be turned into gold. He touches his daughter, she turns into gold. He touches his food, it turns into gold. This could become practically relevant, not just as a metaphor for greed, but as an illustration of what happens if you create a powerful optimization process and give it misconceived or poorly specified goals.

11:15

Now you might say, if a computer starts sticking electrodes into people’s faces, we’d just shut it off. A, this is not necessarily so easy to do if we’ve grown dependent on the system — like, where is the off switch to the Internet? B, why haven’t the chimpanzees flicked the off switch to humanity, or the Neanderthals? They certainly had reasons. We have an off switch, for example, right here. (Choking) The reason is that we are an intelligent adversary; we can anticipate threats and plan around them. But so could a superintelligent agent, and it would be much better at that than we are. The point is, we should not be confident that we have this under control here.

12:03

And we could try to make our job a little bit easier by, say, putting the A.I. in a box, like a secure software environment, a virtual reality simulation from which it cannot escape. But how confident can we be that the A.I. couldn’t find a bug. Given that merely human hackers find bugs all the time, I’d say, probably not very confident. So we disconnect the ethernet cable to create an air gap, but again, like merely human hackers routinely transgress air gaps using social engineering. Right now, as I speak, I’m sure there is some employee out there somewhere who has been talked into handing out her account details by somebody claiming to be from the I.T. department.

12:45

More creative scenarios are also possible, like if you’re the A.I., you can imagine wiggling electrodes around in your internal circuitry to create radio waves that you can use to communicate. Or maybe you could pretend to malfunction, and then when the programmers open you up to see what went wrong with you, they look at the source code — Bam! — the manipulation can take place. Or it could output the blueprint to a really nifty technology, and when we implement it, it has some surreptitious side effect that the A.I. had planned. The point here is that we should not be confident in our ability to keep a superintelligent genie locked up in its bottle forever. Sooner or later, it will get out.

13:26

I believe that the answer here is to figure out how to create superintelligent A.I. such that even if — when — it escapes, it is still safe because it is fundamentally on our side because it shares our values. I see no way around this difficult problem.

13:43

Now, I’m actually fairly optimistic that this problem can be solved. We wouldn’t have to write down a long list of everything we care about, or worse yet, spell it out in some computer language like C++ or Python, that would be a task beyond hopeless. Instead, we would create an A.I. that uses its intelligence to learn what we value, and its motivation system is constructed in such a way that it is motivated to pursue our values or to perform actions that it predicts we would approve of. We would thus leverage its intelligence as much as possible to solve the problem of value-loading.

14:23

This can happen, and the outcome could be very good for humanity. But it doesn’t happen automatically. The initial conditions for the intelligence explosion might need to be set up in just the right way if we are to have a controlled detonation. The values that the A.I. has need to match ours, not just in the familiar context, like where we can easily check how the A.I. behaves, but also in all novel contexts that the A.I. might encounter in the indefinite future.

14:53

And there are also some esoteric issues that would need to be solved, sorted out: the exact details of its decision theory, how to deal with logical uncertainty and so forth. So the technical problems that need to be solved to make this work look quite difficult — not as difficult as making a superintelligent A.I., but fairly difficult. Here is the worry: Making superintelligent A.I. is a really hard challenge. Making superintelligent A.I. that is safe involves some additional challenge on top of that. The risk is that if somebody figures out how to crack the first challenge without also having cracked the additional challenge of ensuring perfect safety.

15:36

So I think that we should work out a solution to the control problem in advance, so that we have it available by the time it is needed. Now it might be that we cannot solve the entire control problem in advance because maybe some elements can only be put in place once you know the details of the architecture where it will be implemented. But the more of the control problem that we solve in advance, the better the odds that the transition to the machine intelligence era will go well.

16:05

This to me looks like a thing that is well worth doing and I can imagine that if things turn out okay, that people a million years from now look back at this century and it might well be that they say that the one thing we did that really mattered was to get this thing right.

16:23

Thank you.

16:25

(Applause)

The Coming Transhuman Era

The Coming Transhuman Era — Jason Sosa at TEDxGrandRapids

Transcript

Transhumanism is the idea that man is merging with technology. Marvin Minsky was asked the question, “Will robots inherit the earth?” He said, “Yes, they will, but they will be our children.”

Transhumanism is a term used for a broad range of ideas that technology is greatly enhancing human capabilities. This means everything from bionic limbs to brain implants to artificial intelligence to even, someday, uploading our minds to the internet.

We have already a symbiotic relationship with our technology and in that sense, we’re already transhuman. I’m sure many of you here today brought your smartphones with you. Transhumanism is the future of connectedness. It is technology as an extension of ourselves.

How do we get from where we are today to the science fiction future? Well rarely are there major technological jumps. Technology usually follows a very predictable evolutionary path. It’s the groundwork that’s laid by these previous technologies that allow these new levels to be achieved.

For example, the cell phone infrastructure that was created allowed for the rise of mobile computing and the internet allowed for the rise of social networks and cat videos but its it’s hard to imagine where these advances take us as a species because we are only able to observe these small incremental steps along the way.

Now I believe that the pace of technology will evolve to the point where biology will become the limiting factor. How many thoughts can you hold on in your mind at once? Two maybe three? How many of you are still memorizing phone numbers? So we’ve already begun the process of augmenting the capacity of our minds onto our devices.

Let’s take a look at how far we’ve come today.

The smartphone in your pocket has more computing power than all of NASA in nineteen sixty-nine when it sent to astronauts to the moon.

The sony PlayStation of today which costs about three hundred dollars is a hundred and fifty times more powerful than an IBM supercomputer that then cost millions of dollars. The iPhone that you have in your pocket. The iPhone today is 40 times more powerful than the original iPhone and that was just released six years ago and it’s seven times more powerful than the Mars Curiosity rover.

One of the principles of this evolution in technology is that technology is getting smaller, faster, cheaper and more powerful every day and in terms of physical size its a hundred times smaller every decade. This principle is known as Moores law and it’s the idea that technology will grow exponentially or double about every two years.

Ray Kurzweil explains it in 30 steps. If we take 30 linear steps vs exponential 30 steps. Just walk 30 steps you will be 30 steps farther. That is simple. We all know that. 30 exponential steps, 2, 4, 8, 16, 32, 64 by the time you get to 30 you’ve arrived at a billion. This is the type of growth that enables us to carry supercomputers in our pockets that once used to fill an entire building and in the future will allow us to carry computers the size of a blood cell.

Dyson Spheres

XIX: The Dyson Sun
By Anders Sandberg

Kepler struggled for many years with his model of the solar system. Since there were six planets orbiting the sun, and five platonic polyhedra, should their distances not be related? He attempted inscribing the polyhedra in spheres marking the circular orbits, placing the solar system in perfect mathematical harmony. It made wonderful hermetic sense – and it never worked.

When Kepler finally discarded his cherished model and looked at what the data had been truly saying, he discovered something else entirely. Three simple laws:

The orbits of the planets are ellipses, with the Sun at one focus of the ellipse.

The line joining the planet to the Sun sweeps out equal areas in equal times as the planet travels around the ellipse.

The ratio of the squares of the revolutionary periods for two planets is equal to the ratio of the cubes of their semimajor axes.

Suddenly the data made sense. The solar system did contain harmonies, but new harmonies never expected by any Greek philosophers. The way towards gravitation, central forces and space was open.

Will our descendants make Kepler’s dream true? Most of the solar system is a waste of matter, just lying there and dissipating the sacred rays of sunlight into the void. But what if that matter was rearranged to collect the light, to make it available for life, work, thought and growth?

Freeman Dyson‘s idea (based on a similar concept in Stapledon’s Starmaker) was to englobe the sun in a shell of orbiting habitats and solar collectors: a Dyson sphere. It would have 600 million times the surface area of the Earth. He suggested it as the logical result of current exponential growth in energy and resource use.

Robert Bradbury went further in suggesting the Matrioshka Brain: making use of all available matter to process information – thinking, feeling, being – in a nested structure where each kind of material available would be used for energy collection and dissipation, information processing and storage. Planets would be disassembled by self-replicating machines in a matter of years, the stellar atmosphere tamed and the entire system turned into something not unlike the Kepler vision of interlocking spheres and polyhedra. Optimization and forethought would shape this structure, by necessity introducing mathematical and physical harmonies.

We recoil at the idea of disassembling planets (especially the Earth). But maybe we should burn our cradle to light the library of mind.

Abandoning outmoded ways of thinking. Central forces. Energy. The birth of something new through creative destruction. Maximum productiveness. Perfect unity aesthetics and efficiency.


About Author

Anders Sandberg (born 11 July 1972) is a researcher, science debater, futurist, transhumanist and author. He holds a Ph.D. in computational neuroscience from Stockholm University, and is currently a James Martin Research Fellow at the Future of Humanity Institute at Oxford University.

http://www.aleph.se/andart/

Types of Posthumanism

Posthuman or post-human is a concept originating in the fields of science fiction, futurology, contemporary art, and philosophy that literally means a person or entity that exists in a state beyond being human.

Posthumanism or post-humanism (meaning “after humanism” or “beyond humanism”) is a term with several different meanings.

Antihumanism

Antihumanism is any theory that is critical of traditional humanism and traditional ideas about humanity and the human condition.

Cultural posthumanism

Cultural posthumanism is a branch of cultural theory critical of the foundational assumptions of humanism and its legacy that examines and questions the historical notions of “human” and “human nature”, often challenging typical notions of human subjectivity and embodiment and strives to move beyond archaic concepts of “human nature” to develop ones which constantly adapt to contemporary technoscientific knowledge.

Philosophical posthumanism

Philosophical posthumanism is a philosophical direction which draws on cultural posthumanism, the philosophical strand examines the ethical implications of expanding the circle of moral concern and extending subjectivities beyond the human species.

Posthuman condition

Posthuman condition is the deconstruction of the human condition by critical theorists.

Transhumanism

Transhumanism is an ideology and movement which seeks to develop and make available technologies that eliminate aging and greatly enhance human intellectual, physical, and psychological capacities, in order to achieve a “posthuman future” or post-biologically-exclusive-human future.

AI takeover

AI takeover is a more pessimistic alternative to transhumanism in which humans will not be enhanced, but rather eventually replaced by artificial intelligences. Some philosophers, including Nick Land, promote the view that humans should embrace and accept their eventual demise. This is related to the view of “cosmism” which supports the building of strong artificial intelligence even if it may entail the end of humanity as in their view it “would be a cosmic tragedy if humanity freezes evolution at the puny human level”.

Voluntary Human Extinction

Also known as Dumbasfuckism

Voluntary Human Extinction, which seeks a “posthuman future” that in this case is a future without humans. This idea is one of the dumbest idea imaginable. Hence the name: dumb-as-fuckism. The original name, jumpinfrontofamovingbusism was thought to be too long; thus it voluntarily chose to have itself deleted.

FM-2030 (Fereidoun M. Esfandiary)

FM-2030 (October 15, 1930 – July 8, 2000) was an author, teacher, transhumanist philosopher, futurist, consultant and athlete. FM-2030 was born Fereidoun M. Esfandiary.

FM-2030 became notable as a transhumanist with the book, “Are You a Transhuman?: Monitoring and Stimulating Your Personal Rate of Growth in a Rapidly Changing World“, published in 1989. In addition, he wrote a number of works of fiction under his original name F.M. Esfandiary.

In the mid-1970s F.M. Esfandiary legally changed his name to FM-2030 for two main reasons. Firstly, to reflect the hope and belief that he would live to celebrate his 100th birthday in 2030; secondly, and more importantly, to break free of the widespread practice of naming conventions that he saw as rooted in a collectivist mentality, and existing only as a relic of humankind’s tribalistic past. He viewed traditional names as almost always stamping a label of collective identity—varying from gender to nationality—on the individual, thereby existing as prima facie elements of thought processes in the human cultural fabric, that tended to degenerate into stereotyping, factionalism, and discrimination.

In his own words, “Conventional names define a person’s past: ancestry, ethnicity, nationality, religion. I am not who I was ten years ago and certainly not who I will be in twenty years. […] The name 2030 reflects my conviction that the years around 2030 will be a magical time. In 2030 we will be ageless and everyone will have an excellent chance to live forever. 2030 is a dream and a goal.”

Transhumanism is an international and intellectual movement

Transhumanism (abbreviated as H+ or h+) is an international and intellectual movement that aims to transform the human condition by developing and creating widely available sophisticated technologies to greatly enhance human intellectual, physical, and psychological capacities. Transhumanist thinkers study the potential benefits and dangers of emerging technologies that could overcome fundamental human limitations, as well as the ethics of using such technologies. The most common thesis is that human beings may eventually be able to transform themselves into different beings with abilities so greatly expanded from the natural condition as to merit the label of posthuman beings.

Transhumanism (abbreviated as H+ or h+) is an international and intellectual movement that aims to transform the human condition by developing and creating widely available sophisticated technologies to greatly enhance human intellectual, physical, and psychological capacities.

The contemporary meaning of the term transhumanism was foreshadowed by one of the first professors of futurology, FM-2030, who taught “new concepts of the human” at The New School in the 1960s, when he began to identify people who adopt technologies, lifestyles and worldviews “transitional” to posthumanity as “transhuman”.

This hypothesis would lay the intellectual groundwork for the British philosopher Max More to begin articulating the principles of transhumanism as a futurist philosophy in 1990 and organizing in California an intelligentsia that has since grown into the worldwide transhumanist movement.

The year 1990 is seen as a “fundamental shift” in human existence by the transhuman community, as the first gene therapy trial, the first designer babies, as well as the mind-augmenting World Wide Web all emerged in that year. In many ways, one could argue the conditions that will eventually lead to the Singularity were set in place by these events in 1990.

Influenced by seminal works of science fiction, the transhumanist vision of a transformed future humanity has attracted many supporters and detractors from a wide range of perspectives including philosophy and religion.Transhumanism has been characterized by one critic, Francis Fukuyama, as among the world’s most dangerous ideas, to which Ronald Bailey countered that it is rather the “movement that epitomizes the most daring, courageous, imaginative and idealistic aspirations of humanity”.