Artificial Intelligence Is Coming, and It Could Wipe Us Out if We're Not Careful, Professor Warns

The Berenson robot strolls among visitors during the exhibition "Persona: Oddly Human" at the Quai Branly museum in Paris on February 23. REUTERS/Philippe Wojazer

The RSA information security conference this week in San Francisco has been a labyrinth filled with professionals who believe they know best when it comes to information and the Internet. But what if in 25 years humans are no longer good enough to be the gatekeepers of the Internet?

Nick Bostrom, a professor of philosophy at the University of Oxford and founding director of the Future of Humanity Institute, spoke in a keynote address at the Moscone Center conference about the "Midas effect" of wanting artificial intelligence (AI) as smart as humans and ignoring potential downsides.

"You would no longer have some cool technology toy [in AI]," Bostrom said. "You would have the last invention that humans will need to make."

During his 30-minute talk, Bostrom outlined three points about AI: Artificial intelligence on a level with human intelligence is a very good likelihood by 2050; the rise of superintelligent machines beyond the control of humans may be a possibility; and superintelligence could either prove a path to "cosmic endowment"—the potential to colonize the universe using technology—or to the extinction of the human race.

While his talk of cosmic endowment may sound a bit New Age, Bostrom has been one of the most prominent scholars on the philosophy and ethics behind AI, frequently speaking in TED talks and featured in articles, including ones in The New Yorker and The Atlantic. His skeptical view of the robotic future has the support of such Silicon Valley executives as Tesla CEO Elon Musk.

Bostrom's talk also surveyed the current innovations in the artificial intelligence community, ranging from cybersecurity (hence the invitation to RSA) to lethal automatic weapons like predator drones. Bostrom expressed both excitement and concern over the recent zeal for AI technology, most notably with the recent news of Google's deep-learning AI system's "dreaming" of psychedelic-looking images. Using the fable of King Midas as an example, Bostrom worried about engineers' "literal fulfillment of a wish but with qualifications."

According to a survey he conducted among industry experts, most predicted a 50 percent likelihood of human-level AI being available between 2040 and 2050. With decades to mitigate the "extinction risk," a phrase he coined in 2002, Bostrom hoped for more pre-emptive caution. "[AI's] still a fragile thing," Bostrom said. "The view that we will have machine intelligence in our lifetimes is not some ridiculous idea but a very mainstream idea."