Free Our Final Invention: Artificial Intelligence and the End of the Human Era Hardcover by James Barrat Page A

who conversed in text. Each session lasted a maximum of two hours. Keeping silent and boring the Gatekeeper into surrendering was a permitted but never used tactic.
    Between 2002 and 2005, Yudkowsky played against five Gatekeepers. He escaped three times, and stayed in the box twice. How did he escape? I had learned online that one of the rules of the AI Box experiment was that the transcripts of the contests cannot be revealed, so I didn’t know the answer. Why the secrecy?
    Put yourself in Yudkowsky’s shoes. If you, playing the AI in the box, had an ingenious means of escape, why reveal it and tip off the next Gatekeeper, should you ever choose to play again? And second, to try and simulate the persuasive power of a creature a thousand times more intelligent than the smartest human, you might want to go a little over the edge of what’s socially acceptable dialogue. Or you might want to go way over the edge. And who wants to share that with the world?
    The AI-Box Experiment is important because among the likely outcomes of a superintelligence operating without human interference is human annihilation, and that seems to be a showdown we humans cannot win. The fact that Yudkowsky won three times while playing the AI made me all the more concerned and intrigued. He may be a genius, but he’s not a thousand times more intelligent than the smartest human, as an ASI could be. Bad or indifferent ASI needs to get out of the box just once.
    The AI-Box Experiment also fascinated me because it’s a riff on the venerable Turing test. Devised in 1950 by mathematician, computer scientist, and World War II code breaker Alan Turing, the eponymous test was designed to determine whether a machine can exhibit intelligence. In it, a judge asks both a human and a computer a set of written questions. If the judge cannot tell which respondent is the computer and which is the human, the computer “wins.”
    But there’s a twist. Turing knew that thinking is a slippery subject, and so is intelligence. Neither is easily defined, though we know each when we see it. In Turing’s test, the AI doesn’t have to think like a human to pass the test, because how could anyone know how it was thinking anyway? However, it does have to convincingly pretend to think like a human, and output humanlike answers. Turing himself called it “the imitation game.” He rejected the criticism that the machine might not be thinking like a human at all. He wrote, “May not machines carry out something which ought to be described as thinking but which is very different from what a man does?”
    In other words, he objects to the assertion John Searle made with his Chinese Room Experiment: if it doesn’t think like a human it’s not intelligent. Most of the experts I’ve spoken with concur. If the AI does intelligent things, who cares what its program looks like?
    Well, there may be at least two good reasons to care. The transparency of the AI’s “thought” process before it evolves beyond our understanding is crucial to our survival. If we’re going to try and imbue an AI with friendliness or any moral quality or safeguard, we need to know how it works at a high-resolution level before it is able to modify itself. Once that starts, our input may be irrelevant. Second, if the AI’s cognitive architecture is derived from human brains, or from a human brain upload, it may not be as alien as purely new AI. But, there’s a vigorous debate among computer scientists whether that connection to mankind will solve problems or create them.
    No computer has yet passed the Turing test, though each year the controversial Loebner Prize, sponsored by philanthropist Hugh Loebner, is offered to the maker of one that does. But while the $100,000 grand prize goes unclaimed, an annual contest awards $7,000 to the creator of the “most humanlike computer.” For the last few years they’ve been chatbots—robots created to simulate conversation, with little success. Marvin