Free Gödel, Escher, Bach: An Eternal Golden Braid by Douglas R. Hofstadter

you if he came back after a while, complaining that he can't get rid of the initial M , and therefore it is a wild goose chase. Even if a person is not very bright, he still cannot help making some observations about what he is doing, and these observations give him good insight into the task-insight which the computer program, as we have described it, lacks.
    Now let me be very explicit about what I meant by saying this shows a difference between people and machines. I meant that it is possible to program a machine to do a routine task in such a way that the machine will never notice even the most obvious facts about what it is doing; but it is inherent in human consciousness to notice some facts about the things one is doing. But you knew this all along. If you punch "1" into an adding machine, and then add 1 to it, and then add 1 again, and again, and again, and continue doing so for hours and hours, the machine will never learn to anticipate you, and do it itself, although any person would pick up the

    pick up the idea, no matter how much or how well it is driven, that it i supposed to avoid other cars and obstacles on the road; and it will never learn even the most frequently traveled routes of its owner.
    The difference, then, is that it is possible for a machine to act unobservant; it is impossible for a human to act unobservant. Notice I am not saying that all machines are necessarily incapable of making sophisticated observations; just that some machines are.
    Nor am I saying that all people are always making sophisticated observations; people, in fact, are often very unobservant. But machines can be made to be totally unobservant; any people cannot. And in fact, most machines made so far are pretty close ti being totally unobservant. Probably for this reason, the property of being; unobservant seems to be the characteristic feature of machines, to most people. For example, if somebody says that some task is "mechanical", i does not mean that people are incapable of doing the task; it implies though, that only a machine could do it over and over without eve complaining, or feeling bored.

    Jumping out of the System

    It is an inherent property of intelligence that it can jump out of the tas which it is performing, and survey what it has done; it is always looking for and often finding, patterns. Now I said that an intelligence can jump out o its task, but that does not mean that it always will. However, a little prompting will often suffice. For example, a human being who is reading a boo may grow sleepy. Instead of continuing to read until the book is finished he is just as likely to put the book aside and turn off the light. He ha stepped
    "out of the system" and yet it seems the most natural thing in the world to us. Or, suppose person A is watching television when person B comes in the room, and shows evident displeasure with the situation Person A may think he understands the problem, and try to remedy it b exiting the present system (that television program), and flipping the channel knob, looking for a better show. Person B may have a more radio concept of what it is to
    "exit the system"-namely to turn the television oft Of course, there are cases where only a rare individual will have the vision to perceive a system which governs many peoples lives, a system which ha never before even been recognized as a system; then such people often devote their lives to convincing other people that the system really is there and that it ought to be exited from!
    How well have computers been taught to jump out of the system? I w cite one example which surprised some observers. In a computer chess: tournament not long ago in Canada, one program-the weakest of all the competing ones-had the unusual feature of quitting long before the game was over. It was not a very good chess player, but it at least had the redeeming quality of being able to spot a hopeless position, and to resign then and there, instead of waiting for the