Free Fortune's Formula by William Poundstone

ones, figures in today’s codes for compressing digital pictures, audio, and video.
    The success of these compression schemes implies that messages are like sponges. They are mostly “air” with little “substance.” As long as you preserve the substance, you can squeeze out the air.
    The question that all of Shannon’s predecessors tried to tackle was: What is the “substance” of a message, the essential part that can’t be dispensed with? To most the answer was meaning . You can squeeze anything out of a message except meaning. Without meaning, there is no communication.
    Shannon’s most radical insight was that meaning is irrelevant. To paraphrase Laplace, meaning was a hypothesis Shannon had no need of. Shannon’s concept of information is instead tied to chance . This is not just because noise randomly scrambles messages. Information exists only when the sender is saying something that the recipient doesn’t already know and can’t predict. Because true information is unpredictable, it is essentially a series of random events like spins of a roulette wheel or rolls of dice.
    If meaning is excluded from Shannon’s theory, what is the incompressible substance that exists in every message? Shannon concluded that this substance can be described in statistical terms. It has only to do with how unpredictable the stream of symbols composing the message is.
    A while back, a phone company ran ads showing humorous misunderstandings resulting from mobile phone noise. A rancher calls to order “two hundred oxen.” Because of the poor voice quality, he gets two hundred dachshunds—which are no good at pulling plows at all. A wife calls her husband at work and asks him to bring home shampoo. Instead he brings home Shamu, the killer whale.
    The humor of these spots derived from a gut-level understanding of Shannon’s ideas that we all share whether we know it or not. Try to analyze what happened in the Shamu commercial: (1) The wife said something like, “Pick up shampoo!” (2) The husband heard “Pick up Shamu!” (3) The husband wound up the conversation, said goodbye, and on the way home picked up the killer whale.
    It is only the third action that is ridiculous. It is ridiculous because “Pick up Shamu” is an extremely low-probability message. In real conversations, we are always trying to outguess each other. We have a continuously updated sense of where the conversation is going, of what is likely to be said next, and what would be a complete non sequitur. The closer two people are (personally and culturally), the easier this game of anticipation is. A long-married couple can finish each other’s sentences. Teen best friends can be in hysterics over a three-character text message.
    It would be unwise to rely on verbal shorthand when speaking to a complete stranger or someone who doesn’t share your cultural reference points. Nor would the laconic approach work, even with a spouse, when communicating a message that can’t be anticipated.
    Assuming you wanted your spouse to bring home Shamu, you wouldn’t just say, “Pick up Shamu!” You would need a good explanation. The more improbable the message, the less “compressible” it is, and the more bandwidth it requires. This is Shannon’s point: the essence of a message is its improbability.
     
     
    Shannon was not the first to define information approximately the way he did. His two most important predecessors were both Bell Labs scientists working in the 1920s: Harry Nyquist and Ralph Hartley. Shannon read Hartley’s paper in college and credited it as “an important influence on my life.”
    As he developed these ideas, Shannon needed a name for the incompressible stuff of messages. Nyquist had used intelligence , and Hartley had used information . In his earliest writings, Shannon favored Nyquist’s term. The military connotation of “intelligence” was fitting for the cryptographic work. “Intelligence” also implies meaning, however,