Free Super Crunchers by Ian Ayres Page A

generating actual sales. The lines of each alternative page stretch across the graph like a horse race with clear winners and clear losers. Imagine it—instantaneous information on 128 different treatments with tens of thousands of observations. This is randomization on steroids. Offermatica shows the way that Super Crunching often exploits technology to shorten the time between data collection, analysis, and implementation. With Offermatica, the time between the test and the marketing change can be just a matter of hours.
    By the way, if you think you have a good graphic eye, try to see which of these two boxes you think tested better:

    SOURCE : Monster.com Scores Millions, http://www.offermatica.com/stories-1.7.htm.
    I personally find the curved icons of the lower box to be more appealing. That’s what Monster thought too. The lower box is the box that Monster actually started with before the test. Yet it turns out that employers spent 8.31 percent more per visit when they were shown the top box. This translates into tens of millions of dollars per year for Monster’s ecommerce channel. Instead of trusting its initial instinct, Monster went out, perturbed the status quo, and watched what happened. It created a new type of data and put its initial instinct to the test.
    Jo-Ann Fabrics got an even bigger surprise. Part of the power of testing multiple combinations is that it lets companies be bolder, to take bigger risks with test marketing. You might not think that JoAnn.com would draw enough web traffic to make Internet testing feasible, but they pull over a million unique visitors a month. They have enough traffic to do all kinds of testing.
    So when JoAnn.com was optimizing their website, they decided to take a gamble and include in their testing an unlikely promotion for sewing machines: “Buy two machines and save 10 percent.” They didn’t expect this test to pan out. After all, how many people need to buy two sewing machines? Much to their amazement, the promotion generated by far the highest returns. “People were pulling their friends together,” says Linsly Donnelly, JoAnn.com’s chief operating officer. The discount was turning their customers into sales agents. Overall, randomized testing increased its revenue per visitor by a whopping 209 percent.
    In the brick-and-mortar world, the cost of running randomized experiments with large enough samples to give you a statistically significant result sometimes severely limits the number of experiments that can be done. But the Internet changes all this. “As the cost of showing a group of people a given experience gets close to zero,” Matt Roche, Offermatica’s CEO, says, “the number of experiences you can give is close to infinity.”
    Seeing how consumers respond to a whole bunch of different online experiences is what Offermatica is all about. It’s a wildly different model for deciding who decides. Matt comes alive when he talks about seeing the battle for corporate control of your eyeballs: “I go to meetings where you have all these people sitting around a table claiming authority. You have the analytic guy who has the amulet of historical data. You’ve got the branding guy who has this mystical certainty about what makes brands stronger. And of course you got the authority of title itself, the boss who’s used to thinking that he knows best. But what’s missing is the consumer’s voice. All these forms of authority are substituting for the customer’s voice. What we’re really doing at Offermatica is listening to what consumers want.”
    Offermatica not only has to do battle with the in-house analytic guy who crunches numbers on historical data, it also has to take on “usability experts” who run hyper-controlled experiments in university laboratories. The usability experts are sure of certain axioms that have been established in the lab—things like