Free Bad Science by Ben Goldacre Page A

cheap course of a steroid might help improve outcomes, and seven trials testing this idea were done between 1972 and 1981. Two of them showed some benefit from the steroids, but the remaining five failed to detect any benefit, and because of this, the idea didn’t catch on.

     
    Eight years later, in 1989, a meta-analysis was done by pooling all this trial data. If you look at the blobbogram in the logo on the previous page, you can see what happened. Each horizontal line represents a single study: if the line is over to the left, it means the steroids were better than placebo, and if it is over to the right, it means the steroids were worse. If the horizontal line for a trial touches the big vertical nil effect line going down the middle, then the trial showed no clear difference either way. One last thing: the longer a horizontal line is, the less certain the outcome of the study was.
    Looking at the blobbogram, we can see that there are lots of not-very-certain studies, long horizontal lines, mostly touching the central vertical line of no effect; but they’re all a bit over to the left, so they all seem to suggest that steroids might be beneficial, even if each study itself is not statistically significant.
    The diamond at the bottom shows the pooled answer: that there is in fact very strong evidence indeed for steroids reducing the risk—by 30 to 50 percent—of babies dying from the complications of immaturity. We should always remember the human cost of these abstract numbers: babies died unnecessarily because they were deprived of this lifesaving treatment for a decade. They died even when there was enough information available to know what would save them , because that information had not been synthesized together, and analyzed systematically, in a meta-analysis.
    Back to homeopathy (you can see why I find it trivial now). A landmark meta-analysis was published recently in The Lancet . It was accompanied by an editorial titled “The End of Homeopathy?” Shang et al. did a very thorough meta-analysis of a vast number of homeopathy trials, and they found, overall, adding them all up, that homeopathy performs no better than placebo.
    The homeopaths were up in arms. If you mention this meta-analysis, they will try to tell you that it was a fix. What Shang et al. did, essentially, like all the previous negative meta-analyses of homeopathy, was to exclude the poorer-quality trials from their analysis.
    Homeopaths like to pick out the trials that give them the answer that they want to hear and ignore the rest, a practice called cherry-picking. But you can also cherry-pick your favorite meta-analyses or misrepresent them. Shang et al. were only the latest in a long string of meta-analyses to show that homeopathy performs no better than placebo. What is truly amazing to me is that despite the negative results of these meta-analyses, homeopaths have continued—right to the top of the profession—to claim that these same meta-analyses support the use of homeopathy. They do this by quoting only the result for all trials included in each meta-analysis. This figure includes all of the poorer-quality trials. The most reliable figure, you now know, is for the restricted pool of the most “fair tests,” and when you look at those, homeopathy performs no better than placebo. If this fascinates you (and I would be very surprised), then I am currently producing a summary with some colleagues, and you will soon be able to find it online at badscience.net, in all its glorious detail, explaining the results of the various meta-analyses performed on homeopathy.
    Clinicians, pundits, and researchers all like to say things like “There is a need for more research,” because it sounds forward-thinking and open-minded. In fact, that’s not always the case, and it’s a little-known fact that this very phrase has been effectively banned from the British Medical Journal for many years, on the ground that it adds nothing; you may say