Free I Think You'll Find It's a Bit More Complicated Than That by Ben Goldacre

in, was killing people: not in large enough numbers to be immediately obvious, but when the trial was finally done, an extra two people died out of every hundred given steroids.
    There are similar cases from the world of education. The ‘Scared Straight’ programme also made sense on paper: young children were taken into prisons and shown the consequences of a life of crime, in the hope that they would be more law-abiding in their own lives. Following the children who participated in this programme into adult life, it seemed they were less likely to commit crimes, when compared with other children. But here, researchers were caught out by the same problem discussed above: the schools – and so the children – who went on the Scared Straight course were different from the children who didn’t. When a randomised trial was finally done, where this error could be accounted for, we found out that the Scared Straight programme – rolled out at great expense, with great enthusiasm, good intentions and huge optimism – was actively harmful, making children more likely to go to prison in later life.
    So we must always be cautious about assuming that things which are new, or expensive, are necessarily always better. But this is just one special case of a broader issue: we should always be clear when we are uncertain about which intervention is best. Right now, there are huge numbers of different interventions used throughout the country – different strategies to reduce absenteeism, or teach arithmetic, or reduce teenage pregnancies, or any number of other things – where there is no evidence to say which of the currently used methods is best. There is arbitrary variation, across the country, across a town, in what strategies and methods are used, and nobody worries that there is an ethical problem with this.
    Randomisation, in a trial, adds one simple extra chink to this existing variation: we need a group of schools, teachers, pupils or parents who are able to honestly say: ‘We don’t know which of these two strategies is best, so we don’t mind which we use. We want to find out which is best, and we know it won’t harm us.’
    This is a good example of how gathering good evidence requires a culture shift, extending beyond a few individual randomised trials. It requires everyone involved in education to recognise when it’s time to honestly say ‘We don’t know what’s best here.’ This isn’t a counsel of despair: in medicine, and in teaching, we know that most of what we do does some good (if we’re not better than nothing, then we’re all in big trouble!). The real challenge is in identifying what works the best, because when people are deprived of the best, they are harmed too. But this is also a reminder of how inappropriate certainty can be a barrier to progress, especially when there are charismatic people who claim they know what’s best, even without good evidence.
    Medicine suffered hugely with this problem, and as late as the 1970s there were notorious confrontations between people who thought it was important to run fair tests, and ‘experts’, who were angry at the thought of their expertise being challenged and their favourite practices being tested. Archie Cochrane was one of the pioneers of evidence-based medicine, and in his autobiography he describes many battles he had with senior doctors, in glorious detail. In 1971, Cochrane was concerned that coronary care units in hospitals might be no better than home care, which was the standard care for a heart attack at the time (we should remember that this was the early days of managing heart attacks, and the results from this study wouldn’t be applicable today). In fact, he was worried that hospital care might involve a lot of risky procedures that could even, conceivably, make outcomes worse for patients overall.
    Because of this, Cochrane tried to set up a randomised trial comparing home care against hospital care, despite great resistance