the apparently positive outcomes are due to self-selection on prognostic variables other than those we tested, such as available social support or willingness to self-disclose.” 28
As with other studies, the McKellar researchers also took no steps to verify the self-reported data about drinking frequency, either through urine tests or check-ins with friends or relatives. Ultimately, the authors acknowledged that the study failed to answer some very basic questions:
Future studies comparing AA with other interventions might help answer important questions such as (a) Does AA provide specialized benefits in lowering long-term alcohol problems when compared with other self-help groups or outpatient after care programs? or conversely; (b) Does AA affiliation (attending meeting, working the steps, etc.) provide the same benefits that any good therapeutic treatment would provide (i.e., hope, treatment rationale, therapeutic alliance, mitigation of isolation; Bergin & Garfield, 1994)?
Remarkably, despite all this, the McKellar study authors concluded that “the findings are consistent with the hypothesis that AA participation has a positive effect on alcohol-related outcomes.”
THE MOTIVATION QUESTION
A recurring theme in AA research is the question of what
kind
of people do well in AA. Is there something about this small group of people, some special stuff, that makes them different? And if so, can we possibly ascertain what that stuff is? One paper, published by L. A. Kaskutas and colleagues of the School of Public Health, University of California, Berkeley, in 2009, took a look at the existing data and found, as the Cochrane Collaboration did, a lack of solid grounding for the claim that AA is a cure for alcoholism or addiction:
Rigorous experimental evidence establishing the specificity of an effect for AA or Twelve Step Facilitation/TSF (criteria 5) is mixed, with 2 trials finding a positive effect for AA, 1 trial finding a negative effect for AA, and 1 trial finding a null effect. Studies addressing specificity using statistical approaches have had two contradictory findings, and two that reported significant effects for AA after adjusting for potential confounders such as motivation to change. 29
This mix of results squares with what we have seen thus far in this chapter. The strong evidence that one would expect if AA were clearly effective is simply not present. At best, the proponents of the 12-step model can claim only what AA claims; namely, that the program “works if you work it.” Which is another way of saying that people who do well, do well. What does this mean about whether AA itself “works”?
In 2005, R. D. Weiss and colleagues at Harvard Medical School conducted a study that looked more closely at what drives people to a higher level of success in 12-step programs. In the study, which randomized 487 cocaine-dependent outpatients to various twenty-four-week behavioral treatments, the authors uncovered a strong indication that attendance alone did not seem to help people with addictions, but that “
active
12-step participation” was predictive: “Twelve-step group attendance did not predict subsequent drug use. However, active 12-step participation in a given month predicted less cocaine use in the next month.” 30
In 2011, J. Majer of Harry S. Truman College in Chicago and colleagues completed a longitudinal study of people in sober houses and reached much the same conclusion:
Participants who were “categorically involved” in all 12-step [recommended] activities [having a sponsor, reading 12-step literature, doing service work, and calling other members for help] reported significantly higher levels of abstinence and self-efficacy for abstinence at 1 year compared with those who were less involved, whereas averaged summary scores of involvement were not a significant predictor of abstinence. 31
Here it was again: evidence that more
engagement
with the program was correlated with greater