Memory of water: Martin Chaplin's response

As any regular readers will no doubt be aware, there was a recent issue of the journal Homeopathy devoted to the concept of memory of water. Ben Goldacre put together a journal club on the issue. I ended up submitting a letter to the editor in response to Martin F Chaplin's paper 'The memory of water: an overview'. Essentially, I made the point that Chaplin's paper was interesting, but didn't really describe anything of relevance to homeopathy; and that, in any case, there was no observed effect of homeopathy that needed explaining. The letter has now been published, alongside Chaplin's response.

Chaplin writes that my argument 'fails on two levels; firstly such an argument [that there is evidence of water memory of relevance to homeopathic treatment] was not the primary purpose of my review and secondly, in fairness to its inclusion in the journal ‘Homeopathy’, I did present such an argument (e.g. Table 1, p. 149).' Let's leave aside for now that this seems to be saying both that he did not make such an argument and that he did. The first point is partly fair enough: it may be true that Chaplin's primary purpose was not to provide a justification for homeopathy. However, the paper was published in Homeopathy, which suggests some relevance to homeopathy. As to the second point, Chaplin's Table 1 doesn't really provide any argument, but simply lists some mechanisms by which water might have something that Chaplin terms a 'memory'. How these could actually apply in practice to homeopathy is not discussed, and none of them show how a homeopathic remedy would be any different from plain old water. That was the point of my letter.

In his final paragraph, Chaplin states that "It is clear from the final paragraph of the letter that its author has his own preconceptions concerning homeopathy and that he was disappointed that I was not able to reinforce these." This refers to my contention that meta-analyses of placebo-controlled trials show that the best-conducted trials show no effect for homeopathy above placebo, referencing as an example the Shang et al. meta-analysis that was published in the Lancet. That being the case, there is nothing in homeopathy that requires any explanation, whether from 'memory of water' or anything else. It is perhaps true that I have some pre-conceptions with respect to homeopathy: it is, after all, extremely implausible given what we know about chemistry, physics and pharmacology. However, my reference to meta-analyses of placebo-controlled trials is clearly a reference to evidence, and as such has nothing to do with 'pre-conceptions'. Chaplin's comment is therefore no more than an ad hominem.

Chaplin goes on to criticise the Shang et al. paper, referencing his website. The Shang paper is one that has been widely misunderstood homeopaths, as I've written previously. Chaplin makes some of the same errors. In his website he writes that "The conclusion [of Shang et al.] was reached, however, in spite of the study apparently showing little evidence of differences between the two groups (homeopathy and conventional) when all the data was considered. There were differences when a tiny percentage of unmatched larger trials were cherry-picked for further analysis (that is, 102/110 of the homeopathy studies and 104/110 of the conventional studies were discarded). The remaining 6% of the studies, however, still showed positive (if not conclusive, possibly as the number of trials left in this final grouping was so small) evidence in favour of a homeopathic effect over placebo." This is just a misunderstanding of the study. Shang et al. did indeed show an effect for homeopathy above placebo when considering the 110 homeopathy trials. But the trials were not 'cherry-picked' for further analysis. They were selected based on stated criteria, and based on the fact the analysis had shown that the best predictor of bias in the studies was study size. Shang et al. showed that when studies of inadequate methodology and studies with small sample sizes were stripped out, there was no statistically significant effect for homeopathy (contrary to Chaplin's assertion that there was positive evidence for an effect greater than placebo). I stand by my interpretation of the study, that it shows that 'the best-conducted trials show no effect for homeopathic remedies beyond placebo'.

On his website, Chaplin also seems to misunderstand the placebo effect, when he writes "It should also be noted that placebo effects constitute real clinical effects, should be judged positively and probably account for a significant proportion of the success of prevailing established medicine". The point here is that the placebo effect is what you get when you give patients a sugar pill containing no active ingredients. If a remedy produces an effect no greater than placebo, then why should it be judged more positively than a blank sugar pill? That's the whole point of conducting placebo-controlled trials!

Having read Chaplin's response, I see no reason not to stand by my original points. Firstly, that while Chaplin has shown that water is interesting stuff, he hasn't shown any water memory effect that could provide a mechanism for homeopathy to work. Secondly, the best evidence shows that homeopathy doesn't work and needs no explanation.