Sunday, 13 January 2008

Memory of water: Martin Chaplin's response

As any regular readers will no doubt be aware, there was a recent issue of the journal Homeopathy devoted to the concept of memory of water. Ben Goldacre put together a journal club on the issue. I ended up submitting a letter to the editor in response to Martin F Chaplin's paper 'The memory of water: an overview'. Essentially, I made the point that Chaplin's paper was interesting, but didn't really describe anything of relevance to homeopathy; and that, in any case, there was no observed effect of homeopathy that needed explaining. The letter has now been published, alongside Chaplin's response.

Chaplin writes that my argument 'fails on two levels; firstly such an argument [that there is evidence of water memory of relevance to homeopathic treatment] was not the primary purpose of my review and secondly, in fairness to its inclusion in the journal ‘Homeopathy’, I did present such an argument (e.g. Table 1, p. 149).' Let's leave aside for now that this seems to be saying both that he did not make such an argument and that he did. The first point is partly fair enough: it may be true that Chaplin's primary purpose was not to provide a justification for homeopathy. However, the paper was published in Homeopathy, which suggests some relevance to homeopathy. As to the second point, Chaplin's Table 1 doesn't really provide any argument, but simply lists some mechanisms by which water might have something that Chaplin terms a 'memory'. How these could actually apply in practice to homeopathy is not discussed, and none of them show how a homeopathic remedy would be any different from plain old water. That was the point of my letter.

In his final paragraph, Chaplin states that "It is clear from the final paragraph of the letter that its author has his own preconceptions concerning homeopathy and that he was disappointed that I was not able to reinforce these." This refers to my contention that meta-analyses of placebo-controlled trials show that the best-conducted trials show no effect for homeopathy above placebo, referencing as an example the Shang et al. meta-analysis that was published in the Lancet. That being the case, there is nothing in homeopathy that requires any explanation, whether from 'memory of water' or anything else. It is perhaps true that I have some pre-conceptions with respect to homeopathy: it is, after all, extremely implausible given what we know about chemistry, physics and pharmacology. However, my reference to meta-analyses of placebo-controlled trials is clearly a reference to evidence, and as such has nothing to do with 'pre-conceptions'. Chaplin's comment is therefore no more than an ad hominem.

Chaplin goes on to criticise the Shang et al. paper, referencing his website. The Shang paper is one that has been widely misunderstood homeopaths, as I've written previously. Chaplin makes some of the same errors. In his website he writes that "The conclusion [of Shang et al.] was reached, however, in spite of the study apparently showing little evidence of differences between the two groups (homeopathy and conventional) when all the data was considered. There were differences when a tiny percentage of unmatched larger trials were cherry-picked for further analysis (that is, 102/110 of the homeopathy studies and 104/110 of the conventional studies were discarded). The remaining 6% of the studies, however, still showed positive (if not conclusive, possibly as the number of trials left in this final grouping was so small) evidence in favour of a homeopathic effect over placebo." This is just a misunderstanding of the study. Shang et al. did indeed show an effect for homeopathy above placebo when considering the 110 homeopathy trials. But the trials were not 'cherry-picked' for further analysis. They were selected based on stated criteria, and based on the fact the analysis had shown that the best predictor of bias in the studies was study size. Shang et al. showed that when studies of inadequate methodology and studies with small sample sizes were stripped out, there was no statistically significant effect for homeopathy (contrary to Chaplin's assertion that there was positive evidence for an effect greater than placebo). I stand by my interpretation of the study, that it shows that 'the best-conducted trials show no effect for homeopathic remedies beyond placebo'.

On his website, Chaplin also seems to misunderstand the placebo effect, when he writes "It should also be noted that placebo effects constitute real clinical effects, should be judged positively and probably account for a significant proportion of the success of prevailing established medicine". The point here is that the placebo effect is what you get when you give patients a sugar pill containing no active ingredients. If a remedy produces an effect no greater than placebo, then why should it be judged more positively than a blank sugar pill? That's the whole point of conducting placebo-controlled trials!

Having read Chaplin's response, I see no reason not to stand by my original points. Firstly, that while Chaplin has shown that water is interesting stuff, he hasn't shown any water memory effect that could provide a mechanism for homeopathy to work. Secondly, the best evidence shows that homeopathy doesn't work and needs no explanation.

6 comments:

Tom said...

To be fair to Chaplin, what he's actually saying in his 'fails on two levels' comment is more along the lines of 'I didn't set out to make this argument, but in fact I did.' Personally my reading of his original paper was that he neither set out to demonstrate a homoeopathy-relevant water memory effect, nor succeeded in doing so. It'd be unfair to assume that any paper published in Homeopathy was necessarily supportive of the, erm, 'discipline', but unnecessary to ascribe this intention to Chaplin's paper: he may or may not have illustrated something that deserves to be called a 'memory of water', but whatever it is it's clearly not relevant to homoeopathy.

ps I'll probably cross-post this to badscience.net if that's ok...

Tom said...

should have re-read my first sentence which is unclear - I meant 'I didn't set out intending to make this argument, but in fact I did end up doing so.'

Paul Wilson said...

Tom:

That's perhaps fair enough. I do wonder, though, whether the statements of various contributors to the 'memory of water' special issue that they are not supporters of homeopathy might be somewhat disingenuous.

In Chaplin's case, I think part of the problem is that 'memory of water' is an ill-defined term. I think Chaplin understands it to mean something very different to what homeopaths think it means. Certainly, homeopaths (e.g. Dana Ullman) use Chaplin's work to support homeopathy as being scientifically plausible. My point is that Chaplin's work cannot sustain that interpretation.

Oh, and feel free to post stuff wherever you like: you wrote it, after all...

Paul Wilson said...

Just as an example, here's a website that claims the work of Rustum Roy and Martin Chaplin as 'proof' about how homeopathy might work.

Ulrich said...

Chaplin has again demonstrated his "objective" appraisal of the literature by adding this paragraph to his website at http://www1.lsbu.ac.uk/water/memory.html#spec

An extraordinary paper authored by Nobel prize-winning Luc Montagnier has shown memory effects in aqueous DNA solutions that depend on interactions with the background electromagnetic field. These effects require the prior processing and dilution of the solutions and are explained as resonance phenomena with nanostructures derived from the DNA and water [1602].

Reference [1602] is this already infamous and thoroughly debunked article, which appeared after a one-day "review process":

L. Montagnier, J. Aïssa, S. Ferris, J.-L. Montagnier, C. Lavallée, Electromagnetic signals are produced by aqueous nanostructures derived from bacterial DNA sequences, Interdiscip. Sci. Comput. Life Sci. 1 (2009) 81-90.

See also http://www.quackometer.net/blog/2009/10/why-i-am-nominating-luc-montagnier-for.html and http://www.sciencebasedmedicine.org/?p=2081

LargelyPolitical said...

Chaplin wants to believe and has been searching for some type of clear evidence. The mysteries to and in water exist on a scale that we can't even begin to contemplate.

I give Martin credit for doing a pretty good job keeping his bias out of it, see his contributions to the David Sereda movie. Frankly, after watching that film Martin lost a little credibility for even being involved.

That said, water has properties that shouldn't exist, that's weird. Its not just one or two properties it is several and they all seem to be very, very beneficial to life. I believe there is some benefit to drinking clean, naturally restructured water but I doubt it will cure advanced cancer or grow new limbs.

You want to listen to a nut, listen to David Sereda in his coast to coast interview with george Noory, I think the guy was just making stuff up! Water and entanglement - c'mon.

Enjoy your posts, thanks.