Answer came there none...

More fall-out from the Homeopathy memory of water issue.

One of the papers in the Homeopathy special issue was by Rao et al., involving work by Rustum Roy, and had been touted around various places by Dana Ullman as evidence that homeopathy could work. The paper purported to use spectroscopic techniques to differentiate between homeopathic remedies, showing that there was some difference between them. It was fairly comprehensively taken apart by people at the JREF forums (including a minor contribution from me), and eventually a letter was submitted to the editor of Homeopathy outlining our concerns. This letter has now been published, alongside a response by Rao.

Hopefully, the other authors will forgive me if I summarise the points in the letter as follows:

1. The presented spectrum of pure ethanol looks nothing like ethanol of any recognised degree of purity.
2. There is no way of knowing whether the 'differences' between different preparations could have been caused by them being prepared with solvent containing different levels of impurities.
3. No statistical information was presented, meaning that no conclusions could be drawn as to whether the remedies were actually different.
4. Spectra that were supposedly representative of the tested preparations were actually the more extreme examples.
5. One graph was reproduced twice, purporting to show different things each time.

You'd imagine that all this was fairly damning, so how did the authors respond? On all of these points, answer came there none. Regarding the possibility of different levels of contamination between different samples, Rao simply says that this "illustrates one of the values of our work to homeopathy producers and users, and other health researchers. We have provided them with potential quality control tools, and a refining of the arguments away from incorrect generalizations". On the critical issue of contamination potentially invalidating the results of the paper, astoundingly, there is no comment.

On statistical analysis, Rao writes "All the analytical data shown in the paper are the result of reproducible analyses, although we appreciate the suggestion of representing such data as an average with a standard deviation, we emphasize that our key identification by display of an envelope demonstrates that, there are indeed differences beyond the standard deviation range among individual homeopathic remedies, as used in practice". What exactly the 'envelope' actually is has never been explained. It is certainly not a standard technique for statistical analysis. Without knowing what the standard deviation range actually is, it is impossible to demonstrate differences beyond it. While it could perhaps be argued that there are spectroscopic differences between 'homeopathic remedies, as used in practice', it can't be argued on this evidence that these differences are specifically related to the process of producing a homeopathic remedy, when different levels of different impurities can explain the results just as well.

Rao also states that s/he and her co-authors are "neither champions nor detractors of homeopathy". I'm not sure how this squares with co-author Rustum Roy showing up in the Grauniad to defend homeopathy.

On the other points, there is not a word. The authors have entirely failed to defend their work. I'm amazed. I thought there would be a more than cursory attempt to invalidate our criticisms, or at least clarify some of the ambiguities in the original paper.

I think that Rao et al. is dead, and in a sane world it would be withdrawn by the journal.