Very bad science...

I already wrote an amateur rebuttal of the Homeopathy special issue on water memory here, but I was following the discussion about the Rao et al. paper at Ben Goldacre's site, where the articles are being journal clubbed. A contributor to Bad Science, RichH, noticed that some of the curves in figure 2, representing 'envelopes of difference' for different dilutions of two different homeopathic remedies, looked similar to the 'representative' examples of curves shown in figure 1, purporting to show that UV-vis spectroscopy can identify different homeopathic remedies.

I did some Corel Drawing, and resized and stretched the graphs in figure 2 so that the scales between figures 1 and 2 were the same. It seems that RichH was correct, and the curves in figure 1 are also shown in figure 2. This suggests that curves the authors compare in figure 1 are the most extreme spectra obtained for the different remedies, rather than representative ones. Most surprisingly, though, the part of figure 2a that shows the 'envelope of difference' for 30C Nat Mur is identical to the graph in figure 1 that supposedly shows the difference between 30C Nat Mur and 30C Nux Vom. In other words, the same graph is presented twice, and purports to show different things each time is presented. Apart from all the errors discussed here, it seems they got their graphs mixed up too. This seems like an extraordinary error to make in a formal paper, even in a pseudojournal.

I reproduce the part of figure 2a (at left), resized and stretched so scales match, next to the part of figure 1 (at right) below. It's easier to see that they're the same if you put them on top of each other, in different layers, in Corel Draw and turn one layer on and off. But hopefully you can see that the curves are the same when presented side by side. The only difference is that the symbols (open vs. closed circles) are swapped between the two graphs.











Of course, you shouldn't have to go through this rigmarole to compare graphs in the first place...

Edit: Philip Ball has picked this up on his blog. If you go there, you can also find links to his original Nature piece and the various comments on it at the Nature weblog. Ben Goldacre has also picked it up on his mini-blog. If I'm not careful, all this fame will go to my head...