Something to prove

There's an interesting debate currently taking place in the pages of the journal Homeopathy. It's interesting in that there is actually an attempt to have a debate about one of the fundamentals of homeopathic practice. It's also interesting to see some of the anti-scientific responses to the debate. The original papers discussed can be found through the website of Homeopathy. Unfortunately, you probably won't be able to access them unless you or your institution subscribes to the journal.

In the January issue of Homeopathy, Dantas et al. published a review of homeopathic provings, or Homeopathic Pathogenetic Trials (HPTs), as the authors prefer to call them. The authors defined HPTs as being "clinical trials designed to investigate the effects of the exposure of human volunteers, in good health, to potentially toxic or pathogenetic substances, diluted and serially agitated according to homeopathic pharmacopoeial methods, with a view to providing data to inform their use as homeopathic medicines". The idea is that symptoms caused by the homeopathic preparations can be cured by the same preparations, under the principle of 'like cures like'. There is no good evidence that this principle can be applied as a general rule, but even so it has become one of the foundation stones of homeopathy. One problem is that the symptoms in an HPT are recorded by the volunteers who take part in the proving. No quantitative data is collected about symptoms, and there are well-known problems with such self-reporting studies. Also, in many cases there is no way of telling whether the symptoms occurred as a result of the homeopathic preparation, or for some other reason, because such trials are not always placebo-controlled (Dantas et al. claim that 58% of the trials in their review were placebo-controlled).

The review by Dantas et al. concluded that "Most studies had design flaws, particularly absence of proper randomization, blinding, placebo control and criteria for analysis of outcomes", and went so far as to state that "The central question of whether homeopathic medicines in high dilutions can provoke effects in healthy volunteers has not yet been definitively answered, because of methodological weaknesses of the reports". Their central point is that while provings often turn up all kinds of symptoms, methodological flaws mean that you can't tell whether the symptoms were caused by the homeopathic preparation or not. The authors recommend that improved methodology should be adopted for future HPTs.

This is interesting stuff, and suggests that there are at least some homeopaths who question the value of HPTs, and on perfectly reasonable scientific grounds. It all starts to go a bit wrong in the responses to the article, which were published in the current issue of Homeopathy.

Sherr and Quirk's response is probably the most fun, and I suggest you track it down for yourself (but only if you've got time for such nonsense). Their point of view can be summarised by a paragraph towards the end of the paper, where they state "Eliminating the majority of symptoms or characteristic single symptoms due to over scientific vigour or a concern about statistical significance or background noise, risks throwing out the baby with the bathwater. It is important to remember the proof of provings is first and foremost their clinical usability and efficiency". Over-scientific vigour or a concern over statistical significance, indeed. This is pre-enlightenment thinking if ever I saw it.

They also say that "A good proving is not about producing every possible symptom. It is about producing enough symptoms of quality so that the intelligent homoeopath can perceive a meaningful totality". I take this to mean that you don't have to worry about using the best possible methodology, because the homeopath has some magic way of 'perceiving a meaningful totality'. Also, the object of the proving is to produce 'enough symptoms', not the ones actually caused by the preparation. (Here I gloss over the fact that homeopathic preparations tend to contain no active ingredient, so will in all likelihood produce no symptoms at all). This is illustrated by a proving of hydrogen mentioned in Dantas et al., where the original trial produced 50 times more symptoms than a subsequent trial with improved methodology. According to Sherr and Quirk, the problem here is not with the original trial, but with the improved one, which produced too few symptoms to constitute a usable proving.

Dantas et al. respond with a paper entitled 'We must distinguish symptoms caused by the medicine from other symptoms'. In this case, the title is probably an adequate response on its own.

Then Harald Walach has a paper in response to Sherr and Quirk, entitled "Potential nonlocal mechanisms make placebo controls in pathogenetic trials difficult". This, once again, is quantum gibberish being used to claim that placebo-controlled trials can't work for homeopathy, because of 'entanglement' between patient, practitioner and remedy. Not to put too fine a point on it, this is bollocks, because entanglement has not been observed for systems containing more than a few particles. This is just homeopaths trying to find a way out of all the negative placebo-controlled trials of homeopathy. The attempts by those sympathetic to homeopathy to explain it via quantum mechanics are taken apart in some detail on shpalman's blog here and also here. My favourite bit of Walach's response is this: "It is a well-known lore of homeopathic proving that those in control groups, relatives, or even the pet dog may develop proving symptoms although they have not taken the remedy. This lore, although anecdotal and not scientific evidence at all, is valuable since it suggests that placebo controls might not be adequate". So, although this 'lore' is 'not scientific evidence at all' it is still valuable as evidence that placebo controls may be inadequate. Hm. Perhaps another explanation is that the proving symptoms recorded in the trial had nothing whatever to do with the homeopathic preparation being trialled, and so could be expected to be found in people (or dogs) not taking the preparation? That's why you do a placebo-controlled trial in the first place, and that's why quantitative data (as opposed to self-reporting) on symptoms are so important.

At the end of it all, you have to wonder what would happen if relatively sceptical authors such as those responsible for Dantas et al. started to address the results from meta-analyses that persistently show that homeopathic preparations have no benefit beyond placebo. Unfortunately, there seems to be no sign of this happening, as the authors conclude their paper by saying "As evidence accumulates for the efficacy and safety of homeopathy from rigorous clinical trials, there is an increasing need to investigate and develop valid methodologies for the experimental pillar of homeopathy—the homeopathic pathogenetic trial". Still, perhaps this drive towards better methodology may have unintended consequences. As we know from the Shang et al. meta-analysis in the Lancet, the better the methodology of your study, the more likely it is to show no effect beyond placebo for homeopathy.

References

Dantas, F., Fisher, P., Walach, H., Wieland, F., Rastogi, D.P., Teixeira, H., Koster, D., Jansen, J.P., Eizayaga, J., Alvarez, M.E.P., Marim, M., Belon, P. and Weckx, L.L.M. 2007. A systematic review of the quality of homeopathic pathogenetic trials published from 1945 to 1995. Homeopathy, 96: 4-16.

Dantas, F., Fisher, P., Rastogi, D.P., Teixeira, H., Eizayaga, J., Alvarez, M.E.P., Belon, P. and Weckx, L.L.M. 2007. Authors' response: we must distinguish symptoms caused by the medicine from other symptoms. Homeopathy, 96: 275-276

Shang, A., Huwiler-Müntener, K., Nartey, L., Jüni, P., Dörig, S., Sterne, J.A.C., Pewsner, D., Egger, M. 2005. Are the clinical effects of homoeopathy placebo effects? Comparitive study of placebo-controlled trials of homoeopathy and allopathy. Lancet, 366: 726-732.

Sherr, J. and Quirk, T. 2007. Systematic review of homeopathic pathogenetic trials: an excess of rigour? Homeopathy, 96: 273-275

Walach, H. 2007. Response: potential nonlocal mechanisms make placebo controls in pathogenetic trials difficult. Homeopathy, 96: 278.