Wednesday, 21 November 2007

Something to prove

There's an interesting debate currently taking place in the pages of the journal Homeopathy. It's interesting in that there is actually an attempt to have a debate about one of the fundamentals of homeopathic practice. It's also interesting to see some of the anti-scientific responses to the debate. The original papers discussed can be found through the website of Homeopathy. Unfortunately, you probably won't be able to access them unless you or your institution subscribes to the journal.

In the January issue of Homeopathy, Dantas et al. published a review of homeopathic provings, or Homeopathic Pathogenetic Trials (HPTs), as the authors prefer to call them. The authors defined HPTs as being "clinical trials designed to investigate the effects of the exposure of human volunteers, in good health, to potentially toxic or pathogenetic substances, diluted and serially agitated according to homeopathic pharmacopoeial methods, with a view to providing data to inform their use as homeopathic medicines". The idea is that symptoms caused by the homeopathic preparations can be cured by the same preparations, under the principle of 'like cures like'. There is no good evidence that this principle can be applied as a general rule, but even so it has become one of the foundation stones of homeopathy. One problem is that the symptoms in an HPT are recorded by the volunteers who take part in the proving. No quantitative data is collected about symptoms, and there are well-known problems with such self-reporting studies. Also, in many cases there is no way of telling whether the symptoms occurred as a result of the homeopathic preparation, or for some other reason, because such trials are not always placebo-controlled (Dantas et al. claim that 58% of the trials in their review were placebo-controlled).

The review by Dantas et al. concluded that "Most studies had design flaws, particularly absence of proper randomization, blinding, placebo control and criteria for analysis of outcomes", and went so far as to state that "The central question of whether homeopathic medicines in high dilutions can provoke effects in healthy volunteers has not yet been definitively answered, because of methodological weaknesses of the reports". Their central point is that while provings often turn up all kinds of symptoms, methodological flaws mean that you can't tell whether the symptoms were caused by the homeopathic preparation or not. The authors recommend that improved methodology should be adopted for future HPTs.

This is interesting stuff, and suggests that there are at least some homeopaths who question the value of HPTs, and on perfectly reasonable scientific grounds. It all starts to go a bit wrong in the responses to the article, which were published in the current issue of Homeopathy.

Sherr and Quirk's response is probably the most fun, and I suggest you track it down for yourself (but only if you've got time for such nonsense). Their point of view can be summarised by a paragraph towards the end of the paper, where they state "Eliminating the majority of symptoms or characteristic single symptoms due to over scientific vigour or a concern about statistical significance or background noise, risks throwing out the baby with the bathwater. It is important to remember the proof of provings is first and foremost their clinical usability and efficiency". Over-scientific vigour or a concern over statistical significance, indeed. This is pre-enlightenment thinking if ever I saw it.

They also say that "A good proving is not about producing every possible symptom. It is about producing enough symptoms of quality so that the intelligent homoeopath can perceive a meaningful totality". I take this to mean that you don't have to worry about using the best possible methodology, because the homeopath has some magic way of 'perceiving a meaningful totality'. Also, the object of the proving is to produce 'enough symptoms', not the ones actually caused by the preparation. (Here I gloss over the fact that homeopathic preparations tend to contain no active ingredient, so will in all likelihood produce no symptoms at all). This is illustrated by a proving of hydrogen mentioned in Dantas et al., where the original trial produced 50 times more symptoms than a subsequent trial with improved methodology. According to Sherr and Quirk, the problem here is not with the original trial, but with the improved one, which produced too few symptoms to constitute a usable proving.

Dantas et al. respond with a paper entitled 'We must distinguish symptoms caused by the medicine from other symptoms'. In this case, the title is probably an adequate response on its own.

Then Harald Walach has a paper in response to Sherr and Quirk, entitled "Potential nonlocal mechanisms make placebo controls in pathogenetic trials difficult". This, once again, is quantum gibberish being used to claim that placebo-controlled trials can't work for homeopathy, because of 'entanglement' between patient, practitioner and remedy. Not to put too fine a point on it, this is bollocks, because entanglement has not been observed for systems containing more than a few particles. This is just homeopaths trying to find a way out of all the negative placebo-controlled trials of homeopathy. The attempts by those sympathetic to homeopathy to explain it via quantum mechanics are taken apart in some detail on shpalman's blog here and also here. My favourite bit of Walach's response is this: "It is a well-known lore of homeopathic proving that those in control groups, relatives, or even the pet dog may develop proving symptoms although they have not taken the remedy. This lore, although anecdotal and not scientific evidence at all, is valuable since it suggests that placebo controls might not be adequate". So, although this 'lore' is 'not scientific evidence at all' it is still valuable as evidence that placebo controls may be inadequate. Hm. Perhaps another explanation is that the proving symptoms recorded in the trial had nothing whatever to do with the homeopathic preparation being trialled, and so could be expected to be found in people (or dogs) not taking the preparation? That's why you do a placebo-controlled trial in the first place, and that's why quantitative data (as opposed to self-reporting) on symptoms are so important.

At the end of it all, you have to wonder what would happen if relatively sceptical authors such as those responsible for Dantas et al. started to address the results from meta-analyses that persistently show that homeopathic preparations have no benefit beyond placebo. Unfortunately, there seems to be no sign of this happening, as the authors conclude their paper by saying "As evidence accumulates for the efficacy and safety of homeopathy from rigorous clinical trials, there is an increasing need to investigate and develop valid methodologies for the experimental pillar of homeopathy—the homeopathic pathogenetic trial". Still, perhaps this drive towards better methodology may have unintended consequences. As we know from the Shang et al. meta-analysis in the Lancet, the better the methodology of your study, the more likely it is to show no effect beyond placebo for homeopathy.

References

Dantas, F., Fisher, P., Walach, H., Wieland, F., Rastogi, D.P., Teixeira, H., Koster, D., Jansen, J.P., Eizayaga, J., Alvarez, M.E.P., Marim, M., Belon, P. and Weckx, L.L.M. 2007. A systematic review of the quality of homeopathic pathogenetic trials published from 1945 to 1995. Homeopathy, 96: 4-16.

Dantas, F., Fisher, P., Rastogi, D.P., Teixeira, H., Eizayaga, J., Alvarez, M.E.P., Belon, P. and Weckx, L.L.M. 2007. Authors' response: we must distinguish symptoms caused by the medicine from other symptoms. Homeopathy, 96: 275-276

Shang, A., Huwiler-Müntener, K., Nartey, L., Jüni, P., Dörig, S., Sterne, J.A.C., Pewsner, D., Egger, M. 2005. Are the clinical effects of homoeopathy placebo effects? Comparitive study of placebo-controlled trials of homoeopathy and allopathy. Lancet, 366: 726-732.

Sherr, J. and Quirk, T. 2007. Systematic review of homeopathic pathogenetic trials: an excess of rigour? Homeopathy, 96: 273-275

Walach, H. 2007. Response: potential nonlocal mechanisms make placebo controls in pathogenetic trials difficult. Homeopathy, 96: 278.

6 comments:

jdc324 said...

"Sherr and Quirk's response is probably the most fun"
Or the most likely to send you into an apoplectic rage. Good post.

apgaylard said...

Good stuff. There are several interesting accounts of one of the many problems with insufficiently controled 'provings' in Anthony Campbell's book:

3x Pulsatilla vs. dummy. The results? "... There was no evidence that Pulsatilla had produced any more symptoms than had the dummy tablets ..." and "... the largest number of symptoms occurred during the first month - that is,at the time when all the volunteers were taking dummy tablets. Some, indeed, withdrew from the trial because of the severity of their symptoms. The incidence of symptoms declined progressively over the whole 3-month period regardless of whether provers were taking Pulsatilla or dummy tablets ..."

Informal Experiment with an expectation of Lachesis
"... Dr H. Walach ... was giving a lecture on provings, in which he was describing his own experience with a homeopathic medicine derived from the rattlesnake (Lachesis). During the lecture he invited the audience to take part in an experiment; he handed round two bottles containing pillules ... Nine people took the pillules. Ten minutes before the end of the lecture Dr Walach asked what effects people had had from the medicine. Of the 9 who had taken the pillules, 4 had had definite symptoms - two with preparation 1 and two with preparation 2. Some were quite striking: one person had felt the whole left side go to sleep and had experienced a choking sensation. These are the effects that would be expected from Lachesis, according to the homeopathic literature. The audience was then asked to say which bottle they thought had contained the real medicine and which the placebo; their assessments were equally divided. And the answer? You’ve probably guessed it: neither - both were placebo. [British Homeopathic Journal,85, 123-5] ..."

HOMEOPATHY IN PERSPECTIVE: MYTH AND
REALITY
http://www.accampbell.uklinux.net/homeopathy/homeopathy-pdf/homeobook.pdf

Paul Wilson said...

Thanks for the comments...

Here's apgaylards link, for which many thanks. I wasn't aware of this book.

For some reason, you need to hand-code links in the blogger comments. Why I don't know, but there it is.

As for Sherr and Quirk, it's a truly astounding piece of work. It surely couldn't have been published anywhere other than a CAM journal. Can you imagine, say, Nature publishing a response like that?

Dr Aust said...

Yes, well done to Adrian/AP for spotting the link to Anthony Campbell's book. I had read it a while back, but last time I looked for it I had the impression Campbell had taken it down from his site (I think he is selling a revised edition).

BTW, have you read Campbell's latest comment on homoeopathy (and alternative medicine in general)?

On the Homeopathic argument, the paper by the breath-takingly bonkers Jeremy Sherr is a true hoot. My favourite bit was near the beginning when they say:

"...
While we believe that it is worthwhile to explore the issues raised by provings through a research model because it invites academic debate and further study, we are concerned with some of the comments in the related guest editorial. It is surprising to read the statement, “Despite a lot of effort, it remains very uncertain that HPTs yield valid results, capable of contributing to the cure of disease”. It seems as if the guest editor is suggesting that none of our remedies work in practice. This is perplexing because we presume that he commonly uses Staphysagria, Aurum or Sulphur and perhaps on occasion, some modern proved remedies. It is one thing to raise controversial questions by publishing original research, but quite another for an editorial in a respected homoeopathic journal to so strongly condemn the historical provings on which our profession is based and which have proved themselves in millions of cases."


Well, out of the mouths of....

Actually, the tone of this comment is quite funny. Reading it I am oddly reminded of a Catholic inquisitor working up to calling for someone to be burned, or a 1930s Stalinist prosecutor's mock- astonished opening gambit in the show trial of some "bourgeois revisionist".

Paul Wilson said...

Dr Aust, thanks for those links. I had no idea that this Sherr character was quite so barmy. It makes the Sherr and Quirk paper seem only mildly batty in comparison.

Dr Aust said...

Gimpy has this take that homeopaths like Sherr have one "line" for public consumption (the thoughtful serious therapist persona) and then the balls-out utterly mad we-believe- -in-the-spirit side they tend to conceal except among themselves.

I only wish more people who brush up against homeopathy would read some of what people like Sherr really believe, and indeed what the fathers of modern homeopathy like Tyler Kent really thought about disease (in essence "disease is all in the mind, so we cure it with spiritual mumbo-jumbo"). The public face of modern homeopathy is very deliberately anodyne and soothing, though their loathing of conventional medicine and science keeps bleeding through.