Saturday, 10 January 2009

Yet one more radically pointless homeopathy study

Generally, when the efficacy of a medical intervention is being assessed, you expect the quality of the trials and evidence available to increase through time. You might start out with uncontrolled case reports and observational studies, proceed through small-scale pilot studies, to large randomised placebo-controlled trials, and finally to systematic reviews and meta-analyses of the literature.

In CAM research this pattern is often not followed. Once systematic reviews and meta-analyses start to show that there is no evidence that the CAM treatment works, more small trials of poor methodology are conducted, many of which inevitably have (spurious) positive results. This allows CAM advocates to claim that there is lots of evidence in favour of their intervention, because they don't bother to account for study size and quality.

There is a fine example of this CAM tradition in the latest issue of Homeopathy. A systematic review, published in 2006, of homeopathy for treatment of allergic rhinitis concluded that "Some positive results were described in rhinitis with homeopathy in good-quality trials, but an equal number of negative studies counterbalance the positive ones. Therefore it is not possible to provide evidence-based recommendations for the use of homeopathy to treat allergic rhinitis, and further randomized controlled trials are needed". Well, perhaps: some would argue that the prior probability (close to nil) and currently existing evidence are enough to conclude that homeopathy does not work for allergic rhinitis (or, indeed, anything else). Be that as it may, it should be clear that the only useful new evidence would come from large and well-conducted RCTs. So what do Maria Goossens and a football team of colleagues do in the latest issue of Homeopathy? Why, publish a "prospective, open, non-comparative study" on homeopathy and allergic rhinitis, of course.

The methodology of the study consists of "treating" some patients suffering symptoms of allergic rhinitis with homeopathy, and getting them to fill in a quality of life questionaire at the start of the study, and after three weeks and four weeks. The physicians involved also assessed the severity of symptoms at baseline, three weeks, and four weeks. Unsurprisingly, the study found that people felt better with homeopathic treatment. But the methodological problems with this study are straightforward to point out. There is no control group. As a result, there can be no randomisation or blinding. Don't take my word for it; here's what the authors say in the discussion of their paper:

We did not distinguish between intermittent and persistent allergic rhinitis. All patients with intermittent allergic rhinitis (symptoms present less than four consecutive weeks a year) will be better after four weeks without any treatment. Patients who consult a homeopathic physician for allergic rhinitis usually have been suffering for a long time and from severe symptoms as the high level of the RQLQ score at baseline indicates. This study cannot be conclusive because there is no control group. Neither the physician, nor the patient was blinded. We cannot conclude that the degree of certainty of the physician about the appropriateness of the homeopathic prescription of a homeopathic remedy and the physician’s impression whether he had sufficient information about the patient’s condition influenced the outcome...it is not possible to draw a conclusion on the effect of the homeopathic treatment. This would require an RCT. To evaluate the effect of homeopathic treatment for allergic rhinitis an RCT should be performed.


So there you have it. The study cannot come to any useful conclusions. And, in the introduction to the paper, the authors write "This study was originally considered as a preliminary to a Randomized Clinical Trial (RCT) comparing standard conventional therapy with homeopathy (non-inferiority study). The RCT was never performed because sponsorship was withdrawn".

OK, that's life. Sometimes planned research funding fails to come off. These things happen. But why then publish the pilot study? Methodologically, it is useless, and it could never have added anything to the previously existing evidence from RCTs and systematic reviews. This study would never have been published anywhere other than a CAM journal, where scientific usefulness can take a back seat to an ideological desire to publish any evidence that looks as though it is in favour of homeopathy, no matter how methodologically weak it is, and in defiance of the higher level evidence that already exists.

Well, no-one is going to die from allergic rhinitis, so how much does it matter? The problem is that homeopaths don't stop at self-limiting conditions like hayfever. Some insist that homeopathy is a complete system of medicine and it can cure anything, including AIDS and malaria. Ben Goldacre's miniblog points to Jeremy Sherr's blog, for example, where Sherr is preparing to begin an unethical experiment on AIDS sufferers. This is a long road of madness, to be sure, but it begins where people believe they can cure hayfever through the use of magic sugar pills.

27 comments:

NJ said...

The lack of progression in CAM research is disappointing, anyone would think they didn't want well conducted trials investigating the efficacy of their treatments.

I also saw the 'Jeremy's Journal from Africa' post BG linked to, which would be funny if it wasn't so serious. Does anyone else get the image of Giles Wemmbley-Hogg popping off to Africa on his gap year to perform some experiments.

Dana Ullman said...

How typical of you to quote "A Systematic Review of Homeopathy" statement that an "equal" number of trials on allergic rhinitis have had negative results as positive results, and yet, you just happened to not mention the author of this "review" was E. Ernst, a man who is notorious anti-homeopathy.

Ernst again is loose the facts and figures, as are you...and in in the defense of "science." Thank you, but no thank you...

symball said...

what an honour- a visit from dana ullman himself.

and it is definitely the real deal- breathtaking hypocrisy, ignoring the questions, and trying to prove his case by making unfounded accusations of balance

zeno said...

I've just had the misfortune to read through and analyse (as best I can) another homoeopathic trial by Harald Walach et al. To me, it seems like a good example of how not to conduct a scientific trial. I've also mentioned it on the Quackometer, where Walach has been in a 'discussion' with Andy about this trial. My comments can be found here on the Think Humanism forum.

Paul Wilson said...

Dana, you lie. The authors of the systematic review I linked to are as follows:

G. Passalacqua , P. Bousquet , K. Carlsen , J. Kemp , R. Lockey , B. Niggemann , R. Pawankar , D. Price , J. Bousquet.

The reference is: Journal of Allergy and Clinical Immunology , Volume 117 , Issue 5 , Pages 1054 - 1062.

Ernst had nothing to do with that particular review. And so what if he had? You say that Ernst is notoriously anti-homeopathy, which is true enough, but he seems to have come to that opinion through an honest and rigorous assessment of the available evidence. It's an approach that I would recommend to you.

Paul Wilson said...

Zeno, thanks for the link to your comments on the Walach paper. It's one that I haven't read: there's so much rubbish out there that I don't think it would be fair to expect me to read all of it...

zeno said...

I understand! Life is too short.

Mojo said...

"But why then publish the pilot study?"

Another study with similar shortcomings was recently published, and generated headlines like "Homeopathy patients often say health improved" and "Study: Homeopathy can yield long-lasting improvements". Maybe that has something to do with it.

http://www.ncbi.nlm.nih.gov/pubmed/19091085

The lead author was quoted in the press coverage pointing out the shortcomings:

"The current study, published in the journal BMC Public Health, looked only at how homeopathy patients tend to fare in their everyday life. Therefore, it does not settle the question of whether the remedies are actually effective, lead researcher Dr. Claudia Witt told Reuters Health.

"This observational study design does not allow (us) to determine any causality between the improvement and the given homeopathic remedy, nor does it exclude the placebo effect," said Witt, of Charite University Medical Center in Berlin."

Neuroskeptic said...

So we can't trust Ernst when he says homeopathy doesn't work because he's anti-homeopathy ... and your basis for calling him that, presumably, is that he's well known for saying homeopathy doesn't work.

Some people would call that invalid circular logic, but don't listen to them, they're anti-circularists.

Paul Wilson said...

There I was, imagining that an honourable man like Dana would return to acknowledge his errors of fact and reasoning. How disappointing.

Mojo said...

Another similarly pointless study has recently been reported by Pulse:

http://www.pulsetoday.co.uk/story.asp?sectioncode=23&storycode=4121906&c=1

Along with a particularly stupid story commenting on it:

http://www.pulsetoday.co.uk/story.asp?sectioncode=20&storycode=4121911&c=2

The Mighty Colquhoun has turned up in the comments to give it a good kicking.

Paul Wilson said...

Mojo:

Thanks for pointing that out. Allow me to enliven those links:

First one.

Second one.

Some of the comments are pretty poor: supposedly "most people" have moved on from well conducted trials, and Prof Colquhoun should stop being fixated by them.

Dr. Nancy Malik said...

Homeopathy cures even when Conventional Allopathic Medicine (CAM) fails

Zeno said...

Nancy

We're STILL waiting for you at Think Humanism so we can do the test of a homeopathic potion you promised you'd do (and which you claimed you could pass).

Are you EVER going to keep your promise - or do you now think you can't tell the difference between a homeopathic potion and water?

Paul Wilson said...

Blimey, Zeno, that was quick. Only just got notified of Nancy's comment myself...

Homeopathy cures even when Conventional Allopathic Medicine (CAM) fails

No it doesn't.

If you have something related to my blog post to discuss, go for it. But at present your comment as as pointless as the study I was talking about.

Zeno said...

LOL!

She posted on jdc's blog at the same time!

She's just doing her usual drive-by comments. Does she really think she'll ever convince anyone???

Paul Wilson said...

Well, that's basically spam, then.

Zeno said...

Nancy Malik spammed Paulo's blog and jdc's this evening as well!

Dr. nancy Malik said...

Homeopathy is the evidence based modern medicine for the 21st century

http://www.hpathy.com/papersnew/milgrom-homeopathy-fundamentalism.asp

Paul Wilson said...

Nope.

Zeno said...

Nancy

We're STILL waiting for you at Think Humanism so we can do the test of a homeopathic potion you promised you'd do (and which you claimed you could pass).

Are you EVER going to keep your promise - or do you now think you can't tell the difference between a homeopathic potion and water?

Dr. Nancy malik said...

Homeopathy is non-toxic system of medical science originated in Germany by Dr. Friedrich Samuel Hahnemann (1755-1843) (the founder and father of homeopathy). He was M.D. in conventional medicine. The term “homoeopathy” was coined in 1807.

The four fundamental principles of Homeopathy are: -

1. Law of similars/Like cures like (1796): Disease can be cured by a medicinal substance given in micro doses that produces similar symptoms in health people when given in large doses.

2. Law of minimum dose (1801): Since the homoeopathic medicines act at a dynamic level, only a minute quantity of the medicine is administered which is enough/sufficient to stimulate the dynamically deranged vital force/innate healing powers to bring about the necessary curative change in a patient

3. Law of simplex (1810): At any given time, only one remedy can be the exact similar to the presenting disease condition of the patient. So a single remedy (one remedy at a time) is given based upon their constitution/totality of the symptoms which includes physical, mental, and emotional aspects/symptoms.

4. Hering’s law of five directions of cure (1845): Cure progresses from above downwards, from within outwards, center to periphery, from more important organ to less important one, in reverse order of coming of the symptoms

Zeno said...

Nancy Malik has now been banned from Think Humanism for trolling and refusing to properly engage in discussion. Since she had posted more than 200 times there, she had had a good crack of the whip, but showed no signs of changing her ways.

She failed - in all those 200-odd posts - to give even one example of a good quality trial that showed that homeopathy had any effect over placebo, even for a minor self-limiting condition.

Her modus operandi is simply to bombard with links to quack sites.

Her latest contribution here may seem good to the uninitiated, but falls down with the slightest scrutiny. There are no 'laws' of homeopathy, just wishful thinking.

Dr. Nancy Malik said...

Studies in support of Homeopathy published in reputed journals

1. Scientific World Journal
http://www.ncbi.nlm.nih.gov/pubmed/17982565

2. Lancet
http://www.ncbi.nlm.nih.gov/pubmed/9310601

3. Neuro Psycho Pharmacology
http://www.nature.com/npp/journal/v27/n2/abs/1395862a.html // Bacopa Monnieri for memory

Paul Wilson said...

These studies are not in favour of homeopathy. See here.

Dr. Nancy Malik said...

Triple Blind studies, Double-Blind Randomised Placebo-Controlled Trial, Systematic Reviews & Meta Analysis, Evidence-base

130+ studies in support of homeopathy medicine published in 45+ peer-reviewed international journals

Medicines for specific disease conditions, Ultra-molecular dilutions, Structure & Memory of Water, Animal Studies, Plant Studies

http://knol.google.com/k/dr-nancy-malik-bhms/scientific-research-in-homeopathy/pocy7w49ru14/2#

Aarex Homeo said...

Nice Post.,,, Really homeopathy treatment for allergies is best treatment for allergies.