Wednesday, 19 November 2008

Science by press release (epic fail)

While I was messing about at the University of Google, doing some research for the thing that I've just sent to homeopathy, I came across this [pdf]. Yes, the homeopaths (in the form of the International Homeopathic Medical League and the European Committee for Homeopathy) have put together a press release, based on the study in Homeopathy by Rutten and Stolper, and the study by Ludtke and Rutten in the Journal of Clinical Epidemiology. Both studies criticise (mostly wrongly) a perfectly good meta-analysis of homeopathy from the Lancet. The headline is "New evidence for homeopathy". No question mark, no caveats. You might guess that the International Homeopathic Medical League and the European Committee for Homeopathy would not be entirely unbiased sources of information. In fact, these studies are very far from being evidence for homeopathy, as I've tried to point out (and not just me). I suppose two studies sounds better than one, but the truth is that these are essentially the same study, with extra nonsense in the part that got published in Homeopathy. The studies apply a post-hoc analysis to the data from the Lancet article, and claim to be able to produce a positive result for homeopathy if you squint at the data and stand on one leg. Then they accuse the Lancet paper of data dredging. What a laugh riot.

Unfortunately, lots of scientific studies get reported based on their press releases. This means that the people who read the reports get no sense of the flaws in the study and the caveats that should be applied to its conclusions; these come from independent scientific scrutiny of the study once it has been peer-reviewed and published. Pushing press releases on your research may be a good way to get brownie points from your university, but it's not usually a good way of fostering an improved understanding of the scientific process. To be fair, it's by no means only homeopaths and their ilk who do this, as Ben Goldacre has pointed out.

Still, the good thing about this is that a search on Google News for this release shows that, at the time of writing, it has only been picked up by a few woo-ish magazines: mainstream western news seems to have ignored this more or less completely. Maybe the press has got fed up of this particular manufactured controversy, at least for now.

6 comments:

apgaylard said...

The FoH have done the same.

I still see that they are crying over the spilt milk of the three and a half months delay in getting the full details - as if that makes any difference now.

jdc325 said...

"The studies apply a post-hoc analysis to the data from the Lancet article, and claim to be able to produce a positive result for homeopathy if you squint at the data and stand on one leg."
Thank you for providing this information. I have applied this method to various data relating to a number of treatment modalities and have now found that I can actually prove that everything works. Brilliant.

"Maybe the press has got fed up of this particular manufactured controversy..."
Unlike homeopaths.
The bogus complaints about Shang et al just don't seem to show any sign of disappearing, do they?

Paul Wilson said...

From the FoH press release:

"The clear implication is that homeopathy for certain conditions is not placebo".

From Shang et al.:

"There was little evidence that treatment effects varied according to...clinical topic (p=0·660 for homoeopathy, p=0·360 for conventional medicine)".

Ulrich said...

Francis Sedgemore has also commented on this strange press release:
http://sedgemore.com/2008/11/new-evidence-for-homeopathy/
He traces it back to a PR company specializing in sports and events. You seem to think it was released by IHML/ECH. Actually it was released by Elsevier for its journal "Homeopathy":
http://www.elsevier.com/wps/find/journaldescription.cws_home/623042/description

Francis Sedgemore said...

The press release may have been published by Elsevier, but that doesn't mean it was an Elsevier press officer who wrote the copy. Note that journal publishers and research institutes occasionally retain the services of freelance copywriters to prepare press releases. What happened in this particular case remains an open question.

Francis Sedgemore said...

I meant to write "...freelance copywriters and outside agencies..."