Blogging is a transferable skill...

I've just returned from Aberdeen, where I was being interviewed for a job at the university (fingers crossed). In my limited experience of these things, it's always a bit nerve-wracking, and you can never quite predict what questions you'll be asked (although there are some that always come up: where do you see yourself in five years?). One of the five-member panel asked me if, apart from the research-related stuff on my CV, there were any other skills or things that I would like the panel to know about. I was unprepared for that, and I figured that saying 'no' would sound like 'no, I have nothing of interest to say about myself whatsoever', and couldn't be a good thing. Although I hadn't planned to, I talked a little bit about this blog, and explained how I used it to try and explain scientific issues in an accessible way. To my surprise, it went down very well with the panel, and I was asked how my blogging experience might help in university teaching. Given my relative lack of teaching experience, having the opportunity to talk about that can't have done my chances any harm.

So, if you're a blogger doing interviews, it's probably worth bringing up your blog. After all, you use it to distill complex issues down to their essentials, don't you? That's definitely a transferable skill.

More intention experiment: the ghost of Benveniste

I should probably avoid the Intention Experiment website, but it's such a fabulous compendium of nonsense and loony ideas that it's hard to leave alone. Now they're channeling the ghost of Jaques Benveniste. As many will know, Benveniste caused a storm of controversy when his lab published a paper in Nature that appeared to show that homeopathic concentrations of a certain type of antibody could have a biological effect, even though the chances of the solution containing any actual molecules of the antibody were tiny. This is the basis of the idea of 'memory of water' in homeopathy. The Nature paper was published with an unprecedented "editorial reservation", and a team assembled by Nature visited the Benveniste lab to look into the results. The results of the investigation were damning, showing that the design of the experiments was poor, and an article outlining the problems was published in a later issue of Nature. Ever since, homeopaths and other brands of quacks have been convinced that Benveniste's results were suppressed by the scientific establishment on behalf of Big Pharma, and so on.

Benveniste, however, was not put off by this setback, and continued in his work, going on to suggest that not only did the 'memory of water' effect exist, but that it could be transmitted digitally, down phone lines or over the internet. Brilliantly, this only gave 'positive' results when the equipment was being run by a particular researcher. Benveniste doesn't seem to have reflected too hard on why that might have been the case. Benveniste called this "digital biology", which would have "immense consequences on medical diagnostic procedures and the agro-food industry, with huge technological and commercial impact", and was only being held back because scientists are "opposed to the evolution of science". You recognise the narrative here, don't you?

Why do I mention all this? Because the Intention Experiment blog carries news of an experiment into "healing by e-mail". Apparently, a "Francesca McCarney, Ph.D., teacher of professional intuitives at the Academy of Intuitive Studies and Intuition Medicine" conducted the experiment, where 88 people were each sent 2 e-mails. One of these e-mails had "healing energy" "encapsulated" into it, and the other one did not. Apparently, the e-mails were indentified correctly 31.9% of the time, against a 25% probablity of getting them right by chance. This seems like a deeply unimpressive result, with no confidence interval to give us an idea of how likely it would be for such a result to occur by chance, but we're told that "scientists would consider it highly significant result" [sic]. We also don't know whether there might inadvertently have been clues in the text of the e-mails.

It seems that loony ideas never die; they just re-appear periodically in a slightly different form.

Another study for homeopathy awareness week...

It's homeopathy awareness week (until tomorrow), with a focus on homeopathic treatment for allergies.

In the spirit of awareness, here's a systematic review of complementary therapies in the treatment of allergic rhinitis and asthma in the Journal of Allergy and Clinical Immunology, with a summary at the NHS National Library for Health here.

For homeopathy, the authors found that "Some positive results were described with homeopathy in good-quality trials in rhinitis, but a number of negative studies were also found. Therefore it is not possible to provide evidence-based recommendations for homeopathy in the treatment of allergic rhinitis, and further trials are needed".

Fun with trials, for homeopathy awareness week...

Occasionally, people sympathetic to homeopathy try to discuss scientific studies. It's usually fun when this happens, because I get chance to do some paper deconstruction. What can I say, I'm a geek. I usually learn something from doing it, even if only that I'm talking rubbish.

In the comments to this post, a lady (I assume) named Vicky posted three studies claimed to support homeopathy. Of these, the Graunke et al. paper on tadpoles has some glaring problems which are discussed here, and the Oberbaum et al. paper simply contains too few subjects (15 in each group) to come to any reliable conclusions. The third paper, by Bernstein et al., was a bit of an oddity, and I'd like to take a slightly closer look at it, in honour of homeopathy awareness week.

Firstly, the paper is published in the American Journal of Therapeutics. This journal does seem to be peer-reviewed, but is not listed in the ISI Journal Citation Reports, which suggests that it is not a top-of-the-line journal. But that in itself doesn't necessarily mean the findings are wrong.

The paper studies the effect of a proprietary homeopathic remedy, Relieva, on psoriasis. First red flag: psoriasis is a condition where patients experience flare-ups and remissions. So any improvement could be a result of this natural cyclicity rather than any treatment effect.

Relieva is an unusually concentrated substance in homeopathic terms. The study describes the active ingredient as "a highly concentrated, proprietary extract of M. aquifolium 10% known as Psorberine". This remedy actually contains some active ingredient, unlike many homeopathic remedies, so there is at least a chance of it working without needing to overturn practically everything we know about pharmacology.

The study is described as a randomised, double-blind, placebo-controlled study, and in general it seems to be well conducted. There were a total of 200 subjects, 100 in each group, suggesting the trial is large enough to get decent results. The authors conducted an 'intention to treat' analysis. In this analysis, all subjects enrolled in the trial are considered, rather than just those who completed it. The authors 'impute' a worst-case score (12 points) for Psoriasis Area Severity Index (PASI), a measure of psoriasis severity, to all the patients who dropped out of the trial. I'm not sure why they did this: they might have a sound reason for doing that, but I don't see it explained in the paper. Notably, 3 subjects dropped out of the treatment group, compared to 26 in the placebo group. This is a large difference, and it would be expected to introduce a bias into the results of the intention to treat analysis. Also, since a whopping 26% of the placebo arm dropped out of the trial, the results are at least of questionable reliability. The authors suggest that the high drop-out rate in the placebo group compared to the treatment group might be an indication of the efficacy of the treatment. However, their analysis of the subjects that completed the trial suggests that the placebo group experienced some improvement in their psoriasis (a PASI point reduction of 2.22 compared to 3.58 in the treatment group), so I'm not sure that explanation stacks up. The maximum increase was +3 for the treatment group and +4 for the placebo group. It's also worth noting that this big difference in drop-out rates has the potential to break the blinding of the trial.

The acknowledgments of the paper are very odd, at least to me. They say "The authors thank Global Clinicals, Inc (www.GlobalClinicals.com ) and its staff for performing this study, Missy Magill for the statistical evaluation, and Andrea Gwosdow (PhD) for preparing this manuscript". It's possible I'm being naive here, but this raises the question of what the authors actually did to earn the right to be credited as authors.

Who then are Global Clinicals, Inc? Here's their webpage. They undertake clinical research on behalf of their clients, so there is at least the appearance of a conflict of interest here; clients are not going to be happy if their treatment is shown to be no good. This is the sort of the thing that Big Pharma are (rightly) criticised for.

So overall, I would suggest that the evidence in the trial does not unequivocally show that Relieva works. There are also some shenanigans surrounding the authorship of the paper that would be highly criticised by homeopaths if they occurred in a trial of conventional medicine. Finally, even if you accept that the trial provides evidence in favour of Relieva, it provides no general evidence in favour of the basic tenets of homeopathy ('like cures like' and 'potentisation'). For that, you would need to show that Relieva causes psoriasis in healthy volunteers in a 'proving', and show that its effects increased with increasing dilution. If it does work, it's not because it's homeopathic.

Homeopathy works! At least in Mailworld...

There's a terrible, terrible article on homeopathy in the Daily Mail online (cheers to Dr. T on the Bad Science forum for spotting it). The title is "Homeopathy works!", and it really is just about the least critical article I can imagine.

For a start, the fact that homeopathic remedies are generally so diluted that they contain no active ingredient is glossed over with the phrase "Critics argue that the active substance is so diluted that homeopathic remedies have no more effect than placebo or dummy treatment".

The article reports on a study by Dr. David Reilly at, God help us, the Glasgow Homoeopathic Hospital. What do we know about the study? Virtually nothing. According to the article, the study looked at 50 patients suffering from nasal allergies. Some were given homeopathy and some a placebo. I tried to find the study, so I could understand what they did. A search on the pubmed database doesn't find it. There is no mention of the status of the study in the Mail article. Has it been submitted to a journal? Presented at a conference? Written up on the back of a fag packet? We don't know. So there is no way the claims made in the article can be evaluated in any sensible way. Was the study blinded? Did it have adequate randomisation? What was the placebo used? We don't know. The only thing I can say about it is that the study size is small, and so more likely to produce a false positive result.

So, the reporting of the particular study that the article discusses is lamentable. But perhaps even worse is the total failure to place the study in the context of the rest of the scientific evidence. Dr Reilly is allowed to get away with saying that the study replicates some of his previous studies, and that "there were positive findings in 70 per cent of a further 180 clinical trials" of homeopathy.

Perhaps 70% of trials of homeopathy did apparently show a positive result. But this could be a result of poor study designs or insufficient sample sizes. Luckily, this can be tested using a technique called a meta-analysis. Essentially, you get the data from all the published trials, pool it together, and apply statistical techniques to the pooled data to evaluate bias. Such an analysis was conducted by Aijin Shang and co-workers, and published in the Lancet in 2005. This is a study much misunderstood by homeopaths, but what it shows is that large and well-designed trials show no effect for homeopathy, and that positive results of some trials can be explained by poor experimental design, bias and small sample sizes. A previous meta-analysis by Linde et al., published in the Journal of Clinical Epidemiology, came to similar conclusions. Dr. Reillys' previous work on homeopathic treatment for allergies was included in the Shang study, and was not deemed to be of high quality.

So, the current state of the scientific evidence is that there is no plausible mechanism for homeopathy to work, and that the results of large, well-conducted studies are consistent with homeopathic remedies being no more than placeboes. Going by the Mail article, you would get the impression that homeopathy was dismissed by blinkered scientists, but that evidence was gradually stacking up in favour of it. That is simply not the case.

But at least the Mail didn't claim that homeopathy can cure cancer...

EDIT: It has come to my attention that the Mail story is actually reporting the results of a trial that was published in the BMJ...in 2000. So why it has appeared 8 years later is anyone's guess. This trial was also included in the Shang meta-analysis, and not deemed to be of high quality. This is perhaps because the trial was not double-blind, even though it was described as such in the BMJ [Edit: my mistake. The wash-in period of the trial was single blind. The intervention was double-blind]. The authors also calculated that they needed to recruit 60 subjects in each group (a total of 120) for the study to have the required statistical power. There was a total of 50 subjects. So the only sensible conclusion would be that there was insufficient evidence either way.

Edit 2: Here's the author of the piece, Jenny Hope, giving evidence to the House of Commons Select Committee on Health regarding the influence of drug companies on research and research reporting. Note this bit:

When a story is in the public domain—or being placed with the specific intention of generating publicity—someone will write about it. If it's me than I know where the information is coming from, the background, and carried out the interviews—there is some quality control.

The quality control on this particular story seems to be negligible. Perhaps Ms. Hope was having a bad day?

Watch your units!

When I was doing GCSE science and A level chemistry, I was always taught to be careful about units of measurement, especially when I was doing calculations. This is a lesson that would not be lost on the odd science journalist. Today's Guardian has an article on a photovoltaic solar farm in Portugal. According to the article, the farm is "expected to supply 45MW of electricity each year, enough to power 30,000 homes”.

A megawatt (MW) is a unit of power, equivalent to 1,000,000 watts. 1 watt is equivalent to one joule per second. So it doesn't make sense to say that the farm will supply 45MW "each year".

Pedantic, I know, but you'd expect a national newspaper to get that sort of thing right.

Lorenzo's Oil: not quite a miracle cure

I've seen a couple of stories in the press about the death of Lorenzo Odone, who had adrenoleukodystrophy (ALD). ALD occurs when an essential protein that removes very long chain fatty acids (VLCFAs) is missing, resulting in a build-up of VLCFAs in the body which damage the brain. Most sufferers die within 1 to 10 years of showing symptoms.

Lorenzo Odone lived until the age of 30, despite doctors giving him little chance of surving beyond the age of 8 or so. His relative longevity has been ascribed to 'Lorenzo's Oil', a mixture of glycerol trioleate and glycerol trierucate. The oil was initially used as a treatment by Lorenzo Odone's parents, who were understandably unwilling to simply watch their son die. They undertook a considerable amount of research, despite being laypeople with no medical expertise. They thought that the oil would help inhibit the creation of VLCFAs. Lorenzo's condition apparently stabilised after the oil became part of his diet. Since two laypeople creating a therapy that had eluded the finest minds of medicine is quite a story, a film was made, titled 'Lorenzo's Oil'. The money the Odone's made from the film went into the Myelin project, which funds research into diseases like ALD and MS.

However, the Odones found the scientific community sceptical. In the Guardian obituary, Lorenzo's sister Cristina Odone writes "Not all members of the scientific community welcomed this approach; or, indeed, their portrayal on film as hard-hearted and arrogant. A backlash against the Odones saw various researchers from around the world denying the efficacy of Lorenzo's oil, and most doctors refused to prescribe it. In 2005, however, the world's top authority on ALD, the late Dr Hugo Moser, published the findings of a 10-year study which showed that a statistically lower incidence of ALD occurred in those boys whose diet included Lorenzo's oil". This is what interested me in the story, as it fits into a neat narrative about laypeople taking on the might of the scientific establishment, and being treated in a cold-hearted and callous manner as a result.

Firstly, I would say that an initial sceptical approach was sensible. The only evidence that Lorenzo's Oil might work was from the apparent stabilisation in the condition of Lorenzo himself. There could be a huge number of other reasons why this happened that had nothing to do with Lorenzo's Oil. The only way to find out whether it really worked or not was to conduct scientific trials. Until this had been done, it was sensible to be sceptical of a therapy for which there was essentially no evidence of efficacy.

So, what of Moser's study? This is certainly not my area of expertise, but I can at least have a look at the methodology of the paper. The trial followed 89 boys who were thought to be at risk of ALD based on an assay of VLCFAs in their plasma. All of the boys were given Lorenzo's Oil and 'moderate fat restriction' in their diet. The study concluded that treatment with Lorenzo's Oil caused a reduction in plasma hexacosanoic acid, which was associated with a reduced risk of developing abnormalities on an MRI scan. However, this was a single arm trial: there was no placebo group. I'm not criticising the study for this; if the treatment is likely to help, then it is probably unethical to withhold it from the subjects of the study. But it does make it more difficult to know whether the results are because of the therapy, or because of something else. What would be more interesting, I think, would be to look at whether there is a difference between subjects given Lorenzo's Oil and a reduced VLCFA diet, and those just given the diet. As far as I can tell, this has not been done. So it is difficult to know if the oil has any effect on its own. A more recent study in the journal Metabolic Brain Disease followed 11 subjects, all of whom were taking Lorenzo's Oil and were on a diet restricted in VLCFAs. Again, this doesn't allow us to separate the effects of the oil from the effects of the diet.

While these studies might show some evidence for an effect of Lorenzo's Oil before symptoms appear, it seems that the treatment doesn't work well in symptomatic subjects. This may be because the compounds that make up Lorenzo's Oil do not cross the blood/brain barrier.

Now, I'm not saying that Lorenzo's Oil doesn't work. This isn't homeopathy, after all, and there does seem to be a plausible mechanism by which it might work. I'm not even saying that what the Odones have acheived is not impressive. What I am saying is that the evidence that it does work is fairly weak, it is far from being a miracle cure, and the story is a little bit more complicated than the narrative that is being presented.