Infinite costs wouldn't be Nice

There's been a bit of a furore over the last couple of days about a decision not to approve a drug for treating advanced bowel cancer, Avastin, on the grounds that it would not be cost-effective. This has led to a huge amount of criticism, and headlines ranging from the relatively sober to the Mail's hysterical "Betrayal of the cancer patients: Rationing watchdog accused of talking down wonder drug's power to save lives".

The decision (or guidance: in fact, the final decision has not yet been made, with the guidance subject to consultation and appeal) was made by the National Institute of Health and Clinical Excellence (Nice), which decides on the cost-effectiveness of treatments and whether they should be available to patients on the NHS. According to trial data submitted by the drug manufacturer, Roche, the drug can extend the mean lifespan of people diagnosed with advanced bowel cancer from 19.9 months on chemotherapy alone, to 21.3 months with chemotherapy and avastin. This is by no means a miracle cure or wonder drug. This is a six-week gain, but it comes at a cost: the drug costs up to £21,000 per patient.

Barbara Moss, a survivor of bowel cancer, appears in both the BBC and Guardian articles. The Guardian says:

Mrs Moss said the National Institute for Health and Clinical Excellence (Nice) had put a "value on life" after it said the price was too high for the extra benefit it gives patients

While Mrs Moss is quoted in both pieces as saying:

It seems immoral to me that, as a result of negative NICE decisions like this one, people's choice of living or dying depends on whether they can afford a drug, because it isn't available to them on the NHS.

It's difficult not to have some sympathy for this point of view. But someone has to put a "value on life" (or in fact, a thing called a Quality Adjusted Life Year, QALY). Otherwise Roche could come up with a life-saving drug that cost £1 trillion per patient, and the NHS would have to fund it because you "can't put a value on life". Ultimately, the question is not just about avastin. There is only a limited amount of money for the NHS to spend on drugs, so if avastin gets funded something else (with probably greater benefit) doesn't get funded . These are clearly not easy decisions to make, which is why we have Nice to do it in the first place.

My point here is not that Nice have necessarily got this right (I'm not qualified to judge), or even that the QALY approach is the best possible one. The point is that Nice is doing a difficult, unpopular and necessary job. Accusing them of betraying cancer patients, when what they're actually trying to do is get the greatest benefit from a limited budget, is not really helpful to anyone.

Bad cycling: I'm the champion (but I'm not very fast)

Last night was the Stockport Clarion club hill-climb, on the infamous Cat and Fiddle road between Macclesfield and Buxton. I rode in this event last year, and had a good ride on a clear and dry day with a good tailwind. I had high hopes of bettering the 26:00 I did last year, but things didn't go exactly to plan.

There's a webcam attached to the side of the Cat and Fiddle pub, which looks over the finish of the course at the road summit. I had a look at it yesterday afternoon, and there wasn't much to see except cloud and massive wagons heading over to Buxton with their fog lights on. It didn't look like a lot of fun. Once I got home, I was having serious second thoughts about the whole enterprise as I got ready for the 18-mile ride to Macclesfield, with the rain rattling the windows in my flat. But I made myself head out, leaving in absolutely filthy weather and gnarly traffic for a fun rush-hour jaunt through Stockport. The weather gradually cleared up as I got towards Macclesfield, and I arrived with about ten minutes to spare before the 7pm start.

Unfortunately, the start had been put back to 7:30, so I ended up waiting around in the cold for 40 minutes. Will, the timekeeper, didn't make it until just before 7:30 because he had to pick someone up from the airport. So, dispensing with numbers, the four riders who had shown up got going as soon as possible. The weather was now dry, but cool and windy. I decided to ride with lights in case it was cloudy at the summit. I went off number 2, and it was one of those days when you know it's going to be rough as soon as you set off. I hadn't had enough of a warm-up, and my legs just didn't want to have any of it. I didn't really get settled in until I was past Walker Barn and had done the most difficult bit. By then, the two guys behind me had already gone past, but I had caught up with my minute man. Emerging into the open country beyond Walker Barn, the gale-force tailwind came into play, and the middle section of the course was very fast. I even had to touch the brakes on some of the corners. Then the last climb onto the top: I almost got blown off the road on a cross-wind section, and struggled past the timekeeper in 26:36, 36 seconds slower than last year. But, with the only other Stockport Clarion rider being my minute man, who I had passed about two and a half miles earlier, I was the club hill-climb champion. This is nice, especially as I was several times hill-climb champion at my former club, Birdwell Wheelers, but there's no pretending that I was particularly quick. Several riders in the club would have put minutes into me had they ridden, but like the lottery, you have to be in it to win it. Jolan managed to capture a webcam image of us at the top: I'm the guy in the red shirt at left...


My reward was a long, windy ride home, mainly in darkness, but I got to see the sun go down over Fiddlers Ferry power station as I headed back down into Macclesfield.

Bad cycling: I get round in 27:45

Following my New Year's Day adventures, I haven't done as much bike racing as I hoped I would this year. I was away In Norway all of June for work, and then the Stockport Clarion evening time-trials took a break during July. However, the last three Monday evenings I've been out on the Chelford 10-mile course, in an attempt to get some race fitness for next week's hill climb on the Cat and Fiddle. My aim was to get under 28 minutes for 10 miles, which is pretty modest. After a horrible day in the first event, where I was recovering from a migraine and should probably have stayed at home, I clocked 29:30 and wasn't optimistic. But the following week was better with 28:24, and last night I cracked it, just about, with 27:45. For the first time in these 10s, there were actually a couple of people slower than me. It was a good night, dry and sunny with a light headwind on the slightly longer outward leg, and times were generally fast-ish. There was a 21-minute ride, which is pretty quick for such a slow course.

Coming into Chelford on the homeward leg, there was an electronic sign that gives a read-out of your speed. I was grovelling up the slight incline at a bare 21 mph, so the days when I could get under 25 minutes seem long ago. One of the other riders went through at 27 mph. Still, I felt good, and I'm looking forward to the Cat and Fiddle next week.

"Dr" Nancy Malik is spamming my blog again...

In the comments to a couple of old posts, homeopathic apologist and internet numpty extraordinaire Nancy Malik says the following:

Studies in support of Homeopathy published in reputed journals

1. Scientific World Journal
http://www.ncbi.nlm.nih.gov/pubmed/17982565

2. Lancet
http://www.ncbi.nlm.nih.gov/pubmed/9310601

3. Neuro Psycho Pharmacology
http://www.nature.com/npp/journal/v27/n2/abs/1395862a.html // Bacopa Monnieri for memory

I thought I'd have a quick look and explain why they're nonsense. Unfortunately, this hasn't proven to afford much in the way of intellectual exercise.

The first paper is by Graunke et al., and concerns, I kid you not, the treatment of tadpoles with homeopathic thyroxin. This is a well-known bad homeopathy paper. The tadpoles in the treatment group were more developed than those in the control group at the start of the experiment, so it wasn't much of a surprise that they were more developed at the end too. There is more discussion of this dreadful rubbish here.

The second paper is the famed Linde et al. meta-analysis, published in 1997. While this paper does say "The results of our meta-analysis are not compatible with the hypothesis that the clinical effects of homeopathy are completely due to placebo", there are some other things to bear in mind:

1. The paper also says "However, we found insufficient evidence from these studies that homeopathy is clearly efficacious for any single clinical condition".

2. A 1999 paper by the same authors, using improved methodology and including new trials, states that "It seems...likely that our meta-analysis at least overestimated the effects of
homeopathic treatments".

3. A subsequent meta-analysis by Shang et al., published in the Lancet in 2005, using further improved methodology concluded that the results were compatible with homeopathy being a placebo.

Finally, the third study, by Roodenrys et al. in the journal Neuropsychopharmocology, is not about homeopathy at all, but rather about herbal medicine. In homeopathy, remedies are typically diluted such that it is very unlikely that they contain any of the original material: there is no active ingredient. In the Roodenrys study, what is being tested is brahmi, an Indian herb, of which the paper says:

Studies have shown that the herb contains many active constituents, including a number of alkaloids and saponins, however, the major constituents are the steroidal saponins, Bacosides A and B.

So it isn't entirely surprising that brahmi might have some effect.

From this fairly cursory glance at the studies provided by Nancy Malik, it's clear that she is from the Dana Ullman school of evaluating journal articles. This involves finding some papers that superficially appear to support your position, and then spamming them all over the internet. Luckily, for this approach there is no need to understand the articles, or even to read them. For people who think that magic water is medicine, that would be rather too much to expect.

Private Eye still believes in Wakefield...

As discussed in my previous post, and many other places all over the internet, the GMC ruling against Andrew Wakefield was damning. But not so much if you write for Private Eye, who have been defending Wakefield for years. So, in the light of the GMC findings that Wakefield behaved unethically, with "callous disregard" for the wellbeing of the children involved, and that parts of his research were fraudulent, is it time for the Eye to admit it was wrong?

Apparently not. In an "In the Back" piece, the Eye had the following to say:

Although the GMC said the hearing was not about vaccination and autism, it is fairly clear that the two and a half years of disciplinary proceedings were to bring the MMR debate to a conclusion. The three doctors were to some degree being accused (and found guilty) of causing a public health scare which led to a fall in vaccination rates, so damaging "herd immunity", particularly with regard to measles.

Nope. They were found guilty of conducting invasive diagnostic procedures, against the children's interests and without ethical approval; of not disclosing serious conflicts of interest; and of presenting false information in the publication of their research (which finally led to the Lancet retracting the paper).

Several parents of the children who featured in the team's original research paper, which was at the centre of the GMC case, stormed out of the hearing in angry protest at the findings - particularly the suggestion that their children's tests were not clinically necessary...They say they would have told the GMC that the treatment they received at the hospital helped their children's symptoms - but they were never called to testify.

Eh? We're not talking about treatment here, we're talking about invasive diagnostic tests. These were done without ethical approval, without the required expertise, and against the interests of the children. Perhaps we should just get rid of ethical approval, as it gets in the way of heroic doctors doing whatever they need to vunerable patients.

And finally:

None of this debate about conduct, however, changes the fact that no subsequent research has supported Wakefield's thesis of a possible link between the MMR vaccine and autism. Nor does it change the fact that despite previous attempts to justify the Eye's coverage at the time, some readers are still deeply critical of the magazine's reporting of the MMR debate between 2001 and 2007, when we wrote about the concerns of Wakefield, the families and their lawyers, and endorsed calls for more research (see Letters).

Well, indeed. So why is the Eye still apparently defending the indefensible? I still buy the Eye, for all the other good that they do, but they've made a mistake over MMR and they ought to be big enough to admit it. They would surely ask the same of any of the politicians and industry figures they regularly lampoon.

It's literature integrity week...

The integrity of the scientific literature is at issue in not one but two news stories this week. Firstly, the 1998 Lancet paper by Andrew Wakefield and several co-authors, which purported to show a link between autism and the MMR vaccination, was retracted [PDF] by the journal. Meanwhile, there is an ongoing brouhaha about a 1990 Nature paper that has some implications for climate change research.

I'm only going to deal with Wakefield here because, you know, I have other things to do. The retraction by the Lancet is really the final nail in the coffin of Wakefield's paper, which has been heavily criticised for numerous reasons. Of the thirteen authors, ten had previously retracted the interpretation that there was any link between autism associated with gastro-intestinal problems and the MMR vaccine. Only Wakefield himself and one P Harvey refused to sign up to this: the remaining author could not be contacted. It has also been known for some time that the methodology of the paper was flawed. So what suddenly changed this week?

For some time the General Medical Council (GMC) has been investigating Wakefield and two colleagues, Simon Murch and John Walker-Smith, over allegations relating to their research on autism. The GMC findings were released on January 28th, and can only be described as devastating. You can find the whole thing here [PDF], or there's a good summary at Respectful Insolence.

The GMC did not look at whether the research findings were right or wrong: rather they looked at the research methodology. Essentially, the major problems were these:

1. Wakefield had an undisclosed conflict of interest, because he was being paid by lawyers whose clients believed their children had been harmed by the MMR vaccine.

2. Wakefield ordered invasive diagnostic tests, including colonoscopies and lumbar punctures, that were unnecessary and not in the children's interests, and he had neither the required ethical approval nor the requisite expertise to order those tests.

3. Wakefield obtained blood samples at his son's birthday party by paying children £5 each. The GMC panel described Wakefield's actions as comprising "callous disregard for the distress and pain the children might suffer".

4. The 12 children were described as having been "consecutively referred", but this was not true.

Heres what the Lancet had to say:

Following the judgment of the UK General Medical Council’s Fitness to Practise Panel on Jan 28, 2010, it has become clear that several elements of the 1998 paper by Wakefield et al1 are incorrect, contrary to the findings of an earlier investigation.2 In particular, the claims in the original paper that children were “consecutively referred” and that investigations were “approved” by the local ethics committee have been proven to be false. Therefore we fully retract this paper from the published record.

So that's that. The problem with the paper is not that it was wrong; that has been known for some time. In fact, simply being wrong would not be a reason to retract the paper. Science often progresses by building on papers that were not quite right. No, the problem with the Wakefield Lancet paper was that it was fraudulent, unethical and incompetent, as well as being wrong. As a result, the takeup of MMR has fallen below the ~95% level at which herd immunity is maintained, and measels has once again been declared endemic in the UK. That's why you should try to avoid publishing fraudulent and unethical research. We'll let you off if your research is merely wrong, as long as it was honestly and competently wrong.

Wakefield eh, what a hero? Unfortunately, as we'll see in my next post, some people in the most unexpected places still believe in the Cult of Andy...

The Observer ignores the evidence on homeopathy

Homeopathy is in the news once again, following a campaign set up by Merseyside Skeptics in which skeptics took an overdose of homeopathic pills, demonstrating that there's nothing in them other than lactose, and then went to the pub for some non-homeopathic beer. The campaign was mainly aimed at high-street pharmacist Boots, whose professional standards director admitted to the science and technology select committee of the House of Commons that there was no evidence that homeopathy worked, but they were very happy to continue charging people money for it. The campaign, and a forthcoming report of the select committee on whether the NHS should fund homeopathy, gave the Observer an excuse to publish a feature article, in which Anushka Asthana and Robin McKie examine the supposed controversy about whether homeopathy works or not.

Of course, scientifically speaking there is no controversy over homeopathy. It's perfectly clear that there is no scientific reason why it ought to work, and that when it is tested in properly conducted trials it works no better than placebo. I grant you that this probably wouldn't make for a very satisfying Sunday newspaper feature, and it would certainly be rather too short to fill up all of page 30 of the Observer. But you might expect there to be some discussion of the actual evidence. Not really, though. Edzard Ernst gets to say that "I have now published more than 100 papers on homeopathy and I am quite clear about its efficacy: you may as well take a glass of water than a homeopathic medicine". The evidence of Jane Lawrence of the Royal Pharmaceutical Society to the House of Commons select committee on science and technology that "There is no basis for [homeopathic remedies] being effective" is also quoted. But then Cristal Sumner of the British Homeopathic Association gets away with saying "Homeopathy helps patients and is not a placebo effect", despite all the evidence showing exactly the opposite.

This is typical of how the press treats scientific issues; they are presented in a superficially even-handed way, but crucially there is no attempt to weigh the evidence. Again, there is no scientific controversy over homeopathy, but by reading the Observer article you could be forgiven for thinking there was considerable room for doubt.

The worst things about the article, though, are the two inset boxes (these only appear in the printed version of the article, not in the online version, as far as I can tell). The first box is headed "Common Renedies" and lists "Popular homeopathic remedies for sale in Britain", including Arnica for clearing up bruises, and mixed pollen for treating hayfever. As these are homeopathic, they contain no Arnica or pollen, and there is no evidence they do anything except lighten your wallet, but this is not mentioned in the box. The second box is headed "Case Study", and recounts the story of one Helen Llewelyn, who claims that homeopathy helped control her endometriosis. This is an anecdote; it tells us that Llewelyn feels better, but it doesn't tell us anything about why, especially as we know nothing of what real medicines she might have been taking. From this website, it looks as though Llewelyn's case history is rather complex, and she is presently using homeopathy in conjunction with several kinds of real medicine. For medical treatments, you need much better evidence than this to show that they work. In the case of homeopathy, the evidence exists and it shows that homeopathy doesn't work. But there isn't an inset box anywhere that sums up the useful evidence: just one uncontrolled case report.

Given that there is genuinely a debate about whether the NHS should fund homeopathy, it would be good for this debate to be informed by the best currently available evidence. That is exactly what the Observer fails to do: all we get is "he said, she said" appeals to various forms of authority, and an anecdote. No wonder the newspapers are struggling.

Medical Hypotheses row resurfaces

Last year, publishers Elsevier got into trouble with HIV-AIDS researchers, after Medical Hypotheses (an Elsevier journal) published two papers on the subject of AIDS: one by Peter Duesberg claiming that the AIDS epidemic in South Africa was overhyped, and another by Marco Ruggiero suggesting that the Italian health ministry did not believe that HIV was the sole cause of AIDS (blog posts at Bad Science and Respectful Insolence). The papers were deeply flawed, and were retracted by Elsevier pending an investigation into how they were published. The story has now resurfaced in the Times Higher Education Supplement (THES), because:

Prominent Aids researchers contacted Elsevier to object to the article and wrote to the US National Library of Medicine requesting that Medical Hypotheses be removed from the Medline citation database - an act that would exclude it from the mainstream scientific-communication network.

Elsevier have now convened an expert panel to decide on the future of Medical Hypotheses, with conclusions due by the end of 2010.

In fact, there is no great mystery as to how these flawed papers came to be published. Medical Hypotheses is not peer reviewed: instead, decisions on publication are taken solely by the journal's editor, Prof Bruce Charlton. Articles are often accepted within days, or even hours, of being submitted, suggesting there is little or no quality control on what gets published. Prof Charlton defends this process on two grounds: firstly, that there ought to be some outlet for speculative and bizarre ideas that will not be published by mainstream journals. Secondly, that Medical Hypotheses is a successful and influential journal. Here's what he has to say on the comments following the THES article:

The basic facts are that Medical Hypotheses - www.elsevier.com/locate/mehy - is explicitly and proudly editorially-reviewed (i.e. by me - not peer reviewed); aims to publish radical and revolutionary scientific ideas; and it is objectively a successful journal. It makes a profit, the Thomson ISI Impact Factor is 1.416 (much better than average, and rising), and I know from internal sources that there are half a million papers downloaded per year - which is equivalent download usage to the prestigious Journal of Theoretical Biology. Clearly, in spite or because of our policy to publish bold and sometimes bizarre ideas, Medical Hypotheses plays a significant role in medical science. Fact; not opinion. The editorial advisory board currently includes such respected figures as Nobelist Arvid Carlsson http://en.wikipedia.org/wiki/Arvid_Carlsson; Sir Roy Calne http://en.wikipedia.org/wiki/Roy_Calne; Antonio Damasio http://en.wikipedia.org/wiki/Antonio_Damasio and V.S. Ramachandran http://en.wikipedia.org/wiki/Vilayanur_S._Ramachandran . Past editorial advisors have included Sir Karl Popper and Nobelist Sir James Black. *** There are only two possible legitimate outcomes to the current process. Either: 1. Medical Hypotheses could continue as an influential, profitable and well-known editorially-reviewed journal with a radical mission. Or else: 2. The journal could be closed-down altogether, and the title abolished. But it would obviously not be ethically acceptable to launch a new ‘imposter’ journal - with utterly different editorial aims, procedures and personnel; yet retaining the 34 year established title of Medical Hypotheses.

As I keep saying, the impact factor of a journal tells you nothing about its quality. For example, here are three peer-reviewed pseudojournals that repeatedly publish abject nonsense and pdeudoscience, with their impact factors according to Journal Citation Reports:

• Homeopathy: 1.041
  

• Evidence-based Complementary and Alternative Medicine: 1.954
  

• Journal of Alternative and Complementary Medicine: 1.628

The articles in these journals are typically written by quacks, and are cited by other quacks writing in quack journals, giving a high-ish but meaningless impact factor. Perhaps Medical Hypotheses is also highly influential among pseudoscientists?

But the main point here is about radical and controversial hypotheses. I think most people would agree that these have their place in scientific discourse, and there ought to be somewhere to publish them. However, this isn't really what the argument is about. In this case, two fatally flawed papers were published with little or no scrutiny: these papers have potential global health implications. In the case of the Duesberg paper, reviews posted on the Denying AIDS blog show the major problems with the paper. There's a difference between publishing provocative ideas that might inspire new research, and ones that are just demonstrably wrong. While the likes of Peter Duesberg have the right to say what they like, they don't have the right to say it in a MEDLINE-indexed journal. This is not an argument about free speech, it's an argument about the integrity of the scientific literature. There may be a place for journals such as Medical Hypotheses, but there has to be some level of quality control. Otherwise, why should anyone take them seriously?

Bad cycling: New Years Day '10'

Not so much bad science, as bad cycling. Last year I returned to bike racing after more than 10 years away, riding a few club events organised by my club, Stockport Clarion. The triumph of my season was my not-too-bad performance in the club hill climb up the Cat and Fiddle. Since then, I've not really been on the bike too much, because it's dark and cold and I'm a bit of a wuss like that. I did have an ill-fated ride before Christmas, where I fell on a patch of ice just before the Great Stone Road roundabout in Stretford, and then almost got run over by the gritter that was following me. But Stockport Clarion traditionally organises a 10-mile time trial on New Years Day, and I thought it would be a good way to kick-start 2010.

It was an 11am start at Chelford, so I reckoned I had to leave about 9:30. This seemed less and less like a good idea as I headed off into the -4 weather, wearing pretty much every piece of bike kit I own. At least the roads were quiet. There was virtually no wind, but riding at about 15 mph was enough to freeze my hands within a few miles, despite my winter gloves. There was a bloke staggering about and yelling as I went through the edge of Didsbury. Hopefully he was on his way home. There was hardly anyone else about. I rattled through a deserted Alderley Edge and made it to the sign-on about 10:45, just as light snow began to fall. Dunc, the timekeeper, had a good story about the previous year's race, when it had also snowed. Apparently a rider behind him noticed that he was leaving two tyre tracks instead of one: he'd twisted his forks in a crash a few days before.

By the time 11:00 came around, there were still only two riders. It was now -3, and the news was that both Snake Pass and the Cat and Fiddle were closed because of drifting snow. One rider from Poynton had taken one look at the thermometer (-6) and decided to go back to bed. Having come this far, I thought we might as well have a ride anyway. Then the perenially late Will arrived, on a full time-trial machine, to make it three riders.

The Chelford course is a bit of a horror, as time trial courses go. But there are just too many traffic lights around Manchester for a selection of courses, so Chelford is what you get. It's hardly a dragstrip, but not really a sporting course either. It's flattish, narrow and typically busy with fast traffic, but being New Years Day it was relatively quiet. The road surface is pretty heavy and there's a couple of lumps to keep you honest, the main one being the railway bridge in Chelford. That probably doesn't sound too bad, but it's amazing how much effort it costs to maintain speed over the damned thing, with only about a mile to go to the finish.

I started number 2, and within a few hundred yards my feet were frozen. There's a roundabout after about a mile, where Ric the marshal was shivering and pacing about to keep warm, and then the turn is just outside Knutsford. By then I'd already been passed by Will at number 3, and was clearly losing time to Dan at number 1. By the time I got to the turn, I was starting to warm up, but my legs just wouldn't have it over the last 4 miles and I finished in 32:53. That's the slowest '10' I've ever done, a minute slower than the first one I ever did riding as a schoolboy under the banner of Penistone Grammar School. I rode back to the timekeeper's car, and took a drink from my bottle: there was ice in it.

Still, the winning time was 28:39, from a rider who would be doing 24 minute rides in summer, so I couldn't feel too bad. In any case, I didn't have time: to avoid hypothermia, I jumped back on the bike and set off home ASAP.

Hopefully there'll be more bad science in the year ahead (there's certainly no shortage of it about), and perhaps even some good science of my own. Whatever else happens, though, I'm certain that there'll be more bad cycling.