Thursday, 15 May 2008

Nadine Dorries makes an attempt to look at the evidence. She fails. [updated]

Nadine Dorries seems to have realised that simply dismissing as 'desperate tosh' a study in the British Medical Journal won't cut the mustard. The study looked at survival rates of premature babies in the Trent region of the UK, concluding that survival rates for births before 24 weeks had not increased since 1994-9. This undermines the argument that those survival rates are increasing, and the time limit for abortion should be brought down to 20 weeks from the current 24 weeks. Dorries has now at least attempted to address the evidence, on the 20 weeks campaign website. This is a marked improvement, but I think her effort still falls somewhat short.

First, some nonsense. Dorries writes:

"We were somewhat surprised to see that the article was not in this week's paper edition of the BMJ which many doctors did not receive until Saturday. Why then rush to get it onto the web and into the public domain? The BMJ editorial said that upper time limits were to be debated in the House of Commons 'this year'. What it didn't say is that they are actually to be debated and voted on in just ten days, on 20 May. So the rush to get this into the public domain was to influence that debate".

If you can't use the latest research to 'influence debate', then what is it for? Apart from that, it is perfectly normal for research articles to appear on journal websites some time before they appear in print. That is precisely so that articles can reach the public domain once they are ready for publication, without having to wait on production schedules.

What of the science, indeed. Dorries says that "the Trent study looks at results from 16 hospitals and has been running for years. It is not new and other studies have been published based on more recent data". She also claims that the results from Trent "have always been poor and well below those seen in top neonatal centres worldwide". As evidence for this, she cites a study from the University College London Hospital (UCLH). This study is a single centre cohort study comparing survival rates in the years 1981-85, 1986-90, 1991-95, and 1996-2000. [The first thing to note here is that the Trent study uses data up to 2005. So it is actually using much more recent data than the UCLH study.] The UCLH study found that "There was a progressive increase in survival at all gestational ages over the 20-year period".

This seems to be a classic case of how you have to be careful with research. The first thing to note is that the number of babies in the UCLH study is small. The total number of babies in the study was 12 for those born at 22 weeks gestation, and 56 for those born at 23 weeks gestation, over 20 years. This compares with 261 born at 22 weeks and 370 at 23 weeks, over 10 years, for the Trent study. The Trent study is likely to give more robust numbers. Dorries, on the other hand, writes that the UCLH study "showed no survivors at 22 or 23 weeks in 1981-85 but 71% (5/7) and 47% (8/17) respectively in 1996-2000". Note the tiny sample size this statement is based on. In the UCLH study, the authors don't give percentage survival rates for births at 22 weeks, because the sample sizes are simply too small to draw any conclusions. There is something odd about the admission figures here too. For births at 22 weeks gestation, there was 1 admission in 1981-85, 3 in 1986-90, 1 in 1991-95, and 7 in 1996-2000. The comparable figures for births at 23 weeks are 7, 23, 9 and 17. There are very large variations here that suggest that the different time periods may not be directly comparable.

Perhaps the most serious problem with comparing the Trent and UCLH studies, though, is that they don't appear to be measuring the same things. In the Trent study, the authors write "We included in the study infants alive at the onset of labour". In contrast, the UCLH authors write "All infants born between 22 and 25+6 weeks of gestation between 1981 and 2000, who were admitted to the tertiary neonatal intensive care unit of UCLH (UK), within 1 week of birth, were enrolled into our study. In addition, the UCLH labour records for the years 1991–2000 were scrutinised to identify all infants who were born alive between 22 and 25+6 weeks of gestation, but who died in the delivery room". Hence the Trent study includes still-births as deaths, but the UCLH study does not. According to the Trent study, of births at 22 weeks, 43% were stillborn in 1994-99, and 40% in 2000-5. Of births at 23 weeks, 28% were stillborn in 1994-99, and 20% in 2000-5. For the UCLH study we don't know what these rates are. The Ministry of Truth blog has noted an anomaly in that the UCLH study reports a massively lower proportion of deaths in the delivery room compared to the Trent study. So there is at least a possibility that the UCLH survival rates are inflated in comparison to the Trent study by selection bias: perhaps more infants are being admitted to neonatal intensive care in Trent than in UCLH.

It's also worth noting that the Trent study covered a whole area (the Trent health region in the UK), while the UCLH study covered a single hospital. As the authors of the UCLH study write, "Large population-based studies [such as the Trent study] have the advantage of avoiding problems of selection bias and enable the accumulation of large numbers to reduce statistical errors. However, they inevitably represent the outcome from a very heterogeneous range of perinatal facilities...It is therefore inevitable that these units encompassed a wide range of staffing, resources and expertise in the care of extremely preterm infants. In addition, it seems likely that the maternity units encompassed a range of management policies (both obstetric and paediatric) towards infants born at the limits of viability, ranging from ‘proactive’ to ‘noninterventionist’. Hence, it is not surprising that average survival figures obtained from large population based studies will be different from those observed from single centre cohorts". As the editorial in the BMJ that accompanies the Trent studies says, "Single centre studies are confounded by selection bias and tend to overestimate the likelihood of survival". In other words, to compare the Trent and UCLH studies in the way Dorries is trying to do, and draw the conclusion that survival rates in the Trent health region are 'poor', is not a sensible thing to do.

So Dorries has looked at the evidence, which is better than dismissing it as 'tosh'. However, she seems to be drawing erroneous conclusions from an inappropriate comparison of two different studies. By coincidence, this inappropriate comparison supports Dorrie's agenda. How convenient.

8 comments:

Holfordwatch said...

It's good to see that Dorries has people like you following up her warblings and making an appropriate analysis of their 'quality'.

Maybe we might find it in our hearts to rejoice that Prof Wyatt finally made up his mind about the precise status of the infants that he was discussing.

Dr Aust said...

Nice work, Paul.

As ever, one is left wondering whether Mad Nad simply can't tell the difference (fool) or can but just insists on pressing the figures that support her own pre-ordained point of view (liar).

The Riley et al. (Wyatt) Acta Paed paper itself is fairly restrained in how it states things. Apart from the points you have already identified, a couple of other interesting sentences from the paper:

"A limitation of our study is that the outcome was assessed at a corrected age of 1 year and hence it is likely that the true incidence of neurodisability in the cohort was underestimated... ...[In a previous study from our group ] about half of children who showed impairment without disability subsequently required extra educational provision at the age of 8"

Indeed, from reading the paper one would hardly recognise the thing that got all that hysterical over-egged coverage in the media last Autumn.

One does rather wonder whether the difference lies in Prof Wyatt having to stick to the facts and some reasonable inferences in a real referee-d paper, while in his evidence to the Select Ctte, and his work for the Christian Medical Fellowship, he gets to exercise his, erm, "opinion muscles " far more.

pj said...

Why 'fool' or 'liar'? I think Dorries can be both. She doesn't care about scientific evidence because she is simply seeking to bolster her pre-formed view ('liar') but she isn't bright enough to understand the evidence sufficiently to make any sensible objections anyway ('fool').

Interestingly, if you believe that pre-term neonatal survival should inform the abortion debate, the relevant survival figures probably are those related to live births rather than including still-births (since aborted foetuses are presumably alive before they are aborted).

pj said...

When I had a brief look at the published literature, I found that essentially 100% of 22wk births that survived had major disabilities.

I notice she lists these reasons for lowering the limit:

"3D ultrasounds of babies smiling, yawning, stretching, walking in the womb and scratching their noses well below 20 weeks on television or seen their own babies on ultrasound in the prenatal clinic"

That's right, smiling - funny how newborns can't smile isn't it, think they forgot how? And I wonder how these foetuses scratch their noses with no connections between their cortex and sensory fibres? Just maybe 4D ultrasound tells us fuck all?

"Professor Anand's work showing that fetuses have the capacity to feel pain down to 18 weeks"

This is complete bullshit, Anand has published absolutely no research relating to this question, and has simply asserted it as fact.

Paul Wilson said...

Thanks all for your comments.

I should point out that I've made an error here (another reason why you need to be careful with research). The 18% and 19% figures from the Trent study are actually percentages of those infants born at 23 weeks who were admitted to the neonatal intensive care unit. So they should be comparable to the figures from the UCLH study, which are also percentages of those admitted to the NICU. But there is still a question of inclusion bias.

If you have a look here, there is an anomaly in that the UCLH study seems to have a massively lower proportion of deaths in the delivery room compared to the Trent study.

Paul Wilson said...

pj:

I think the distinction between 'lying' and 'bullshit' is a useful one here: see the Quackometer for an interesting discussion.

Paul Wilson said...

"Interestingly, if you believe that pre-term neonatal survival should inform the abortion debate, the relevant survival figures probably are those related to live births rather than including still-births (since aborted foetuses are presumably alive before they are aborted)."

That is certainly one way of looking at it. Another way of looking at it is that still-births are telling you something about 'viability', if that is what we're arguing about, so they should be included.

pj said...

I'm not sure that's true - because being born premature is normally a sign of something bad going on.

Most normal foetuses will never be born at 22-23wks because, by definition, they will be born at term.