Just when one might think the controversy over the Wax et al planned homebirth vs. planned hospital birth study might be dying down, it is instead heating up again.

The American Journal of Gynecology - in which the study was originally published in September, 2010 - released its April 2011 issue last Friday, full of Letters to the Editor criticizing the study. Written by social science, epidemiology, bioethics and public health doctoral university researchers and midwives, it almost seemed as if the journal was preparing to retract the Wax article by these inclusions. But really, they were providing the platform for the study authors to respond to the critical letters, as suggested in an excerpt from one of those responses:

"Although our findings may be unpopular in certain quarters, they result from appropriate rigorous scientific methods that have undergone appropriate peer review."

(More on the "scientific methods" later.)

In the concluding Editor's comment, we are told an (anonymous) independent panel of maternal fetal medicine experts convened to review the data in the Wax study - a move to theoretically decide whether or not the journal should retract the article. The panel's recommendations, following this analysis of the Wax et al data, were that AJOG should publish online, a full summary of the graphs for each outcome included in the study and that no retraction of the article is warranted. (They did admit to finding 'minor' discrepancies in the data - but none that seemed to warrant rescinding the article altogether.)

I will say, I have to give AJOG a little credit here: they didn't have to print those letters. I can only imagine how many letters in a month a large journal like that receives. The managing and chief editors certainly could have kept all those letters to themselves, never to print a single one. Perhaps their PR department convinced them that doing so would have created a larger fire storm than the one (still) brewing.

Coming on the heels of articles in Nature and the Lancet, which seriously called into question the conclusions of the Wax study, there has certainly been a lot of pressure on AJOG to address its decision to publish the article at all.

Medscape is also on board the debate - having already re-published the Nature article and now making available a rebuttal article from a group of researchers - including the authors of the British Columbia and Netherlands homebirth studies the Wax article incorrectly attributed and irresponsibly excluded, respectively. This latest article, entitled Planned Home vs. Hospital Birth: A Meta-Analysis Gone Wrong, by Carl Michal, PhD, et al, was also published last Friday.

An excerpt from the Michal paper encapsulates the problem with the Wax meta-analysis:

The statistical analysis upon which this conclusion was based was deeply flawed, containing many numerical errors, improper inclusion and exclusion of studies, mischaracterization of cited works, and logical impossibilities. In addition, the software tool used for nearly two thirds of the meta-analysis calculations contains serious errors that can dramatically underestimate confidence intervals (CIs), and this resulted in at least 1 spuriously statistically significant result. Despite the publication of statements and commentaries querying the reliability of the findings, this faulty study now forms the evidentiary basis for an American College of Obstetricians and Gynecologists Committee Opinion, meaning that its results are being presented to expectant parents as the state-of-the-art in home birth safety research.

Critical analysis of the Wax study by Michal et al includes the following:
°data included in the study suggests a higher neonatal death rate (for both home and hospital births) compared to perinatal death rates. This, of course, is not possible as neonatal deaths ought to be included in the perinatal death numbers - therefore the data here are paradoxical in nature.

°multiple numerical errors including inconsistencies in data provided in both original and supplemental source papers; incompatibilities between data from cited sources and that which is represented in the Wax study

°inconsistencies in the authors' definitions of perinatal and neonatal mortality

°miscalculations of some odds ratios (OR) and confidence intervals (CI) - sometimes to the tune of drastic underreporting of the CI - resulting in inaccurate statistically significant positive or negative results

°inappropriate data inclusion criteria (such as that for perineal tears, in which the Wax study only included data on first and second degree tears, rather than all perineal lacerations)

°the meta-analysis spreadsheet used to calculate 13 out of the 21 outcomes contained a computational error - making all data computed with that spreadsheet incorrect

°inclusion of reference works that, themselves, have been highly criticized for statistical inaccuracies (such as the Pang et al study that included unplanned home births when it had set out to only assess planned homebirths)

°misappropriated causation:

...the discussion of causes of neonatal mortality focuses on findings from studies that were not included in the meta-analysis, including studies that mix high-risk with low-risk cases. Of the studies that are included in the meta-analysis, none associates rates of intervention with rates of neonatal mortality.

AJOG, in an attempt to rule on the debate of whether or not the data presented in the Wax study are even valid, may have just shot itself in the foot. Since when did 'we published an inaccurate study, and we stand by those inaccuracies' become acceptable?

(see appropriate references attached to above-linked articles)

Published: April 03, 2011

Tags