Peer review is dead, long live blog review

By Marc Hendrickx writing in ABC’s The Drum

In January 2009, Nature splashed its front cover with the results of a new study titled ‘Warming of the Antarctic ice-sheet surface since the 1957 International Geophysical Year’.

The article was accompanied by a glowing editorial from Nature and was widely reported on in the media.

A very short time after the paper was published, a number of factual errors were found in the paper, along with significant issues with the methodology used to obtain the surprising results. The errors and the methodological problems were reported and discussed by climate change blogs Watts Up With That, The Air Vent, Climate Audit and Real Climate.

Imagine if at this stage Nature’s editor in chief looked at the reported blog commentary and decided the journal had published a paper, which while it had gone through the normal peer review processes, based on some of the blog commentary, was basically fundamentally flawed and should not have been published.

Furthermore, the original reviewers may have shared some of the climate alarm notions of the authors, bringing the veracity of the original review into question. Media coverage also sensationalised aspects of the results. The editor in chief is so embarrassed by the publication of the erroneous paper, he decides to resign.

Sounds farcical? In fact Nature’s editor did not resign. Indeed there was no need to resign, there was no expectation on the part of the scientific community that a resignation was called for, regardless of the issues with the paper.

Subsequently Nature published a correction by the authors that dealt with some of the factual errors. And later, the blog commentary dealing with the methodological problems, ended up being published as a peer reviewed paper, by Ryan O’Donnell, Nicolas Lewis, Steve McIntyre and Jeff Condon, in the Journal of Climate.

Unlike the original paper however, this received very little media attention. Perhaps the long time the paper spent in peer review (10 months) and the less sensational results dulled the media’s interest.

This is just one example of how the peer review system works. Papers are written, reviewed, rejected accepted, acclaimed, criticised, corrected, refuted and debunked. When they are significantly in error they may even be retracted. The process of science, and the reason why it works so well, is because it is one of continual correction and revision. Theory stands until a better theory comes along to replace it. Peer review acts as a general screening tool, but it is by no means perfect, and it is ridiculous to expect it to work perfectly every time.

============================

Read the full article at ABC’s The Drum