By Marc Hendrickx writing in ABC’s The Drum
In January 2009, Nature splashed its front cover with the results of a new study titled ‘Warming of the Antarctic ice-sheet surface since the 1957 International Geophysical Year’.
The article was accompanied by a glowing editorial from Nature and was widely reported on in the media.
A very short time after the paper was published, a number of factual errors were found in the paper, along with significant issues with the methodology used to obtain the surprising results. The errors and the methodological problems were reported and discussed by climate change blogs Watts Up With That, The Air Vent, Climate Audit and Real Climate.
Imagine if at this stage Nature’s editor in chief looked at the reported blog commentary and decided the journal had published a paper, which while it had gone through the normal peer review processes, based on some of the blog commentary, was basically fundamentally flawed and should not have been published.
Furthermore, the original reviewers may have shared some of the climate alarm notions of the authors, bringing the veracity of the original review into question. Media coverage also sensationalised aspects of the results. The editor in chief is so embarrassed by the publication of the erroneous paper, he decides to resign.
Sounds farcical? In fact Nature’s editor did not resign. Indeed there was no need to resign, there was no expectation on the part of the scientific community that a resignation was called for, regardless of the issues with the paper.
Subsequently Nature published a correction by the authors that dealt with some of the factual errors. And later, the blog commentary dealing with the methodological problems, ended up being published as a peer reviewed paper, by Ryan O’Donnell, Nicolas Lewis, Steve McIntyre and Jeff Condon, in the Journal of Climate.
Unlike the original paper however, this received very little media attention. Perhaps the long time the paper spent in peer review (10 months) and the less sensational results dulled the media’s interest.
This is just one example of how the peer review system works. Papers are written, reviewed, rejected accepted, acclaimed, criticised, corrected, refuted and debunked. When they are significantly in error they may even be retracted. The process of science, and the reason why it works so well, is because it is one of continual correction and revision. Theory stands until a better theory comes along to replace it. Peer review acts as a general screening tool, but it is by no means perfect, and it is ridiculous to expect it to work perfectly every time.
============================
Read the full article at ABC’s The Drum
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
It is one of the deceits of the Warmists that “peer-reviewed = good, correct” etc. No it doesn’t and its not meant to. Except in the special case of medical publishing, before the Warmists came along with their tendency to change commonly acccepted meanings, what ‘peer reviewed’ did was answer the question, ‘is this paper OK to publish’? And generally the reviewers are in the business of screening out glaring errors, ensuring the author makes a coherent case, is au fait with recent work in the field etc etc. If the paper gets hammered post-publication thats neither their business nor their responsibility: its the author’s work, not the reviewers. Its the Warmists who have quite deliberately given the false impression that reviewers in effect re-do the authors work and, having re-done it, can vouch for its being correct, and that anyone who subsequently disagrees with the paper will have to take them on too, because they’ve ‘guaranteed’ the paper’s OK-ness, haven’t they?. Well, no, because thats not what reviewers do. But it suited the Warmists very well to create the impression, by changing the meaning of peer review, that their work was flawless, guaranteed OK and attacks necessarily came only from cranks and crackpots. And when a contrarian paper popped up which they couldn’t immmediatley rubbish, they made sniffy remarks about ‘standards of peer review’ in whatever journal not being good enough, ie if they and mates had been doing the reviewing, any such paper would have been kept out of the literature. All around Warmism there is a smell of intellectual dishonesty.
It’s an instructive exercise to visit the stacks at a university library and dig out some New Scientists and Scientific Americans from more than a decade ago. By my rough estimate at least 50% of the claims published there have turned out to be wrong, and another 45% have simply been dropped as unrewarding. And these are the findings deemed to be of international importance!
Science is a shotgun. It fires off a couple of hundred pellets, and if you’re lucky one or two of them will leave a mark. But betting the farm on the findings of any one paper — or any dozen or hundred papers — is stark lunacy.
I regularly referee papers in my field and it is a necessary part of modern academic publishing. However the stuff I deal with is rarely contentious. So the system works well at making good papers better and killing the rubbish. It is an OK system. The main gripe is that it can slow down to publication process. However with the use of the internet and working papers, this is not necessarily a hindrance to just getting the new stuff out there.
However when the field is contentious and as politically riven as climate science, then peer reviewers can become blockers of new research as shown in the Remote Sensing case – ok the paper was not blocked but the point is that with hindsight the editor wishes he had blocked it for spurious reasons despite having following the journal’s review policy to the letter. Shame on him.
But we must ask ourselves why do we need peer review. Most scientists would not care about journals and would be happy to just put a pdf online. The reason is that scientists are evaluated by funding bodies on the number of publications/citations they get in journals which are themselves ranked. It is how funding bodies rate their researchers. As long as this is the case we are stuck with this problem. So there is no easy way out.
This is a joke, right? The Drum? No, it’s not April 1st. Must have been an earthquake or something. Wow. Warmists will not be happy to be found out here.
The climategate emails showed that they would “keep it out … if they had to redefine what peer review meant”. We thought the media would get the hint … start investigating the corruption, but no. Now we have a clear public case of that corruption. The concerted pressure put on editors to toe the line, the protracted obfuscation tactics to keep contrary evidence from being published and the obscenely fast track system which allows supposedly pro “evidence” (and I can’t see how it can be described as such) to get published in days.
And this is not just a question of notches on the bedpost. Science now seems to be a “publishocracy”. Those who get published get the kudo to get promotions, get grants, get to be reviewer and editors. Those who don’t end up in dead end careers, not getting promotion, not getting onto the grant bodies and not getting those all important editorships.
We wouldn’t accept a political system whereby the government were able to so skew the electoral system that only they would get on the TV shows, only they would have the funding, etc. so that only one party had any realistic prospect of getting into power. We value debate in democracy, we should also value debate in science.
Science, and particularly climate “science” seems to have lost its way. It’s doesn’t value debate, it doesn’t value diversity. It brooks no contradiction, it ruthlessly pursues those who disagree, and as as I know given the lack of any real science the main occupation of climate “scientists” seem to be running attack sites like (un)realclimate and Wikipedia.
I found the piece well balanced, good reading.
The comments are running 95%:
A. The author is not a CLIMATE SCIENTIST, only a lowly geologist, thus is completely unqualified to speak of such matters.
B. The author is an Anti-Science DENIER (“serial climate change denier”?) thus his words are beneath consideration.
C. S&B was garbage, completely and fatally flawed, the reviewers were all biased, Dessler 2011 in GRL already soundly rebuked it [note: I thought “Dessler 2011” was merely commentary, not a paper], thus the editor in chief of Remote Sensing was right to resign for allowing this travesty to be published. Indeed, it was the only proper thing he could have done!
I’d be taking offense to many of those comments, if they weren’t so laughably predictable.
Oh, I didn’t read all of the comments, too repetitive with too many anonymous names resembling a bot attack rather than actual thinking humans attempting an actual discussion. Likely the final percentage for all comments will be higher than 95%. 😉
This summary misses the point of the article in the Drum. It is specifically about the Spencer/Braswell paper which is not mentioned here.
Just read through comments on the site, this article brought out the hornets, and are lashing out at anything, especially the author. Pass the popcorn, I’ve run out again.
The ABC published this? I’m absolutely stunned. Maybe there is hope for the organisation afterall.
Even the editor of the Drum may be alarmed by the amout of contra evidence to the AGW hypothesis. The world itself is not helping their cause the sea level hypists are having a hard time of it as are the collapsing ice sheet alarmists. JUdge by the audience for the recent Gorathon. Not one mention anywhere in the press here that a. it was on and b. what the results were.
I remember IGY 59. Mind you I was at school but that year of geophysical research did confirm a long held but ridiculed theory. Plate tectonics, or Continental drift as it was then called.
Magnetometer readings across the Atlantic showed a bar code like change in magnetisation of the ocean bed rocks. This proved two things, polar magnetic switching and fact that the Americas and Europe/Africa were slowly drifting apart.
The rest is history as they say.
Can we now get some sanity into ‘climate alarmism’ which is at present riding on the claims that climate is fixed, rather like the old geologists who shouted down the Plate Tectonic theory of continual slow change which is also what climate does without any help from us.
A better title: “Peer review is dead; long live the Climate Clergy”
Kurt in Switzerland
Wow, the cult members are out at The Drum defending their beliefs … not a shred of facts produced.
>> “The errors and the methodological problems were reported and discussed by climate change blogs Watts Up With That, The Air Vent, Climate Audit and Real Climate.”
Uh, I’m confused why the author linked to the Real Climate posting. It didn’t discuss errors or methodology problems, but explained what the results were capable of showing or not showing.
>> “Subsequently Nature published a correction by the authors that dealt with some of the factual errors.”
The Nature correction indicates that the corrected confidence levels did not change the significance of trends nor any of the primary conclusions. I don’t see much in the way of factual errors, just minor mathematical ones.
There are a lot of people who seem not to understand what the phrase ‘peer review’ actually means. What it does not describe is the process of getting a paper published, despite the regrettable tendency of some journals to use the phrase. Getting a paper past a referee is only the very first and least important step in ‘peer review’ , which continues after publication – well after in most cases – as other people working in the field read and comment on the article. So Spencer’s paper is part of the peer review of Dessler’s earlier paper, and Dessler’s reply is part of the review of Spenser’s etc etc. And yes, blogs can be part of the review process, provided that they actually provide facts.
The whole process can take years however, so the current rush to instant judgement (on both sides) is unhealthy. Come back in 2 or 3 years and it may have reached a conclusion.
“Theory stands until a better theory comes along to replace it.”
No, no, no, no, no.
Theory stands until evidence is identified which disagrees with/refutes the theory, there’s no requirement to come up with an alternative at the same time. It’s fine to simply say “We don’t know” and leave space for new theories to develop at their own pace.
Have you forwarded this to the erstwhile Editor of the Remote Sensing journal who resigned recently?
There is after all a new Nature Climate Science journal which will I am sure, be in need of staff in future……….
He could use this as a bit of coercive bribery, couldn’t he??!!
Even when it works, peer review can be expected only to ensure that a paper is worth exposing to wider, general scrutiny. Peer review is not proof of a paper’s veracity, still less its worth. By the same token when peer review fails the worst than can happen is that a worthless paper is offered for wider scrutiny, which is no more than a waste off the readers’ time.
Wasting some time is not a resigning matter unless you make a habit of it.
Streetcred says:
September 21, 2011 at 2:10 am
“Wow, the cult members are out at The Drum defending their beliefs not a shred of facts produced.”
It is highly likely that the majority of the negative commenters at The Drum derive some or all of their incomes from the CAGW climate industry. Climate science is about to collide head on with government budget realities, and the climate elites know it. If they lose public support for CAGW (and make no mistake, they HAVE lost support), then the game is OVER for them.
I remember when that paper was published, the BBC picked it up and raved about the impending doom of humanity. However, included on the BBC page was a link to the authors website which I checked out. The authors explained that the reason they needed the website was to explain what their paper said in plain language, because the media was badly misinterpreting what they had actually written. When you read the authors explanation you realized that the BBC had cherry picked their paper and hyped the most sensational portion only, ignoring the parts which were not as alarmist in nature. It was this particular experience which helped me remain on the skeptic side of the debate. So kudos to the authors for trying to communicate with readers back in 2009, and a big thank you to the BBC for royally screwing up.
cheers
Anyone want to offer odds on which papers will be used by the IPCC? What will be taken as the most appropriate to use in their next outpouring of catastrophe – the original paper, the errors or the corrections.
Science works when it functions as a meritocracy (the best stuff wins) and fails when it slides toward fascism (an authoritarian, governmental, political ideology). Peer-review has become a tool of the failure side of this spectrum, which thankfully, is giving way to technology-empowered, egalitarian popular-review. Yes, long live blog-review.
Scottish Sceptic says:
September 21, 2011 at 1:17 am
Very well said.
Multiscience has a great point. But it does extend to medical publishing. I am a PhD, not an MD, so I have been steeped in scientific epistemology to a much greater extent than most MDs, including those with “MPH.” I find it easier to get published in medical journals than in contingent fields where the researchers have PhDs (epidemiology, psychology, statistics, etc.).
In medicine, people do change definitions and criteria. And, there is the naive adherence to statistical significance to the detriment of respect for clinical significance.
I review papers. No, we reviewers do not get the documentation on the whole study, so we cannot vouch for the validity of the study. IRB can audit the study procedures, but that is abt it.
There are calls for more open-ness in medicine if the data are publically funded. Fed-sponsored resch over 500,000 must somehow make the eventual data available to other researchers, which will clarify a lot of lousy analyses and funny business: overdependence on central tendency while ignoring subgroups, un-reported multiple testing, drop-outs/missing data, etc.
When it is scientifically or politically relevant, I push for more complete info. I obviously can have my views trumped by the other reviewers, but I have been trumped by the editors who simply do not recognize the validity of some of these data/analysis issues.
SteveW is right about theory not being accepted as the end of story until something obviously superior comes along. Any theory is a model, and always has weaknesses. Every study has its weakenesses. A theory can be terribly suspect simply because someone has a decent criticism of it, or because the methods of gathering data have not yet, by design, ruled out some source of bias.
In an election, we have to pick the lesser of two evils, and someone has to “win,” and occupy the position, until a better candidate comes along. Science is not the same.
SteveW says:
September 21, 2011 at 3:00 am
“Theory stands until a better theory comes along to replace it.”
No, no, no, no, no.
Theory stands until evidence is identified which disagrees with/refutes the theory, there’s no requirement to come up with an alternative at the same time. It’s fine to simply say “We don’t know” and leave space for new theories to develop at their own pace.
==================================================================
Thanks Steve, I thought it bore repeating. There is no requirement to present an alternate theory, only to prove or disprove the current one.
To the general discussion, as the author of the opinion piece noted, the “peer-review” process for cli-sci at the least is dead. It has shown to be meaningless. Some papers get published that are of value, but I’m certain that’s entirely accidental.