The row over the issue of Antarctica warming continues. After a number of articles appeared at the Air Vent, Lucia’s, and Climate Audit, Dr. Steig responds at RealClimate with some accusations of his own. I offered Dr. Steig a guest post here, with no caveats, so that he could get maximum exposure, twice. He didn’t bother to respond.
This whole incident illustrates exactly why authors of competing scientific papers should not be reviewers of other papers critical of their own. This failure of peer review falls squarely into the lap of the Journal of Climate for allowing such nonsense in the first place.
But IMHO, Dr. Steig bears responsibility too, he should have said “no”, realizing what a conflict of interest this was.
He confirms in the latest RealClimate essay that he was in fact “Reviewer A”. He also complains that he wasn’t allowed to see the final draft. This is due to the fact that JoC had to bring in another reviewer to break the 88 page log jam created by “Reviewer A”.
The analysis of the difference between the 3rd and 4th (final) drafts at Climate Audit reveal this:
MrPete
Posted Feb 9, 2011 at 10:06 PM | PermalinkHere is a comparison of Rev 3 and Rev 4. All text changes are marked up — including totally minor changes. I hope this works for the reader. (Personally, I would primarily trust this to provide pointers to areas of change as it is not obvious how to reliably discern exactly what the old/new text was.)
To my admittedly inexperienced eyes, the changes appear relatively minor.
Perhaps one of the authors can speak authoritatively on a) whether Wm C’s question (about round 4 reviews) has any standing, and b) whether Eric Steig’s disclaimer (based on not having seen these changes) is appropriate.
So it seems Dr. Steig’s complaint is empty, and the situation mostly a result of his own doings. Still it points back to the failure of peer review at JoC. They should not have invited Dr. Steig to be a reviewer in the first place. had they not, this whole ugly row would be non-existent.
At CA, this commenter sums it up pretty well:
movielib
Posted Feb 9, 2011 at 5:03 PM | PermalinkEric Steig has replied to Ryan:
http://www.realclimate.org/index.php/archives/2011/02/odonnellgate/
There seems to be a lot of arm waving about O’Donnell being wrong about… well, everything.
There is what I’d call a personal attack against “O’Donnell, Condon, and McIntyre,” comparing them unfavorably with such “legitimate, honest commenters” as “Susan Solomon or J. Michael Wallace, or, for that matter, Gavin Schmidt or Mike Mann or myself [i.e. Steig].” You see, he thinks people like O’Donnell and McIntyre are not legitimate honest commenters. The compulsory word “deniers” is also thrown in.
Steig claims O’Donnell is going to “retract [his] allegations” against Steig. It’s very vague and I sure don’t know what he’s talking about.
He says he was a reviewer for the first three drafts of the O’Donnell et al. paper but not for the “markedly different” fourth draft so he hadn’t seen it before publication.
Curiously, Steig does not address the point that is the subject of this thread.
I’ll carry ODonnell’s statement here when he completes it, including making whatever changes/retractions he sees fit.
In the meantime, the Journal of Climate editors should probably be made aware of the mess they created by allowing this conflict of interest to occur in the first place.
The bottom line that has been lost in the fog of this war is that Antarctica isn’t warming as much as is claimed, and most of the statistically significant warming is confined to the peninsula.
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Pielke Sr. regarding Watts not being a referee on Menne et al.:
“I was quite surprised to learn that despite the central role of Anthony Watt’s analysis in the paper, he was not asked to be a referee of the paper. This is inappropriate and suggests the Editor did not provide a balanced review process. ”
http://pielkeclimatesci.wordpress.com/2010/01/15/professional-discourtesy-by-the-national-climate-data-center/
it is completely normal, or expected even, that authors whose paper is being critized are one of the reviewers. They are most familiar with the issues, plus it enables the editor to hear both sides.
Since by his own admission ES is not a statistician, what was he doing writing statistical papers without a statistician co-author in the first place?
Since that paper was show to use poor statistical methods, which invalidate the results, how did it pass peer review?
Who were the reviewers of the original ES paper, and why were the poor statistical methods not highlighted during it’s review?
What the climate science community needs to ask it’s self is, how did a flawed paper make it onto the front page of the global media, and why has it not been retracted already?
Until the original ES paper is retracted by the journal, the journal, and climate science as a whole has zero credibility IMO.
RDCII February 10, 2011 at 1:03 pm
On the issue of understanding of these issues I was amused by the authors’ response to the following third review suggestion from Reviewer A about the need for more information on iridge:
The main thing is that the ‘iridge’ procedure is a bit of a black box, and yet this is now what is emphasized in the manuscript. That’s too bad because it is probably less useful as a ‘teaching’ manuscript than earlier versions. I would love to see O’Donnell et al. discuss in a bit more details (perhaps just a few sentences) how the iridget caclculations actually work, since this is not very well described in the original work of Schneider.
The exasperated response from the authors:
We hold a rather different opinion of which algorithm is a “black box”. Tikhonov regularization (which is called ridge regression primarily in the statistical literature, but Tikhonov regularization elsewhere) has a substantial body of published literature dating back to the 1960s. Much more has been written concerning ridge regression than any other shrinkage estimator of which the present authors are aware. It is a far more common tool in applied mathematics, statistics, and signal / image processing than TTLS.
Schneider’s 2001 paper spends but two paragraphs (page 866) on TTLS in a 12,000+ word article. The remainder of the article is dedicated to EM and ridge regression. We disagree rather strongly that the ridge regression procedure in Schneider (2001) is not well described – it is quite thoroughly described. On the other hand, TTLS is hardly mentioned, and most of the important calculations that appear in the algorithm are not even shown, much less discussed.
We’ve had pal review and closer than pal review. Surely the next logical step is self review.
tonyb
Global temps are dropping like a rock.
Climate charlatans constantly assert that a well documented USA cooling trend is utterly irrelevant for a land mass as small (and, in their “minds”, inconsequential) as the continental USA.
But, the same charlatans want us to get our panties in a bunch over extremely questionable “science” related to a FAR smaller land mass (the Antarctic Peninsula)?
They’re joking, right? If not, then they should meet my challenge.
Perhaps to aid understanding of my February 10, 2011 at 2:09 pm for those not obsessed by this topic, TTLS is the method used in Steig (09) and initially used in the drafts of O’Donnell (2010) until Reviewer A argued that the results from iRidge were more likely and therefore agreed should be reported. TTLS and results were removed from the main paper.
In light of this one has to suspect TTLS was originally used to best replicate what Steig et al had done, rather than any particular commitment to it by O’Donnell et al.
And of course that Reviewer A wasn’t really particularly up with the play.
pat says:
February 10, 2011 at 9:36 am
Nearly right. The canaries were supposed not to sing (or in fact pass out) if levels of toxic gasses rose to a level far below what we could perceive. So in the case of danger, the canary would not sing. If the canary sang, all was OK!
Eric is not just duplicitious, he’s a liar. Take a look at blog entry 84 from RC:
———————————————————————————–
84BPW says:
8 Feb 2011 at 7:44 PM
Did my questions get snipped/moderated and if so, why? I should have kept a copy of my questions in case they failed to make it through. They were perfectly legitimate questions giving you a chance to explain this issue from your side rather than letting folks like myself simply take O’Donnell at his word that you acted in an unprofessional and potentially dishonest manner. Simply asking you to clarify your position.
1) Were you, as he states, one of the referees on his paper?
2) If so, do you think that the conflict of interest that would seem to come from that being the case is meaningful and if not, why not?
3) Did you, as part of your review, ask that they change their method only to later criticize that method?
Like it or not, if you don’t directly address these accusations, the impression lay people are left with is that the whole thing smacks of dishonest use of peer review.
I am not qualified to suss out the science, but i am qualified to understand O’Donnell’s accusations. I am willing to accept that there is something lost and that he is misrepresenting the situation, but if you waive away these type of questions and quash those who try to ask, what are we left to think?
[Response: Perhaps you should try thinking, instead of asking me what to think. Let me turn this question around on you: why do you take O’Donnell at his word? And now he’s my word: His allegations have no basis in fact. Now you have my word against his. Now try thinking,-eric]
———————————————————————————-
The link is here:
http://www.realclimate.org/index.php/archives/2011/02/west-antarctica-still-warming-2/comment-page-2/#comments
Note Eric’s evasive answer. It is a lie by omission. He didn’t actually deny that he was a reviewer but the wording is intended to convince the blogger that he could be trusted (ie, I wasnt a reviewer and my comments are not conflicted).
I tried to post messages to RC seeking clarification because it was clear someone was lying. None of my posts made it past the moderator.
Aside from the fact that nothing Eric says can be relied upon, the folk at RC are behaving as though they are Gods that cannot be questioned.
RC and the Team cannot be trusted until they allow their opinions and their science to be critiqued by people outside their clique.
Sierra17 sez:
“RC and the Team cannot be trusted until they allow their opinions and their science to be critiqued by people outside their clique.”
Hell will freeze before that happens. But, at this rate, that may come sooner rather than later.
There are several points that I feel should be emphasized for clarity.
1. Eric Steig did not write 88 pages of review as some have mistakenly claimed, the 88 pages covers the entire review process including the author’s responses.
2. Eric Steig IS reviewer A, he has publically stated so.
3. Steig did not insist on iridge being used, however he encouraged it, and after it was used commented favourably upon that use in his 3rd review. If he had clear criticisms of the sort subsequently expressed at RC then either he has had a complete change of heart on the matter (which he should explain publicly), or he is being underhand. If he has objections, then he most certainly should have said so as a reviewer, and having not done so as a reviewer he should not do so subsequently without explanation. Simple ethics I suggest.
There are other points at issue, but these are three that seem to be widely misunderstood.
Ryan if report here is true that you intend to apologise for something you said in temper in an email, DONT DO IT. It will be spun and spun and spun. The finer points will be left out and headlines will shriek “Denier apologises for errors” or “Denier retracts attack on legit science” etc. etc. It will be on every AGW blog/newspaper on the planet and will haunt you for years.
WE understand you are a nice guy and nice guys apologise when they do something they see as not nice. THEY on the other hand, are at war with anyone who criticises them and threatens their livelihood, and will use this as another weapon.
Bart Verheggen says:
February 10, 2011 at 2:00 pm
Pielke Sr. regarding Watts not being a referee on Menne et al.:
“it is completely normal, or expected even, that authors whose paper is being critized are one of the reviewers. They are most familiar with the issues, plus it enables the editor to hear both sides.”
IF EVERYBODY KNOWS ABOUT IT AND AGREES TO IT, INCLUDING THE OTHER REVIEWERS. DUH!
If there is one other person in this forum who has worked for an editor of an academic journal, they are doing a fine job of hiding that fact. People are talking about the work of editors as if they existed in Ivory Towers with a staff of 200. Editors live from hand to mouth and are constantly begging for help. They are the hardest working people in academia and they truly sacrifice their lives for their journals. An editor’s journal lives through his editorial practices and they are a direct expression of his personality. If an editor permitted a conflict of interest then he knew about it and he endorsed it. Sorry, it just does not work any other way.
Again, I am responding to a troll. If editors routinely select reviewers who have a conflict of interest then I will eat my hat. That is just plain stupid. It is another example of Ivory Tower thinking. The job of an editor is to ensure that the article is well written, will be of interest to some readers of his journal, and make no blunders that will embarrass the journal. Any editor will tell you that 80% of what is published in journals is worthless. It is published to let people show their stuff and get promotions. Editors do not have the means to determine the quality of the contribution made by an article. To suggest that they do is akin to confusing a lonely little academic office with the Pentagon.
Perhaps I am misreading but in trying to gain understanding, I thought this was significant. It seems clear to me that a change to the paper was made based on a reviewers request and that all reviewers had been made aware of this change in response to that request.
http://www.climateaudit.info/data/odonnell/2%2020100826%20General%20Note%20to%20All%20Reviewers.pdf
General Note to All Reviewers
Based on a request from one of the reviewers, we have agreed to incorporate our “most likely” reconstructions into the main text. These reconstructions do not infill the ground station data using TTLS; instead, they utilize ridge regression. Verification statistics are mildly improved and solution stability is much improved. The smooth regularization and ability to adapt the regularization parameter to the number of predictors in ridge regression proves to be of significant benefit (which was noted as a possibility in Schneider, 2001). Because of this, the TTLS/TSVD reconstructions now serve only to show that cross-validation testing provides a superior means of determining a truncation parameter than the heuristic tool used by S09, and have been relegated to the SI.
We also decided to make additional simplifications for clarity. As using RegEM in a non-standardized (i.e., covariance) mode results in degraded verification statistics and was far more subject to overfitting than using it in the default standardized (i.e., correlation) mode, we feel it adds little to the paper to include the non-standardized infilling operations. This includes the non-standardized ground station infilling and E-W reconstructions. These have been removed.
xyzlatin says:
February 10, 2011 at 3:29 pm
What xyzlatin said. They are a pit of vipers.
Ed Snack says:
February 10, 2011 at 3:19 pm
“1. Eric Steig did not write 88 pages of review as some have mistakenly claimed, the 88 pages covers the entire review process including the author’s responses.”
Let us be clear about this matter. The result of the editorial exchanges was a document that was 88 pages long. Part of it was written by Steig. He began with a review of an 8 page essay that was 24 pages long. Some of the 88 pages was written by O’Donnell and maybe by other reviewers and the journal editor. However, the fact that this exchange went on for 88 pages demonstrates, at least to us who have experience in these matters, that something was seriously amiss. The editor is responsible for permitting this process to get entirely out of control. By the way, in using the word ‘responsible’, I mean that errors were made and a reckoning must be faced, at least a reckoning of conscience. Why haven’t we heard from the editor? He has a lot of questions to answer.
The real trouble is the damage is done, Steig’swork has been BBC news , it’s been on the front page of Nature its entered the dogma of AGW. The fact it’s been proven to be rubbish does no matter unless this proof receives the same exposure, and with gate keepers on these that make Cerberus look like the Andrex puppy can anyone see that happing?
So although they have lost the science argument, and you do need to keep in mind that the ‘Team’ regard it has impossible for them to be wrong, they actual won the more important argument. All that is left now is some ‘pissing into the wind ‘exercises.
jorgekafkazar says:
February 10, 2011 at 11:46 am
What I find so troubling is the “Global Average Fairy.”
Bart Verheggen says:
February 10, 2011 at 2:00 pm
Pielke Sr. regarding Watts not being a referee on Menne et al.:
“it is completely normal, or expected even, that authors whose paper is being critized are one of the reviewers. They are most familiar with the issues, plus it enables the editor to hear both sides.”
Whether or not it was appropriate for Eric Steig to review the paper is one thing, the fact that he criticized the paper at RC without declaring he had been a reviewer was nothing more than deceitful. The article at RC was an attempt to protect his credibility which would have been utterly ineffective had his reviewer status been known when the article was posted.
Moreover, until he had been caught red handed, he acted deceptively to prevent his reviewer status from being revealed (see my post above).
People who lie have something to hide.
When is the Team and RC going to understand that an honest debate cannot be had without transperancy?
How many times do they have to be caught out before the media and our political leaders get it?
When will the moderators at RC stop acting like they are Winston Smith from George Orwells 1984?
sharper00 says:
February 10, 2011 at 11:04 am
@Sam Parsons
“No editor in his right mind would assign as reviewer someone who adamantly believes, and rightly so, that the paper he is reviewing has major negative implications for a paper he has published. “
“The editor is free to completely disregard everything the reviewer produces if they think it has no basis.”
sharper00, why do you quote me and fail to respond to the quote? Do you not understand the concept of “conflict of interest?” Let me explain. Being a reviewer means being an impartial judge. Being an impartial judge means having no personal interest in the fate of the essay that you are judging. However, the essay of O’Donnell’s that Steig judged was a criticism of Steig’s work. Given that fact, Steig should have said to the editor that I have a conflict of interest. If the editor replied that it does not matter, then Steig had a responsibility to ask if O’Donnell agreed. These questions were not answered because O’Donnell did not know that Steig was a reviewer. Therefore Steig had a conflict of interest. So, do you understand now that Steig had a conflict of interest and was morally wrong not to reveal it to all other interested parties? If this practice is common among climate scientists, then all of them are morally wrong, should admit their malfeasance, and accept the punishment. You do not engage in this practice, do you?
KnR says:
February 10, 2011 at 4:18 pm
The real trouble is the damage is done, Steig’swork has been BBC news , it’s been on the front page of Nature its entered the dogma of AGW. The fact it’s been proven to be rubbish does no matter unless this proof receives the same exposure, and with gate keepers on these that make Cerberus look like the Andrex puppy can anyone see that happing?
———————————————————–
Well the BBC and Nature are going to look like right royal idiots by the time this is over.
Steig’s credibility is shot and news of his behaviour will eventually filter out.
Part of it was written by Steig. He began with a review of an 8 page essay that was 24 pages long.
Both incorrect. You haven’t actually read the documents you are describing have you? The ‘essay’ runs to 45 pages and the Steig’s first response was 16. Review is an iterative process and reviews 2 and 3 added another 10 pages. All perfectly unremarkable. Steig was not involved in round 4 which is why he requested a preprint of the final published paper, in case it had been further revised. The only ‘duplicity’ in this affair was in O’Donnell posting the reviews online after he had explicitly given his word he would not do so.
http://www.climateaudit.info/data/odonnell/1%2020100209%20Submission.pdf
Be thankful for Realclimate
I have a question. Tamino has a knack for defending the math of team members from skeptics/deniers. How many have defended Steig’s results mathematically since the Corrigendum? Perhaps Tamino would like to weigh in on the appropriateness of the methods used in the two papers.
There is very little Steig can say about this matter to remove the egg from his face and re-establish any shred of credibility he thought he had. He has been well and truly discredited as a modern snake-oil salesman…
From the peeps at ‘the other place’:
…I’d describe it as “tantric peer review”….