The row over the issue of Antarctica warming continues. After a number of articles appeared at the Air Vent, Lucia’s, and Climate Audit, Dr. Steig responds at RealClimate with some accusations of his own. I offered Dr. Steig a guest post here, with no caveats, so that he could get maximum exposure, twice. He didn’t bother to respond.
This whole incident illustrates exactly why authors of competing scientific papers should not be reviewers of other papers critical of their own. This failure of peer review falls squarely into the lap of the Journal of Climate for allowing such nonsense in the first place.
But IMHO, Dr. Steig bears responsibility too, he should have said “no”, realizing what a conflict of interest this was.
He confirms in the latest RealClimate essay that he was in fact “Reviewer A”. He also complains that he wasn’t allowed to see the final draft. This is due to the fact that JoC had to bring in another reviewer to break the 88 page log jam created by “Reviewer A”.
The analysis of the difference between the 3rd and 4th (final) drafts at Climate Audit reveal this:
MrPete
Posted Feb 9, 2011 at 10:06 PM | PermalinkHere is a comparison of Rev 3 and Rev 4. All text changes are marked up — including totally minor changes. I hope this works for the reader. (Personally, I would primarily trust this to provide pointers to areas of change as it is not obvious how to reliably discern exactly what the old/new text was.)
To my admittedly inexperienced eyes, the changes appear relatively minor.
Perhaps one of the authors can speak authoritatively on a) whether Wm C’s question (about round 4 reviews) has any standing, and b) whether Eric Steig’s disclaimer (based on not having seen these changes) is appropriate.
So it seems Dr. Steig’s complaint is empty, and the situation mostly a result of his own doings. Still it points back to the failure of peer review at JoC. They should not have invited Dr. Steig to be a reviewer in the first place. had they not, this whole ugly row would be non-existent.
At CA, this commenter sums it up pretty well:
movielib
Posted Feb 9, 2011 at 5:03 PM | PermalinkEric Steig has replied to Ryan:
http://www.realclimate.org/index.php/archives/2011/02/odonnellgate/
There seems to be a lot of arm waving about O’Donnell being wrong about… well, everything.
There is what I’d call a personal attack against “O’Donnell, Condon, and McIntyre,” comparing them unfavorably with such “legitimate, honest commenters” as “Susan Solomon or J. Michael Wallace, or, for that matter, Gavin Schmidt or Mike Mann or myself [i.e. Steig].” You see, he thinks people like O’Donnell and McIntyre are not legitimate honest commenters. The compulsory word “deniers” is also thrown in.
Steig claims O’Donnell is going to “retract [his] allegations” against Steig. It’s very vague and I sure don’t know what he’s talking about.
He says he was a reviewer for the first three drafts of the O’Donnell et al. paper but not for the “markedly different” fourth draft so he hadn’t seen it before publication.
Curiously, Steig does not address the point that is the subject of this thread.
I’ll carry ODonnell’s statement here when he completes it, including making whatever changes/retractions he sees fit.
In the meantime, the Journal of Climate editors should probably be made aware of the mess they created by allowing this conflict of interest to occur in the first place.
The bottom line that has been lost in the fog of this war is that Antarctica isn’t warming as much as is claimed, and most of the statistically significant warming is confined to the peninsula.
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
@ur momisugly sharper00
” sharper00 says:
February 10, 2011 at 10:51 am
@ur momisuglyJames Sexton
“I’ve already quoted it twice now, if you’re going to rest your claim on the supposed strength of “Reviewer A”‘s recommendation then you have no argument.”
======================================================
That’s horse $hit. Ok, insisted or recommended or suggested…….however you want to frame it.
Read this and tell me what it means to you….“Yet at least two independent groups who have tested the performance of RegEM with iridge have found that it is prone to the underestimation of trends, given sparse and noisy data (e.g. Mann et al, 2007a, Mann et al., 2007b, Smerdon and Kaplan, 2007) and this is precisely why more recent work has favored the use of TTLS, rather than iridge, as the regularization method in RegEM in such situations. It is not surprising that O’Donnell et al (2010), by using iridge, do indeed appear to have dramatically underestimated long-term trends—the Byrd comparison leaves no other possible conclusion…..”
Later…….“The choice of kgnd that yields the best agreement with the iridge calculations (which, remember, is already known to create problems) happens to be kgnd = 7, and it just so happens that this yields the minimum trends……”
========================================================
So, did he know his suggestion would lead him to state this later? Or did he have an epiphany after he was done reviewing? Because if he “knew” his suggestion carried these inherent criticisms, then it would be entirely disingenuous to suggest it, without so much as a warning about its use.
Duplicity.
sharper00 says:
February 10, 2011 at 11:31 am
Calling it “an effort to exert influence” seems quite a stretch since the more obvious he makes his attachment then the less likely the editor is to accept it.
_______________________________
So, now the editor picked a reviewer he felt had a problematic attachment? Why on earth would he pick a man he knew to be potentially biased….UNLESS he shares his bias, maybe?
First you imply that you see nothing wrong with Steig being a reviewer. Then you point out the editor’s knowledge of a “problematic attachment” which could make his review biased. I hope you’re not a criminal investigator.
Sir Arthur Conan Doyle believed in fairies. When others claimed to see fairies, he believed them. He thought the Cottingley fake photographs likely showed fairies.
The Team believe that adding CO2 to the atmosphere increases the greenhouse effect, producing higher temperatures. Since there’s no way to detect that process other than indirect measurements, they pursue novel methods to tease a warming signal out of some of the noisiest data on the planet. If those methods don’t show warming, they tweak the methods until they do. If the data don’t show warming, they tweak the data until they do. There simply must be fairies!
They are so certain there are fairies, they see fairies wherever they look. If you don’t see the fairies, they’ll provide the Cottingley photographs, and the printout from their Global Fairy Models, too. Do you believe in fairies yet?
Secondly sharper00
Eric then reads the iRidge version and has THIS to say about it
“As expected from this, the new method that O’Donnell et al. now
emphasize — using their implementation of the ‘regem’ algorithm with individual
ridge regressions for each data point, and hence individual values of kgnd and
ksat being used as well, gives results in much better agreement with lower values
of k”
and
“* The use of the ‘iridge’ procedure makes sense to me, and I suspect it really
does give the best results.”
and
“The main thing is that the ‘iridge’
procedure is a bit of a black box, and yet this is now what is emphasized in the
manuscript. That’s too bad because it is probably less useful as a ‘teaching’
manuscript than earlier versions. I would love to see O’Donnell et al. discuss in a
bit more details (perhaps just a few sentences) how the iridget caclculations
actually work, since this is not very well described in the original work of
Schneider. This is just a suggestion to the authors, and I do not feel strongly that
they should be held to it.” < here not saying to toss it out, but to beter explain it so it can be used to teach!
And then….then after writing that he goes onto RC and kills them for using it.
Open and shut case. Duplicity. If he had a problem with it, maybe he should have said so as a reviewer! The fourth version didn't change the iridge calculations so he can't claim that it is now the reason he doesn't like it.
Although this ongoing controversy has provided some valuable insights into the many flaws in the peer reviewed publication process for scientific literature, it has seemed to me from the beginning to be a colossal waste of time and effort. The temperature data from Antarctica has always been so sparse and unreliable that even if, by some unimaginable miracle, you could get the entire statistical community to agree on a set of “best practices” on how to deal with it, you still couldn’t produce anything more meaningful than a statement that Antarctica is colder than a Wiccan’s mammary and is likely to stay that way, no matter what global mean temperatures do in the next couple of centuries.
As I tried to point out when Steig’s work first surfaced, this effort is analogous to trying to construct long term temperature trends for the continentalU.S. by taking data from stations in the Florida Keys and the Everglades, combining it with data for the other 47 states from a handful of randomly scattered stations which were some of the worst rated in Anthony’s surface station survey, and producing a nicely scarlet swathed map of the lower 48. The only real difference is that Antarctica is actually twice the size of the lower 48.
I realize that dedicated number crunchers have a psychological need to believe in the efficacy of their efforts. A realistic assessment of the true uncertainties of their efforts would likely subject them to profound existential doubts about the meaningfulness of their life’s work. The rest of us shouldn’t feel compelled to share their delusions.
Forgot to add a further link…
http://ts1.mm.bing.net/images/thumbnail.aspx?q=410822575592&id=afbdac3312f35edcace1030cb84970ab&url=http://i285.photobucket.com/albums/ll57/mm53054/Forum/fail.jpg
“Goodness” indeed!
Hasn’t it occurred to you that not one of your contributions would ever have seen the light of day at the temple of your oracles – Real Climate.
As somebody once said – “Goodness has nothing to do with it”
Some wrong facts in circulation here. From the OP:
“This is due to the fact that JoC had to bring in another reviewer to break the 88 page log jam created by “Reviewer A”.”
First the 88 pages furphy. Reviewer A wrote three reviews. The first was 14 pages, the second 6 and the third 4.
JoC did not bring in reviewer D to break a logjam. They brought him in after the first round of reviews, explicitly in response to a complaint from Ryan. Broccoli said:
“To allay some of the concerns you have expressed about Rev. A, I have sought the advice of an additional reviewer (Rev. D). Please note that several of Rev. D’s comments are similar to points made by Rev. A and thus warrant especially close attention.”
Obviously, Steig was a poor choice of reviewer as he doesn’t have the necessary expertise in the field(s) in question (i.e. statistics and methodology). Further, in the performance of his duties as reviewer he displayed a greater allegiance to protecting his reputation than actually engaging with the submitted material.
All in all its a rather amateurish outing for Prof. Steig.
Steig says ” I’m not a statistician “. That’s the only true thing he has said in this whole sordid saga.
So a self confessed non-statistician reviews a paper which is criticial of his crappy statistical methods. That itself is a dishonourable and fraudulent act.
And then he goes on to further show his idiocy by having 88 pages of obstructive review, 10 times more than the paper.
And then he goes and posts crap in RC and gets ripped apart more in detail about his statistical methods. To think that this guy is a professor!!! God help his students, seeing his knowledge and ethics.
@AJ Abrams
“Because you can copy and paste what is over at RC does NOT mean you read the reviews, or at the least you are cherry picking”
Er ok but you just quoted precisely the same section I did. Is your one better somehow? So the recommendation starts with
“My recommendation is that the editor insist that results showing the ‘mostly likely’ West Antarctic trends be shown in place of Figure 3.”
And then later (third time I’ve quoted this now)
“Perhaps, as the authors suggest, kgnd should not be used at all, but the results from the ‘iridge’ infilling should be used instead.”
Does the “insist” from several sentences previously somehow leak into this one? The use of iridge and Steig’s subsequent criticism of it is the basis for the “duplicity” charge.
Maybe O’Donnell interpreted it as insistence but then I already pointed out in my first comment that this is a misunderstanding, not a failure of peer review.
“So smarty..if he insists they replace figure three with something else, but says not to use a different K_gnd number in place of 7, then says hey why not iRidge you mentioned..how on THIS earth is he not insisting on iRidge given he’s given NO OTHER CHOICES here????”
It’s not the job of the reviewer to write the paper! He identifies problems with the figure, wants it replaced and offers suggestions about what could be used instead. It’s upto the authors to replace it with something else.
Now this apparent crisis in peer review apparently boils down to how forcefully you can interpret “Perhaps, as the authors suggest, kgnd should not be used at all, but the results from the ‘iridge’ infilling should be used instead.”.
This is an extremely weak argument.
It would appear that EVERY “peer-reviewed” science article (and reply/notes/correction/addendum published thereafter) needs an anonymous reviewer cycle as often done today – but only during the reviewing process.
Once accepted for publication, every article (or correction or note) should be printed WITH the reviewers’ names and title included immediately below the authors’ names and titles.
Peer-reviewing without attribution leads to four failures:
1) The hours and days spent by the reviewer are NOT visible to his peers, to his students or fellow authors at his own institute, and to his bosses and department heads. No visibility, no recognition by the reviewers’ peers = No rewards for being a credible and honest reviewer.
2) An anonymous reviewer is protected by his/her very anonymity. Protected against correction or exposure that is – the writer of an unpopular piece is “stuck” flailing against Galileo’s accusers in the “church’s authorities” = “The Paper Inquisition” but cannot defend himself/herself against those who bring their charges of witchcraft and heresy. The Paper Inquisition are protected against THEIR abuses and THEIR own prejudices and THEIR (deliberate and incidental) errors because no one knows who is behind the charges.
3) A person who has performed dozens (or hundreds) of reviews is more powerful, more effective than ten writers/one hundred writers who have submitted one paper each. That reviewer MUST be known and their reputation established and enhanced. Or that reputation removed, if the reviewer’s prejudices are known.
4) The EDITOR who is shown to blindly pass too many articles to only a “Chosen Few” in his Paper Inquisition will lose his power and HIS ability to throttle scientific investigation. Papers and Journals who support a Paper Inquisition by THEIR selection and prejudices inside the review process need flushing and exposure to the light too!
jorgekafkazar,
Fine post. I’m tempted to plagiarize it!☺
sharper00 says:
“I already quoted the actual recommendation and you’re telling me to get current.
Here, again, is the “strong recommendation”:
“Perhaps, as the authors suggest, kgnd should not be used at all, but the results from the ‘iridge’ infilling should be used instead. “”
You are absolutely correct that this politely worded suggestion was in the 88 page review. However, as you also quoted without highlighting, it was in a paragraph where the opening sentence is:
“My recommendation is that the editor insist that results showing the ‘mostly
likely’ West Antarctic trends be shown in place of Figure 3.”
I would suggest that recommending “that the editor insist” on a change can be categorized as a “strong recommendation”.
Couple of factual points.
1. Steig / Reviewer A wrote 24 pages of review, an utterly proportionate amount, the ’88 pages’ comes when you include the author’s reply.
2. Montford’s ‘Jesus Paper’ essay, a putative demolition of peer-review, is hilarious. He rants on (and on) about the occlusion of r-squared stats:
And he therefore now knew that Wahl and Amman’s work suffered from exactly the same problem as the hockey stick itself: the R2 number was so low as to suggest that the hockey stick had no meaning at all, […] McIntyre’s first action as a peer reviewer was therefore to request from Wahl and Amman the verification statistics for their replication of the stick. Confirmation that the R2 was close to zero would strike a serious blow at Wahl and Amman’s work. [….]. (etc etc,)
You’d think someone wanted to be regarded as credible would have checked what the NAS panel on paleo-reconstructions had to say about the usefulness of r-squared in the context of assessing proxy skill.
They said ‘it is not in itself a useful indication of merit’ (Page 93).
Not in itself a useful indicator of merit. Take away this meat from the Bish’s flight of fancy and you’re left with a thin soup indeed….
I’m still pretty sure sharper00 hasn’t read (or understood) http://bishophill.squarespace.com/blog/2008/8/11/caspar-and-the-jesus-paper.html
Am I right, sharper00? Or do you just toss reality away?
Colin K at Bishop Hill got it:
“Surely Eric Steig has admitted defeat:
– He concedes he might be wrong;
– He states that the “technical aspects of the methodology are completely legitimate subjects of discussion” but doesn’t go on to discuss and defend them;
– Earlier in his post he agrees “I am not a statistician”;
– Since the entire basis of the paper is a statistical analysis of data that supposedly demonstrates alarming warming across wide areas of Antarctica, but the author agrees he’s not a statistician and might be wrong, why should anyone attach any weight to this particular piece of work?”
ColinK
That’s the meat in Steig’s post: “With respect to O’Donnell’s lengthy discussion of the technical aspects of the difference between our papers, I’m not complaining. ”
The rest is noise.
Richard Horton, editor of the British medical journal The Lancet, has said that
“The mistake, of course, is to have thought that peer review was any more than a crude means of discovering the acceptability — not the validity — of a new finding. Editors and scientists alike insist on the pivotal importance of peer review. We portray peer review to the public as a quasi-sacred process that helps to make science our most objective truth teller. But we know that the system of peer review is biased, unjust, unaccountable, incomplete, easily fixed, often insulting, usually ignorant, occasionally foolish, and frequently wrong.”
http://en.wikipedia.org/wiki/Peer_review#Criticisms_of_peer_review
I’ve not read through all the comments to see if this is old news, but according to Andrew Revkin, O’Donnells planning to apologize. Distressing if so…From D.E. below
“That was then. Civility evaporated in a series of blog posts on Realclimate and Climateaudit that crested a few days ago when O’Donnell lobbed a heap of accusations against Steig. (O’Donnell has, in e-mail exchanges between the combatants that I’ve been copied on, said he recants the worst of them and plans to post an apology.)”
The reason for banning the trolls is that they are here to create obfuscation and for no other reason. They and “The Team” want to be able to point to this site and say that many people disagree with claims such as the claim that Steig knowingly and deceitfully engaged in a conflict of interest that was blessed by the journal editor. Not one of the trolls will debate me on that claim. They should be banned.
So…Eric’s reply, “Some thoughts on Personal Responsibility and the Peer Review Process”.
Firstly:
“First, I never suggested to the authors that they use ‘iridge’.”
Yes, Steig unquestionably did, in one of the reviews:
“Perhaps, as the authors suggest, kgnd should not be used at all, but the results from the ‘iridge’ infilling should be used instead.”
“Perhaps” makes it a suggestion. It’s not debatable.
What Steig could have said that would make his first statement above a truth is that he did not raise the idea in the first place, but encouraged it. But what Steig says is that he never suggested they use iridge, and that’s patently untrue.
Steig’s assertion is to try to avoid the duplicity charge. The duplicity charge does not actually come from his criticizing the choice of the iridge procedure; he could have, in fact, changed his mind in the interim. People are allowed to do that. BUT…having made the suggestion in “secret”, he would have a moral responsibility, when going public with his criticism, to ‘fess up to the fact that he influenced this choice in the first place; to publicly state that, at the time, he suspected it did provide better results and, for that reason, he recommended it.
That’s where the duplicity comes in; he influenced that choice in “secret”, and then criticized it in public without revealing his secret influence.
If Steig really does not see this as duplicitious, then it does highlight a flaw of the Peer Review process. If scientists really believe that it’s ethical to influence changes to a paper in secret, then later criticize those changes, then the Peer Review process is flawed.
Secondly, if scientists really believe this next Steig quote:
“But once they have signed off on the paper, it is their paper, and blaming someone else — reviewer or editor — for its content is simply passing the buck.”
…the Peer Review process again has a serious problem.
The way the Peer Review process works, the reviewer becomes a co-writer. To the extent that their suggestions are ignored, the reviewer isn’t responsible; to the extent that their suggestions are accepted and the paper is changed, they become co-responsible. If scientists don’t get this, if scientists are not willing to take personal responsibility for the effects of their reviews, then the Peer Review process is again hopelessly flawed.
Thirdly, a conflict of interest is not inherently a problem. A SECRET conflict of interest is, indeed, a problem. It doesn’t help that the editor knows, or makes the final decision, if, when the reviewer makes public criticisms, the public doesn’t know. I think the editor/publisher enabled this tempest with a poor choice.
And lastly, Steig admits he’s not a statistician. Think about that for a minute. Let that bounce around inside the head for a bit.
If he’s not qualified to be a statistician…what is he doing writing S09, which defined and applied new statistical techniques? Why is someone asking him to review Ryan’s paper, if he’s not qualified to understand the stats? What is he doing recommending one technique over another, especially when, as a “secret” person, the authors have no way to determine whether his advice comes from an amateur statistician or an expert? And lastly, what is he doing criticizing iridge at all?
It really, really bothers me that so many of these papers are based on Statistical analysis without there having been a statitician involved. If there’s no other reason to suspect that AGW is a paper boat, it’s this: these Scientists produce their “Science” using tools they don’t understand, and they really resent it when someone notices.
Ryan, if you really do intend to “retract allegations”, think carefully. The only thing I can see to retract is the assertion that Eric asked for a copy of the paper when in fact he had already seen it…if Eric didn’t see the final draft, then he wasn’t being manipulative. But that’s it; Eric’s outrage is unfounded. Eric was duplicitious, as in two-faced, recommending “X” over here as secret persona A and attacking “X” over there as public persona B.
However, all that said…it never pays to lose your temper. A good public relations firm (and I’m convinced that Eric’s response was written mostly by a public relations firm; it’s just too good) can work it against you.
And thanks for all your hard work, and for bearing the frustrations inherent in getting anything “skeptical” published. You have no idea how much I admire you folks.
>>sharper00 says:
February 10, 2011 at 11:50 am
And then later (third time I’ve quoted this now)
“Perhaps, as the authors suggest, kgnd should not be used at all, but the results from the ‘iridge’ infilling should be used instead.”
Does the “insist” from several sentences previously somehow leak into this one? The use of iridge and Steig’s subsequent criticism of it is the basis for the “duplicity” charge. <<
Even taking the mildest meaning from your above quote, it says that Reviewer A found iridge to be acceptable, and voiced that back to the author via his review. To then come out and criticize the author for using it is clearly duplicity.
Would it perhaps be wise…. given the trillions riding on all this…
To invest in a number of state of the art (designed specifically for the harsh antartica ennvironment) automatic WEATHER STATIONS, to be spread across the continent and interior of Antartica…
It may well cost 100 million, 200 million….(it may well offer serious engineering challenges), would it not be money well spent, after all this is (maybe/maybe not) the biggest crisis that man-kind faces (sarc off)
But using statistics to measure temperature, where there are no thermomers is not the best approach?
My response is best typified by the quote below.
“Oh what a tangled web we weave,
When first we practise to deceive!”
Sir Walter Scott
Marmion
Canto vi. Stanza 17.
Phil Clarke says:
February 10, 2011 at 12:11 pm
Couple of factual points.
“1. Steig / Reviewer A wrote 24 pages of review, an utterly proportionate amount, the ’88 pages’ comes when you include the author’s reply.”
Laughable. Absurd. Have you no common sense whatsoever? What do you think journal editors do? If the average review were 24 pages long, no journal editor would finish reading half the reviews sent him? So, why would he ask for them?