From Climate Audit, more on the ongoing Antarctic kerfluffle.
By Ryan O’Donnell
Subsequent to my post on Feb 7, 2010 here, Eric Steig informed me by email that he had not seen our Response to his Third Review, as I had previously assumed. I apologize for my misunderstanding on this point, which was, however, incidental to the major concerns expressed in my post. A more detailed response on matters raised in Steig’s most recent RC post and other issues will be forthcoming.
My misunderstanding over whether Steig received the Response to the Third Review does not alter the fact that Steig acting as Reviewer A, in his Second Review, had asked the editor to “insist” that we present the “most likely” West Antarctica trends, specifically proposing iridge, although, in an email yesterday, Steig expressed “total surprise” that we had complied with his iridge proposal and, in his Feb 1 RC post, even criticized us for complying with his proposal. We will have more to say on this near the future, as his explanations remain insufficient.
Nor do I agree that the criticisms in his RC posts of the methodology have any merit. In his Third Review, Steig had raised similar points against iridge, notwithstanding his Second Review proposal of the approach and his Third Review comment that “use of the ‘iridge’ procedure makes sense to me, and I suspect it really does give the best results”. We responded fully to these concerns in our Response to his Third Review, although we were unaware until Feb 8, 2011 that Steig had not received a copy of our Response.
In any event, Steig knew or ought to have known that our response must have satisfied the editor of Journal of Climate and should have familiarized himself with our response before condemning the method that he had previously encouraged. Had Steig informed me that he had not seen a copy of our Response to his Third Review, I would have been delighted to send it to him. Instead, he chose to publicly disparage our paper using arguments that were both irrelevant and satisfactorily addressed – which was, unfortunately, no different than the tactic he used during review.
Steig’s recent outbursts are merely his most recent effort to obfuscate the underlying point of our critique: that whatever was original in Steig et al 2009 was based on faulty mathematics; and that whatever was correct in Steig et al 2009 was already known.
Alan Millar says:
February 11, 2011 at 10:14 am
“……It begs the question of who were the Reviewers of Steig’s original paper? How could it have passed through this process if it had been reviewed by an objective expert statistician? Was it ‘Pal’ review rather than Peer review? Will we ever know?”
////////////////////////////////////////////////////////////
This is a point that has been going through my head for sometime but which does not seemed to have been suffieiciently emphasised. It is a fundamental point and should be stressed.
The fact that Steig’s original paper got through the peer review system despite its obvious flaws demonstrates how unsatisfactory the peer review system is.
The fact that reviewer A submitted an 88 page response, shows the other problem with peer review namely attempts to stiffle oposing views.
Writers from earlier eras seem to have forseen this or similar situations – I guess there is nothing new in human conflict.
HL Mencken’s quip takes the prize for me – “For every problem there is a solution, simple, neat…and wrong!”!
The Team have certainly lost their way, along with their sense of humour.
Steig did not write all of the 88 pages… it was a total of 88 pages that resulted, which includes, as I recall, 24 pages from Steig.
Just to be clear since this keeps getting claimed.
Mark
richard verney says:
February 11, 2011 at 10:42 am
The fact that reviewer A submitted an 88 page response, shows the other problem with peer review namely attempts to stiffle oposing views.
Reviewer A did not submit 88 pages in response, even when all three reviews are added together. The original submission was however 95 pages long.
sp:
kerflufflekerfuffleREPLY: no, the “fluff” was intentional – Anthony
Mike Davis says: “Even if Steig was a part of the group that became reviewer A it is more likely that he was not aware of all the comments made by reviewer A! The responses from him show that reviewer A was a “Team Effort” with no one individual actually knowing what all was said in the process…!”
The comments went to the editor over Steig’s name. He bears full responsibility for them.
Alan Millar says: “That has to be one of the biggest own goals the ‘Team’ has ever made.”
The disconnect between AGW “science” and reality necessitates that Team activity will increasingly result in them “scoring” on themselves.
Ron Cram says: “My conclusion is that Eric Steig has been hanging around Michael Mann too much.”
Looks like Sister Michael Joseph was right when she used to lecture us about “avoiding bad companions.”
It begs the question of who were the Reviewers of Steig’s original paper? How could it have passed through this process if it had been reviewed by an objective expert statistician? Was it ‘Pal’ review rather than Peer review? Will we ever know?
The reply to McLean et al. provides a clue. Same cast of characters.
CLosed comments, huh? They know the game is over…
One more thing. If Steig does try to sue you for defamation, you should just point here for proof and he will not persue it. It is obvious that you are being calm and rational about the whole thing and that it is the ignorant scientists who are doing obfuscating outbursts.
One thing I don’t understand though is why he wrote to the editor of the journal that your paper is a worthwhile contribution when he obviously wanted to stifle publication. I suppose it was just so he could criticise you on RC for not doing what he suggested where people who don’t agree are not allowed to post. I don’t see him being game to post on here. Just shows the scientists are so ignorant of the peer-by-peer review process.
I look forward to your next post rebutting the science.
Regards, Denny.
” I would like to remind all that open/public criticism is generally not considered to be good form amongst actual scientists.”
This is the heart of the problem. The wall of secrecy, hidden behind authority. The unspoken “gentlemen’s agreement” that allows scientific errors and mistakes to persist, unknown to the public that largely foots the bill.
The Internet has changed this as both scientists and governments are discovering.
Unfortunately the demonstration that Stiegs original claims were rubbish will not get anywhere near the exposure afforded to the original claims themselves.
What are the chances of this matter making it to the cover page of a glossy science magazine?
And here’s “Fast Jimmy” Annan, holding two different groups of fellow scientists up, not just for criticism, but for ridicule, in a very open/public forum.
One set of rules for the climate conmen, another set for everyone else.
The flawed methodology of Steig et al 2009 is now well understood by all – thanks to Ryan’s graphs and Bishop Hill’s eloquent annotation – http://www.bishop-hill.net/blog/2011/2/8/steigs-method-massacred.html
Equally damning is Eric’s admission that statistical analysis is not his area of expertise. (So why was he lead author of S09 and why was he asked to review Ryan’s paper? )
I am also very intrigued by what Phil wrote last night on http://wattsupwiththat.com/2011/02/10/reviewer-a-responds/ – where he suggests that the satellite data available to Steig (with potential error of 3C due to cloud cover data uncertainties) could not have permitted the claimed the resultant temperature reconstruction accuracy to 1/10th of a degree. I repost Phil’s comment below in the hope that its significance is recognised:
Phil says:
February 10, 2011 at 9:33 pm
From Response to Third Review A, page 8:
…the reviewer seems to misunderstand the difference between spatial and serial [auto]correlation.
From http://climateaudit.org/2011/02/07/eric-steigs-trick/#comment-254159:
Steig stated by email today that he did not see the Response to Reviewer A’s Third Review…
It seems to me that Dr. Steig should not have launched into a public criticism of O’Donnell et al., when he, admittedly, was not a statistician and apparently did not understand a subtle, but key, statistical distinction.
Futhermore, it should be emphasized that Steig, et al. have not been completely transparent with regard to Steig 2009. They initially promised to electronically publish “all” of their data, but, subsequently, they have withheld and are continuing to withhold the raw satellite data as well as the details of the cloud masking. I don’t think that this data will ever be disclosed, because, if they do disclose it, IMO any remaining credibility would vanish. However, I could be wrong. Here is why I think that Steig et al. may need to be withdrawn:
1. Steig et al. claim that their study is based on the satellite data (NATURE| Vol 457|22 January 2009, pg 462):
We use passive infrared brightness measurements (TIR) from the Advanced Very High Resolution Radiometer (AVHRR), which are continuous beginning January 1982 and constitute the most spatially complete Antarctic temperature data set. (emphasis and acronym added)
They reference: Comiso, J. C. Variability and trends in Antarctic surface temperatures from in situ and satellite infrared measurements. J. Clim. 13, 1674–1696 (2000) (Comiso 2000) and state as the only explicit change to Comiso 2000:
We have updated the data throughout 2006, using an enhanced cloud-masking technique to give better fidelity with existing occupied and automatic weather station data. We make use of the cloud masking in (Comiso 2000) but impose an additional restriction that requires that daily anomalies be within a threshold of ±10 °C of climatology, a conservative technique that will tend to damp extreme values and, hence, minimize trends. (parenthetical comment added)
2. Comiso 2000 states:
Except for 1979 and 1992, when infrared data for the entire year were processed, the derived temperature data that are currently available are only for a winter (July) and a summer (January) month during (1978-1997). (parenthetical comment added)
References to seasonal trends are made in O’Donnell 2010, Steig’s first review (Review A) and its response, in Review C and its response and in Review D. In no way should this comment be taken as a criticism of O’Donnell 2010 as the processed AVHRR data was provided by Dr. Steig without further clarification and without any gaps (Steig et al Antarctica ant_recon.txt is what I had downloaded – it may have a different name now). The reader should be cautioned that Steig et al may have processed the infrared data for all months of the year and for each year, instead of what Comiso 2000 states, but, given the reference to Comiso 2000 and the refusal to disclose the raw satellite data, this question is not frivolous.
3. Steig 2009 makes no mention of how much AVHRR data is lost due to cloud masking. Kato et al. 2006 (S. Kato, N. G. Loeb, P. Minnis, J. A. Francis, T. P. Charlock, D. A. Rutan, E. E. Clothiaux, and S. Sun-Mack, Seasonal and interannual variations of top-of-atmosphere irradiance and cloud cover over polar regions derived from the CERES data set, GRL, VOL. 33, L19804, doi:10.1029/2006GL026685, 2006) states (pg 3):
The mean cloud cover over Antarctica is relatively constant, ranging between 0.62 and 0.75 during all seasons.
Thus, it can be assumed that only 25% to 38% of the AVHRR data is retained after cloud masking. Furthermore, Comiso 2000 states as one of their conclusions:
Among the key results of this study are the following: (a) satellite infrared data provide spatially detailed maps of surface temperature in the Antarctic region with an accuracy of 3°C…
Thus, each datum of the satellite data matrix provided by Dr. Steig should have an individual uncertainty probably greater than 3°C, but certainly in whole degrees C, yet Steig 2009 asserts that there is statistically significant warming by calculating trends in tenths of degrees C with confidence intervals expressed in hundredths of degrees C. I would submit that such an assertion is highly questionable and should be believed only upon rigorous demonstration. (Once again, this is no reflection on O’Donnell 2010 as they specifically state in the response to Reviewer D that:
Because our expertise is with the mathematics, we prefer to limit our paper to the mathematics.
)
Given all of the above, I respectfully submit that Steig 2009 should be withdrawn in its totality as the claimed warming trends appear to be a fantasy, given the cloud masking data losses of about two thirds and the uncertainty for each remaining datum of at least 3°C according to Comiso 2000 (Dr. Comiso is a co-author of Steig 2009). I remain open to withdrawing this comment if Steig et al. can rigorously demonstrate that the total uncertainty of the satellite data doesn’t swamp the small warming trends that they claim (I would likewise say that any claim that there is a small cooling trend would also be a fantasy on the same grounds). Withdrawal of Steig 2009, however, IMO should not impact O’Donnell 2010 as they have conclusively demonstrated that the statistical methodology employed in Steig 2009 is erroneous and that result would survive in any event. In closing, I would like to congratulate the O’Donnell 2010 authors for their fine scholarship.
My only comment on O’Donnell 2010 is that they should have included somewhere the response to Reviewer D’s comment on pages 1-3 of said response that the O’Donnell trend was not statistically different than the Steig 2009 trend, as this may be a common misunderstanding when comparing two different trends. I thought the explanation by O’Donnell et al. was very instructive and it shouldn’t be buried in a review response.
Phil. says: “…Reviewer A did not submit 88 pages in response, even when all three reviews are added together. The original submission was however 95 pages long.”
“The total number of pages dedicated by that reviewer alone – and our subsequent responses – was 88 single-spaced pages, or more than 10 times the length of the paper. ”
http://wattsupwiththat.com/2010/12/01/skeptic-paper-accepted-on-antarctica-rebuts-steig-et-al/
D. Nyall says: “…One thing I don’t understand though is why he wrote to the editor of the journal that your paper is a worthwhile contribution when he obviously wanted to stifle publication.”
It appears inconsistent, but makes sense if you assume that Steig, himself, wrote it, whereas the bulk of the review was forced upon him by meddlers. Still, it went out over his name, so he gets the full blame while the others (if there were any) can just slink away.
“Unfortunately the demonstration that Stiegs original claims were rubbish will not get anywhere near the exposure afforded to the original claims themselves.
What are the chances of this matter making it to the cover page of a glossy science magazine?”
I agree of course. And yet, you’d think one wise and savvy mainstream editor somewhere would see where this is headed and try to get out in front of it as much as possible. Because otherwise one would have to assume they are truly as benighted as they appear..
It’s a scary and depressing thought…
lapogus says: “…I am also very intrigued by what Phil wrote last night on http://wattsupwiththat.com/2011/02/10/reviewer-a-responds/ – where he suggests that the satellite data available to Steig (with potential error of 3C due to cloud cover data uncertainties) could not have permitted the claimed the resultant temperature reconstruction accuracy to 1/10th of a degree. I repost Phil’s comment below in the hope that its significance is recognised….”
Mindblowing! If correct, Steig et al 2009 should be withdrawn immediately. In fact, if only the part about Steig refusing to post his input data is right, the paper should be withdrawn. No data, no methods, no science.
I think Ryan’s outburst was more than warranted, but still:
If you muddy the waters by getting personal, you let others escape through the same muddy waters by getting equally personal.
Stick to the science.
That’s what we are here for.
@Alan Millar
good comment. and I do think that there has been some kind of ‘peer’ pressure or influence on Steig – probably by the ‘Team’ ?
I guess it’s the same with the majority of any AGW ‘denouncing’ work – any authors of such work will tend to have to face the ‘Team’ response?
ES deserves his criticism for his duplicitous behaviour IMO – but if the peer review process wasn’t so corrupt it may not have arisen.
@Ron Cram – I wasn’t specifically trying to lecturing anyone – and I agree with you. But I suppose I just don’t want to see ‘us’ stoop to the level of the other side! But I do agree that ES comments warranted strong response and O’Donnell has seemed pretty cool with his actions – I just don’t think it should have been quite so public. But then again, I guess that was the intentional purpose of the ‘Teams’ tactics?
ge0050 says:
February 11, 2011 at 11:15 am
Yes – we know errors in climate science are ‘persistent’ beggers! – and we know this is due entirely to the Team efforts.
My meaning/point is that normally such discussion is mostly sorted before publication. It’s not fair to tar all science with the same brush. Usually, work would be forwarded to others for proper CRITICAL review and discussed between reviewer(s) and authors appropriately. Clearly in ES’ case the original publication was not correctly peer reviewed (if at all? – one can imagine the warmists gleefully sending it to print!). As others have commented, the outcome of all this is yet more egg on the faces of the Team for their biased ‘peer’ review system.
jorgekafkazar says:
February 11, 2011 at 11:26 am
Phil. says: “…Reviewer A did not submit 88 pages in response, even when all three reviews are added together. The original submission was however 95 pages long.”
“The total number of pages dedicated by that reviewer alone – and our subsequent responses – was 88 single-spaced pages, or more than 10 times the length of the paper. ”
Indeed, they did say that on WUWT, however it is rather misleading. Reviewer A over three reviews contributed 24 pages, not 88.
Also the original submission was 45 pages with 51 pages of supplementary information.
http://www.climateaudit.info/data/odonnell/
Richard:
The fact that Steig’s original paper got through the peer review system despite its obvious flaws demonstrates how unsatisfactory the peer review system is.
I don’t see it that way at all. It was almost guaranteed to pass review.
* It had cool looking graphics,
* it verified some things that were already known (peninsula warming)
* and also showed something new (other areas warming)
* Even a good statistician would need to be suspicious that an erroneous conclusion was reached before he attempted to unravel the statistics.
* Since the procedures / codes used were not available, it’s an overwhelming task for a disinterested party or casual reviewer to try to review the statistics in detail, (but the study COULD be replicated by someone who WAS curious and interested and willing to invest hundreds of hours on replication).
* Only the level of analysis that Jeff, Ryan, Steve committed can reveal the underlying procedure, which is when the problems can be noticed.
* Withholding methods and code is the norm in “Mainstream” Climate Science, and probably other areas as well. Had methods been provided instead of having to be reverse engineered over weeks and months, these issues would have surfaced much earlier, and a real statistician probably would have seen the issues fairly quickly by doing just what Ryan demonstrated yesterday, namely, showing what happens when other data is used… Is the result sensitive to the change? Does this behavior make sense? Answer in this case, NO.
* There are two possible reasons researchers make review difficult by leaving out key information, either
a) They are not confident in their work, and don’t want it subject to scrutiny, or,
b) They are hiding something and don’t want to be discovered.
They know that no reviewer is expected to take so much time that they could write their own paper on the subject, so the paper is only given a cursory examination for glaring errors. I think the notion that most papers are subject to intense scrutiny is wrong.
The worrisome aspect of withholding key procedural information is that the paper becomes essentially useless, even if it is correct in all of its analysis. If replication is difficult, the lessons learned cannot be applied to a new situation, because the method is not understood well enough to even be evaluated, much less re-used.
Failure of transparency is the same outcome as no science at all. It inhibits replication and understanding by others, and can mask issues that become obvious when a fair and thorough review is conducted.
Contrast Steig’s paper with Jeff and Ryan’s replication instructions:
Download R
Copy / Paste
Press Start
Wait 17 minutes
Obviously, one technique is preferred over the other – by real scientists, that is.
Can you imagine if the preferred method were used in mainstream Climate Science? POOF. As it is, it can take some time to unravel and refute these papers, which is by design, I suspect.
Many are impossible to unravel. That’s how you know they are correct. 🙂
@Kev-in-UK
There is a phenomenon of “group think”. This isn’t the same as peer pressure, which is an active process of influence, but more of a subconscious want to appear pleasing and in agreement before ones immediate associates and people held in high regard.
As Ryan O’Donnell said, Dr. Steig acts completely different when he puts his “RealClimate” hat on.
Put another way, we become like those we associate with, especially while in their presence. Put yet another way, bad company corrupts good character.
@Phil
No, it isn’t misleading. They said the sum total of the reviewer’s pages and their pages in response to the reviewer came out to 88 single spaced, dense pages. If you want to put it in math sum(reviewer A, O’Donnall Response) = 88.
You are also forgetting formatting differences (line spacing, font size, margin size) and space taken up by figures when counting the page number of the paper itself compared to the reviewer and responses.
Scientific papers are way shorter than they seem text wise when you distill out all the fluff.
I have an idea from the real world.
These who support this hockey stick construction method should invest in said facts.
All 3,000 who signed the letter of approval should each get treasury stock in the AGW, Inc. inital offering on the NY Stock Exchange. If in fact this science is real and investment in CO2 trading is a good investment these 3,000 should make a load of cash and they will then be able to point long boney fingers at us others and laugh all the way to the bank.
If its not so, they can ask U.S. for more tax money hand outs and get a firm no.