I’m a bit late in covering this due to other issues of interest taking precedence, but I want to make sure this is widely known. We’ve covered the issue of the Gergis et al paleoclimatology paper this past summer, as well as noted the retraction, but the real action as usual, is behind the scenes in the emails, emails now made available via FOI thanks to Michael Kottek who posted this on Climate Audit to announce the emails were now available:
Posted Oct 28, 2012 at 7:04 AM | PermalinkThe results of my FOI request to the University of Melbourne can be seen here:
I requested all correspondence between the authors and the journal regarding the paper. The referees reports were exempted as were documents relating to the resubmitted paper.
I also requested correspondence between the authors after the paper was accepted. Once again emails relating to the resubmitted paper were exempted, and personal material redacted.
I note that emails regarding the paper that were received by one author and not forwarded to the others would not have been covered by my request.
Despite the embarrassment of the withdrawn paper, the University is to be commended for their no nonsense approach to this request. As an alumunus, I am pleased that the response is far more sensible than the approach taken by the UEA and UVa.
That’s true, because there appears to be no holding back of any important correspondence. Here’s some select portions.
Karoly’s first technical response (June 7 Melbourne) to Neukom’s confession was a surprisingly strong endorsement of criticism of non-detrended correlation, going as far as to even agree with [McIntyre] by name:
”If the selection is done on the proxies without detrending ie the full proxy records over the 20th century, then records with strong trends will be selected and that will effectively force a hockey stick result. Then Stephen Mcintyre criticism is valid”.
And then there’s this from Gergis
[…]
”Over recent days we have been in discussion with colleagues here in Australia and internationally about the use of detrended or non detrended data for proxy selection as both methods are published in the literature .
People have argued that detrending proxy records when reconstructing temperature is in fact undesirable (see two papers attached provided courtesy of Professor Michael Mann) .
While anthropogenic trends may inflate correlation coefficients, this can be dealt with by allowing for autocorrelation when assessing significance. If any linear trends ARE removed when validating individual proxies, then the validation exercise will essentially only confirm the ability of the proxies to reconstruct interannual variations. However, in an exercise of this nature we are also intrinsically interested in reconstructing longer-term trends. It therefore appears to be preferable to retain trends in the data, so that we are also assessing the ability of the proxies to reconstruct this information.”
And this admission from co-author Phipps:
Based on the various emails circulated over the past few days, it appears that we will not have a viable millennial-scale reconstruction if we pursue the detrended approach. I therefore feel that we should use the raw data to validate the proxies…
My preference is therefore for David’s Option 2, with Option 1 as my second choice. I dislike Option 3 as it will not leave us with a viable reconstruction. I also dislike Option 4 as it strikes me as essentially starting again from scratch – which seems unnecessary given how far this work has already progressed, and also seems out of proportion to what is only a matter of fixing a technical issue.
Mann, in correspondence with the authors Gergis and Karoly, in his typical style tried to sell a collection different workarounds for the problems they brought on themselves, and in the end, his advice was rejected, the JC editors told the authors the paper was not viable, and the authors were forced to withdraw the paper. Full stop.
Journal of Climate editor Chiang wrote:
After consulting with the Chief Editor, I have decided to rescind acceptance of the paper- you’ll receive an official email from J Climate to this effect as soon as we figure out how it should be properly done. I believe the EOR has already been taken down.
Also, since it appears that you will have to redo the entire analysis (and which may result in different conclusions), I will also be requesting that you withdraw the paper from consideration. Again, you’ll hear officially from J CLimate in due course. I invite you to resubmit once the necessary analyses and changes to the manuscript have been made.
I hope this will be acceptable to you. I regret the situation, but thank you for bringing it to my prompt attention.
The end result of the AMS putting the authors on notice, plus the admissions in the emails, rather puts this early defamatory remark from Dr. Mann in perspective:
Well I’m afraid Mclntyre has probably already leaked this anyway. I probably don’t have to tell you this, but don’t trust him to behave ethically or honestly here, and assume that anything you tell him will be cherry-picked in a way that maximally discredits the study and will be leaked as suits his purposes.
Read the whole episode here at Climate Audit.
It should be noted that commenter Jean S. contributed the first valid critique, which then later grew into this full retraction. Kudos to him too.
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
richardscourtney,
1.
Where did you get this nonsense: “the divergence problem is thermometer calibration drift” ???
2.
“You are claiming a proxy indication of temperature is more accurate than thermometer data which is used to calibrate it!”
This comment just shows that you do not understand the question.
3.
“The method REQUIRES “screening”, i.e. selection. It is part of calibration.”
Absolutely not, these two statements are completely false.
4.
“The growth of trees is dependent on the limiting factor for their growth, and this is rarely temperature.”
We are talking about MXD not growth. What is valid for TRW is not necessarily for MXD.
Any discussion with you is useless, I wish you a good continuation, Mr. Courtney.
phi:
re your post at October 31, 2012 at 1:34 pm.
There was no discussion.
You posted illogical and factually inaccurate nonsense.
I pointed out your errors.
You have gone off in a huff.
I am saddened that you left before starting a discussion.
Richard
From Mr. Zip to Mr. Zippo.
One thing about this “trended” / “detrended” argument I don’t get. Surely the proxy has to match both the detrended and the trended for it to be useful? I can see it has to match the detrended in order to be a legitimate temperature proxy. But then if it doesn’t match the trended, it still isn’t a temperature proxy. Or am I missing something?
Mike Jonas:
You ask October 31, 2012 at 2:40 pm
You are missing nothing. You are right.
In fact, rejecting ANY trees biases the sample: strange but true.
A true proxy
(a) varies in a known way
(b) with the parameter it is indicating
(c) as a result of a known mechanism.
Thus the variations of the proxy indicate variations of the parameter for a known reason.
In treemometry (a) and (c) is each unknown. The method assumes that tree ring growth indicates temperature. There is no reason to think this is true and several reasons to doubt it.
If trees were proxies for temperature then all trees (e.g. of a species) would indicate temperature. Most don’t. But the method assumes the trees which vary with temperature over a calibration period will indicate temperature at other times. In the unlikely event that the assumption were true then trees would only indicate temperature in the growing season. And it is a very unlikely event. The rate of tree ring growth varies for several reasons so it is always possible to find some trees that seem to indicate temperature over a calibration period. Other trees will give a similar seeming indication over a different calibration period.
Simply, as a science, treemometry makes astrology look good.
And considerations of whether tree ring data should be detrended or not for treemometry is like considering whether the influence of the Moon or the planets should be considered when making a horoscope. The relevant ‘experts’ have detailed debates to decide these matters, but whatever decision they reach does not stop the results of their analyses being rubbish.
Richard
Okay, I’ll bite. . . .
“Well I’m afraid Mclntyre has probably already leaked this anyway. I probably don’t have to tell you this, but don’t trust him to behave ethically or honestly here, and assume that anything you tell him will be cherry-picked in a way that maximally discredits the study and will be leaked as suits his purposes.”
1. How in the world could Steve McIntyre “leak” something that had been publicly published?
2. Even if he wanted to, how could Steve McIntyre behave either “ethically” or “unethically?” Is there some code of honor that I’ve never heard about that proscribes someone who reads something by a “Climate Scientist” from commenting about the paper?
Ok, I’ll bite too. Steve McIntyre is not playing by club rules, as set out by people who believe that they are the arbiters of club decorum. In effect, he’s worn non-regulation length shorts to the annual club tournament. It matters not that he plays his ball fairly, counts every stroke and questions only to ensure fair play.
Despite being able to not only play but teach a great game, he should be disqualified since his shorts are an inch too short.
phi,
I think both trended and detrended screening is biased to hockey sticks.
This is obvious, if you assume that proxies are thermometers of varying quality over time.
Passing through screening suggests that tree rings have been sufficiently correlated with thermometers in the modern times calibration period.
But this excludes proxies, which have been good thermometers in the past but fail to be in modern times.
And includes proxies which have been poor in the past but good in modern times.
Both inevitably will reduce temperature maxima before the calibration method and results in a bias towards a hockey stick.
Additional and perhaps even more Hockey Stickness comes from the habit of data non archiving in climate science. Non hockey stick data is very likely to be not archived and available for reconstruction.
And on top of that, there is a bias of selection by some scientists. Despite some of the proxies have long been classified as false, upside down or corrupt, and despite this fact has been known to researchers, the have been used again and again.
Jan – not quite. It’s not his shorts but the fact he is better than the Club Coach who Steve shows to be playing with crooked kit.
I’m also biting.
I lack Jan’s wit so I cannot provide so accurate, amusing and sarcastic a summary as that.
Instead, I say it as I think it.
The Team runs a closed shop because they all know treemometry is nonsense. Outsiders may notice and report their errors. IImportantly, real statisticians such as Steve McIntyre can not only see and report their errors but can also demonstrate those errors to others.
Ethics are rules of correct conduct.
1.
The first rule of any closed shop is to exclude outsiders. An outsider who enters the shop is – by definition – being unethical.
2.
The second rule of the Team is to ensure that only they know how bad their work really is. Indeed, this is the main reason they run a closed shop. Clearly, the release of anything which shows the truly bad quality of their work is a “leak” from the shop.
3.
The climategate emails reveal the members of the Team define what is “honest” behaviour, and their definition is anything which suites their “cause”. Clearly, by their definition, finding and exposing faults in their work is dishonest.
4.
Point 2 demands that any fault in their work must be hidden (hence, “hide the decline”) and only points complimentary of their work are allowed to be reported. Therefore, mentioning – especially “leaking” – a fault is not complimentary of the work so can only be “cherry-picked in a way that maximally discredits the study”.
Hence, in terms of the Team’s nature and ethics, Mann’s comment is seen to be correct.
Richard
slow to follow – of course, you are absolutely right!
In statistics you are not allowed to use temperature to select the trees you want to analyze, if it is temperature that you are trying to correlate with the trees. Formally it is known as “selection on the dependent variable”. It is forbidden in statistics because it leads to spurious correlations.
Imagine that you wanted to find out why people were getting sick, so you selected a number of sick people to study. After all, it seems reasonable that if you want to find what is making the people sick, you should look at sick people.
And what you found after studying the people is that they mostly had brown hair. And from this you concluded that brown hair was making them sick, because it had the highest correlation.
The problem is that you selected the people based on the very thing you wanted to study, and this gave a misleading statistical result. If you want to find out why people are getting sick, then you need to include both sick and not sick people in the study, to find what is different.
This is the effect of selecting tree rings based on temperature (calibration) when temperature is the thing you are trying to study. It returns statistical nonsense that on the surface appears reasonable. Nothing says that climate is anything different that “brown hair”.
phi says:
October 31, 2012 at 7:34 am
I broadly share your analysis. But that’s about screening and not specifically dendro. There are many local, regional or hemispheric studies which use especially rings dendities without any screening. These studies provide quite remarkable results.
=========
This result argues strongly that tree rings should not be screened. As such, it is further evidence that the screening process is introducing selection bias into the results.
The problem for the authors in this case is that trees do not respond globally, they respond to local conditions. The screening process is effect is trying to select trees that are ignoring local conditions and responding to global conditions, something that is physically impossible.
As such, I could see some validity in the use of tree rings if you calculate the result regionally without screening and then use this to compute a longer term average trend, but you would likely need quite a large number of samples to be representative.
From what I read in the emails, when the authors tried this the significance dropped to the point where only a single proxy before 1400 was useable, which argues that the result is not showing a global or southern hemisphere trend, rather a local or regional trend.
Every paper will have faults. The question is whether the faults are significant. In every case that I’m aware of where McIntyre noted faults, they were significant enough to require correction or retraction of said papers.
Manfred,
What you describe can happen but screening on detrended data prevents a statistical artefact which increases the variance over the screening period. This implies in principle a general loss of quality of the construction but at least the variance is homogeneous.
ferd berple,
“As such, I could see some validity in the use of tree rings if you calculate the result regionally without screening and then use this to compute a longer term average trend, but you would likely need quite a large number of samples to be representative.”
That’s exactly it.
Manfred,
It should be added that if the screening is done against temperatures whose trend is biased (eg UHI), screening and calibration on detrended data becomes absolutely necessary.
Ferd (way up the thread):
What the events show is that peer review is not robust. It failed to catch a fairly significant error in the paper, in that the methodology described did not match the methodology used. This is strong evidence that peer review does not guarantee correctness.
Absolutely correct, especially with regard to peer review as currently performed on analytical data that are produced by ‘black box’ computer routines. If you are given a copy of the paper text but not the complete dataset and breakdown of the code used in the computer routines, then it is not possible to pick up this sort of error. Given that peer review is essentially a voluntary and unpaid / unappreciated* aspect of (mainly) academia, the quality of peer review is inevitably variable and frequently little more than cursory.
What the events show is that internet review is more robust than peer review. This is strong evidence that the results published on the internet are more reliable than the results published in leading journals.
This argues strongly that leading journals should require more than peer review as a condition of publishing. They should require that the papers pass internet review.
Not sure I entirely agree with these comments – perhaps better to say that effective peer review needs to learn from the strengths of internet review as performed by the likes of Steve McIntyre and Jean S. You have to take the time and get into the heart of the data, not just the written description. I recall Phil Jones’s response on refusing Steve Mc’s request for data backing up the findings of a paper – ‘no-one has asked for the data before’.
I think a lot of internet or other open review simply generates more heat than light, especially when on controversial topics, and so it becomes very difficult to identify the important points. Too many eyes with too many personal biases…
What the events also show is the professional climate scientists are no better at catching errors than anyone else. A degree in science does not make you an expert in spotting mistakes. An eye for detail is not something learned in university. Some folks have it, some don’t, and it is a skill that appears early in life, during early childhood. What university teaches (hopefully) is how to apply this skill to different types of problems.
Absolutely – I write a lot of technical reports for my job, but each report must be fully reviewed and counter-signed by a colleague prior to being issued (similarly, I review reports for colleagues). It often needs a second set of eyes to spot the mistake, and even with our review system some errors get through.
* No-one boost their CV or research profile by listing how many papers they review. Publish or perish is the only mantra, not ‘make sure your colleagues and rivals don’t screw up’.