NOTE: This has been running two weeks at the top of WUWT, discussion has slowed, so I’m placing it back in regular que. – Anthony
UPDATES:
Statistician William Briggs weighs in here
Eduardo Zorita weighs in here
Anonymous blogger “Deep Climate” weighs in with what he/she calls a “deeply flawed study” here
After a week of being “preoccupied” Real Climate finally breaks radio silence here. It appears to be a prelude to a dismissal with a “wave of the hand”
Supplementary Info now available: All data and code used in this paper are available at the Annals of Applied Statistics supplementary materials website:
http://www.imstat.org/aoas/supplements/default.htm
=========================================
Sticky Wicket – phrase, meaning: “A difficult situation”.
Oh, my. There is a new and important study on temperature proxy reconstructions (McShane and Wyner 2010) submitted into the Annals of Applied Statistics and is listed to be published in the next issue. According to Steve McIntyre, this is one of the “top statistical journals”. This paper is a direct and serious rebuttal to the proxy reconstructions of Mann. It seems watertight on the surface, because instead of trying to attack the proxy data quality issues, they assumed the proxy data was accurate for their purpose, then created a bayesian backcast method. Then, using the proxy data, they demonstrate it fails to reproduce the sharp 20th century uptick.
Now, there’s a new look to the familiar “hockey stick”.
Before:
After:
Not only are the results stunning, but the paper is highly readable, written in a sensible style that most laymen can absorb, even if they don’t understand some of the finer points of bayesian and loess filters, or principal components. Not only that, this paper is a confirmation of McIntyre and McKitrick’s work, with a strong nod to Wegman. I highly recommend reading this and distributing this story widely.
Here’s the submitted paper:
(PDF, 2.5 MB. Backup download available here: McShane and Wyner 2010 )
It states in its abstract:
We find that the proxies do not predict temperature significantly better than random series generated independently of temperature. Furthermore, various model specifications that perform similarly at predicting temperature produce extremely different historical backcasts. Finally, the proxies seem unable to forecast the high levels of and sharp run-up in temperature in the 1990s either in-sample or from contiguous holdout blocks, thus casting doubt on their ability to predict such phenomena if in fact they occurred several hundred years ago.
Here are some excerpts from the paper (emphasis in paragraphs mine):
This one shows that M&M hit the mark, because it is independent validation:
In other words, our model performs better when using highly autocorrelated
noise rather than proxies to ”predict” temperature. The real proxies are less predictive than our ”fake” data. While the Lasso generated reconstructions using the proxies are highly statistically significant compared to simple null models, they do not achieve statistical significance against sophisticated null models.
We are not the first to observe this effect. It was shown, in McIntyre
and McKitrick (2005a,c), that random sequences with complex local dependence
structures can predict temperatures. Their approach has been
roundly dismissed in the climate science literature:
To generate ”random” noise series, MM05c apply the full autoregressive structure of the real world proxy series. In this way, they in fact train their stochastic engine with significant (if not dominant) low frequency climate signal rather than purely non-climatic noise and its persistence. [Emphasis in original]
Ammann and Wahl (2007)
…
On the power of the proxy data to actually detect climate change:
This is disturbing: if a model cannot predict the occurrence of a sharp run-up in an out-of-sample block which is contiguous with the insample training set, then it seems highly unlikely that it has power to detect such levels or run-ups in the more distant past. It is even more discouraging when one recalls Figure 15: the model cannot capture the sharp run-up even in-sample. In sum, these results suggest that the ninety-three sequences that comprise the 1,000 year old proxy record simply lack power to detect a sharp increase in temperature. See Footnote 12
Footnote 12:
On the other hand, perhaps our model is unable to detect the high level of and sharp run-up in recent temperatures because anthropogenic factors have, for example, caused a regime change in the relation between temperatures and proxies. While this is certainly a consistent line of reasoning, it is also fraught with peril for, once one admits the possibility of regime changes in the instrumental period, it raises the question of whether such changes exist elsewhere over the past 1,000 years. Furthermore, it implies that up to half of the already short instrumental record is corrupted by anthropogenic factors, thus undermining paleoclimatology as a statistical enterprise.
…
We plot the in-sample portion of this backcast (1850-1998 AD) in Figure 15. Not surprisingly, the model tracks CRU reasonably well because it is in-sample. However, despite the fact that the backcast is both in-sample and initialized with the high true temperatures from 1999 AD and 2000 AD, it still cannot capture either the high level of or the sharp run-up in temperatures of the 1990s. It is substantially biased low. That the model cannot capture run-up even in-sample does not portend well for its ability
to capture similar levels and run-ups if they exist out-of-sample.
…
Conclusion.
Research on multi-proxy temperature reconstructions of the earth’s temperature is now entering its second decade. While the literature is large, there has been very little collaboration with universitylevel, professional statisticians (Wegman et al., 2006; Wegman, 2006). Our paper is an effort to apply some modern statistical methods to these problems. While our results agree with the climate scientists findings in some
respects, our methods of estimating model uncertainty and accuracy are in sharp disagreement.
On the one hand, we conclude unequivocally that the evidence for a ”long-handled” hockey stick (where the shaft of the hockey stick extends to the year 1000 AD) is lacking in the data. The fundamental problem is that there is a limited amount of proxy data which dates back to 1000 AD; what is available is weakly predictive of global annual temperature. Our backcasting methods, which track quite closely the methods applied most recently in Mann (2008) to the same data, are unable to catch the sharp run up in temperatures recorded in the 1990s, even in-sample.
As can be seen in Figure 15, our estimate of the run up in temperature in the 1990s has
a much smaller slope than the actual temperature series. Furthermore, the lower frame of Figure 18 clearly reveals that the proxy model is not at all able to track the high gradient segment. Consequently, the long flat handle of the hockey stick is best understood to be a feature of regression and less a reflection of our knowledge of the truth. Nevertheless, the temperatures of the last few decades have been relatively warm compared to many of the thousand year temperature curves sampled from the posterior distribution of our model.
Our main contribution is our efforts to seriously grapple with the uncertainty involved in paleoclimatological reconstructions. Regression of high dimensional time series is always a complex problem with many traps. In our case, the particular challenges include (i) a short sequence of training data, (ii) more predictors than observations, (iii) a very weak signal, and (iv) response and predictor variables which are both strongly autocorrelated.
The final point is particularly troublesome: since the data is not easily modeled by a simple autoregressive process it follows that the number of truly independent observations (i.e., the effective sample size) may be just too small for accurate reconstruction.
Climate scientists have greatly underestimated the uncertainty of proxy based reconstructions and hence have been overconfident in their models. We have shown that time dependence in the temperature series is sufficiently strong to permit complex sequences of random numbers to forecast out-of-sample reasonably well fairly frequently (see, for example, Figure 9). Furthermore, even proxy based models with approximately the same amount of reconstructive skill (Figures 11,12, and 13), produce strikingly dissimilar historical backcasts: some of these look like hockey sticks but most do not (Figure 14).
Natural climate variability is not well understood and is probably quite large. It is not clear that the proxies currently used to predict temperature are even predictive of it at the scale of several decades let alone over many centuries. Nonetheless, paleoclimatoligical reconstructions constitute only one source of evidence in the AGW debate. Our work stands entirely on the shoulders of those environmental scientists who labored untold years to assemble the vast network of natural proxies. Although we assume the reliability of their data for our purposes here, there still remains a considerable number of outstanding questions that can only be answered with a free and open inquiry and a great deal of replication.
===============================================================
Commenters on WUWT report that Tamino and Romm are deleting comments even mentioning this paper on their blog comment forum. Their refusal to even acknowledge it tells you it has squarely hit the target, and the fat lady has sung – loudly.
(h/t to WUWT reader “thechuckr”)
Mikael:
I think you have to look at it historically, the science projects have
grown in the past decades, both in complexity, scope and also concerning
the stakes from a societal viewpoint.
Well, there’s a durn good reason why they encourage history PhDs to take statistics courses as well as languages.
“Although we assume the reliability of their data for our purposes here”
This is the data put together by the Man(n) who is willing to cheat on the published record of how many papers he has had published and repeatedly uses inverted graphs to substantiate his conclusions.
Two historic posts in 24 hours – Lord Monkton blasts the estimates for C02 projections and now the Hockey stick is shown to be the hocus pocus we all guessed it to be. With this evidence even non scientists like me can go into enemy territory and slay the illusory scare stories put up in the Guardian and other tin pot newspapers.
Anthony,
I have never tried to participate at Tamino’s place before. Occasionally at RC, but not often.
Anyway, really want to know more about the ad hominen against McShane and Wyner right from the initial source there.
John
Ric Werme says: August 15, 2010 at 4:57 am
There’s an entry for paleoclimatology, see
http://home.comcast.net/~ewerme/wuwt/categories.html
http://home.comcast.net/~ewerme/wuwt/cat_paleoclimatology.html
Rick, that’s a really useful search resource for WUWT, far more useful than the current search bar. Anthony is there any chance we can have a prominent link to this at the top of your page or in the sidebar?
“The hockey stick is broken” is a great rallying cry, but has zero substance in the world of qualified scientists who actually produce the charts in question. Some climate scientists have actually investigated the broken hockey stick claim in detail. Here’s what they found: nothing. If, on the other hand, one chooses to believe that IPCC and NASA scientists are part of a grant-seeking world-government-installing cabal, than it is difficult to dispute your argument. It’s considerably more difficult to believe it.
I am not concerned with the politics. But the Medieval Warm period is in the literature, architecture, geology, and archaeology. Even the statistics, such as they were (e.g., “the emperor’s cherry trees”). Climate scientists have got to face up to the fact that they are, as on wit put it, like Truman Capote trying to marry Dolly Parton. The job is just too big for them. They sound like the History channel trying to convince us that Philadelphia won WWII (and never mind Vasilevsky’s army groups).
Climatology takes a village. A Full and Complete village.
As it is, they are even trying to cut the oceanographers out of the picture. It simply won’t do.
Latitude
My branch of science is medicine, specifically anaesthesia, and I spent 9 years as an Associate Editor to one of our journals. Peer review to me means asking “was the methodology, including sample size and selection, and the statistical analysis appropriate to test the hypothesis put forward?” and “are the conclusions justified by the results?”.
Just checking the spelling and grammar and asking whether the topic is interesting does not constitute peer review. If we are moving to a system of publishing followed by on-line review, that gives us a major problem of how the various criticisms are integrated and a revised, hopely now improved, paper published. Where is the pressure on the authors to accept criticisms and change the paper? It means also that initial publications must contain a health warning that policy makers and users should not rely on the results.
For my money, peer review occurs prior to publication not afterwards.
evanmjones-
Agree, but if the climatoligists let in real scientists and statisticians their lie will be exposed!
[REPLY – There is a distinct possibility they have gone too far out on the limb to get back. Nonetheless, we must endeavor to persevere. ~ Evan]
Not sure why everyone is jumping up and down. The “hockey stick” is still within the uncertainties cited by these authors.
Two comments:
1. Any researcher worth his/her salt takes a graduate level stats class. We get to do things like ANOVA’s and COANOVA’s with just a simple calculator and without the use of a computer program. That experience SHOULD help us understand that unless there is a University level independent statistician available and required to be used by the lab, the lab is nothing but a playpen for too big for their breeches, self-important, puffed up scientists putting out embarrassingly poor and possibly dangerous products. Have you noticed that there is an entire legal industry built around just such a lab?
2. Getting peer reviewed and published is one of the most political processes there is, resulting in, when the process is done, the need for a detox scrubbing of all body parts till those parts are raw and bleeding. Trust me, in the dog eat dog world of journals, good studies go unpublished while poor ones get top billing, and on a fairly regular basis.
Might it be that only those who have never been through the process believe that checking for spelling and performing a simple “merit check” is the bulk of the review/publish process? Might it be that only those who are not sure about the statistics of their endeavor but since the results reject their null hypothesis would turn aside from a thorough independent University stat review?
Its curious that periods prior the the MWP are not so researched as the period 1000-present. Modern proxies tend to forget the holocence optimum some 9000-5000 before present – which excelled the MWP and certainly today in temperature, both temperature period and elevation by rapport to the two warm periods (The long MWP and the short late 20th C) that concern climate reconstructions. The Roman warm period is also excluded.
When a long period legend are included, what emerges is that 1850-1875 were by far the coldest years of the entire holocence. An ice age compared to the median of the last 12,000 years. 1875, atypically cold, is also the year chosen by AGW proponents as the starting point of the climate.
I did a first read of the McShane & Wyner paper. I am not a climatologist or a statistician.
I think everyone agrees there are large uncertainties when “backcasting” global temps a thousand years. M&W take issue with how proxies are calibrated with the instrumental record. If you delete a block of data you can try to “predict” it by using the remaining temp data (and its derivative) or using your proxies. You’d like for the proxies to do better than interpolation. What M&W call pseudo-proxies or “fake” data is really a form of interpolation. They aren’t really using pure noise. Climatologists, according to M&W, block out a 50 year window in the middle of the instrumental record. M&W block out various 30 year windows. With the shorter window interpolation does as well as the proxies they claim and therefore the proxies are of little use. It makes sense to me that interpolation would do better with shorter window length. So, I don’t see the paper as punching that big a hole in the “hockey stick” constructions. I look forward to seeing what people who know more about this than I do have to say.
The other issue brought up by this paper is the fact that the model program said to be very similar to Mann’s, could not reconstruct the proxy slope but could the observation slope from Mann’s graph. That means two things to me:
1. The study confirms that the two data sets are apples and oranges and should have never been spliced together, as has been stated by many skeptics.
2. The fact that the model reconstructed the current steep slope does not solidify Mann’s version of temperature rise as the correct one. It simply confirms his code used to give that temperature rise works on the data in similar fashion to the authors’ code. Importantly, the authors caution the rise might be contaminated by anthropogenic data. To me that means human sourced data and includes ALL the various ways the temperature data used by Mann has been compromised and contaminated.
To me it is not of primary importance that the hockey stick is step-by-step being trivialized, it is that Mann is being marginalized. It says to other climate scientists that he, as a role model, doesn’t look promising.
John
Mikael Pihlström says:
August 15, 2010 at 11:33 am
But, the accusation that Mann and others neglected to do so, just to be
able to manipulate and distort results;I don’t believe it.”
M&M (with no AGW axes to grind) have shown clear evidence that Mann et al hadn’t used correct statistical analysis yet Mann et al have refuse to acknowledge any of this evidence. In light of this new paper does their continuing [refusal to provide] this evidence have any influence on your disbelief stated above?
I guess I don’t get it! The Mann temperature data (red) ends at the date of publication (1998) as does the data. The McShane and Wyner temperature data (black) ends in 2000 and does not extend beyond the proxy data. Yes, the extent that the inclusion of this data would raise the “blade” of the stick is slight. Mann puts that in his graph to strenghthen his point just as McShane and Wyner omit this data to enhance theirs.
Mikael Pihlström says:
August 15, 2010 at 4:31 am
Philemon says: August 15, 2010 at 2:44 am
Mikael Pihlström says: August 15, 2010 at 2:22 am
“…why not use fig. 17, which brings it alltogether: the warming of the last decades is bigger than any backcast, H&W 2010 included.”
Look at the uncertainty bands.
“In fact, our uncertainty bands are so wide that they envelop all of the other backcasts in the literature. Given their ample width, it is difficult to say that recent warming is an extraordinary event compared to the last 1,000 years. For example, according to our uncertainty bands, it is possible that it was as warm in the year 1200 AD as it is today.” (McShane and Wyner, AOAS 2010, p. 37)
—————
“You are right, but uncertainty works both ways: It could have been as warm in 1200 AD, or considerably cooler.”
_________
Brrr! However, as TerryS says:
August 15, 2010 at 5:02 am
“The error bars are so big in the graph that it encompasses everything from the MWP being colder than the LIA to the LIA being warmer than today and pretty much everything in between.”
_________
Since the Mannian proxy reconstructions were used to claim that modern warming was unprecedented, the demonstration that statistically they can show no such result is the more interesting conclusion.
Moreover, whatever the limitations of peer review in checking statistical acumen, the fact that grants were obtained, without the benefit of a statistician on board, for the analysis of data containing “…complex spatial and temporal dependence structures which are not easily captured with simple models…” (McShane and Wyner, AOAS 2010, p. 1) is even more interesting.
Doug McGee:
You’re joking, right?
“According to Steve McIntyre, this is one of the “top statistical journals”. ”
A very minor point here, the journal is new, only in its forth year. So, it is not likely to be a top journal yet. It takes many years to establish a reputation. The journal claims to have the 6th highest impact factor among stat journals, but it is not clear what that really means. The authors of the paper work in business schools (good ones) and have little background in science. But of course the paper should be judged on its merits and that will take some time.
hehehe
James Sexton: no offence taken mate. Nice to see you and Smokey at work, good team of Beaters at Quidditch.
From New York Times:
http://www.nytimes.com/2010/08/15/science/earth/15climate.html?ref=global-home
“If you ask me as a person, do I think the Russian heat wave has to do with climate change, the answer is yes,” said Gavin Schmidt, a climate researcher with NASA in New York. “If you ask me as a scientist whether I have proved it, the answer is no — at least not yet.”
Is this the first time Gavin has expressed anything but total certainty?
And elsewhere in the article nytimes says:
‘Seemingly disconnected, these far-flung disasters [floods, fires] are reviving the question of whether global warming is causing more weather extremes’
Note “reviving”; but when asked to help in the revival Gavin seems to have declined. What’s going on?
The development here in New Zealand this morning has been mentioned a
couple of times, and although it is off-topic, it is certainly worth looking at.
NIWA, the “official body”, has been a proponent of GW, and proclaimed, along
with many other sources, that the most recent decade has been the ‘warmest’.
When pressed for figures, they repaired to their Seven Key Stations (SKS),
which are widely dispersed over the country, and which have been collecting
temperature data for over 100 years. On the basis of data from the SKS,
they announced the following:
that the mean temperature in the decade to end December 2009 was
ONE-TENTH OF A DEGREE ‘warmer’ than the mean temperaure in the
thirty year period from 1970 to 1999.
Please note: that from a mean established for the most recent decade,
NIWA had to go back a further THIRTY YEARS before they could get to
a ‘cooler’ difference of one-tenth of a degree !
It is an abuse of the English language to describe this as ‘warming’.
In the NIWA website, however, and referring again to the SKS, the
statement is made that there has been a 0.9 degree C increase in mean
temperature in the hundred years up to the end of 2009.
?
So we have 0.2 degree differences in the last 40 years, and 0.7 degree
differences in the previous 60 ? This does not compute.
So far as New Zealand is concerned, take the ‘G’ out of AGW: we don’t fit.
So whats next?
Once the knee-jerk protecting of Mann/Gore starts to subside and reality sets in, where will the AGW mob go next?
IMHO, Gore will be left to fend for himself, he has the resources and lack of morals to succeed (in his own mind) against reality, but what of Mann?
I strongly suspect that quite soon now he will be gone from the scene. He is an obnoxious personality, and I doubt that many of his compatriots would shed too many tears over throwing him to the wolves.
It has taken far to long (IMHO) for reputable scientists to do what was required and expose cargo cult science for what it is. However, it looks like its about to happen, which restores a lot of my faith in the scientific method not being dead and buried.
Great post Rex from NZ. Thank you!
Mike Roddy says:
August 14, 2010 at 7:13 pm
“[…] Glaciers and Arctic ice are melting at unheard of rates. […]”
Maybe some people are just hard of hearing. If one listens to him carefully, Otzi the Ice Man just might tell you differently about the melt rate of glaciers. Unfortunately, the Viking farmers buried on Greenland didn’t have much to say about the rate of Arctic melt, though the way they left things suggest they’d have something to say about how it can get pretty darn cold in a hurry in that neck of the woods.
If we keep up with the satellite monitoring for a few centuries, humans might have something more concrete to say about the rate of Arctic melting (and icing) we’ve observed over the past 30 years. Right now, not so much.
My take is that in the paper figures 15 and 18 show that their model needs some work to explain the modern warming.
Figure 16 still looks like a hokey stick, and if I could see the whole of the grey areas of figure 17, I would be able to predict how much further warming is necessary to say that the current warming is unprecendented in the last 1000 years.
Still have a way to go before the current warming is unprecedented for the Holocene, let alone the Cretaceous.
Still warming after all these years