NOTE: This has been running two weeks at the top of WUWT, discussion has slowed, so I’m placing it back in regular que. – Anthony
UPDATES:
Statistician William Briggs weighs in here
Eduardo Zorita weighs in here
Anonymous blogger “Deep Climate” weighs in with what he/she calls a “deeply flawed study” here
After a week of being “preoccupied” Real Climate finally breaks radio silence here. It appears to be a prelude to a dismissal with a “wave of the hand”
Supplementary Info now available: All data and code used in this paper are available at the Annals of Applied Statistics supplementary materials website:
http://www.imstat.org/aoas/supplements/default.htm
=========================================
Sticky Wicket – phrase, meaning: “A difficult situation”.
Oh, my. There is a new and important study on temperature proxy reconstructions (McShane and Wyner 2010) submitted into the Annals of Applied Statistics and is listed to be published in the next issue. According to Steve McIntyre, this is one of the “top statistical journals”. This paper is a direct and serious rebuttal to the proxy reconstructions of Mann. It seems watertight on the surface, because instead of trying to attack the proxy data quality issues, they assumed the proxy data was accurate for their purpose, then created a bayesian backcast method. Then, using the proxy data, they demonstrate it fails to reproduce the sharp 20th century uptick.
Now, there’s a new look to the familiar “hockey stick”.
Before:
After:
Not only are the results stunning, but the paper is highly readable, written in a sensible style that most laymen can absorb, even if they don’t understand some of the finer points of bayesian and loess filters, or principal components. Not only that, this paper is a confirmation of McIntyre and McKitrick’s work, with a strong nod to Wegman. I highly recommend reading this and distributing this story widely.
Here’s the submitted paper:
(PDF, 2.5 MB. Backup download available here: McShane and Wyner 2010 )
It states in its abstract:
We find that the proxies do not predict temperature significantly better than random series generated independently of temperature. Furthermore, various model specifications that perform similarly at predicting temperature produce extremely different historical backcasts. Finally, the proxies seem unable to forecast the high levels of and sharp run-up in temperature in the 1990s either in-sample or from contiguous holdout blocks, thus casting doubt on their ability to predict such phenomena if in fact they occurred several hundred years ago.
Here are some excerpts from the paper (emphasis in paragraphs mine):
This one shows that M&M hit the mark, because it is independent validation:
In other words, our model performs better when using highly autocorrelated
noise rather than proxies to ”predict” temperature. The real proxies are less predictive than our ”fake” data. While the Lasso generated reconstructions using the proxies are highly statistically significant compared to simple null models, they do not achieve statistical significance against sophisticated null models.
We are not the first to observe this effect. It was shown, in McIntyre
and McKitrick (2005a,c), that random sequences with complex local dependence
structures can predict temperatures. Their approach has been
roundly dismissed in the climate science literature:
To generate ”random” noise series, MM05c apply the full autoregressive structure of the real world proxy series. In this way, they in fact train their stochastic engine with significant (if not dominant) low frequency climate signal rather than purely non-climatic noise and its persistence. [Emphasis in original]
Ammann and Wahl (2007)
…
On the power of the proxy data to actually detect climate change:
This is disturbing: if a model cannot predict the occurrence of a sharp run-up in an out-of-sample block which is contiguous with the insample training set, then it seems highly unlikely that it has power to detect such levels or run-ups in the more distant past. It is even more discouraging when one recalls Figure 15: the model cannot capture the sharp run-up even in-sample. In sum, these results suggest that the ninety-three sequences that comprise the 1,000 year old proxy record simply lack power to detect a sharp increase in temperature. See Footnote 12
Footnote 12:
On the other hand, perhaps our model is unable to detect the high level of and sharp run-up in recent temperatures because anthropogenic factors have, for example, caused a regime change in the relation between temperatures and proxies. While this is certainly a consistent line of reasoning, it is also fraught with peril for, once one admits the possibility of regime changes in the instrumental period, it raises the question of whether such changes exist elsewhere over the past 1,000 years. Furthermore, it implies that up to half of the already short instrumental record is corrupted by anthropogenic factors, thus undermining paleoclimatology as a statistical enterprise.
…
We plot the in-sample portion of this backcast (1850-1998 AD) in Figure 15. Not surprisingly, the model tracks CRU reasonably well because it is in-sample. However, despite the fact that the backcast is both in-sample and initialized with the high true temperatures from 1999 AD and 2000 AD, it still cannot capture either the high level of or the sharp run-up in temperatures of the 1990s. It is substantially biased low. That the model cannot capture run-up even in-sample does not portend well for its ability
to capture similar levels and run-ups if they exist out-of-sample.
…
Conclusion.
Research on multi-proxy temperature reconstructions of the earth’s temperature is now entering its second decade. While the literature is large, there has been very little collaboration with universitylevel, professional statisticians (Wegman et al., 2006; Wegman, 2006). Our paper is an effort to apply some modern statistical methods to these problems. While our results agree with the climate scientists findings in some
respects, our methods of estimating model uncertainty and accuracy are in sharp disagreement.
On the one hand, we conclude unequivocally that the evidence for a ”long-handled” hockey stick (where the shaft of the hockey stick extends to the year 1000 AD) is lacking in the data. The fundamental problem is that there is a limited amount of proxy data which dates back to 1000 AD; what is available is weakly predictive of global annual temperature. Our backcasting methods, which track quite closely the methods applied most recently in Mann (2008) to the same data, are unable to catch the sharp run up in temperatures recorded in the 1990s, even in-sample.
As can be seen in Figure 15, our estimate of the run up in temperature in the 1990s has
a much smaller slope than the actual temperature series. Furthermore, the lower frame of Figure 18 clearly reveals that the proxy model is not at all able to track the high gradient segment. Consequently, the long flat handle of the hockey stick is best understood to be a feature of regression and less a reflection of our knowledge of the truth. Nevertheless, the temperatures of the last few decades have been relatively warm compared to many of the thousand year temperature curves sampled from the posterior distribution of our model.
Our main contribution is our efforts to seriously grapple with the uncertainty involved in paleoclimatological reconstructions. Regression of high dimensional time series is always a complex problem with many traps. In our case, the particular challenges include (i) a short sequence of training data, (ii) more predictors than observations, (iii) a very weak signal, and (iv) response and predictor variables which are both strongly autocorrelated.
The final point is particularly troublesome: since the data is not easily modeled by a simple autoregressive process it follows that the number of truly independent observations (i.e., the effective sample size) may be just too small for accurate reconstruction.
Climate scientists have greatly underestimated the uncertainty of proxy based reconstructions and hence have been overconfident in their models. We have shown that time dependence in the temperature series is sufficiently strong to permit complex sequences of random numbers to forecast out-of-sample reasonably well fairly frequently (see, for example, Figure 9). Furthermore, even proxy based models with approximately the same amount of reconstructive skill (Figures 11,12, and 13), produce strikingly dissimilar historical backcasts: some of these look like hockey sticks but most do not (Figure 14).
Natural climate variability is not well understood and is probably quite large. It is not clear that the proxies currently used to predict temperature are even predictive of it at the scale of several decades let alone over many centuries. Nonetheless, paleoclimatoligical reconstructions constitute only one source of evidence in the AGW debate. Our work stands entirely on the shoulders of those environmental scientists who labored untold years to assemble the vast network of natural proxies. Although we assume the reliability of their data for our purposes here, there still remains a considerable number of outstanding questions that can only be answered with a free and open inquiry and a great deal of replication.
===============================================================
Commenters on WUWT report that Tamino and Romm are deleting comments even mentioning this paper on their blog comment forum. Their refusal to even acknowledge it tells you it has squarely hit the target, and the fat lady has sung – loudly.
(h/t to WUWT reader “thechuckr”)
And sea levels are not rising as predicted. Journal of Geophysical Research – Oceans, 15th August 2010
http://www.agu.org/pubs/crossref/2010/2009JC005630.shtml
So how many wheels are left on the (band)wagon?
Many are obviously missing the main point. The study is not intended to indicate that its predictions are BETTER only that making accurate predictions of any true significance is not possible even using their bayesian method (and though not stated specifically, any other method for that matter). Too many potential causal variables which are themselves intercorrelated and really of unknown past and future value. Time series assumes that the only real causal variable is the continuation of time.
two moon,
I like your comment overall.
However, your “M&W are not scientists and their point is not scientific” is misleading in that it misrepresents by oversimplification the overall area of relationships between of all the disciplines of study that are called sciences.
It seems that a discipline of study like statistics (a part of applied mathematics) that can audit and validate the theories of a physical science (such as the physics, chemistry and biology that constitute climate science) is a part of science itself. If part of science then it is not “not-science”.
I do not think we need to go further into the history of the science of the philosophy of science in this post. That would be OT, therefore a separate post.
John
James Sexton at 11.14 – absolutely spot on!
This paper clearly shows that any paper, publication or policy statement which cites MBH98 and bases its conclusions on MBH98 is discredited and falsified. A fact which is apparently not lost on Anthony, which is one reason I am sure that he has made this article a ‘sticky’ remaining at the top of the website for some days.
Hilary Barnes says:
August 16, 2010 at 9:22 am
Excellent and exciting post, but headline is a bit of a mess: no wickets in hockey, let alone the ice hockey of the hockey stick debate. You’re thinking of cricket, when the wicket may be sticky, but the bat is always straight!
—-Reply:
That’s ok, Hillary. The “sticky wicket” applies to ice hockey about as much as the hockey stick interpretation applies to climate science–that is, not at all.
“If you ask me as a person, do I think the Russian heat wave has to do with climate change, the answer is yes,” said Gavin Schmidt, a climate researcher with NASA in New York. “If you ask me as a scientist whether I have proved it, the answer is no — at least not yet.”
This simply shows Gavin’s personal bias and predisposition to a particular outcome. A malaise that inflicts many of his cohorts likely causing them to suffer from extreme confirmation bias.
-Yet the TREND is clearly upward Ron. And the observed data correlates with the modeled data where it exists.
You need to pay attention. Two clearly bogus proxies have been left in the reconstruction. The paper is mostly about the efficacy of using proxies to model temps.
evanmjones says:
August 15, 2010 at 9:18 pm
Well, geo, as I have commented in the past, for every $billion wasted (or never produced) anywhere in the world, babies starve somewhere in the world……
____________________________________________________________
What bothers me the most about CAGW and the “precautionary principle” argument is that all those who use it leave out the OTHER half of the equation. It is even stated in a Warmist peer reviewed paper:
Lesson from the past: present insolation minimum holds potential for glacial inception
“Because the intensities of the 397 ka BP and present insolation minima are very similar, we conclude that under natural boundary conditions the present insolation minimum holds the potential to terminate the Holocene interglacial. Our findings support the Ruddiman hypothesis [Ruddiman, W., 2003. The Anthropogenic Greenhouse Era began thousands of years ago. Climate Change 61, 261–293], which proposes that early anthropogenic greenhouse gas emission prevented the inception of a glacial that would otherwise already have started….”
On top of that Woods Hole Oceanographic Institution states:
“Most of the studies and debates on potential climate change, along with its ecological and economic impacts, have focused on the ongoing buildup of industrial greenhouse gases in the atmosphere and a gradual increase in global temperatures. This line of thinking, however, fails to consider another potentially disruptive climate scenario. It ignores recent and rapidly advancing evidence that Earth’s climate repeatedly has shifted abruptly and dramatically in the past, and is capable of doing so in the future.
Fossil evidence clearly demonstrates that Earthvs climate can shift gears within a decade….
But the concept remains little known and scarcely appreciated in the wider community of scientists, economists, policy makers, and world political and business leaders. Thus, world leaders may be planning for climate scenarios of global warming that are opposite to what might actually occur…“
As far as I am concerned neglecting a possible change towards a COOLING world is down right criminal negligence – my biggest gripe with CAGW. We are so busy watching the yapping little poodle we can not see the mammoth that just walked into the room.
If politicians and scientists were really concerned about the welfare of people instead of the welfare of their wallets we would be seeing great strides made in converting most of the world towards nuclear power ASAP.
Re: Ben U
“It’s not unusual in science to work on a theory that one knows not be in full accordance with reality, if one has no alternative theories nearly as good. The least bad scientific theory often gets to get worked on. People work both inside the box and outside the box of the theory, in hopes of ending up with a better theory.”
I hear you and agree in principal though sometimes you have to accept that you just know too little to have any theory whatsoever. I believe that this case is also a little different since it does not refer to a theory but to a method for fishing out temperatures of the past. If you believe as I do that this method is crap then all interpretations of past climate using this method are also invalidated.
I agree that in order to further the science one should optimally come up with a new method for fishing out past temperature that is better. In lack of such innovative skills I do think however that admitting the crapiness of the mentioned method and disregarding the message that has been distributed by using it allows for previously documented archeological findings to once again be the best knowledge to date. By this I mean the previously melted glaciers in the Alps etc. For people claiming this to be a regional European event I would very much like to see a sense making theory for how a regional effect made it possible to farm on Greenland for a couple of hundred years.
Sticky-wiki?
I find the sticky entry to be extremely annoying; it’s way too annoying to look for new items this way… 🙁
REPLY: Wow that’s a new one, use of the scroll bar is too annoying. The “annoying” story comes off the top tomorrow, until then, use of the scroll bar will be required. – Anthony
Interesting topic and reactions, especially at Tamino where there seems to be a lack of an “Open Mind” amongst the regulars.
On another note, it would seem Gavin has disappeared his original response to this at RC. The comment was there yesterday, now it is gone. As if the topic had never been mentioned.
Thoughts?
James Sexton quoted:
It is an unfortunate fact that, most likely, this is completely true. And it’s not just hurling insults to point out that the discipline of “climate scientist” does not appear to include high quality data selection, collection or processing.
Mann, to me, aspires to be the Ringo Starr of climate science, but alas, has only risen to the level of Justin Beiber’s drummer.
This tale is slightly off-topic but it illustrates how politics and statistics can collide.
Sixty years ago cases of Gram negative (G-) septicemia were associated with a 50% mortality. Today, with all our advances in antimicrobial therapy, the mortality remains at about 50%. What kills is not the infection, per se, but rather exotoxins that G- bacteria release which initiate a cascade of events in the body that lead to end organ failure and death. All G- bacteria share a common J-chain which acts as a toxin. Some smart folks figured out how to develop a human-mouse chimeric antibody which binds the G- common J-chain toxin. This product came within a gnat’s hair of being a new, approved drug for the treatment of G- sepsis. The manufacturer had already prepared marketing pieces and had hired a sales force.
The problem was that the drug would only be efficacious in cases of G- sepsis. It would be utterly ineffective in cases of G+ sepsis, viral infections or fungemia. What’s more, the drug would cost $4,000 a dose. Our diagnostic acumen was not sufficiently developed to immediately differentiate between G-, G+, viral or fungal causes of sepsis (and still isn’t). Here’s the hitch…most cases of G- sepsis occur in patients > 65 (i.e. Medicare age). Further, the drug only conferred a 10% improvement in mortality. The government went into panic mode. If approved, this drug could cost well over $1,000,000 per life saved…maybe as much as $2,000,000 or more.
So the FDA responded. On a mere technicality (end point of death at 30 days rather than 31 days), they insisted that that the manufacturer run their studies again. This was enormously expensive and difficult as it required that study subjects be drawn from hundreds of sites and that G- sepsis be confirmed before therapy was initiated (and, of course, the study must be double blinded so there was always a placebo group). It turned out badly for the manufacturer. The second study was halted half way through when the drug could not be shown to be efficacious and this promising drug was essentially dead.
Statistics killed it. I actually believed (and still believe) that this drug could theoretically save lives. But when do you use it and at what cost to society? This story reminds me of CO2 mitigation. MAYBE it could prevent some future warming, but at what cost?
Perhaps a more important lesson in this case is the need to challenge and retest methodologies. The manufacturer of this drug (which went on to make a fortune on other products) followed all the “rules” to bring a drug to market. Their study design and statistical analysis were largely above reproach…yet they were still wrong. Now, the government had a financial interest in not allowing this drug to be brought to market (it could have been devastating to Medicare), but all it took to kill it was to insist they run their experiment again. The favorable results could not be replicated.
Today we have a similar situation with CAGW except that the financial incentives have been reversed. Government makes out like a bandit under CO2 mitigation schemes, but for how much “good” and at what cost to society? The M&W paper illustrates why it is important to insist that “conclusions” be restudied.
I apologize if this comment is long and rambling, but the moral of the story is that at the end of the day good statistics are our friends and, if unchallenged, the improper interpretation of bad statistics can lead to financial ruination with no net societal gain.
BPW says:
August 16, 2010 at 11:52 am
Interesting topic and reactions, especially at Tamino where there seems to be a lack of an “Open Mind” amongst the regulars.
On another note, it would seem Gavin has disappeared his original response to this at RC. The comment was there yesterday, now it is gone. As if the topic had never been mentioned.
Thoughts?
—–Reply:
Hear no logic, see no logic, speak no logic. (Or substitute “truth” for “logic” and the same applies.)
Jaye Bass says:
August 16, 2010 at 10:55 am
“You need to pay attention. Two clearly bogus proxies have been left in the reconstruction. The paper is mostly about the efficacy of using proxies to model temps.”
Agreed. I’m wondering if what we are seeing here isn’t something worth studying itself? It seems an overwhelming majority of the alarmists seem only to look at the pretty pictures and assume a story from there, discerning nothing from the colors of the graph or even noting the “high water” marks about the year 1100. They don’t seem to even read the caption under the fig. 16. and either don’t know what to make of fig 15 or simply can’t see it. So, my question is this;
Is this why there are so many CAGW believers out there? Do they simply look at the pictures and let someone else tell them what it means or is it left to them to make wild assumptions to the meaning of the pretty pictures without actually reading an explanation? It is true some skeptics did the same, but not many and once explained what it means, they endeavored to READ THE DAMNED PAPER!!!!
It is a small wonder MSNBC thought they could put a penguin with a polar bear and get away with it. All they have to do is photo-shop something and let the believers imagination tell the story. Dear God! Has it come to this? Has this world, as the majority of the population, capitulated its God given right to contemplate and even think for itself? I guess I now know why when I make a reference to 1984 I get very few responses from the alarmists. There aren’t any pictures to assume a story. I’m simply flabbergasted at the either willful refusal to read or the inability to accomplish the reading.
Dr Dave,
Excellent analogy and relevant to the context of M and W.
Marc Blank says:
I find the sticky entry to be extremely annoying; it’s way too annoying to look for new items this way… 🙁
===================
I use the page down button (2 taps to reach recent comments, three for links) 🙂
@ur momisugly davidc says:
August 15, 2010 at 3:07 pm
From New York Times:
“If you ask me as a person, do I think the Russian heat wave has to do with climate change, the answer is yes,” said Gavin Schmidt, a climate researcher with NASA in New York. “If you ask me as a scientist whether I have proved it, the answer is no — at least not yet.”
Is this the first time Gavin has expressed anything but total certainty?
And elsewhere in the article nytimes says:
‘Seemingly disconnected, these far-flung disasters [floods, fires] are reviving the question of whether global warming is causing more weather extremes’
Well, the Russians themselves aren’t buying it, but I guess Gavin doesn’t think they’re aware of what’s going on in their own country. The fires were caused by poor forest management, not global warming.
http://notrickszone.com/2010/08/12/russian-scientist-extreme-central-russian-heat-wave-not-an-indication-of-a-future-climate-change/
The original article is in German.
John Whitman: Point taken. Thank you.
New Zealand sceptics to challenge climate science in court http://www.abc.net.au/news/stories/2010/08/16/2984597.htm?section=justin
The kiwi’s don’t f**k around, man.
Off topic:
Met office acknowledges problems with siting of weather stations…
‘Putting measuring instruments on your roof isn’t technically the best place to have them because they might absorb more sunlight and therefore record a temperature a few degrees hotter than it actually is.’
Read more: http://www.mailonsunday.co.uk/news/article-1303400/One-man-weather-centre-Simon-Cansick-accurate-farmers-snubbing-Met-Office.html#ixzz0wnk4TMcY
BPW: August 16, 2010 at 11:52 am
On another note, it would seem Gavin has disappeared his original response to this at RC. The comment was there yesterday, now it is gone. As if the topic had never been mentioned.
Thoughts?
He was probably rushed and inadvertently deleted himself…
2010. Blakeley B. McShane and Abraham J. Wyner.
“Natural climate variability is not well understood and is probably quite large”.
1950: Charles Ernest Pelham Brooks
The weather of one year differs from that of another year, the weather of one decade from that of another decade ; why should not the climate of one century differ from that of another century ?
Please read “Climate Through the Ages”!
http://books.google.co.uk/books?id=4PLu8lIfSFEC
http://people.wku.edu/charles.smith/chronob/BROO1888.htm
The main conclusion is that we do not know. We do not know whether CO2 is the origin of dangerous climate change and we do not know why the climate of one century may differ from that of another century.
We do not know.