Guest Post by Willis Eschenbach
The discussion of the 1998 Mann “Hockeystick” seems like it will never die. (The “Hockeystick” was Dr. Michael Mann’s famous graph showing flatline historical temperatures followed by a huge modern rise.) Claims of the Hockeystick’s veracity continue apace, with people doggedly wanting to believe that the results are “robust”. I thought I’d revisit something I first posted and then expanded on at ClimateAudit a few years ago, which are the proxies in Michael Mann et al.’s 2008 paper, “Proxy-based reconstructions of hemispheric and global surface temperature variations over the past two millennia” (M2008). This was another salvo in Mann’s unending attempt to revive his fatally flawed 1998 “Hockeystick” paper. I used what is called “Cluster Analysis” to look at the proxies. Cluster analysis creates a tree-shaped structure called a “dendrogram” that shows the similarity between the individual datasets involved. Figure 1 shows the dendrogram of the 95 full-length proxies used in the M2008 study:
Figure 1. Cluster Dendrogram of the 95 proxies in the Mann 2008 dataset which extend from the year 1001 to 1980. The closer together two proxies are in the dendrogram, the more similar they are. Absolute similarity is indicated by the left-right position of the fork connecting two datasets. The names give the dataset abbreviation as used by Mann2008, the type (e.g. tree ring, ice core) the location as lat/long, the name of the princiipal investigator, and if tree rings the species abbreviation (e.g. PIBA, PILO).
What can we learn from this dendrogram showing the results of the cluster analysis of the Mann 2008 proxies?
First let me start by describing how the dendrogram is made. The program compares all possible pairs of proxies, and measures their similarity. It selects the most similar pair, and draws a “fork” that connects the two.
Take a look at the “forks” in the dendrogram. The further to the left the fork occurs, the more similar are the pairs. The two most similar proxy datasets in the whole bunch are ones that are furthest to the left. They turn out to be the Tiljander “lightsum” and “thicknessmm” datasets.
Once these two are identified, they are then averaged. The individual proxy datasets are replaced by the average of the two. Then the procedure is repeated. This time it compares all possible remaining pairs, including the average of the first two as a single dataset. Again the most similar pair is selected, marked with a “fork” (slightly to the right of the first fork), and averaged. In the dataset above, the most similar pair is again among the Tiljander proxies. In this case, the pair consists of the “darksum” proxy on the one hand, and the average of the two Tiljander proxies from the first step on the other hand. These two are then removed and replaced with their average.
This procedure is repeated over and over again, until all of the available proxies have been averaged together and added to the dendrogram and it is complete.
In this case, the clustering is clearly not random. In general a cluster is composed of measurements of similar things in a single geographical area (e.g. Argentinian Cypress tree rings). In addition, the proxies tend to cluster by proxy type (e.g. speleothems and sediments vs. tree rings).
Next, the dendrogram can be read from the bottom up to show which groups of proxies are most dissimilar to the others. The more outlying and more unusual group a group is, the nearer it is to the top of the dendrogram.
Next, note that many of the groups of proxies are much more similar to each other than they are to any of the other proxies. In particular the bristlecone “stripbark pines” end up right at the top of the dendrogram, because they are the most atypical group of the lot. Only when there is absolutely no other choice are the bristlecones at the top of the dendrogram added to the dendrogram.
So how does this type of analysis clarify whether the “Hockeystick” is real? The question at issue all along has been, is the “hockeystick” shape something that can be seen in a majority of the proxies, or is it limited to a few proxies? This is usually phrased as whether the results are “robust” to the removal of subsets of the proxies. And as usual in climate science, there are several backstories to this question of “robustness”.
The first backstory on this question is that well prior to this study, the National Research Council (2006) “Surface Temperature Reconstructions for the Last 2,000 Years” recommended that the bristlecone and related “stripbark” pines not be used in paleotemperature reconstructions. This recommendation had also been made previously by other experts in the field. The problem for Mann, of course, is that the hockeystick signal doesn’t show up much when one leaves out the bristlecones. So like a junkie unable to resist going back for one last fix, Dr. Mann and his adherents have found it almost impossible to give up the bristlecones.
The next backstory is that a number of the bristlecone proxy records collected by Graybill have failed replication, as shown by the Ababneh Thesis. Not only that, but one of the authors of M2008 (Malcolm Hughes) had to have known that, because he was on her PhD committee … so the M2008 study used proxies that were not only not recommended for use, but proxies not recommended for use that they knew had failed replication. Bad scientists, no cookies.
The final backstory is that the Tiljander proxies a) were said by the original authors to be hopelessly compromised in recent times and who advised against their use as temperature proxies, and b) were used upside-down by Mann (what he called warming the proxies actually showed as cooling).
With all of that as prologue, Figure 2 shows the average signals of the clusters of normalized proxies (averaged after each proxy is normalized to an average of zero and a standard deviation of one). See if you can tell where the Hockeystick shaped signal is located …
Figure 2. Left column shows average signals of the clusters of proxies shown in Figure 1, from the year 1001 to 1980. Averages are of the cluster to which each is connected by a short black line.
You can see the problems with the various Tiljander series, which are obviously contaminated … they go off the charts in the latter part of the record. In addition, if the Tiljander data were real it would be saying record cold, not record hot, but the computational method of Mann et al. flipped it over.
The reason for the unending addiction of Mann and his adherents to certain groups of proxies becomes obvious in this analysis. The hockeystick shape is entirely contained in a few clusters—the Greybill bristlecones and related stripbark species, the upside-down Tiljander proxies, and a few Asian tree ring records. The speleothems and lake sediments tell a very different story, one of falling temperatures … and in most of the clusters, there’s not much of a common signal at all. Which is why the attempts to rescue the original 1998 “hockeystick” have re-used and refuse to stop re-using those few proxies, proxies which are known to be unsuitable for use in paleotemperature reconstructions. They refuse to stop recycling them for a simple reason … you can’t make hockeysticks without those few proxies.
To sum up. Is the mining of “hockeystick” shaped climate reconstructions from this dataset a “robust” finding?
Not for me, not one bit. While you can get a hockeystick if you waterboard this data long enough, the result is a chimera, a false result of improper analysis. The hockeystick shaped signal is far too localized, and occurs in far too few of the clusters, to call it “robust” in any sense of the word.
w.
PS – The entire saga of the Ababneh Thesis, along with lots and lots of other interesting information, is available over at ClimateAudit. People who want to improve their knowledge about things like the proxy records and the Climategate FOI requests and the whole climate saga should certainly do their homework at ClimateAudit first … because in the marvelous world of Climate Science, things are rarely what they seem.
[UPDATE] Some commenters asked for the data, my apologies for not providing it. It is located at the NOAA Paleoclimate repository here.
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Very nice explanation. The visuals are really helpful. Took me a while to get what you meant by further left, but it all makes sense now. The further left you drew the fork, the closer the match. It’s not just the fork itself (as I had thought), but ALSO how far to the left it is drawn that represents a closer match.
The graphs next to each group is extremely clear.
BTW, I’m a programmer. What kind of dataset is this? Is it sparse data? Point data? How are different graphs compared? Are their offsets automatically adjusted to match when doing comparisons? What about scale? I’d like to learn more about this. I might be interested in creating some really nice graphs as I have some really good drawing tools I created that can draw pretty much anything.
Oh, and where can I get the data?
The Tiljander screw up is utterly amazing. If you want to see how absurd the Team is, study it over at Climate Audit. In a nutshell, the sediment layers in a lake are correlated with temperature. However, after a certain date, sediments from construction contaminated the record. Qualitatively, looking at the sediment record using scientific knowledge, the sediment record shows massive COOLING after that date. The author knew that the sediment record was worthless after that date. However Mann incorporated this record into his chronology model. The model was “tuned” with modern temperature measurements. So the model used the inverse sign for the correlation coefficients, i.e. the data is “upside down”. Anyhow, mathematically the model glommed on to the massive “signal” which “matched” the slight temperature rise in modern times. The effect was that the MWP was flattened out and you got a hockeystick.
It really is this preposterous. Throw out Tiljander, the one tree in Yamal, and the strip bark trees, and you get a nice peak during the MWP, and a warming in current records up to about 2005.
Clear and useful analysis, Mr. Eschenbach — for those who are inclined to analyze.
But tell me this:
Would any analysis, criticism, logical argument or factual evidence help to persuade and bring around those who firmly believe in their irrational ideology, sacred book or mock-scientific dogma? Or those who derive their livelihood from these lies?
My point is, our most important and immediate task is to find effective practical ways and means — financial, organizational, and legal — to overcome the nascent green faith, to deprive it of political support, to take away its access to public funds, and — which is absolutely necessary! — to see that the most active fraudsters stand trial and go to jail.
Scientific bankruptcy of the green scaremongering is obvious not only to us but to its preachers themselves. The most influential priests of “man-made climate change” scare are smiling when they see us debunking their swindle in our blogs. While we are at it, they do the real thing, making political connections, finding rich sponsors, controlling professional and mainstream magazines and associations, dominating in academic institutions and international organizations, incessantly brainwashing the masses with total impunity.
They have money and power. We have none. Money and power are what we need to extinguish this poisonous source of lies before the whole world becomes one faceless, Chinese-style dictatorship spewing pious propaganda, gagging all dissidents, and keeping the large minority of working people under control by feeding products of their ingenuity and labor to the majority of parasites.
In essence, to fight off green lies, we must radically change the structure of democracy. In its present form, there will be soon no real difference between what we call “democracy” in the United States, and what they mean by “democracy” in Syria, Russia, and China.
I don’t see my comment.
What I try to post it again, I see the “duplicate comment detected” message.
Which is a pity — I spent some time trying to lay out my thoughts clearly.
[Reply: Very sorry about that. The only thing in the spam folder was “penis extender”. I don’t think that was your post. WordPress drops comments occsionally, so it’s best to keep a copy until you see yours posted. ~dbs, mod.]
Very tidy. When you’re good, you are very good.
If a picture (or a diagram) was ever worth a thousand words, it is this one.
As always, way to blow the lid off it, Willis.
Chris
Norfolk, VA, USA
Wilis
With regard to the Ababneh Thesis, as with any PhD dissertation, there is should be an embargo on the use of the material by anyone else for a year or two after it is presented to the examiners by the student. The student has sole right to publish the results during that period. After the end of that period the results are available for others to use so long as proper attribution is made. Has anyone examined the question of when the embargo period is due to end?
It pays to look at the source data. Доверяй, но проверяй!
gnomish said: “Very tidy. When you’re good, you are very good”
Mae West: “When I’m good, I’m good, when I’m bad I’m better”.
Sounds a bit like dubious climate research claims
Willis,
Many thanks for your reply to my comments re presumed upside down use by Mann et al of the Tiljander proxies. I followed your suggestion and found considerable information is available at climateaudit.org. Particularly interesting is the mention there of a grudging acknowledgement contained in a “draft Corrigendum” that they had used the Tiljander proxies upside down, but apparently the “Team” was not convinced. Maybe the horse didn’t like the murky, tainted water so it wouldn’t drink.
But in pursuing the spirit of issuing corrigenda where needed, I must confess to a couple of errors in my original posting. Sorry about my typo — the Tiljander data wasn’t “haplessly compromised”, it was “hopelessly compromised”. Also, on rechecking the news story about Gov. Christie’s proposed dumping of NJ’s destructive RGGI cap and trade, I found that the amount stated in the news story was not $3.50, it was “… $3.24 on average toward RGGI on their utility bills.”
There was no billing period defined in the news story, but since it clearly stated “bills” in plural form, I assumed this meant monthly. After digging up the actual rate involved, I now believe the period should have been stated as “yearly payments” instead of just “bills.” I sincerely regret my unintentional misinformation.
Bob
RE: carol smith: (May 30, 2011 at 1:59 pm)
“When I first saw the hockey stick alarm bells started to ring.”
I second the motion. This (combined with the revelation of the logarithmic nature of carbon dioxide warming) is when CO2 Global Warming jumped the shark for me. At first, I tried to give them the benefit of doubt, but I found too many indications that the Medieval Warm Period and the Little Ice Age were global events. It is too bad that your average news journalist does not seem to understand what this fuss is all about. For many people, these numbers and charts are meaningless–all they see are pictures of smoke spewing into the ‘fragile’ atmosphere.
Great explanation Willis!
Am I right in assuming that if each of the 12 proxy groups was given equal weighting and combined (eg 1/12 each) the resultant average would be a cooling trend, especially with the Tiljander set inverted?
I wonder what makes the first Greybill bristlecone & related pine SW USA group so different, when the two later groups seem to show almost no “warming”?
The Hockey Stick Illusion
The whole sorry saga is dealt with in detail in Andrew Montfords’s book that appears on this site and is available from Amazon.
It explains how the HS shape is entirely dependent on a few defective series, the algorithm devised by Mann literally hunts for HS shapes and Mann’s acrobatic filibustering and denial.
Why note buy a copy and support Bishop Hill.
Paul
Mann Made Global Warming
Another excellent addition to the Willis index. It’s also interesting that if one single tree – YAD061 – was deleted, there would be no alarming chart that Keith Briffa used to fabricate a fictitious hockey stick.
Mann knowingly used a corrupted proxy [Tiljander]. But he had been informed before he published that the Tiljander proxy was no good due to road construction, which had overturned the sediment layer. Mann deliberately used Tiljander’s proxy anyway, because it gave him the coveted hockey stick shape. Is that scientific misconduct, or what?
DBS, thank you, my post appeared.
Sometimes there’s an inexplicable delay, which leads to unnecessary complaints and/or repeats. I understand that this is rather beyond your control.
From the article
How is this comparison made?
Thanks Willis. This was my general feel about the subject, but it helps that you itemized it. I read Ababneh’s paper and saw that the non-strip bark trees showed no hockey stick shape at all. In fact, the data was rather flat. I also remember that McIntyre had found that not all of the samples that Graybill had collected had been archived – making it look like there might have been some cherry picking on Graybill’s part. Graybill’s objective was to show CO2 feeding when he collected those samles. Furthermore, the weighing that Mann did also served to increase the hockey stick shape.
A couple of things that I’m still unclear about. What was Mann’s explanation for using the Tiljander data upside down?
The Misc. Asian tree rings do seem to give a bit of a hockey stick shape. Has anyone looked at those?
Rob R: “Has anyone examined the question of when the embargo period is due to end?”
Ababneh’s paper came out in 2006. It seems like Mann should have both known and had access to it for his 2008 paper. Ababneh shows very clearly that the hockey stick rise at the end is a split bark phenomena. She broke her trees into groups that had gone split bark and those that had not. All the trees that had gone split bark showed and acceleration in tree ring growth. Likely, the parts of the tree that died left more nutrients and structure available for the parts that did not die. And of course core samples were pulled from the living bark.
Ababneh used more trees and did a more complete analysis than Graybill. But the fact that Mann is unwilling to update all his old work by replacing the Graybill data with the Ababneh data shows clearly that Mann has zero interest in producing good science. His entire career is dedicated to his personal and political agenda – science be damned.
The ideological grandfather of the hockey stick is in the 1960 cover article
of Science, 4 November 1960, Vol. 132 #3436 pp. 1291-1295
by von Foerster “Doomsday: Friday 13 November, A.D. 2026”
I was in high school and read Science every week
and vividly remember this population-explosion screed.
And yes, it’s every bit as shoddy as its equally wretched AGW descendents.
The cover has a graph from the article, and thus is the original Hockey Stick.
Another bet: all the ClimateGate conspirators have seen and adored this graph .
Irony: the Doomsday Collision of the NEO asteroid Apophis
(if it Keyholes on its close pass on Friday the 13th April 2029)
will be on Friday the 13th of April 2036.
Willis:
This is the simplest, most clear explanation of the HS phenomena that I have seen.
Hats off to you and keep them coming….
“Jimbo says:
May 30, 2011 at 1:55 pm
Here is a sceptics intro to dendrochronology and data keeping.
http://uts.cc.utexas.edu/~wd/courses/373F/notes/lec20den.html
http://scienceblogs.com/aardvarchaeology/2009/06/open_source_dendrochronology.php”
Jimbo,
Your links rock (as usual – you know a lot about this stuff!). I checked out the second one and reading a comment came up with this link:
http://web.utk.edu/~grissino/
I haven’t yet delved into this site much, but it was mentioned as a good source for dendro. Any comments and insights about the site or about Dr. Henri Grissino-Meyer?
bob paglee says:
May 30, 2011 at 3:03 pm
If it’s any consolation, RGGI’s cost to consumers is totally unclear. Here in New Hampshire, on power producer has come up with estimates of $0.065 cents per month and also $0.36 per month.
From the $10 million “returned” to NH in 2010 and the some 500,000 households in NH, I figure the total impact is $10/year at the current low prices per allowance. (People had hoped for 5-10X the current $1.89.) Even so, the major defense for RGGI in NH has been economic. The program has only been in effect for a few years but the recipients of the funds are already very, very attached to them.
Note – my “total impact” includes what you pay to support power bills to your employer, grocery store, traffic lights, etc.
See more at http://wermenh.com/rggiwatch/finance_notes.html
Leif Svalgaard says:
May 30, 2011 at 1:29 pm
The program compares all possible pairs of proxies, and measures their similarity.
How is ‘similarity’ measured in your analysis?
———————————————————————————
The silence is deafening. Can we have an answer please???
BTW—Fantastic work Willis. You will get a Whistleblower fee from all the money we’ll save from abandoning the CO2 imbargo Industry!! But I’m worried Lief has spotted something in your analysis.
Gary says:
May 30, 2011 at 1:30 pm
Yes to both. One caveat, some data was linearly interpolated by Mann et al. to yearly measurements from decadal measurements.
No clue. In general trees are not good thermometers. In dry zones they can be passable rain gauges, but thermometry is generally beyond their vocal range, with loads of unmeasurable confounding variables.
w.