Following yesterday’s first look at emails from the 2 year effort to get NOAA to release emails from FOIA requests, where we learn that some scientists felt ‘Hit on the head with a hockey stick’ and that “The paleodata always got a lot more attention from the general public than it deserved.”.
There’s a great article today at SPPI today Michael Mann — the ghost of climate past which summarizes the whole hockey stick affair quite well. It draws heavily on a post from May of this year on PJ Media by Rand Simberg titled The Death of the Hockey Stick?
In that post I found this little nugget below from December 2009, one that I apparently missed in the furor immediately following Climategate1. I’m now correcting that oversight.
What is it? It’s the hockey stick recreated in Excel, using proxy data and instrumental data freely available on the web, something you can easily do at home. I figure the more people that explore this themselves using the easy to follow steps, the more people will understand what a statistical abuse the hockey stick is. The source of this tutorial is as surprising to me as it may be to you, and there’s no special software or secret Mannian Excel plugins needed to do the work yourself. I’ll let Rand Simberg explain.
From his May 2012 essay:
Ultimately, in addition to Mann’s claim for the dramatic recent uptick (which we are supposed to presume was a result of the late industrial revolution and equally dramatic increase in carbon dioxide into the atmosphere as a result of the liberation of carbon from burning long-buried fossil fuels), Keith Briffa of the Climate Research Unit (CRU) at the University of East Anglia in England controversially declared, based on Eurasian data, that the well-documented Medieval Warm Period (MWP), from around 950 to 1250 CE — the European Middle Ages — didn’t actually exist.
This claim was important, if not essential, to Mann’s thesis, because his initial formulation only went back to 1400, the beginning of the so-called Little Ice Age. Critics of the theory thus argued immediately upon its presentation that it shouldn’t be surprising that the earth was warming now, given that we are still coming out of it, and that the medieval warming in the absence of late Carolingian SUVs and coal plants argued that the climate naturally cycled, with no need to invoke Demon Carbon. That is to say, to the degree that the hockey stick has a blade in the twentieth century, it would have another a millennium ago.
The theory has continued to take blows over the years since it was first presented. About a decade ago, a paper was published by Willie Soon and Sallie Baliunis claiming that there was good evidence that both the (still extant) MWP and current warming were driven by solar activity rather than carbon emissions. But these initial attacks were beaten back by the climate mafia (as we now know from the leaked emails between Mann and his partners in crime in East Anglia from two and a half years ago). The real damage came when a retired Canadian mining engineer, Steve McIntyre, and a professor at the University of Guelph, Ross McKitrick, started digging into Mann’s methodology, and found flaws in both his statistical analysis and data interpretation, and published a paper describing them in Geophysical Research Letters in 2005. They showed that Mann’s methodology would generate a hockey stick almost independently of the data input, by feeding it spectral noise. Later, Internet satirist (and apparent statistician by day) Iowahawk provided a primer on how to create a hockey stick at home, using a standard spreadsheet program.
Yep, Iowahawk.
I liked this part of Iowahawk’s “how to” primer the best:
Is there anything wrong with this methodology? Not in principle. In fact there’s a lot to recommend it. There’s a strong reason to believe that high resolution proxy variables like tree rings and ice core o-18 are related to temperature. At the very least it’s a more mathematically rigorous approach than the earlier methods for climate reconstruction, which is probably why the hockey stick / AGW conclusion received a lot of endorsements from academic High Society (including the American Statistical Association).
The devil, as they say is in the details. In each of the steps there is some leeway for, shall we say, intervention. The early criticisms of Mann et al.’s analyses were confined to relatively minor points about the presence of autocorrelated errors, linear specification, etc. But a funny thing happened on the way to Copenhagen: a couple of Canadian researchers, McIntyre and McKitrick, found that when they ran simulations of “red noise” random principal components data into Mann’s reconstruction model, 99% of the time it produced the same hockey stick pattern. They attributed this to Mann’s method / time frame for selecting of principal components.
To illustrate the nature of that debate through the spreadsheet, try some of the following tests:
- Run step 3 through step 7, but only use the proxy data up through 1960 instead of 1980.
- Run step 5 through step 7, but only include the first 2 principal components in the regression.
- Run step 3 through step 7, but delete the ice core data from the proxy set.
- Run step 2 through step 7, but pick out a different proxy data set from NOAA.
Or combinations thereof. What you’ll find is that contrary to Mann’s assertion that the hockey stick is “robust,” you’ll find that the reconstructions tend to be sensitive to the data selection. M&M found, for example, that temperature reconstructions for the 1400s were higher or lower than today, depending on whether bristlecone pine tree rings were included in the proxies.
What the leaked emails reveal, among other things, is some of that bit of principal component sausage making. But more disturbing, they reveal that the actual data going into the reconstruction model — the instrumental temperature data and the proxy variables themselves — were rife for manipulation. In the laughable euphemism of Philip Jones, “value added homogenized data.” The data I provided here was the real, value added global temperature and proxy data, because Phil told me so. Trust me!
I urge readers to replicate it yourselves. Knowledge is power, especially first hand knowledge. Here’s all you need to do it. I’ll be happy to publish what you learn.
Clearly you missed the letter in the Wall Street Journal signed by 49 of those stellar folk telling NASA to knock it off with their climate antics.
Not quite, Tim. He said that he thought the models were useless. It’s worth pointing out that Mann’s “boss” said nothing about GISS data in general, only the models.
That said, I think he’s wrong about transparency, the models, and the observations. In general, the data, processes, and results have been published. While I think that there’s still room for improvement, my experience is that much of the problems with “transparency” actually lies with those who aren’t familiar with the scientific literature.
For instance, Mike’s “trick” to “hide the decline” was widely held up as an example of deception or fraud by climatologists, even though there’d been several papers in big journals explaining use of the “trick”. In the end, the only legitimate complaint was that Mann et al. should have been clearer about the chart which made it into the IPCC report.. but given that this chart was well-explained in the literature, and quite well-publicized, the problem was one of communication, rather than of willful intent to deceive.
Most of the hullabaloo was by people who hadn’t bothered to read the literature, and see that, yes, his “trick” was well-published. It’s dishonest to say that someone’s hiding something if they’ve put it up in plain sight.
Now Mann et al have published a new paper, corrected for the legitimate math errors that McIntyre brought up, and the shape of the graph is still pretty much the same. There’s still a hockey stick. Additionally, scientists have now published a dozen or so other proxy records, from places scattered around the globe, again showing a sharp increase in temperatures in the last 100 years.
Last, while the actual degree of warmth during the MWP is still somewhat up-in-the-air, evidence is pointing towards it being warmer now. Most of the proxy records that are coming out line up with this. (Of course, different records from different places will give different results. The temperature trends of Australia and Greenland are only loosely correlated).
Mark T,
Always willing to be corrected. Are you saying that numbers in a phone book are an example of red noise? That would affect my main point. Are you saying that numbers in a phone book are not random, or are not white noise? Yes, I can see an error on my part there. Is that what you meant?
When people new to this area and trying to figure it out ask me why the models shouldn’t be trusted, I generally give them the simple introductory answer “Because models are just little imitations of the real thing.”
“Right…hmmm…….”
I always love it when skeptics provide complex and comprehensive data and the believers come along and use rhetoric to try and quash it rather than alternative figures. The vast majority of appeals to authority and simple playground insults can only illustrate the utter emptiness of a movement who would rather see poor people freeze to death and third world farmers starve as their food crops are sequestered for biofuel than what could only be a slightly warmer planet. The incredible selective blindness to specific data presented is as telling as the half baked attempts to justify their general claim without ever managing to go into details eg “So you think polluting the planet is a good thing”, can only mean they haven’t a clue.
Even the highest names of all rarely if ever reply using any more than the identical tactics followed by their vast army of lower orders, the best most can manage there are copying and pasting the same links from John Cook often written before proved thoroughly wrong by later material, while such greats as Phil Jones simply says in BBC interviews as 90% of scientists agree then it’s simply not up for discusssion.
So to wind this up to an actual point and punchline. In this long list of extremely valuable comments, including a couple who challenged the piece using arguments better suited to a remedial school, lies one little link I am all too familiar with (and only referred to indirectly by Andrew) showing something which (using a statistical term in its general meaning) has more weight than the others put together. The IPCC kept the medieval warm period until someone removed it. There was never a single claim it was wrong beforehand, so clearly not challenged by them or anyone outside at the time, and quietly vanished as if cutting off a limb and expecting no one would notice it had gone. I would suggest Anthony (may I use the familiar) should devote an entire post to this alone, as (with a background myself in law but a great interest in science), it’s not always the complex and byzantine frauds which undo the criminal, but the schoolboy errors like dropping a fag end at a crime scene covered with DNA or putting stolen money in your own bank account.
Once a crook does something in an unguarded moment which more or less opens the cell door for the state, the other evidence doesn’t really matter, mainly as nearly all criminals by now would have made a full confession, negating the need for a trial and raising all the other complex evidence. Now I’m not saying this shtick alone could derail AGW, if only this was true, but every single warmista I’ve flung this in the face of has skated over it and reverted to generalities ‘you want to kill my unborn grandchildren’ for instance. They can’t defend the data, why not? Because it’s indefensible. I’ve spent over 10 years investigating it and now published my findings. If the general public could see my material in large numbers they would lynch the IPCC and its lackeys around the world’s once esteemed scientific institutions. But this MWP excision is not a smoking gun, it’s a fingerprint. A whacking great one on the murder weapon. And one I’ve never seen a single excuse for despite many attempts to ask for one. Anthony, over to you, please highlight this phenomenon as it’s a biggie.
David Andrew Howard, an excellent reply. So much wisdom can be encapsulated in simple aphorisms.
On another forum I was challenged with the “You want to kill my unborn grandchildren?” type of specious non-sequitur argument.
I replied, “So you believe Jerry Sandusky was innocent?”
As fire drives out fire, so fallacy fallacy. (ack Julius Caesar, Act 3 Scene I)
It would be lovely to see the hockey stick graph tested in a Court of Law. Hopefully, it might happen as a consequence of the legal action brought by Mann against Dr Tim Ball.
Thanks Mike, preferring what I would call leisure areas of research by choice, I began by investigating this area out of curiosity after a few year’s worth of remarkable media reports, and simply wondered why after all that time I hadn’t noticed anything happening, and wanted to check how everything was going. What I found was like opening an old container of food and finding it full of maggots. I couldn’t believe how different the actual figures were from what the papers had said, how tiny rises in temperature and an almost negligible change in sea level could cause such a performance despite the fact nothing unusual had happened. The answer itself was in the future, all these disasters hadn’t actually happened, but would for sure. Except they weren’t sure how much, it could rise 6C and it could rise 0.5C. Now that’s not a prediction but the most extreme form of hedging you’d get from any dodgy psychic who was only in it for the money and fame. And worst of all these predictions, made around 2000 at the time, were for 2100. Science requires observation and completion of experiments. They had openly set the finish date beyond the point it could be known, thus killing the theory stone dead on day one.
Then, on top of the diabolical data I discovered the Club of Rome and others, mainly led by David Rockefeller (he seems to be behind everything and proud of it), had openly stated back in 1991 that the threat of global warming would be the key that would unlock the new world order. Oh really, so what does that mean? Inventing a problem which most people wouldn’t be able to tell was happening and then offering a world government and massive taxes to solve, and then when it didn’t happen as expected say the medicine worked and laugh all the way to the bank. I then went about gathering the data, and now have it all collected and am looking for a company to publish it. Once the public catch this scandal it would die overnight.
[Windchaser says:
August 24, 2012 at 3:44 pm]
I believe you need to retake reading comprehension, from his letter:
“He was never muzzled even though he violated NASA’s official agency position on climate forecasting
(i.e., we did not know enough to forecast climate change or mankind’s effect on it).
He thus embarrassed NASA by coming out with his claims of global warming in 1988 in his testimony before Congress.
My own belief concerning anthropogenic climate change is that the models do not realistically simulate the climate system because there are many very important sub-grid scale processes that the models either replicate poorly or completely omit.
Furthermore, some scientists have manipulated the observed data to justify their model results.”
I just thought of something, being a web analyst.
Has anyone considered what influence the lowering of the amount of temperature stations in the statistical material for global average temperature does for random fluctuations in data and their significance for average calculations, especially when adjusting data upwards in a artificial fashion.
Wouldn’t that create a higher amount of temperature extreme highs, just like we always see smaller communities have higher cancer rates, better education scores, etc.
I see this constantly, when working with web analytics tool that samples data above 250k visits (Google Analytics) and I just Read Daniel Kahnenmans book “Thinking fast and thinking slow” which demonstrates human intuition to accept data, even though it has been influenced by lower sample rates?
David Andrew Howard says:
August 24, 2012 at 9:42 pm
“But this MWP excision [without any debate] is not a smoking gun, it’s a fingerprint.”
Yes, John Daly noted the same thing as per Smokey’s link above, and there are many other “fingerprints”. Anecdotally, the mere fact that the ipcc released its scientific conclusions from the TAR months before its supporting science touched me off the first little bit back in 2001. I’d never heard of that in the publication of “peer reviewed” science! I stuck with their “science” in the spirit of not expecting a group of scientists to not be practicing real science and given my own ignorance about the usual or acceptable practices of Climate Science, but the maggots just kept on multiplying.
The problem is that for whatever reason a great many people who should know better just won’t make the call as to the fact that “mainstream” Climate Science is simply not real science, but instead only a massive Propaganda Operation – a fact which explains nearly everything we see from its players. That’s what we’re up against in getting these “fingerprints” before the “court”.
The correct way to make a professional hockey stick: http://gorillavid.in/s3s8xbeqz29a
See also Willis Eschenbach’s article at http://climateaudit.org/2008/11/23/cant-see-the-signal-for-the-trees/: “CDA shows that the hockeystick shape is entirely due to Tiljander proxies plus high-altitude southwestern US “stripbark” pines (bristlecones, foxtails, etc). When these are removed, the hockeystick shape disappears entirely.”
The basic problem is that the models use proxy data for paleo-temperature data (since, of course, no thermometers were there at the time) but proxies are sensitive to other phenomena than simply temperature. The Mann et al didn’t properly account for other factors in creating their curve.
The major problem in getting these items of forensic evidence to court is no such process exists. The climate machine was created pretty much outside the law. In fact one organisation (someone will remember which branch or twig of the UN) is currently applying for full legal immunity for their actions as I write. There was never any mechanism set up in order to question in any way the findings of such organisations. The best they’ve managed are pretty much internal enquiries over Climategate headed by such as Lord Oxburgh whose main personal wealth comes from green industry. But technically unless they have clearly appeared to break the law in known areas there is no wrongdoing they can be accused of which can be tried in a court, and furthermore no authority to either report them to or send them to trial, as ‘they’ as such are the judges already, being the governments of the world who sponsor and produce these wrongdoings.
In fact it seems the sole way of catching them out officially is backwards, using the old route of lateral thinking’s brother, lateral acting. This means not trying to take them to trial as currently there is no known way, but get them to take someone else to trial, and thus having to reveal their own data in a court of law, much like the only case I know as yet of Al Gore being found to have made a list of errors so long the judge drew a line and said he’d counted enough to make his decision already. As this was a film and not a scientific paper Gore was excused as only a PR man, but now Michael Mann is taking his detractors to court one by one, he may well go the way of the Scientologists who ended up having all their secrets published as evidence. The rule here is all that needs doing is to get them in court, not how it is done, and Mann seems to be the only one stupid or crazy enough to do it.
Of course the second way to catch them is simply by getting the dirt on TV, it’s happened here once in The Great Global Warming Swindle, which must have won a few over, but since then the new data and outright confessions I’ve collected alone would sink the movement dead in an hour if presented to the viewing public. No one likes to know they’ve been taken for a mug and would be the equivalent of the French Revolution. If anyone here is in the media please let me know as I have completed the full package and is now ready for release. I studied law so did not reveal my material until it was completed, unlike the joke story about the Arctic melt which is doing as much to dismantle their case (if not more) than support it (as it’s weather!). You can contact me by clicking my name here.