Climategate 2 email – Rob Wilson replicates McIntyre & McKitrick – produces hockey sticks out of noise

It is not Briffa but Rob Wilson who wrote this. Why the confusion?

The email is from Rob Wilson?

“It is certainly worrying, but I do not think that it is a problem so long as one screens
against LOCAL temperature data…”
Great work – he has successfully and smoothly “debunked” any idea that McIntyre’s work challenges the Mann hockey stick.

“I then screened the time-series against NH mean annual temperatures and retained those
series that correlated at the 90% C.L.”
By that, does he mean that he compared the “tree ring” time series against the expected answer and only kept those that matched? If so, is this the normal way they go about these things? “Oh this tree’s record has *clearly* been affected by something other than temperature so we’ll ignore it… and this one too…. and another…. and yet another…. oh but this one’s ok so we’ll use that… [some time later] It’s ok we’ve got 12 that tell us what we need”.

What is a “target” climate?

The whole USP of Mann was that he took lots of disparate proxies and regressed them against a global trend. Not a trace of local screening if I remember correctly.

“but I do not think that it is a problem so long as one screens
against LOCAL temperature data and not large scale temperature where trend dominates the correlation”
That sounds like speculation. Was it ever checked out to see whether it is true?
Did Mann screen against local, regional, global or at all ?

oakwood says:

“It is certainly worrying, but I do not think that it is a problem so long as one screens against LOCAL temperature data…”
Great work – he has successfully and smoothly “debunked” any idea that McIntyre’s work challenges the Mann hockey stick.

You really don’t understand how the hockey stick handle is generated do you? Neither does Rob Wilson for that matter.
It must be nice to be so believing that somebody only has to state they think something is true and you believe it. Rob states “but I do not think that it is a problem”. Rob does not perform any experiments to verify this statement. He does not provide any reasoning as to why it would not be a problem. Since he does not reveal any understanding of why the random data produces hockey sticks, he could not know whether using local temperatures would as well.
Your reaction and unquestioning acceptance of Rob Wilson’s “thought” is what I would expect from a brainwashed follower of a cult.

So he found ‘hockey sticks’ after processing all the random series that matched the trend of the instrumental record…. hardly surprising.
What may be more surprising is why Wegman et al FAILED to replicate the M&M findings relying instead on the original flawed results from M&M which were cherry-picked for ‘hockey sticks’from many radnom series that showed no end-effect trend..
Hockey sticks however that were at least an order of magnitude smaller than the proxy and instrumental results!

Actually.. I have seen Edward Cook’s name… wrt other concerns about Mr Mann
Is this is sub group of reasonable climate scientists at CRU?

Juliette Jowit of The Guardian has contributed a thorough “Nothing to see here; move along” article explaining just why nothing in Climategate 2.0 is damaging to UAE or “the cause”:
http://www.guardian.co.uk/environment/2011/nov/24/leaked-climate-science-emails

That’ a joke, right? One of the esteemed climate scientists tests the algorithm of a colleague on WHITE NOISE, it produces the HOCKEY STICK SHAPE, and the scientist says “I don’t think it’s a problem”?????
Could we have these guys defunded and BANNED from any further research activities?

2011: A Climate Odyssey – Arthur C. Clarke’s Bowman meets the climate debate.
“The thing’s hollow—it goes on forever—and—oh my God—it’s full of Hockey Sticks?!

“I first generated 1000 random time-series in Excel” Maybe Jones should try that. Oh right I forgot he doesn’t know how to use Excel.

It is clear that Mann has not completed graduate level forestry or agricultural research design coursework. He mistakes one step of the scientific process for another. Experimental plot design is different than plot observation. The AGW scientists get the two mixed up when studying trees as a proxy for temperature. Experimental plot design sets up a grow period carefully designed to minimize (or at least control) all variables except the one under study. Plot observation simply grids a natural plot and observes it for interesting phenomenon to later hypothesize and study under controlled plot conditions. Mann needs to take the next step and develop an experimental plot design to see to what degree temperature anomalies observed in the past affect tree ring growth when all other variables are controlled to a steady state. That he and we are massaging his observational phenomenon data suggests that we are falling into the same trap instead of pointing out the more egregious design error.

oakwood says:
November 27, 2011 at 11:10 pm
“It is certainly worrying, but I do not think that it is a problem so long as one screens
against LOCAL temperature data…”
Great work – he has successfully and smoothly “debunked” any idea that McIntyre’s work challenges the Mann hockey stick.
What “LOCAL” temperatures is Wilson talking about, oakwood? And where is the work that even shows the local temps. track the GMT? All Mann did is to take tree ring data already collected in the field by someone else from the obviously flawed stripbarks and try to “fit” it to GMT via his Hockey Stick seeking algorithm, not updating the data after 1980 because it would be ~”too expensive and difficult”. Which McIntyre disproved by his “Starbucks hypothesis” by collecting field data himself at the Almagre site, Colorado. Has Mann ever collected any data himself?
Then McShane and Wyner showed that Mann’s data does not even track the “in sample” data of the calibration period itself, nor the “gradient” of the blade, and that similar methods produce Hockey sticks sometimes with flat shafts, but “most do not”, from random series given “time dependence in the temperature series”.
Such that, “Consequently, the long flat handle of the hockey stick is best understood to be a feature of regression and less a reflection of our knowledge of the truth.”
So what temperatures of any kind or source does Mann’s Hockey Stick replicate or track, oakwood?

Meanwhile, on the planet where the Associated Press lives:

“The overwhelming majority of climate scientists say the earth’s surface temperatures have been rising rapidly because man is burning fossil fuels such as coal and oil. Studies show the rising heat already has led to animals migrating, crop yields decreasing, glaciers shrinking and shore lines shifting. Developing nations, where overwhelmingly poor populations are vulnerable and infrastructure is fragile, are particularly hard hit, leading to calls on industrialized nations that got a head start on pumping greenhouse gases into the atmosphere to do, and pay, more to deal with the results.”

Posted moments ago at: http://www.google.com/hostednews/ap/article/ALeqM5g4Cf1d5NRoFFdHJCRybrNjwvCqlg?docId=f0aaf0e2d85b46eb8aaf322bd7913159

Rob Wilson did not drop dead moments after publishing this. It can only be taken to represent then the results at this moment in time. Not known is if Rob went on to “play” with the data further to explore the consequences of this discovery.
Or to put it more plainly, this post is a data point, and a data point does not define a trend. Don’t conclude anything until you know what Rob did next. Certainly don’t assume this is the entirety of his investigation. And don’t conclude that even if this were the entirety of his investigation don’t conclude it is conclusive. That is what the other side does.

The Wall Street Journal (Nov. 28) op-ed that Jack Mclaughlin, November 28, 2011 at 5:19 am refers to mentions: … email written by Mr. Mann in August 2007. “I have been talking w/ folks in the states about finding an investigative journalist to investigate and expose McIntyre, and his thus far unexplored connections with fossil fuel interests. …
McIntyre has indirectly received $20 in financial assistance from fossil fuel interests. My brother-in-law has prospered from the natural gas exploration company that he owns and has been generous with a few of his dollars. Has helped street people and others and has even been generous with his siblings. That generosity has allowed me to be uncharacteristically generous as well, and I once donated $20 to McIntyre’s Tip Jar.
Just the evidence that Mann was looking for?

TerryS says:
November 28, 2011 at 1:46 am
‘Rob states “but I do not think that it is a problem”.’
This means “I remain a member of The Team, don’t hit me!” As TerryS and others have pointed out, he found the problem and immediately dropped it. That should tell you everything you need to know about whether it is a genuine problem. Anyone who is serious about this matter will read McIntyre’s extensive analyses of the problem.

I heard (endured) some global warming propaganda this morning on NPR. The segment lamented about the dire need for “drastic” CO2 reductions immediately by industrialized nations or, basically, the world will burn to a crisp very soon. No mention of the Climategate 2.0 emails and the obvious (to any sane, reasonable individual) mass collusion of “climate scientists” to create a mass hysteria about non-existant global warming. Just the same old tired propaganda from National Propaganda Radio—publicly funded, of course.

@TerryS
“It must be nice to be so believing that somebody only has to state they think something is true and you believe it. Rob states “but I do not think that it is a problem”. Rob does not perform any experiments to verify this statement. He does not provide any reasoning as to why it would not be a problem.”
++++++
I am quite surprised by your statement above as it is diversionary. The method used by McIntyre to test the method used by Mann et al was clearly described and anyone can reproduce it. You infer that Rob did not know how it worked. Mann knew how it worked and described it in his paper (’98). They all knew how it worked (if they could understand the MM paper examining it).
Why is it wrong? Principle Components analysis is not a process applicable to the problem Mann was working on. McIntyre’s objection was not specifically that the result was a hockey stick (which it inevitably happens to be) but an observation that the mathematical approach used was invalid. Whether it produced hockey sticks or not would not address the root problem which makes them appear out of the mathematical void.
I am forced to conclude that it is you who do not understand what the core issues are and thus you have made the statements above thinking that some verification of the statement or why the shape occurs is needed. The experiment was the verification you seek. He was reproducing an experiment and getting the same result. This is called the “scientific method”. I find you comment diversionary. We should rather look to the matter of the Team knowing full well in 2006 that MM98 was junk science.
Back up one step from the outputs from random inputs with autocorrelations of zero. Principle Component analysis is an invalid step in the chain of events. Further, it was not the only defect as you will quickly find out by reading McIntyre and McKitrick’s paper. The output, whether a flat line, a hockey stick, or coincidentally a reasonable reproduction of the known temperatures, is invalid because of the method used. Even you guess the correct answer in math, you still get a ‘fail’. If you get a wrong answer using wrong methods the mark is no different.
When one claims that the temperature of the past 1000 years is hockey stick-shaped because of man-made CO2 emissions, one needs to show some correlation between CO2 and temperature. Mann’s paper was supposed to be that proof and the IPCC made much hay repeating his claim to have shown the temperature flat for 1000 years and a steep rise in the 20th century. Mann’s paper is mathematically invalid and would have been failed if submitted as an undergrad term paper, at least at any Canadian university.
McIntyre and McKitrick reproduced Mann’s method and showed that, because it was a defective method, any large data sets fed into it resulted in a hockey stick output, even when the input was known to be meaningless gibberish like desert tree ring data or random red noise. Rob reproduced McIntyre’s reproduction of Mann et al. Rob claims to have gotten hockey sticks by using white noise, not red, which extends the issue to even greater proportions!
The hockey stick temperature chart produced by Mann et all in 1998 is mathematically, logically and statistically indefensible, yet indirectly you defend it. Please stop. It has been enough of an embarrassment to the climate community already.

When one claims that the temperature of the past 1000 years is hockey stick-shaped because of man-made CO2 emissions, one needs to show some correlation between CO2 and temperature.

What bothers me is that we do have a very CLEAR hockey stick in the relatively past and yet nobody attempts to correlate it with CO2. Temperatures when we started to come out of the LIA were a pretty nice “hockey stick” if you were to do a 400 year plot. It would have been a pretty cold period (albeit with some noisy bumps up and down here and there) and then suddenly the temperatures take off skyward. Nobody is blaming that on human activity. So we do have evidence of various hockey sticks in the past. Heck, coming out of the last glacial was the mother of all hockey sticks. What set that off? It wasn’t gradual at all, it was *boom* glacial over! Hockey sticks happen all the time when you don’t have the full context and carefully pick your start and end times. We have one at my house most mornings.
I personally believe that the way we are measuring things is whack. First of all, the majority of the land on this planet is in the Northern Hemisphere. Land responds differently to things than ocean does. Temperatures over land can be quite noisy. I would expect that a global change that causes little noticeable change in the SH might cause a quite noticeable change over more of the surface of the Earth in the NH. Does CO2 have an impact? Probably. I’m not convinced it has as much of an impact as is claimed, though. I think what has more of an impact are the normal cyclical things we see such as the various multi-decadal oscillations in the atmosphere plus likely oscillations in ocean currents plus things like changes in cloud cover, solar spectrum, solar wind/cosmic ray impacts, etc. These things move about at their own pace. I believe that sometimes they just sort of accidentally line up in such a way to amplify their impact. It is like looking at a signal that is the sum of a lot of different frequencies. At various points they all add up and you might get a large peak. At other points they might all add up in the opposite direction and you get a large dip. The problem is that the periods of these things in nature are only approximate. They aren’t on a fixed period. A cycle might last anywhere from 40 to 70 years. So now you take that same mix of frequencies and begin to randomly frequency modulate them and the result is even more unpredictable. I believe that sometimes things just all line up in the “wrong” direction and we get a major impact. Maybe we have a solar grand minimum (or maybe a few more or less back to back to back [Wolf Minimum 1280-1350, Spörer Minimum 1460-1550, Maunder Minimum 1645-1715, Dalton Minimum 1790-1820] and maybe you get the Atlantic and Pacific oscillations and ENSO all lining up just right and kaboom, it gets really cold where any of those events would not by themselves be enough to create a cold period but all lined up together they can make a doozy. And that probably only happens at very infrequent intervals.
Or heck, maybe the distribution of cosmic rays in the space through which the solar system flies isn’t even. Maybe every 100,000 years or so we break out into an area that has less of them and so even with a given amount of solar activity there are fewer cosmic rays available to reach the inner solar system and cloud cover drops and for a relatively little while (15% of the time, roughly) Earth has less cloud cover and warms up and then the density of those rays begins to increase again and slowly, over several thousand years, the cloud cover increases and we get progressively colder until we slip into another “shadow” of some sort and the number of these cosmic rays is again reduced.
I can speculate all day because the whole thing is open to speculation. Nothing has been nailed down except billions of dollars of taxpayer money that is being tossed into programs to “save” the planet from becoming exactly what it has been in the not so distant past and survived that period just fine. You don’t have to go back so far in geological time to find a period when CO2 and temperatures were higher than today and we survived. No “saving” required. It’s just nuts.

“I do not think that it is a problem so long as one screens
against LOCAL temperature data and not large scale temperature”
So, with this specualtion, we must conclude that the hockey stick that this method generates when fed real data (presumably screened against LOCAL temp data) is really real. Not like the bogus hockey stick the method generates when fed bogus data.
Did he really not notice what a logical pretzel he had twisted himself into?

“The possibility that the results in all cases were heap of dung has been missed by commentators.”
You deniers have taken this phrase out of context, and it’s a typo on top of that. I originally wrote:
“The possibility that the commentators in all cases were heap of dung has been missed by the results.”
Cheers,
Ed
/sarc

@Jeremy says:
November 28, 2011 at 6:54 am
exactly – I respect Robs observations on this problem, he gave a heads up to the top nobs, ball in your court sort of thing, would any of us potentially end our career going further?
he knew the story, how else to put it?

Galane says
So Rob Wilson suspected something was hinky with the plotting program and conducted a proper scientific test of the program and proved beyond any doubt the program was rigged – yet chose to keep it quiet?
————-
Nope. He told a lot of people about it. As is crystal clear if you actually read the email.
Also he produced a hockey stick from noise, but concluded that it was not relevant to the analysis of the real data. In other words it was a hockey stick but not the same hockey stick.
So the email is actually evidence that what skeptics say is considered, checked and judgement applied about whether it is relevant.