Reader Crosspatch writes in comments:
4241.txt is where Briffa Rob Wilson apparently believes he recreates what McIntyre is talking about the hockey stick showing up no matter what data you feed into it. Briffa Wilson creates randomly generated time series, feeds them in to Mann’s maths and bingo … out pops a hockey stick.
I first generated 1000 random time-series in Excel – I did not try and approximate the persistence structure in tree-ring data. The autocorrelation therefore of the time-series was close to zero, although it did vary between each time-series.
Playing around therefore with the AR persistent structure of these time-series would make a difference. However, as these series are generally random white noise processes, I thought this would be a conservative test of any potential bias.
…
The reconstructions clearly show a ‘hockey-stick’ trend. I guess this is precisely the phenomenon that Macintyre has been going on about.
Here’s the full email:
cc: edwardcook <REDACTED>, <REDACTED>, “David Frank” <REDACTED>, “Jan Esper” <REDACTED>, “Tim Osborn” <REDACTED>, <REDACTED>, “Brian Luckman” <REDACTED>, “Andrea Wilson” <REDACTED>, “rosanne” <REDACTED>, “Watson,Emma [Ontario]” <REDACTED>, “Gordon Jacoby” <REDACTED>, “Brohan, Philip” <REDACTED>
date: Tue, 7 Mar 2006 17:44:46 +0700
from: edwardcook <REDACTED>
subject: Re: Emailing: Rob’s Hockey Sticks
to: “Rob Wilson” <REDACTED>
Hi Rob,
You are a masochist. Maybe Tom Melvin has it right: “Controversy about which bull caused mess not relevent. The possibility that the results in all cases were heap of dung has been missed by commentators.”
Cheers,
Ed
On Mar 7, 2006, at 5:20 PM, Rob Wilson wrote:
Greetings All,
I thought you might be interested in these results.
The wonderful thing about being paid properly (i.e. not by the hour) is that I have time to
play.
The whole Macintyre issue got me thinking about over-fitting and the potential bias of
screening against the target climate parameter.
Therefore, I thought I’d play around with some randomly generated time-series and see if I could ‘reconstruct’ northern hemisphere temperatures.
I first generated 1000 random time-series in Excel – I did not try and approximate the
persistence structure in tree-ring data. The autocorrelation therefore of the time-series
was close to zero, although it did vary between each time-series. Playing around therefore
with the AR persistent structure of these time-series would make a difference. However, as
these series are generally random white noise processes, I thought this would be a
conservative test of any potential bias.
I then screened the time-series against NH mean annual temperatures and retained those
series that correlated at the 90% C.L.
48 series passed this screening process.
Using three different methods, I developed a NH temperature reconstruction from these data:
1. simple mean of all 48 series after they had been normalised to their common period
2. Stepwise multiple regression
3. Principle component regression using a stepwise selection process.
The results are attached.
Interestingly, the averaging method produced the best results, although for each method
there is a linear trend in the model residuals – perhaps an end-effect problem of
over-fitting.
The reconstructions clearly show a ‘hockey-stick’ trend. I guess this is precisely the
phenomenon that Macintyre has been going on about.
It is certainly worrying, but I do not think that it is a problem so long as one screens
against LOCAL temperature data and not large scale temperature where trend dominates the correlation.
I guess this over-fitting issue will be relevant to studies that rely more on trend
coherence rather than inter-annual coherence. It would be interesting to do a similar
analysis against the NAO or PDO indices. However, I should work on other things.
Thought you’d might find it interesting though.
comments welcome
Rob
REDACTEDREDACTEDREDACTEDREDACTED—
Dr. Rob Wilson
Research Fellow
School of GeoSciences,
Grant Institute,
Edinburgh University,
West Mains Road,
Edinburgh EH9 3JW,
Scotland, U.K.
Tel:REDACTED
Publication List: [1]http://freespace.virgin.net/rob.dendro/Publications.html
“…..I have wondered about trees.
They are sensitive to light, to moisture, to wind, to pressure.
Sensitivity implies sensation. Might a man feel into the soul of a tree
for these sensations? If a tree were capable of awareness, this faculty
might prove useful. ”
“The Miracle Workers” by Jack Vance
REDACTEDREDACTEDREDACTEDREDACTED—
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Is this another silver bullet? Is this the end of Mann made global warming?
Juliette Jowit of The Guardian has contributed a thorough “Nothing to see here; move along” article explaining just why nothing in Climategate 2.0 is damaging to UAE or “the cause”:
http://www.guardian.co.uk/environment/2011/nov/24/leaked-climate-science-emails
izen:
You are kidding, right? Please specify how the McIntyre and McKitrick analysis is flawed. You do understand the issue of the Bristle Cone Pines and Gaspe Larches, right? And you do understand the issues with Briffa’s Yamal sample?
That’ a joke, right? One of the esteemed climate scientists tests the algorithm of a colleague on WHITE NOISE, it produces the HOCKEY STICK SHAPE, and the scientist says “I don’t think it’s a problem”?????
Could we have these guys defunded and BANNED from any further research activities?
I thought it interesting that in todays Wall Street Journal (Nov. 28) on the op-ed page there is an article by James Delingpole where he mentions how Mann in one of his e-mails wants to discredit McIntyre. At least now, this story is making it`s way to influential readers. The Journal is a must read for conservative politicians.
2011: A Climate Odyssey – Arthur C. Clarke’s Bowman meets the climate debate.
“The thing’s hollow—it goes on forever—and—oh my God—it’s full of Hockey Sticks?!
DirkH:
“That’ a joke, right? One of the esteemed climate scientists tests the algorithm of a colleague on WHITE NOISE, it produces the HOCKEY STICK SHAPE, and the scientist says “I don’t think it’s a problem”????? ”
Unfortunately its not a joke. McIntyre & McKitrick demonstrated this in 2003 (actually strictly with “red noise”, not white noise, but clearly Mann’s method fails with all noise). It was when I read this most trivial test of Mannian Hockey Stick algorithm that it was obvious the hockey stick was nonsense. When I posted on the New Statesman nomination of Steve M that on reading MM2003 that I just new it was correct a CAGW nut then made sarcastic comments about how I could just “know” they were right. Any rational person with the minimum of scientific or numerate training can see that failing such a trivial test means the method of Mann et al cannot work. I have the slight advantage of working on spatially/temporally correlated stochastic data sets for my day job, so I just “knew” McIntyre & McKitrick were correct.
Peter Ward:
“By that, does he mean that he compared the “tree ring” time series against the expected answer and only kept those that matched? If so, is this the normal way they go about these things? “Oh this tree’s record has *clearly* been affected by something other than temperature so we’ll ignore it… and this one too…. and another…. and yet another…. oh but this one’s ok so we’ll use that… [some time later] It’s ok we’ve got 12 that tell us what we need”.”
Yep, that’s exactly right. The whole appraoch of paleoclimate is based on a tautology. You compare your proxy with temp, if it correlates you keep it, if it doesn’t you discard it. By only keeping the postive correlating series, hey presto – result.
Now imagine testing against random noise where there is no correlation. Some series are positive correlation, some negative correlation. But we reject all negative correlations and so magically…trees are thermometers. Just a shame they didn’t listen to an actual Dendro scientist like John Daly whose email to all of them is in the Climategate 2 release. All the reasons why trees are not thermometers.
I cannot believe that so many people have been believing such thin and tautological nonsense for so long, and trying to hide such inadequate and pathetic evidence for CAGW by dressing up feeble and flawed analyses as though they were the cutting edge of science. Take a look at Steig’s admission concerning Steig 09 in the emails, he knew the reviewers were right and his case could not be supported, but it took RyanO et al sweat, blood and tears to dissect the truth about Steig’s paper. Anyone that believes Steig 09 shows Antarctic warming just needs to look at RyanO’s sensitivity tests to know it isn’t true:
http://climateaudit.files.wordpress.com/2011/02/nailcoffin.png
I have that picture on my wall to remind me what nonsense all this stuff is from “The Team”. They are charlatans, the lot of them.
“I first generated 1000 random time-series in Excel” Maybe Jones should try that. Oh right I forgot he doesn’t know how to use Excel.
You didn’t publish my first comment, so my second comment, beginning ‘added to:’ doesn’t quite make sense, I think…
It is clear that Mann has not completed graduate level forestry or agricultural research design coursework. He mistakes one step of the scientific process for another. Experimental plot design is different than plot observation. The AGW scientists get the two mixed up when studying trees as a proxy for temperature. Experimental plot design sets up a grow period carefully designed to minimize (or at least control) all variables except the one under study. Plot observation simply grids a natural plot and observes it for interesting phenomenon to later hypothesize and study under controlled plot conditions. Mann needs to take the next step and develop an experimental plot design to see to what degree temperature anomalies observed in the past affect tree ring growth when all other variables are controlled to a steady state. That he and we are massaging his observational phenomenon data suggests that we are falling into the same trap instead of pointing out the more egregious design error.
Rob Wilson occasionally jumped in the discussions at Climate Audit, most notably, the now famous Strip Bark Standardization post (where a link to foi2009.zip was first posted). He was commenting with the knowledge that he had replicated McIntyre’s results three years prior. Interesting.
oakwood says:
November 27, 2011 at 11:10 pm
“It is certainly worrying, but I do not think that it is a problem so long as one screens
against LOCAL temperature data…”
Great work – he has successfully and smoothly “debunked” any idea that McIntyre’s work challenges the Mann hockey stick.
What “LOCAL” temperatures is Wilson talking about, oakwood? And where is the work that even shows the local temps. track the GMT? All Mann did is to take tree ring data already collected in the field by someone else from the obviously flawed stripbarks and try to “fit” it to GMT via his Hockey Stick seeking algorithm, not updating the data after 1980 because it would be ~”too expensive and difficult”. Which McIntyre disproved by his “Starbucks hypothesis” by collecting field data himself at the Almagre site, Colorado. Has Mann ever collected any data himself?
Then McShane and Wyner showed that Mann’s data does not even track the “in sample” data of the calibration period itself, nor the “gradient” of the blade, and that similar methods produce Hockey sticks sometimes with flat shafts, but “most do not”, from random series given “time dependence in the temperature series”.
Such that, “Consequently, the long flat handle of the hockey stick is best understood to be a feature of regression and less a reflection of our knowledge of the truth.”
So what temperatures of any kind or source does Mann’s Hockey Stick replicate or track, oakwood?
Before attacking Rob Wilson (the messenger) , consider that someone was upset enough about this situation to release all these emails. Possibly that “someone” was one of those on this email. For a climate scientist working within this clique, to openly criticize the dogma of “the cause” is career suicide.
Think about it. Who would be willing commit total career suicide to correct a BS paper?
This is the crux of the problem created by Government funding and eco-extremists….nobody can speak out against the dogma without ending their career and being forced to find another area of research. How many would be willing to throw away 5 to 10 years of specialized training and knowledge and start again?
If I were Rob, I would at least feel somewhat justified in that I had raised the issue to my superiors and copied a whole bunch of people.
Meanwhile, on the planet where the Associated Press lives:
Posted moments ago at: http://www.google.com/hostednews/ap/article/ALeqM5g4Cf1d5NRoFFdHJCRybrNjwvCqlg?docId=f0aaf0e2d85b46eb8aaf322bd7913159
Noticed this email was NOT sent to Mann or Jones. Wilson knew what he was doing.
Rob Wilson did not drop dead moments after publishing this. It can only be taken to represent then the results at this moment in time. Not known is if Rob went on to “play” with the data further to explore the consequences of this discovery.
Or to put it more plainly, this post is a data point, and a data point does not define a trend. Don’t conclude anything until you know what Rob did next. Certainly don’t assume this is the entirety of his investigation. And don’t conclude that even if this were the entirety of his investigation don’t conclude it is conclusive. That is what the other side does.
Maybe it’s best to let Rob Wilson be. At least he was willing to try something, and tell about it, in stead of dismissing it out of hand.
The Wall Street Journal (Nov. 28) op-ed that Jack Mclaughlin, November 28, 2011 at 5:19 am refers to mentions: … email written by Mr. Mann in August 2007. “I have been talking w/ folks in the states about finding an investigative journalist to investigate and expose McIntyre, and his thus far unexplored connections with fossil fuel interests. …
McIntyre has indirectly received $20 in financial assistance from fossil fuel interests. My brother-in-law has prospered from the natural gas exploration company that he owns and has been generous with a few of his dollars. Has helped street people and others and has even been generous with his siblings. That generosity has allowed me to be uncharacteristically generous as well, and I once donated $20 to McIntyre’s Tip Jar.
Just the evidence that Mann was looking for?
Jeremy says:
November 28, 2011 at 6:54 am
Before attacking Rob Wilson (the messenger) , consider that someone was upset enough about this situation to release all these emails. Possibly that “someone” was one of those on this email. For a climate scientist working within this clique, to openly criticize the dogma of “the cause” is career suicide.
Think about it. Who would be willing commit total career suicide to correct a BS paper?
This is the crux of the problem created by Government funding and eco-extremists….nobody can speak out against the dogma without ending their career and being forced to find another area of research. How many would be willing to throw away 5 to 10 years of specialized training and knowledge and start again?
If I were Rob, I would at least feel somewhat justified in that I had raised the issue to my superiors and copied a whole bunch of people.
=================
You make a valid point.
There could be major cracks within the ‘Team’ foundation and as the truth unfolds any ethical / honest players may step forward. I hope it becomes a landslide when it begins. Nevertheless, it should have happened years ago.
TerryS says:
November 28, 2011 at 1:46 am
‘Rob states “but I do not think that it is a problem”.’
This means “I remain a member of The Team, don’t hit me!” As TerryS and others have pointed out, he found the problem and immediately dropped it. That should tell you everything you need to know about whether it is a genuine problem. Anyone who is serious about this matter will read McIntyre’s extensive analyses of the problem.
See also 1527.txt, where the same guy Rob Wilson writes
” From both my TR NH recon and the coral recon, I noted that RE values seem to be grossly inflated.
The attached figure shows a pseudo-proxy example – in this situation, calibration was made over the 1940-1981 period and verification over the 1897-1939 period.
Clearly, the predicted values (Lrec) are too high.
RE = 0.63, while CE = -0.74.
In this example, a sole reliance on RE would be wrong and could result in a highly bias reconstruction. CE is telling the correct story – i.e. the predicted values should not be trusted.
I was wondering what you and Keith have experienced over the years.
There has been criticism by Macintyre of Mann’s sole reliance on RE, and I am now starting to believe the accusations. “
I heard (endured) some global warming propaganda this morning on NPR. The segment lamented about the dire need for “drastic” CO2 reductions immediately by industrialized nations or, basically, the world will burn to a crisp very soon. No mention of the Climategate 2.0 emails and the obvious (to any sane, reasonable individual) mass collusion of “climate scientists” to create a mass hysteria about non-existant global warming. Just the same old tired propaganda from National Propaganda Radio—publicly funded, of course.
I’d suggest he get in touch with Peter Schweizer though the investigation might just take a U-turn on him, I don’t know. Schweizer seems to be pretty good at ferreting out who benefits from various government programs.
@TerryS
“It must be nice to be so believing that somebody only has to state they think something is true and you believe it. Rob states “but I do not think that it is a problem”. Rob does not perform any experiments to verify this statement. He does not provide any reasoning as to why it would not be a problem.”
++++++
I am quite surprised by your statement above as it is diversionary. The method used by McIntyre to test the method used by Mann et al was clearly described and anyone can reproduce it. You infer that Rob did not know how it worked. Mann knew how it worked and described it in his paper (’98). They all knew how it worked (if they could understand the MM paper examining it).
Why is it wrong? Principle Components analysis is not a process applicable to the problem Mann was working on. McIntyre’s objection was not specifically that the result was a hockey stick (which it inevitably happens to be) but an observation that the mathematical approach used was invalid. Whether it produced hockey sticks or not would not address the root problem which makes them appear out of the mathematical void.
I am forced to conclude that it is you who do not understand what the core issues are and thus you have made the statements above thinking that some verification of the statement or why the shape occurs is needed. The experiment was the verification you seek. He was reproducing an experiment and getting the same result. This is called the “scientific method”. I find you comment diversionary. We should rather look to the matter of the Team knowing full well in 2006 that MM98 was junk science.
Back up one step from the outputs from random inputs with autocorrelations of zero. Principle Component analysis is an invalid step in the chain of events. Further, it was not the only defect as you will quickly find out by reading McIntyre and McKitrick’s paper. The output, whether a flat line, a hockey stick, or coincidentally a reasonable reproduction of the known temperatures, is invalid because of the method used. Even you guess the correct answer in math, you still get a ‘fail’. If you get a wrong answer using wrong methods the mark is no different.
When one claims that the temperature of the past 1000 years is hockey stick-shaped because of man-made CO2 emissions, one needs to show some correlation between CO2 and temperature. Mann’s paper was supposed to be that proof and the IPCC made much hay repeating his claim to have shown the temperature flat for 1000 years and a steep rise in the 20th century. Mann’s paper is mathematically invalid and would have been failed if submitted as an undergrad term paper, at least at any Canadian university.
McIntyre and McKitrick reproduced Mann’s method and showed that, because it was a defective method, any large data sets fed into it resulted in a hockey stick output, even when the input was known to be meaningless gibberish like desert tree ring data or random red noise. Rob reproduced McIntyre’s reproduction of Mann et al. Rob claims to have gotten hockey sticks by using white noise, not red, which extends the issue to even greater proportions!
The hockey stick temperature chart produced by Mann et all in 1998 is mathematically, logically and statistically indefensible, yet indirectly you defend it. Please stop. It has been enough of an embarrassment to the climate community already.
Wake me up [when] India, Brazil, and China implement “drastic” CO2 reductions. China might actually manage this trough massive nuclear power build-out which it is currently undergoing. While they are building a lot of coal power plants, they are also in a massive nuclear power building phase, too.
Also, it looks like China won’t have the nuclear waste problem because it appears they will reprocess spent fuel: http://www.telegraph.co.uk/news/worldnews/asia/china/8238032/China-masters-nuclear-fuel-reprocessing-technology.html which is something the Jimmy Carter administration put the kibosh on in the US, Regan attempted to revive and Clinton drove a stake into the heart of.