CRU Emails "may" be open to interpretation, but commented code by the programmer tells the real story

When the CRU emails first made it into news stories, there was immediate reaction from the head of CRU, Dr. Phil Jones over this passage in an email:

From a yahoo.com news story:

In one leaked e-mail, the research center’s director, Phil Jones, writes to colleagues about graphs showing climate statistics over the last millennium. He alludes to a technique used by a fellow scientist to “hide the decline” in recent global temperatures. Some evidence appears to show a halt in a rise of global temperatures from about 1960, but is contradicted by other evidence which appears to show a rise in temperatures is continuing.

Jones wrote that, in compiling new data, he had “just completed Mike’s Nature trick of adding in the real temps to each series for the last 20 years (i.e., from 1981 onwards) and from 1961 for Keith’s to hide the decline,” according to a leaked e-mail, which the author confirmed was genuine.

Dr. Jones responded.

However, Jones denied manipulating evidence and insisted his comment had been taken out of context. “The word ‘trick’ was used here colloquially, as in a clever thing to do. It is ludicrous to suggest that it refers to anything untoward,” he said in a statement Saturday.

Ok fine, but how Dr. Jones, do you explain this?

There’s a file of code also in the collection of emails and documents from CRU. A commenter named Neal on climate audit writes:

People are talking about the emails being smoking guns but I find the remarks in the code and the code more of a smoking gun. The code is so hacked around to give predetermined results that it shows the bias of the coder. In other words make the code ignore inconvenient data to show what I want it to show. The code after a quick scan is quite a mess. Anyone with any pride would be to ashamed of to let it out public viewing. As examples [of] bias take a look at the following remarks from the MANN code files:

Here’s the code with the comments left by the programmer:

function mkp2correlation,indts,depts,remts,t,filter=filter,refperiod=refperiod,$

datathresh=datathresh

;

; THIS WORKS WITH REMTS BEING A 2D ARRAY (nseries,ntime) OF MULTIPLE TIMESERIES

; WHOSE INFLUENCE IS TO BE REMOVED. UNFORTUNATELY THE IDL5.4 p_correlate

; FAILS WITH >1 SERIES TO HOLD CONSTANT, SO I HAVE TO REMOVE THEIR INFLUENCE

; FROM BOTH INDTS AND DEPTS USING MULTIPLE LINEAR REGRESSION AND THEN USE THE

; USUAL correlate FUNCTION ON THE RESIDUALS.

;

pro maps12,yrstart,doinfill=doinfill

;

; Plots 24 yearly maps of calibrated (PCR-infilled or not) MXD reconstructions

; of growing season temperatures. Uses “corrected” MXD – but shouldn’t usually

; plot past 1960 because these will be artificially adjusted to look closer to

; the real temperatures.

;

and later the same programming comment again in another routine:

;

; Plots (1 at a time) yearly maps of calibrated (PCR-infilled or not) MXD

; reconstructions

; of growing season temperatures. Uses “corrected” MXD – but shouldn’t usually

; plot past 1960 because these will be artificially adjusted to look closer to

; the real temperatures.

 

You can claim an email you wrote years ago isn’t accurate saying it was “taken out of context”,  but a programmer making notes in the code does so that he/she can document what the code is actually doing at that stage, so that anyone who looks at it later can figure out why this function doesn’t plot past 1960. In this case, it is not allowing all of the temperature data to be plotted. Growing season data (summer months when the new tree rings are formed) past 1960 is thrown out because “these will be artificially adjusted to look closer to the real temperatures”, which implies some post processing routine.

Spin that, spin it to the moon if you want. I’ll believe programmer notes over the word of somebody who stands to gain from suggesting there’s nothing “untowards” about it.

Either the data tells the story of nature or it does not. Data that has been “artificially adjusted to look closer to the real temperatures” is false data, yielding a false result.

For more details, see Mike’s Nature Trick

UPDATE: By way of verification….

The source files with the comments that are the topic of this thread are in this folder of the FOI2009.zip file

/documents/osborn-tree6/mann/oldprog

in the files

maps12.pro

maps15.pro

maps24.pro

These first two files are dated 1/18/2000, and the map24 file on 11/10/1999 so it fits timeline-wise with Dr. Jones email where he mentions “Mike’s Nature trick” which is dated 11/16/1999, six days later.

UPDATE2: Commenter Eric at the Climate Audit Mirror site writes:

================

From documents\harris-tree\recon_esper.pro:

; Computes regressions on full, high and low pass Esper et al. (2002) series,

; anomalies against full NH temperatures and other series.

; CALIBRATES IT AGAINST THE LAND-ONLY TEMPERATURES NORTH OF 20 N

;

; Specify period over which to compute the regressions (stop in 1960 to avoid

; the decline

;

Note the wording here “avoid the decline” versus “hide the decline” in the famous email.

===============

I’ll give Dr. Jones and CRU  the benefit of the doubt, maybe these are not “untowards” issues, but these things scream for rational explanations. Having transparency and being able to replicate all this years ago would have gone a long way towards either correcting problems and/or assuaging concerns.


Sponsored IT training links:

Need help for EX0-101 exam ? We offer self study 642-436 training program for all your 642-974 exam needs.


0 0 votes
Article Rating

Discover more from Watts Up With That?

Subscribe to get the latest posts sent to your email.

480 Comments
Inline Feedbacks
View all comments
stevemcintyre
November 22, 2009 8:22 pm

The quote and credit is due to reader “Neal”.

Noelene
November 22, 2009 8:30 pm

Keep it up,I am watching with fascination.You(and the others who love science) must be so angry,they have besmirched the whole field of climate science.Dirty rotten scoundrels.Politicians are carrying on as if this whole issue will go away soon,I hope it doesn’t.

Policyguy
November 22, 2009 8:32 pm

The trillions and gazillions of dollars that are now at risk is the pathetic consequence of a programmer gone berserk. What a travesty. Can it be corrected? Who knows. A call to Science is in order so that reliable individuals can correct this unballance between political/social myth and reality. The challenge will be, how to encourage informed individuals to speak up.

November 22, 2009 8:34 pm

I may be dense here, but what’s the issue? The red comment says “don’t plot beyond 1960”, because the results are unreliable. So is there any indication that anyone has plotted beyond 1960? This came up on the Bishop Hill thread, where he drew attention to an email by Tim Osborn where he said that they never plot some treering set beyond 1960 because of a divergence issue. Turns out that that is what Briffa/Osborn say also in Briffa et al 2001. This Briffa/Osborn context may be unrelated, but it seems to me that it may simply just mean what it says. Don’t plot beyond 1960 using this code. And people don’t.

Fred
November 22, 2009 8:35 pm

Its highly unlikely mere facts will be able to stop the AGW religion for advancing from one victory to the next. After all, AGW was never about science, it was about raisng taxes and controlling other people’s lives.

P Gosselin
November 22, 2009 8:36 pm

[snip]
Add that to violating FOIA.
When we’re done, we’ll be able to throw the book at them.

Mann O Mann
November 22, 2009 8:36 pm

But remember – if you question Real Climate Scientists ® then you are a DENIER.
Oooga Boooga!
/sarcasm

Glenn
November 22, 2009 8:38 pm

I’ve been unable to find the definition of “artificial adjustment” in the climatologist’s handbook. It must be called something else. Sure sounds like something else to me. More! More!

November 22, 2009 8:40 pm

Spin that to the moon.
Looks like a duck, quacks like a duck….smells like a duck’s butt.

November 22, 2009 8:41 pm

If that’s the best you can do then you’re whole case is in trouble. Data from independent sources often has to be massaged (the programming terminology is “munged”) in order to make independent series compatible with each other.
The most interesting thing about this is the practice of “copy and paste” coding indicated by the duplicated comments, which ‘real’ programmers don’t like but which are an unfortunate necessity when programming for scientific research.

Alvin
November 22, 2009 8:42 pm

Very telling. I just authored a letter to Senator Graham asking if he is keeping up with this new finding. I also insisted that he provide a public appology to the people of South Carolina and to Senator James Inhofe.

BarryW
November 22, 2009 8:43 pm

[sarcasm on]Don’t you understand? There is reality and then there is TRUTH. If reality doesn’t fit the TRUTH then reality must be adjusted. [sarcasm off]

November 22, 2009 8:44 pm

No wonder they refused to release it even under FOI.

Richard Sharpe
November 22, 2009 8:44 pm

When people use cut-n-paste coding, they sometime even copy the comments and forget to change them.

P Gosselin
November 22, 2009 8:47 pm

WHOA!
I can’t believe it!
One of Germany’s biggest highly respected dailies has it on the front website page.
http://www.welt.de/wissenschaft/article5294872/Die-Tricks-der-Forscher-beim-Klimawandel.html#xmsg_comment

Viktor
November 22, 2009 8:50 pm

Uh, wow. Some fence sitters wanted hard evidence, more than what they perceived to be mere conjecture within the email spool. Well, there you go.
This story has reached another level. Hard evidence of such blatant data manipulation mustn’t be allowed to simply vanish into the news cycle. I hope the few MSM outlets accurately reporting this story pick up on this, because the notes in the code indeed appear to be a smoking gun.

November 22, 2009 8:51 pm

It may have been posted before so, if so, apologies but I do like this one – to be found in the HARRY_READ_ME.txt file:
“OH F*** THIS. It’s Sunday evening, I’ve worked all weekend, and just when I thought it was done I’m
hitting yet another problem that’s based on the hopeless state of our databases. There is no uniform
data integrity, it’s just a catalogue of issues that continues to grow as they’re found.”
(I confess to manipulation of the f-word myself, but only to comply with WUWT policy).
So Copenhagen Comrades, what’s a trillion dollars or so here and there based on “no uniform data integrity” ??

UKIP
November 22, 2009 8:52 pm

No resignations or sackings yet then?

Hank Hancock
November 22, 2009 8:54 pm

I think it is noteworthy that Steve McIntyre comments on Mann and Briffa truncating their MXD data at 1960.
http://www.climateaudit.org/?p=4221

artwest
November 22, 2009 8:54 pm

A poster called Asimov has quoted extracts from the “HARRY_READ_ME.txt” file – deeply shocking stuff:
Asimov’s very plausible suggestion is that Harry is a programmer trying, and often failing, to make sense of the garbage data which he’s been lumbered with.
Several posts over several pages:
http://www.tickerforum.org/cgi-ticker/akcs-www?post=118625&page=13

November 22, 2009 8:57 pm

Can someone point me what file is this code from?
I have downloaded the package, and found only one file (FOIA/documents/osborn-tree6/mkp2correlation.pro) that includes the above mentioned function. Lines 1 to 7 are identical, but the rest has nothing to do with the screenshot above.

Leon Brozyna
November 22, 2009 8:58 pm

Emails may be just chit-chat between various parties, but the coding is the receipe for cooking the books. No wonder AGW leaves such a bad taste — it’s way too overcooked leftovers.

November 22, 2009 8:58 pm

Stokes: the real issue is that temperatures derived from tree rings are known to not match measured temperatures after 1960.
If tree ring-based temperatures are known to be false compared to actual measurements, then how can they be true in earlier decades or centuries?

Michael Jankowski
November 22, 2009 9:01 pm

Nick,
Exactly who is responsible for “artificially adjusted to look closer to the real temperatures” that the code is advising against plotting, and why would they have done such a thing?
I agree that it’s no problem with the code saying not to plot past 1960, but it is certainly a problem that the code says someone has taken liberty with post-1960 data (or the methodology used to process it) for the purpose of making it “look” more like the instrumental record.

D. King
November 22, 2009 9:01 pm

Wow, the coming week will be most interesting.
Good job Anthony, Steve, and all. Thank you.
Time to look at the sea ice satellite AGC, pointing,
and receiver gain. I think we may find some missing ice.

1 2 3 20
Verified by MonsterInsights