When the CRU emails first made it into news stories, there was immediate reaction from the head of CRU, Dr. Phil Jones over this passage in an email:
From a yahoo.com news story:
In one leaked e-mail, the research center’s director, Phil Jones, writes to colleagues about graphs showing climate statistics over the last millennium. He alludes to a technique used by a fellow scientist to “hide the decline” in recent global temperatures. Some evidence appears to show a halt in a rise of global temperatures from about 1960, but is contradicted by other evidence which appears to show a rise in temperatures is continuing.
Jones wrote that, in compiling new data, he had “just completed Mike’s Nature trick of adding in the real temps to each series for the last 20 years (i.e., from 1981 onwards) and from 1961 for Keith’s to hide the decline,” according to a leaked e-mail, which the author confirmed was genuine.
Dr. Jones responded.
However, Jones denied manipulating evidence and insisted his comment had been taken out of context. “The word ‘trick’ was used here colloquially, as in a clever thing to do. It is ludicrous to suggest that it refers to anything untoward,” he said in a statement Saturday.
Ok fine, but how Dr. Jones, do you explain this?
There’s a file of code also in the collection of emails and documents from CRU. A commenter named Neal on climate audit writes:
People are talking about the emails being smoking guns but I find the remarks in the code and the code more of a smoking gun. The code is so hacked around to give predetermined results that it shows the bias of the coder. In other words make the code ignore inconvenient data to show what I want it to show. The code after a quick scan is quite a mess. Anyone with any pride would be to ashamed of to let it out public viewing. As examples [of] bias take a look at the following remarks from the MANN code files:
Here’s the code with the comments left by the programmer:
function mkp2correlation,indts,depts,remts,t,filter=filter,refperiod=refperiod,$
datathresh=datathresh
;
; THIS WORKS WITH REMTS BEING A 2D ARRAY (nseries,ntime) OF MULTIPLE TIMESERIES
; WHOSE INFLUENCE IS TO BE REMOVED. UNFORTUNATELY THE IDL5.4 p_correlate
; FAILS WITH >1 SERIES TO HOLD CONSTANT, SO I HAVE TO REMOVE THEIR INFLUENCE
; FROM BOTH INDTS AND DEPTS USING MULTIPLE LINEAR REGRESSION AND THEN USE THE
; USUAL correlate FUNCTION ON THE RESIDUALS.
;
pro maps12,yrstart,doinfill=doinfill
;
; Plots 24 yearly maps of calibrated (PCR-infilled or not) MXD reconstructions
; of growing season temperatures. Uses “corrected” MXD – but shouldn’t usually
; plot past 1960 because these will be artificially adjusted to look closer to
; the real temperatures.
;
and later the same programming comment again in another routine:
; ; Plots (1 at a time) yearly maps of calibrated (PCR-infilled or not) MXD ; reconstructions ; of growing season temperatures. Uses “corrected” MXD – but shouldn’t usually ; plot past 1960 because these will be artificially adjusted to look closer to ; the real temperatures.
You can claim an email you wrote years ago isn’t accurate saying it was “taken out of context”, but a programmer making notes in the code does so that he/she can document what the code is actually doing at that stage, so that anyone who looks at it later can figure out why this function doesn’t plot past 1960. In this case, it is not allowing all of the temperature data to be plotted. Growing season data (summer months when the new tree rings are formed) past 1960 is thrown out because “these will be artificially adjusted to look closer to the real temperatures”, which implies some post processing routine.
Spin that, spin it to the moon if you want. I’ll believe programmer notes over the word of somebody who stands to gain from suggesting there’s nothing “untowards” about it.
Either the data tells the story of nature or it does not. Data that has been “artificially adjusted to look closer to the real temperatures” is false data, yielding a false result.
For more details, see Mike’s Nature Trick
UPDATE: By way of verification….
The source files with the comments that are the topic of this thread are in this folder of the FOI2009.zip file
/documents/osborn-tree6/mann/oldprog
in the files
maps12.pro
maps15.pro
maps24.pro
These first two files are dated 1/18/2000, and the map24 file on 11/10/1999 so it fits timeline-wise with Dr. Jones email where he mentions “Mike’s Nature trick” which is dated 11/16/1999, six days later.
UPDATE2: Commenter Eric at the Climate Audit Mirror site writes:
================
From documents\harris-tree\recon_esper.pro:
; Computes regressions on full, high and low pass Esper et al. (2002) series,
; anomalies against full NH temperatures and other series.
; CALIBRATES IT AGAINST THE LAND-ONLY TEMPERATURES NORTH OF 20 N
;
; Specify period over which to compute the regressions (stop in 1960 to avoid
; the decline
;
Note the wording here “avoid the decline” versus “hide the decline” in the famous email.
===============
I’ll give Dr. Jones and CRU the benefit of the doubt, maybe these are not “untowards” issues, but these things scream for rational explanations. Having transparency and being able to replicate all this years ago would have gone a long way towards either correcting problems and/or assuaging concerns.
Sponsored IT training links:
Need help for EX0-101 exam ? We offer self study 642-436 training program for all your 642-974 exam needs.
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Here is something from Briffa that says a lot about their mindset (from 938031546.txt):
“There is still a potential problem with non-linear responses in the very recent period of some biological proxies ( or perhaps a fertilisation through high CO2 or nitrate input) . I know there is pressure to present a nice tidy story as regards ‘apparent unprecedented warming in a thousand years or more in the proxy data’ but in reality the situation is not quite so simple. We don’t have a lot of proxies that come right up to date and those that do (at least a significant number of tree proxies ) some unexpected changes in response that do not match the recent warming. I do not think it wise that this issue be ignored in the chapter.”
Mann’s response in part (938018124.txt):
“But that explanation certainly can’t rectify why Keith’s series, which has similar seasonality *and* latitudinal emphasis to Phil’s series, differs in large part in exactly the opposite direction that Phil’s does from ours. This is the problem we all picked up on (everyone in the room at IPCC was in agreement that this was a problem and a potential distraction/detraction from the reasonably concensus viewpoint we’d like to show w/ the Jones et al and Mann et al series.
So, if we show Keith’s series in this plot, we have to comment that
“something else” is responsible for the discrepancies in this case. Perhaps
Keith can help us out a bit by explaining the processing that went into the series and the potential factors that might lead to it being “warmer” than the Jones et al and Mann et al series?? We would need to put in a few words in this regard. Otherwise, the skeptics have an field day casting doubt on our ability to understand the factors that influence these estimates and, thus, can undermine faith in the paleoestimates. I don’t think that doubt is scientifically justified, and I’d hate to be the one to have to give it fodder!”
Seems like they’re more interested in putting up a united front than figuring out why their data tells different stories. Sounds like bunker mentality.
I have to get back to mopping floors now.
TerryBixler (07:25:17) :
If I am remembering correctly you have been a programmer for a long time. So your viewpoint carries a lot of weight with me.
Lol are you guys for real? Whatever this is about, you dont need to understand programming to see climate change. Come to northern Canada and take a look – the north west passage is open people! All the ice melted! It has been closed up with ice for a long [snip] time.
There is no argument against global warming. It is warming. The question is how much do people contribute to it? And do we want to slow it down?
All the data needs to be in a RDBMS with the sql published.
@ur momisugly Gene Nemetz (22:52:46) :
“Look under ‘cha-ching’”
I believe the term is ‘ka-ching’… ;-))
E.M.Smith (00:47:58) :
A cracking post, hats off to you Sir! 30 odd years of experience here too. Mission critical software engineering and IT projects with 30+ teams in manufacturing, insurance, banking, telco and military sectors. Two directorships, 3 years consultancy to overseas government. From assembler programmer to board room and most roles in between on PCs to mainframes. Bespoke hardware design, manufacture, OEM integration and enough ‘T’ shirts to winter the rhubarb.
Never mind the cobbled up code, it’s the seemingly abject management of the whole enterprise that alarms me the most. Where’s the QA, where’s the change control? Where’s the BS EN ISO 900x conformance/paper-pushing nonsense that’s a part of all UK government IT these days? At least that would have raised a few red flags. No need to ask about documentation and operational procedures, a quick skim of HARRY_READ_ME.TXT paints the picture, even if it is a tad over the top. My heart bleeds for the poor guy, we’ve all been there! Doesn’t even appear to be a DBMS anywhere, it’s all in discrete files with virtually no integrity checks whatsoever. And politicos are betting the future shape of the world and billions of peoples’ livelihoods on this alligator swamp!? Give me a break, please.
Where’s the design, who’s managing the overall system architecture, who’s dealing with archival, disaster recovery and so on? Like as not the same uninformed amateurs with science PhDs that did a couple of modules on Fortran 66 once upon a time plus a couple of general purpose “IT gofers”. Not really surprising, after Brown’s ten year IR35 vendetta against UK IT professionals, that’s about all that’s left here now.
There’s only one thing to be done here: Send in experienced independent IT auditors, shut the whole festering mess down and box it up before valuable raw data is potentially lost forever. Do not enlist the usual milking machine offshoots of big accounting firms that employ hordes of green-horn IT graduates at max mark-up, this is serious business.
Beyond that I fully endorse Nigel Lawson’s statement in The Times today: “A high-level independent inquiry must be set up without delay.”
Concern should be raised over the size of budgets being handled here too, this is all public money regardless of the source. Just look at some of those grant application PDF files. Who is managing this at the coal face, the same people? In the same sloppy way?
And what about management of the new pig iron the Met Office just spent a huge sum of our money on? I sincerely hope it’ll be better than what appears to be going on here but perhaps we ought to see the justification first.
If you do everything by bozo sledge hammer approach from first principles at every turn (and just look at the state of this code for crying out loud!) then yes, maybe you do need a 1 petaFLOPS machine. But has anyone with years of practical experience of architecting big performance bound systems from the ground up looked at the problem? Sadly, this all reminds me of nut jobs doing complex iterative floating-point actuarial computation from first principles in big insurance systems and forever screaming for more hardware. Sounds suspiciously like another pig iron salesman’s wet dream to me, never mind that its carbon footprint is the equivalent to the CO2 emitted by 2,400 homes!
Harry can you READ_ME?
Harry can you heed me?
Harry can you lead me?
Read me
Run me
Feed me
Fund me
Prof Watson starts by saying “These scientists at East Anglia are both honourable and world class.”
Yes, isn’t it nice to think so?
rbateman: The news claims that “hackers” broke into and stole emails/data.
The real “hackers” destroyed science data long before that, under the guise of science.
You can say that again.
Just think, if only Mann, Jones, et al. were competent, they could gone to Canada and taken a few snapshots of the Northwest Passage, instead of cooking their data, deleting documents named in FOI requests, and strong-arming editors of learned journals into not publishing papers by those who disagree with them.
Yertizz:
Any comments or moves from Cameron and the Tories? I’d think this might be another issue for them in the upcoming elections.
John Smith (07:39:24) :
“Lol are you guys for real? Whatever this is about, you dont need to understand programming to see climate change. Come to northern Canada and take a look – the north west passage is open people! All the ice melted! It has been closed up with ice for a long [snip] time.
There is no argument against global warming. It is warming. The question is how much do people contribute to it?”
There are more questions than that, John. How much of Arctic melting is due to warming, for instance. Another is how much warming is really happening. Another is how people contribute to it, in addition to how much. These are just a few of the basic questions.
According to Paul Hudson at the BBC he saw the emails on 12th October. http://www.bbc.co.uk/blogs/paulhudson/2009/11/climategate-cru-hacked-into-an.shtml#comments
I guess Al Gore wishes he would not have invented the internet. Live by the Internet, die by the internet. When I use to write code, I would ask my boss what he wanted for results? You can make the data come out as you wish. Even a woman could do this.
Jack Okie (08:01:11) :
“Any comments or moves from Cameron and the Tories? I’d think this might be another issue for them in the upcoming elections.”
Cameron’s been burnishing his green credentials since Day 1 as Tory leader. It’s been part of his drive to “disinfect” the Conservative “brand” of its image as the “nasty” party. There’s not the slightest chance he’ll take any notice of this, more’s the pity.
Kudos, Evanjones (07:52:46), from another Tommy affictionado. This has exposed a lot of “fiddlin around, fiddlin around”!
This is what happens when a bunch of uninformed amateurs try and “debunk” real scientists. Leave the science to the scientists and go back to your day jobs as custodians, wal-mart employees and laborers.
Appalling. This is the sort of attitude we may expect should a scientocracy emerge. Be warned. (It goes further than that. Only the “credentialed” will be heard. And three guesses who arbitrates the credentials.)
While realizing that’s it’s a very general and imprecise yardstick, IMO it is still worth noting that yesterday when I threw ”climategate” at Google, it came back with ~38K hits. As of 08:14 US PST same Google returns 81.5K hits.
Clearly climategate has gone viral; big-time.
One question that interests me.
They have stated they cannot realease data (to a FOI request?) as its been lost.
Isnt it time to send a forensic team in and see what has actually vanished…??
Given the standards set so far (typical academia science, basicaly 🙂 I’d bet on there being a lot if not all of what has been claimed to have been lost sitting around somewhere.
(and yes, multiple code and data versions and repositories are not good software practice. However its quite prevelant in these situations….)
Jack Okie
We can but hope….I sent the content of my earlier post to a Conservative MP who had commented on the CRU in the Daily Mail.
All those times they used the phrase, “It’s worse then you think,” we thought they were talking about the temperatures. They were actually talking about their deceptions.
Aligner, I agree with your comments but,
I’m afraid, as Sir Humphrey has demonstrated on more than one occasion (Stern?), independent inquiries are almost never independent.
This gives whole, new meaning to “paint-by-numbers.”
John Smith (07:39:24) :
Lol are you guys for real? Whatever this is about, you dont need to understand programming to see climate change. Come to northern Canada and take a look – the north west passage is open people! All the ice melted!
_______________________
Lol John are you for real?
Why don’t you go to northern Canada and take a look –
John Smith (07:39:24) :
“Lol are you guys for real? Whatever this is about, you dont need to understand programming to see climate change. Come to northern Canada and take a look – the north west passage is open people! All the ice melted! It has been closed up with ice for a long [snip] time.”
As you clearly admit that the North West Passage used to be open (indeed, it is even called a passage 😉 ), shouldn’t you actually be asking why the climate became so cold that the passage was closed for the last few decades?
“When a dam breaks, it always starts with a trickle.”
A storm starts with a single raindrop.
Prof Watson starts by saying “These scientists at East Anglia are both honourable and world class.”
We’ll see about that.
“If you look a bit further into some of the other files I think its likely the person providing the comments in the HARRY_READ_ME.txt is a ‘contract programmer’ brought in to assist Ian Harris in sorting out this mess.”
That would fit nicely with the title of the file, which would be addressed to Harry thusly (with a colon) in non-computer-filename-English: “Harry: Read Me”.