When the CRU emails first made it into news stories, there was immediate reaction from the head of CRU, Dr. Phil Jones over this passage in an email:
From a yahoo.com news story:
In one leaked e-mail, the research center’s director, Phil Jones, writes to colleagues about graphs showing climate statistics over the last millennium. He alludes to a technique used by a fellow scientist to “hide the decline” in recent global temperatures. Some evidence appears to show a halt in a rise of global temperatures from about 1960, but is contradicted by other evidence which appears to show a rise in temperatures is continuing.
Jones wrote that, in compiling new data, he had “just completed Mike’s Nature trick of adding in the real temps to each series for the last 20 years (i.e., from 1981 onwards) and from 1961 for Keith’s to hide the decline,” according to a leaked e-mail, which the author confirmed was genuine.
Dr. Jones responded.
However, Jones denied manipulating evidence and insisted his comment had been taken out of context. “The word ‘trick’ was used here colloquially, as in a clever thing to do. It is ludicrous to suggest that it refers to anything untoward,” he said in a statement Saturday.
Ok fine, but how Dr. Jones, do you explain this?
There’s a file of code also in the collection of emails and documents from CRU. A commenter named Neal on climate audit writes:
People are talking about the emails being smoking guns but I find the remarks in the code and the code more of a smoking gun. The code is so hacked around to give predetermined results that it shows the bias of the coder. In other words make the code ignore inconvenient data to show what I want it to show. The code after a quick scan is quite a mess. Anyone with any pride would be to ashamed of to let it out public viewing. As examples [of] bias take a look at the following remarks from the MANN code files:
Here’s the code with the comments left by the programmer:
function mkp2correlation,indts,depts,remts,t,filter=filter,refperiod=refperiod,$
datathresh=datathresh
;
; THIS WORKS WITH REMTS BEING A 2D ARRAY (nseries,ntime) OF MULTIPLE TIMESERIES
; WHOSE INFLUENCE IS TO BE REMOVED. UNFORTUNATELY THE IDL5.4 p_correlate
; FAILS WITH >1 SERIES TO HOLD CONSTANT, SO I HAVE TO REMOVE THEIR INFLUENCE
; FROM BOTH INDTS AND DEPTS USING MULTIPLE LINEAR REGRESSION AND THEN USE THE
; USUAL correlate FUNCTION ON THE RESIDUALS.
;
pro maps12,yrstart,doinfill=doinfill
;
; Plots 24 yearly maps of calibrated (PCR-infilled or not) MXD reconstructions
; of growing season temperatures. Uses “corrected” MXD – but shouldn’t usually
; plot past 1960 because these will be artificially adjusted to look closer to
; the real temperatures.
;
and later the same programming comment again in another routine:
; ; Plots (1 at a time) yearly maps of calibrated (PCR-infilled or not) MXD ; reconstructions ; of growing season temperatures. Uses “corrected” MXD – but shouldn’t usually ; plot past 1960 because these will be artificially adjusted to look closer to ; the real temperatures.
You can claim an email you wrote years ago isn’t accurate saying it was “taken out of context”, but a programmer making notes in the code does so that he/she can document what the code is actually doing at that stage, so that anyone who looks at it later can figure out why this function doesn’t plot past 1960. In this case, it is not allowing all of the temperature data to be plotted. Growing season data (summer months when the new tree rings are formed) past 1960 is thrown out because “these will be artificially adjusted to look closer to the real temperatures”, which implies some post processing routine.
Spin that, spin it to the moon if you want. I’ll believe programmer notes over the word of somebody who stands to gain from suggesting there’s nothing “untowards” about it.
Either the data tells the story of nature or it does not. Data that has been “artificially adjusted to look closer to the real temperatures” is false data, yielding a false result.
For more details, see Mike’s Nature Trick
UPDATE: By way of verification….
The source files with the comments that are the topic of this thread are in this folder of the FOI2009.zip file
/documents/osborn-tree6/mann/oldprog
in the files
maps12.pro
maps15.pro
maps24.pro
These first two files are dated 1/18/2000, and the map24 file on 11/10/1999 so it fits timeline-wise with Dr. Jones email where he mentions “Mike’s Nature trick” which is dated 11/16/1999, six days later.
UPDATE2: Commenter Eric at the Climate Audit Mirror site writes:
================
From documents\harris-tree\recon_esper.pro:
; Computes regressions on full, high and low pass Esper et al. (2002) series,
; anomalies against full NH temperatures and other series.
; CALIBRATES IT AGAINST THE LAND-ONLY TEMPERATURES NORTH OF 20 N
;
; Specify period over which to compute the regressions (stop in 1960 to avoid
; the decline
;
Note the wording here “avoid the decline” versus “hide the decline” in the famous email.
===============
I’ll give Dr. Jones and CRU the benefit of the doubt, maybe these are not “untowards” issues, but these things scream for rational explanations. Having transparency and being able to replicate all this years ago would have gone a long way towards either correcting problems and/or assuaging concerns.
Sponsored IT training links:
Need help for EX0-101 exam ? We offer self study 642-436 training program for all your 642-974 exam needs.
All,
Fox News Channel (US) has picked up the story. As of ~0830 US Central Time they were doing a segment on air.
as expected, my post has been deleted from RC
Great post on NYT weaseling
http://www.weeklystandard.com/weblogs/TWSFP/2009/11/nytimes_we_wont_publish_statem.asp
Some of the revelations in the emails have been quite shocking – even worse than I thought. There’s some interesting samples on Andrew Bolt’s blog. However, what I find even more interesting is the response of the warmists. I am waiting to see if any of them – even one – says, hmm, maybe I was wrong; maybe these guys have been subverting science.
But no. The reaction is one of absolute denial (in the non perjorative sense). I am reminded of our old friend the cognisant dissonance again. Psychology teaches us that this is exactly the behaviour to be expected – mentally trying to rearrange the facts to somehow remove the dissonance. Hence we have “mountains out of molehills”, “taken out of context” etc. All very interesting and revealing behaviour.
Yet, in a way, these individuals have a metaphorical noose around their necks. The floor on which they stand is very slowly moving downwards, but they still have time to remove the noose before the rope tightens. The price they must pay is the repudiation of their cherished beliefs. Yet they do nothing except argue, in the hope that their arguments will somehow be heard and the floor will stop descending. The longer they leave it the worse it gets. Too bad.
Latest gloss-over from the Guardian.
http://www.guardian.co.uk/environment/cif-green/2009/nov/23/leaked-email-climate-change
Nothing to see here, folks, move along, all is well..
If the instrument temperatures do not agree with tree ring data over significant recent time periods, then why should we think the tree ring data is accurate for pre-instrumental times?
on the optinons how to handel data:
Jones:
Options appear to be:
1. Send them the data
2. Send them a subset removing station data from some of the countries who made us pay in the normals papers of Hulme et al. (1990s) and also any number that David can remember. This should also omit some other countries like (Australia, NZ, Canada, Antarctica). Also could extract some of the sources that Anders added in (31-38 source codes in J&M 2003). Also should remove many of the early stations that we coded up in the 1980s.
3. Send them the raw data as is, by reconstructing it from GHCN. How could this be done? Replace all stations where the WMO ID agrees with what is in GHCN. This would be the raw data, but it would annoy them.
This lot is just stunning!
– nd SteveMc must be feeling pretty vindicated at the moment!
– I bet he had a good weekend!
– I love the harry_readme file – it’s just amazing
– the code/data quality of the HadTemp product is so great (sarcasm)
– I think they’ll have to withdraw that from publication after this!
– and the code comments about adjusting the data post 1960 ‘to hide the decline’
– great stuff.
The other thing the emails give is a great insight into how the HockeyTeam operates
– suppressing disent, controlling publications & reviews, hiding data they don’t like, – and diverting attention from the real issue (e.g. ‘hiding the decline’) onto something they can toss back & forth ad infinitum (e.g. ‘trick’)
By the way, the ‘decline’ is the way the northern hemisphere tree-ring data doesn’t track temp after about 1960
– it does the opposite (i.e. declines)
– it does seem amazing to me that they seems to have been no attempt to find the scientific explantion for this, and just loads of software tweaks to hide it….
– for if our great proxies don’t track temp reliably in the current time & recent past, why should we suppose that they do 1000 years ago??
Well, we live in interesting times!
kdkd wrote:
1. Make the unstated assumption that the paleoclimate data is the whole of the co2 forced global warming story,
The skeptics didn’t build up the importance of paleoclimate/proxies. That was done by the Hockey Team and the IPCC (to their detriment, I believe). The warmest it’s been in XXXX (insert your favorite number) has been trumpeted in countless press releases. That’s the doing of the warmies, not the skeptics. McIntyre is on record at CA saying the proxies aren’t really that import to the question of whether or not Co2 causes warming–if that’s also your point, why is Phil Jones/Michael Mann et al in such a tizzy?
2. Hone in on increasingly small parts of the story and ignore the big picture. You have to do this because the big picture doesn’t support your position at all.
Besides the climatology/proxy stuff I guess there’s also the all the modeling/sensitivity issues. Is that the “big picture”? Most of us here are less than convinced.
3. Fail to respond to counter arguments from the originators of the research.
Do you mean the lack of peer-reviewed “skeptic” articles? If so then you might be interested in the emails talking about manipulating the peer review process and efforts to remove the editors of journals that dare to publish anything the Hockey Team doesn’t support. You strike me as a recent refugee from RealClimate so if you mean instead the lack of followup “skeptic” posts at RC, then you you might want to hang around here long enough to see the references to censorship over there. It’s easy to win debates when you have the power to silence your critics.
If you are in fact a frequentor of RealClimate you might like to try a little experiment. Why don’t you try posting something along the lines of “The folks at ClimateAudit and Watts Up with That claim that you practice very selective censorship of the skeptical arguments and response posts at RC. Is this true?” You might also ask if McIntyre or Watts could have a thread on RC to make their case. Report back.
“Spin that, spin it to the moon if you want. I’ll believe programmer notes over the word of somebody who stands to gain from suggesting there’s nothing “untowards” about it.
Either the data tells the story of nature or it does not. Data that has been “artificially adjusted to look closer to the real temperatures” is false data, yielding a false result”
,,,,,,,,,,,,,<<<<<<<Yes indeed. I posted that they would claim all comments were taken out of context last Friday. They did as I predicted. Now Real climate is starting a thread to do so. This is SPIN. Covering what they wrote privately to save face. As a Psychologist, this is common as a defense mechanism when people feel threatened.
Over here in Britain this story is starting to build up a head of steam. There are few voices objecting to transparency.
My reply to the Revkin/Pierrehumbert blog post
Oh please.
First of all, there’s no indication there was any vandalism involved here. In fact, it’s not even clear there was hacking involved; this looks very much like an inside job. There are no “honey, pick up a pint of milk on the way home” emails, which is interesting in itself. My own suspicion is this particular collection is one Jones himself made of documents he didn’t want released in an FOIA.
Second, scientists, such as myself, who work at public institutions are or should be aware that our email accounts are subject to FOIA. I have my own laptop with a non-university wireless modem that I use for personal business. Use of public property for personal purposes may be tacitly permitted in many places, but it shouldn’t be protected or excused.
And third, this would not have been such an issue if Jones and his cohorts had not been actively trying to hide their raw data, a practice that modern science increasingly frowns on. This is now the third embarrassment that has come from that practice (the first being the revelation of the loss of a lot of original climate data — though one must now ask if that loss was accidental — and the second the serious issues revealed a couple of months ago with Briffa’s analysis of data.). One would hope scientists would learn from their mistakes.
Jesse (21:24:11):
This is what happens when a bunch of uninformed amateurs try and “debunk” real scientists. Leave the science to the scientists and go back to your day jobs as custodians, wal-mart employees and laborers.
REPLY: So what do you do down there in Norman? NSSL? U of OK? You might be surprised at the sort of professionals that frequent here. I invite them to sound off. – A
Hi Jesse,
Your ad hominem failed because the custodians and day laborers regularly show themselves to be equivalent to climate scientists and occasionally prove to be better than proponents of Anthropogenic Global Warming. That does not say much about those professional folks at CRU.
As for myself, I am a laborer as an Electrical Engineer who works on weapon systems to be used in a time of war. My day job frequently becomes my night job and sometimes my weekend job. The products of my organization incorporate this feature called “Configuration Management” and if the product has software in it, the software is written to these things called “Standards”. Before any product of my organization is transitioned from development to the military, it has to go through several program reviews, design reviews, technical evaluation, and operational evaluation. Software used in modeling a weapon has to go through Verification, Validation, and Accreditation before it is accepted. Funny, I do not see any of that in the products from CRU.
If you would like to find out about building software to a professional standard, get two programmers together who had to write MIL-SPEC code back in the late ‘80s or early ‘90s and use the words ‘ADA’ and ‘twenty one sixty seven A’ in the same sentence. Bring popcorn; the discussion will extend well into the night.
I see they mention the MWP in file 0845217169:
“There were also long warm spells between 900 and 1100, known as the medieval warm period, and 1360 to 1560. ”
I thought the hockey stick got rid of it?
I’m gonna need a whole boat load of popcorn!
Dr A Burns (21:29:27) :
“Another strange happening at Hadley … all the hadcrut3 data for this year, except Jan/Feb, has been deleted.
http://www.cru.uea.ac.uk/cru/data/temperature/hadcrut3gl.txt”
That maybe the case because they went over to a “backup system”
If that is the case then that “missing now but not then” object of the FOI requests is also still in existence. Maybe like sort of huh?
Since we are now told that “Mike’s Nature Trick” simply refers to a clever way to get something done, I think that “Mike’s Nature Trick” should become part of common language use.
For example – If a student’s GPA starts to fall off in his senior year in High School, s/he can simply graft on someone else’s grades. That’s Mike’s Nature Trick helping students get in better universities!
Or if you lost your job and can’t refinance your house, use Mike’s Nature Trick! Just submit someone else’s pay stubs to the lender for the period you are jobless and have a declining income. Mike’s Nature Trick solves the mortgage crisis!
There have to be thousands of ways that Mike’s Nature Trick can be put to work!
The manipulating of the debate by the Team at RealClimate goes on. Just as they manipulated the peer review and publication process to make heavily lopsided the presentation of AGW science Gavin Schmidt is using RealClimate posts and responses to distort the picture of the CRU hack.
By disallowing every post he can’t squirm out of or pass off as nothing to see he’s creating the impression there’s nothing of subtance in the hack/whistleblower content.
So once again the team is in fact defrauding the public.
Gavin, along with the loyalist regulars at RC, pile on their routine rhetoric to marginalize the easiest to do so posts allowed through.
This is science to the RC regime.
When push comes to shove Gavin bails.
Even the most generic question he disallows when he doesn’t want to answer. It’s like a trial where only one side gets to clear all questions.
I tried to post this at RC and it was blocked.
“[Response: Thanks. But I don’t know what comment response you are referring to, and your claim that WUWT and CA have no agenda is laughable. – gavin]”
Well then you must know what their agenda is and you’ll be glad to share it with us?
Tom Fuller has done postings on this which are well worthwhile reads. His 5th in the series provides one of the most succinct overviews of the development of corruption in the scientific process over the last 10 years.
http://www.examiner.com/examiner/x-9111-SF-Environmental-Policy-Examiner~y2009m11d21-Evidence-of-a-desperate-push-to-pump-global-warming-up-and-up?#comments
debreuil (04:25:44) :
“Ok, haven’t done fortran in 20 years, but if I read this right, it is creating a weighting hash for each 5 year period starting in 1904 (two arrays, 1st is year, second is weighting). The forties area are multiplied by as much as -.3, then in 1960 the ‘fudge’ creeps positive, up to 2.6 in 1980 onwards. It then interpolates this over the data. Please correct if this is wrong”
<<<<<<my fortran is from the early 70's and it also sees the "weighting factors" attached.
The code archive is hardly an archive as it is a one shot picture of the code. A current standard archive is fully version controlled. The version control has the code version number, the name of the modifier, an explanation of the modifications, the date and time of the modification. You can call on the version control software to produce the previous version, or any version in the archive and further compare any version to any other version. These are the minimal standards that should be applied to any code or dataset. It appears that CRU brought no experienced software expertise to this important process. Further within the code in each routine there needs to be commentary, in plain English, that describes what the routine is to do. As the code unfolds a description of what the code is doing. Commentary like truncate the series at 1960 as a header might be OK but then how and why you were going to do it are important. To say that the data set is junk and write code to fabricate data needs to have a header “fabricating data here”. If after writing such a header you did not complain to the highest person you could access and succeed in getting the issue resolved, I cannot imagine writing one more line of code for anyone in that organization, no matter how august appearing that organization is. The issue of ethics is always present on every line of code written.
We have not even covered code design, code writing, code review and code debugging. Just some commentary on archiving and commentary.
Trillions of dollars on amateur night at the CRU. Such organized corruption by educated people is beyond belief. The damage to the universities involved is immense. These universities need to be held to task for failing the most basic supervision of their professors and research grants.
You can discuss Climategate on BBC here. Be “careful” or your comment will get pulled knowing what their censorship is like.
http://www.bbc.co.uk/blogs/paulhudson/2009/11/climategate-cru-hacked-into-an.shtml#comments
Great summary of code and data issues found so far at Devil’s Kitchen. Very entertaining read as well.
the 62 mb isn’t the smoking gun. it’s the smoking howitzer.
“Bang, bang, they shot themselves down. bang, bang, they hit the ground…”
~~cher’s new version
@ur momisugly Jesse and Mike McMillan:
…build a grad school their football team could be proud of…
I remember those days. Where did they go?
I guess OU overdosed on too much elitism, eh? This week’s gridiron game should be interesting. Got popcorn?