When the CRU emails first made it into news stories, there was immediate reaction from the head of CRU, Dr. Phil Jones over this passage in an email:
From a yahoo.com news story:
In one leaked e-mail, the research center’s director, Phil Jones, writes to colleagues about graphs showing climate statistics over the last millennium. He alludes to a technique used by a fellow scientist to “hide the decline” in recent global temperatures. Some evidence appears to show a halt in a rise of global temperatures from about 1960, but is contradicted by other evidence which appears to show a rise in temperatures is continuing.
Jones wrote that, in compiling new data, he had “just completed Mike’s Nature trick of adding in the real temps to each series for the last 20 years (i.e., from 1981 onwards) and from 1961 for Keith’s to hide the decline,” according to a leaked e-mail, which the author confirmed was genuine.
Dr. Jones responded.
However, Jones denied manipulating evidence and insisted his comment had been taken out of context. “The word ‘trick’ was used here colloquially, as in a clever thing to do. It is ludicrous to suggest that it refers to anything untoward,” he said in a statement Saturday.
Ok fine, but how Dr. Jones, do you explain this?
There’s a file of code also in the collection of emails and documents from CRU. A commenter named Neal on climate audit writes:
People are talking about the emails being smoking guns but I find the remarks in the code and the code more of a smoking gun. The code is so hacked around to give predetermined results that it shows the bias of the coder. In other words make the code ignore inconvenient data to show what I want it to show. The code after a quick scan is quite a mess. Anyone with any pride would be to ashamed of to let it out public viewing. As examples [of] bias take a look at the following remarks from the MANN code files:
Here’s the code with the comments left by the programmer:
function mkp2correlation,indts,depts,remts,t,filter=filter,refperiod=refperiod,$
datathresh=datathresh
;
; THIS WORKS WITH REMTS BEING A 2D ARRAY (nseries,ntime) OF MULTIPLE TIMESERIES
; WHOSE INFLUENCE IS TO BE REMOVED. UNFORTUNATELY THE IDL5.4 p_correlate
; FAILS WITH >1 SERIES TO HOLD CONSTANT, SO I HAVE TO REMOVE THEIR INFLUENCE
; FROM BOTH INDTS AND DEPTS USING MULTIPLE LINEAR REGRESSION AND THEN USE THE
; USUAL correlate FUNCTION ON THE RESIDUALS.
;
pro maps12,yrstart,doinfill=doinfill
;
; Plots 24 yearly maps of calibrated (PCR-infilled or not) MXD reconstructions
; of growing season temperatures. Uses “corrected” MXD – but shouldn’t usually
; plot past 1960 because these will be artificially adjusted to look closer to
; the real temperatures.
;
and later the same programming comment again in another routine:
; ; Plots (1 at a time) yearly maps of calibrated (PCR-infilled or not) MXD ; reconstructions ; of growing season temperatures. Uses “corrected” MXD – but shouldn’t usually ; plot past 1960 because these will be artificially adjusted to look closer to ; the real temperatures.
You can claim an email you wrote years ago isn’t accurate saying it was “taken out of context”, but a programmer making notes in the code does so that he/she can document what the code is actually doing at that stage, so that anyone who looks at it later can figure out why this function doesn’t plot past 1960. In this case, it is not allowing all of the temperature data to be plotted. Growing season data (summer months when the new tree rings are formed) past 1960 is thrown out because “these will be artificially adjusted to look closer to the real temperatures”, which implies some post processing routine.
Spin that, spin it to the moon if you want. I’ll believe programmer notes over the word of somebody who stands to gain from suggesting there’s nothing “untowards” about it.
Either the data tells the story of nature or it does not. Data that has been “artificially adjusted to look closer to the real temperatures” is false data, yielding a false result.
For more details, see Mike’s Nature Trick
UPDATE: By way of verification….
The source files with the comments that are the topic of this thread are in this folder of the FOI2009.zip file
/documents/osborn-tree6/mann/oldprog
in the files
maps12.pro
maps15.pro
maps24.pro
These first two files are dated 1/18/2000, and the map24 file on 11/10/1999 so it fits timeline-wise with Dr. Jones email where he mentions “Mike’s Nature trick” which is dated 11/16/1999, six days later.
UPDATE2: Commenter Eric at the Climate Audit Mirror site writes:
================
From documents\harris-tree\recon_esper.pro:
; Computes regressions on full, high and low pass Esper et al. (2002) series,
; anomalies against full NH temperatures and other series.
; CALIBRATES IT AGAINST THE LAND-ONLY TEMPERATURES NORTH OF 20 N
;
; Specify period over which to compute the regressions (stop in 1960 to avoid
; the decline
;
Note the wording here “avoid the decline” versus “hide the decline” in the famous email.
===============
I’ll give Dr. Jones and CRU the benefit of the doubt, maybe these are not “untowards” issues, but these things scream for rational explanations. Having transparency and being able to replicate all this years ago would have gone a long way towards either correcting problems and/or assuaging concerns.
Sponsored IT training links:
Need help for EX0-101 exam ? We offer self study 642-436 training program for all your 642-974 exam needs.
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Lord Lawson announced this morning:
About time!
kdkd wrote:
Vested interests and delusionals leading the ignorant here, now that you are starting to lose your grip on policy makers internationally it’s becoming amusing to watch you, if we’ve got the time that is.
Interesting. I would have written the same, word for word, speaking to the green crowd.
Who has a vested interest in green scaremongering? Skeptics? No, the environmentalist totalitarians. That’s why they always suppress any talk about the nuclear solution of energy and pollution problems.
Who is losing their grip on policy makers internationally? Al Gore & Co. Skeptics never had such a grip.
Who is becoming the laughing stock of the world? Briffa, Jones, Overpeck, Mann, Al Gore et al. Yes, it is very amusing to watch them now, running around like cockroaches on the hot skillet, their shenanigans exposed.
Who is delusional? Those who respect the facts and have guts to admit that they don’t know enough about climate to be able to predict it, or those who believe in their omniscience, and ascribe to the humanity a disproportional influence over the processes of astronomical scale?
Who judges people not by what thy do but by who they know, not by the correctness of their predictions but by pre-orchestrated fraudulent “peer reviews”?
Finally, who is always afraid to post under their real names, and has neither respect nor tolerance, not even an elementary human decency, toward their opponents?
Mann smells like a ducks but at the very behind…
http://www.timesonline.co.uk/tol/comment/columnists/guest_contributors/article6927598.ece
I think this is an important article.
JNL (22:29:08) : I’m a statistical programmer for “BIG PHARMA” . For every new drug application, the FDA requires that we give them:…
Nice List.
FWIW, I’ve done “qualified installs” for Pharma companies.
What the non-Pharma folks might not know: For every single bit of hardware and software used for all the stuff JNL listed, it must be installed “just so”. Every Single Step of Every Single Procedure must be defined in advance. Even if it is just “Open box. Plug in cord. Turn power switch on.”.
A “Qualified Install” has a predefined process and it has an implementor. It also has a Manager (that was me) and a QA officer (that may have been specific to the particular company, but the rest is FDA mandated).
The Manager watches the Implementor (i.e. systems admin) do each step.
Each step must be done EXACTLY AS WRITTEN. Then both the sysadmin and the manager sign off the step. At the end of the entire process a PREDEFINED QA process is performed and the output must match EXACTLY.
Then the manager gets to sign off the whole package and hand it over to the Company QA guy (who was watching over the shoulder of the manager watching over the shoulder of the SysAdmin…).
The whole package is copied, filed with the company, and sent to the FDA.
This is so that they can exactly duplicate everything the drug company did, including “Open the box, plug it in, turn on power”… (Though that second list would not pass a “Qualified Install” since I used commas instead of periods and said “turn on power” instead of “Turn power switch on”; it did not match my first list… Yes, it IS that picky…)
So not only must all the stuff JNL listed be sent to the FDA, but also every single bit of hardware assembly, software installation, software configuration, (the works) must be a “Qualified Install” and documented. And lord help you if NetApp changes the power-on button from Red to Orange and your Qualified Install says “Turn On Red Power Button Lower Left”…
IF the FDA decides to test something you sent, and the Qualified Install docs don’t match what they experience when, oh, booting up a Sun Server, guess what: Your drug gets rejected until you get it right… So when you say “install Solaris” you’d better have the exact release number noted and it better behave exactly the same each time…
So that is what you must do if you want to sell a an aspirin with a new type of inert binder in it or even just wanted to make a “kosher aspirin” with an enteric coating blessed by a Rabbi …
But enslave the world with carbon taxes? Destroy world economies? Claim thermageddon happening now? That can be done with completely undefined and substantially broken software with no comments, no procedures, irreproducible runs (as the comments in HARRY_README show), and with no clue if the product works.
Grab an aspirin tin / bottle and think about it, for just a few moments…
“If tree ring-based temperatures are known to be false compared to actual measurements, then how can they be true in earlier decades or centuries?”
Because no one can prove otherwise, silly.
E.M.Smith (00:47:58) :
Ever found yourself standing on a Fire Ant hill? Visit Texas… .
I did that once. ONCE.
Mike in Houston
.
savethesharks (23:13:36) :
CORRECTION: The “we scientists” part is extremely pungent given the fact that, empirically speaking, some of the first individuals to cave INTO the Third Reich….were the scientists.
So “into” not “in”.
“in to” is correcter.
Mike, grammar n*zi
.
Anyone heard any comments from M&M ?
I always look forward to Nick Stokes’ contributions; he must be nearly the most hard-working supporter of the increasing dishelveled AGW edifice; certainly he is the politest, admittedly against poor opposition, and he must have the constitution of a Mallee bull, having to digest the tripe, offal and dreck that passes as AGW evidence these days.
I’ve also been fascinated with the divergence ‘problem’ which Nick has applied his cudgels to on this thread; here we have AGW, proxified through history with all sorts of weird and wonderful samples and series correlated with each other according to strange incantations, with all the power of quantum exotics; in this way the wizards of these bits of yore and fairy stuff can construct a mighty hafted stick, as strong as any Bradman bat, to slay the doubters. But when we hit 1960, the start of the dreaded AGW, the magic disappears and the proxy magic path, so firm in the past, wilts like a viagra-less, old man’s dreams. It just isn’t fair that in the modern era when we have access to all the tree-rings we can shake a stick at that none work and we have to instrumentalise the modern duds up to speed; typical younger generation!
Found a quotlet on another blog…*
“…we are having trouble to express the real message of the reconstructions – being
scientifically sound in representing uncertainty…”
It’s like they’re channeling the Hitchhiker’s Guide to the Galaxy. “We demand rigidly defined areas of doubt and uncertainty!” 😉
*I have too slow of a net connection to download the whole lump of stuff.
“Leave the science to the scientists and go back to your day jobs as custodians, wal-mart employees and laborers.”
This is the sort of comment that those who know that their argument can not stand up to logic will resort to using. It fits in with so-called scientists who refuse to make public the raw data, code, and methods used to obtain the hockey-stick.
Without public transparency, there can be no science. The idea of putting your results out there for all to try to discredit is the very heart of the scientific method. (or at least, that is what they said in the science courses I took)
Does the truth not matter to these alarmists?
LORD LAWSON CALLS FOR PUBLIC INQUIRY INTO UEA GLOBAL WARMING DATA ‘MANIPULATION’
Lord Lawson, the former chancellor, has called for an independent inquiry into claims that leading climate change scientists manipulated data to strengthen the case for man-made global warming.
http://www.telegraph.co.uk/earth/environment/globalwarming/6634282/Lord-Lawson-calls-for-public-inquiry-into-UEA-global-warming-data-manipulation.html
Fry (00:46:31) :
“There is no much difference between CRU and UAH/RSS dataset”
Of course not, because the difference would be too difficult to explain.
Pre-satellite era data fiddling is free territory for CRU/GISS so they could argue that climate has been warming during last decades.
The BBC coverage has reopened, with the proviso that: “We have now re-opened comments on this post. However, legal considerations mean that we will not publish comments quoting from e-mails purporting to be those stolen from the University of East Anglia, nor comments linking to other sites quoting from that material.”
See http://www.bbc.co.uk/blogs/thereporters/richardblack/2009/11/copenhagen_countdown_17_days.html#comments
It would appear that the BBC is not going to give this any publicity. Any surprises there?
I have an idea of where these files may have come from. Many e-mail programs store each “folder” of messages and attachments in a single file with a simple indexing scheme.
When an e-mail is deleted, it’s not actually removed from the “folder” file, only the e-mail’s pointer in the index is removed. To actually DELETE the e-mails, the “folder” file has to be compacted or purged or whatever term the mail client software uses. There are many mail recovery programs that can quickly and easily create an index to all deleted messages in a mail client’s trash/deleted “folder” file, though they may not be able to fully recover the complete headers.
This “feature” is one thing computer forensics often uses to find evidence on computers.
In spite of this being fairly common knowledge amongst people with mid to high level computer skills, it’s surprising how few actually bother to ensure their mail clients are configured to automatically purge/compact deleted messages.
This could be the e-mail analog to a famous case where a murder suspect sneaked a pair of pinking shears into an interrogation room where the police had brought the actual floppy disks they’d obtained from his house. (Had they been a bit more on the ball they’d have stuck some blank disks into the sleeves or made backup copies first.) The suspect managed to chop the disks into pieces, but other people were able to develop a process to put them back together well enough to recover large enough fragments to prove the suspect had written a lot about the murder. A powerful magnet would’ve been a better way to destroy the incriminating evidence.
Pinking shears = “deleting” e-mail. Big magnet = purging the deleted e-mail.
JNL wrote:
“The FDA, acting as a public protector, has to assume we are trying to “cheat” (and that is a reasonable approach)… But then again, we are the evil, capitalist, profit-seeking, “BIG PHARMA” and the people need to be protected from us.”
I agree – the FDA is only “acting as” etc.
But – isn’t it possible that the FDA and “BIG PHARMA” have BOTH lost “some” credibility “lately”? And by lately I mean the past few decades…
Does the “revolving-door” policy mean anything to people working for the “BIG PHARMA – and I mean other than “career opportunities”?
Well, just wondering – and yes, I realize this is OT etc. – and I promise not to continue… 😉
E.M.Smith (00:47:58) :
“To calibrate your trees ……….”
You are absolutely right. One factor regarding forest growth that can’t be easily accounted for is effects of the excessively strong winds and hurricanes. I do regular walks in woodland which was affected by 1987 hurricane in the South East England. In some areas where trees were more exposed, most of large mature oak and beech trees were uprooted, while in the sheltered parts they survived.
Young saplings from 1987 are now established trees in all areas, but in the exposed parts where mature trees were uprooted, young ones are now nearly twice the size of those of the same age in the sheltered parts. This could be attributed to extra sunlight and nutrients available in the areas where large old established trees were uprooted.
kdkd (00:44:40)
Can you fill us in on the big picture? For example can you tell us how high sea levels were in the MWP? Can you give us the arctic ice extent for the 40’s? Do you know when the forward speed of Greenland’s glaciers peaked? We have all heard the litany of how things are worse than ever and accelerating, but what is lacking in those litanies is numerous details and a long term perspective.
Some more files with the same comment….
./FOIA/documents/osborn-tree6/summer_modes/hovmueller_lon.pro
./FOIA/documents/osborn-tree6/summer_modes/maps24.pro
./FOIA/documents/osborn-tree6/summer_modes/maps_general.pro
./FOIA/documents/osborn-tree6/summer_modes/maps1.pro
./FOIA/documents/osborn-tree6/summer_modes/maps15.pro
./FOIA/documents/osborn-tree6/summer_modes/maps1_movie.pro
./FOIA/documents/osborn-tree6/summer_modes/maps1_poster.pro
./FOIA/documents/osborn-tree6/summer_modes/maps12.pro
./FOIA/documents/osborn-tree6/mann/oldprog/hovmueller_lon.pro
./FOIA/documents/osborn-tree6/mann/oldprog/maps24.pro
./FOIA/documents/osborn-tree6/mann/oldprog/maps1.pro
./FOIA/documents/osborn-tree6/mann/oldprog/maps15.pro
./FOIA/documents/osborn-tree6/mann/oldprog/maps1_movie.pro
./FOIA/documents/osborn-tree6/mann/oldprog/maps1_poster.pro
./FOIA/documents/osborn-tree6/mann/oldprog/maps12.pro
INSIDERS?
Interesting that entitled “FOIA.zip” came at Jeff Id’s blog (nOV.13) with a posting that asked 18 leading US scientific associations about their letter to the US Senate on Oct.21,2009, at http://www.whatisclimate.com/
Would Russian hackers have done that?
Link to interview [0735] with Nigel Lawson and Robert Watson on R4 Today programme
http://news.bbc.co.uk/today/hi/today/newsid_8373000/8373594.stm
I’ve never heard Robert interviewed before but he did seem a little defensively assertive to my ears.
Alec J (23:59:14) :”This is starting to be noticed at the BBC.
On this morning’s Radio 4 Today program at about 0735am there was a five minute slot with Former Chancellor Nigel Lawson and a Prof Watson. It was reasonably well balanced – surprising given that the presenter James Naughtie usually tries to work Global Warming into everything he can.”
It’s now online here for today
http://news.bbc.co.uk/today/hi/listen_again/default.stm
at 0735
Prof Watson starts by saying “These scientists at East Anglia are both honourable and world class. Their data is not being manipulated in any bad way”.
Was Yamal not bad? How does honourable square with ‘Not allowing data release because people will try to find fault with it’?
Discuss.
tim (23:24:49) :
Well, tim, there is a big problem with what is indicated in HARRY_READ_ME.txt and the present records for historical temp data.
The skullduggery does not start and end at CRU.
Great damage has been inflicted to irreplacable?? data.
Somebody gave the warmists the means to destroy data, and they did just that.
HARRY_READ_ME understood the gravity of what happened to the data he was given to work with.
Nope, nothing wrong with the code, it finished the job of data destruction as designed.
The Librarian at Alexandria weeps yet again, for the same type of people have once more robbed history.
The news claims that “hackers” broke into and stole emails/data.
The real “hackers” destroyed science data long before that, under the guise of science.
I’m just imagining some of the behind the scenes conversations (probably by phone 😉 ) between the interested parties that must be going on now. I’d think the mother of all ‘damage limitation’ plans is being drawn up.
Along the same vein, it’s amusing to see the usual ‘nothing to see here, move along’ articles appearing in the various pro AGW newspapers / blogs (either that or the ostrich ‘it never happened’ behaviour as typified by the BBC). However, try as they might , I can’t see this one going away…
P Gosselin (01:33:38) :
No sunspots today, and for most of the past 2 weeks those that have been ‘officially listed’ as sunspots were SOHO only visible.
We really should be talking about the Sun, but a very sad day has dawned with the realization that Science Barbarians have sacked and burned irreplacable data worldwide in an effort to support a political bent.
When you look at the unsmoothed proxy data it looks like noise. It’s only when the data has been smoothed and only those proxy series that match twentieth century temperatures, (as measured by thermometers), are included do you begin to get something that looks like a hockey stick. Everything else is discarded.
Even when they get something that looks like a hockey stick, it has a divergence problem that needs a ‘trick’ to hide it. This trick is to truncate the data and substitute the thermometer data that was used to ‘calibrate’ the original data.
Lots of reasons have been put forward as to why this is necessary and, from some, why it doesn’t matter, but for me the most convincing reason, the one that old Occam would adopt, is that the original, unsmoothed, unadjusted, uncalibrated data is correct.
It isn’t temperature the proxies are showing, it’s noise.
When you’ve thrown away 90% of the data in the smoothing and calibrating process and what you’ve got left still needs padding with data from another source, then you don’t have anything at all.
No wonder they needed to rig the peer review process and bully the journals.