NOTE: Part 2 of this story has been posted: see The Smoking Code, part 2
The Proof Behind the CRU Climategate Debacle: Because Computers Do Lie When Humans Tell Them To
From Cube Antics, by Robert Greiner
I’m coming to you today as a scientist and engineer with an agnostic stand on global warming.
If you don’t know anything about “Climategate” (does anyone else hate that name?) Go ahead and read up on it before you check out this post, I’ll wait.
Back? Let’s get started.
First, let’s get this out of the way: Emails prove nothing. Sure, you can look like an unethical asshole who may have committed a felony using government funded money; but all email is, is talk, and talk is cheap.
Now, here is some actual proof that the CRU was deliberately tampering with their data. Unfortunately, for readability’s sake, this code was written in Interactive Data Language (IDL) and is a pain to go through.
NOTE: This is an actual snippet of code from the CRU contained in the source file: briffa_Sep98_d.pro
[sourcecode language=”text”]
;
; Apply a VERY ARTIFICAL correction for decline!!
;
yrloc=[1400,findgen(19)*5.+1904]
valadj=[0.,0.,0.,0.,0.,-0.1,-0.25,-0.3,0.,-0.1,0.3,0.8,1.2,1.7,2.5,2.6,2.6,2.6,2.6,2.6]*0.75 ; fudge factor
if n_elements(yrloc) ne n_elements(valadj) then message,’Oooops!’
yearlyadj=interpol(valadj,yrloc,timey)
[/sourcecode]
Mouse over the upper right for source code viewing options – including pop-up window
What does this Mean? A review of the code line-by-line
Starting off Easy
Lines 1-3 are comments
Line 4
yrloc is a 20 element array containing:
1400 and 19 years between 1904 and 1994 in increments of 5 years…
yrloc = [1400, 1904, 1909, 1914, 1919, 1924, 1929, … , 1964, 1969, 1974, 1979, 1984, 1989, 1994]
findgen() creates a floating-point array of the specified dimension. Each element of the array is set to the value of its one-dimensional subscript
F = indgen(6) ;F[0] is 0.0, F[1] is 1.0….. F[6] is 6.0
Pretty straightforward, right?
Line 5
valadj, or, the “fudge factor” array as some arrogant programmer likes to call it is the foundation for the manipulated temperature readings. It contains twenty values of seemingly random numbers. We’ll get back to this later.
Line 6
Just a check to make sure that yrloc and valadj have the same number of attributes in them. This is important for line 8.
Line 8
This is where the magic happens. Remember that array we have of valid temperature readings? And, remember that random array of numbers we have from line two? Well, in line 4, those two arrays are interpolated together.
The interpol() function will take each element in both arrays and “guess” at the points in between them to create a smoothing effect on the data. This technique is often used when dealing with natural data points, just not quite in this manner.
The main thing to realize here, is, that the interpol() function will cause the valid temperature readings (yrloc) to skew towards the valadj values.
What the heck does all of this mean?
Well, I’m glad you asked. First, let’s plot the values in the valadj array.
Look familiar? This closely resembles the infamous hockey stick graph that Michael Mann came up with about a decade ago. By the way, did I mention Michael Mann is one of the “scientists” (and I use that word loosely) caught up in this scandal?
Here is Mann’s graph from 1999
As you can see, (potentially) valid temperature station readings were taken and skewed to fabricate the results the “scientists” at the CRU wanted to believe, not what actually occurred.
Where do we go from here?
It’s not as cut-and-try as one might think. First and foremost, this doesn’t necessarily prove anything about global warming as science. It just shows that all of the data that was the chief result of most of the environmental legislation created over the last decade was a farce.
This means that all of those billions of dollars we spent as a global community to combat global warming may have been for nothing.
If news station anchors and politicians were trained as engineers, they would be able to find real proof and not just speculate about the meaning of emails that only made it appear as if something illegal happened.
Conclusion
I tried to write this post in a manner that transcends politics. I really haven’t taken much of an interest in the whole global warming debate and don’t really have a strong opinion on the matter. However, being part of the Science Community (I have a degree in Physics) and having done scientific research myself makes me very worried when arrogant jerks who call themselves “scientists” work outside of ethics and ignore the truth to fit their pre-conceived notions of the world. That is not science, that is religion with math equations.
What do you think?
Now that you have the facts, you can come to your own conclusion!
Be sure to leave me a comment, it gets lonely in here sometimes.
hat tip to WUWT commenter “Disquisitive”
========================
NOTE: While there are some interesting points raised here, it is important to note a couple of caveats. First, the adjustment shown above is applied to the tree ring proxy data (proxy for temperature) not the actual instrumental temperature data. Second, we don’t know the use context of this code. It may be a test procedure of some sort, it may be something that was tried and then discarded, or it may be part of final production output. We simply don’t know. This is why a complete disclosure and open accounting is needed, so that the process can be fully traced and debugged. Hopefully, one of the official investigations will bring the complete collection of code out so that this can be fully examined in the complete context. – Anthony
Sponsored IT training links:
Join today for 646-985 exam prep and get a free newsletter for next 642-072 and 1z0-050 exams.
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Obviously the majority of all the comments were made by very knowledgeable persons. I tell you the truth, I don’t understand all those codes and I could not tell from this blog, if those scientist did really make up all the story …
I know only one thing, it is a shame that some scientist, people we put our trust in, are lying and making up “stuff”,. We really don’t need the government to spend money on phony claims, BUT we really need to be “environment conscious”, we have to respect our wonderful Earth.
A balance has to be found between extreme spending on ghost claims and the obvious attitude we should all have by protecting our Earth.
It begins in our backyard, empowering our kids, teaching them respect for the environment, picking paper on the ground, learn not to waste electricity or water. This is all fundamentals that begin right her at our door, and the snowball will grow….
I know, I know, you are going to say we already know this, good! So it doesn’t hurt repeating it, as I still see people throw stuff out of their car’s window…there parent’s must have forget to “repeat them” the fundamentals…LOL..by the way, I can’t help biping my horn to them when I see that!!
Now concerning the research and the governmental part in all this, yes we need it, we need to find other sources of energy, we need to fight again pollution and we will still need to trust scientist, but as we can’t do the job without them they cannot do it without us.
It is a world wild consciousness! It is Everybody’s job.
Save the planet and SHE will save us!!!
NN
stop the WARm-mongers
Tiger Woods: The Media’s Allegorical Substitute for Climategate Reporting
Sound familiar, but in another context????
http://www.esquire.com/the-side/opinion/tiger-woods-accident-updates-legacy-120109
Whistleblowers–Make History, Make a Fortune
The AGW Scammers’ Achilles heel is their dependence on government funding.
There is a very robust anti-fraud mechanism in place in the Federal government.
We need to help whistleblowers provide the truth.
Can you imagine working for that superscilious “scientist” at Penn State? You know that there are numerous grad students who helped the scam. They just need to speak up.
The beautiful thing is that grant fraud is nothing new. And the federal government has a robust anti-fraud Task Force. To combat the rampant fraud in federal grants, they have established a program in which whistleblowers are rewarded with a share of funds recovered from scammers.
Get the word out to Penn State, NASA, NOAA, and any other recipients of federal funding.
Whistleblowers can make a lot of money.
Tell the truth.
Here’s a lawyer that specializes in fraud recovery:
http://howtoreportfraud.com/examples-of-federal-fraud/grant-fraud
If you are a potential whistleblower, or know someone who might be, you can also join our group: http://tech.groups.yahoo.com/group/co2isplantfood/
Anonymity is gauranteed. We’ll help you find the right government or private entity to help you take the proper action.
It’s been a long nasty time for those who believe in science and ethics. Being part of the scammer groups must have been painful and difficult. Now’s the time to do the right thing. Make history. End the most massive financial scam of all time.
Please join the group: http://groups.yahoo.com/group/co2isplantfood
And sign the petition: http://www.ipetitions.com/petition/NOC_NOW/
Join Facebook group: http://www.facebook.com/group.php?gid=191580771509
We can stop this scam, together.
Kent Clizbe
NOC-NOW
Stop the Scam—Halt the IPCC
No Consensus—No Warming
I’ve been an independent computer consultant for almost 19 years. In a past career I worked in the lab as a chemist for 16 years. I have worked in many places that have similar code control as the CRU, meaning none. The problem here is that nobody knows what the code was when it was run. As some have mentioned some of the code is commented out. We don’t know when it was commented out and when it wasn’t commented out. One would expect that if a programmer wrote the commented code he did not do it just for snicks and giggles and at some point the commented code was run.
Having said all this it points out why if you are trying to prove something that involves vast amounts of data and intricate statistical analysis the data is only part of the proof, the code that was run to arrive at your conclusions is the other. The fact that CRU instead just releases results and is so secretive of both data and code makes it impossible for anyone to evaluate their methods or repeat their findings. Ask yourselves this if they had released the code and the data with their findings would we necessarily be calling this a smoking gun? But they didn’t and that makes this appear to be exactly that. Without the data and the code THAT WAS RUN to produce their findings we have no reason to believe anything the CRU says, or for that matter anyone using CRU datasets. Science that cannot be repeated is not science, I don’t know what it is but I do know what it is not.
Reading the HARRY_ReadMe file it describes an environment where raw data was indiscriminately tossed about with no safeguards to prevent its corruption and program suites that do not perform as advertised in what little documentation that was available.
We programmers refer to shops like this as the ‘wild west’ or ‘hairballs’. I’ve worked in a bunch of them so I know one when I see it and from looking at the code the CRU is one.
So aside from all the other scientific reasons to doubt AGW, what we have here is a theory where all the observed data is of questionable providence and the methods used to interpret that data was itself a moving target. We are left with no way to verify anything they have said for the last decade or so. I don’t know what others use to describe ‘science’ like that but as far as I can see it’s nothing but a hodgepodge of rumors.
For the religious and political motivations behind the AGW scam, check out this site from an insider — http://www.green-agenda.com
Much of this comes from the UN’s Agenda 21, which includes much more than the AGW scam. There are a few other scams just like it, all part of the same global plan.
Let’s not forget that the author of this “information” is a social conservative that has railed against political solutions to climate change on his blog.
I’m not sure that ANY information would’ve led to to reach a different conclusion.
This is just more propaganda from the right.
The integrity of science must be preserved above everything else. If the data is no longer verifiable, then painful as it is, we may have no choice than to scuttle our work and start from scratch. If a more rigorous set of checks and balances are required to preserve transparency and validity of data — even revising the peer review process — then so be it. The integrity of science is worth it, no matter the embarrassment, time or cost. At stake is our ability to verify truth itself — and this is worth any price.
I understand the point the OP is making but I’m not sure I’d simply write off the emails as cheap talk.
Putting aside all the blatant deception revealed, at least one of the emails calls for recipients to delete material that was already subject to a freedom of information enquiry.
I am given to understand that is a criminal offence, if proven, and I’m very interested to see the outcome of the so-called independent investigation being carried out into the activities of Phil Jones and other members of the CRU.
Though my first expectation is to see an attempt to whitewash the thing, after all, that’s the primary purpose of ‘independent’ investigations.
what is briffa_Sep98_d.pro and how do you link this information to the CRU
It’s 8 degrees out. Time to go running but I don’t want to bundle up. No worries.
rloc=[today]
valadj=[77] ; fudge factor
Now it’s 85! Thanks Dr Jones!
I bet the good doctor could make a killing in the stock market!
Your plot doesn’t seem to match the valadj. The Valadj goes negative, to zero, negative, and then climbs into the positive. Your chart just shows one drop into negative.
All you need to ask those who believe in made made global warming is:
Do you think GREENLAND was named as a joke?
Couldn’t agree with you more. I don’t really have an opinion at this point of AGW, because I’m a good scientist and I believe in substantiating claims with data, and there’s not enough data to make the AGW-CO2 linkage, in my opinion.
The failure to disclose the codes really got me all hot and bothered, however. Not only is it detrimental to the scientific community, it hurts the integrity of the practice of science and relationship between the scientist and society as well.
The appropriate action would be for the journals which published the results-in-question to give the following ultimatum: disclose your codes and data analysis in full, or we’re yanking your papers.
Don’t know how many of you have seen this 1 minute clip that makes the point brilliantly…
http://iowntheworld.com/blog/?s=Green+Fakers
In your conclusion you state: “However, being part of the Science Community (I have a degree in Physics) and having done scientific research myself makes me very worried when arrogant jerks who call themselves “scientists” work outside of ethics and ignore the truth to fit their pre-conceived notions of the world. That is not science, that is religion with math equations.”
I know exactly how you feel. I have been dealing with that for years in another area of pseudo-science. Your statement is also a perfect definition of evolutionists. Science has very little to do with the theory of evolution. It, like man-made global warming, is just another religion. “Science” takes a backseat to the pre-conceived notions of atheists. It’s a shame so many people are taken in by both of these hoaxes.
Some of these posts are amazing. People actually putting no smoking gun in this code? One with any common sense would think that if man made global warming was not a hoax the code, the raw data and every body involved would want to have all of this information published. There is only one reason to hide this information so that people cannot come to an honest conclusion.
Until raw data, the entire modeling process and any addtional relevant information is made public intelligent people can assume this is a hoax and the idiots can buy green waisitng their money. None of these so called scientists should ever have credibility again.
I have a question.
Why don’t they just use a simple spreadsheet like excel, and do the calculations and analysis in there or something like minitab to do all of this. It seems to me that those 2 pre-written programs could “calibrate”, adjust, average, and trend just about any kind of data you wanted. I mean I have HUGE data files from work that I treat this way, with no problems what so ever. Plus that way all of your “raw data” is sitting right beside any kind of HONEST adjustments that are made.
To me I think over complicating all the data manipulations, in a way that makes it harder for someone to double check their work, just makes it more suspicious.
Exactly GoreGetsRichFromMorons (17:12:52) :…
There is no need for a smoking gun…
These guys should never have been listened to whatsoever!! People have been requesting, then demanding (via FOIA) their data and methods for years, to no avail. That in and of itself tells you they are and were lying.
“RU was deliberately tampering with their data.”
Oh no, that’s horrible. You mean they were taking their data and replacing inconvenient parts of it with made up data?
“…(potentially) valid temperature station readings were taken and skewed to fabricate the results the “scientists” at the CRU wanted to believe, not what actually occurred.”
Oh, so they weren’t replacing data outright, but they were interpolating valid temperature readings in a way that skews the result. That still seems bad. Why would they need to skew valid temperature readings?
“This closely resembles the infamous hockey stick graph that Michael Mann came up with about a decade ago.”
Interesting. But… that graph show’s Mann’s 1999 “hockey stick” actually under-projecting the actual temperature readouts. Why would a scientist (even using the word loosely) interested in massaging data to show evidence of global warming massage it to show less global temperature change than actual direct measurement of the temperature shows?
Hey, something else that’s interesting: Mann’s graph shows actual temperature measurements starting sometime after 1800. That makes sense… probably not a lot of reliable direct temperature measurements from before the 19th century. But you said that CRU’s code is manipulating valid temperature readings, and their code also has a skew factor (of 0.0) for 1400. What gives? Where are CRU’s valid temperature measurements from 1400 coming from? And how is Mann’s model going all the way back to 1000?
“the adjustment shown above is applied to the tree ring proxy data (proxy for temperature) not the actual instrumental temperature data.”
Ohh… so these *aren’t* direct, valid temperature measurements after all. They are temperature estimates based on tree ring data.
“we don’t know the use context of this code.”
Perhaps not… but you know, this whole thing sounds a lot like one of those emails that are being bandied about where Phil Jones mentions using Mann’s “trick” regarding tree-ring data.
The explanation I’ve heard about that is that tree-ring data is not not entirely reliable after a certain data for some not completely understood reasons, and Mann found a way to integrate older tree-ring data with other temperature data, but had to employ some method to do so, which was published in Nature.
Could that be what this code is about? If so… is it really the smoking gun you think it is? You clearly have been following all of this enough to have some additional insight, right?
“I really haven’t taken much of an interest in the whole global warming debate and don’t really have a strong opinion on the matter.”
Oh I see. You *haven’t* been following it, but you decided because you have a degree in Physics, you are qualified to pick some sample code apart and level accusations against others based on your interpretation of what the code is doing, but without actually knowing much more than any of the rest of us laymen about tree-ring temperature series.
Do you really have enough here to challenge people’s credentials and question their ethics?
“having done scientific research myself makes me very worried when arrogant jerks who call themselves “scientists” work outside of ethics and ignore the truth to fit their pre-conceived notions of the world.”
If by “work outside of ethics” you mean post excerpts of code whose context one admits not knowing, covers a data in a field in which they don’t specialize, but yet they feel entitled to interpret anyway and cast aspersions upon those who allegedly wrote/used said code… then I agree: that is worrying indeed.
For MangoChutney at (05:54:06), who asked “just a thought, if the emails and code were obtained by a hacker and not released by a whistle blower, would the evidence be inadmissible in a court of law”
Yours truly is only a new attorney but I nonetheless believe US law is well setttled here. I can’t speak for the Brits but don’t think the UK has a Mapp v. Ohio (1961)-style exclusionary rule. If so, they wouldn’t have a “fruit of the poison tree” doctrine. Thus I would expect the Brits would admit these files into evidence.
In the US we do have that rule, so evidence illegally obtained is not admissable. Thus the outcome would not turn on any distinction between a “hacker” or “whistle blower” in the US. If a private person acting without the knowledge of or at the direction of the police steals the data then that data would be admissable. It doesn’t matter if that stealing person were a hacker or a whistle blower. As long as the police did not tell them to steal it or know beforehand that it would be stolen it’s admissable. If the police come into into possession of material without they themselves violating a law then the material is admissable.
For example, suppose a technician in a shop stumbles across child porn on a customer’s laptop while attempting to resolve a software conflict between two unrelated programs. Tech calls the police. They examine the laptop without a warrant. The porn is admissable. Same situation except now the police tell the tech to examine every computer brought in and call them whenever he finds kiddie porn. In the US that evidence would be excluded. No probable cause; not in plain sight; search warrant needed but not obtained.
However, in this case, I believe the “hacker/stolen” theory is highly unlikely to be the cause of the release. An army of wireheads have analyzed the directory file structure and content of the materials and of course the near universal conclusion is that the Zip file was NOT the work of hackers.
I will admit my bias as a skeptic of MAN MADE global warming.
However as a scientist myself (not in these fields) I am absolutely horrified at the lack of integrity in the “climate science” field.
Destroying orignal data, rigged statistical modeling.
It’s sad to say, but basically money has corrupted science to the point that ANY claims without ALL data being published OPENLY should be immediately put in the trash.
I’m not familiar with this language, but it appears a continuation of a line of code uses a hyphen “-“. So the IF statement is not commented out.
What is interesting is why anyone would program this in the first place. Clearly something nefarious is going on. And “fudge factor” isn’t jargon you expect to find in any comment.
Destroying the original datasets is kind of the clincher though. Because there is no way of testing the code as it changes over time. With a program of even moderate complexity, there is no way to test all the possible error conditions, and a good way to see if something is really squirrelly is to use the same datasets for checking.
Not to be unkind, but I get the feeling the key programs were being modified to reach some predetermined conclusion.
I hate to break it to you climate guys, but exactly the same sort of stuff has been going on AND EXPOSED for years with regard to the antismoking movement. Data deletion from the internet records, chopping off of end points to produce desirable looking graphs, problems with FOI requests. Nothing new at all, just being done in a different area where people are still open minded enough to look at the exposure and see it for what it is.
Michael J. McFadden
Author of “Dissecting Antismokers’ Brains”
I agree with Michael, and this is not the only realm where science has been misuded by ‘policy’ as justification. What does this signify? Science as a substitute for ethics?? Science as a tool of control when religion no longer works?? Science as a new form of authority that stifles debate and dissent?
Scientists and their institutions had better watch out, for that possible role, while attractive and surely useful for raising research funding, would also mean the end of science as ‘speaking truth to power’ and future generations. Perhaps we have reached that stage already.