From a Judicial watch press release
NASA Scientists Go on Attack After Climate Data Error Exposed
Contact Information: Press Office 202-646-5172, ext 305
Washington, DC — January 14, 2010
Judicial Watch, the public interest group that investigates and prosecutes government corruption, announced today that it has obtained internal documents from NASA’s Goddard Institute for Space Studies (GISS) related to a controversy that erupted in 2007 when Canadian blogger Stephen McIntyre exposed an error in NASA’s handling of raw temperature data from 2000-2006 that exaggerated the reported rise in temperature readings in the United States. According to multiple press reports, when NASA corrected the error, the new data apparently caused a reshuffling of NASA’s rankings for the hottest years on record in the United States, with 1934 replacing 1998 at the top of the list.
These new documents, obtained by Judicial Watch through the Freedom of Information Act (FOIA), include internal GISS email correspondence as NASA scientists attempted to deal with the media firestorm resulting from the controversy. In one exchange GISS head James Hansen tells a reporter from Bloomberg that NASA had not previously published rankings with 1998 atop the list as the hottest year on record in the 20th century.
Email from Demian McLean, Bloomberg to Jim Hansen, August 14, 2007: “The U.S. figures showed 1998 as the warmest year. Nevertheless, NASA has indeed newly ranked 1934 as the warmest year…”
Email Response from James Hansen to Damian McLean, August 14, 2007: “…We have not changed ranking of warmest year in the U.S. As you will see in our 2001 paper we found 1934 slightly warmer, by an insignificant hair over 1998. We still find that result. The flaw affected temperatures only after 2000, not 1998 and 1934.”
Email from NASA Scientist Makiko Sato to James Hansen, August 14, 2007: “I am sure I had 1998 warmer at least once on my own temperature web page…” (Email includes temperature chart dated January 1, 2007.)
(This issue also crops up in email communications with New York Times reporter Andrew Revkin a little over a week later.)
According to the NASA email, NASA’s incorrect temperature readings resulted from a “flaw” in a computer program used to update annual temperature data.
Hansen, clearly frustrated by the attention paid to the NASA error, labeled McIntyre a “pest” and suggests those who disagree with his global warming theories “should be ready to crawl under a rock by now.” Hansen also suggests that those calling attention to the climate data error did not have a “light on upstairs.”
“This email traffic ought to be embarrassing for NASA. Given the recent Climategate scandal, NASA has an obligation to be completely transparent with its handling of temperature data. Instead of insulting those who point out their mistakes, NASA scientists should engage the public in an open, professional and honest manner,” stated Judicial Watch President Tom Fitton.
=================================
Here’s a large package of emails from NASA GISS in one large PDF with 215 page which I’ve made available on the WUWT server which can handle the traffic this is likely to get.
783_NASA_docs (warning large PDF 11 MB)
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Before adding the “WUWT” home page to my “favorites” list on my browser, I first put this WUWT blog entry on the list:
http://www.norcalblogs.com/watts/2007/08/1998_no_longer_the_hottest_yea.html
It showed the change of 1998 and 1934 so that 1934 became the warmest in the US. It also contained a link to a GISS page that gets updated:
http://data.giss.nasa.gov/gistemp/graphs/Fig.D.txt
If you go to that GISS page today, you will find that 1998 has regained its position as the warmest year in the US: 1998 is 1.29 and 1934 is 1.26.
Strange how that GISS “blackbox” calculation of annual means works. Depending on when you look at the posted results, you may find they have changed since the last time. Someone ought to take screenshots once in a while, so he can put together a “blinking” picture of the dancing numbers.
I do not know the details of HADCRU’s correction scheme, but it cannot be very different from NCDC. If you go to this page right here, you will find a pretty thorough explanation of NCDC adjustment/correction scheme. The graph of effect of individual adjustments plus the graph of over-all adjustment tells the entire story of how the current decades and 1998 in particular appear on graphs so warm compared to the 1930s. The pertinent question to ask is “Are these adjustments correct and justified?” There is no answer at present, but there is reason for doubt.
I hope this helps.
Micajah (16:27:33) :
If you had been keeping tabs, you would also have noticed that 2006 moved from +1.10ºC to +1.29ºC, now tying with 1998.
DaveE.
Okay, some stupid questions from the guy who rides the short bus to this site. Am I correct when I look at the first, complete e-mail and see that in July 1999, they have 1934 at a plus anomaly of 1.459, and 1998 at a plus anomaly of 0.918? And then, by August 2007, they state that the plus anomaly for 1934 is down to 1.249, while the plus anomaly of 1998 has risen to plus 1.226? Is that what it says?
I’m somewhat familiar with the issues of diddlin’ with the data, and don’t understand the whys and wherefores of most of it, but some of it I understand. But if I am interpreting this e-mail correctly, that’s some heavy- duty diddlin’ of data. Why are historical figures changed this often, and changed to this degree? All I can say is: “VIVA VIAGARA!”
So, in order to answer a FOIA request, the people who believe the planet is in imminent danger of catastrophe printed all the emails onto paper, scanned them, and then sent them electronically.
The UK CRU version of Climategate centered around whether the 1990’s were warmer than any time in the past 1000 years. The US GISS version could be about whether 1998 was warmer than 1934!
It seems the temperature readings were adjusted six times after analysis in July 1999 indicated that the temperature anomaly for 1934 was nearly 60% higher than for 1998. I’ve graphed the seven versions to show how GISS systematically adjusted 1934 down and 1998 up until 1998 was warmer than 1934 (the January 2007 analysis) or at least virtually indistinguishable (the March and August 2007 analyses).
And this is just from one email in a treasure trove of 216 pages of them!
In the UK CRU case, the Medieval Warm Period vanished to present a “nice tidy story”. In the US GISS case, a nearly 60% temperature anomaly difference vanished to show that 1998 was as warm as 1934! Are these guys serious scientists or just skilled magicians?
As the Atomic Doomsday Clock’ post has shown “settled sciences” scaring tales for a world governance “project” which began back in 1945:
Changes in the Doomsday Clock are determined by the Bulletin’s Board of Directors in consultation with its Board of Sponsors. No less a personage than Albert Einstein helped found the Board of Sponsors (which currently includes eighteen Nobel laureates), and Robert Oppenheimer was its first chairman.
So, now “Climate Change/Global Warming” , by the foolish behaviour of its now desesperate promoters has become an unexpected and until now unknown wider and older conspiracy:
http://www.agoodmanonline.com/pdf/free_range_2007_04.pdf
Their aim:
To equalize the economies of rich and poor nations, he proposed that funds from nations with trade surpluses would be confiscated by an international bank (or clearing house) and transferred to poorer nations whose goods had been exported at low prices
http://www.thebulletin.org/web-edition/features/bringing-climate-change-global-governance
When I was a child, NASA were my heroes. Childhood ended some time ago.
Paddy (16:13:27)
Judicial watch has gone after many cases of corruption over the years since I have followed them (at least since the Clinton era). I read their newsletter for many years. But they also glom on to various issues where they perceive some populist political sentiment, and then they run their press releases with just one or two facts and large amounts of innuendo.
When they do thorough research to uncover corruption where they bring strong legal cases, they are quite effective. But when they go after issues like this one they are not going to have the knowledge and depth to make an effective case. I notice for example that there is no discussion of any legal issues which is their speciality. They really need to stick to their speciality.
True to form:
NASA = Never A Straight Answer
u.k.(us) (15:34:35) :
“history used to be written by the victor, not any more.”
history used to be written by the vicar, not any more
FTFY
A number of commenters on this thread helped me write this post:
http://powerandcontrol.blogspot.com/2010/01/diddlin-with-data.html
Thanks to you all and to our gracious host for bringing this to light.
SteveSadlov (17:15:44) :
When I was a child, NASA were my heroes. Childhood ended some time ago.
I was an adult when our joint heroes were held so highly. Isn’t it sad when we find out that heroes end up as villains. Apologies to the true NASA Champions, those that stole your mantle are beneath contempt! You guys still rock, thank you!
In mid-January 2009, the notorious Hansen declared unequivocally that “this coming year [’09] will be the warmest yet on record.” When Roger Pielke, Jr. queried the factual, scientific basis for this transparent folly, Big Jim responded with Warmists’ usual Argument from Authority, by Stipulation: “It’s true because we Ascended Masters say it is.” Pielke, Jr. coughed discreetly and named Hansen recipient of his coveted Bonehead of the Year Award, averring that despite their best efforts not even Climate Cultists like Briffa, Jones, Mann, Trenberth et al. would likely outdo James Stylites’ pronunciamento from atop his self-anointed NASA pillar.
Anyone in contact with Pielke, Jr. might request his nominee for 2010. As Edward Lorenz said of Earth’s “climate” in general (1960), “The answer, at first glance obvious, improves on acquaintance.”
What this all tells me is that my suspision that NASA has a time machine has to be true. Its the only way they could know that the temperatures back in th 30’s weren’t as high as the written records say they are.
Actually the big baddie in all this really isn’t NASA it’s NOAA. NOAA is the one that compiles the GHCN and USHCN data sets and then mangles them and sends that output off to NASA GISS were they try a one size fits all de-mangle proceedure, then apply there own mangling proceedure which they hold up as something of real value.
Re: David Alan Evans (16:39:59)
You’re right: 1934 now gets the “purse” for third place, since 1998 and 2006 are tied for first place. I had only looked to see how 1998 and 1934 had changed.
I wonder if things will change again when the figure for 2009 is posted. The GISS procedure seems to involve “spooky action at a distance,” since prior years are often changed by the entries for later years.
One of my favorites from 23 Aug 2007 (p 111)
Andrew Revkin Asks –
“..finally, do you agree that generally we (globally) should be doing a lot more to improve surface temperature tracking? i never, til today visited http://www.surfacestations.org and found it quite amazing. if our stations are that shoddy, what’s it like in Mongolia?”
James Hansen replies –
“…Of course it is good to improve the station data. Temperature is an absolute measurement, however, so errors over time are not cumulative. When there are several thousand stations it is easy to find what seem like a huge number of stations with problems.”
I’d say “it is easy to find what seem like a huge number of stations with problems” when 89% of the stations surveyed by Anthony rate below CRN 2
Not A Carbon Cow (17:38:41) :
u.k.(us) (15:34:35) :
“history used to be written by the victor, not any more.”
history used to be written by the vicar, not any more
FTFY
=========
had to look up meaning of “ftfy”, now i know 🙂
quick internet search brings up: Winston Churchill prophetically said: “history is written by the victors.”
if victors was wrong, my next guess would have be generals.
i was debating which to use in my comment, and was correct i guess.
I spent some time eyeballing some of the the emails this evening after work, and the ones that left a particularly bad taste in my mouth were the ones by US-taxpayer-funded James Hansen.
His tone in the emails matches his tone whenever he speaks: Sheer arrogance.
That type of arrogant, insulting tone is not befitting for someone who draws his salary from public funds.
But….what do you expect from a megalomaniac.
Hey JIM!!! You and your ilk….Gavin….Michael…and the rest of “the team” and henchmen like Gore and Holdren that feed you, listen to these words:
YOUR REIGN IS OVER.
Its only a matter of time….and not a minute too soon!
Chris
Norfolk, VA, USA
I’ve read through 87 of these e-mails and this is no Climategate Episode 2. It makes McIntyre look bad in a way since they are somewhat successfully pointing out to reporters and each other some things that McIntyre may have misunderstood (perhaps justifiably). GISS has had its code available all along, I believe, whereas McIntyre was asking for additional code that simply did not exist, based on rumors from the SurfaceStations.org project that NASA adjusted for bad station siting in some manner that was not included in the public version of GISS (?).
I also must note that GISS does not evidently use “already adjusted” data. That’s another rumor I believe is false. The actual GISS source code that I have seen grabs data from GHCN raw and not GHCN adjusted.
It speaks of McIntyre using a software robot to grab all the GISS station data from their public site that only allows listing of one station at a time. It operated too fast for their servers to keep up with and still provide a fast response to general users, so they blocked his IP, prior to knowing that it was his IP and prior, evidently, to knowing who he himself was (they claim). They then did seem to prevent him from getting their own data rather than what they claimed was identical data from GHCN, however.
As of page 87 it’s all about GISS’s ‘Y2K bug’, in which they used a kludge to extend the US data after the USHCN stopped applying corrections, by using GHCN data instead. So they add a further kludge to artificially pull the GHCN data downwards to mend the step jump. Sloppy, indeed, but nothing along the lines of CRU machinations. The tone is one of somewhat hassled defensive damage control in a reactive sense, not the pro-active sort of offensive planning of CRU e-mailers. I would call it a case of a bad first impression, since Hansen seems to learn about ClimateAudit.org for the first time in the context of a literal attack on their web site (over-zealous spider software acting on a site that uses lots of CPU intensive scripts). So they wrote back that he had been un-blocked and could data mine in the wee hours of the night. Case closed, except for their discomfort with how the story would play out in the media, which is understandable since they are getting a barrage of inquires from reporters about it.
But later, Gavin (confusingly, to me) tells chief programmer Ruedy (http://www.giss.nasa.gov/staff/rruedy.html) that “It does however highlight the rhetorical power of saying that the code is secret and things are being kept from the public. It may still be worth putting up a clean version of the adjustment program on the website in order to have something to point to in such cases.” (page 120)
They make a further confusing point that the bug did NOT effect 1934 nor 1998 so could not have altered their ranking (even though it did?). Yet they make the point too that it only applied to US data, well within the margin or error, and make only minuscule difference to the global average (the US being only 2% of the surface of the planet).
What is *highly* confusing to me is that they point out again and again that from 2001 (when they published a big paper), that 1998 and 1934 were a virtual tie (within 0.01°), but now (http://data.giss.nasa.gov/gistemp/graphs/Fig.D.lrg.gif) 1998 is considerably hotter. However, as I state above, I believe the full source code and the exact raw data is available at present so should be fully auditable (ClearClimateCode.org is an example of this in progress).
Ah! I was looking at the red line instead of the yearly points in black. They are still a virtual tie in the US (but not global) data.
They do however mention that they are being stubborn about using the latest USHCN data rather than an older version. Discussion about this does not sound conspiratorial but indeed sounds very round about and bumbling in tone. They simply don’t care what difference it will make except in the sense that they want to head off any bad PR if the difference seems odd. I’m up to page 130 now.
Not much more intrigue, but by page 180 the story is hitting the press, with the typical back-and-forth pro/anti slants, with them providing further pro-spin damage control as additional interest is being generated. It’s pretty boring stuff though. They discuss using the episode as a way to get their message out better. Their is no Michael Mann figure causing more harm than good in these. Hansen is curt instead of verbosely ponderous, and Gavin, always present, merely provides some textbook PR advice about not getting dragged into every argument.
Then BOOM, at the very end (page 205 of 215) we confront possible and seemingly casual/sloppy data destruction as they attempt to produce a before/after plot of the correction. The crud has really hit the fan now and they are getting inquiries from student types but they don’t yet have a good form letter reply, it seems.
At one point (page 209) Hansen quaintly seems to refer to the Internet the “ethernet”: “Your e-mail should be framed, as a counterweight to all the viscous ad hominem e-mails that have descended through the ethernet.”
In conclusion, the thrust of the issue, the “Y2K bug” in GISTEMP, which is what these are all about, is not a huge smoking gun. What *remains* as a smoking gun is casually sloppy data and code archiving, over time, so that a historical audit of changes over time has become quite likely impossible to document and analyze. However, the idea that current code and current raw data is being withheld by GISS is not supported by these e-mails.
“Temperature is an absolute measurement, however, so errors over time are not cumulative”
Then how come those “absolute measurements” keep changing, Jimmy?
Peter of Sydney (14:17:11) :
It’s a pity to see what was once a vibrant and useful organization as NASA deteriorate down to it’s current level of embarrassment and malpractice.
It seems to be an endemic condition at NASA. See also the laxity in adhering to procedure which led to the 1986 loss of Challenger and the 2003 loss of Columbia. The “What? Me worry?” attitude also led to the Apollo 1 fire.
Not too long before the loss of Columbia, there was a documentary called “Space Shuttle Garage”, about what a Shuttle Orbiter goes through to get it ready for launch.
The part I found especially amazing/disgusting was when a group of mechanics could not figure out how to use the special torque wrench for tightening the landing wheel nuts. They went through the procedure manual, which is a very large looseleaf binder (why is that not on a computer?!), and they still couldn’t get it to work. So they got the other special torque wrench (there are only two in existence) and still couldn’t get the nuts torqued correctly.
Finally they gave up and put a note on the preflight checklist saying they didn’t get the wheel nuts torqued correctly. Can’t make it work? Leave it up to the next shift and hope they don’t lose the Post-It notes.
I don’t recall now which it was on, History Channel, Discovery Channel or The Learning Channel. I did in 2003 and went to their website right after the Columbia crash and it wasn’t available to buy. Anyway, quite the poor choice of video to showcase how NASA operates.
Hooo boy! Someone at NASA is going to get the boot! (Possibly.) Someone’s stash of cocaine was just found in the Shuttle garage. http://www.heraldsun.com.au/news/breaking-news/cocaine-found-in-nasa-space-shuttle-garage/story-e6frf7jx-1225819834366?from=public_rss
Gavin, 16 Aug, 2007 18:16:22 emails R. Ruedy:
“the issue is here that you are dealing with a hostile interviewer. In such circumstances, it is much better simply to point out clear errors. If you open up another front they will dive on that instead and abandon all the previous positions (since they are not sincere in any case).
It does however highlight the rhetorical power of saying that the code is secret and things are being kept from the public. It may still be worth putting up a clean version of the adjustment program on the website in order to have something to point to in such cases.”
————
“Clean version”? Meaning, one without a “fudge factor”??? Now, me thinks, we need to see the “dirty” version so we can judge for ourselves. I’m not trusting these code monkeys.
boballab (18:25:39) :
Actually the big baddie in all this really isn’t NASA it’s NOAA. NOAA is the one that compiles the GHCN and USHCN data sets and then mangles them and sends that output off to NASA GISS…
My take on the progression of raw data to “value added” data is as follows:
(1) NOAA (V2.mean.z vs V2.mean_adj.z) adjusts for station moves. (mostly minor and defendable adjustments).
(2) NCDC (Menne) makes major adjustments for a multitude of factors – See:
http://www.ncdc.noaa.gov/oa/climate/research/ushcn/
My favorite section is Homogeneity Testing & Adjustment Procedures – It made my head spin. I’d copy it here but it is pretty long.
(3) GISS probably uses the NCDC dataset. I’ve compared the 2 datasets for San Antonio and they are basically the same – but other locations may vary.
Ditto. The Earthbound, ingrown (like an infected toenail) NASA of Hansen is not the NASA of the space-faring pioneers of Mercury, Gemini, Apollo, the Space Shuttle, the Jet Propulsion Laboratory, the Hubble Space Telescope, and many other noble and exciting endeavors. The Agency just needs a good rinse and an antibiotic to clean out the nasty detritus of politicized ideological enviro-whackoism.
/Mr Lynn