Guest post by Christopher Horner, Planet Gore at National Review Online
One thing about “ClimateGate” nagging at the back of my mind is the absence of any discussion by ringleader Phil Jones (or others) of the remarkable, shocking discovery that Jones now claims he had that his precedessor destroyed the raw data in the 1980s.
That is the data that scientists have for years been seeking from Jones under the UK’s freedom of information law. Against numerous such requests he offered equally numerous excuses for refusing access culminating with the September 2009 claim — when it looked like he’d been cornered and had no excuses not to provide it to Prof. Ross McKitrick who met all of his long-stated qualifications — that in fact he’d lost it.
First, it does seem odd that Jones would so firmly and crisply articulate his many, very specific excuses for so many years about why he could not provide something that in fact they had, as he now tells it, lost. His refusals all clearly imply that a belief that he had it.
But where are the emails putting out the word, oh, snap, you guys aren’t gonna believe this?
Among all that has been revealed, there does not appear to be one. Let alone a chain discussing the importance of not at long last actually having the raw, how this loss might relate to the scores of emails they wrote about whether to release the data and how to avoid releasing the data and how they’d rather destroy it (I don’t know, “pretend to have lost it”) than give it to the folks who seem to be on to them.
This seems like a big email, and a chain of discussions that would pervade that which has been revealed. It doesn’t.
To the contrary, we have numerous emails from Jones explaining how turning over the raw data is one option, but he’d much rather destroy it than let the intrepid start pawing over it which could only lead, as he admits in one email, to figuring out what CRU et al did to said raw data in order to come up with their alarming claims.
So there is a reasonable conclusion, and it is not that the data was lost or destroyed twenty years ago.
But who knows, maybe Jones wrote James Hansen at NASA, or Gavin Schmidt — for so long a taxpayer-funded activist for Environmental Media Services’ RealClimate.blog and now implicated as a major player in these emails (Capo number 6 according to this analysis). Those should turn up when the courts help NASA figure out how to come into compliance with their legal obligations and provide me similar data and correspondence that they have been, similarly and by chance, refusing me for over two years.
Christopher C. Horner Senior Fellow Competitive Enterprise Institute 1899 L. St, NW 12th Floor Washington, DC, 20036 +1.202.331.2260 (O)
Author of the newly released: Red Hot Lies: How Global Warming Alarmists Use Threats, Fraud, and Deception to Keep You Misinformed http://www.amazon.com/Red-Hot-Lies-Alarmists-Misinformed/dp/1596985380/ref=pd_bbs_sr_1?ie=UTF8&s=books&qid=1231180047&sr=8-1
Author of The New York Times Bestselling The Politically Incorrect Guide to Global Warming (and Environmentalism) http://www.amazon.com/Politically-Incorrect-Global-Warming-Environmentalism/dp/1596985011
D. Patterson (06:21:21) :
edriley (03:21:19) :
where there’s a will, there’s a way
Indeed see NCDC dataset id C00144…
A Surface Weather Observations is a historical manuscript collection of records archived at the National Climatic Data Center (NCDC). This data file contains original manuscript records of raw meteorological data collected by 1st order and 2nd order station located in the U.S., U.S. Pacific Islands, U.S. Virgin Islands, Puerto Rico, and by military weather stations located worldwide. Temperature, precipitation, pressure, wind, visibility and cloud data are covered from 1872 to the present. Observed parameters and number of daily observations varies among military and second order stations. Hourly and/or 3-hourly data and summary of the day data that are entered on these manuscript records are also available on file as DSI-3280 (C00215) and DSI-3210 (C00314). In addition, these data are published as Local Climatological Data, DSI-3715 (C00128).
Did you notice it costs big $$$. To obtain the original data covering 100 years monthly for all stations would cost tens of thousands. Pretty big impediment there.
ozspeaksup:
Is this what you were thinking of?
http://camirror.wordpress.com/2009/11/25/willis-eschenbachs-foi-request/
“We cannot produce a simple list with this format and with the information you described in your note of 14 April. Firstly, we do not have a list consisting solely of the sites we currently use. Our list is larger, as it includes data not used due to incomplete reference periods, for example. Additionally, even if we were able to create such a list we would not be able to link the sites with sources of data. The station database has evolved over time and the Climate Research Unit was not able to keep multiple versions of it as stations were added, amended and deleted. This was a consequence of a lack of data storage in the 1980s and early 1990s compared to what we have at our disposal currently. It is also likely that quite a few stations consist of a mixture of sources.”
As Jones FOIA lackey pointed out in responses, the original data does still exist. . . in the repository of the various NWS (national weather services).
If Jones successor really wishes to make amends for this period of travesty of the principles of science, CRU will make the effort to go around and reacquire that data, securing at the same time the necessary agreements to make it all public so that all researchers may use it as a common resource.
If the AGW issue is as important and urgent as claimed, I see no reason to think it should be all that hard for CRU to do, nor that any NWS would refuse them.
How much data are we talking about? Here’s my back of the envelope calculation:
Back in 1984 an IBM magnetic tape cartridge was 4″ x 5″ x 1″ and stored 200 MB.
According to Gavin Schmidt of RealClimate, 95% of the adjusted CRU data or its equivalent is available at http://data.giss.nasa.gov/gistemp/ .
I’ve been to that site and based on the various download pages, that data plus various software programs is 100 MB tops. Now maybe there was additional metadata and other source materials with the raw data but surely those wouldn’t have gone over 100 MB.
In other words the original data would easily have fit on one 200 MB cartridge. Throw in a couple of backups and the original data wouldn’t have taken up more room than a decent hardback dictionary.
Or perhaps in those endarkened days, the data was uncompressed. So call it a shoe box worth of cartridges.
I just don’t believe the CRU scientists idly lost or threw out that data while housecleaning or moving to a new building.
Comments or corrections?
Ron’s 22:02:21 comment rings true. If the only data set now available is “value-added” data that has gone through some preprocessing phase, why did the HARRY_READ_ME guy spend three recent years going half out of his wits trying to make sense out of chaotic directory structures and inconsistent data formats? How could anyone, when modifying the original raw numbers, fail to normalize the data in other ways so as to organize it into a coherent, accessible form? I honestly don’t get it.
Of course, which is why I mentioned the dataset.
I’ve been trying for years to learn precisely what happened to the paper forms we forwarded to NCDC, but I’ve always been diverted to the NCDC dataset catalog you see online without enough information to determine what really happened to those forms. I am now informed some of the paper forms, I don’t know to what extent, have been destroyed by neglect and/or discarding. It is unclear the extent of which raw surface weather observations of air temperatures remain available in completely unadjusted and unedited formats. Based on the verbal reports I’m presently receiving, the situation is highly dubious at best. We already know the air temperatures in the historical climate networks have been compromised by adjustmetns which likely cannot be undone to restore the raw observation values. So far, it isn’t looking much better for the great majority of the other datasets. The story is still unfolding, so we’ll have to see what develops….
Larry Sheldon (06:46:31) :
I still have some Shugart 8-inch FDD out in the garage. Don’t know if they still work or not.
Looks like the MET still has the data anyway.
If so, then “Jones FOIA lackey” is deceiving the public. At least some lesser or greater part of the original paper records were destroyed and damaged in the repositories where they were archived. The digital datasets have with distressing frequency been adjusted, making the original values unretrievable from the dataset. It will require a detailed investigation to determine the extent to which unadjusted and unedited raw surface weather air temperatures can be retrieved.
chemist (06:41:38),
You referred to a CSIRO study which
Could you supply a reference for that? I am interested in applying a similar analysis to modern educational papers, particularly since the use of APA style of citations means that any paraphrase of a cited work requires no page numbers (thereby adding great difficulty to the checking of that work’s accuracy and applicability).
If you deplore the fudging of data and the poor arguments of some scientists, you probably won’t be impressed by the loose thinking, sweeping conclusions from extremely small samples, and outright idiocy of many educational theorists.
I don’t remember seeing this mentioned anywhere—
Is there enough information still in the system to back out the distortions and recover the original inputs?
Or was it all one-way, or logically random, distortions?
Stop fretting about reading old media.
A cache of tapes storing the ORIGINAL broadcasts from the Lunar Landings, showing ultra-high-resolution photographs of the activity, still exists.
Volunteers have cobbled together parts from old tape machines and managed to read the tapes. See Sky and Telescope magazine for details.
Just put out an APB. Some hobbyist has a garage full of these old beasts.
As for paper, we now have optical scanners.
Punch tape? Punch cards? I really don’t care.
If we can scan an old palimpsest and recover a lost treatise from Archimedes, we can find whatever we want to find.
By the way, wasn’t the Surface Stations project done by volunteers?
Huxley suggests the original data cold have been put on one 200MByte cartridge.
Unless there was a system for the 4000+ weather stations to write their data first onto that single cartridge, that would not be _original_ data. It would be a copy of it.
I shut down Canadian Blue Lemons a few months ago because I thought the work was done by weight of fact. And now that it seems that the sins and sinners are revealed, it might be more important than ever to keep ploughing through the freezing nonsense that continues to dominate the news casts (although the worm is on its back and suffering).
The first time I took on the evil establishment our advisor said when you get your foot on their throat you push down, long and hard.
The data are still out there.
Jones, or whomever, might have destroyed the data they had.
But, those data are still around. The CRU reports their sources.
Do it again. Publish them. Show the gaps.
I have worked on hundreds of statistical time series, and plugging is the only way to fill gaps in data, based on the knowledge of the plugger. Doing this I was always ready to explain my plug and the degree of variance that might be expected.
Demand that Jones reproduce his data.
If they were available then – they are available now.