While I’ve been getting lots of attention for “take down” of a single file that infringed on my copyright, another, much more broad and serious event is unfolding in the UK at the Hadley Climate Centre.
It appears that the “mole” has caused a Centre-wide panic and they are purging publicly available climate data.
If their climate science is so solid, so unassailable, why would they need to do this? Why hide the climate data gathered from public domain sources worldwide such as NOAA and NCDC? Steve McIntyre tells the story and wonders also. I’m sure all of those who complained in my case will do the same about Hadley, since it will affect the climate community worldwide. I suppose we now have a new term: “Climate Data Deniers” – Anthony
“Unprecedented” Data Purge At CRU
On July 31, 2009, the purge of public data at CRU reached levels “unprecedented” in its recorded history. Climate Audit reader Super-Grover said that the data purge was “worse” than we expected.
On Monday, July 27, 2009, as reported in a prior thread, CRU deleted three files pertaining to station data from their public directory ftp.cru.uea.ac.uk/.
The next day, on July 28, Phil Jones deleted data from his public file – see screenshot with timestemp in post here, leaving online a variety of files from the 1990s as shown in the following screenshot taken on July 28, 2009.
The following day, the following listing of station data available since 1996 (discussed in my post CRU Then and Now) was deleted from public access: ftp.cru.uea.ac.uk/projects/advance10k/cruwlda2.zip, though other data in the file remained.
This morning, everything in Dr Phil’s directory had been removed.
This is part of a broader lockdown at CRU. Ian Harris, Dave Lister, Kate Willett, Tim Osborn, Dimitrios, Clive Wilkinson and Colin Harpham all altered their FTP directories this morning. Only one directory (Tim Osborn -see below) has added material.
Revisiting the Advance 10K webpage this morning, all Advance 10K data was deleted from their FTP site. None of the Advance 10K data links at www.cru.uea.ac.uk/advance10k/climdata.htm work any more.
If you go to the directory page ftp.cru.uea.ac.uk/projects which formerly hosted ftp.cru.uea.ac.uk/projects/advance10k directory, it now contains only two directories between Sept 1999 and the present, both dated 8/1/2008, but containing data from 2001.
On July 31, 2009 at 10:41 am, Tim Osborn published a webpage entitled “controversy.htm”. It is located in a folder entitled www.cru.uea.ac.uk/people/timosborn/censored/ and the webpage www.cru.uea.ac.uk/people/timosborn/censored/controversy.htm itself is of course censored.
I presume that the data has not been totally destroyed, only that, after many years of public availability, it has been put under lock and key. It’s as though CRU is having a collective temper tantrum.
Perhaps the files were taken offline because they show massive alteration when compared to earlier versions. Wouldn’t want incriminating files lying about. Hard to tell what the thinking is. Most science sites I know of put up a “Temporarily down due to maintenace” note. They don’t just start deleting files under full view.
Wade (16:52:02) :…. “I think of this like the right to remain silent and right to not allow search without a warrant. Some may say that if you have nothing to hide then you should allow your property to be freely searched by law enforcement. The US government was founded on preventing such broad assumptions.”
————————————————————–
You miss the point Wade…. These people are publicly funded. That data is public data, it belongs to the the UK taxpayer.
Therefore it is not CRU’s property to hide…… These people are getting above themselves.
I speculate that even the AGW Scammers might be getting weary of The Scam (I sure am) at this point (because nobody intelligent falls for it), and the inexplicable behavior belongs in the classic ‘Criminal Who Wants To Get Caught’ scenario. 😉
Andrew
Mac (19:49:54) :….. But Mac, it is CRU that won’t share data an who is stopping other people from examining their findings and methodology. CRU is abusing the FOI act on publically funded research data….. Why shouldn’t Anthony kick up a stink at every possible excuse?
Plus they don’t answer his “why’s”.
I don’t know where you get the idea that Anthony’s behaviour is “tiresome”…. CRU’s behaviour is beyond childish.
…. Anyway, we’ll wait and see what this tree shaking exercise does. See what nuts fall out;-)
Wade (16:52:02) :
“I think of this like the right to remain silent and right to not allow search without a warrant. Some may say that if you have nothing to hide then you should allow your property to be freely searched by law enforcement. The US government was founded on preventing such broad assumptions.”
In asserting this may be nothing, Wade is correct.
But the legal rights he cites here are specifically designed to protect individual citizens from the overweening power of the state. They are not designed to protect the overweening power of the state from individual citizens.
Nor to hide public data from the tyrannical citizenry which, in fact, owns it.
Surely, the mole has already backed up all of this data.
This data was visible to the public, whether it was intended to be visible or not. If any of these files have been published, such as through links from their web site, then their deletion is odd.
Of course, they should be keeping copies as required by their policies.
Perhaps they are responding to the FOI requests- gathering all data that has been asked for by Steve and others. After collation the entire data set with code and intermediary steps will be forwarded with best wishes.
So,what if a researcher,say AlGore, wants the data,is it kept at the CRU HQ, under ground in a bomb proof vault on say, a series of linked 60’s vintage mainframe IBM computers,all accessible with the appropriate knowlege of FORTRAN and use of punchcards?
There are subtleties here in the UK which our US friends might not appreciate.
I cannot go further at this stage until I have found out why the data has been removed, which is the first step.
It might be wholly innocent you know, servers down, it’s August and we are revamping the site or whatever.
But once I have answers to that, credible or otherwise, I can start to dig into the procedures and protocols by asking individual ministries, departments and government agencies to provide answers.
That is because this information is not the sole province of any one of them: so each must answer as to its interest in and usage of the data.
And there are more of them than you might think: and more Acts of Parliament that define their powers than you might imagine. And strange ways and means apart from an FOI that you have probably never heard of.
Nor is the money they spend unaccountable either. Far from it. We can ask about that too: and track it.
This is far from filing a simple FOI and hoping for an answer, it is a slow and cumbersome process but eventually like the mills of God it grinds exceeding small.
It is true that civil servants working for a new and robust administration can be very obstructive, especially when the government has the Meeja behind it. Then you really do have to flex some muscle to get anywhere.
But this is a dying administration which many civil servants at any level, especially the middle and upper echelons, have no cause to love, and in any event they know it must necessarily fall by the middle of next year.
So I don’t imagine they will put up too robust a defence.
We shall see.
Kindest Regards.
a jones,
Ploy Away!
and heartfelt thanks in advance….
It appears that the “mole” has caused a Centre-wide panic and they are purging publicly available climate data.
Things aren’t always what they appear to be. This might be true, but if it isn’t, you better believe they’re going to use this to their advantage to discredit this blog and Climate Audit as over-reactionary.
Just erring on the side of caution for now, as others have stated. There could be another explanation. However, if it turns out to be true, this has 8 more legs than “CarbonGate” ever did.
Call me a denier for the next few days until this one shakes out.
REPLY: Well the timing is suspect. Some of these these files have been there for years. – Anthony
Two things; first, a jones, it will be most interesting to hear your updates. Thank you.
Second, SunSword brought up a matter that concerns me. I am not mathematically or technically trained, as are the commenters and our gracious host, so I am dependent upon clear expository writing and honest data and graphs to even begin to keep up with you. The idea of controversialists jacking with online sources such as Wikipedia so as to falsely promote their position makes me more angry than I care to abuse the moderators’ patience by expressing. This is exactly the sort of (snip) I expect from them — they can’t support their position with science so they resort to dirty tricks to fool people like me who can’t catch them without help. Whatever it takes to win. There’s money at stake here!
Infuriating.
Re: Philip_B (19:34:57) :
“If tax dollars paid for the info to be gathered, then on what basis is it withheld?
The same basis as info gathered by another entity.”
Huh? A person or company in the private sector is looking to make money off their product or information. How is that the same basis as public funded non-profit, government work? Once again, WHY, in specific, should such info be withheld?
crosspatch (19:36:34) – good point on fairness, though that is a different topic than what I was addressing
Wade sez:
“I think of this like the right to remain silent and right to not allow search without a warrant…”
Dude, in science the hypothesis is always assumed guilty and does not have the right to remain silent or not have it’s data or methods searched.
Huh? A person or company in the private sector is looking to make money off their product or information. How is that the same basis as public funded non-profit, government work? Once again, WHY, in specific, should such info be withheld?
You appear to be confusing,
Whether data is the property of some entity.
Whether data should be the property of some entity.
Many public bodies make money from selling their data, which requires that it be proprietary, and I believe the UK’s Met office is one of them.
regards
SunSword (18:31:41) :
I was going to give CRU the benefit of the doubt until I found this on the UEA web site. It looks Phil Jones wants to ‘adjust’ the MSU temperature record now. A self funded research project entitled ‘Can we create a better Microwave Sounding Unit climate record through the use of high-quality in-situ data?’ with Professor Phil Jones as one of the supervisors can be found here:
http://biobis.bio.uea.ac.uk/Resproject/show.aspx?ID=23
‘However, the [MSU] measurements have been made with forecast input in mind leading to inevitable and insidious non-climatic influences permeating the record’.
‘Taken together these data should be sufficient to allow a fundamentally different approach to be undertaken to MSU dataset development and hence to re-evaluate currently available MSU-based series’.
Self funding probably means funding by some environmental group. The outcome of the research is ‘inevitable’. No doubt the ‘insidious non-climatic influences’ will be removed and the warming will reappear.
Anthony, this might be worth a separate post. I believe the cops call this MO.
I bet the order came down to clean up all the folders that can be acessed by the pubnlic. Jones has always maintained that no data can be made public, and there it was, on a publicly accessible ftp server.
I have submitted a FOI request to the Met Office seeking an explanation fo the denial of this data.
Previuosly, I had asked whether their model could be run backwards in time. Their response was :-
> > Date: Mon, 5 Jan 2009 11:35:35 +0000> > From: enquiries@metoffice.gov.uk> > Subject: re[2]: FWD: Climate prediction model> > To: john_edmondson@hotmail.co.uk> >> > Dear John,> >> > We do not run climate models with time going backwards> > in a literal sense as climate models are complex numerical tools> that> > are definitely not designed for this purpose (an analogy being to> try to> > operate a car by injecting exhaust gases through the tail pipe and> > expecting it to drive itself backwards and for petrol to accumulate> in> > the tank – clearly nonsensical).> >> > Running a model “prediction” with time going forwards but climate> > forcing (such as greenhouse gases and aerosols) changing in a way> that> > reverses what has happened in the past is possible, but we have no> > particular reason to do this since it does not help much in> answering> > relevant scientific or modelling questions.> >> > We do, however, run climate models regularly (in a forward sense) as> a> > means of evaluating their ability to reproduce past climate changes> (of> > the last 150 years, or of more ancient periods when the Earth’s> orbit> > around the Sun presented a significantly different pattern of solar> > insolation through the year at given latitudes).> >> > I hope this helps.> >> > Regards,> >> > Suzanne.> > (On behalf of an expert)> >> > Weather Desk Advisor> > Met Office, FitzRoy Road, Exeter, Devon, EX1 3PB, United Kingdom.> > Tel: 0870 900 0100 Fax: 0870 900 5050 Email:> enquiries@metoffice.gov.uk http://www.metoffice.gov.uk
I later asked if the model could be run from 1600 AD , but all I got back was some scientific paper sidestepping the question.
Can anybody answer my question as to whether running the model backwards in time or running from 1600 AD is a valid method of testing it’s accuracy?
Hadley and the CRU are intertwined, hence ‘HadCRUt’. The giveaway is in the name.
It may of course be possible that:
1. The “mole” was actually the ftp server itself.
2. That would account for the public announcements from CA and WUWT indicating that the source was valid and the IP address had been confirmed as within CRU.
3. It would also account for why CA et al. were happy to discuss it openly (despite many posters being concerned regarding the “mole’s” future employment.)
4. It would also account for why the ftp server is now being “purged”.
So, my theory (for what it’s worth) is that data was discovered on a publicly accessible ftp server in folders which the owners thought were private.
Just my theory :o)
It may of course be possible that:
1. The “mole” was actually the ftp server itself.
2. That would account for the public announcements from CA and WUWT indicating that the source was valid and the IP address had been confirmed as within CRU.
3. It would also account for why CA et al. were happy to discuss it openly (despite many posters being concerned regarding the “mole’s” future employment.)
4. It would also account for why the ftp server is now being “purged”.
So, my theory (for what it’s worth) is that data was discovered on a publicly accessible ftp server in folders which the owners thought were private.
Just my theory :o)
P.S. – Sorry, forgot to tell you great post!
J.Hansford (20:17:26) :
Wade (16:52:02) :…. “I think of this like the right to remain silent and right to not allow search without a warrant. Some may say that if you have nothing to hide then you should allow your property to be freely searched by law enforcement. The US government was founded on preventing such broad assumptions.”
————————————————————–
You miss the point Wade…. These people are publicly funded. That data is public data, it belongs to the the UK taxpayer.
Therefore it is not CRU’s property to hide…… These people are getting above themselves.
————————————————————————————
There are plenty of things funded by the UK taxpayer that are not public-domain. There may be commercial reasons (data can be worth money) or other sensitivities – remember that MI5/MI6 are publically funded – would you send them an FOI request about the latest Al-Q plots?
So to talk about this being public-funded as a major objection is a red herring.
My taxes fund the British Geological Survey but if I want mundane geological datasets from them I have to pay a lot for the privilege – and that’s the stuff that CAN be accessed. It started getting tight like this during the Thatcher years and really needs looking at.
The CRU data as I understand involve a lot of stuff obtained via various international agreements.
Bye for now – John
Anthony, you and SteveM’s got some good coverage on this week’s podcast by Jim Puplava. The link to the audio is here:
http://www.financialsense.com/fsn/main.html
Click on 3rd Hour with Jim & John – part 2. The climate part starts 34 mins from the start. The first part is about the health care bill which is even more scary than the climate change scam. I always find these podcasts great to listen at the wekend.
the story finally made the press today
http://www.telegraph.co.uk/comment/columnists/christopherbooker/5955955/Weather-records-are-a-state-secret.html