Let me review the request situation for readers. There are two institutions involved in the present round of FOI/EIR requests: CRU and the Met Office. Phil Jones of CRU collects station data and sends his “value added” version to the Met Office, who publish the HadCRU combined land-and-ocean index and also distribute the CRUTEM series online.
I requested a copy of the “value added” version from the Met Office (marion.archer at metoffice.uk.gov) which has been refused for excuses provided in my last post. On June 25, 2009, learning that Phil Jones had sent a copy of the station data to Peter Webster of Georgia Tech, I sent a new FOI request to CRU ( david.palmer at uea.ac.uk) requesting the data in the form sent to Peter Webster. This too was refused today.
We now have a new excuse to add to our collection of excuses – each excuse seemingly more ridiculous than the previous one.
My most recent request was as follows:
Dear Mr Palmer,
Pursuant to the Environmental Information Regulations, I hereby request a copy of any digital version of the CRUTEM station data set that has been sent from CRU to Peter Webster and/or any other person at Georgia Tech between January 1, 2007 and Jun 25, 2009.
Thank you for your attention,
Stephen McIntyre
The full response was as follows. (I’ve included full address particulars for readers that may wish to follow up):
Dear Mr McIntyre
ENVIRONMENTAL INFORMATION REGULATIONS 2004 – INFORMATION REQUEST (FOI_09-44; EIR_09-03)
Your request for information received on 26 June 2009 has now been considered and it is, unfortunately, not possible to meet all of your request.
In accordance with Regulation 14 of the Environmental Information Regulations 2004 this letter acts as a partial Refusal Notice, and I am not obliged to supply this information and the reasons for exemption are as stated below:
Exception Reason Reg. 12(5)(f) – Adverse effect on the person providing information Information is covered by a confidentiality agreement Regulation 12(5)(f) applies because the information requested was received by the University on terms that prevent further transmission to non-academics
Regulation 12(1)(b) mandates that we consider the public interest in any decision to release or refuse information under Regulation 12(4). In this case, we feel that there is a strong public interest in upholding contract terms governing the use of received information. To not do so would be to potentially risk the loss of access to such data in future.
I apologise that not all of your request will be met but if you have any further information needs in the future then please contact me.
If you have any queries or concerns, or, if you are dissatisfied with the handling of your request please contact me at:
University of East Anglia
Norwich
NR4 7TJ
Telephone: 0160 393 523
E-mail: foi AT uea.ac.uk
You also have the right of appeal against the decision. If you wish to appeal please set out in writing your grounds of appeal and send to me at the same address as noted above.
Subsequent to our determination of your appeal, you also have a further right of appeal to the Information Commissioner at:
Information Commissioner’s Office
Wycliffe House
Water Lane
Wilmslow
Cheshire
SK9 5AF
Telephone: 01625 545 700
www.ico.gov.uk
Yours sincerely
David Palmer
Information Policy and Compliance Manager
University of East Anglia
This is the first time that we’ve heard that their supposed confidentiality agreements merely restrict “further transmission to non-academics”. A couple of observations on this. I’m sure that CRU will soon receive a similar request from someone to whom this excuse does not apply.
However, aside from that, there are other troubling aspects to this refusal. If there actually are confidentiality agreements, I would expect the relevant language to be framed in terms of “academic use” as opposed to guild membership i.e. I’d be surprised if the language were framed in terms of institutional affiliation as opposed to use. I’ve published relevant articles in peer reviewed literature, acted as an IPCC reviewer, been cited in IPCC AR4, been invited to present to a NAS panel – my use of data is “academic” by any legal standard.
Secondly, over at the Met Office, they say “it cannot be determined which countries or stations data were given in confidence as records were not kept.” But over at CRU, they purport to “know” nuanced details of the contractual language of the confidentiality agreements – clauses that have the effect of justifying the refusal of the data.
In May of 1999, they put the entire data set up on the CRU website. It was a zipped file called cruwlda2 and was about 3.5 mb. I downloaded the file. My analysis using the file didn’t really support CRU’s analysis. Among other things, the error bars on the trend looks to be much larger than 0.2 C to me. The data file also contains lots of typos.
In June, they noticed I was looking at the data and getting somewhat different answers. They took it down then and refused requests to other people. I know, because other people asked me for the file.
The point should be made that the data was not confidential in 1999.
bon fife place of learning.
hmmm!
bona fide is probably what I meant!
Instead of using FOI requests for past data, wouldn’t it be more useful to ask for future data?
If we know what the data are, for this time next year, then that gives us more time to calculate what adjustments need to be made to current data to meet the planned trend.
I should have added that the Met Office seems to have no problems with issuing detailed information about long range future climate data – even down to post code zones.
It’s thise pesky historical data they have issues with!
Steve’s standards in statistical Math is a lot higher than those at CRU or the Met Office. He and fellow Scientists have continuously seen corrupt information being used to forcast future Climate events. I know Steve hasn’t come out and said this because His interest is strictly in the Math side and try’s His best to keep out of the Politics end…He does a very good job at His site, Climate Audit in controlling the responses there! Thank God for all who contribute!
Just The Facts (12:32:34) :
Those who have manipulated and misrepresented Earth’s temperature data should be held responsible for wasting our limited resources on this phony fight against global warming
I would suggest you the following (I am an foreigner so I can not propose anything): Issue a bill creating a tax of 55% or more to the sales of “carbon shares”. This would stop inmediately all this foolishness.
Alec J (10:24:32) wrote:
“I suggest that you contact Peter Lilley MP who is one of the few of our tribunes who has been willing to stand up against the AGW hysteria which has take over Westminster (sadly so far unsuccessfully).”
I have tried Peter Lilley a couple of times in the past and have got no reply. I understand he does believe in AGW (or accepts that the AGW position may be right) but is sceptical about the calculations made by the UK Government of the cost/benefits of their environmental law compelling the UK to reduce man made CO2 by 80% by 2050. David Cameron is his leader and David C. has suggested that Peter Lilley may be brought into a future Conservative government so he is probably not going to rock the boat at this point since there is a very good chance he may be a minister a year from now.
Please pursue this under the UK FOI Law. You will win.
I have sent the following email:
Subject: FOI_09-44; EIR_09-03
I find your reasons for withholding information ridiculous, at best.
The Information Commissioner will undoubtedly rule in favour of the applicant.
Douglas Hoyt,
I just found cruwlda2.zip by using the following internet file search engine: Try it!
http://www.gegereka.com/
I have downloaded it for safekeeping…
bill (12:41:47) said:
“bon fife place of learning.
hmmm!
bona fide is probably what I meant!”
Actually, I think you meant Barney Fife.
Only has one file, and is forced to keep it in his pocket, otherwise he’d probably shoot himself in the foot.
Freedom of Information Act. Congress didn’t let NASA get away with sitting on data collected by taxpayer money, and if it finds out about the Met Office using the US to launder it’s public funded data I am willing to bet the fur will fly.
This would make a great story for the NY Times, and some hot ammo for the Republicans in Congress.
David (09:48:12) :
The adverse affects of letting the data out is that when it’s monkeyed with to support agendas it easy to expose. Keep it hidden and nobody knows how badly it’s been tampered with, how much of it was altered and what the real data looks like.
If it isn’t done above board, it’s wide open to suspicion.
“I am sure it is not covered by secret stuff it is simply commercial data ”
Now that makes the most sense of any explanation I have heard to date. If an entity that normally charges for this data gave it to the Met office with the caveat that it must not be distributed, then that might explain things but if that is the case, why don’t they just say so?
I do remember reading that some places (Australia?) charge for weather data. Might be interesting to make a request with the addition of wanting to know how much it might cost for reimbursement for any data that is normally available only from commercial sources.
bill (12:27:34) :
See my Comments below:
“I am sure it is not covered by secret stuff it is simply commercial data (i.e. normally paid for). Just see if you can obtain the data at no cost from all the sites.”
I find it odd that you can be more sure that Jones who controls the data.
If the reason were merely “The data must be purchased” then Jones could say that. He hasn’t. His only written comment on the data is that he refuses
to give the data to people because they will just find something wrong with it. Further, if it were merely commercial limitations, then What prevents him from identitfying those entities? According the FOI requests, he doesnt have any records indicating who demands “confidentiality”
“If these commercial interests are willing to give CRU the data under the condition it is not passed on then surely that is OK?”
Again you assume they are commercial interests. There are no such facts in evidence. If the data is WMO data then those organizations are under the obligation, according to their guidelines, to produce the data free of charge.
Second, Jones is under an obligation as a scientist to “source” his data
in such a way that it can be passed onto others. Hansen, for example, does a global average without using any confidential data.
Continuing:
“If CRU have agreed to this then that is ok?”
Not necessarily. That is why in my FOI request I asked for CRU policies with regard to entering into confidentiality agreements. The organization will have policies and the employees will have obligations. Jones, for example, may have violated CRU policies about ACCEPTING confidential data without a proper legal review. He may have violated document maintainance policies by “losing” the records of these agreements. So, it’s not necessarily OK. That’s why in my FOI I requested the relevant CRU policies and procedures.
Continuing:
“If these agreements were by word of mouth 20 years ago then there would be no records. isn’t that OK?”
Again, not necessarily. That is why in my FOI request I asked for CRU policies and procedures WRT verbal agreements. For example, Jones may
be under obligations to create no “verbal agreements” or under obligations to create written documentation for verbal agreements. CRU now claims
that the agreements cover transfer to “non academics” This is highly implausible as confidentilaity agreements cover USE. these are called
USE RESTRICTIONS. They do not cover use by particularly individuals:
ei. “non acedemics” for the simple reason that I can instantly enroll in an on-line school and become an academic. The Reason why one DOESNT make verbal agreements to confidentiality is to avoid this kind of issue.
Further, Since Jones is interested in science it would be an easy matter for him to contact the parties and request them to update the verbal agreement with a written understanding.
If they were written agreements then digging them out from past encumbents lair may be more trouble than it is worth?
Continuing:
“Haven’t CRU given the data to Peter Webster at Georgia Tech because they are a bon fife place of learning.”
1. Sorry but Ross McKintrick has now requested the data as have other professors at [sic] bon fife place of learning.
2. CRU claims the agreements preclude sending the data non “non academics” It says nothing about how this term is defined. Phd? currently
employed by a university? by a non profit? emeritus professor? graduate student? undergraduate? Does publishing in an acedemic journal make one an “acedemic”. Now you see why agreements cover USE and not the user.
Continuing:
“Doesn’t that prove that the data is not secret?”
No it doesn’t. Secret is a relative term. Top Secret data is not secret to those who have a clearence. So, you point is rather trivially true. The point is the data is relied on for public policy and good public policy requires open and transparent records.
finaly
“As I said – get it if you can from Georgia Tech if this is public domain data.
The data does not seem to be hidden to me!”
That legal process is most likely underway as I write. Now, re read your last sentence and see if you get the self defeating irony. the data is hidden to YOU. and me too for that part. I have no trouble supporting pblic policy when I have access to the data it was based on. When I don’t have access I raise my right to have access to it.
Doesn’t this explain the problem?
In accordance with Regulation 14 of the Environmental Information Regulations 2004 this letter acts as a partial Refusal Notice, and I am not obliged to supply this information and the reasons for exemption are as stated below:
Exception Reason
Reg. 12(5)(f) – Adverse effect on the person providing information Information is covered by a confidentiality agreement
Regulation 12(5)(f) applies because the information requested was received by the University on terms that prevent further transmission to non-academics
Regulation 12(1)(b) mandates that we consider the public interest in any decision to release or refuse information under Regulation 12(4). In this case, we feel that there is a strong public interest in upholding contract terms governing the use of received information. To not do so would be to potentially risk the loss of access to such data in future.
ie:
the information requested was received by the University on terms that prevent further transmission to non-academics
it is being witheld because of conditions placed on the University.
Canada has similar legislation to the FOI in the UK. Having worked for the Canadian government for 12 years, I know how these rules work (I’ve been on the other end of these requests too often). The rules are quite rigid and bureaucrats tend to get very defensive of their internal documents and data.
Not knowing the specifics of the confidentiality agreements regarding the CRUTEM data, I will not say that they’ve “acted stupidly”. In fact, they may well be within their legal rights to withhold this info.
That being said, the penalties on bureaucrats for unjustifiably withholding information from the public are very severe. Moreover, from a political standpoint, the optics are very bad if they refuse to be transparent. Consequently, if you push hard enough and make it into a political issue, they are likely to cough up the data at some point.
For those who did not get the message yet….
WE ARE AT WAR!
Hi, Douglas.
I’ve got a copy of that file – the corresponding station data is what I used in the Then and Now comparison in my most recent post. Warwick Hughes also downloaded the data at the time and sent it to me. I agree 100% with your point that the data was not “confidential” at the time. I re-visited the CRU FTP site today to see if it was still there by any chance. It wasn’t but there was something interesting. Stay tuned.
re-visited the CRU site looking for old data to see if it was still there
Re: good story for Republicans in the US. Hardly. Many of them have voiced an opinion about climate change (which was coined by conservatives wanting to appear concerned about climate), that would mitigate against them using this story. The term “flip flopper” comes to mind. I also think that conservatives (as well as quite a few dems) are willing to believe just about anything, if you have the right credentials. Notice the religious beliefs touted ad nauseum. The dems are willing to believe the snake oil salesman touting low carbon fixes and the repubs are willing to believe that God will fix you right up. What IS a sane person to do?????
Hmmm… I wonder how long it will be before AGW papers will be published in Latin so that only ‘academics’ can read them?
So what? Most of the cranks in the [snip] also have articles in peer reviewed literature. You have a habit of posting findings which are sensational, out of line with the consensus view and without qualification of the limitations or the literature which covers the established view, and without paying respect to the nature of emerging results. You pimp every smallest conceivable doubt you find slanted to fit your blog’s brand image to increase your reader base.
The University has every reason to believe you’ll just be finding the first plausible-sounding flaw – whether or not it has any credit – and throwing it to this pack of hungry wolves. It wouldn’t matter if your challenges get proven wrong or not – the damage to the images of the people involved would suffer.
You can still come up with significant results acting as you are – science has a long history of the black sheep coming up with robust findings which have long-lasting change – just don’t expect helping hands taking this approach.
Sorry, “most of the cranks” should read “most of the star cranks”
We need about 10-15 more people to write FOI.
Please donate 5 minutes of your time.
http://www.climateaudit.org/?p=6623#comment-350474
A simple academic explanation: Dean Wormer has you on Double Secret Probation.
steven mosher (20:25:52) :
[edited for privacy reasons~ctm]
arguably NOT i.e. you do not know.
I would suggest Jones will not suffer, but the CRU will by loosing data sources
What would be your honest reaction to this scenario:
You sold data that was bringing in money to your home/company.
You have an agreement that allows your data to be merged with other data to create a product of global importance, but would not allow your data to be accessed individually.
Then your data gets given to others (who could be your customers) without permission.
Would you would be a mite upset?
Would you in future be willing to allow your source of income to be given away?
I think you might decide not to allow your data to be used any more.