Swedes call out Jones on data availability

PRESS RELEASE

Stockholm March 5, 2010

Climate scientist delivers false statement in parliament enquiry

It has come to our attention, that last Monday (March 1), Dr. Phil Jones, head of the Climatic Research Unit at the University of East Anglia (CRU), in a hearing with the House of Commons Science and Technology Committee made a statement in regards to the alleged non-availability for disclosure of Swedish climate data.

Dr. Jones asserted that the weather services of several countries, including Sweden, Canada and Poland, had refused to allow their data to be released, to explain his reluctance to comply with Freedom of Information requests.

This statement is false and misleading in regards to the Swedish data.

All Swedish climate data are available in the public domain. As is demonstrated in the attached correspondence between SMHI (Swedish Meteorological and Hydrological Institute), the UK Met Office and Dr. Jones (the last correspondence dated yesterday March 4), this has been clearly explained to Dr. Jones. What is also clear is that SMHI is reluctant to be connected to data that has undergone “processing” by the East Anglia research unit.

STOCKHOLM INITIATIVE

Göran Ahlgren, secretary general

Kungsgatan 82

12 27 Stockholm, Sweden

===================================

They included attached PDF files. which I have uploaded to WUWT below. Click for PDF files:

Request_from_Professor_Phil_Jones_regarding_the_release_of_data_from_the_HadCRUT_dataset__dnr_SMHI_

Data_from_the_HadCRUT_dataset_100304

DOC111209


Sponsored IT training links:

100% pass guarantee on first try with help of best quality 1Y0-A17 study material including 642-456 dumps and 1Y0-A08 practice exam.


The climate data they don't want you to find — free, to your inbox.
Join readers who get 5–8 new articles daily — no algorithms, no shadow bans.
0 0 votes
Article Rating
315 Comments
Daniel
March 6, 2010 7:27 am

Turboblocke (03:51:19) :
“Has anyone considered that the current position of the Swedes is not necessarily the same as at the time of the FOI requests? Seeing as how they are only now putting their data in the public domain, I think that these letters serve to show that they didn’t want their data made available at the time of those requests.”
Not really true because here in Sweden you can ask for release of data (what ever) and no one can refuse to release them. It is big crime to do that

Mal
March 6, 2010 7:27 am

It seems this Jones guy is just unable to tell the truth. Twisting the “progressive” phrase to suit, he “Lies to Power”.
As the rest of his crew run down the rat-lines, Phil remains on deck. Saluting with one hand, crossing his fingers with the other; lying with all his heart.

Sou
March 6, 2010 7:36 am

The Stockholm Initiative is a private organisation that is anti-global warming mitigation. This is what it says about the person who wrote the ‘press release’:
Göran Ahlgren, tekn. dr. docent organic chemistry. Active at the Royal Institute of Tecnology until 1990. After that active as an entrepreneur and consultant. Engaged in climate politics since several years.
The refer to themselves as ‘climate realists’ – and their site suggests they are not in favour of any action that would mitigate global warming. It states in part:
“Do you think that the ongoing and planned climate policy measures are meaningless and constitutes a gigantic waste of resources that will increase poverty and jeopardize our environment?”
Why is WUWT even posting this article?

March 6, 2010 7:50 am

Sou (07:36:57),
Why the ad hominem attacks? The relevant statement seems clear:
“All Swedish climate data are available in the public domain.”

Jeremy Thomas
March 6, 2010 7:56 am

doc111209 has Jones asking SMHI if he can release the data for 30 Swedish stations that has been recovered from multiple sources over the last 30 years, and has been homogenized and quality controlled, by implication by CRU. He says it’s highly likely that the data he wants to publish will differ from the data held by SMHI.
SMHI replies that given that since the data Jones proposes to release as coming from SMHI is different from that held by SMHI, it withholds its consent. It offers Jones access to its own data.
SMHI is being reasonable – it does not want its data misrepresented.
If Jones was acting in good faith he would have replied to SMHI saying that in the light of its concerns he would release his modified data under the heading “CRU Multiple Sourced, Quality Controlled, Homogenized Version of Swedish Data”.
But he did not, instead stating that SMHI had refused release of its data.
That was misleading.

Sou
March 6, 2010 9:34 am

Smokey (07:50:46)
What ad hominem attacks? All I did was quote from the website of the Stockholm Initiative.
That private lobby group (Stockholm Initiative) has put out a press release (as above) that made false allegations as shown in the pdf files (also above).
At the hearings UEA stated that although some services had not given permission for the Met or CRU to publish the data, most countries made the point that the data was available by approaching those countries directly. That is entirely consistent with the documentation provided above.

Tim Clark
March 6, 2010 9:36 am

janama (19:24:14) :
I’ve eventually found, after an intense search, where the Met Office has released it’s data. Somehow I get the feeling they didn’t really want anyone to find it.
http://www.metoffice.gov.uk/climatechange/science/monitoring/subsets.html

Below is a couple of the “frequently asked” questions from that webpage:
2. What about the underlying data?
Underlying data are held by the National Meterological Services and other data providers and such data have in many cases been released for research purposes under specific licences that govern their usage and distribution.
It is important to distinguish between the data released by the NMSs and the truly raw data. e.g. the temperature readings noted by the observer. The data may have been adjusted to take account of non climatic influences, for example changes in observations methods, and in some cases this adjustment may not have been recorded so it may not be possible to recreate the original data as recorded by the observer.
Back to top
3. Why is there no comprehensive copy of the underlying data?
The data set of temperatures, which are provided as a gridded product back to 1850 was largely compiled in the 1980s when it was technically difficult and expensive to keep multiple copies of the database.
For IT infrastructure of the time this was an exceedingly large database and multiple copies could not be kept at a reasonable cost. There is no question that anything untoward or unacceptable in terms of best practices at the time occurred.
Back to top

So, Jones is stating that there are 3 datsets in some cases;
1. the original raw data,
2. the data adjusted by the National Meterological Services of a particular country,
3.CRU readjusted previously adjusted spam.
He then blames lack of IT storage for not storing multiple copies???….WUWT.
As to which set of data he, or anyone else is specifically referring to, ask the dog.
But rest assured, There is no question that anything untoward or unacceptable in terms of best practices at the time occurred.
I love a well-fed canine.

Tim Clark
March 6, 2010 9:51 am

Forgot to add;
and in some cases this adjustment may not have been recorded so it may not be possible to recreate the original data as recorded by the observer.
I’m depressed.

Paul Vaughan
March 6, 2010 10:46 am

Re: Leif Svalgaard (06:06:09)
Note the hypothetical “would”.
Explanation or not, people see the successful forecasts.

Ingemar
March 6, 2010 11:36 am

I agree with Jeremy Thomas. What Jones and Acton do is definitely misleading. The reluctance of SMHI to allow CRU to publish the “value-added” data as SMHI’s is just another obstacle for Jones to “hide behind”. And it is not fair to the Parlament Committee to give the impression that some countries do not want their data to be published. The data from SMHI could, as far as I understand, have been published years ago.
Much more serious is the reluctance of the CRU-team to be open about how they have reconstructed the Nordic temperature curve. See the following report of the climategate letters by Willis Eschenbach:
http://wattsupwiththat.com/2009/11/29/when-results-go-bad/, see also
http://hidethedecline.eu/pages/posts/scandinavian-temperatures-ipccacutes–scandinavia-gate–127.php?id=127
You may read about the Stockholm Initiative here:
http://www.stockholminitiative.com/eng/about-us
Ingemar Nordin /member of SI

March 6, 2010 11:41 am

Paul Vaughan (10:46:51) :
Explanation or not, people see the successful forecasts.
And ignore the failed ones. This is a very normal human trait.

Brendan H
March 6, 2010 12:13 pm

Smokey: “All Swedish climate data are available in the public domain.”
The point at issue is whether Jones was authorised to release the Swedish data. The Swedish Meteorological and Hydrological Institute says no, as per the communication dated 21 December 2009:
“Given the information that the version of the data from the SMHI stations that you hold are likely to differ from the data we hold, SMHI do not want the data to be released on your web site. “
That is, SMHI does not want its original data to be released on the CRU website. The second communication is a clarification which does not change the thrust of the first. Therefore, the information given to the UK Parliamentary enquiry was correct.
One compelling reason for the SMHI policy of making a clear distinction between the sources of data is the sort of confusion we’ve seen in this thread.
Interestingly, given the focus on processing data, the SMHI communications have themselves undergone a kind of double processing: first by the Stockholm Initiative in its press release, and then by the person who wrote the headline to the story that heads this thread.

March 6, 2010 1:50 pm

The data from SMHI could, as far as I understand, have been published years ago.

Anyone can go to the SMHI website and download data but you have to agree not to disseminate it. I believe the website that includes the confidentiality agreement is here:
http://data.smhi.se/met/climate/time_series/html/essential20.html
Apparently, the SMHI wishes to retain the raw data so it can sell it commercially and if people can publish it anywhere they choose or send it to whomever they want, they would lose this ability.

3.2 The Licensee owns no right to use the data or products provided under this agreement for commercial purposes and not for development or production of meteorological, hydrological and oceanographic value added-value services. The licensee does not own nor authorized to redistribute, sell, assign or otherwise transfer data products or documentation without further processing to third parties unless the parties have received written permission from SMHI.

Robert Burns
March 6, 2010 2:22 pm

re Brendan H (12:13:48) :
You said “The point at issue is whether Jones was authorised to release the Swedish data. ” That is not the point at issue. The point at issue is that he does not have the data he started with, the data that was ‘raw’ to him, and without that data his work cannot be reproduced, cannot be confirmed or rejected.
All else is smoke and mirrors to confuse and obstruct. His letter to the SMHI and their answers is about data AFTER the CUR manipulated the data, not the raw (to Jones) data needed to reproduce his work… again smoke and mirrors. Jones talks about ‘value added data’ which changes the subject from raw (to him) data. It is also not known that one could reproduce his work even if one had his value added data, because AFAIK he has not released the code.

geo
March 6, 2010 8:14 pm

Burns (06:55:37) :
I mean the reproducibility point, and yes I pointed out pretty much the same thing you did right there in one of my posts a few back that Jones may not be able to provide enough detail on what he did to the old data before he stored what is really an “intermediate result” as data instead.
Having said that, obviously this is not a dead data set. They are still adding new data to it every month. I wonder what the processes done to new data are like, and how data and adjustments are stored. Depending on the answer, that might possibly provide a path towards making at least a stab at reverse engineering the old data set.
But really it more and more feels like they need to toss it all out and start over again from the real raw data, stored that way, and a separate store of business rules (versioned and archived as they change) applied to it each time results are needed. Then see how close those new results come to the old results. That might be about the best that can be done now.

Brendan H
March 6, 2010 8:18 pm

Robert Burns: “The point at issue is that he does not have the data he started with…”
Look again. Here are the claims (numbers added) made by the Stockholm Initiative in the above press release.
“(1) Dr. Jones asserted that the weather services of several countries, including Sweden, Canada and Poland, had refused to allow their data to be released, to explain his reluctance to comply with Freedom of Information requests.
(2) This statement is false and misleading in regards to the Swedish data.
(3) All Swedish climate data are available in the public domain.”
Claim (1) is misleading. Jones said the Swedes et al had refused to allow him to release the data, not that they had refused to release the data themselves. Claim (3) is correct; the data is publicly available.
Claim (2) is itself both false and misleading. As shown by the communications from the Swedish Meteorological and Hydrological Institute to Jones, it did not want the original SMHI data to be released from the CRU website.

Paul Vaughan
March 6, 2010 8:57 pm

Re: Leif Svalgaard (11:41:44)
85% success with forecasts. 15% failure. Both visible (to anyone paying attention).

Robert Burns
March 6, 2010 9:29 pm

Brendan H (20:18:27) :
What data are we talking about? The data Jones asked for permission to release is not the original data. As per Jones letter “…the data we hold has arisen from multiple sources, and has been recovered over the last 30 years. Subsequent quality control and homogenisation of the data have been carried out.” This is not the data that is needed to reproduce Jones work.
This was explained by many posters here. An example is by
Jeremy (15:42:53) :
So, the cliff-notes rundown now…
0) CRU makes gridded temperature anomaly charts of the globe.
1) Original data is “lost”, Value-added data retained.
2) Bloggers request original data and methods from Jones.
3) CRU Stonewalls and drags their feet.
4) When finally called out, CRU says, “we can’t due to agreements with other people.”
5) Climategate e-mails are leaked.
6) It is at this point that CRU finally asks for permission to release from other orgs who supposedly wont allow it.
7) Other orgs say, “If you only have your manipulated data, then don’t release a thing.”
8) CRU claims before MPs they cannot release their data at all.
9) Scientists fail to call out their peers for malfeasance.
Please re-read the post by Willis Eschenbach (21:49:29) of 5/3/10. The data in question is the data Willis requested. All the discussion about the data referred to in Jones’s letter is a red herring and a smoke screen because it is not the data Willis asked for.

Sask Resident
March 6, 2010 9:36 pm

From Environment Canada, don’t just ask for a data dump for the climate monitoring stations but also the calculated gridded climate data, which is the data sets used by the climate models. The data from http://www.climate.weatheroffice.gc.ca is verified recorded data and has gaps.

Sask Resident
March 6, 2010 9:46 pm

Let’s face it, the world’s climate data base would take up giga if not tera bytes of space and would need a good data base to handle it. Plus the models have been modified to use gridded data, each parameter is estimated for each world grid, about 4° by 6°.
Too bad we cannot trust the two or three groups that have taken the effort to produce the data or the 7 or 8 organizations that wrote their models to use only gridded data.
My level of trust about climate analysis drops more every time Gore, Hanson, Jones or Weaver open their mouths.

March 6, 2010 10:01 pm

Paul Vaughan (20:57:50) :
85% success with forecasts. 15% failure. Both visible (to anyone paying attention).
Says he in sales promotional material. Not verified. There is a standard meteorological skill score calculation. Corbyn does not use the standard, as far as I can see.

March 6, 2010 10:03 pm

Paul Vaughan (20:57:50) :
85% success with forecasts. 15% failure. Both visible (to anyone paying attention).
If I say the weather tomorrow will be like today, I also get a 85% success rate [depending a bit on location]. The success rate should be adjusted for persistence. I don’t see him do that. Perhaps, you can point me to a relevant citation.

Brendan H
March 7, 2010 2:40 am

Robert Burns: “What data are we talking about? The data Jones asked for permission to release is not the original data.”
Yes, but it was the data that he had. SMHI said they did not want him to release that data as the original or “raw” Swedish data; rather, that they would make the original data available on their own website.
The accusation being made against Jones is:
“Dr. Jones asserted that the weather services of several countries, including Sweden, Canada and Poland, had refused to allow their data to be released, to explain his reluctance to comply with Freedom of Information requests.”
This is not quite correct. Compare with the Guardian report:
“Jones said some issues raised by the emails, such as an apparent reluctance to comply with Freedom of Information requests, were because the CRU did not have permission to release requested data, which had been supplied by foreign weather services. Several countries, including Sweden, Canada and Poland had refused to allow their information to be supplied, he said.”
The crucial difference in accounts is: “…the CRU did not have permission to release requested data.” The Stockholm Initiative press release neglects to mention this important factor.
The focus on the various versions of the data only muddies the waters. The important factors are:
1. Did Jones have permission to release the original Swedish data?
2. Was his statement to the UK Parliamentary enquiry correct?
Going from the reports available, the answers are No and Yes, hence the Stockholm Initiative accusation is false.

March 7, 2010 6:32 am

Brendan H (12:13:48),
IIRC, many of the original data requests and FOIA complaints were from people like Willis, McKitrick and McIntyre, who had no interest in commercializing the data, but who only wanted to verify CRU’s conclusions.
Instead of complying, Jones and his cohorts stonewalled at every opportunity. Now they’re playing word games to try and explain why they refused to cooperate with requests for data – data that they provided to their pals, but not to other scientists:
See here and here and here to understand the mind-set of Phil Jones and others at CRU, who resorted to devious and unethical [and largely successful] attempts to deny publicly funded information and to game the peer review process.

geo
March 7, 2010 7:03 am

Smokey–
I don’t know how Steve McI would have dealt with that kind of license restriction in his framework of doing business. What Steve typically does is make the data available with the code used to manipulate it, and thus allows others to confirm, tweak, re-argue, whatever.
With that license not allowing redistribution, then Steve’s usual methodology for transparency is prevented. So what does he do at that point? I don’t know. It would certainly be unsatisfying (to Steve, I suspect, and to others I’m fairly sure) if Steve would have been forced into the same kind of “trust me” reporting that Jones has been engaged in for so many years based on license restrictions.

1 6 7 8 9 10 13