Met Office and CRU bow to public pressure: publish data subset and code

Just over a month after Climategate started, we have breaking news from Climate Audit

Steve McIntyre writes:

The UK Met Office has released a large tranche of station data, together with code.

Only last summer, the Met Office had turned down my FOI request for station data, saying that the provision of station data to me would threaten the course of UK international relations. Apparently, these excuses have somehow ceased to apply.

Last summer the Met Office stated:

The Met Office received the data information from Professor Jones at the University of East Anglia on the strict understanding by the data providers that this station data must not be publicly released. If any of this information were released, scientists could be reluctant to share information and participate in scientific projects with the public sector organisations based in the UK in future. It would also damage the trust that scientists have in those scientists who happen to be employed in the public sector and could show the Met Office ignored the confidentiality in which the data information was provided.

However, the effective conduct of international relations depends upon maintaining trust and confidence between states and international organisations. This relationship of trust allows for the free and frank exchange of information on the understanding that it will be treated in confidence. If the United Kingdom does not respect such confidences, its ability to protect and promote United Kingdom interests through international relations may be hampered…

The Met Office are not party to information which would allow us to determine which countries and stations data can or cannot be released as records were not kept, or given to the Met Office, therefore we cannot release data where we have no authority to do so…

Some of the information was provided to Professor Jones on the strict understanding by the data providers that this station data must not be publicly released and it cannot be determined which countries or stations data were given in confidence as records were not kept. The Met Office received the data from Professor Jones on the proviso that it would not be released to any other source and to release it without authority would seriously affect the relationship between the United Kingdom and other Countries and Institutions.

The Met Office announced the release of “station records were produced by the Climatic Research Unit, University of East Anglia, in collaboration with the Met Office Hadley Centre.”

The station data zipfile here is described as a “subset of the full HadCRUT3 record of global temperatures” consisting of:

a network of individual land stations that has been designated by the World Meteorological Organization for use in climate monitoring. The data show monthly average temperature values for over 1,500 land stations…

The stations that we have released are those in the CRUTEM3 database that are also either in the WMO Regional Basic Climatological Network (RBCN) and so freely available without restrictions on re-use; or those for which we have received permission from the national met. service which owns the underlying station data.

I haven’t parsed the data set yet to see what countries are not included in the subset and/or what stations are not included in the subset.

The release was previously reported by Bishop Hill and John Graham-Cumming, who’s already done a preliminary run of the source code made available at the new webpage.

We’ve reported on a previous incident where the Met Office had made untrue statements in order to thwart an FOI request. Is this change of heart an admission of error in at their FOI refusal last summer or has there been a relevant change in their legal situation (as distinct from bad publicity)?

0 0 votes
Article Rating
169 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
December 22, 2009 9:47 pm

Congratulations, Steve!
That’s a great step in the right direction.
Best wishes for the Holidays,
Oliver K. Manuel

docattheautopsy
December 22, 2009 9:53 pm

Nice. I’m wondering if the data had been “homogenized” before release to minimize the CRU shennanigans.

December 22, 2009 9:58 pm

docattheautopsy – I wonder the same thing. There’s been enough devious , underhand stuff happen to make ruling this out somewhat difficult.

Michael R
December 22, 2009 10:00 pm

According to the met website FAQ’s
(blockquote)The data that we are providing is the database used to produce the global temperature series. Some of these data are the original underlying observations and some are observations adjusted to account for non climatic influences, for example changes in observations methods or site location.
The database consists of the “value added” product that has been quality controlled and adjusted to account for identified non-climatic influences. It is the station subset of this value-added product that we have released. Adjustments were only applied to a subset of the stations so in many cases the data provided are the underlying data minus any obviously erroneous values removed by quality control. The Met Office do not hold information as to adjustments that were applied and so cannot advise as to which stations are underlying data only and which contain adjustments.(/blockquote)
and
(blockquote)The data set of temperatures, which are provided as a gridded product back to 1850 was largely compiled in the 1980s when it was technically difficult and expensive to keep multiple copies of the database.
For IT infrastructure of the time this was an exceedingly large database and multiple copies could not be kept at a reasonable cost. There is no question that anything untoward or unacceptable in terms of best practices at the time occurred.(/blockquote)
Meaning there is a high chance this data set is still largely the “value added” data making verification of the temperatures impossible.

boballab
December 22, 2009 10:06 pm

couple of things
1. If you read the FAQ page at the Met office you find out this isn’t the Raw data that CRU used. This is the adjusted data that makes up CRUTEM3 and goes into the HadCRUT Global temp data. John Graham-Cumming in his latest blog post has it plotted against CRUTEM3 and to my eye they look almost identical (very slight differences).
http://bishophill.squarespace.com/blog/2009/12/22/met-office-code.html
2. John Graham-Cumming once he looked at the code released believes this is not the CRU code but code the Met office whipped up just for this release of the subset because of the bug he found in it.
http://www.jgc.org/blog/

Peter of Sydney
December 22, 2009 10:07 pm

Comments by other blog sites say the data is not raw but “value add”. ALso, it’s not all there. Too early to tell yet. In any case, I for one have a lot of suspicion so I hope the analysis involved checking the data to see if it makes sense. One way to do this is to compare it with raw data obtained from other sources.

savethesharks
December 22, 2009 10:10 pm

The measured….methodical….patient approach of M&M strikes again.
Something to be said for the Canadian Scots.
Get ’em boys!
Chris
Norfolk, VA, USA

George Turner
December 22, 2009 10:17 pm

However, the effective conduct of international relations depends upon maintaining trust and confidence between states and international organisations. This relationship of trust allows for the free and frank exchange of information on the understanding that it will be treated in confidence.

My head is still spinning.
I can write everything from engineering equations to Shakespeare, but the doublespeak behind that second sentence has me floored.
It’s almost like “I’ll tell you, but then I’ll have to kill you.”
We’re not talking about the formula for Coca-Cola, Kentucky Fried Chicken, or W-87 Minuteman III warhead designs. We’re talking about once-a-day outdoor thermometer readings.

Rereke Whakaaro
December 22, 2009 10:19 pm

Come on guys – don’t be so skeptical – they have given you a Christmas present.
I was probably made by an “emerging economy”, it will undoubtedly break the first time you use it, it may well poison you if you ingest it, but it was still a nice thought …
Have a happy and safe holiday, and thanks for all the fun.

jorgekafkazar
December 22, 2009 10:21 pm

savethesharks (22:10:25) : “The measured… methodical… patient approach of M&M strikes again. Something to be said for the Canadian Scots.”
Och, aye, eh?

Dave F
December 22, 2009 10:22 pm

Peter of Sydney (22:07:55) :
Well, they’ve lost the raw data. Not like it is a set of car keys, but they managed.

Michael R
December 22, 2009 10:23 pm

Ok i tried standard quote marks, then i tried quote marks suggested by another reader and i still cant make quotes, can someone clarify what tags i need to add to make them?
REPLY:

Like this are block quotes

Use the word blockquote and /blockquote inside of left and right arrows, which I can’t display here, but are the ones above the comma and period on your keyboard – Anthony
<i> gives Italics </i> and you can do the same with <b> for bold </b> or <blockquote> for inset block quotes as noted by Anthony. Use the <strike> tag for strikeout font. -ems

Bulldust
December 22, 2009 10:31 pm

Merry Christmas M&M! I sense a busy holiday season for you two 🙂

Olle
December 22, 2009 10:40 pm

Well??
Im thinking about Proffessor Wibjörn Karlen here in Sweden.Doesnt this give him the opportunity to check his national and nordic original “raw data” with the “massaged” CRU s ? And isnt ALL historical rawdata still availeble in most cuntries? If this is the case,… shit can really hit the fan.Am i wrong??The real audit is now possible?

Jay
December 22, 2009 10:40 pm

I agree with the previous posts. Unless the FULL RAW data gets published this if only to give the appearance of cooperation. I am not a conspiracy theorist but I still am very suspicious of these people. What I know of them from their emails makes me think, How long does it take to look through the data, throw together a nice mixture of numbers that reflect a warming trend and enough outliers to appear genuine, and then some code that does not do much but reflect what you want it to. I would not be suprised to hear that the code is not the original and the data is manipulated to reflect what they want. Now they are going to go out and say how the data was released in full along with the code and we are still not happy. This is a smart bunch. They have everything to lose at this point.
Someone smarter than me please catch them in this new lie! It is a battle of David and goliath for modern times. I live for the day when those with vested interests will be exposed. I watch NBC with their GE commercials about “Renewing America”.
Follow the money. It is all Psychology. If someone did not benefit in some way, they would not do it. Green used to be as uncool as Captain Planet. Only when it became profitable did everyone start to care so
much.
My final point is a quote from my father after I go over all of the peer-reviewed skeptical arguments, the emails, the UHI effect, and all the other things I have learned since actually questioning what I was hearing.
“I see what your saying, but I still believe in global warming.”
How do you argue with that?

Phillip Bratby
December 22, 2009 10:41 pm

I’d like to see the QA procedures they use for control and use of the data and code. Ten to one there is no QA trail for the code released and that used prior to release. We need to see every version of the code since it was first used years ago, the reasons for all the changes and the effects of the changes.

December 22, 2009 10:42 pm

Steve, Anthony:
Congratulations to you two for your tenacity, and to all you others who’ve hung in there. And, good luck to all those who dig in to this material to see what’s left.
Here’s hoping that “raw” means “raw”. And that it’s complete.

Michael
December 22, 2009 10:42 pm

For those who missed the conversation on a previous thread.
Oh how deep the rabbit hole goes.
Agenda 21 For Dummies

George Turner
December 22, 2009 10:47 pm

Anthony,
I lost the slash in front of my closing blockquote above, where I said my head was still spinning. The comment still reads okay, but you do have to flip your laptop upside-down to get a feel for the intended indentation. I’m confident that WUWT readers won’t have a problem with it since they’ve already learned to look at Mann’s sediment samples upside-down.

Bulldust
December 22, 2009 10:49 pm

Ah I see Darwin Airport is there… I sense a follow up article in the Darwin series…

boballab
December 22, 2009 10:53 pm

@Olle (22:40:02) :
Yes and no.
No because what the Met office released is a subset not the full set of data and yes he can check to see what is in the subset against the raw back in his home country. However he still will have no idea why any adjustements were made, just that they were.

Dan Martin
December 22, 2009 10:57 pm

My first reaction to this was “this is great”. After thinking about it though I have to be suspicious of any data that was being held from the public like this. It’s temperature records for goodness sakes, what is so secretive about that? Unless the providers of this data are worried about the quality of their sensor sites and they have fudged the data. Mr. Watts has shown what kind of quality we can expect from US stations, who’s to say that other countries don’t have the same problem.
Hopefully all of you who are much more learned than I am can sort through this mess and show everyone what is really happening.

Ray
December 22, 2009 11:06 pm

I would not trust the data they released since these individuals could not be trusted. How can we make sure the raw data have not been modified to fit their agenda? Maybe the only way would be to get the original sheets of paper from individual stations and compare them with what CRU and MET gave up.

Michael
December 22, 2009 11:23 pm

The more rocks you turn over, the more secretes you find that they have been hiding from you. Don’t look so surprised. They’ve been at it for decades. Who are they? Your owners. They see you as useless eaters and the unwashed masses. You are nothing to them. What I just said doesn’t sound so crazy anymore does it, not after what you have seen and read on WUWT.

MarkR
December 22, 2009 11:34 pm

Off-topic:
It looks like Build-a-Bear has pulled the global warming animated propaganda kiddie videos from their website (though still available on YouTube http://biggovernment.com/author/mflynn/). Here’s the email I just got from the Build-a-Bear CEO.
“Thank you for your email and candid comments regarding our animated holiday webisodes. We are sorry we disappointed you.
Our goal is to entertain and engage the imagination of children with our stuffed animals, our store environment, and online. Our intention with the “Under the North Star ” webisodes was to tell a story through the voices of our animal characters of how kids can make a difference in their own individual ways. We did not intend to politicize the topic of global climate change or offend anyone in any way. The webisodes concluded this week with Santa successfully leaving on his journey to deliver gifts around the world. The webisodes are no longer available on the site.
I started Build-A-Bear Workshop as a place for families and children to come for a …
Sincerely,
Maxine Clark
Maxine Clark
Founder and Chief Executive Bear
Build-A-Bear Workshop®
1954 Innerbelt Business Center Drive
St. Louis, MO 63114-5760
maxine@buildabear.com

Malaga View
December 22, 2009 11:53 pm

As we get closer to the CRU kitchen [where they prepare the raw data ingredients] the smell of cooking is becoming overpowering….
Global Warming Goulash
First take your Darwin Zero raw station data and add in some step changes to spice up this otherwise bland dish:
http://wattsupwiththat.files.wordpress.com/2009/12/eschenbach_before-after1.jpg
Then add some additional spice by boiling down the number of station locations since 1850 with some step changes identified by John Graham-Cumming
http://www.jgc.org/blog/uploaded_images/a-709767.png
Please note that you should coordinate your step changes if you wish to get this mixture to rise properly in the oven.
The next step is to combine these ingredients in a computer so that they can be blended to perfection by removing any unwanted lumps and bumps.
The final steps in seasoning this dish are really for the connoisseur as detailed by Musings from the Chiefio at http://chiefio.wordpress.com/gistemp/
Moving your raw ingredients to lower altitudes (preferably a beach or an airport) will enable you to add some BBQ flavour and heat… if you find that the mix is not spicy enough then please move your raw ingredients to higher latitudes where they can be heated and roasted by the sun….
The finished dish should look something like this:
http://www.appinsys.com/GlobalWarming/GW_Part2_GlobalTempMeasure_files/image007.jpg
Global Warming Goulash is Mann made by good cooks everywhere.

December 22, 2009 11:57 pm

This is not the raw data. This is a bone thrown to the dog to make him quiet.

December 23, 2009 12:07 am

I have compared the CRU data for some of the UK sites to the Met Office Station data and there are differences, I have asked the Met Office to check my assumptions. You can see the difference for Oxford at my website, generally it’s small but suddenly after 1979 there are large differences.
http://www.akk.me.uk/Climate_Change.htm

MartinGAtkins
December 23, 2009 12:08 am

Michael R (22:23:26) :
Ok i tried standard quote marks, then i tried quote marks suggested by another reader and i still cant make quotes, can someone clarify what tags i need to add to make them?
Like this. (I think)
<blockquote>

Turns blockquote on.

</blockquote>
Turns blockquote off.

Chris Schoneveld
December 23, 2009 12:10 am

Jay (22:40:56) :
“I see what your saying, but I still believe in global warming.”
As an atheist I am used to that convincing line of argument.

December 23, 2009 12:11 am

I have already compared the temperatues from Met Office and ECA for my hometown, Lisbon, Portugal, and they show some interesting difference between them. Please check out the difference graph at http://ecotretas.blogspot.com/2009/12/receita-de-tratamento-de-temperaturas.html
Ecotretas

Ed Zuiderwijk
December 23, 2009 12:13 am

Before you do anything with this data, do a Benson analysis on it.

Michael
December 23, 2009 12:15 am

I think Nike and Apple should re-think their actions of pulling out of the Chamber of Commerce because of their beliefs in AGW. I know I’ll be shunning those companies.

tallbloke
December 23, 2009 12:15 am

So, the value added station data is released but the explanations for the values which have been added have not.
Ho Hummm. Oh well, it gives us a bone to chew over the holiday break I guess.

December 23, 2009 12:26 am

I’ve used this new UK Met Office data source to create linear graphs for 24 BoM monitoring sites in Western Australia which compare it against historic records from the BoM, high quality data from the BoM and GISS data. Where available, I refer to it as HadCRUT3 data but I must admit I’m not too sure what to call it. UKMO09?
Albany
Bridgetown
Broome
Busselton
Cape Leeuwin
Cape Naturaliste
Carnarvon
Derby
Donnybrook
Esperance
Eucla
Eyre
Geraldton
Halls Creek
Kalgoorlie
Katanning
Kellerberrin
Marble Bar
Merredin
Perth
Rottnest Island
Southern Cross
Wandering
York

December 23, 2009 12:26 am

I wonder if the Met Office has been put under legal pressure here. From the outset, the original claim that Met Office/CRU couldn’t hand over data because of various legal restrictions sounded very thin. First, did Jones ever do more than assert that that was the case – did he prove in any way his claim that he couldn’t share data? Part of the claim was that the data had commercial value. Help out an ignoramus like me: in what way are, say, Singapore temperature readings 1930-1935, commercially valuable. Second, of the 193 countries in the world, strange that they all took the same line, that you can’t share this stuff with anyone because its ours and its jolly valuable. Strange that there weren’t one or two more generous spirited countries, at least. So maybe this is how we’ve got to here: someone has finally said, OK, show us the agreements where it says you can’t. And do you know, no-one could find any such agreements.

johnh
December 23, 2009 12:31 am

This is classic UK Govt spin, please remember I paid for this and its not worth the paper its written on, please use it for the only use left for recycled paper.

Steven Douglas
December 23, 2009 12:35 am

Some of the information was provided to Professor Jones on the strict understanding by the data providers that this station data must not be publicly released and it cannot be determined which countries or stations data were given in confidence as records were not kept.”
That has GOT to be the most weasled and weak explanation ever. There is a finite number of countries – NO recollection of those with whom agreements/”strict understandings” were made? No records kept? It’s THAT SLOPPY? Could be anyone? Can’t state or name anyone specific with any degree of confidence?
Worse yet — just how difficult would it have been — even over this past week — to make contact, or send out less than two hundred pieces of correspondence even, to get everyone on record as agreeing or objecting to the release of the data they provided…and then publicly name them as such?
The stench of coverup is still very thick in the air. The information they released? Virtually worthless. They can’t even distinguished between raw and adjusted data, and don’t even know what adjustments were made to what sets, or why. Good luck sorting that out. Meanwhile, good luck to the credibility of ANYONE who cites the CRU datasets in their work.

Charles. U. Farley
December 23, 2009 12:40 am

I knew this is what theyd do.
I sent them a request for the RAW, unadjusted, unmolested data weeks ago….”we’ll get back to you” they said, they never did.
What-a-surprise, more “adjusted” stuff.

Perry
December 23, 2009 12:48 am

Jay (22:40:56) :
You wrote about your father, “I see what your saying, but I still believe in global warming.”
How do you argue with that?
You say (with tongue in cheek) “Ah, elderly and respected parent, please step outside the igloo for a short time”. http://www.straightdope.com/columns/read/2160/did-eskimos-put-their-elderly-on-ice-floes-to-die
Check please! I’ll get my bearskin.

Max
December 23, 2009 12:54 am

I see the only data set used in Northern Ireland is Aldergrove and guess what it is an international airport… Cherry picking or what… We have an Observatory here in Armagh that has been diigently collecting data since the 1840’s……
It has an accurate forecast that shows no significant warming, even after its data has been homogenised by it own climate scientists.
Nothing to see here, move along

stephen richards
December 23, 2009 12:57 am

Michael R (22:00:19)
Meaning this is ‘probably’ data manipulated by the CRU ‘Harry’ software. In other words ‘Crap’.
It is not the raw data, which I believe they must have, but the ‘value added’ data.
And what is it with the software they have provided?

Dev
December 23, 2009 1:05 am

The is the equivalent of the Washington DC Friday “Data Dump”.
Anyone familiar with Washington knows that bad news, bad economic numbers, possibly troublesome policy decisions, etc, are always released publicly on Friday afternoon. Nobody reads the weekend papers or watches the news broadcasts. By the time Monday comes around, the administration can claim that it is “old news” and ‘nothing to see here’.
It’s the day before Christmas Eve, and most people are busy with the holidays and family until after the First of the year. Smart timing by the MET, but something tells me CRU isn’t going to get away with saying ‘old news, nothing to see here’ over this data…
My browser will be autorefreshing for all the latest from Anthony, Steve, Roger, William, Lucia, Jeff, Bishop, John, and all the other blogs over the holidays. Should be fun reading!

Malaga View
December 23, 2009 1:06 am

Global Warming Goulash – Notes for Wikipedia and Chefs
The original recipe for Global Warming Goulash is Top Secret and is believed to have originated in East Anglia although others claim it comes from Pennsylvania… perhaps the truth with never be really know…. However, it is now generally acknowledged that this dish is very smelly and is best swallowed with a large pinch of salt… However, dedicated individuals have managed to reverse engineer the main ingredients of this toxic concoction based upon random sampling, analysis and hard work. But please note that this recipe should never be released into the public domain without prior approval and peer review by the Magic Circle of AGW.
Buying your raw ingredients from the same manufacturing is most likely to produce the best results. However, some Chefs do experience problems with the ingredients and techniques used to produce the perfect Global Warming Goulash. If you should find that your mixture fails to rise, or is half-baked, then please do not release your dish for public consumption. Please place your failures in the Hide The Decline bin for safe disposal.

DavePrime
December 23, 2009 1:08 am

Now the domers can say, ” We have given them the [whisper](cut/recut,adjusted/re-adjusted,value added/value reduced,location added/location reduced)[/whisper] data. We are sure *smerk* that it will show everything we said it did….”
It is time to take a VERY close accounting of this crud they have just handed out and nt let up until either they release the true RAW DATA, of admit OUT LOUD and IN PUBLIC that they no longer have it. (Thus invalidating their ENTIRE premise…)

DavePrime
December 23, 2009 1:10 am

That should have read “Doomers”.

DavePrime
December 23, 2009 1:11 am

…and *smirk*.
*sigh*

P Wilson
December 23, 2009 1:16 am

Yet a year previously the they published such information openly. Then it was cross referenced against satellite data, which was more reliable and accurate, and put into doubt due to the enormous and growing divergence with GISS satellite data, (and then quickly withdrawn from the Met’s information pages).
it would be interesting to know why this precept isn’t applied equally across all fields of information they make available – such as willingness to tell us their projected temperatures, erroneous or otherwise, all around the world for the next 100 years, the weather for the next season, or indeed tomorrow’s weather forecast. During the week they wrote this international crisis that would come of telling us the temperatures, the daily forecast changes enormously, for example. At the beginning of that week it was going to be “hot and dry by Thursday” then at mid week it was changed to the weekend, and on Friday evening they apologised and said it would be later – hot and dry this week instead.
So its unsure that the Hadley centre and CRU do anything of serious value, especially in view of the erroneous (mistaken) forecasts for the last 8 consecutive seasons, which are quite entertaining to read in their archives. On their web pages they almost boast about not only uncertainty about their own models, but uncertainty about the science behind them. They make quite an issue of it in fact, and then, on that basis, proceed to make very certain predictions up to 100 years ahead.
on their info page they say they take different computer models that all produce different results, mash them all together (like custard with the steak tartare) to produce what is probably a thoroughly confused set of predictions about even the very near future – eg the next few days..

P Wilson
December 23, 2009 1:20 am

Above addressed to George Turner (22:17:52) :

P Wilson
December 23, 2009 1:31 am

George i take a different view. That statement issued last summer would make a better comedy sketch than Steve Coogan

Richard
December 23, 2009 1:39 am

matsibengtsson (23:57:59) : This is not the raw data. This is a bone thrown to the dog to make him quiet.
Right!
“The database consists of the “value added” product that has been quality controlled and adjusted to account for identified non-climatic influences. It is the station subset of this value-added product that we have released. Adjustments were only applied to a subset of the stations so in many cases the data provided are the underlying data minus any obviously erroneous values removed by quality control. The Met Office do not hold information as to adjustments that were applied and so cannot advise as to which stations are underlying data only and which contain adjustments.
Remember the email?
“The two MMs have been after the CRU station data for years. If they ever hear there is a Freedom of Information Act now in the UK, I think I’ll delete the file rather than send to anyone. Does your similar act in the US force you to respond to enquiries within 20 days? – our does ! The UK works on precedents, so the first request will test it. We also have a data protection act, which I will hide behind. Tom Wigley has sent me a worried email when he heard about it – thought people could ask him for his model code. He has retired officially from UEA so he can hide behind that. IPR should be relevant here, but I can see me getting into an argument with someone at UEA who’ll say we must adhere to it !”
Sort of adds up doesnt it?

P Wilson
December 23, 2009 1:39 am

On another subject, what is also interesting is that they openly admit they don’t understand the two dominant climatic forces and the effect they have on climate, despite there being a large amount of research on them: oceans and clouds/water vapour, which comprise 98% of the climate, (Oceans have over 1000 times more ability to retain shortwave heat than air does longwave, and a much greater heat capacity, and 70% of the earth surface is ocean) and then go onto make pronouncements about what it will be like in years to come.

Boudu
December 23, 2009 1:43 am

On the first day of Christmas
HADCRut sent to me
A portion of raw data
An a partidge in a Yamal tree

Telboy
December 23, 2009 1:46 am

“Timeo Danaos et dona ferentes” (Vergil) – I fear the Greeks even if they bring gifts.
I think we should certainly look this gift horse in the mouth (and elsewhere).

amortiser
December 23, 2009 1:53 am

The statement from the met FAQ:
“For IT infrastructure of the time this was an exceedingly large database and multiple copies could not be kept at a reasonable cost. There is no question that anything untoward or unacceptable in terms of best practices at the time occurred”
What a lot of tosh!! They have just accepted what Phil Jones has fed them. This data is the very basis for all their findings yet they discard it because it was expensive to retain.
They want to change the global economic basis on the strength of their “scientific” findings yet they cannot produce the data on which those findings are based.
If a PHD student proferred up that excuse to get his thesis accepted he would be laughed out of the faculty yet these guys expect billions of people to change the way they live their lives on this flimsy evidence.
They should be told to go back and reconstruct the data at their own expense. Their baseless “evidence” has been provided at taxpayers expense to date and it is not worth two bob.
The Met Office has been complicit blithely accepting the CRU assertions of confidentiality agreements existing to prevent disclosure. When the CRU could not produce such agreements – probably discarded with the original station data, they just complied. The emails regarding FOI indicate that various agencies were prepared to accept the CRU position in the absence of any lawful reason. If the CRU cannot produce such agreements then the data should have been released.
The credibility of the CRU is being further eroded with these latest releases. As the data is further picked apart, the reputations of politicians who have stood by these charlatans will also be destroyed.
The way some politicians have hitched their wagons to this train to expand their power, they will fully deserve all the approbrium that will come their way.

Archonix
December 23, 2009 1:55 am

If anyone wants a layman’s explanation for this, it’s like asking for the ingredients and recipe that are used to make a blended soup and getting the actual soup instead, then asking again and getting the uncooked soup. You still don’t know what’s actually in it, in what ratios, and how it was prepared and it’s impossible to unblend the soup to find out.

UK Sceptic
December 23, 2009 2:12 am

What’s this? A smokescreen laid down by the Met Office to distract skeptics from derailing the real villain of the piece – the carbon trading money train?

December 23, 2009 2:18 am

DavePrime (01:08:12) :
“Now the domers can say,…”
Are you talking about Schmidt and Mann by any chance?…

TerrySkinner
December 23, 2009 2:29 am

“blah, blah, blah…show the Met Office ignored the confidentiality in which the data information was provided.
However, the effective conduct of international relations depends upon maintaining trust and confidence between states and international organisations. This relationship of trust allows for the free and frank exchange of information on the understanding that it will be treated in confidence…blah, blah, blah”
Sir Humphrey would be so proud.
“And how are things going at the Freedom of Information Society”
“I can’t talk about that.”

tty
December 23, 2009 2:40 am

I have taken a quick look at the 8 Swedish stations in the lot. Six of these are incidentally in the GHCN database, two are not. Results are as might be expected.
One has extreme UHI problems (Stockholms observatorium, right in the center of the city)
Four are airports (Östersund-Frösön, Karlstad, Jönköping, Visby)
One is in a small town (Haparanda)
Two (both in northern Sweden) are rural: Karesuando, Kvikkjokk

Steven Douglas
December 23, 2009 2:41 am

Figuring that by elimination I might be able to narrow down which countries’ data the Met Office might be reticent about releasing, I compiled a list of all countries named in the data “subset” files. There was not single country that I could find for which data had been withheld (and the subsets all generally include data leading up to 2009).
Furthermore, if you look at the station map for the subset data that the MET posted, you can see clearly that data has been released from virtually all countries (can anybody see one not covered?):
http://www.metoffice.gov.uk/climatechange/science/monitoring/locations.GIF
If the Met is really hesitant about releasing data on the premise that they are somehow protected by a “strict (albeit undocumented, unremembered) understanding”, it would have to be a rather specific list, which would suggest that they know exactly which country objects, and which do not. And if only a portion of a particular countries’ data are being withheld, why was other data from that same country released? How could the Met distinguish between these data without having some record? And if there’s a record, and they do have names of countries with whom their are “strict understandings”, what would preclude them from naming those countries?

December 23, 2009 2:51 am

Some of the information was provided to Professor Jones on the strict understanding by the data providers that this station data must not be publicly released and it cannot be determined which countries or stations data were given in confidence as records were not kept.
If thats an official explanation its beyond laughable. Are we supposed to believe that documents/information were handed over – to a university mind you, not Freds Footwear Store – and that the contracts/legal agreements realting to them have been lost? Or that records were not kept? Well how is any agreement binding if there’s no contract, setting out the terms of the agreement. If this is actually the Met Office’s position they’re making fools of themselves. Far more likely, Jones just thought this one up off the cuff to protect his data: there never were any such agreements. And what are the implications of that?

stephen richards
December 23, 2009 2:56 am

The Met Office do not hold information as to adjustments that were applied and so cannot advise as to which stations are underlying data only and which contain adjustments
How the hell do you do “Quality control” without knowing what the product should look like. What [snip] are they controlling the quality against when they don’t know what adjustments have been made. These people are clowns.
Shut the lot down. They provide nothing of value to the people of the UK or anywhere else. They are there solely to give Brown somewhere to LEAD everyone.

Andrew
December 23, 2009 3:04 am

So, the CRU is still not practicing the scientific method, it has been almost two decades since they published the data, models and procedures needed for a skeptical analysis of any of their work (part of the scientific method). If they are not using the scientific method then they simply are not doing science. If it is not science then why have these ‘peer reviewed’ papers not yet been purged from the scientific record, and why do we continue to discuss the output of this non-science as if it were anything other than fiction?
Ponds and Fleishman may have been wrong, but they were at least real scientists. They did their experiments, published the results and their data, procedures and description of the apparatus and the methods. They where shown to be in error, fine, but they were still scientific in their work. The CRU on the other hand is junk, so I ask again, why are people not going after the papers and getting them ALL withdrawn?
If the papers were withdrawn then it would serve notice on the community that science is science and all else is fiction. If you publish fiction, it will be scrapped.

JBean
December 23, 2009 3:09 am

“Some of the information was provided to Professor Jones on the strict understanding by the data providers that this station data must not be publicly released and it cannot be determined which countries or stations data were given in confidence as records were not kept.”
This is hilarious. If they don’t know which countries they signed an agreement with, how do they know what data to withhold?
Is it in a file marked ‘SUPER SEKRIT’ with secret station codes that no one can decipher because they threw the secret decoder away? Or perhaps the data is from secret countries, such as Hockeystickastan, ready to be included when needed for super secret calculations…

Alan the Brit
December 23, 2009 3:10 am

I await the review of this information ealry in the New Year. Interesting little video on Bishop Hill of Phil Mcateer in Copenhagan looking for Phil Jones, trust the greens to get agreessive & start to get violent!
In the meantime, from one who has just returned to the south-west from Gloucestershire where I attended a delightful wedding on Monday, where they were experiencing some non-global warming-like weather, where I needed two glasses of mulled wine just to thaw the toes through so that I could walk again, as did many of the guests, I wish everyone who has posted, blogged, in favour or against the issues discussed, a Merry Christmas & a Happy New Year! May 2010 be a wonderful time to Hide the Decline! We down here are also experience some rather non-global warming-like weather, it’s freezing cold, as is most of the country, as is most of Europe (except the Peoples Democratic Republic of the European Union of course, where all is sunny & rosey, & the beautiful people are happy dedicated workers, who worship the Beautiful Father (or Mother) whoever they might happen to be! Whoops, I’ve used up my carbon credits for the week, the Carbon Police will be at my door in less than 20 minutes I’ll have to………….aarghh………………………………………………?
AtB

VG
December 23, 2009 3:14 am

Anthony: It does not matter what met office or cru release anymore.. Its over…..

observa
December 23, 2009 3:32 am

You are all being far too polite and reserved here. The volcanic eruptions are occurring everywhere with the Climate Changers and their mad panic as they head for the exits as the tectonic fault lines are spreading at breathtaking speed. As Mann tries desperately to distance himself from Jones, et al at EAU and blame it on Palin in editorials, the UK Met has flip-flopped from trust us and the nice homely lady with the big computer in the background, to besides lots of ‘signed up’ scientists do, to in any case it will take us at least 3 years to sort this lot out, to here have what we’ve got and don’t blame us cos the EAU are responsible for that bloody land temp record- http://www.metoffice.gov.uk/climatechange/science/monitoring/subsets.html
Sweet Jesus, read between the lines as the headless chooks are trying to backpeddle all over the place with their Questions and Answers about the data and methodology and the massive contradictions with their past stance. Basically it’s throw the mob a bone and hope like hell they don’t get trampled to death in the frenzy. Copenhagen has been their erupting Pompeii and it’s every Climate Changer for himself now.
If this AGW agnostic is struggling to come to grips with the enormity of what’s unfolding, I can only imagine what the prior cheer squads in the Mainstream Media are making of it all right now. How can I put the distinct possibility to them now? Ummm,errr….Never in the annals of modern science have so many been duped so much by so few for so much political embarrassment…? Or nothing to see here with these trusted repositories of data like the UK Met, etc and move along folks…? Other ructions aside, the UK Met’s past and present stance now point unerringly to Climategate.

Steven Douglas
December 23, 2009 3:33 am

“We also have a data protection act, which I will hide behind.” – Phil Jones -(and now the Met Office)
It’s all too transparent. They’re buying themselves some intellectually dishonest “plausible deniability”, wherein they can say, “We released data to the public, what more do they want?” — as if a vast swath of the public is too stupid to smell a rat.
“It’s all been publicly available!” is PRECISELY what NASA, RealClimate and others are claiming now, even though James Hansen, Michael Mann, et al, TO THIS DAY have not released or made public all their materials, codes, programs, data and methodologies.

December 23, 2009 3:41 am

Wasn’t this data released over two weeks ago?

3x2
December 23, 2009 3:46 am

Not sure how this MO release is much different from previous releases.
The data downloadable from this page are a subset of the full HadCRUT3 record of global temperatures
and
To assist users in making these for themselves [gridded fields], we are providing two Perl programs:
Leaving the data subset and permission issues aside, I’m still not sure this is much to do with transparency. The code isn’t the CRU code.
If somebody finds a problem (not suggesting they will) the answer will be “ah that’s because you are not using the CRU code” or “you have obviously misinterpreted Jones et al.. 19xx”. In terms of replication or transparency I don’t see that this release moves anything forward from the FOI requests.
This is the equivalent of GISS releasing a couple of utilities that perform something similar to a step or two in the Gistemp process rather than their current offering.
I can see Gavin over a RC preparing his replies as we speak .. “see you give them the tools and the data and they are still not satisfied … always it’s more more more (and obviously best left with those who know what they are doing)”.
(knowing full well what has been released would never have allowed replication or investigation)

Stephen Prower
December 23, 2009 3:57 am

Anthony
Met Office: Release of code & data
To the credit of the Met Office as scientists, according to the
UK Times and Daily Telegraph on 5 December, they intended to go
further, and undertake over the next three years a full
re-analysis of the temperature data. But seemingly the British
Government intervened, and successfully exerted pressure upon
the Met Office to deny the project.
Here are the relevant parts of the Times, Daily Telegraph,
and BBC News stories!
Note (1) that the Met Office denial as reported in the
BBC News story seems deliberately worded so as to confuse the
three-year project in the audience’s mind with the impending
release by the Met Office of temperature data for a subset
of over 1500 land stations:
Note (2) that I am forced to rely upon a recital of the BBC News
story by the Weston-super-Mare website, WSM Weather because
some unhelpful person has removed the story from the BBC’s
archive, and made the link to the story that the WSM Weather
gave now point irrelevantly to the next story in the archive:
* Per the Times on 5 December:
http://www.timesonline.co.uk/tol/news/article6945445.ece
‘The Met Office plans to re-examine 160 years of temperature
data after admitting that public confidence in the science
on man-made global warming has been shattered by leaked
emails.
The new analysis of the data will take three years, meaning
that the Met Office will not be able to state with absolute
confidence the extent of the warming trend until the end of
2012.

The Government is attempting to stop the Met Office from
carrying out the re-examination, arguing that it would be
seized upon by climate change sceptics.

The Met Office is confident that its analysis will
eventually be shown to be correct. However, it says it
wants to create a new and fully open method of analysing
temperature data.’
* Per the Daily Telegraph on 5 December:
http://www.telegraph.co.uk/earth/environment/climatechange/6732011/Scientists-may-reexamine-temperature-data-to-prove-climate-change.html
‘To try to restore public confidence the Met Office is
talking to other meteorological organisations around the
world about recreating the model using the same raw data
but more modern computers.
The whole process will also use any new information and
be more open to the public.
However, it could take up to three years for the study to
complete, meaning the scientific world would have to wait
until after 2012 to provide updated proof of the extent of
global warming.’
* But per the BBC on 5 December:
http://www.wsmweather.co.uk/?p=3102
‘The Met Office (MO) is to announce it will publish the
raw data it uses to analyse man-made global warming.
It follows a row about the reliability of data from the
Climatic Research Unit (CRU) at the University of East
Anglia which has been dubbed “Climategate”.
An MO spokesman denied it would spend up to three years
re-examining the climate change data, and said it had
already planned to publish the material long before the
“Climategate” controversy broke.

The material, dating back 160 years from more than 1,000
weather stations around the world, is expected to be
released this week.
It comes as an independent review is announced into leaked
e-mails at the CRU in Norwich to see whether there is
evidence of manipulation or suppression of data “at odds
with acceptable scientific practice”.

An MO spokesman denied it would spend up to three years
re-examining the climate change data, and said it had
already planned to publish the material long before the
“Climategate” controversy broke.’
Stephen Prower
37 Buckthorn Avenue
Stevenage
Herts SG1 1TW
Wednesday 23 December 2009

December 23, 2009 4:00 am

Being interested in the North Atlantic temperature anomaly, I’ve chosen remote station in Lerwick (Shetland isles, half way Scotland to Norway, far away from major urban areas or large land mass), and calculated 10 year running mean as shown here:
http://www.vukcevic.talktalk.net/Lerwick-TA.gif

Chris Edwards
December 23, 2009 4:01 am

Given the source (when did the met office last tell the truth?) we can dependon the lies, except hat the FOI reason that talks, probably freudianly, about causing an international incident, to true it has, it has upset the plans of a lot of scam artists, Gore and obama among them. We should keep a list, then make sure they never get a position of responsibility again! These assholes can never be seen as fit to serve again.

Rhys Jaggar
December 23, 2009 4:04 am

‘Phillip Bratby (22:41:33) :
I’d like to see the QA procedures they use for control and use of the data and code. Ten to one there is no QA trail for the code released and that used prior to release. We need to see every version of the code since it was first used years ago, the reasons for all the changes and the effects of the changes.’
Well said. I agree 100% with you.
To me, this reads like: ‘we’ve got to release something, so we’ll release something ‘new’, which makes us look as if we’re cooperating, but we’ll drag it out as long as we can’.
You’ll note the words ‘this reads like’. I’m not saying my intuition is correct, more what my sniff on the situation is……

Engineer
December 23, 2009 4:13 am

Phillip Bratby is completely right in commenting on QA procedures.
If we are to believe any data or processed data full QA procedures need to be provided. Any REPUTABLE organisation will be able to provide this together with audit procedures and results. Software Verification and Validation should be to IEEE – STD-610. Professional code should be numbered and commented for testing . We must demand from the MET or CRU Quality assurance information on data as important as this. I doubt whether the schoolboy code I have seen will pass any professional scrutiny.

Stephen Prower
December 23, 2009 4:16 am

Anthony
Oops!
The extracts from the BBC story should read:
* But per the BBC on 5 December:
http://www.wsmweather.co.uk/?p=3102
‘The Met Office (MO) is to announce it will publish the raw data
it uses to analyse man-made global warming.
It follows a row about the reliability of data from the Climatic
Research Unit (CRU) at the University of East Anglia which has
been dubbed “Climategate”.
The MO has written to 188 countries for permission to publish
the historic data it says proves that the world is warming up
due to man-made emissions.
A spokesman denied reports ministers had tried to block the
publication.

The MO’s database is a main source of analysis for the UN’s
climate change science body, the Intergovernmental Panel on
Climate Change (IPCC), which joins talks next week at the
long-awaited Copenhagen summit.
An MO spokesman denied it would spend up to three years
re-examining the climate change data, and said it had already
planned to publish the material long before the “Climategate”
controversy broke.

Story from BBC NEWS:
http://news.bbc.co.uk/go/pr/fr/-/1/hi/uk/8396696.stm
Published: 2009/12/05 05:31:20 GMT’

December 23, 2009 4:18 am

Is this the data they lost? Now they found it? I would be very careful drawing conclusions from data supplied by known fraudsters.

Sean O'Hare
December 23, 2009 4:27 am

The Met Office state in answer to the question “Why is there no comprehensive copy of the underlying data?”:
The data set of temperatures, which are provided as a gridded product back to 1850 was largely compiled in the 1980s when it was technically difficult and expensive to keep multiple copies of the database
The is patently BS. Although hard disk space was a bit limited you could still get an awful of of bytes on station data on a 2400ft reel of magnetic tape at 56Kbpi.

December 23, 2009 4:36 am

Michael (00:15:54) :
I think Nike and Apple should re-think their actions of pulling out of the Chamber of Commerce because of their beliefs in AGW. I know I’ll be shunning those companies.

Difficult for me, as I’ve been a Mac user (and fan) since 1987.
As for this data dump, I’d say file more FOIA demands for immediate access to whatever RAW data they have, wherever they have it. And the same for the other climate manipulators. Maybe a class-action lawsuit would help.
/Mr Lynn

Anand Rajan KD
December 23, 2009 4:43 am

But seemingly the British Government intervened, and successfully exerted pressure upon the Met Office to deny the project.
One just hopes for a Met whistleblower.

ShrNfr
December 23, 2009 4:44 am

When as a scientist you lose my respect for the quality of your data, I will never again trust your data no matter what. CRU you have lost my respect, not that you had much anyway.

tallbloke
December 23, 2009 4:53 am

Boudu (01:43:45) :
On the first day of Christmas
HADCRut sent to me
A portion of raw data
An a partidge in a Yamal tree

And on the second day of Christmas
Hadley sent to me
Two hashed up PERL scripts
A portion of raw data
And a partridge in a Yamal tree

David Shepherd
December 23, 2009 5:03 am

This is doing my head in. On the one hand the MO say:
An MO spokesman denied it would spend up to three years
re-examining the climate change data, and said it had
already planned to publish the material long before the
“Climategate” controversy broke.’ (from Stephen Prower (03:57:42) )
but on the other they say:
The Met Office received the data information from Professor Jones at the University of East Anglia on the strict understanding by the data providers that this station data must not be publicly released.
So they were planning to release supposedly confidential data long before their sudden discovery that it isn’t confidential after all – if only they could show such foresight with their weather forecasts.

3x2
December 23, 2009 5:24 am

Malaga View (23:53:02) :
Global Warming Goulash

There are several recipes.
GWG (FOI-A)
There is a photograph of the dish on our printed menu which is freely available at any of our restaurants. You can pry the recipe from our cold dead hands.
GWG (FOI-B)
Our GWG uses ingredients freely available at any good supermarket and might include potatoes.
GWG (FOI-C)
Look, here is a list of ingredients one could include and there is a supermarket over there so have at it.
GWG (climategate)
THE NORWICH RESTAURANT IS CLOSED UNTIL FURTHER NOTICE
It appears that somebody may have planted covert surveillance devices in our Norwich kitchen in a criminal attempt to faithfully replicate our delicious GWG along with many other customer favourites.
GWG (MO-A)
Although we have every confidence that the food served at our restaurant in Norwich has never contained Cockroaches or Rat poison, we feel it is our duty to investigate. This investigation may take some time and we hope the public will demonstrate patience.
We can confirm however that our delicious GWG contains, at the very least, the following ingredients .. Potatoes (maris piper), carrots (large firm) … seasoning.
GWG (MO-B)
Our parent company feels that any major investigation of our Norwich restaurant facility at this time may reflect badly on our other Restaurants. It was felt that even the suggestion of Cockroaches may have resulted in irreparable damage to our restaurants across the country, particularly as we move into this festive period.
We can confirm though that there are other ingredients in our delicious GWG such as … vegetable stock, onions (large mild), beef (stewing) …
InstructionsPortion ingredients utilising something like the tool presented here (see diagram).

Peter Salonius
December 23, 2009 5:25 am

ARE CURRENT WARMING AND SEA LEVEL RISE PROBLEMATIC
The paper entitled: ‘Past Temperatures Directly from the Greenland Ice Sheet’ at:
http://www.sciencemag.org/cgi/content/full/282/5387/268/
-offers a fascinating, and pretty ironclad record of temperatures extant when the upper layer of the Greenland ice sheet was laid down — see especially Figure 3 showing:
A. Last 100,000 years
B. Last 10,000 years
C. Last 2,000 years
It is obvious that — contrary to the assertions of the Intergovernmental Panel on Climate Change (IPCC) — there have been several periods since the last glaciation ended and modern agriculture began, when the climate has been considerably warmer than it is now.
————————————————————-
Also in the light of the documented greater than 20 degree C temperature rise during the approximately 25,000 year period since the last DEGLACIATION began (Figure 3 A in the paper above), it is interesting to examine documentation of the sea level rise that accompanied that 25,000 year / greater than 20 degree warming history –– see graph entitled:
‘Post-Glacial Sea Level Rise’ at:
http://gregladen.com/wordpress/wp-content/graphics/Post-Glacial_Sea_Level.png
— and then speculate on the sea level rise (resulting from both melting ice and sea water temperature-related volume expansion) that we might reasonably expect if the average documented natural climate warming (less than 1 degree C per century), that we have experienced since the end of the Little Ice Age, continues for the next 100 years // bearing in mind that there is MUCH LESS ICE AVAILABLE for melting than there was when the vast continental ice sheets were present as the last DEGLACIATIN got underway – and – that the Holocene period, starting about 8,000 years ago, featured temperatures at least 2 degrees warmer than those we are experiencing now.
Peter Salonius

rukidding
December 23, 2009 5:30 am

Seems to only be a limited number of Australian stations and the fact they have Darwin Airport starting in 1882 is not a good start.
Wonder how Australia managed to keep powered flight secret from the rest of the world for 20 odd years.:-)
Australian records start at folder 94 for those interested
Cheers & Merry Christmas

WakeUpMaggy
December 23, 2009 5:31 am

I cannot imagine how we would expect actual temperatures to be reported from corrupt third world countries, now that all they have to do is imply “climate change” of any type to lobby for a big handout from the UN. Did anyone ever go to jail over the Oil for Food program?
AGW was bad enough, using the concept “Climate Change” creates a worldwide angry victim class where every weather event must be blameworthy.
Massive worldwide welfare fraud in the brew. That’s what makes me wonder sometimes if the scientists didn’t see this coming and out themselves. The unforeseen consequences could be just enormous.
So I would expect temperature readings to be pre-adjusted from many countries. How could you trust them if our “top” scientists, under a microscope, are skewing things?

Editor
December 23, 2009 5:32 am

Michael R (22:23:26) :

<blockquote>
Ok i tried standard quote marks, then i tried quote marks suggested by another reader and i still cant make quotes, can someone clarify what tags i need to add to make them?
</blockquote>

See http://wattsupwiththat.com/resources/#comment-65319 for more hints.

Editor
December 23, 2009 5:43 am

Jay (22:40:56) :

My final point is a quote from my father after I go over all of the peer-reviewed skeptical arguments, the emails, the UHI effect, and all the other things I have learned since actually questioning what I was hearing.
“I see what your saying, but I still believe in global warming.”
How do you argue with that?

You can’t. Instead you pray for a snow storm, preferably one comparable one in the 1970s during the last cool phase of the PDO.
Or one that immobilizes Washington DC. Those are always good for the soul as long as you aren’t inconvenienced.

Caius Petroleus
December 23, 2009 5:54 am
Kevin Kilty
December 23, 2009 6:24 am

Ray (23:06:06) :
I would not trust the data they released since these individuals could not be trusted. How can we make sure the raw data have not been modified to fit their agenda? Maybe the only way would be to get the original sheets of paper from individual stations and compare them with what CRU and MET gave up.

Just make a comparison of some random stations. If anything suspicious shows up, then do the whole lot.

December 23, 2009 6:30 am

The code up at the MET Office does not appear to be the code used by CRU:
http://strata-sphere.com/blog/index.php/archives/11957

Basil
Editor
December 23, 2009 6:42 am

climategatestuff (03:41:27) :
Wasn’t this data released over two weeks ago?

Yes.

James Chamberlain
December 23, 2009 6:44 am

It’s a publicity stunt for RC and others to say, “See, we supplied everything. Those deniers are crazy!” Value added data and likely medled code. It’s worth something, but mostly PR.

Henry chance
December 23, 2009 6:45 am

We couldn’t trust them to release data.
We can’t trust what they released.
There was a motive to refuse to cooperate.
When Climate Progress in the last days suddenly flood the market with new studies and graphs, ask why they release graphs and not data.

JonesII
December 23, 2009 6:47 am

BUT…wasn´t it the preferred massaging method “Cherry Picking”?, as shown previously, the warmer stations were graciously selected.

JonesII
December 23, 2009 6:52 am

They have changed, adjusted, massaged, cherry picked data so many times that I would bet that not even them know which data is correct and which it is not.

Robert Kral
December 23, 2009 7:12 am

Believe me, when you execute a binding confidentiality agreement with someone you keep a copy. Their statement that they don’t know who they had an agreement with is hogwash.
Beyond that, since they did not release the underlying raw data and limited the release to a subset (chosen how, exactly?) this looks like they are not acting in good faith. Might as well be negotiating disarmament with North Korea.

December 23, 2009 7:21 am
ew-3
December 23, 2009 7:33 am

“The data set of temperatures, which are provided as a gridded product back to 1850 was largely compiled in the 1980s when it was technically difficult and expensive to keep multiple copies of the database.”
Nonsense, I began my civilian career in the mid-70s on Wall St doing maintenance on the computers in their data centers. We always had back up to mag tape. Not expensive and not a big problem. Their dataset was certainly was a lot smaller then the banks back then.

Brass Monkey
December 23, 2009 7:33 am

email 1112622624 may help resolve some of these issues:
From: Phil Jones
To: “Brohan, Philip”
Subject: Re: HADCRUT various
Date: Mon Apr 4 09:50:24 2005
Cc: Peter Thorne
Philip,
I’m not unhappy at all. If I am it is more about HadCRUT2 and 3.
I read through the report to DEFRA and will be sending some comments later today. I also commented on what Harry has written as a report for you. I’ve left those comments with him as he’s away this week and I’m off April 6-15.
It is a bit odd with HadCRUT2 that the problem has surfaced now and my old mask hasn’t made any difference.
Cheers
Phil
At 15:33 01/04/2005, Brohan, Philip wrote:
Phil.
I’ve just had a chat with Peter Thorne about HadCRUT2 and 3, and I get the impression that you are concerned, so we thought I should clarify what is going on. In particular I want to assure you that we are not trying to change the system without your approval.
To make things quite clear, we have two HadCRUT systems here:
1) Peter is running HadCRUT2. This is our operational system which produces the new data every month that we send to you and everyone. This is a fixed system, it does exactly what you agreed with Peter a couple of years ago. We don’t plan to change it at all.
We did, unfortunately, make a mistake while running the system; we think a land-mask file was changed. This is what Peter’s recent messages have been about. We’re still not quite sure how this happened, but whatever fix we apply will be to restore the system to the original, agreed state.
2) I am coordinating HadCRUT3. This currently encompasses Harry’s work on the data, Simon’s work on blending, John Kennedy’s work on variance correction, and my work on errors and gridding. Some combination of this work will become the new dataset.
I have a clear picture of what I think should form the new dataset. However, we won’t produce HadCRUT3 unless you (and all the other contributors) agree. If I can’t persuade you of the value of a change, it won’t happen. In particular, I see the land station data as entirely under your control, both now and in the future.
If I (or Peter) misread the vibes and you were not worrying about any of this, please don’t start. There are not serious problems with either system.
Have fun,
Philip.
Prof. Phil Jones
Climatic Research Unit Telephone +44 (0) 1603 592090
School of Environmental Sciences Fax +44 (0) 1603 507784
University of East Anglia
Norwich Email p.jones@uea.ac.uk
NR4 7TJ
UK

mojo
December 23, 2009 7:33 am

Sounds like averaged data, not day-to-day records. Useless, most likely.

Andrew Suprun
December 23, 2009 7:37 am

From Met Office Q&A:
“The database consists of the “value added” product that has been quality controlled and adjusted to account for identified non-climatic influences. ”
“The Met Office do not hold information as to adjustments that were applied and so cannot advise as to which stations are underlying data only and which contain adjustments.”
My take on it, this data is useless.

bill
December 23, 2009 7:40 am

David Shepherd (05:03:14) :
The data was not confidential it was commercial as Jones said about a year ago. You cannot redistribute commercial data – try putting commercial DVD data on youtube and see the result.
The data is not raw data – That is available from the originating met offices if no available from CRU.
Jones said the data would be released when all MOs (who considered their data to be commercially valuable) had given approval. He also said that gt 90% of the data was already available as GISS sites.
The MO have now released the gt 90% of data as a file. I.e they have only released data that has no commercial value.
This is a good example of what Jones had as raw data
http://climate.arm.ac.uk/scans/1878/08/187808a1.jpg
other locations would have been more difficult.
To store such data in the 80s would have been difficult – 1000 stations, 150kB per picture per month per am/pm=1000*150*2= 300MB/month
To store the text hand converted would be much less but then you loose the raw data.
Armargh noted in a post above has a discontinuity of 1 deg C in september 1878 which raises all preceeding months by 1 deg C. Overall the hockey stick shape is preserved. when this is taken into account:
http://img51.imageshack.us/img51/3635/armargh1878discontinuit.png
Note that the met office has had on line for some time unadjusted temps of a few uk stations

Andrejs Vanags
December 23, 2009 7:51 am

Fixed it:
Boudu (01:43:45) :
On the first day of Christmas
HADCRut sent to me
A portion of raw value added data
An a partidge in a Yamal tree
And on the second day of Christmas
Hadley sent to me
Two hashed up PERL scripts
A portion of raw value added data
And a partridge in a Yamal tree

David Segesta
December 23, 2009 8:02 am

1) There is still no raw data.
2) In response to a question about FOI requests they said:
“We take our responsibilities under the Freedom of Information Act very seriously and have, in all cases, handled and responded to requests in accordance with its obligations under the legislation.”
That doesn’t seem to square well with the leaked e-mail messages.

P Solar
December 23, 2009 8:13 am

Interesting aside. CRU in frech means uncooked. Seems something was lost in the translation !

Roger Knights
December 23, 2009 8:15 am

“I see what your saying, but I still believe in global warming.”
“How do you argue with that?”

Don’t. The crux of the matter isn’t whether the globe is warming, which it is, but what is the cause. Dr. Akasofu argues that it’s the rebound from the LIA plus the warm phase of the PDO, in: “Two Natural Components of Recent Climate Change,” (as a 50-Mb PDF):
http://people.iarc.uaf.edu/~sakasofu/little_ice_age.php

observa
December 23, 2009 8:22 am

“I see what your saying, but I still believe in global warming.” and how do you argue with that? Well given the importance and relevance of the UK Mets flip-flopping and total buffoonery now, you have to ask yourself which national leader will crack first and say ‘enough is enough’ and we need to stop this juggernaut right now while we all cool off and take a calm, calculated look behind the walls of current climatology. Nothing like a decent commission of enquiry to deflect the usual political approbrium and the temptation must be growing. They must be watching their scientists and experts at the commanding heights covering their butts and heading for the exits and getting increasingly nervous. Once one cracks the rest will follow with alarming speed I’ll warrant. Look how quickly the Berlin Wall fell once a certain momentum had gathered and then a tipping point was reached.

John Sims
December 23, 2009 8:31 am

The Met office says:
“For IT infrastructure of the time this was an exceedingly large database and multiple copies could not be kept at a reasonable cost. There is no question that anything untoward or unacceptable in terms of best practices at the time occurred”.
OK, they didn’t keep multiple copies. But the question is, did they keep *any* copies – or even just the original tapes (or possibly disk packs)? Note that a a “copy” of something is not the “original” something.

December 23, 2009 8:49 am

Wait. I thought ALL the data was already released and publicly available. Hasn’t that been the defense for the last month?

SteveS
December 23, 2009 8:54 am

”a network of individual land stations that has been designated by the World Meteorological Organization for use in climate monitoring”
World Meteorological Organization?? – What’s that? Not another UN body,I hope? Do some of ‘The 42’ work there?

Tenuc
December 23, 2009 8:56 am

Jay (22:40:56) :
“My final point is a quote from my father after I go over all of the peer-reviewed skeptical arguments, the emails, the UHI effect, and all the other things I have learned since actually questioning what I was hearing.
‘I see what your saying, but I still believe in global warming.’
How do you argue with that?”

I’ve had to cope with the same problem convincing friends, acquaintances and even my two adult sons.
One problem is that belief is almost impossible to defeat with facts alone, as it usually based on perception rather than reality. To find out what’s happening with your father try some inquiry using some planned questions and a technique which I call the ‘Depth Gauge’ – remember just listen during the questioning phase and don’t refute anything he says, no matter how daft it may sound to you! Oh, and no ‘closed’ questions.
Build the question deck you as questioner are going to ask the respondent in this sequence:-
What, how, when, why, where –
1. Understanding of facts. (e.g. How is global average temperature calculated?)
2. Context of facts in relation to hypothesis. (e.g. How does this impact CAGW?)
3. Importance of topic to respondent. (e.g. What does this mean to you?)
4. Feelings of respondent to topic (e.g. How do you feel about it?)
At the end of this process, you should have a good idea of the level of your fathers factual knowledge, the context of its importance to him and his feelings about it. Now you can give him the correct information a bit at a time and hopefully change his perception of reality.

Larry Sprague
December 23, 2009 9:00 am

I figured why the MET is able to release the data now when they previously could not do so due to confidentiality requirements. The confidentiality agreement was that they could not share the original data. But this data has been massaged and homogenized, such that it in no way resembles the original data, and thus the MET can release the data and stay in compliance with their confidentiality agreements!

Kitefreak
December 23, 2009 9:03 am

amortiser:
“As the data is further picked apart, the reputations of politicians who have stood by these charlatans will also be destroyed.”
I’d love to believe that… In the UK ALL the major political parties’ leaders are fully, indeed rampantly, behind the AGW cause. I call it a cause, we’d call it a scam, of course.
Personally, UKIP will get my vote. Only credible people at least standing up to all this EU/UN/AGW/NWO BS.
Then again, they’re politicians as well and once you start giving them power then, well, you know what power does, don’t you?
Just have to look to the biggest political organisation in the world to see the global height of corruption. Not the pinnacle of corruption, for that place is occupied by the people who set up and funded the UN from the beginning: the money men.

Hangtime55
December 23, 2009 9:07 am

Again , anyone had to conclude that the data and emails ‘ obtained ‘ from the University of East Anglia’s Hadley Climate Research Unit (CRU) were in fact leaked and not hacked .
On October 12, 2009 , Paul Hudson , a BBC weatherman was ‘ forwarded ‘ the ‘ ClimateGate ‘ files from a ‘ anonymous person ‘ or ‘ mole ‘ a month before the internal breach of security was discovered and the ClimateGate files went public . I seems that the question of whether the ClimateGate files were leaked or hacked should had already been determined .
I stated a month ago that :
” To verify this one would have to compare the ClimateGate file that Paul Hudson from the BBC was forwarded on October 12 , 2009 . again , the word ‘ forwarded ‘ implies to the files being ‘ Copied ‘ , not Taken as Jones had stated to Invesigate Magazine .”
” This means that the ‘ anonymous person ‘ must be a ‘ insider ‘ within the Climate Research Unit , as he had opportunity to ‘ update ‘ the FOIA2009 folder from within until November 12th , 30 days after Hudson at the BBC possibly had an altermatum to either expose ClimateGate to the public OR the ‘ insider ‘ would leak the data him/her/them selfs .”
I wondered if Paul Hudson had been contacted or interviewed to get his side of this story until I found that most BBC forecasters are not directly employed by the BBC, but by the Ministry of Defence (MOD) ‘s Meteorological Office ? Yet since 2007 Paul Hudson is reported to be a full-time member of the BBC staff , and not the Meteorological Office ?
September 11 , 2007. ( or 911 ? ) a new Integrated Climate Programme (ICP) was for the first time combined for the needs of the Department for Environment, Food and Rural Affairs and the Ministry of Defence, to gather information on climate change.
The Met Office Hadley Centre is the UK’s official centre for climate change research. Partly funded by Defra (the Department for Environment, Food and Rural Affairs) and the Ministry of Defence. The MOD has signed a three-year deal with the Met Office Hadley Centre worth £12m while Defra has signed a five-year deal worth £74m.
While it is stated that Paul Hudson is no longer directly associated with the Meteorological Office , you can be sure that personal and professional ties still exist .
With this information , it is more clear to me now that on the day that Phil Jones was informed of the ‘ Breach of Security ‘ (internel?) by the Administrators at RealClimate , that same afternoon it was reported that Jones was interviewed by ‘ police ‘ about the scandal , AND that Two Plain Clothes Officers arrived in an unmarked car and took Jones to Norfolk Police’s headquarters in nearby Wymondham to give a statement ?
Two Plain Clothes Officers in a Unmarked Car ? Doesn’t that seem more like a Ministry of Defence visit rather then a conventional local police matter ? While it is fact that the Ministry of Defence had signed that three-year deal with the Met Office Hadley Centre worth £12m , this could be the reason why the Ministry of Defence was involved , but if not ? ? ?
Like I’ve Always Said . . . FOLLOW THE MONEY .

Grace
December 23, 2009 9:12 am

From Aaron Shenck, Deputy Director, (PA)Senate Education Committee:
“Penn State has informed Senator Piccola that university policy requires an internal inquiry process to occur first before a full investigation can be undertaken. In most cases the internal inquiry process may take no more than 60 days by university policy, and if this process determines a full investigation is warranted, that investigation must be completed no later than 120 days after it is initiated. The internal inquiry process in Dr. Mann’s case started on November 30. The Senate Education Committee will monitor this process and will review the outcomes of it. “

Raymond
December 23, 2009 9:15 am

Actually,publishing the data late is no excuse for not prosecuting them. Delaying the publishing is illegal; I just hope it is punishable with something painful enough.
What is the purpose of criminalising something if there is no punishment?

Hank Hancock
December 23, 2009 9:28 am

If the subsets are based on the TS2.1 dataset they are homogenized. CRU claims that the later TS3.0 is not homogenized but Harry_Read_Me seems to make it clear that TS3.0 is derived from TS2.1. That’s if the subsets are from the TS data products. They could be from some other data product or scraped together source data discovered in the Recycle bin (Trash bin for Mac users).

steven mosher
December 23, 2009 9:38 am

I’ve got FOIAs in process submitted a couple weeks ago to see if CRU complied with their regulations regarding confidential data.
Plus the code just displays the numbers as best as I can see. So they havent released the code or the data. Just some code and some data. When will they learn that we won’t stop.

Malaga View
December 23, 2009 9:51 am

3×2 (05:24:26) :
GWG (MO-A)
Although we have every confidence that the food served at our restaurant in Norwich has never contained Cockroaches or Rat poison, we feel it is our duty to investigate.

The latest information indicates that the Norwich restaurant will be closed for sometime because cockroaches and rats still have free access to the building. Additionally, the latest environmental health inspection of the Norwich premises has identified that the Chefs have actually been using genetically modified ingredients in the kitchen. This severely undermines their claims that the Global Warming Goulash only contains 100% pure organic ingredients.

December 23, 2009 9:58 am

If the subsets are based on the TS2.1 dataset they are homogenized. CRU claims that the later TS3.0 is not homogenized but Harry_Read_Me seems to make it clear that TS3.0 is derived from TS2.1.

I wouldn’t know TS-1 from a nine iron, but if the data release has been homgenized, I think the popular case (probably legal as well) may be building to demand meta-data showing exactly what was changed, when, and how.
Can the stewards of individual stations legally turn over raw data to the public? (either past or future?)

December 23, 2009 10:00 am


John Sims (08:31:13) :
The Met office says:
“For IT infrastructure of the time this was an exceedingly large database and multiple copies could not be kept at a reasonable cost.

A T50 (50 MB) disk could be backed up on one (1) 9-track Pertec tape – a T200 (200 MB) disk took three tapes I think. And a 990 minicomputer usually had one T50 (system disk) and several (2 or 3) T200 ‘user’ disks. Incremental backups took place each night, and full backups took place once a week; there were sufficient ‘tapes’ on a storage rack that were cycled through to accomodate this practice without reusing the prior tape made the day or week before.
A mini-datacenter at a site (we had three sites at least in the Dallas area) had about five (5) 990 mincomputer systems each (not counting any specific ‘project’-owned 990 minis) run by the group known as ‘DCS’ (Distributed Computer Services) within the company (TI). I say mini-datacenter because this does not count the BIG IBM iron that was at a couple different ‘CIC’ (Corporate Information Center) sites (South Bldg and Lewisville at the time) and to which the 990’s had been networked via applications on the 990’s written to IBM’s SNA arcitecture and protocols (RJE, BDT, TSO).
And this was at a defense contractor in the 80’s (before the VAX’s began to sweep through and replace the 990 minis) …
.
.

Ryan Stephenson
December 23, 2009 10:09 am

I see they include Gatwick airport near London but exclude the “gold standard” site of Ross-on-Wye (a site that has only changed once in 80 years). I could understand if the sites had been chosen to be evenly distributed as they claim they are – but the map shows that they have a funny idea of “even”.
A suspicious person might claim that Ross was excluded because it doesn’t show warming, whereas for some strange reason London Gatwick does. Curious.

karl.heuer
December 23, 2009 10:09 am

Chris Gilliam
The “value added” is clear and striking.
The differences between the BoM and “adjusted data” are obvious to see, as is the intent of the adjustments.
Have they no shame?
This is blatant fraud.
Good work.

December 23, 2009 10:50 am

The CRU station data can be plotted at: http://www.appinsys.com/GlobalWarming/climate.aspx
Individual or multiple stations can be plotted in terms of temperatures or anomalies.
When plotting a station, the NOAA unadjusted GHCN data can be plotted for the same station for comparison – the amount of adjustment that CRU made to the station is then observable.

December 23, 2009 10:56 am

OMG! Don’t you people realize that this release of state secrets is going to cause an INTERNATIONAL INCIDENT???
Do you think countries like Berserkistan are ever going to TRUST the HadMetCRU ever again?
This is the end of science as we know it, and the beginning of World Data War I. Looks like it’s back to the bomb shelters for the plucky British as the affronted leaders of Zimbabwe, Venezuela, and other countries operating under the Rule of Circumstantial Law turn their collective backs and flatulize the former Empire.

J.Peden
December 23, 2009 10:57 am

The data show monthly average temperature values for over 1,500 land stations…
I didn’t see that anyone’s mentioned it again, so are they saying they’ve used only about 300 stations for the whole rest of the landed World apart from the U.S.? [And where’s the Ocean, which was supposedly Jones’ great contribution to Humanity, otherwise known as “ipcc Climate Science”?]
Or is the number of stations the CRU used for land up to a whopping 1500 for the landed “ROW” compared to the U.S.’s ~1200, a country which represents “only” ~2% of the land+sea ROW [actually about 6.7% of landed ROW],
So that according to the Team ~”the U.S. doesn’t matter”, since the instrumental U.S. temp. record, such as it is, doesn’t show much warming at best?*
While the U.S. record would also comprise at least about 12/27 = 44% of the total land record. So that the ROW must really be burning up [no doubt due to U.S. “imperialism”].
I know I know, I start sounding pretty crazy to even try to put an apparently little thing into “context” within Climate Science. But hey, not as crazy as “Climate Science”!
*According to yet another spurious debate the evil anti-scientist, Steve McIntyre, caused with the Team, at least as claimed by that Climate Science “tribe”
-“tribe”, so hat tip also to Little Judith Curry Rambling Wreck, from Georgia Tech..

rbateman
December 23, 2009 11:17 am

I have put up a page of the combined data for Sacramento 5 ESE back to 1853
http://www.robertb.darkhorizons.org/SacMonthlyAMS_COOP.htm
which I have plotted with overlap.
Included on page is the station data for Sacramento from the MET release.
I’ll plot it later, but it looks to preserve the Urban Heat Island effect.

David Gay
December 23, 2009 11:34 am

I keep wondering if the missing legal agreements that Jones was talking about were ones that prevented the agency supplying the data from releasing the raw data. Then the CRU or Met Office would become the official source….

tallbloke
December 23, 2009 11:54 am

GWG (MO-B)
Our parent company feels that any major investigation of our Norwich restaurant facility at this time may reflect badly on our other Restaurants. It was felt that even the suggestion of Cockroaches may have resulted in irreparable damage to our restaurants across the country, particularly as we move into this festive period.

“Roaches blame light switch operator in restaurant food heist”

Hank Hancock
December 23, 2009 12:58 pm

Bill Parsons (09:58:25) :

I wouldn’t know TS-1 from a nine iron, but if the data release has been homgenized, I think the popular case (probably legal as well) may be building to demand meta-data showing exactly what was changed, when, and how.

Unfortunately, while Harry was reconstructing the CRU TS2.1 data to produce TS3.0, he complained constantly that there was no information available to him that explained how and why changes were made to the source data. Much of the source data used by his predecessors appears to have been lost (deleted) along with any documentation. So, it becomes evident in his project notes that he just assumed they got it right before dumping the project into his lap. Near the end of the reconstruction of the dataset, he exclaimed how fortunate he felt to have achieved 0.5C to 1C accuracy. So much for two place precision in the CRU time series (TS) data product.

JonesII
December 23, 2009 1:10 pm

Mike D. (10:56:08) :Don’t forget Chavez also subsidizes London buses with low prize oil. All you need now is some bananas.

Basil
Editor
December 23, 2009 1:36 pm

rbateman (11:17:46) :
Included on page is the station data for Sacramento from the MET release.
I’ll plot it later, but it looks to preserve the Urban Heat Island effect.

As it should. Temperature is temperature, regardless of the factors influencing it. I think it is a mistake to try to homogenize the UHI out of urban temperature records, a la GISS. Let’s get as “clean” a data set as we can, and then argue about what has caused any long term temporal, or secular, changes in temperature.

Jason
December 23, 2009 1:46 pm

Look at the map.
“The subset of stations is evenly distributed across the globe and provides a fair representation of changes in mean temperature on a global scale over land.”
Does it look that way? No. Look, not one station at ROC Taipei is used in the subset, but every station in New Zealand–remember the posts about the scandal there?
There is all sorts of imbalances-look at the preference for industrialized Western Europe.
I believe the map alone, and this topic, may deserve its own thread.

Jason
December 23, 2009 1:55 pm

Also, the t rise graph. from about 1920 on the subset and crutem3 look so similar the “subset” might be near the crutem3 total. But earlier years crutem had some colder records. Must be dodgy.
Where is the t rise 1910-40, and decline 40-70?

Jason
December 23, 2009 1:57 pm

And does not crutem3 use Ocean temps? How can crutem3 without ocean temps seem so identical in the graph to the subset, a subset that is only land stations?

Invariant
December 23, 2009 2:20 pm

Tenuc (08:56:58): Now you can give him the correct information a bit at a time and hopefully change his perception of reality.
Exactly! I always read you posts with great interest Tenuc! It is difficult to remove a delusion. You have to do this slowly. Otherwise arguments usually backfire! Another technique is to not have too many arguments. A single argument or possibly two trivial arguments may be sufficient. Too many arguments and each argument become weaker. More here:
Eight ways to get exactly what you want
http://www.newscientist.com/article/mg19826551.400-eight-ways-to-get-exactly-what-you-want.html?full=true&print=true
Merry Christmas to Anthony, Svalgaard, Steve, rbateman, Tenuc, Lord Monckton and the rest of all the brilliant and honest people here at WUWT:
Christmas greetings from Invariant:
http://www.neurotheory.columbia.edu/~ken/cargo_cult.html
So I have just one wish for you – the good luck to be somewhere where you are free to maintain the kind of integrity I have described, and where you do not feel forced by a need to maintain your position in the organization, or financial support, or so on, to lose your integrity. May you have that freedom.

Steven Douglas
December 23, 2009 3:04 pm

I just caught this from their FAQ:
During the 1980’s the UEA/CRU was funded, primarily by the United States ‘Department of Energy’, to collate a global land temperature record. Since then they have undertaken several major updates to the record increasing station density and time series completeness. This is why the UEA/CRU owns the primary IPR (Intellectual Property Rights) for the land climate records.
Well, that clears it up. The “land climate records” were funded primarily by the United States Department of Energy (back in the day), but since the UEA/CRU later did some “value adding” since then, it became dib-hox-dice-we-own-it-all-now.
“OWNED” by the UEA/CRU. As in “private property of”.
Well, since we’re not interested in the “several major updates to the record” anyway, give us just that part that was paid for by the public – and namely, the U.S. Public.
Better yet, get out of intellectual dishonesty mode, stop implying that funding from the U.S. was somehow limited to the 1980’s, and recognize that the UEA/CRU doesn’t “OWN” data that is PAID FOR BY THE PUBLIC, and WHICH AFFECTS THE PUBLIC INTEREST, and is SUBJECT TO FOIA REQUESTS.

Ryan Stephenson
December 23, 2009 3:04 pm

I took a look at the data for about twenty of the stations in the UK. All but two of these stopped at 1998 (Why?). Of the remaining two, one stopped at 2004. Ross-on-Wye which was an official Met Office measurement site which used mercury thermometers for eighty years and only had one minor site change during that time as the site was the pet station of one keen meteorologist – is not included in the data.
The only site I could see that had a bang up to date record (right up to November 2009) was Lerwick in the Shetland Islands. Funnily enough I took the known lat and long of the Lerwick site and tried to find this station using satellite images. The location seems to refer to a barren patch of land with no structures of any kind on it.
The Met Office take max and min readings every day from a large number of stations not included here. Just in my locale Bristol and Fairford weather stations are not included here. The Met Office should release their own records, as paid for by the taxpayer, for all sites over all years.
This data has been cherry-picked, massaged and chopped about in every conceivable way. It is useless as a resource.

December 23, 2009 3:23 pm


bill (07:40:28) :

To store such data in the 80s would have been difficult – 1000 stations, 150kB per picture per month per am/pm=1000*150*2= 300MB/month
To store the text hand converted would be much less but then you loose the raw data.

ARE you not aware of a process called ‘transcription’?
It used to be that ‘raw input’, human written data, records, forms filled out etc. would be key punched by individuals known as key punch operators, bascially a process of transcription.
NOBODY kept ‘scans’ of raw documents on-line in the 80’s (define for me then the system, the processor, the software by which this was accomplished then. Show for me a a ‘page scanner’ the likes of which a business or univeristy department could afford … now price for me computer-interfaced graphics terminals from this era …)
TI used to have a columnar-based ‘form’ that served specifically as the input for key punching onto data onto an 80-column Hollerith IBM punch card.
ANY raw data would have been transcribed to an ASCII or EBCDIC form then stored on line … possibly followed by incorporation into a crude database/datarecord format of the era …
.
.

December 23, 2009 3:25 pm

When the wolves are at the door, you toss the dog a bone, and then shut the door real fast.
Also you make sure to release the tasty tidbit of data just after Copenhagen, not just before.
I am quite certain these CRU fellows are hoping all they release has the effect of dragging a red herring across the trail which howling hound dogs are following. I fear I have lost all faith that they want to get to the bottom of anything. They are not friends of the truth.
The question is, will they succeed in their attempts to divert, confuse and befuddle those who seek truth?
I doubt it. It is truly amazing how many willing minds get to work, all across the web, when the Alarmist obfuscators attempt any obfuscating tactic. A red herring which might have side-tracked the hounds for months in the old days barely slows the pursuit, now.
(Isn’t it wonderful how these Alarmist fellows raised the word “obfuscation” from the dead?)
The joke of it all is that air temperatures don’t really measure whether the world is warming. They measure what the sea is up to, and the sea has poorly understood responses to warming and cooling, many of which take decades, centuries and (in the case of thermohaline circulation) millennium to occur. So the very subject of air temperatures is itself a red herring of sorts.

December 23, 2009 3:35 pm

Plotted up the data for Vardoe Norway (file 010980) from 1840 to the last date of 10/2009. Showed a warming trend of 0.0092±0.0014 C/yr. Slope from 1840 to 1930 was 0.0092±.0037 and from 1930 to 2009 0.0095±0.0041
So it looks like a slow gradual warming trend that was not effected much by industrialization in the 1900’s.

Jason
December 23, 2009 3:38 pm

More on the map of stations.
To my eye, most, but not 100% of station appearing oceanic are actually small islands, and likely all the subset stations are islands.
On land other anomalies are the preference of Western US stations over eastern. Taiwan has not one station used and nearby Philippines seem to have a lot omitted from the subset. There is favoritism to stations in large coastal cities of Australia–already noted every NZ station is in subset (already known they know how to “play ball”)
All Somalian stations are used. Have they had one work since when? (shown inland, prob. coastal ports.) No nearby Yemen sites included. What’s wrong with them?
The Selectors do not like Italy or Portugal. Lower Volga River not used either. Recall how temp red/blue maps often leave this area valueless. No cone of South America used except in Atlantic ocean side. The Selectors ‘dis Cuba too.
Hawaii is loved, but US Navy observers such as at Midway are omitted.

Evan Jones
Editor
December 23, 2009 3:46 pm

Okay for now. But until we have the entire set of stations included plus any excluded along the way, cherrypicking is to be presumed.
I’ve gone from not trusting their adjustments to not trusting their selection of stations. I’d like to see what every official station — HCN or not — says.

Rob H
December 23, 2009 5:01 pm

If the data is not the original raw data, but the “value added” data, what good is it? It will only show the “warming” they have claimed. This is an attempt to claim they are being open with critics. Don’t let them get away with that.

Editor
December 23, 2009 5:19 pm

_Jim (15:23:55) :

bill (07:40:28) :

To store such data in the 80s would have been difficult – 1000 stations, 150kB per picture per month per am/pm=1000*150*2= 300MB/month
To store the text hand converted would be much less but then you loose the raw data.
ARE you not aware of a process called ‘transcription’?
It used to be that ‘raw input’, human written data, records, forms filled out etc. would be key punched by individuals known as key punch operators, bascially a process of transcription.

The transcribed product might have mistakes, it may not have the observers comments (especially if being key punched). It would no longer be raw data, information would be lost if it were destroyed.
I will grant you that “transcribed raw data” is likely the best starting point and may already exist. Errors could be kept under control by spot checks and homogenization software that reports likely errors so they can be checked against the raw data.

pby
December 23, 2009 5:54 pm

since original raw temp data is gone or unreliable why not start now as ground zero and see what happens in the next 75 years as far as spike in temps. Bet this will not be considered. These power grabbers are only interested in one thing – power.

Mesa Econoguy
December 23, 2009 6:52 pm

Well done, everyone who contributes here. Very, very well done.
Let’s keep going….

Alvin
December 23, 2009 6:52 pm

Maybe Bill Whittle at PJTV had something to do with this http://www.pjtv.com/?cmd=video&video-id=2889

rbateman
December 23, 2009 7:39 pm

_Jim (15:23:55) :
It’s not the transcribed data that I really want to see, it’s the hand-written data.
The one’s where you get them online in PDF form.
And if they did the trasnscribing, then they have the hand-written data.
Some of that is clearly missing.

jorgekafkazar
December 23, 2009 8:15 pm

Gregg E. (02:58:03) : “September 1995, Hewlett Packard produces the first CD-R drive priced under $1000, the Sure Store 4020i…”
I, for one, am not convinced that the original data files were deleted until recently.

Steven Douglas
December 23, 2009 8:51 pm

Bill wrote,
“The data was not confidential it was commercial as Jones said about a year ago….Jones said the data would be released when…He also said that…”
At what point will this madness stop? At what point will there be a realization that a mere rebuttal is sufficient to deny or explain away facts that are well established and not in controversy, but instead are completely ignored so as not to be acknowledged, let alone squarely addressed?
I am so tired of being patronized, and having my intelligence insulted.
Unfortunately for your defense of Jones and what he might have said publicly, we also have on record other things that he wrote privately, like the fact that he would rather delete files than send them, and that he would “hide behind” the confidentiality act that you so clearly, reasonably, calmly and patronizingly alluded to. Which he did.
Try to explain that away as “an unfortunate turn of phrase”, or boyz-will-be-boyz sentiment. There’s no trick to understanding this. He said, and I quote, “I WILL HIDE BEHIND IT”.

December 23, 2009 9:14 pm


Ric Werme (17:19:04) :
The transcribed product might have mistakes, it may not have the observers comments (especially if being key punched). It would no longer be raw data, information would be lost if it were destroyed.

a) You live (or die) with it *after* the transciption (you move ON with life; you make a call as a professional instead of worrying over every bit of frayed, unreadable paper. There are deadlines in this life and there aren’t enough hours to spend near-endless hours on a ‘personal project’).
b) Do you really think another transcription working off a ratty copy (or a COURSE image as linked by Bill) is going to improve the interpretation on the second transcription? Of course not …
c) You aren’t (I would think not) doing archaelogy with your ‘temperature’ dataset at this point; otherwise you’re going to be handling every data-record like they tried doing with the punched card ballots in Florida re: 2000 presidential election … ‘individual interpretation’ of chad, crease, extra punched holes etc. I’ll take the efforts and ‘reliability’ of a trained keypunch operator any day … One you’ve got XX MB of data and a time period of xx hours – it is a very simple process to assess what percentage of the ‘records’ you’ve got time to examine. 100% QA you’re NOT going to be able to do (one person or a few people anyway). Verification of a few select sites: sure. Does this make it ‘right’? Now you’re in the realm of making a value judgement on a situation that was controlled mostly by available resoures e.g. as ‘time’.
d) I don’t think other things have been considered. We have not considered what age, rot, fading, fires, sprinklers going off is going to do to these ‘paper’ records.
e) What is the ‘error rate’ on these records anyway? One percent (1%)? Five percent (5%)? No establishment at present as to what the ‘error limts’ are.
Extra credit: can any of YOU guys transcribe Bill’s linked raw record image? I didn’t think so … so the point is moot if *this* is the form the ‘raw’ records are in.
.
.

rbateman
December 23, 2009 9:34 pm

I have converted the MET data for Sacramento, CA and added it plus a graph to the bottom of the page
http://www.robertb.darkhorizons.org/SacMonthlyAMS_COOP.htm
The graph doesn’t compare with the raw data at all, until I noticed the
1880 cool year was on the opposite side, so I flipped the graph round.
Now, it looks like they read the tape backwards, and spit the data into the forward sucession of years. And their correction makes Sac. flat, as it is in real life (very flat place).
Is this a frontiere of science institution?
C’mon, MET, surely you can do better than this.
Or, maybe not. Times are hard, I know, and good help is hard to come by.
You sure you Metters wouldn’t be better off outsourcing the trudgy data to
the capable hands of M&M? I’m sure they can get the job done right.

rbateman
December 23, 2009 9:39 pm

_Jim (21:14:48) :
I’ll take those frayed pieces of paper, all of ’em you can dig up, thank you.
Read my post on Sac, CA data.
Any questions about the frayed paper?

rbateman
December 23, 2009 10:03 pm

_Jim (15:23:55) :
No, Jim, they didn’t buy scanners, they built them.
That’s why you entrust your raw science data to higher learning, they invent what they need.
“Accordingly, in 1991 Caltech and the ST ScI completed a Memorandum of Understanding which defined The Palomar–ST ScI Digitized Sky Survey. Major features of this program are digitization based on scans of the original plates, processing of all fields in the three passbands (2682 plates total), the use of a 1 sampling interval, and distribution of the full-plate pixel data to the community.
In support of this, the microdensitometers used for the original scanning for the Guide Star Catalog (Lasker et al. 1990) were rebuilt as laser-illuminated 5-channel systems capable of scanning rates well in excess of 1000 plates per year. The metrology is stable to 0.5 m, and the densitometry extends three density units above the sky (note that typical sky values on the original plates are in the range 1.5–2.5). The scans are of dimension 23040, which corresponds to 1.1 Gbyte per plate (for a 2.8 Tbyte survey total). ”
But you never, ever, throw away the raw science data.
That is the equivalent of sacrilege.
Scientists DON’T do that.
And what’s all this garbage about data storage capacity?
Universities invented the darn computers and storage systems.
If they need a bigger hammer, they build it.
Simple as that.

December 24, 2009 12:58 am

rbateman (21:34:12) :
“I have converted the MET data for Sacramento, CA and added it plus a graph to the bottom of the page
http://www.robertb.darkhorizons.org/SacMonthlyAMS_COOP.htm
The graph doesn’t compare with the raw data at all”
You are certainly right; something odd there.
http://www.vukcevic.talktalk.net/ScrCa.gif
I have made a short program to convert all data (1890 – 2009) to a temperature anomaly (with a 10 year running mean), in order to compare locations.
Ps.
Anyone: Is there a way of finding country and location without unzipping folders?

Chris Edwards
December 24, 2009 4:00 am

This is another total lie, All government agencies, the met office in Bracknel all had huge ICL mainframes, some had more than one, these all had a large room with perhaps 40 tape cabinets and another room with tapes in, also hard disk drives, perhaps 20 or so, I do not remember the capacity but as the taxpayer was picking up the tab, endless, this lot was likely to have been backed up on to modern media as technology improved. Yes the likelyhood is that some of this was deleted but more likely hidden in the mountain of data they have on various storage methods, they are career liars.

Steven Douglas
December 24, 2009 10:20 am

Well I checked. The UEA does have a history department as well as an American Studies department with more than a few Ph.D.’s wandering around. Why are they not consulting them for guidance? They might tell them that, historically speaking, the lie is often not nearly as damning as the subsequent coverups. That this is precisely the kind of thing that justifies putting the “gate” in Climategate.
Ground temp data from the UEA/CRU are worthless, so what are we left with?
The NOAA data sets aren’t even cited or used for conclusions without James Hansen’s “value-added” laundry services.
The NASA GISS data are now being checked against the raw data (that NASA has now attempted to hide), and what we find are station data that are so corrected, adjusted, homogenized and otherwise morphed into little hockey sticks that they bear VERY little resemblance to the raw ingredients that they were all cooked from.
Which leaves us with what…satellite records that they couldn’t get their hands on, and which have already been thoroughly checked reviewed (and even objected to and subsequently corrected) that haven’t been tampered with, and which show nothing at all unusual?
How about those thousands of “robust” “overwhelming” regional studies around the world that relied on some combination of the IPCC, NASA GISS and UEA data, that cite everything catastrophic warming related under the sun, but aren’t even qualified to conclude changes or human CO2 atmospheric attribution for their observations and conclusions?
Likewise, we have a “putative” consensus that says, in effect, that “The evidence may have been fabricated, but that doesn’t make the story untrue.”?
Ask CBS or Dan Rather how well that line sells.

Brass Monkey
December 24, 2009 10:30 am

With regard to the physical evidence for recent Global warming, I consider the NOAA records of the Barrow melting date on Alaska’s North Slope to be convincing. See Fig. 3 in:
http://www.esrl.noaa.gov/gmd/grad/snomelt.html
I have visited Barrow and Sagwon and have experienced the fairly rapid snowmelt on the slope.

Steven Douglas
December 24, 2009 1:39 pm

“With regard to the physical evidence for recent Global warming, I consider the NOAA records of the Barrow melting date on Alaska’s North Slope to be convincing.”
You do realize that you’ve cited something regional as evidence for something global, don’t you? That’s like citing local weather changes (Kennedy, I’m looking in your Virginial direction), mistaking weather for climate. Would you be equally convinced if you saw record freezing and ice accumulation and growth elsewhere? There’s plenty of that to go around, and not just in one Hemisphere. If I showed only that, I could make a very strong case for catastrophic tipping into the next ice age.

December 24, 2009 1:58 pm

North Atlantic temperature anomaly decreases with the latitude.
All locations are islands far from either urban areas or any large land mass.
http://www.vukcevic.talktalk.net/NA-TA.gif
Happy C’MASS and NEW YEAR to all of you

cdn.infidel
December 24, 2009 2:28 pm

This, while seemingly nice, means nothing. Releasing homogenized temps and some code that corresponds to their desired output is garbage.
What’s needed for a true picture is the raw data and the code they use for the hokus-pokus adjustments.
This is nothing more than smoke and mirrors.

Steven Douglas
December 24, 2009 7:55 pm

This, while seemingly nice, means nothing. Releasing homogenized temps and some code that corresponds to their desired output is garbage.
What’s needed for a true picture is the raw data and the code they use for the hokus-pokus adjustments.
This is nothing more than smoke and mirrors.

That’s exactly right. For just two out of literally hundreds of examples, here’s what you get with NASA GISS’ wonderful “USHCN adjusted” and then subsequently “homogenized” station plots:
http://examples.com/giss/santarosa_phased.gif
http://examples.com/giss/marysville_phased.gif
NASA took down its option to view the raw code that you see in the initial frame, leaving only their “adjusted” and then “homogenized” data.
That’s just two examples of garbage. with temp data that we can look and verify for ourselves. It’s bad enough when you can plainly see with a blink comparator what NASA is doing. With the CRU, they’re claiming that they can’t even give anyone the option to check.
Without the raw station plots, the data MUST be assumed as completely worthless as NASA GISS data.

Pamela Gray
December 25, 2009 12:56 pm

Brass Monkey, you seem to have not read the report you are referring to. I did. You conclude that the 10 day overall increase in early snowmelt is evidence of global warming. I did not see that conclusion in the report.

Brass Monkey
December 25, 2009 2:16 pm

Steven Douglas and Pamela Gray, when I spent the summer of 1964 on the Slope we could still land aircraft on ice bound lakes in early June and the snow melt happened about mid-June, except in gullies and north facing hill slopes.
NOAA’s data seem to show a consistent trend to earlier melt; although I haven’t been back to check, this seems significant. I don’t think it can be due to more open water along the north coast as it is the rivers of melt water that starts the coastal sea-ice melt, not vice-versa.
The area is underlain by deep permafrost, so unlike the Arctic Ocean ice it cannot be affected by warm water melting from below. (Except in a few places where there are hot springs at the mountain front.)
My guess is that the earlier snow melt is caused by solar radiation and has nothing to do with increasing CO2. The very recent possible reversal could suggest that we are entering a renewed cooling phase.
Perhaps the area could be regarded as a ‘coal-mine canary’ candidate to provide early warning of Northern Hemisphere temperature trends.