From 
Wed, 27 Jul 2011
All data sent to the Climatic Research Unit (CRU) at the University of East Anglia by National Meteorological Services around the globe to complete its global land temperature dataset CRUTEM3 will be released today, apart from data from 19 stations in Poland.
The University has been working closely with the Met Office to arrange the release of the remaining data not already in the public domain.
CRU has made its gridded datasets available online for many years, but climate sceptics had asked to see the data as received from National Meteorological Services and research colleagues around the world, who had sent data to the Unit for its research purposes.
Some countries’ Meteorological Services, including Poland’s, had been unwilling to have their data publicly released – some, who charge for this information, for commercial reasons.
Data from Trinidad and Tobago are being released against that state’s wishes. This is because the University is complying with the Information Commissioner’s Office’s instruction to release part of the database which covered the latitude zones 30° N to 40° S.
Professor Trevor Davies, UEA Pro-Vice-Chancellor for Research, said: “We regret having to release data from Trinidad and Tobago against that state’s express wish but we want to place beyond all doubt our determination to be open with our data and to comply with the ICO’s instruction.
“To demonstrate that determination we have made the decision, in discussion with the Met Office, to release the data from latitudes outside the 30° N to 40° S zone, with the exception of some stations in Poland which has explicitly refused permission. This means that data from 5113 weather stations around the world are now released.
“We are very pleased to be in the position now to release data for all but 19 stations and are grateful to the Met Office for its support over the past 18 months and for its major effort in contacting National Meteorological Services to seek their permission for release. In the interest of openness, we have released data from those which have not responded to requests to release.
“We remain concerned, however, that the forced release of material from a source which has explicitly refused to give permission for release could have some damaging consequences for the UK in international research collaborations.”
Research findings from the analysis of the CRUTEM dataset, on the course of global-scale land temperature changes, tally with those of other independent research groups across the world, including NOAA and NASA.
The data are available from Met Office website:
And from CRU:
http://www.cru.uea.ac.uk/cru/data/temperature/station-data/
With explanations at:
http://www.cru.uea.ac.uk/cru/data/temperature/
Steve McIntyre has a summry complete with historical details here.
Further to Brian H’s post which brooks no disagreement. Last year I suggested that using (Tmax +Tmin)/2 was not a suitable approximation for Tmean. Steven Mosher put me right and, after testing the data to which he had pointed me, I had to agree with him.
That was, however, before I came across the following post by Anthony http://wattsupwiththat.com/2010/04/17/giss-metar-dial-m-for-missing-minus-signs-its-worse-than-we-thought/.
Consider the hypothetical station report where Tmin is -10 and Tmax is +60, Tmean is approximated as 25. But, if as Anthony shows happens occasionally, the minus sign (M) has been dropped from recording the 60 reading and Tmin is really -60 while Tmax is, perhaps, 0 the approximated Tmean should actually be -30. Quite a difference. Even if this single small error was averaged over a whole month the records would overestimate the temperature by more than 1.5 degrees for that month alone.
Of course if stations used continuous temperature recording rather than daily max and mins this could not happen but although the facilities are available I understand that, for the sake of consistency with the past, error prone recording and averaging are still the norm.
“Brian H says: July 28, 2011 at 4:03 am
Even the definition of average is at issue.”
It’s not at issue. CRUTEM, like all such induces, is based on the monthly average of daily min/max means. No-one claims it’s any different.
Whether some other measure might be preferred is fanciful – what we actually have is a historic record of daily min and max temps. Even climate scientists can’t change that.
@- Brian H says:
July 28, 2011 at 4:03 am
“Even the definition of average is at issue. In close-enough-for-government-work terms, envisage a day with 23 hours of 10°C temps, and a spike in the afternoon when the clouds broke, up to 34°C. By the “splitting of min & max” system, the day would get recorded as 22°C. By weighted average (hoursxdegrees), it comes to 11°C.
NO calculation based on the 22°C figure would be meaningful or other than totally misleading.
Yet that’s how it’s done.”
And as long as the method is consistent it provides a useful metric for the detection of trends and variance.
But your scenario is a little outlandish, and not as clear-cut as you imply. It would require most unusual weather conditions to create just a 1 hour spike 24degC higher than the rest of the day just from changing cloud cover.
There is also the matter that IF conditions where such that the temperature could rise to 34degC in this rapid manner the min-max average probably does represent a better measure of local thermal energy levels than the time-weighted version. And is much easier (until recently the only method possible) than a time-weighted figure.
Both have their advantages and problems, but both can provide a useful metric if consistant over time.
Newsflash! Trinidad Hot Over Release of Temperature Data.
The release yesterday by the UEA/CRU reveals the ups and downs in Trinidad’s on-going stormy relationship with Tobago which violates their privacy rights. In a related story, independent investigators had discovered that the data was compiled from on-line responses of listeners to a leading talk radio show in Trinidad.
sarc/off
Poland is in the EU and the Polish data are thus subject to the Access to Environmental Information directive — regardless of whether the directive was transposed in Polish law, regardless of the legal status of data collectors, and regardless from where in the EU data access is requested
What’s up with Poland anyway? And why withhold such a tiny number of stations?
What a truly bizarre bunch they are at CRU.
The ICO decision is here…
http://www.ico.gov.uk/~/media/documents/decisionnotices/2011/fer_0280033.ashx
Yes but it’s all tainted now.
Who will believe the figures when they had over a year to manipulate them ?
Are these figures even relevant anymore, because it seems like the debate has moved on ?
Comments from Trevor Davies ,Gavin Schmidt and Bob Ward wrt the UEA data release-
“We released [the dataset] to dispel the myths that the data have been inappropriately manipulated, and that we are being secretive,” says Trevor Davies, the university’s pro-vice-chancellor for research. “Some sceptics argue we must have something to hide, and we’ve released the data to pull the rug out from those who say there isn’t evidence that the global temperature is increasing.”
“One can hope this might put an end to the interminable discussion of the CRU temperatures, but the experience of GISTEMP – another database that’s been available for years – is that the criticisms will continue because there are some people who are never going to be satisfied,” says Gavin Schmidt of Columbia University in New York.
“Sadly, I think this will just lead to a new round of attacks on CRU and the Met Office,” says Bob Ward, communications director of the Grantham Research Institute on Climate Change and the Environment at the London School of Economics. “Sceptics will pore through the data looking for ways to criticise the processing methodology in an attempt to persuade the public that there’s doubt the world has warmed significantly.”
http://www.newscientist.com/article/dn20739-ok-climate-sceptics-heres-the-raw-data-you-wanted.html?DCMP=OTC-rss&nsref=online-news
@Garacka
You say : “Trinidad’s on-going stormy relationship with Tobago”
But that isn’t Cricket old bean, what-oh, chip-chip.
In the Carribean, Cricket is FAR MORE important than any
of the climate-wittering than goes on in here and elsewhere.
http://www.ttcricketboard.com/
No mention of HADCRUT or any other kind of CRUT at that
website which has the following “Mission Statement” …..
“To develop and sustain cricket as the most successfully organized sport in Trinidad and Tobago and the National Team as the best in the West Indies, in collaboration with its players, zones, clubs, administrators, affiliates and all stakeholders.”
—–
Just to pursue a point, and to show how seriously Cricket is taken, over the antics of hubristic climate wittering, Europeans. Here is an abstract from a book, entitled, “Brian Lara in Poetic Form”……..
“This essay considers the place of Brian Lara in Caribbean poetry through two literary framing devices. It first situates the iconic image of Lara within the literary frames provided by T.S. Eliot’s ‘Tradition and the Individual Talent’ and Kamau Brathwaite’s notion of ‘mwe’, before unpacking Lara poems composed by writers and performers as well known as Jean Breeze, Howard Fergus and Paul Keens-Douglas. The discussion suggests that by reading these pieces as a collection Lara is shown to represent the Caribbean’s ongoing negotiation between the one and the many, as well as the potentially hazardous over-investment in the individual hero. Consequently, the discussion is less concerned with the actions and personality of Lara than it is with his heroic image in poetry and the critical messages West Indies cricket and the Caribbean more generally may take from such literary representations.”
That’s SERIOUS Cricket analysis !!!!!
Yet I bet that most readers will be unable to understand why the folks of Trindad & Tobago would want to dissect the anatomy of such a folk hero, and maybe don’t even know who Brian Lara really is. Cricket isn’t about life and death, in the Carribean. It’s far more important than that !
Funny, the British Advertising Standards Authority can require companies to pull advertisements when the advertisers fail to provide un-retouched images for comparison if someone believes the ad to be inaccurate, but CRU does not have to pull their temperature products when they fail to provide original information to people attempting to verify their results. Guess there is no oversight group for climate scientists.
http://shine.yahoo.com/channel/beauty/loreal-ads-of-julia-roberts-and-christy-turlington-banned-for-airbrushing-2516498/
@Nick Stokes,
That’s a great gadget and work you did.
It’s too rare around here that good things are said of others and honor given where it’s due, so I just wanted to thank you, and give you encouragement. I enjoy your demeanor when it’s calm and focused like this, and you’ve added a lot to the discussion here.
“Perhaps the good Professor Jones can enlighten us on…”
I wouldn’t trust Prof Jones to enlighten a candle.
What a shameful and humiliating exercise this has been for the UEA in general and the CRU in particular!
izen;
deliberately misleading answer, I think. Not surprised. My example was constructed to maximize the effect, but the size of the effect is almost secondary to the inevitability of it. The core problem is the sparseness of the data used, and the amount discarded. That makes it, as Anthony’s observations about the dramatic effect of sign reversal (far from unknown) shows. Intelligently applied filters (‘reasonableness tests’) catch such boo-boos, but are surprisingly rarely used. Sometimes a boiling hot Lake Michigan sneaks into the records!
I believe “climate skeptic” to be a step up from “climate denier,” but no less meaningful. Are “climate skeptics” sitting on the fence as to whether or not the climate actually exists, while “climate deniers” categorically state the climate does not exist?
Oops! “no less meaningful” should read as “equally meaningless.”
Luboš Motl says: (July 28, 2011 at 2:11 am) says, “My guess is that the old Polish data are probably not excessively high-quality ones haha.”
I’m not a scientist, but a historian. A few years ago, I looked at the the pre-war GISS weather station data from Poland. I did this only out of curiosity, and chose Poland only because I speak Polish and have done some archival work in that country. All I can say is that what I found confirmed my preconceptions about the quality of the temperate record. Many of the records did not differentiate between high and low temperatures (most records which I saw only had one temperature recorded). None of them indicated what time of day the reading was taken. Many dates were simply missing.
It should surprise no one that the Polish pre-war records were in such bad shape. Most of the pre-war governmental records in Poland were destroyed during the war. I have not seen the weather station data taken during the Polish People’s Republic. However, I would not be surprised if it were in just as bad of condition. It is entirely possible that many of those records were merely faked by the meteorologists, as they spent their days drinking vodka or standing in bread lines rather than doing their jobs. That’s just the way it was back in those days. I would not be surprised if the Poles are simply embarrassed about the state of their records. A proud people, the Poles.
@erik in Tehachapi
That’s where around 1% of California’s electricity is made (sometimes, if the wind is a-blowing), by the use of over 5000 windmills, isn’t it ? I took a look on the Google Satellite view. What a boondoggle that lot is. Maybe there is enough to power the towns of Mojave & Tehachapi on a windy day, but what an eyesore it is, and is there any Eagles left in the area at all ?
And what are all these “roads to nowhere” in the area ?
The whole area is cris-crossed with scores of “roads” and few actual buildings.
It is like a large town that has burned down and blown away in a desert sandstorm.
http://url2it.com/gedc
Maybe those windmills didn’t sustain as large a development as was first envisaged ?
Erik;
Proud but embarrassed Poles they may be, but if their vodka-soaked data got into the computations, it needs to be disclosed.
Alphabet Soup says:
July 27, 2011 at 11:24 pm
Trying to get the UEA to answer FOi requests is like trying to pull teeth…
… from a Pharaoh, in a locked sarcophagus, in a walled in chamber, deep in the core of a solid pyramid, buried in the sand, at an unknown location, in a massive desert somewhere within 500 miles of one of the banks of the longest river on Earth.
But first you have to find prove the teeth exist.
Erik in Tehachapi, it’s a small planet indeed- I live in Tehachapi myself, work in Mojave. Have you ever read the “Crosstime Engineer” novels by Leo Frankowski? They’re a lot of fun, very politically incorrect.
On topic, I’m much too busy to dig into the data files, but I’m sure there are some people in the skeptic community who have the time and skills to audit the data. I’m looking forward to their summaries.
Brian H (July 28, 2011 at 11:01 am), “Proud but embarrassed Poles they may be, but if their vodka-soaked data got into the computations, it needs to be disclosed.”
Agreed. I don’t post often on science blogs. Or, any blogs for that matter. But, the issue of complete scientific transparency is one of longstanding interest. Note: http://climateaudit.org/2010/02/07/a-small-document/#comment-220677
I was Erik in Cairo back then. That was before I was evacuated from Egypt.
Doug, Tehachapi is much nicer than Cairo. Mojave, not so much. (-;
Axel, it’s not really that bad. A new power line is being installed to carry the wind power over to Castaic lake, where there is a 1.4 GW pumped-hydro storage system. The wind power can be stored in Pyramid Lake and released when needed, so it’s one of the few wind-hydro combinations that actually can be reliable. I’d be happier if the surplus wind power lowered my utility rates in Tehachapi, though… supply and demand ought to count for something, but then, I’m in the people’s republic of Californica.
Duke C. says:
July 28, 2011 at 7:29 am
CRU did not release data. What they released is a summary of monthly averages taken from original data. It does not tell anything about the process of how they came up with thier summary in of itself.
Did they borrow from previous works, contemporary works, original observation data, etc., and what specific data came from what source? Without such knowledge, thier data sets cannot be duplicated, and no measure of uncertainty from thier in-house processing can be assigned.
One may take their procuct at face value and attempt to replicate their findings.
If, as Phil Jones has avered to, original data from which the station monthly averages has been lost, then the value of the CRU product is less than 100%.
How much orginal station data can be accounted for at this time? Key.