From 
Wed, 27 Jul 2011
All data sent to the Climatic Research Unit (CRU) at the University of East Anglia by National Meteorological Services around the globe to complete its global land temperature dataset CRUTEM3 will be released today, apart from data from 19 stations in Poland.
The University has been working closely with the Met Office to arrange the release of the remaining data not already in the public domain.
CRU has made its gridded datasets available online for many years, but climate sceptics had asked to see the data as received from National Meteorological Services and research colleagues around the world, who had sent data to the Unit for its research purposes.
Some countries’ Meteorological Services, including Poland’s, had been unwilling to have their data publicly released – some, who charge for this information, for commercial reasons.
Data from Trinidad and Tobago are being released against that state’s wishes. This is because the University is complying with the Information Commissioner’s Office’s instruction to release part of the database which covered the latitude zones 30° N to 40° S.
Professor Trevor Davies, UEA Pro-Vice-Chancellor for Research, said: “We regret having to release data from Trinidad and Tobago against that state’s express wish but we want to place beyond all doubt our determination to be open with our data and to comply with the ICO’s instruction.
“To demonstrate that determination we have made the decision, in discussion with the Met Office, to release the data from latitudes outside the 30° N to 40° S zone, with the exception of some stations in Poland which has explicitly refused permission. This means that data from 5113 weather stations around the world are now released.
“We are very pleased to be in the position now to release data for all but 19 stations and are grateful to the Met Office for its support over the past 18 months and for its major effort in contacting National Meteorological Services to seek their permission for release. In the interest of openness, we have released data from those which have not responded to requests to release.
“We remain concerned, however, that the forced release of material from a source which has explicitly refused to give permission for release could have some damaging consequences for the UK in international research collaborations.”
Research findings from the analysis of the CRUTEM dataset, on the course of global-scale land temperature changes, tally with those of other independent research groups across the world, including NOAA and NASA.
The data are available from Met Office website:
And from CRU:
http://www.cru.uea.ac.uk/cru/data/temperature/station-data/
With explanations at:
http://www.cru.uea.ac.uk/cru/data/temperature/
Steve McIntyre has a summry complete with historical details here.
steven mosher says:
July 27, 2011 at 9:55 pm
State a hypothesis about the adjustments. do you think they are big? small? large in number? small.
If its an interesting hypothesis I will test it. If you are just speculating, well then not so interesting
****
Hypothesis,
A graph of the adjustments over time will conform to the shape desired by someone attempting to create a hockey stick shape out of the combined dataset. Much like what you see with the GISS adjustments.
Doug;
Pumped hydro can handle maybe a day’s worth of missing output due to too soft/too strong/too cold/too whatever winds. Which is not a rare occurrence. Contrary to assurances and mythos, large areas can be simultaneously affected. And all such storage kluges are very lossy, hence expensive. Lipstick and mascara on a pig.
RobertM.
“Hypothesis,
A graph of the adjustments over time will conform to the shape desired by someone attempting to create a hockey stick shape out of the combined dataset. Much like what you see with the GISS adjustments.”
Your hypothesis is not falsifiable since it:
1. refers to a non observable: someone’ desire
2. has no magnitude.
For people who demand a return to normal science, I would expect better.
I’ll give you a do over.
“CRU did not release data. What they released is a summary of monthly averages taken from original data. It does not tell anything about the process of how they came up with thier summary in of itself.
Did they borrow from previous works, contemporary works, original observation data, etc., and what specific data came from what source? ”
Do any of you read the data before you vomit nonsense.
1. the sources are identified as a part of the station identifier
2. Some source data IS MONTHLY ONLY
3. if you read the request Letters you would understand what CRU requested from people.
“A graph of the adjustments over time will conform to the shape desired by someone attempting to create a hockey stick shape out of the combined dataset. Much like what you see with the GISS adjustments.”
Except we don’t see that with the GISS adjustments. If you are just going to make stuff up why not just flat out lie and claim you know the data are faked by Phil Jones because you saw him edit the numbers with a biro pen?
Ged says: July 28, 2011 at 8:40 am
Thanks, Ged
jaymam says:
July 28, 2011 at 2:50 am
Is this data from before or after manipulation by the programs described in HARRY_READ_ME.txt?
############
entirely different dataset. not an issue.
Look at what Australia will do to those committing “fraud” to counter a “fraud”
http://www.news.com.au/breaking-news/abbott-slams-draconian-carbon-cops/story-e6frfku0-1226104042303
Not quite on topic here but didn’t want to see this buried elsewhere.
I look forward to re-analysis with the new data, but I doubt that anyone will undertake that job. Has anyone reanalysed GISS, which has been maligned as CRU has, using the data and code that has been publicly available for years?
The negative response from the peanut gallery was predictable. It wouldn’t matter if CRU somehow managed to retrieve all the original station data – still they would be accused of doctoring it. Some people will never be satisfied, but rather than write CRU off and move on, they will wallow in the snark fest because it feels so good.
A far smaller number, approaching zero, may actually attempt hemispheric and global reconstructions with the newly released data. That would be useful, at least. I hope to see a post on that here.
steven mosher says:
July 28, 2011 at 3:41 pm
Yes, I have read through the station data, and the station data headers.
I have used Phil Jones CRU data before, and I have found additional sources both in monthly and daily, and compared the sets. So what if some sources are only Monthly? It does not mean they are the only game in town. Somebody at some point took original observations and/or tabulated it.
My point is this: The closer you can get to the original observation, the more information you have.
If that is nonsense to you, so be it. I stand by my conclusion.
I think another explanation might help explain temperature records. In statistics, when you are measuring a system, you can measure similar things in about 40 different ways and come to the same conclusion. High and low temperature is how the records have been kept for years, and although maybe not ideal, it does work out to give us a relative trend.
You have to prove that this method introduces systematic error otherwise its just as good as a good old average temperature anyway. When you are discussing statistics, when you have so many different unaccurate measurements, the sheer volume of entries will make the holes disapear so to speak in many ways. Sure, you probably can not trust completely what the weather was like in say 1913 Poland at all, but you can use that data with other incomplete data (assuming you don’t just throw it away anyway) and the error might be larger, but overall you can also say that its more accurate.
Where is the pitfall in this? Well in the GCM design, we take heat content as a measurement. Since high and low temperatures could very well differ from day to day on this measurement (in comparison to heat content) our GCM data is rather lackluster for this reason. Something to think about anyway… Its not systematic error but more or less accuracy error perhaps. Year to year measurements will not match up with two different scales and you are left with the conclusion that you are indeed comparing apples to oranges.
But that is a note of caution more then anything else. This is just one of the many reasons that the idea of BR in GCM’s (heat content) is just a terrible idea in the first place. Nothing really new there, but it is something to note. So there is nothing wrong statistically with the way we measure temperatures, but the problem comes in when we model based on different metrics such as “heat content.”
But enough of that…..Lets discuss systematic errors (such as UHI) which itself turns out to be on the magnitude of 0.1C or so. This is a rather large error, and it is important to check and double check it, but remember that since 1850 we have warmed. That is rather factual. None of the errors I can see in the data can back that up.
But adjustments can and will change slopes.
I am curious to the reasonings behind adjustments. Anything besides UHI or other systematic adjustments (to correct systematic error) are the places to look for issues. With temperature, I am especially not convinced that adjusting has been done correctly or for good reasons. The UHI adjustments are necessary. Other adjustments for the F to C conversion (for instance maybe?) are also necessary (think systematic bias). But what other systematic bias’s need to be fixed? That is the question and if every adjustment can not be explained due to a systematic bias, then the adjustment is worthless.
This is a point of contention. Its not up to us as sceptics to prove that an adjustment is wrong. Its up to the scientists to prove that the adjustment was necessary. Don’t let them switch the game on us. Adjustments should be throw away unless they are reasoned and well written. If we can not figure out the why, there is absolutely no reason to use the adjusted data. Just use the raw data and go from there.
“Just use the raw data and go from there.”
You realize this has already been done by various groups, notably by staunch skeptics at The Air Vent, and the results are a very close match to the surface records?
(The Air Vent reconstruction produced a higher centennial temperature trend than CRU)
Underlying data are held by the National Meteorological Services (NMSs) and other data providers. Such data have in certain cases been released for research purposes under specific licences that govern their usage and distribution.
It is important to distinguish between the data released by the NMSs and the truly raw data, e.g. the temperature readings noted by the observer. The data may have been adjusted to take account of non-climatic influences, for example changes in observing methods, and in some cases this adjustment may not have been recorded, so it may not be possible to recreate the original data as recorded by the observer.
THAT is what you REALLY want to see.
Because there are versions and revisions of the Monthly data sets:
Jones 99c
703710 5707 -13535 20 SITKA————— USA———- 1832 19897037100 70371000
1832 11 -13 17 31 89-9999-9999-9999-9999 61 56 25
1833 40 19 49 66 83 122 144 156 128 98 76 20
1834 1 19 40 48 74 118 119 115 108 84 68 40
1835 36 32 37 55 77 103 117-9999 106 55 46 -2
1836 -5 31 47 62 100 121 117 131 108 87 50 -1
1837 49 47 52 63 101 105 118 130 102 89 32 23
1838 14 25 19 51 89 109 128 128 101 79 53 51
1839 1 63 34 69 98 118 143 156 136 85 68 52
1840 73 4 79 59 87 102 127 144 116 69 21 21
1841 21 34 47 56 74 125 139 131 101 74 55 21
1842 26 21 -5 40 62 115 123 120 103 62 41 37
1843 -11 -30 18 44 76 109 121 124 98 77 32 24
1844 -20 32 12 51 77 131 133 130 101 60 21 15
1845 7 -16 25 52 87 106 130 142 97 73 28 49
1846-9999-9999-9999-9999-9999-9999-9999-9999-9999-9999-9999-9999
1847-9999-9999-9999-9999 84 106 121 133 98 63 21 11
1848 -30 -16 5 33 79 112 141 122 103 65 45 -9
1849 -33 -37 -14-9999-9999 92 112 116 97 66 36 -2
1850 -57 15 -22 37 74 89 125 131 102 68 38 23
1851 -3 31 49 79 114 131 152 169 137 113 66 -10
1852 52 26 18 70 99 124 156 158 138 100 32 -36
1853 16 39 42 71 107 113 134 142 129 82 -70 22
1854 -14 19 26 73 84 119 142 154 133 81 73 14
CRU/UEA today
703710 571 1353 20 SITKA MAGNETIC OBS. ALASKA 18322007 101832
1832 11 -13 17 31 89 -999 -999 -999 -999 61 56 25
1833 40 19 49 66 83 122 144 156 128 98 76 20
1834 1 19 40 48 74 118 119 115 108 84 68 40
1835 36 32 37 55 77 103 117 -999 106 55 46 -2
1836 -5 31 47 62 100 121 117 131 108 87 50 -1
1837 49 47 52 63 101 105 118 130 102 89 32 23
1838 14 25 19 51 89 109 128 128 101 79 53 51
1839 1 63 34 69 98 118 143 156 136 85 68 52
1840 73 4 79 59 87 102 127 144 116 69 21 21
1841 21 34 47 56 74 125 139 131 101 74 55 21
1842 26 21 -5 40 62 115 123 120 103 62 41 37
1843 -11 -30 18 44 76 109 121 124 98 77 32 24
1844 -20 32 12 51 77 131 133 130 101 60 21 15
1845 7 -16 25 52 87 106 130 142 97 73 28 49
1846 -999 -999 -999 -999 -999 -999 -999 -999 -999 -999 -999 -999
1847 -999 -999 -999 -999 84 106 121 133 98 63 21 11
1848 -30 -16 5 33 79 112 141 122 103 65 45 -9
1849 -33 -37 -14 -999 -999 92 112 116 97 66 36 -2
1850 -57 15 -22 37 74 89 125 131 102 68 38 23
1851 -14 12 24 56 90 106 126 141 108 90 52 -18
1852 41 7 -7 46 75 99 130 130 109 76 17 -45
1853 5 20 17 47 83 88 108 114 100 58 -70 14
1854 -25 0 1 49 60 94 116 126 104 57 59 6
Which set is closer to reality, and how would you know without access to what the sets were derived from?
I am considering adding the CRU temperature chart;
http://www.cru.uea.ac.uk/cru/info/warming/gtc.gif
to the WUWT Global Temperature Page;
http://wattsupwiththat.com/reference-pages/global-temperature/
based on this comment;
http://wattsupwiththat.com/reference-pages/global-temperature/#comment-702046
and the release of their data.
I have one issue, in that the chart confusingly states that it is “Air Temperature” versus Surface Temperature, when it is in fact a “global land and marine surface temperature record”;
http://www.cru.uea.ac.uk/cru/info/warming/
but I am otherwise unaware of any convincing evidence that the chart is erroneous. Can anyone present a compelling argument as to why Phil Jones CRU temperature chart should not be included on the WUWT Global Temperature Page?
rbateman, those concerns drill down past CRU to the weather stations/nat met offices involved, worldwide. Do you have a constructive suggestion for obtaining the data?
I do. In the manner of the surfacestations project, how about an effort, led by skeptics, (so it will be unimpeachable), to gather up data from source worldwide? Not only can the skeptical community then get their data free from the CRU taint and do their own temperature series, a new, raw data set will be available for any group to use (assuming no proprietary issues). Even better, Anthony and his colleagues (Fall et al?) can examine the question of bias in the global trend, as suggested in their paper earlier this year. It would be interesting to see for example, whether it was a fluke that biases cancel each other out for the US mean temperature record, and whether the diurnal range biases are manifest in other country’s records.
I’ve poked my nose into a couple of locations in Western Australia – the capital Perth Regional Office (BoM 9034) where I live and the Cape Leeuwin station (BoM 9518) which is a long-term lighthouse reading with no surrounding urban development at all over 100 years.
Perth’s temperature history is very messy with four different locations (inc Perth Airport) since 1897, each about 4km distant. See http://www.waclimate.net/perth-temperature-history.html
I’ve done a couple of charts comparing CRU, BoM raw and BoM HQ data (trendline polynomial order 4):
Perth Regional Office – http://www.waclimate.net/imgs/perth-regional-cru-bom-hq.gif
Cape Leeuwin – http://www.waclimate.net/imgs/cape-leeuwin-cru-bom-hq.gif
For Perth Regional Office, the CRU mean temp from 1910 to 1919 was 18.96C. The raw BoM data has it at 17.85C while the BoM’s adjusted HQ data is 17.92C. I’ve included Perth Airport in the chart because that’s effectively the official BoM record for the capital, starting 1910 even though the airport didn’t open till 1945.
If anything, the CRU data is adjusted to level out the temperature records at these two sites, to such an extent there’s been hardly any warming at Cape Leeuwin over 100 years and similar at Perth Regional Office taking into account the relocation of station 9034 from Mt Eliza (elevation 61 metres) to inner city Wellington St (elevation 19 metres).
For Perth Regional Office, I can only assume the CRU data has a UHI adjustment in the early 20th century up to 1.1C, which is probably fairly accurate (even though the adjusted BoM HQ data doesn’t seem to agree).
If the temperature data were compared to seed plots without quality control, the very idea of compiling such data for analysis would be nixed. The non-quality controlled seed plots would not be homogeneous and could not be used to determine seed characteristics under controlled conditions. No more than the temperature data from various countries and their poorly controlled stations. In my opinion, the most that could be done here would be to google map the stations for quality control issues which would shed light on the voracity of statistical conclusions made about global warming based on the just released compiled CRU data.
I wonder who we could get, someone experienced in surface station quality control reviews, to do this?
barry says:
July 28, 2011 at 10:17 pm
I USED to get original observation digitals from http://www7.ncdc.noaa.gov/IPS/coop/coop.html
but this popped up in March, and has been there ever since:
Notice: NCDC is working toward restoring access to the full period of record for all IPS documents as quickly as possible. Full access is not expected until mid-summer (July). We apologize for any inconvenience.
Failing that, newspaper archives are a good place to dig. We need an army of volunteers to do this.
I have done so for my area, and you’d be surprised at some of the things that went on 150 years ago.
Pamela Gray says:
July 29, 2011 at 8:54 am
You need seasoned/veteran meteorologists. Lots of them.
rbateman – i don’t think getting your data from NCDC (or GIur run of the mill skeptic.SS or CRU) is going to satisfy skeptics, who appear to trust none of the established sources. Basically, if data comes through any of these portals, it’s going to be seen as Team-tainted, right?
(If the skeptical community trust any of the above, please let me know, so I can defer to that source in future)
The only way I can see for the skeptical community to get at the truth is to gather the data from source, bypassing the suspect middle-men (NASA, NOAA, UEA/CRU), and creating an independent data base.
rbateman, do you think it is at all possible to create a workable data set? Or do you think the defects make it impossible?
Sorry about the bafflegab in the first para – big hands, sensitive touchpad. The brackets should say “(or GISS or CRU)”.
planet’s atmosphere is not same as human body – to be same temperature everywhere. Flactuation in Europe /USA by 8degrees is = to flactuation in Oceania by 0,8degrees = should be 10 times more monitoring places over Pacific than Europe /USA. Is it?! The truth always wins on the end boys. You from East Anglia; log on http://www.stefanmitich.com.au to see the questions one day will be asked in court of justice, under oath!
Shouldn’t this have been handled in advance?
Proprietary scientific data? What the heck is that?
They shouldn’t have included data they couldn’t publish. Period, end of story.
the fact that anybody can withold information of that kind is really frustrating and scary.