Berkeley Earth has just released a new version of the Berkeley Earth dataset, which is more comprehensive than the version released in October 2011, and fixes some bugs in the initial release. You can access the new dataset here: www.BerkeleyEarth.org/data.
The new dataset includes:
- Additional data not included in the first release of the dataset (e.g. early data from South America, data through 2011, etc.)
- TMIN and TMAX (in addition to TAVG)
- Intermediate versions of the data (including multi-valued, single valued, with and without seasonality removed, with and without quality control)
- Source data in a common format, as well as links to the original sources
All files are in Text format, but if there is enough interest we can also provide them in Matlab. Steven Mosher has independently put together a R function to import the Berkeley Earth data, which is available here:
http://cran.r-project.org/web/packages/BerkeleyEarth/index.html
In making these data accessible to professional and amateur exploration we hope to encourage further analysis. If you have questions or reflections on this work, please contact, info@berkeleyearth.org. We will attempt to address as many inquiries as possible, and look forward to hearing from you.
Best regards,
Elizabeth Muller
Founder and Executive Director
Berkeley Earth Surface Temperature
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Philip Bradley says:
February 18, 2012 at 1:45 am
This is the same nasty data we have ALWAYS had access to more or less. We have a dodgy series generally of one minimum and one maximum temperature value per day
AFAIK it is exclusively one minimum and one maximum temperature value per day.
##############
wrong
Gee, couldn’t the Best team afford at least one competent computer scientist on the project?
Max and Min are data (of unknown quality, for what its worth) but AVG is NOT data, it is a derived quantity, and its inclusion serves no purpose other than to bloat the database by 50% without adding value. Or was that inclusion for the benefit of “Excel-challenged” climate scientologist.
In fact, if AVG is computed as (Max + Min)/2, then it isn’t even a value with any physical definition or validity.
Thank you for the explanation Steven, if I have time this week I’ll throw my extra clock cycles your way if that would be useful. I’ll have to figure out what I need for R packages, but hopefully that’s not going to require a lot of thought to get running 🙂
Zeke and Mosher have a guest post at Climate Etc. explaining the new data set and also include some new data plots
http://judithcurry.com/2012/02/18/new-version-of-the-berkeley-earth-surface-temperature-data-set/
Is there any data on the history of urbanisation at ,and local to, each of the stations providing data?
A lot of urban expansion happened in the 1976-2000 period.
Until this historical, urban change, type of data is available to properly account for UHI effects in the data sets, they are pretty much useless for calculating real global temperature trends.
AndyG55 says:
February 18, 2012 at 12:57 pm
Is there any data on the history of urbanisation at ,and local to, each of the stations providing data?
A lot of urban expansion happened in the 1976-2000 period.
Until this historical, urban change, type of data is available to properly account for UHI effects in the data sets, they are pretty much useless for calculating real global temperature trends.
#############################
1. yes there is some historical data both on population and the percentage of land that is urban.
2. We can also just select stations that have no “built” area surrounding them. There are
roughly 14,000 such stations. They have no built area, zero to small population.
Looking at rural only you see that the answer doesnt change much. Its warmer. the real question is why
Philip Bradley says:
February 18, 2012 at 1:45 am
This is the same nasty data we have ALWAYS had access to more or less. We have a dodgy series generally of one minimum and one maximum temperature value per day
AFAIK it is exclusively one minimum and one maximum temperature value per day.
I’d like to see someone compare the BEST min/max derived average to the average derived from hourly measurements available for quite a number of sites.
###############
there is literature on this. read that before you suggest that someone else do busy work.
steven mosher says:
February 18, 2012 at 7:31 am
Philip Bradley says:
February 18, 2012 at 1:45 am
AFAIK it is exclusively one minimum and one maximum temperature value per day.
##############
wrong
How about telling us what is correct and providing us with a link.
The BEST website gives no indication they use other than Tmin and Tmax.
They do provide Taverage, but there is no indication this isn’t derived from Tmin/Tmax.
http://berkeleyearth.org/data/
Here is a long paper on their mathmatical methods, but surprisingly no mention of what the source data is.
http://berkeleyearth.org/pdf/berkeley-earth-averaging-process.pdf
Smacks of amateurism if the first issue was substandard. This reminds me of everything in the discussion of the possible AGW – comments and conclusions, including the big ones to introduce carbon taxes and subsidies, are based on premature publication. Everyone is going off half cocked for some reason – is it the glory of saving the world or proving the world does not have to be saved that causes this? As an aging engineer I just want true facts (all facts should be true!) and a good explanation of cause and effect. Everything else is unnecessary and offensive!!. At the moment egos and profile rule. In time all those driven by ego and self serving profile will look like fools.
Until you get past the drudgery of getting the raw numbers in the best form possible, you can use exquisitely sophisticated mathematics to derive little more than a big argument.
Please, stop the argument and turn your minds to materially assist with data quality validation. Major errors reside in some countries. Please help fix them.