Posted by Jeff Id on January 5, 2010
So I’ve learned a great deal playing around with the GHCN data, I think this is a reasonably significant post. Ya know, it’s hard to know anything until you try it yourself and I hope more of the readers here will. Again, there was a problem in my last CRU post, however, the more I look for it, the more avenues there are to explore. The issues have been corrected by avoiding the remaining possibilities in this post.
Of all the details worked out over the last two days, one is a decent gridded average of temperature data. Unfortunately for us skeptics it looks like Figure 1 which is pretty similar to the CRU plot.
Figure 1 – all data gridded
Yes, there is warming according to our temp stations, but I don’t think comrade Phil Climategate Jones would like this curve, because the warming in this curve happens entirely after 1975.
It’s nice to see a good quality CRU similar curve after the previous effort, but that’s how things happen when you do your work in public. The plot above uses all the data with each 5 digit temp station code averaged together individually, as my first post did. Anomaly is calculated over the entire series length.
The concern which was explored in some detail, regarded the hypothesis that the loss of stations in recent years created or biased the trend. It came about since so many stations are lost in recent years as Ken Fritsch pointed out in the recent CRU #3 thread.
Figure 2 – Stations per year GHCN per Ken Fritsch from KNMI database
I’ve run dozens of plots over the last several days, some of which contained an error in them created from data selection or a code problem in my previous post. Using the algorithm which averages together individual station ID numbers, I get very consistent CRUesque patterns. the warming is common to a variety of data sorting processes. This methods avoids the issues of data selection or code problems in the other methods and I’m confident in the accuracy of these results, but you should check them.
Several methods were employed to test the consistencey of result, including sorting for Rural and Urban, and sorting for several different time lengths of station data. All varieties so far produced very similar same results. There are, however, interesting revelations from examination of the slight differences.
Figure 3 – 100% Urban Data
Figure 3 is a plot is the urban data only. Of note is that the warming starts at 1978 with only slight warming beforehand and launches up about 1.2 C with no end in sight. Also, 1982 isn’t much reduced from around 1940 which is different from the global average in Figure 1. So the next thing I did was to plot the rural data.
Figure 4 – Rural station data.
That looks a great deal more like the satellite data. The temp rose and fell again prior to 1978 and rose again since 1978 is maybe 0.5C total. I tend to ignore data prior to 1900 due to the very small number of stations. I don’t think the drop in temps to 1900 levels in the early 70’s is the kind of curve that supports the high CO2 sensitivity claimed by climate science. Does anyone remember the snow storms of the early 70’s? Yeah, yeah just weather, I know.
One of the other avenues explored at great length , yet still isn’t finished, was how station starts and stops affect the trend in recent years. To explore that, one of the several methods I used was to sort data according to number of available data points. Below, I presented the gridded global average for all stations with at least 100 years (1200 points) of available data, since many of the stations in Figure 2 were started in 1950.
Figure 5 – Urban gridded data from stations with at least 1200 points
The urban data only in Figure 5 has an even steeper curve, you would expect this from longer series in this type of analyis. The temp rise since 1978 is about 1.2C. The rural 100 year curve is below.
Figure 6 Rural temperature stations with at least 1200 points
So the Rural stations show about 0.7C of warming since 1978. Visibly less warming than the urban stations by themselves. Also note the slight downtrend in recent years. Since the industrial revolution occurred a hundred years ago, it’s hard to imagine this curve is created by CO2. Still I’m not denying the heat capturing ability of CO2, just that the curves here don’t show a continuous warming but rather a short term recent spike.
So of course we should look at the difference between urban and rural stations.
Figure 7 – Difference between urban and rural data as identified by GHCN
Figure 8 – Difference between urban and rural data from GHCN stations with at least 100 yrs of data (1200 monthly points)
Look at that curve! Despite the crudeness of the categorization of thermometers, there is a clear warming bias for big city data. The curve in Figure 8 ends at 0.6C difference. What’s more, the trend between the two looks statistically significant. If Phil Climategate Jones and Michael Marx Mann can choose which data they want to show and hide the rest, I think it’s only fair to choose to look at trends only from Figure 8 since 1978 (even though it won’t make much difference). After all, one hundred percent of global warming has apparently occurred since that time. Let’s do a simple significance test.
Figure 9 – Statistical significance between rural and urban stations
Woah, it’s not even close, a trend of 0.12 and a no trend null hypothesis limit of +0.04. The difference between urban and rural warming is as great as the entire trend in UAH data over the same timeperiod.
Just how much trend do the ground stations show.
Figure 10 – All urban stations different Y scale from 11
Figure 11 – All rural stations different Y scale from 10
Even Figure 11 is still greater than UAH and RSS satellite data but it’s one heck of a lot less than the urban stations. Of course we would be remiss to not mention that WUWT has taught us what rural stations often look like.
What could go wrong with sophisticated technology like that?
The R code for this post is here.
Wasn’t sure which thread to add this too, but it’s a pretty amazing video interview with John Hirst, head of the Met Office.
Jeff: Is this GHCN raw or GHCN adjusted data? It’s not indicated, so I am assuming adjusted.
USHCN raw data shows +0.14C per station (equal weighting) per century.
Adjusted data shows +0.59 per century.
Urban sites warm c. 0.5C faster than non-urban. 9% of USHCN sites are designated as urban (17% suburban, the rest, rural).
Jeff ID:
Now you’re breaching a whole new topic: Where the measurements need to be taken and which ones are meaningful: If you’re discussing tropospheric measurements, should they be taken near ground level, or above the atmospheric boundary layer?
But if you are discussing measurements in the boundary layer , obviously urban warming (and more generally local environmental changes associated with changes in surface covering) is an important regional effect.
There are multiple layers of errors here.
1) Climate data is known to be corrupt.
2) As noted by Stephen, if some are cooling and some are heating… we may have to look very closely as to why – data is insufficient.
3) Microclimates are everywhere and have been alluded to based upon altitude and wind, but there are many more factors to consider.
4) The CEC in Calif. identifies 16 zones but I have Identified at least that many SUB-zones within the “official climate zone”
5) Lets consider the strength of the data prior to making decisions or recommendations.
Steve
@Pete
Thanks for this. This is the first time I’ve ever seen anyone on the BBC actually question the assumed consensus on AGW. Normally it is presented as a foregone conclusion, for Instance Prof. Iain Stewarts “Climate Wars” series. Whoever had researched Andrew Neil’s questions had done a very good job – will probably be looking for alternative employment shortly! So now the Met are openly admitting that temperatures have not risen since 1998. But luckily their model had already predicted this would happen. It looks like history is being rewritten somewhere.
What next? The Met Office “discover” actually no link between 0.04 % CO2 in the atmosphere and atmospheric temperature ?
You can indeed do what you want with data.
To amuse myself with the snow outside, I decided to analyse CET winter and summer seasons since 1798.
I wanted to see how temperature changed through complete Hale Cycles.
Course, you can start with an odd cycle or not, and interestingly, the CET mean for a Hale cycle shows apparently different results depending on which you use – OE or EO.
If odd cycles start the 22-odd year winter seasons, you get: from 1843 – 1913 – stasis. Then: POW!! A 0.6C rise 1913 – 1933, before dropping back to stasis and rising steadily to 1998 but not coming close to the mildness of the early 20th century.
But if you do it the other way round, the result is a delight to warmers: from 1855 to 1986: STASIS. But 1986 to 2008 – close to 1 degree increase in winter temperature!!
Similar things in summer: POW in the early 20th century, with later 20th century less warm but warmer than 19th century; OR step change in 20th century with statis to 1986; then POW!!
To editor: please clip prior posting – accidently hit submit half way through……
Pete (12:04:46) :
“Wasn’t sure which thread to add this too, but it’s a pretty amazing video interview with John Hirst, head of the Met Office.”
Thanks for posting the Video, Pete, it was well worth watching.
John Hirst was so poor I was cringing for him most of the way through – what a complete and utter plonker. No wonder that the Met Office is a joke here in the UK and no-one I know believes anything they try to tell us.
For WUWT non-UK readers, this guy was taken apart in the video by Andrew Neil, a political presenter/consultant. Imagine what Lord Monckton would have done to the poor guy :-))
jorgekafkazar:
We’re to the question of quantifying how much that matters.
If you are averaging the temperature over a 250-km column of air being dragged across the microphone by advection, that tends to moderate the effect somewhat, especially if we are discussing temperature anomalies or trends.
Carrick “While I agree on the need to be cautious, urban islands are real sources of heat.”
Not necessarily. For example, if you have a temperature station in a middle of a field and then build a wall just to the north of the station, the average temps will all increase. There has been no net heating in that area though. If you place a temperature station in the shade just north of the wall it should register cooler temps. The average of the two stations should be the same as one station with no wall.
The more interesting issue is asphalt. I have measured temps from black roofs and white roofs. The white roofs have higher temps in the day and lower temps at night than the black roofs. It is my understanding that the higher temps we are seeing is not daytime highs rather it is nighttime lows being higher. Which is what I would expect asphalt to do.
The other issue is rural settings which typically have plants in the area. Areas with plants are cooler than areas without, because plants convert heat to chemical reactions and don’t reflect as much heat. Plants are heat sinks.
The UHI is the difference in temperature between a thermometer being surrounded by heat sinks and reflective surfaces. I have measured the differences within 60 feet and found 30+ degree temperature differences.
Re: Richard Wakefield’s analysis of historical Canadian temperatures.
You may be interested in this tidbit…About 6 years ago my son compared historical temperatures for 6 stations in Alberta (3 rural, 3 urban) for his Grade 6 science fair. I don’t recall all of the details, but his main summary was that over time the temperatures were getting “less cold” (ie, the summer maximums were staying about the same, but the winters were not as cold as they were in the past). Sorry I don’t have the details with me, but I thought you might appreciate this anecdotal confirmation of what you have been finding.
ps…his evaluators at the science fair did not appreciate his contention that “it isn’t getting warmer – it’s just getting less cold”!
Hamish
Thanks for the back up. I was in Cairo a while back and it stank and the air was yellow. In India the burning of dung fires for domestic cooking is causing particulate pollution – worst indoors of course – and aren’t the Chinese building a new coal fired power station every week now? I thought that I’d read somewhere that all these particulates in the air and settling out could affect the albedo of ice thousands of miles away…
I don’t think the Chinese are ahead of the west in the application of solar power. But I could be wrong!
TonyB (09:42:44) : Jeff Id and Lucy Skywalker
Temperature information only means something if we are comparing like for like…
Perhaps I’m being thick but I don’t understand what you are addressing in my comments… What I was pointing to was a graph of Salehard with Spring, Summer, Autumn and Winter records scaled to the mean of each. For a long time, patterns can be seen echoed across the seasons… then in the last few years, the WINTER record shoots up with a serious obvious anomaly that is not echoed in the other months. This is surely comparing apples with apples – it doesn’t matter here if the station is moved etc. What matters is that the winter record changes so drastically, out of step with the other seasons, as to show something is suspect, and make UHI a likely suspect. What this does not show is the UHI effect at all seasons. But it does show that the annual temperature is already suspect on account of the winter temperature.
Lucy Skywalker, thanks for that. It indeed follows a very similar line that my average temp does for Belleville. However, this appears to be “anomaly” (however that is definted) of the yearly average temperature trends.
What I did was plot out the extreme ends of each year. That is, the number of days in the year that were above 30C and the number of days below -20C. When you plot that along with the average temperature you get a very interesting result. That plot you see in the Russian location may show the same thing, but it needs to be evaluated on the daily, not monthly temps, so you can count the days at the extremes.
I’ve taken this further looking at the onset of spring and the onset of winter (winter’s duration). Getting interesting results from that, but not yet done.
The bottom line from what I see is in the 1930-1950’s there were far more hotter days and colder days that today. As seen in the first upswing of that Russian plot. Then, at the same period that the Russan data drops, Belleville shows the lowest hot days, and a somewhat lower number of cold days. Then from 1985 onwards, the number of hot days increased (but less than the first increase) but the number of cold days continued to drop. Hence the trend seems clear. A narrowing of the variation within the years since 1921 for this location.
Soon as I get this done, I’ll post it for all to have a look and verify. This MUST be verified.
GeneDoc (08:54:22) :
Yep, that is EXACTLY why I wanted to see what was going on in far more detail.
Besides, can someone explain to me what “anomaly” temperature actually means? I seem to see different definitions. How is it actually calculated?
Here is a challenge. See if any of you can answer this. Using daily temperature data of the max, min and mean, what calculation can be used to deturmine when winter ends and begins?
This is related to the article but first…
_________________________________________________
Has anyone else looked at this article from NOAA?
According to our current instruments, we know that, on average, Earth receives about 1,365 W/m2 in total solar irradiance. Piecing together clues from proxy datasets such as tree rings, scientists have evidence that this value has increased slightly over the last century. However, this increase accounts for less than 10 percent of the warming our world has experienced over the same period. Thus, the increase in total solar irradiance alone cannot account for all of the global warming observed since 1900.
http://www.climatewatch.noaa.gov/2009/articles/climate-change-incoming-sunlight
After Anthony’s blog on the new Climate.gov site I ran across that and was astounded that there was, in fact, an admitted up-trend in the output of the sun. Just out of curiosity, does anyone know why this graph ends in 2000?
From the NOAA graph, just eyeballing trough to trough, the delta from 1880 to 2000 looks like it’s about 1 W/m2 (close to 2/3 the IPCC forcing attributed to CO2 here: http://upload.wikimedia.org/wikipedia/commons/b/bb/Radiative-forcings.svg), and that the solar output increased and stabilized at the new higher level around 1950.
I’m having a little trouble reconciling a 1 W/m2 change from 1880 to 2000 indicated by NOAA, with the .15-ish value assigned to it by the IPCC (which baselines pre-Industrial as 1750) unless solar output was significantly higher in 1750 than it was in the 1880’s. Could anyone set me straight on this?
_________________________________________________
OK, now that I digressed for a sec: I’d be very curious about how significant the increase in solar output is when used against the rural or satellite records
I’d also be curious to see what T. J. Nelson and Ted Ladewski’s calculations (http://www.brneurosci.org/co2.html) on climate sensitivity would look like using the rural record instead of “0.57 degrees C (using the value cited by Al Gore and others)” between 1900 and 2000 – both with and without the solar output change factored in.
Jeff Id (10:48:54) :
“You and I could sit in a room and disagree about the color of the walls. It would not be my fault either. You can’t be serious in claiming gridding data compensates for bias in warming? Somehow, I’m sure you are though.”
I claim no such thing – you are picking an argument where there is none. Gridding as a procedure is used to produce a uniform dataset from nonuniformly distributed station data – it is not a procedure to compensate for UHI. I wrote “gridded temperatures” just because this is the dataset that CRU and GISS produce.
“I don’t … believe it’s as large as these curves show. Urban warming effects also apply to many rural stations.”
So how then do you explain the good agreement between the trends of ground and satellite data since 1979?
Easy. CO2 goes on summer vacation every year ….
Great post and great analysis. This is what I’ve been waiting to see. Cheers
Tom the difference between ground and sat is statistically significant. The ground data is biased high or the sat data low.
Lucy
I was AGREEING with your earlier comment abour undetected UHI whilst pointing out to Jeff that most data is highly suspect as we don’t know its provenance. Stations move. UHI affects the data. There are ‘known unknowns.’ It means we end up inadvertently comparing apples and oranges as the micro climate currently being measured is not the same as the one we started with.
Ideally-if not realistically-all data points need to be checked.
Tonyb
Some people have asked about how stations are categorized as rural or urban. I don’t know the answer to that but a better way is to grade the stations, rather than have a binary classification.
McKitrick and Michaels have a paper published – JOURNAL OF GEOPHYSICAL RESEARCH, VOL. 112, D24S09, doi:10.1029/2007JD008465, 2007
– which compared temperature records to economic statistics for the area in question – as a proxy for urbanization. They found a strong correlation between temperature and urbanization.
“Local land surface modification and variations in data quality affect temperature trends in surface-measured data. Such effects are considered extraneous for the purpose of measuring climate change, and providers of climate data must develop adjustments to filter them out. If done correctly, temperature trends in climate data should be uncorrelated with socioeconomic variables that determine these extraneous factors.”
“We conclude that the data contamination likely leads to an overstatement of actual trends over land. Using the regression model to filter the extraneous, nonclimatic effects reduces the estimated 1980–2002 global average temperature trend over land by about half.”
To hoots of derision from the IPCC marketing department, once again demonstrating that “peer-reviewed literature” isn’t really the important issue.
Later Ren et al published a paper about China which demonstrated the UHI effect was about 38% of the stated Chinese warming.
Ren, G., Zhou, Y., Chu, Z., Zhou, J., Zhang, A., Guo, J. and Liu, X. 2008. Urbanization effects on observed surface air temperature trends in north China. Journal of Climate 21: 1333-1348.
So, quite amusing to find that Phil Jones has subsequently published a paper:
Jones, P. D., D. H. Lister, and Q. Li (2008), Urbanization effects in large-scale temperature records, with an emphasis on China, J. Geophys. Res., 113, D16122, doi:10.1029/2008JD009916
– which demonstrates large UHI effects, probably a necessity after his “seminal” 1990 paper with Wang was shown to be “flawed”. His 1990 paper had a large dependence on Chinese “data”, well fictitious data, well allegedly fictitious, but let’s not go there.
His early paper along with his control of the IPCC process on global temperature records has led to the UHI effect being effectively ignored.
So it will be fascinating to read the IPCC report in 2011.
By the way, the IPCC marketing department, realclimate.org, don’t seem to have done a post on the errors in Ren at al. (I couldn’t find one in a search). This leads me to conclude that they attacked the McKitrick paper because it was written by McKitrick – or, once Ren et al was in progress Phil Jones had told them he was soon to publish a recantation of his 1990 – 2007 viewpoint.
And a disclaimer that I have cherry picked 3 papers which show a large UHI effect. There are plenty of peer-reviewed papers which don’t.
But fascinating that Phil Jones is now in the “skeptics” camp! Bring on IPCC 2011!
Jeff Id (14:22:56) :
“Tom the difference between ground and sat is statistically significant. The ground data is biased high or the sat data low.”
There’s no statistical difference in the warming trends:
http://img85.imageshack.us/img85/2432/rssvscru.png
and hence no evidence for UHI bias.
Does anyone have any sources for ocean temp data? Rural and Urban trends are all very interesting but you’re only talking 30 percent of the planet.
[woodfortrees.org has HADsst2GL for download – RT mod]
There is some published literature on many of the points being discussed here.
For those interested some scientists to google are:
Eugenia Kalnay
De Laat & Maurellis
Happy Hunting