Posted by Jeff Id on January 5, 2010
So I’ve learned a great deal playing around with the GHCN data, I think this is a reasonably significant post. Ya know, it’s hard to know anything until you try it yourself and I hope more of the readers here will. Again, there was a problem in my last CRU post, however, the more I look for it, the more avenues there are to explore. The issues have been corrected by avoiding the remaining possibilities in this post.
Of all the details worked out over the last two days, one is a decent gridded average of temperature data. Unfortunately for us skeptics it looks like Figure 1 which is pretty similar to the CRU plot.
Figure 1 – all data gridded
Yes, there is warming according to our temp stations, but I don’t think comrade Phil Climategate Jones would like this curve, because the warming in this curve happens entirely after 1975.
It’s nice to see a good quality CRU similar curve after the previous effort, but that’s how things happen when you do your work in public. The plot above uses all the data with each 5 digit temp station code averaged together individually, as my first post did. Anomaly is calculated over the entire series length.
The concern which was explored in some detail, regarded the hypothesis that the loss of stations in recent years created or biased the trend. It came about since so many stations are lost in recent years as Ken Fritsch pointed out in the recent CRU #3 thread.
Figure 2 – Stations per year GHCN per Ken Fritsch from KNMI database
I’ve run dozens of plots over the last several days, some of which contained an error in them created from data selection or a code problem in my previous post. Using the algorithm which averages together individual station ID numbers, I get very consistent CRUesque patterns. the warming is common to a variety of data sorting processes. This methods avoids the issues of data selection or code problems in the other methods and I’m confident in the accuracy of these results, but you should check them.
Several methods were employed to test the consistencey of result, including sorting for Rural and Urban, and sorting for several different time lengths of station data. All varieties so far produced very similar same results. There are, however, interesting revelations from examination of the slight differences.
Figure 3 – 100% Urban Data
Figure 3 is a plot is the urban data only. Of note is that the warming starts at 1978 with only slight warming beforehand and launches up about 1.2 C with no end in sight. Also, 1982 isn’t much reduced from around 1940 which is different from the global average in Figure 1. So the next thing I did was to plot the rural data.
Figure 4 – Rural station data.
That looks a great deal more like the satellite data. The temp rose and fell again prior to 1978 and rose again since 1978 is maybe 0.5C total. I tend to ignore data prior to 1900 due to the very small number of stations. I don’t think the drop in temps to 1900 levels in the early 70’s is the kind of curve that supports the high CO2 sensitivity claimed by climate science. Does anyone remember the snow storms of the early 70’s? Yeah, yeah just weather, I know.
One of the other avenues explored at great length , yet still isn’t finished, was how station starts and stops affect the trend in recent years. To explore that, one of the several methods I used was to sort data according to number of available data points. Below, I presented the gridded global average for all stations with at least 100 years (1200 points) of available data, since many of the stations in Figure 2 were started in 1950.
Figure 5 – Urban gridded data from stations with at least 1200 points
The urban data only in Figure 5 has an even steeper curve, you would expect this from longer series in this type of analyis. The temp rise since 1978 is about 1.2C. The rural 100 year curve is below.
Figure 6 Rural temperature stations with at least 1200 points
So the Rural stations show about 0.7C of warming since 1978. Visibly less warming than the urban stations by themselves. Also note the slight downtrend in recent years. Since the industrial revolution occurred a hundred years ago, it’s hard to imagine this curve is created by CO2. Still I’m not denying the heat capturing ability of CO2, just that the curves here don’t show a continuous warming but rather a short term recent spike.
So of course we should look at the difference between urban and rural stations.
Figure 7 – Difference between urban and rural data as identified by GHCN
Figure 8 – Difference between urban and rural data from GHCN stations with at least 100 yrs of data (1200 monthly points)
Look at that curve! Despite the crudeness of the categorization of thermometers, there is a clear warming bias for big city data. The curve in Figure 8 ends at 0.6C difference. What’s more, the trend between the two looks statistically significant. If Phil Climategate Jones and Michael Marx Mann can choose which data they want to show and hide the rest, I think it’s only fair to choose to look at trends only from Figure 8 since 1978 (even though it won’t make much difference). After all, one hundred percent of global warming has apparently occurred since that time. Let’s do a simple significance test.
Figure 9 – Statistical significance between rural and urban stations
Woah, it’s not even close, a trend of 0.12 and a no trend null hypothesis limit of +0.04. The difference between urban and rural warming is as great as the entire trend in UAH data over the same timeperiod.
Just how much trend do the ground stations show.
Figure 10 – All urban stations different Y scale from 11
Figure 11 – All rural stations different Y scale from 10
Even Figure 11 is still greater than UAH and RSS satellite data but it’s one heck of a lot less than the urban stations. Of course we would be remiss to not mention that WUWT has taught us what rural stations often look like.
What could go wrong with sophisticated technology like that?
The R code for this post is here.
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Of course, that is a useful observation, because there are obviously times when the urban stations appear colder than rural stations, and one way to explain this is through inversions in valleys where these cities sit. However, the UHI I think is also complicated by heated environments. A day or so ago I posted an observation that our campus, which is at the edge of the city, was about 4F warmer than my home which is about 4 miles east of the city. There was no “sky-view” effect (which many people insist is the bulk of UHI) because the sky was heavily overcast and snowing. I suspect the heated campus buildings and utility vaults explains this.
One question that I never see answered is –
‘is the concept of an average valid when applied to temperatures measured at different stations?’
Personally I believe the answer is no. What is the average temperature of Australia? The entire globe? To me it is about as valid as stating the average skin colour of humans. You are not in fact comparing the same data. You can only give an average of dependent values. The temperature for a station in the Antarctic is totally independent of the temperature of a station in the tropics. Even when you start talking about the average temperature of a single station over a period of a year you may not even be comparing the same thing.
Genghis:
And you will get a discontinuity in the temperature anomaly when the wall got put in place. If you have an algorithm for detecting these infrequent changes in local environment, the effect of the environmental change would subtract out to first order.
I’m not sure I understand your measurements here.
Are you measuring the surface temperature of the roof or the air above the roof? And if you did, did you have a radiation shield on your thermometer? Because surface radiation can interfere with your measurement in situations like this.
In the day time, the surface temperature of the black roof should be much warmer than the white surface. And if it is a black body radiator, its surface temperature should be cooler at night, not warmer.
I’m thinking it’s a bit more complex than this. The chlorophyl in the plants absorbs most of the infrared radiation from the Sun, so in the case of short grass, it will actually raise the temperature.
You do get cooler temperatures inside of a tall forest..but the explanation is more complex than just how much heat is being absorbed and reradiated by the surface of the plant.
Tom P:
I’ll note your “proof” contained no confidence intervals. That’s kind of sloppy on your part to make an assertion based on a graph with no uncertainty information.
In fact the trends are quite different. I get for the trends, 1.26°C/century UAH and 1.57°C/century HadCRU3v.
Subtracting the two two data sets (this operation removes a lot of common correlations in the separate data sets) and computing the trend + error, I get 0.31°±0.12°C/century (p < 0.01). Even inflating the uncertainties well above their admitted values (0.4° measurement uncertainties), I still get p < 0.1.
Anyway, even if there was no UHI, there's reason to expect satellite measurements (which don't include the boundary layer) to be different than measurements taken inside the boundary layer
I am no kind of expert but I think it is important to remember the point made by Jo Nova, http://joannenova.com.au/global-warming/, There is a lot of evidence about warming but very little evidence that mankind’s contributions are causing it. Again, not an expert, but I remember learning long ago that correlation is not causation.
Tom P (14:39:24) :
I learned something with this point. RSS is a very good trend match to CRU for the recent 30 years. Of course it still has the huge ugly step in the center of the timeseries which makes it generally inferior to UAH in my opinion. Christy demonstrated using sonde data that the step favors UAH, I independently came to the same conclusion using ground data. When the step is corrected, the curves match almost perfectly.
http://noconsensus.wordpress.com/2009/01/19/satellite-temp-homoginization-using-giss/
My point should have been that CRU has a statistically significant positive bias when compared to UAH satellite data. Hence, evidence for urban warming bias.
USA Today has a particularly un-academic (and very poorly argued) guest editorial today from Orrin H. Pilkey and Rob Young in North Carolina. Bottom line? We don’t need no stinkin’ measurements to know that it’s getting warm, and give us more money so we can convince the public better! Unbelievable gall.
http://blogs.usatoday.com/oped/2010/01/column-doubt-global-warming-the-planet-wont-tell-a-lie-.html
Maybe earlier than “now”; maybe around 2002, when temperatures began plateauing.
Broaching.
If it gets any colder they are going to have to site stations inside people’s homes to get the ‘right’ results!
TonyB (09:42:44) :
Jeff Id and Lucy Skywalker
Temperature information only means something if we are comparing like for like. In this respect we should always bear in mind that Thermometers are designed purely to measure the micro climate immediately around them.
And to inform the surrounding community of day to day temperature movements. A favorite ‘sport’ on our local ABC radio in the morning is to bait the duty forecaster about the previous days forecasting errors, chatting about max/min is obligatory (“wasn’t it warm last night”). I guess it makes sense that if you live in a city you are interested in the max/min’s of that city overnight. Your world is that city environment so the UHI is meaningful to you as an individual. I am saying this as there seems to be a disconnect between local air temperature that is measured at human height for local people and the ageing and extrapolation for climate purposes. One side says ‘so what’ about UHI , ‘this city is where I live and I want to know the temperature” and the other employs statistical gymnastics trying to remove UHI to create global stats. The global estimates here seem to be the looser.
Just a thought, if the aim is get a meaningful record of land insolation, would not the lowest maximum (not a max/min average) in a gridded area be the best estimate ?. A higher maximum may reflect local heating anomalies and the night minimum’s are just a re-radiation of the daily IR intake and as has been discussed,subject to local distortion. If the aim is record solar heating for climate purposes,the daily maximum only would be a better estimate that the current average.
Kent McQueen, wow, amazing. I think we may be on to something. Agazzis BC is showing the same trend. I’ll likely be hitting the same AB stations soon. Thanks.
Anthony, we need to pursue this much further than what I’m doing. Please contact me.
For those wanting to plot station data and compare GHCN and HadCRU stations as well as the HadCRU 5×5 grid, there is a new map-based graphing interface at http://www.appinsys.com/GlobalWarming/climate.aspx – click the Map Interface For Graphing button
GeneDoc:
Well a couple of points here. We know from other observations besides surface temperature measurements that it’s been warming for about 150 years. The climate models themselves suggest that until 1980 all of the warming was natural (anthropogenic CO2 was balanced by sulfate emissions). Again temperature measurements not needed there.
Finally, if we are going to take climate change seriously enough to discuss radically changing our economic system, yes more money to help monitor the system would be nice.
The Rural graph (Figure 6) reminds me of the 100+ year trend of the Barents Sea temperature, published last year as a peer-reviewed paper. But I am unable now to identify it. Someone please guide me to the paper?
Thank you Evan for your reply at Stephen (10:28:13) :
But I still have a bad feeling in my gut that there is something wrong with the concept of grids, or averaging or homogenizing, temp data to find a global temp base line. Is there a way to help me passed this block. To me it is like taking a plot of land and pile up dirt on one little spot. Yes we changed the average height of the plot of land, but except for the little plot, the rest of the land remains at the base line. To me, it doesn’t matter what the island effect is doing at its little pile, or even if we spread a little dirt on a few more little surrounding spots, we haven’t yet changed the base line, and as far as the base line is concerned, averaging etc. is meaningless. What I would like to see is all of the “good” rural stations, which show a zero to cooling trend, set to a grid to find out how big our holding and cooling baseline is; and compare that to “good” rural stations that show warming; and also compare those to “good” urban, etc. stations. Until we know what the baseline is doing, the rest is just local! In other words, if our plot of land is actually sinking, spreading the pile of dirt and more, may be a good thing, ha!
Stephen
Carrick:
Yes, there is warming. Yes, it’s unclear to what extent humans are responsible and it might be worth understanding whether and to what extent (I read you to be convinced by the models, I for one, am not, and there are many issues all over this site and others that provide ample latitude for doubt.
I read the USA Today piece to suggest that additional funding was needed to help convince the public that there’s a problem, not to actually study anything (after all, we know already!) There’s an odd claim that it’s not worth doing anything quantitative:
“Why is there so much confusion about whether the planet is warming? We believe a big part of the problem centers on the use of earth surface temperature data as a direct measure of warming. Where do you stick the thermometer? What matters most: Daytime highs? Nighttime lows? Summer or winter temperatures? Trying to determine whether the planet is warming in this fashion seems fraught with peril.
One doesn’t need to measure thousands of temperatures to find conclusive evidence that the planet is warming. The earth does the averaging for us. Many physical and biological characteristics show that the earth is warming and has been for decades.”
OK, so what’s that evidence?
“Glacier National Park in Montana is down to 26 named glaciers from 150 in 1850, and if this trend continues, the park is expected to be ice-free by 2020. Glaciers in the Himalayas are shrinking so rapidly that the summer flow of the major rivers (Indus, Ganges, Mekong, Yellow, Yangtze) they feed might eventually be affected.”
Hmm, glaciers have been retreating since the end of the Little Ice Age. Why wouldn’t the trend continue? Perhaps a rigorous, quantitative analysis would be informative? And they cite the now debunked alarm about the Himalayan glaciers.
“The rate of rise in sea level has sped up over the past century. A tide gauge on a pier in Duck, N.C., says the sea level is rising at 1.5 feet a century. The Arctic Ocean’s ice cover is shrinking and could disappear, endangering animals that depend on the ice for their survival.”
One tide gauge is evidence of global sea level rise of 0.5m/100 years! (off by an order of magnitude or two–and probably local subsidence). Arctic sea ice is back since 2007 and will likely be above average levels after this winter.
“One can argue for hours whether this year was warmer or colder than last. To us, it doesn’t matter. We should be reading the earth, not thermometers. The earth is clearly warming, and sea level is clearly rising.”
The science is settled. No use arguing. Don’t bother us with numbers. To me, this is “un-academic”. If you can’t quantify it, it’s not science. Rates matter. Whether it’s out of historical bounds matters.
“In order to convince a skeptical public that global change is real, scientists and funding agencies need to invest more in field measurements and monitoring rather than computer modeled predictions.”
I’m not sure how this is meant to be interpreted. My take was: More measurements are needed to convince the skeptical public, even though we don’t need any more convincing…but we’ll take your money. Maybe you see this differently, but I find it offensive.
“As for the skeptics, it’s time to get off the atmospheric temperature kick and read the earth. We also believe the science clearly indicates that humans are playing a new, and critical role in driving that warming. But, lack of clarity in the exact degree that humans are causing global warming should not be used as an excuse to ignore the monumental changes that rising sea level and changing climate will bring to the planet, and our society.”
Belief is not a concept that is helpful in science. Maybe it’s ok in climate science, but it’s a word that I forbid in my laboratory. And I encourage skepticism about data. This article takes the opposite tack: we know and we don’t appreciate anybody who’s skeptical. So what if the data are messy? We know the answer and don’t need to be bothered by problems with the data. But we’ll take more money because that will improve our proselytizing to convert the skeptics…Lack of clarity of the degree…again, don’t bother us with quantitative aspects. No one’s going to ignore sea level rises when they occur. I thought the argument was about whether there’s anything to be done about it and how soon it might occur. But of course that requires numbers…
In sum, I found this piece poorly argued, with blatantly false or misleading statements and insulting to the intelligence of the reader/taxpayer.
I think these results have not been interpreted correctly in this thread.
The rural data is land only ! Land only temperatures have warmed by a factor of 2 more than ocean temperatures since 1980.
As land covers 30% of the earth’s surface, land temperature increase should be about 1.5 times higher than land + ocean combined.
If instead this rural land only data matches well with HadCrut land + ocean data, there must be something wrong !
If we believe this rural data is correct, HadCrut land + ocean data should overstate warming by a factor of 1.5.
Actually, this matches well with what we expect from the comparison with UAH satelite data::
Here, HadCrut trend is approx. 20% above the UAH satellite data trend.
However, physics and climate models predict, that the troposphere should warm faster than the surface by a few ten percent.
In sum it can be concluded, that the HadCrut trend overstates warming by a factor of approx. 1.5.
This is still
Genedoc:
No, not really. I think they are pretty much broken actually.
To be clear, I do think that the physics of CO2 as a greenhouse gas is well settled (but that only gives about a 1.2°C/doubling of CO2), but I don’t think the rest of the physics is fully understood (water vapor feedback loop that supposedly enhances the CO2 feedback to as much as 6°C/doubling of CO2, effect of clouds etc). Nor do I think the models adequately capture the short-term fluctuations of climate (e.g. periods less than 10 years, a point agreed to by most modelers as far as I can tell), however, I have doubts it can capture all of the important long-term fluctuations either.
So in my opinion, the models get the physics of CO2 sensitivity wrong and completely misses most of the rest of regional scale climate fluctuations like the Arctic Oscillation that is wrecking havoc. On the up side, they do make produce some nice animations, great for PowerPoint slides to pitch for more funding. So there is that!
Carrick (16:26:41), Jeff Id (16:40:21) :
Both of your points rest on UAH and RSS have a different trend, 1.27 vs 1.53 C/century. There are three points here:
1. Is the difference in trends statistically significant? Any calculation which ignores serial correlation, such as Carrick’s, will overestimate the significance. You need to do the complete calculation based on an appropriate correlation model to show significance here – I very much doubt there is any based on a plot of the residuals, but I’m willing to be proved wrong.
2. Is the difference scientifically significant? This is a 20% difference in trends, not an important difference given the instrumental uncertainties. Here’s John Christ of UAH himself on the difference:
“When global trends are compared for 70S-85N (RSS domain) the difference [between UAH and RSS] is only 0.02 C/decade and is getting closer as the relative warm shift of RSS in 1992 is being mitigated by the relative cooler drift over the NOAA-15 period. So we are looking at relatively small difference issues in the larger context.”
3. Is the difference relevant to UHI? Obviously the difference in the trends is to do with the methodologies RSS and UAH and nothing to do with UHI. Hence attributing the satellite differences to UHI is incorrect.
Manfred (23:24:22) :
“If instead this rural land only data matches well with HadCrut land + ocean data, there must be something wrong !”
But it doesn’t match well. Here are the two plots, Jeff’s in black, CRU in red, offset to show the different recent trends:
http://img340.imageshack.us/img340/5856/cruvsruralid.png
In fact the difference in the slopes looks like it’s a little more than 1.5 – Jeff could give a more accurate figure for the period of 1978 to present. Hence, taking into account the ocean data as you quite rightly suggest, Jeff’s work is in good agreement with the CRU global average.
The thing that really jumps out at me when looking at any of these graphs, is just how small the scale is. We’re talking half degrees, which are dwarfed by daily, not to mention seasonal temperature ranges.
Urban Bias
I live in village in the UK that lies to the north of a railway line. The railway line is on a raised embankment for much of its path through the village and all the traffic passes through one short narrow tunnel under the line.
When I was cycling to work regularly through this tunnel, it was noticeable that if there was frost on the south side of the tunnel there was no frost on the north side, this before the sun had risen. So if a temperature sensor had been moved just 45 feet from one side of the tunnel to the other the record would have shown quite a different temperature profile.
Tom P (02:12:17) :
I checked it before I posted my last comment. UAH is statistically significantly different from CRU based on a Santer style lag 1 difference.
Tom, you keep saying this is in good agreement with the global average, this is not a gridded global dataset presented here. This is gridded LAND ONLY data which is not even averaged by individual hemisphere leaving the southern nations underrepresented.
The trend drops substantially when you do a north south average CRU style, but even that leaves the oceans unmeasured. It’s premature to look at trends in this data as global trends as the station weightings are not appropriate.
The purpose of this post was to demonstrate the urban warming bias, nothing more. I’m working toward a better global average but would like to improve the step detection algorithm and individual station ID number averaging before moving on.
Some Guy (02:33:56) :
I agree, none of these plots is terribly scary. Copenhagen was scary, cap and trade— scary. A little warming– kinda nice.
Jeff Id (05:53:26) :
“UAH is statistically significantly different from CRU based on a Santer style lag 1 difference.”
Wouldn’t the correlation be expected to reach far beyond just the next month of data – after all there is a seasonal signal in the series. If so, might restricting to lag 1 substantially overestimate the significance.
“… you keep saying this is in good agreement with the global average…”
Please see my comments to Manfred above – your rural data shows a larger warming trend than CRU global data.
“…none of these plots is terribly scary… little warming– kinda nice.”
Warming of about one degree is seen in the last thirty years. That’s much less than the total warming of 8 C from the last ice age 12,000 years ago. But there’s certainly a cause for concern if the warming trend continues at this rate – that wouldn’t be so nice.
As for crying over glacier melt, as long as the average sea level temp of the globe is 59 deg F, they will continue to melt, even without warming. The only way to get the glaciers back, is to reduce the average global temp to below freezing. And, you may get your wish… See:
http://www.c3headlines.com/2010/01/satellite-confirms-that-global-temps-continue-decline-trend-a-minus-151f-per-century-rate.html
And if your really scared about today’s melting, take another look at:
http://wattsupwiththat.com/2010/01/03/swiss-eth-glaciers-melted-in-the-1940s-faster-than-today/
Stephen
“”Carrick (00:56:00) :
Genedoc:
I read you to be convinced by the models…
No, not really. I think they are pretty much broken actually.
To be clear, I do think that the physics of CO2 as a greenhouse gas is well settled (but that only gives about a 1.2°C/doubling of CO2), but I don’t think the rest of the physics is fully understood (water vapor feedback loop that supposedly enhances the CO2 feedback to as much as 6°C/doubling of CO2, effect of clouds etc). Nor do I think the models adequately capture the short-term fluctuations of climate (e.g. periods less than 10 years, a point agreed to by most modelers as far as I can tell), however, I have doubts it can capture all of the important long-term fluctuations either.
So in my opinion, the models get the physics of CO2 sensitivity wrong and completely misses most of the rest of regional scale climate fluctuations like the Arctic Oscillation that is wrecking havoc. On the up side, they do make produce some nice animations, great for PowerPoint slides to pitch for more funding. So there is that!
“”
Your co2 sensitivity is still a bit high. However, absolute humidity variations with temperature really tend to quash the hypersensitivity of h2o vapor. Increasing temperature by 5 deg. C while assuming relative humidity stays constant (a common climatology assumption) results in less forcing due to h2o than is due to the co2 doubling. It would add about 1.3x more h2o which is a long way from an h2o doubling. The result is that the co2 and h2o could not cause even half the warming necessary to raise the T by 5 degrees. If we assumed the T increased by 2 deg. C, the increase in h2o vapor would be somewhat less than a 5 deg. C warming could cause. Again, though, the total warming caused by the co2 doubling and corresponding h2o vapor increase would amount to hardly more than the co2 warming by itself. The other factors which are ignored here are negative feedback and so will reduce the effect further. While the h2o attenuation effect is over twice that of co2, the limitation that absolute humidity with temperature places upon its growth gives one some serious limits as to how big an effect could be had.
Of course, the notion that a 1 deg C rise could generate a 5 deg C rise is simply a runaway condition that could never exist due to a total lack of stability. The absolute humidity limitation shows it cannot be the case.
This is based on the fundamental physics of radiative transfer in the atmosphere. It suggests that the complex models that do promote high sensitivity due to h2o feedback must be highly flawed and may not even based upon the actual physics.