In case you missed it, Roy Spencer performed a unique and valuable analysis comparing International Hourly Surface data to population density to provide a simple gauge for the Urban Heat Island (UHI) effect. It was presented at WUWT yesterday with this result:
There were lots of questions on the method. Dr. Spencer adds to the discussion below.
===========================================
UPDATE #2: Clarifications and answers to questions
After sifting through the 212 comments posted in the last 12 hours at Anthony Watts’ site, I thought I would answer those concerns that seemed most relevant.
Many of the questions and objections posted there were actually answered by others peoples’ posts — see especially the 2 comments by Jim Clarke at time stamps 18:23:56 & 01:32:40. Clearly, Jim understood what I did, why I did it, and phrased the explanations even better than I could have.
Some readers were left confused since my posting was necessarily greatly simplified; the level of detail for a journal submission would increase by about a factor of ten. I appreciate all the input, which has helped clarify my thinking.
RATIONALE FOR THE STUDY
While it might not have been obvious, I am trying to come up with a quantitative method for correcting past temperature measurements for the localized warming effects due to the urban heat island (UHI) effect. I am generally including in the “UHI effect” any replacement of natural vegetation by manmade surfaces, structures and active sources of heat. I don’t want to argue about terminology, just keep things simple.
For instance, the addition of an outbuilding and a sidewalk next to an otherwise naturally-vegetated thermometer site would be considered UHI-contaminated. (As Roger Pielke, Sr., has repeatedly pointed out, changes in land use, without the addition of manmade surfaces and structures, can also cause temperature changes. I consider this to be a much more difficult influence to correct for in the global thermometer data.)
The UHI effect leads to a spurious warming signal which, even though only local, has been given global significance by some experts. Many of us believe that as much as 50% (or more) of the “global warming” signal in the thermometer data could actually be from local UHI effects. The IPCC community, in contrast, appears to believe that the thermometer record has not been substantially contaminated.
Unless someone quantitatively demonstrates that there is a significant UHI signal in the global thermometer data, the IPCC can claim that global temperature trends are not substantially contaminated by such effects.
If there were sufficient thermometer data scattered around the world that are unaffected by UHI effects, then we could simply throw away all of the contaminated data. A couple of people wondered why this is not done. I believe that there is not enough uncontaminated data to do this, which means we must find some way of correcting for UHI effects that exist in most of the thermometer data — preferably extending back 100 years or more.
Since population data is one of the few pieces of information that we have long term records for, it makes sense to determine if we can quantify the UHI effect based upon population data. My post introduces a simple method for doing that, based upon the analysis of global thermometer and population density data for a single year, 2000. The analysis needs to be done for other years as well, but the high-resolution population density data only extends back to 1990.
Admittedly, if we had good long-term records of some other variable that was more closely related to UHI, then we could use that instead. But the purpose here is not to find the best way to estimate the magnitude of TODAY’S UHI effect, but to find a practical way to correct PAST thermometer data. What I posted was the first step in that direction.
Clearly, satellite surveys of land use change in the last 10 or 20 years are not going to allow you to extend a method back to 1900. Population data, though, ARE available (although of arguable quality). But no method will be perfect, and all possible methods should be investigated.
STATION PAIRING
My goal is to quantify how much of a UHI temperature rise occurs, on average, for any population density, compared to a population density of zero. We can not do this directly because that would require a zero-population temperature measurement near every populated temperature measurement location. So, we must do it in a piecewise fashion.
For every closely-spaced station pair in the world, we can compare the temperature difference between the 2 stations to the population density difference between the two station locations. Using station pairs is easily programmable on a computer, allowing the approx 10,000 temperature measurements sites to be processed relatively quickly.
Using a simple example to introduce the concept, theoretically one could compute:
1) how much average UHI warming occurs from going from 0 to 20 people per sq. km, then
2) the average warming going from 20 to 50 people per sq. km, then
3) the average warming going from 50 to 100 people per. sq. km,
etc.
If you can compute all of these separate statistics, we can determine how the UHI effect varies with population density going from 0 to the highest population densities.
Unfortunately, the populations of any 2 closely-spaced stations will be highly variable, not neatly ordered like this simple example. We need some way of handling the fact that stations do NOT have population densities exactly at 0, 20, 100 (etc.) persons per sq. km., but can have ANY population density. I handle this problem by doing averaging in specific population intervals.
For each pair of closely spaced stations, if the higher-population station is in population interval #3, and the lower population station is in population interval #1, I put that station pair’s year-average temperature difference in a 2-dimensional (interval#3, interval#1) population “bin” for later averaging.
Not only is the average temperature difference computed for all station pairs falling in each population bin, but also computed are the average populations in those bins. We will need those statistics later for our calculations of how temperature increases with population density.
Note that we can even compute the temperature difference between stations in the SAME population bin, as long as we keep track of which one has the higher population and which has the lower population. If the population densities for a pair of stations are exactly the same, we do not include that pair in the averaging.
The fact that the greatest warming RATE is observed at the lowest population densities is not a new finding. My comment that the greatest amount of spurious warming might therefore occur at the rural (rather than urban) sites, as a couple of people pointed out, presumes that rural sites tend to increase in population over the years. This might not be the case for most rural sites.
Also, as some pointed out, the UHI warming will vary with time of day, season, geography, wind conditions, etc. These are all mixed in together in my averages. But the fact that a UHI signal clearly exists without any correction for these other effects means that the global warming over the last 100 years measured using daily max/min temperature data has likely been overestimated. This is an important starting point, and its large-scale, big-picture approach complements the kind of individual-station surveys that Anthony Watts has been performing.
There is a huge exclusion zone around Chenobal, there must have been thermometers in the exclusion zone before and after the reactor did the big firework. This would give you the UHI backwards, as the population left, vegetation returned.
debreuil (14:30:03) :
“I think it might also be useful to go by economic output rather than population”
Why not the amount of KwH delivered by energy company’s. They should keep records of that. Gr. M.
Jim (14:41:26) :
There are enormous methodological problems with that assertion. The most obvious is defining “well placed.” Ideally, “well placed” would mean located away from any human induced environmental alterations that alter regional “climate” as reflected in thermometer data. The problem with that idea is that until that advent of electrical methods of data recording and transmission, thermometers must be located where they can be accessed by a human reader daily. That means that for all older data, no thermometer is or could be located away from some source of UHI as defined by Dr. Spencer. In fact the “best” will be located in situations where the effect is greatest (0-10 people per square kilometer). I would speculate that the manner in which Dr. Spencer’s curve approaches a linear trend after population density reaches about 250 people/sq. km. (about one person per 0.4 hectares) is because no new agricultural changes to vegetation are likely. After that the effects are primarily due to increasing urban changes – the development of market and manufacturing centers and other large population aggregates that are not agriculturally based.
If an agreement (consensus) could be reached concerning what “well placed” implied, there is still a problem concerning what an adequate sample period would be. The climate is a natural system and is 100s of millions of years old. If 100 years of data were good enough to sort out natural trends in the climate, there are numerous data sets that span at least that range. It seems pretty clear though, even among the AGW school, that 100 years is not regarded as an adequate sampling period. There is a sound reason for the search for temperature proxies, even if the selection of such proxies has heretofore in some cases been pretty questionable.
Nor is there anything like a real consensus on just how to define “climate.” There is broad agreement that it is not weather, but just what does comprise climate seems to be a very active area of debate – read some or Dr. Pielke, Sr.’s discussions regarding climate for an idea.
Jim (14:41:26) :
“Someone once postulated that 50 well placed thermometers would be adequate to gauge average global temperature trends. Any statisticians care to comment? Are there 50 uncontaminated thermometers that have 100 year records and are somewhat well placed?”
Jim, i would narrow it down to two, one Northern hemisphere one Southern hemisphere mid latitude. Obviously well placed rural.
Everything else is hand waving.
@DocMartyn
“There is a huge exclusion zone around Chenobal, there must have been thermometers in the exclusion zone before and after the reactor did the big firework. This would give you the UHI backwards, as the population left, vegetation returned.”
Another good candidate for measuring the reverse-UHI effect is the Detroit area. Over the past 50 years it has been depopulating nearly as fast as Chernobyl did during the 1980s!
http://upload.wikimedia.org/wikipedia/en/thumb/b/bf/Detroit_population_and_rank.svg/500px-Detroit_population_and_rank.svg.png
I still don’t understand (with some earlier commenters) how a population of 1 – 20 can be considered “Urban”. Further, if it is true that an area with such minimal population has a warmer temperature, this means that most of the global landmass is affected, that is everywhere that humanity has settled. If this is the case, “Urban Heat Island” would have to have both “Urban” and “Island” removed from the label (Urban because the effect is clearly greatest in rural areas, and Island because urban areas are no longer islands under this definition) and we’d be left with, um, “Heat”.
And the ‘spurious’ signal becomes the actual…
Maybe on the whole its better to stick with satellites?
Outstanding work!
Regional differences in construction and coloring will affect the UHI, as well as changes in construction methods. Asphalt roofs have a different thermal spectrum than Bermuda Tile roofs, etc. All of this needs to be included in a detailed analysis to get improved accuracy, but I doubt that doing so will change the post knee portion of the curve significantly.
Brilliant approach, just elegant.
Roy- Does the station warm bias = difference between pairs of stations within 150 km? Why not go with something more descriptive? Also, is the zero point of this graph being set by the curve fit?
Does this also mean that all stations w/in 150km where the population density is higher were warmer than all stations where the population density is lower? If so, I can suggest that mountains have a much, much lower population density and should almost always be colder than the lower-lying cities. Maybe that’s a second-order correction? But how many of those extremely low-density station pairs represent mountain vs. low-lying city?
BTW, that explanation you pre-pended works pretty well.
Given the current emotional set of journal editors, I don’t think your research has a chance to see the light of day. Not unless it is tightened down. You must reduce the unknown variables or else your thesis will be torn to shreds.
Couple comments on this post and the previous one.
First, I think this is a very important piece of work to consider. Once a reasonable and quantitative correction factor for UHI is implemented and a modified temperature dataset presented, the picture of possible AGW will be much clearer (and harder for “warmers” to dismiss if the skeptics are correct).
As others have pointed out, population isn’t the greatest proxy, but I think it’s the only one where data is good enough for the past 100+ years to make an effective UHI proxy. Presumably, the UHI/population density figure changes with both population density AND date. Dr. Spencer – do you think there’s enough population data resolution in older records to estimate the effect from before 1990 (the older stuff is what you’re targeting)? Also, this approach (or something similar) is still useful on current data, IMO.
One note – I’ve gone by the rule of “3 degrees Fahrenheit per 1000 ft” since my professor mentioned it in a college geography class. Thus, I was very happy to see the 5.4 C/km number calculated in your work, which computes to 2.96 F/1000 ft, or 3.0 if we keep track of sig figs ;-).
As for the comment from Jim @ur momisugly 14:41:26, I’ve always thought of something similar…a few excellent temperature readings are far better than masses of bad ones. Assuming that “bad” thermometers are 5 times “noiser” (note that bad ones add systematic errors, not necessarily noise, but the method used to correct for the errors likely introduce the equivalent of noise), then it would take 1250 “bad” thermometers to match 50 “good” ones. Actually finding excellently-placed thermometers or ensuring the data are acceptable is an entirely different, and very difficult, matter (as noted by other commentors).
Well it would seem from reading the posts from Kevin Kilty and others, that there are a lot of people who have never watched a horse opera on television (or the movies) in which the runaway wagon wheels are clearly revolving backwards.
That by itself would seem adequate proof that you cannot expect to get believable results from insufficient data.
The problem is similar to that of the traffic cop, who presents the judge with a picture that clearly shows the defendant’s car on the wrong side of the road, and rotated 90 degrees to the direction of other traffic on the road.
Whereupon the judge slaps the hapless drive with a reckless driving conviction, for being sideways on on the wrong side of the road.
A movie of the actual circumstances; i.e. a data set with enough “thermometers” to reveal what actually happened, would show that the defendant was simply driving through on a cross street, when the cop snapped him “sideways on on the wrong side of the road.”
The general theory of sampled data systems is VERY WELL KNOWN, and in practice IT WORKS because our entire high bandwidth data communications, and telephonic voice and other communication high bandwidth traffic, are entirely dependent on proper sampling of all of those signals.
Those who ignore that theory; to save some thermometers do so at their own risk.
So on a nice hot northern summer midday, in a North African or middle-eastern desert, the ground temperature might be +60 deg C or higher (140 F or 333K). Simultaneously it is the dark of winter midnight at or near Vostok station in Antartica’s highlands, and the temperature as low as -90 deg C (-130 F or 183 K). And due to a famous argument in Galileo’s “Dialog on The two World Systems.” we know that every possible value of temperature between thsoe two extremes will exists somewhere on the planet at the same time ( ok to be pedantic; IF those are the two extremes at the moment).
So now where do we want to put those two thermometers; the NH and SH temperature monitors; and be sure it is well away from urban blight.
You cannot statistically create information where none exists.
Whilst I commend Dr Spencer’s effort and work, mainly because it throws up arguments against using thermometers at all, to “measure” the Earth’s temperature, and throws a bit more doubt on the methodology, a useful holding strategy.
However, I believe that measuring the treeline and measuring the extent of the snowline, throughout all countries including the Southern Hemisphere, is a more valid way of measuring what is happening to the Earth. The response of the vegetation worldwide at the “coalface” of the snowlines, would be equivalent to millions of thermometers.
One of the interesting things I have noted about the debate over the trees showing a cooling trend opposing the thermometer record, which led to the whole “hiding the decline” debacle, is the complete absence of anyone standing up for the trees! What if the trees WERE correct and the thermometers were wrong?
The trees have a “dog in the fight” in that if they don’t adapt and quickly, they die. The thermometers, however, by themselves as inanimate, man made items, can be wrong and nothing happens to them. ( I believe that each and every mercury thermometer ever made, has an inbuilt error factor of +-1 degree, which is greater than the amount of warming per century supposedly measured by them.)
(By the way, OT but with reference to Australia, you may not be aware that Western Queensland is having widespread floods through the Channel Country, which are breaking records going back over 100 years in many towns.)
Dr Spencer – Dr Simon Torok did a similar kind of calculation back in 1999
http://reg.bom.gov.au/amm/docs/2001/torok_hres.pdf
Urban heat island features
of southeast Australian towns
Simon J. Torok and Christopher J.G. Morris
School of Earth Sciences, University of Melbourne, Australia
and
Carol Skinner and Neil Plummer
National Climate Centre, Bureau of Meteorology, Australia
(Manuscript received December 1998; revised June 2000)
Sorry the link is here for the flood information http://www.couriermail.com.au/news/st-george-charleville-brace-for-more-rain/story-e6freon6-1225837127753
UPDATE 11:30AM WEARY Charleville residents confront their third flood in five days as St George prepares for worst flood in 120 years with predictions of more rain
George E. Smith (14:53:17)
UHIs do tend to get hotter than the average landscape they used to be, so it is proper to measure them.
The error comes in trying to extend the influence of the measured UHI far beyond its real sphere of influence.
That to me seems to be an error in methodology; and not something which calls for “correction” of the UHI measurment. The correction called for is in curbing the radius of influence assigned to the UHI; not in changing its value.
I really do enjoy, appreciate, and second this point. From a temperature record standpoint, UHIs should be seen as blips on a map (maybe something like this) – not giant blobs… and I think that from a climate modeling standpoint, UHI should be seen as a local forcing not something that needs to be corrected out
Well said!
A very interesting article and some great debate – perhaps a role model for a model open peer review process for future climate debate discussion.
On a slight tangent, Dr Spencer, as described, the methodology you used involved the averaging of the hourly temperature data for each station to calculate mean average temperature for that station. My understanding of the average daily temperatures generally used in historical analysis of temperature trends, including climate models, are actually the mean of the maximum and minimum temperatures recorded for that day, not an average of hourly measurements (presumably because the historical record is based on measurements recorded using those old max-min thermometers I vaguely remember from my long distant schooling). As its looks like you already have the data, I am curious as to how closely the hourly computed average daily temperature correlates with the min-max average and, if there are systematic differences, what implications/caveats it has for assuming changes of the min-max average over time are the same as changes in the average temperature.
George E. Smith,
If the temperature measurements really do resemble 1/f noise, well, I know of a guy who has a Fortran program that can convert it into a hockey stick. ^_^
Back to Dr. Spencer’s post, it tells me that none of the methods being used to account for UHI account for all of it, because the rural stations are also being affected (Rural Heat Reefs?) .
We’ve probably encountered this problem before, with some cave men during the last ice age arguing that the existing climate data sets included too many measurements from inside their caves, while others argued that rural stations were unreliable because of station moves due to advancing and retreating glaciers.
latitude (14:56:35) :
Latitude has it right. There is no good reason to resqueeze rotten fruit in the hopes that the juice will come out fresh.
Further, we won’t know if there are enough “clean rural sites” until we look for them. As the SPPI report suggests, they are clearly not as dirty as the urban sites, and as our work is begining to show, there probably will be enough to fill a 5×5 grid in the US.
I believe that in quite a few places you have written “population” where you may meant “population density”. Saying “population” makes one think you are talking about absolute population levels not densities.
A more substantive question is — why not just estimate a multivariate regression with population density and station elevation and distance from water as explanatory variables? In fact, you could use log(population density) to allow for the non-linearity and perhaps also higher order terms such as (log(population density))^2. The coefficients on the population density variable(s) would then give you the UHI correction you seek.
I can see one advantage of the pairing idea is that two paired stations might have a common unmeasured factor affecting their temperatures. Looking at the difference then cancels the common error. Even if you added other control variables, such as climate zone indicators, you would still no doubt miss things that would then appear in the error term and could have been eliminated via differencing. Still, it would be interesting to do the multivariate regression and compare the coefficients across methods.
Thinking of using regression methods of course brings up the McKitrick & Michaels paper. Why not regress on the satellite temperatures too and use coefficients on population density or other socioeconomic variables to correct for UHI?
NickB. (18:06:14)
Yea for the literal meaning of UHI… if youre meaning a change in micro climate through human alterations in environment, maybe not… id imagine there in the USA at the beginning o the century, you were pastoral farmers by n large… and now you are factory farmers, because you can grow more KGs a hectare in grain than grass, and get more productivity a hectare… Things like this could conceivably play a role in giving artificial bias’s. But how would you find out?
Hell chances are in cold areas last century, the guy doing the recording, just guessed on cold nights, cause he didnt want to go out side :-0 Really the satellite record is the only one you could claim any certainty on, what a conundrum in this age o instant answers eh!
Several commentors on both this thread and the original one have talked about the large effect/slope at “rural” sites, mentioning that it doesn’t make sense. I think one possible rationale is to think about where the monitoring site’s positioning is relative to the population. I’m guessing the thermometer/sensor is closer to the local population at these sites than one might assume.
Think about it this way – if one placed a grid of sensors every 5 meters in sq. km, one would have 4000 readings. In a heavily urbanized environment, the average of these readings is probably close to the reading of the sited sensor, simply because the UHI is uniform. However, in more rural areas, my hunch is that the true sited sensor would read near the high end of the 4000 sample distribution because it is located unrealistically close to a local HI (house cluster, airport, factor, you name it).
That’s just a hunch, but it would explain the trend. Perhaps one way to verify this would be to compare the variance at the rural sites compared to the urban sites. My hypothesis is that the rural sites will show more variance.
Any thoughts?
-Scott
Roy, I applaud your approach. I assume you mean that if this relationship can be firmly and correctly established; two different parameters can be inferred at the same time. One is the amount that the pre-urbanization globe has warmed up to and including today and two, the increase in temperature any city of a given size should experience on any windless day due to the UHI. Is that close?
Even though the relationship of station heat bias and population density can be statistically proven, those two other parameters mentioned above may prove harder to statistically establish, but I and many others will know they are true in spite of the lack of a statistical stamp of approval. It only requires a logical mind.
George E Smith
Any extensive analysis of “climate data” over any geological time scale you want to examine; will clearly show that the data has all the ear marks of 1/f noise. Not that I am claiming that the data DOES fit a 1/f noise spectrum.
Another way of putting it would be to sday the data is fractal in nature; and no matter what time scale one chooses to look for a trend; similar appearances, will occur at both longer and shorter time frames.
This would have not made much sense to my before I read Deep Simplicity by John Gribbin. He made a good fist at explaining fractals,etropy and order at the edge of chaos. I do amateur audio work and the 1/f starts (in audio) with white noise which is completely random to music that is well structured. In between there is pink noise, brown noise and so it goes down the line with increasing order(information).
John Gribbin is very much a warmist and the book did discuss weather as 1/f noise in very much detail all.
George has mentioned this concept before, perhaps he can elaborate on chaos, fractals,1/f noise in relation to weather at some time. We need to come to grips with this factor, its effect may be a directional ‘forcing’ or may be random, like white noise, we just do not know.
It just may help clarify Spencer’s excellent work.
All I can say it seems that everything has been done to cool down past temps and “heat up” current ones to validate AGW