McKitrick & Michaels Were Right: More Evidence of Spurious Warming in the IPCC Surface Temperature Dataset
Guest post by Roy W. Spencer, Ph. D.
The supposed gold standard in surface temperature data is that produced by Univ. of East Anglia, the so-called CRUTem3 dataset. There has always been a lingering suspicion among skeptics that some portion of this IPCC official temperature record contains some level of residual spurious warming due to the urban heat island effect. Several published papers over the years have supported that suspicion.
The Urban Heat Island (UHI) effect is familiar to most people: towns and cities are typically warmer than surrounding rural areas due to the replacement of natural vegetation with manmade structures. If that effect increases over time at thermometer sites, there will be a spurious warming component to regional or global temperature trends computed from the data.
Here I will show based upon unadjusted International Surface Hourly (ISH) data archived at NCDC that the warming trend over the Northern Hemisphere, where virtually all of the thermometer data exist, is a function of population density at the thermometer site.
Depending upon how low in population density one extends the results, the level of spurious warming in the CRUTem3 dataset ranges from 14% to 30% when 3 population density classes are considered, and even 60% with 5 population classes.
DATA & METHOD
Analysis of the raw station data is not for the faint of heart. For the period 1973 through 2011, there are hundreds of thousands of data files in the NCDC ISH archive, each file representing one station of data from one year. The data volume is many gigabytes.
From these files I computed daily average temperatures at each station which had records extending back at least to 1973, the year of a large increase in the number of global stations included in the ISH database. The daily average temperature was computed from the 4 standard synoptic times (00, 06, 12, 18 UTC) which are the most commonly reported times from stations around the world.
At least 20 days of complete data were required for a monthly average temperature to be computed, and the 1973-2011 period of record had to be at least 80% complete for a station to be included in the analysis.
I then stratified the stations based upon the 2000 census population density at each station; the population dataset I used has a spatial resolution of 1 km.
I then accepted all 5×5 deg lat/lon grid boxes (the same ones that Phil Jones uses in constructing the CRUTem3 dataset) which had all of the following present: a CRUTem3 temperature, and at least 1 station from each of 3 population classes, with class boundaries at 0, 15, 500, and 30,000 persons per sq. km.
By requiring all three population classes to be present for grids to be used in the analysis, we get the best ‘apples-to-apples’ comparison between stations of different population densities. The downside is that there is less geographic coverage than that provided in the Jones dataset, since relatively few grids meet such a requirement.
But the intent here is not to get a best estimate of temperature trends for the 1973-2011 period; it is instead to get an estimate of the level of spurious warming in the CRUTem3 dataset. The resulting number of 5×5 deg grids with stations from all three population classes averaged around 100 per month during 1973 through 2011.
RESULTS
The results are shown in the following figure, which indicates that the lower the population density surrounding a temperature station, the lower the average linear warming trend for the 1973-2011 period. Note that the CRUTem3 trend is a little higher than simply averaging all of the accepted ISH stations together, but not as high as when only the highest population stations were used.
The CRUTem3 and lowest population density temperature anomaly time series which go into computing these trends are shown in the next plot, along with polynomial fits to the data:
Again, the above plot is not meant to necessarily be estimates for the entire Northern Hemispheric land area, but only those 5×5 deg grids where there are temperature reporting stations representing all three population classes.
The difference between these two temperature traces is shown next:
From this last plot, we see in recent years there appears to be a growing bias in the CRUTem3 temperatures versus the temperatures from the lowest population class.
The CRUTem3 temperature linear trend is about 15% warmer than the lowest population class temperature trend. But if we extrapolate the results in the first plot above to near-zero population density (0.1 persons per sq. km), we get a 30% overestimate of temperature trends from CRUTem3.
If I increase the number of population classes from 3 to 5, the CRUTem3 trend is overestimated by 60% at 0.1 persons per sq. km, but the number of grids which have stations representing all 5 population classes averages only 10 to 15 per month, instead of 100 per month. So, I suspect those results are less reliable.
I find the above results to be quite compelling evidence for what Anthony Watts, Pat Michaels, Ross McKitrick, et al., have been emphasizing for years: that poor thermometer siting has likely led to spurious warming trends, which has then inflated the official IPCC estimates of warming. These results are roughly consistent with the McKitrick and Michaels (2007) study which suggested as much as 50% of the reported surface warming since 1980 could be spurious.
I would love to write this work up and submit it for publication, but I am growing weary of the IPCC gatekeepers killing my papers; the more damaging any conclusions are to the IPCC narrative, the less likely they are to be published. That’s the world we live in.
UPDATE: I’ve appended the results for the U.S. only, which shows evidence that CRUTem3 has overstated U.S. warming trends during 1973-2011 by at least 50%.
I’ve computed results for just the United States, and these are a little more specific. The ISH stations were once again stratified by local population density. Temperature trends were computed for each station individually, and the upper and lower 5% trend ‘outliers’ in each of the 3 population classes were excluded from the analysis. For each population class, I also computed the ‘official’ CRUTem3 trends, and averaged those just like I averaged the ISH station data.
The results in the following plot show that for the 87 stations in the lowest population class, the average CRUTem3 temperature trend was 57% warmer than the trend computed from the ISH station data.
These are apples-to-apples comparisons…for each station trend included in the averaging for each population class, a corresponding, nearest-neighbor CRUTem3 trend was also included in the averaging for that population class.
How can one explain such results, other than to conclude that there is spurious warming in the CRUTem3 dataset? I already see in the comments, below, that there are a few attempts to divert attention from this central issue. I would like to hear an alternative explanation for such results.
Philip Bradley says:
March 30, 2012 at 5:47 pm
……….
I think the problem here is that UHI itself is widely misunderstood. It’s assumed to be greater heat release and retention in urban areas.
I think decreased urban aerosols and aerosol seeded clouds since the 1970s is the primary cause of urban warming since that time in the developed world (quite different trends exist in the developing world). This explains why larger urban areas show a greater trend, because aerosol concentrations are largely a function of the size of an urban area. It also explains why the effect is smaller in the southern hemisphere, because aerosol levels in the SH (ex tropics) have always been lower than in the NH (Brazil possibly excepted)……..
Well first there is the effect of all the buildings and concrete and tarmac that reflect a lot of heat and also get hot and act as heat stores for overnight. Then the buildings have air-conditioning heat pumps dumping waste heated dry air to the atmosphere or heating leaking heated air to the atmosphere. All the power being consumed normally turns to heat and leaks from the buildings and lights. Then most cities have a lot less plant life trees, grasses than the rural areas. Plants in rural areas transpire water into the atmosphere raising its humidity and therefore its enthalpy and increasing the amount of heat required to raise the temperature. So the dryer air over towns and cities will be warmer for the same amount of energy and the amounts of energy available in cities at night is higher for the reasons given.
And of course – atmospheric temperature is in any case the incorrect metric to measure atmospheric heat content.
“There has always been a lingering suspicion among skeptics that some portion of this IPCC official temperature record contains some level of residual spurious warming…”
A life born of fire like the Phoenix rises again: “…evidence that CRUTem3 has overstated U.S. warming trends during 1973-2011 by at least 50%.”
Dr Spencer,
Now that you have done this, and shown your main message, you might do two things and then submit to a statistics journal like Annals of Applied Statistics: then instead of an outright rejection, you might get a debate such as the debate stimulated by McShane and Wyner.
the two things:
1) address Steven Mosher’s points in detail — it seems from your responses that you can, and that they do not affect the main message;
2) if it isn’t too much trouble, follow this up with analyses of the 4 time points separately, to see if you can discover at what time of day the effect is greatest. I think that it will be a really good idea for climate scientists to step away from the “daily mean”, and look at particular times of day. Usually (I don’t know if it is true in this case), doing the disaggregated analysis after doing the aggregated analysis is not that much extra programming time, but is a lot of additional reading and writing time.
I think this would be worthwhile, but that is only an opinion from a non-expert.
Eric Adler says on March 30, 2012 at 6:27 pm:
According to BEST, “distant” can be as close as 6 nautical miles (1 tenth of a degree of latitude or longitude). That hardly qualifies as being very far from population centers. Add to that the point made by Philip Bradley regarding irrigation and, at best, the “very rural” are a mixture of sites with anthropogenic influences of some sort and probably a minority that have minimal anthropogenic influences. This makes their “conclusions” questionable. Furthermore, BEST, to my knowledge, has not been published in a peer-reviewed publication and they have not really addressed the issues raised on this august forum (see links here).
Left out of discussions above is heat released from burning fossil fuels which have totally released energy to warm the atmosphere by .2 C. Touched on was irrigation in CA that increased the atmosphere temperature by 3 C. However, worldwide there is about 900 cubic kilometers per year of fossil ground water from slow or no recharge aquifers being produced, which has resulted in increasing the temperature of the atmosphere by about 1.6 C per year since 1950. Also left out is the release of energy from condensation at night on grass blades used for lawns and production of food and fodder. Area extent of both urban and rural grasses has increased steadily since roughly 1950.
It appears to me that there is a temperature control process at work in the upper Troposphere.
Smokey says: “If the runaway global warming predictions were right, we would see a recent rise in the long term trend” Smokey: those “predictions” are “projections” and projections are logically neither right nor wrong.
Correction:
A highly questionable method called
“polynominals”“polygons” is currently being used to make this jump, but far superior methods exist.(Sorry, sometimes math overwhelms my vocabulary).
NH = Northern Hemisphere, it’s not exactly an isolated region is it, you know, one whole half of the Earth. If it’s significant in the NH and non existent in the SH then it’s still significant over all.
Give constructive feedback where it’s due and let him further improve the analysis. That’s science. Some write as if anything that isn’t perfect gets tossed in the bin.
They try to estimate numbers of non-existent climate refugees but something as important as UHI is neglected. Go figure.
What does “rural” actually mean in 2012. My rural locale, the Fraser Valley is arguably the best farm land in Canada. It produces mouth-watering corn, delicious blueberries and all manner of veggies and livestock but it also has the Trans Canada Highway running through it, land fills, two international airports, acres of natural gas heated greenhouses, bio gas plants, composters, chicken, turkey, pig and mushroom “farms” – steel sheds – and enough diesel powered combines, tractor-trailers, locos and tractors to give Gore a heart attack. With mechanised farming i would think that the number of people working farms has dropped as production has increased over the last 50 years. The measure of the “UHI” of rural areas by population density is, i’ll wager, underestimating warming effects of intensive,energy -dependant, modern farming and “rural” life.
At the time of this post (after 9.18 pm WUWT time) Hugh Pepper had not responded to those commenting on his observations about the BEST data. This is not surprising given his obvious misunderstanding, whether due to genuine incomprehension of or deliberately ignoring, the effect of UHI on the temperature records and how this has affected and is affecting political decisions. A number of posters have commented on the southern temperature records and have recommending using data from Australia and New Zealand. You all should be aware that the temperature data records in both of these countries have been recently “modified” with some stations, usually rural. being excluded. In other cases the modifications appear to be upward. I am not aware of any downward modifications but this doesn’t mean these have not occurred.
Thanks again Dr. Spencer,
I have published quotes from this excellent article in the section on Global Temperatures of my page “Climate Change (“Global Warming”?)
– The cyclic nature of Earth’s climate”, at http://www.oarval.org/ClimateChangeBW.htm
(Spanish version at http://www.oarval.org/CambioClimaBW.htm)
Roy said:
“I would love to write this work up and submit it for publication, but I am growing weary of the IPCC gatekeepers killing my papers; the more damaging any conclusions are to the IPCC narrative, the less likely they are to be published. That’s the world we live in.”
___________________________
A proposed new procedure to publish scientific papers and conduct peer review:
Roy, you may recall that in early 2008 I asked Joe d’Aleo to published a paper for me on icecap.us and he kindly did so. I published all my data and calculations and my paper was critiqued in ClimateAudit.org by Willis and others. I believe this ad-hoc process constituted a far more rigorous peer review than the typical “pal review” that the hockey team and other global warming acolytes apply to their friends’ papers in the major journals.
Technology has made these journals and their review methods obsolete anyway. Why don’t you, or Anthony or Joe, etc. establish a website specifically for parties to publish their climate research papers and have them critiqued – the rules could be simple – publish your paper with all data and all supporting calculations. Anyone can critique your paper provided they mind their manners, dot their i’s and cross their t’s. No gates and no gatekeepers. I expect that you would soon leave these once-prestigious journals and their gatekeepers in the dust.
This is precisely why I am only interested in satellite data.
I generally consider the surface data to be worthless in any use for measuring climate effects, as I think many of us do…
Great work Dr. Spencer!
I caught your stint on Fox the other day and thought you were excellent.
@JFD
“It appears to me that there is a temperature control process at work in the upper Troposphere.”
Well done sir !!
Since the atmosphere is a “regulator” operating under, (for the most part), the combined gas laws,
a rise in total atmospheric temperature must either increase the atmospheric pressure, (only possible locally), or increase the atmpospheric volume. Now since the tropopause could be considered the point at which convection stops, this means that the tropopause must raise slightly in altitude (I believe this has been measured ?) This will greatly increase the surface area available for radiation loss thus maintaining the system balance.
The whole thing is controlled PURELY by atmospheric pressure and incoming radiation. The lapse rate is controlled by the combined gas laws, La = Cp/G, (Cp = specific energyof the atmosphere), the only thing that can change the lapse rate is atmopheric condensation of H2O, which lowers the lapse rate because it increases Cp), but the energy transfer rate is still the same !!
If a parcel of air has more energy than it can hold wrt the air above it, it MUST rise. Its called convection, and along with conduction is the dominate method of moving energy within the atmosphere.
Thus, since no gas (apart from H2O condensation, which is really a phase change) can change the lapse rate (certainly NOT a trace amount of CO2), there can be no greenhouse effect in the Earth’s atmosphere.
Dr. Spencer,
I think your analysis understates UHI because the period you analyzed was one of the steeper sections of the record since it occurred from the bottom of the trough (mid-’70’s) to present. By analyzing this section, your baseline slope is much steeper than the overall long term trend, so any absolute UHI °C/yr identified will be a smaller % of the long term slope in that section.
You identified a method by which UHI goes from 15% to 30%. I identified a difference in slope of 2.5x after cyclical elements were removed. I went ahead and posted my own analysis of CRUTem3 tonight so you could see the details. Please see my analysis here:
http://naturalclimate.wordpress.com/2012/03/31/northern-hemisphere-uhi-crutem3-18/
Based on our two studies taken together, and then taking the idea of using all 5 population density categories, I pose the question on whether between the two of us we have identified UHI as MORE THAN 100% of the total observed temperature increase. I don’t thing it is out of the question. I would be most interested to hear your perspective on this. As always, thanks again for your interesting articles. Good show on Stossel.
As far as giving up on the gatekeepers, I’m thinking the gates they are guarding aren’t worth walking through anymore. Somebody might see you. Mike S.
One problem that Dr. Spencer will have to address is the lack of spatial coverage in his approach.
Effectively his selection of grid cells is biased to select cells that must have all three population types. This does two things
1. It limits you to NH where UHI is worse
2. It eliminates grid cells that are only rural, cells that have rural and medium population
and cells that have rural and urban but no medium population.
The spatial coverage is consequently very small.
Here is another way I can illustrate the problem.
In the Berkeley Earth Data ( which includes the data Dr. Spencer uses plus more) there are roughly 36K stations that pass data quality checks ( basically have more than a few months of data )
Of those 36853 stations: 15,348 are in Dr. Spencers LOWEST population class, using his
population dataset ( GrumpV1) 12797 are in his “medium” population data class.
and 6471 are in the high population dataclass.
For those of you doing the math ( 12.7+15.3+6.4 =34.4) there are OVER 2000 additional stations that have “missing”
population data. These are “missing” because they are stations that are
a) bouys
b) in antarctica
c) Atols
d) stationary ships at sea.
So, basically 17500 or roughly 50% are in the lowest class.
If you compare the lowest population to the highest population you will get an estimate of
what the bias is for any give pair. However, you have to look at the fraction of stations that actually have that bias. If the actual number of high population stations is significant
then the total bias will be high. But if there are very few stations that have high population
then the bias between high and low, WHILE REAL, will not pollute the entire sample.
Lets use Dr. Spencers estimate to work with
At the lowest population we have a trend of .22C decade
At the medium .24C decade
at the max .28C decade
As Zeke and I noted this difference between the high and the low ( .28–.22)
is fairly close to what we found in our AGU work ~.04C decade. In fact,
In some of our sensitivity work we could find differences as large as .06C
But to see that we had to really restricted the spatial coverage ( <25% of the earth)
basically we had to compare the best to the worst.
When you look at a more complete dataset than ISH you get
50% of the stations are Spencers low population
33% of the stations are medium population A bias of .02C decade
17% of the stations are high population a Bias of .06C decade
Weight those biases by sample and you get an idea of the bias in the whole sample.
17% of your sample will have a Bias of .06C, 33% will have a bias of .02
If you are concerned about Bias in the lowest population class…
there are roughly 8000 stations with population density less than 1. That is,
More stations with zero population than stations with populations over 500 people per sq km
Ian W says:
March 30, 2012 at 8:12 pm
Well first there is the effect of all the buildings and concrete and tarmac that reflect a lot of heat and also get hot and act as heat stores for overnight. Then the buildings have air-conditioning heat pumps dumping waste heated dry air to the atmosphere or heating leaking heated air to the atmosphere. All the power being consumed normally turns to heat and leaks from the buildings and lights. Then most cities have a lot less plant life trees, grasses than the rural areas. Plants in rural areas transpire water into the atmosphere raising its humidity and therefore its enthalpy and increasing the amount of heat required to raise the temperature. So the dryer air over towns and cities will be warmer for the same amount of energy and the amounts of energy available in cities at night is higher for the reasons given.
I should have my point clearer. I was referring to any UHI trend.
If you look at power consumption, per capita USA energy consumption peaked in 1973. So there is no contribution to the trend from this source during the period of Dr Spencer’s study. Although there may be a contribution from increasing population densities, where this has occured.
http://epb.lbl.gov/homepages/rick_diamond/LBNL55011-trends.pdf
Otherwise I agree completely about enthalpy, but unfortunately the debate is around temperatures.
Roy’s “middle graph” shows almost 0.2C difference due to UHI effect in ~30 years.
In 2008 I calculated a similar difference of ~0.2C as the difference between global Hadcrut3 Surface Temperatures (ST) and UAH Lower Troposphere temperatures (LT), also in 30 years (I thought this point was also made by Roy or someone else above, but I cannot find it now).
I concluded in 2008 that there was a probable warming bias of ~0.07C/decade in the ST data, since at least 1979, if not earlier. I know it’s somewhat “apples and oranges” because of different altitudes, but still the numbers appear close.
I collected a number of approaches to UHI discernment and they actually start with Goodridge’s graph at the top here, and include earlier work of Roy Spencer’s.
I found it good to see many different approaches producing basically the same result. My page also shows evidence of the UHI effect increasing more rapidly in very rural areas urbanizing slightly over time, than in urban areas.
Okay. There is an UHI bias in land temperatures; this conclusion does not seem to be controversial. By looking at ocean data — whether via CruTemp or via UAH — we have over 0.1 C increase per decade over the last three decades. Any comments on that other than the increase has softened in the last several years?
Following BEST press release has any actual peer reviewed work be seen in journal form this organization or its that still fence they have not jumped over despite the dramatic claims?
Steven Mosher:
The above article by Roy Spencer proves that
(a) UHI provides significant error to temperature trends in gridded localities
and
(b) the degree of the error relates to population density in the regions.
But at March 31, 2012 at 12:17 am you yet again assert;
“One problem that Dr. Spencer will have to address is the lack of spatial coverage in his approach.”
And the remainder of that post is expansion of your assertion which you first made (but less clearly) at March 30, 2012 at 12:36 pm and have repeated several times since.
I am surprised that you have repeated and pressed the point because it was completely refuted by others after you first made it.
I think ‘atheok’ provided the most clear refutation at March 30, 2012 at 4:43 pm where he asks and says;
“Are you implying that it is not possible to identify a specific urban area and then identify the UHI bias of that area?”
And
“To me, one should first prove there is UHI bias in the record and if possible, quantify it explicitly by region.”
So,
Dr. Spencer DOES NOT have to address the lack of spatial coverage in his approach because it has no relevance of any kind to his analysis.
If you want to extend his analysis to include the entire globe then do so because Dr Spencer’s analysis demonstrates that such an extension would have value. And that demonstration is the value of his analysis.
Simply, your carping is like telling the Wright brothers that they needed to expand their work to provide a Boeing 747.
Richard
An Inquirer says:……..
and what is the error in that 0.1c trend?
what is the error in the measuremensts.
how much has it been “adjusted “……do we even know if it has or hasn’t been adjusted ?
Very useful. Thanks for the hard work and good luck with publishing efforts. It deserves to be widely known; increasingly WUWT will serve that goal!
An Inquirer says:
March 31, 2012 at 2:29 am
///////////////////////////
I have long held the view that the land based temperature data should simply be consigned to the bin. The reasons are fourfold.
First, given the heat capacity of the oceans which dwarfs that of land air temperature, it is ocean temperature not land air temperature that is important.
Second, it is ocean temperatures not land temperatures that drive climate (this is something that follows from the first point).
Third, the land air temperature data does not even measure the correct metric. It tells us nothing about energy.
Fourth, the land air temperature record is almost certainly corrupted by UHI.
The problem with sea temperature is that we do not have good quality data on it and are only just beginning to compile this. We therefore cannot reliably answer the enquiry made by An Inquirer.
The oceans are the key to global warming. Without the oceans warming there can be no global warming. Understanding the reasons why the oceans may be warming (if they are) is also the key.
As I have often observed, not sufficient thought has been given to how DWLWIR works over the oceans. For example:
1. Do the oceans reflect some part of the DWLWIR, if not why not?
The K & T energy diagram shows that not all the solar is absorbed by the surface and some part of this is reflected. The reflected part is predominantly from the oceans which reflect solar when striking at low angles of incidence. Some part of the DWLWIR must strike the oceans at a similar low angle of incidence, so why is that part not simply reflected?
2. The optical physics is such that 20% of all DWLWIR is absorbed within 1 micron of water and 60% of all DWLWIR absorbed within 4 microns of water. Theoretically, there would be so much energy being absorbed within the first few microns of the ocean that this would drive a huge amount of evaporation which is not being observed. Why not? For example, is it because much of the DWLWIR is simply being reflected?, or perhaps it lacks sensible energy? Some explanation is required.
Following on from point 2, given the absorption characteristics of water, water is essentially a LWIR block (much like sun cream may block harmfull UV rays), how does any energy from DWLWIR effectively heat the ocean? How does it find its way down from the top micron level? Indeed, does not wind swept spray/spume in itself act as an effective block. IF wind swept spray/spume is more than 4 microns thick it in itself would capture 60% of all DWLWIR and would carry it away in the air before it even gets to the bulk ocean.
I am merely pointing out that there are substantial issues with DWLWIR and the oceans and much consideration needs to be given to this. To date, I do not see that proper consideration has been given to the optical physics involved and the consequences of this..