McKitrick & Michaels Were Right: More Evidence of Spurious Warming in the IPCC Surface Temperature Dataset
Guest post by Roy W. Spencer, Ph. D.
The supposed gold standard in surface temperature data is that produced by Univ. of East Anglia, the so-called CRUTem3 dataset. There has always been a lingering suspicion among skeptics that some portion of this IPCC official temperature record contains some level of residual spurious warming due to the urban heat island effect. Several published papers over the years have supported that suspicion.
The Urban Heat Island (UHI) effect is familiar to most people: towns and cities are typically warmer than surrounding rural areas due to the replacement of natural vegetation with manmade structures. If that effect increases over time at thermometer sites, there will be a spurious warming component to regional or global temperature trends computed from the data.
Here I will show based upon unadjusted International Surface Hourly (ISH) data archived at NCDC that the warming trend over the Northern Hemisphere, where virtually all of the thermometer data exist, is a function of population density at the thermometer site.
Depending upon how low in population density one extends the results, the level of spurious warming in the CRUTem3 dataset ranges from 14% to 30% when 3 population density classes are considered, and even 60% with 5 population classes.
DATA & METHOD
Analysis of the raw station data is not for the faint of heart. For the period 1973 through 2011, there are hundreds of thousands of data files in the NCDC ISH archive, each file representing one station of data from one year. The data volume is many gigabytes.
From these files I computed daily average temperatures at each station which had records extending back at least to 1973, the year of a large increase in the number of global stations included in the ISH database. The daily average temperature was computed from the 4 standard synoptic times (00, 06, 12, 18 UTC) which are the most commonly reported times from stations around the world.
At least 20 days of complete data were required for a monthly average temperature to be computed, and the 1973-2011 period of record had to be at least 80% complete for a station to be included in the analysis.
I then stratified the stations based upon the 2000 census population density at each station; the population dataset I used has a spatial resolution of 1 km.
I then accepted all 5×5 deg lat/lon grid boxes (the same ones that Phil Jones uses in constructing the CRUTem3 dataset) which had all of the following present: a CRUTem3 temperature, and at least 1 station from each of 3 population classes, with class boundaries at 0, 15, 500, and 30,000 persons per sq. km.
By requiring all three population classes to be present for grids to be used in the analysis, we get the best ‘apples-to-apples’ comparison between stations of different population densities. The downside is that there is less geographic coverage than that provided in the Jones dataset, since relatively few grids meet such a requirement.
But the intent here is not to get a best estimate of temperature trends for the 1973-2011 period; it is instead to get an estimate of the level of spurious warming in the CRUTem3 dataset. The resulting number of 5×5 deg grids with stations from all three population classes averaged around 100 per month during 1973 through 2011.
RESULTS
The results are shown in the following figure, which indicates that the lower the population density surrounding a temperature station, the lower the average linear warming trend for the 1973-2011 period. Note that the CRUTem3 trend is a little higher than simply averaging all of the accepted ISH stations together, but not as high as when only the highest population stations were used.
The CRUTem3 and lowest population density temperature anomaly time series which go into computing these trends are shown in the next plot, along with polynomial fits to the data:
Again, the above plot is not meant to necessarily be estimates for the entire Northern Hemispheric land area, but only those 5×5 deg grids where there are temperature reporting stations representing all three population classes.
The difference between these two temperature traces is shown next:
From this last plot, we see in recent years there appears to be a growing bias in the CRUTem3 temperatures versus the temperatures from the lowest population class.
The CRUTem3 temperature linear trend is about 15% warmer than the lowest population class temperature trend. But if we extrapolate the results in the first plot above to near-zero population density (0.1 persons per sq. km), we get a 30% overestimate of temperature trends from CRUTem3.
If I increase the number of population classes from 3 to 5, the CRUTem3 trend is overestimated by 60% at 0.1 persons per sq. km, but the number of grids which have stations representing all 5 population classes averages only 10 to 15 per month, instead of 100 per month. So, I suspect those results are less reliable.
I find the above results to be quite compelling evidence for what Anthony Watts, Pat Michaels, Ross McKitrick, et al., have been emphasizing for years: that poor thermometer siting has likely led to spurious warming trends, which has then inflated the official IPCC estimates of warming. These results are roughly consistent with the McKitrick and Michaels (2007) study which suggested as much as 50% of the reported surface warming since 1980 could be spurious.
I would love to write this work up and submit it for publication, but I am growing weary of the IPCC gatekeepers killing my papers; the more damaging any conclusions are to the IPCC narrative, the less likely they are to be published. That’s the world we live in.
UPDATE: I’ve appended the results for the U.S. only, which shows evidence that CRUTem3 has overstated U.S. warming trends during 1973-2011 by at least 50%.
I’ve computed results for just the United States, and these are a little more specific. The ISH stations were once again stratified by local population density. Temperature trends were computed for each station individually, and the upper and lower 5% trend ‘outliers’ in each of the 3 population classes were excluded from the analysis. For each population class, I also computed the ‘official’ CRUTem3 trends, and averaged those just like I averaged the ISH station data.
The results in the following plot show that for the 87 stations in the lowest population class, the average CRUTem3 temperature trend was 57% warmer than the trend computed from the ISH station data.
These are apples-to-apples comparisons…for each station trend included in the averaging for each population class, a corresponding, nearest-neighbor CRUTem3 trend was also included in the averaging for that population class.
How can one explain such results, other than to conclude that there is spurious warming in the CRUTem3 dataset? I already see in the comments, below, that there are a few attempts to divert attention from this central issue. I would like to hear an alternative explanation for such results.
oldgamer56 says:
March 30, 2012 at 1:09 pm
Would it be worthwhile to focus on some long term stations that meet the Cat 1 or 2 standard and have experienced transition from rural to urban if they can be linked to some nearby long term Cat 1 or 2 station that has stayed rural? Would suggest that the light density photos from satellite would be the best way to define urban/rural, as it is more infrastructure specific than population.
Would seem this approach would get away from models and be strictly observation driven.
_____________________________
Here is a set of data points that illustrate the problem. These are the only city & close by airport listed for North Carolina. The NC state population, 2011 estimate, is 9,656,401. For a comparison the New York–Newark, NY –NJ–CT Urbanized Area has an estimated population of 18,319,939 double that of the entire state of NC. The city is on the North Carolina/Virgina border and right on the ocean.
Take a look at the city vs the airport! Norfolk City and
Norfolk International Airport
………..
North to south thru the middle of the state of North Carolina
North – Raleigh NC (from 1890 to present)
Large city in the middle of NC – Fayetteville NC (from 1890 to present)
South – Lumberton NC (from 1890 to present)
Rural
North – Louisburg
North – Louisburg (from 1890 to present)
South – Southport
South – Southport
Here is the raw 1856 to current Atlantic Multidecadal Oscillation
@ur momisugly rgbatduke who said “Suggestions: I know you are doing apples to apples, but 5×5 degree gridding is absurd as it builds in a horrendous projective correction near the poles, and one that is reasonably accurate only near the equator. I’d strongly suggest building an icosahedral tiling of the sphere at a scale-adustable granularity.”
Here is a link to an example of such a projection:
http://www.rwgrayprojects.com/rbfnotes/maps/graymap1.html
Gail Combs:
In your post at March 30, 2012 at 4:40 pm you say;
“To (sic) bad this very important work will never make it into publication.”
It HAS made it into publication. It has been published on WUWT.
And please do not provide any BS about the ‘worth’ of scientific work being related to peer review prior to publication. It is not.
A third-rate patents clerk published two papers on (what he called) relativity. The worth of those papers is demonstrated by their having revolutionised physics, and the fact that they were not peer reviewed prior to publication does not change that.
Two bicycle salesmen published a seminal paper on aviation in a magazine about bee-keeping. The worth of that paper is demonstrated by commercial air traffic, and the facts of its lack of peer review and where it was published do not change that.
etc.
Science assesses information on its merits. The source of the information is not relevant (nullius in verba).
Richard
Hugh Pepper says:
March 30, 2012 at 12:46 pm
This question was dealt with by the BEST study. They concluded that, since only 0.5% of the world is urbanized, even a 2 degree rise in urban temperature would contribute negligibly to the global average. Were they too part of the great conspiracy?
That would be all well and good if the stations were spread evenly over the globe, but they are not. Many are near urban centers because that’s where they can be serviced more conveniently.
In 2008 several stations in Ireland were closed down due to urban sprawl.
John Finn says:
March 30, 2012 at 5:04 pm
“I can’t see any reason why urban TRENDS (not temperatures) should be any different to rural TRENDS (not temperatures) unless ALL urban populations were growing while ALL rural populations remained static.”
I actually agree with John Finn here. Let’s look at the long term global temperature trend line since the LIA:
http://i.minus.com/idAOoE.gif
As we see, the trend has not accelerated. If it had it would have exceeded its long term parameters by now.
Therefore, the ≈40% rise in CO2 has had no measurable effect. It’s right there in the record. If the runaway global warming predictions were right, we would see a recent rise in the long term trend. But we don’t [note that the green long term trend line is slowly declining].
That long term temperature trend line is the same whether CO2 was 280 ppmv, or 392 ppmv. Does that not cause a major problem for those claiming that “carbon” is rapidly pushing up the temperature? But the long term trend is indistinguishable from natural variability. And no one has falsified the hypothesis that the observed temperature changes are the result of natural variability.
The planet is simply warming from the LIA — one of the coldest episodes of the Holocene. A warming ocean outgases CO2. And the geologic record shows conclusively that rises in CO2 follow temperature rises, not vice versa.
Thank you Dr. Spencer for a most enlightening and wonderful paper. Real analysis of real data — an increasingly novel idea it seems.
Please don’t give up on publication. I realize how frustrating and maddening it must be battling those who choose to place agenda before facts. But keep up the fight because the facts will prevail in the end.
I would love to write this work up and submit it for publication, but I am growing weary of the IPCC gatekeepers killing my papers; the more damaging any conclusions are to the IPCC narrative, the less likely they are to be published. That’s the world we live in.
==================================================
Dr. Spencer, I think it’s time you and some of the more responsible adults simply start ignoring the current lunatic infested journals. And, simply publish in the alternative places. Your work on this issue (UHI) and others is so well known, very few alarmists actually dispute it, they simply hand wave and ignore this glaring fact.
Perhaps some of you guys can set up a publishing network of your own….
Anymore, I only use published studies on the climate issue to be held up for scorn and ridicule….. as do most here. My recent favorite was the one about if the stuff making us cooler wasn’t happening, we’d still be getting warmer!! I’m thinking up publishing a study about if frogs had glass asses they wouldn’t jump so high. But, then there’s always the lack of arctic ice causing all the snow, when before it didn’t, but it does now…. that we’re at a 7-8 year high in the ice extent.
You really shouldn’t risk tarnishing your name and reputation by publishing in some crap magazine anyway. I understand there are different venues available today.
Hugh Pepper says:
March 30, 2012 at 12:46 pm
“This question was dealt with by the BEST study. They concluded that, since only 0.5% of the world is urbanized, even a 2 degree rise in urban temperature would contribute negligibly to the global average. Were they too part of the great conspiracy?”
Are you alluding to Dr. Muller’s connections to the geo-engineering NOVIM group or to his activities in his own company when talking about a conspiracy?
http://jer-skepticscorner.blogspot.com/2011/04/best-novim-and-other-solution.html
http://www.mullerandassociates.com/index.php
more about Muller here…
http://wattsupwiththat.com/2011/11/13/the-waxman-markey-circus-is-coming-to-town-dr-richard-muller-to-showcase-best-under-the-bigtop/#comment-796111
Roy Spencer says:
March 30, 2012 at 4:15 pm
Following up on a few comments about the strength of the warming bias increasing (not decreasing) with average population density, I agree this is opposite of what I expected. I don’t have an explanation for it, but I haven’t taken the time to think about it, either.
I think the problem here is that UHI itself is widely misunderstood. It’s assumed to be greater heat release and retention in urban areas.
I think decreased urban aerosols and aerosol seeded clouds since the 1970s is the primary cause of urban warming since that time in the developed world (quite different trends exist in the developing world). This explains why larger urban areas show a greater trend, because aerosol concentrations are largely a function of the size of an urban area. It also explains why the effect is smaller in the southern hemisphere, because aerosol levels in the SH (ex tropics) have always been lower than in the NH (Brazil possibly excepted).
there are hundreds of thousands of data files in the NCDC ISH archive, each file representing one station of data from one year. The data volume is many gigabytes.
I did try and download this data but gave up under the sheer immensity of the task. If you could make it available in a more accessible form, I and I expect others would appreciate it.
One final point, I believe that an analysis of this data by time of day would produce some interesting results. I believe you will find the warming is predominantly a daytime phenomena. This would be directly contrary to analyses based on min/max temps, which IMO wrongly attribute increasing min temps to increasing nighttime temperatures, when the cause is increased solar insolation due to decreased aerosols.
Is UHI being attributed to human population?
The great dying of the thermometers can not be explained in terms of too much data to analyze, as computing power during this time grew by leaps and bounds.
If I was tasked to find a human signal in global temperature, the killing of thermometers that showed no warming would be an obvious step to take. The then fudging of the UHI is an easy cover as no effort was taken to show its full extent and an ever upward temperature signal can be foisted on the world.
The poor dears at the moment are having trouble finding enough fudge factors to keep the temperate level, as even the UHI is not enough to cover the decline. The coming years will be such a disappointment to the poor souls.
Thank you Dr Spencer.
Gregory Prinsze says: @ur momisugly March 30, 2012 at 1:25 pm
….. I would be very interested in hearing a comment from the author or another expert in response to Hugh Pepper’s point about the BEST study….
___________________________________
The discussion of the BEST study at WUWT:
http://wattsupwiththat.com/2011/10/20/the-berkeley-earth-surface-temperature-project-puts-pr-before-peer-review/
http://wattsupwiththat.com/2011/10/20/pielke-sr-no-surprise-about-best/
http://wattsupwiththat.com/2011/10/21/sceptical-berkeley-scientists-say-human-component-of-global-warming-may-be-somewhat-overstated/
A mathematician’s response to BEST: http://wattsupwiththat.com/2011/10/21/a-mathematicians-response-to-best/
http://wattsupwiththat.com/2011/10/21/best-what-i-agree-with-and-what-i-disagree-with-plus-a-call-for-additional-transparency-to-preven-pal-review/
http://wattsupwiththat.com/2011/10/22/a-preliminary-assessment-of-bests-decline/
http://wattsupwiththat.com/2011/10/24/what-the-best-data-actually-says/
And More….
http://wattsupwiththat.com/2011/10/24/real-climate-pans-best-and-muller/
http://wattsupwiththat.com/2011/10/25/singers-letter-to-wapo-on-best/
http://wattsupwiththat.com/2011/10/27/nature-pans-best-and-muller-pr-antics-prints-letter-from-dr-singer/
http://wattsupwiththat.com/2011/10/28/explaining-muller-vs-muller-is-best-blissfully-unaware-of-cosmic-ray-cloud-theory/
http://wattsupwiththat.com/2011/10/29/uh-oh-it-was-the-best-of-times-it-was-the-worst-of-times/
This maybe of interest also: Unadjusted data of long period stations in GISS show a virtually flat century scale trend http://wattsupwiththat.com/2011/10/24/unadjusted-data-of-long-period-stations-in-giss-show-a-virtually-flat-century-scale-trend/
the maps at these links show how US population shifts continue to increase the size of urban areas…
http://www.forbes.com/2010/06/04/migration-moving-wealthy-interactive-counties-map.html
http://www.usgcrp.gov/usgcrp//Library/nationalassessment/images/PopMap-o.jpg
I think that either Dr. Abdussamatov is right and the year 2014 will show transition into a 200 year “Little Ice Age” or Professor Vladimir Paar is right and it will be a new glaciation era of 70,000 years. Both could be right. We will never know since our lives are so short.
Roy Spencer said @ur momisugly March 30, 2012 at 4:58 pm
Ag scientists are like that 😉
Dr Spencer,
One thing that appears to be missing is the distribution of trends in each class. How significant are the differences between the trends, compared to the standard error of the trends?
It would be interesting to see your opinion of why your result is different from the BEST esitimate of the UHI, which is based on a difference between all the very rural stations, known to be distant from population centers, and the rest of the world’s stations. Using a larger number of stations they got and uncertainty estimate of +/- 0.19C/century for the slope.(95% confidence)
I was confident you had it under control but as a fellow aviator who earned his wings at John Wayne Field (long before it was JWF), I’ve seen first hand why a lot of areas of California are sparsely populated and know how quickly one can transit climate zones in the Golden State. From the air the highest and lowest points in the state can be seen at the same time as they’re not too far apart, and not far from either are high and low deserts, the central valley, glaciated mountains, sparse coastal regions, etc. I’m looking forward to learning how it is all quantified in your paper. Thanks for responding.
Thanks Dr. Spencer. Again UHI is shown to be a major “forcing” on thermometer’s data.
And even then, it showed a cooling trend after 1998.
Dr. Spencer,
If you’re still reading this thread, please expound on this quote Steve Goddard says is from you. I have asked Steve Mosher to comment on this a few times, but apparently he didn’t read it.
http://www.real-science.com/important-point-from-roy-spencer
Steve Mosher, care to comment now?
One further point about urban – rural comparisons.
In many rural areas irrigation is practiced and there have been large increases and decreases in the amount of irrigation over the last 50 years in different areas.
Dr Christy found a 3C rise in temps in the Central Valley over the 20th C, he attributed to increased irrigation and albedo changes resulting from irrigation mostly.
http://www.livescience.com/563-irrigation-fuels-warmer-temps-california-central-valley.html
Which makes me think an analysis by size of urban area would reveal more about UHI than an urban – rural comparison.
Although irrigation isn’t exclusively a rural practice. Here in Perth, Australia, when they moved the official Perth site from opposite an irrigated park to a non-irrigated field a couple of Ks away, nighttime temperatures immediately rose 1.5C. This is because the park was irrigated at night, increasing the thermal capacity of the air and decreasing nighttime temperatures.
Did you not notice that the plot of California sites was labeled ‘Goodridge 1996’ and was not part of Dr. Spencer’s analysis?
See the graphs below. Is it possible that UHI is at least partly responsible for the difference between the satellite slopes and the slope of the others?
http://www.woodfortrees.org/plot/uah/from:1978.9/trend/offset:0.31/plot/rss/from:1978.9/trend/offset:0.22/plot/gistemp/from:1978.9/trend/plot/hadcrut3gl/from:1978.9/trend/offset:0.08
“I am attempting to correlate County population changes changes from 1900 to 2010 with cooling or warming from 1900 to 2011.
Warming Counties had a mean temperature change of .0692C/decade.
Warming counties had a mean population increase of 174,361.
Warming counties on average grew by 648% from 1900 to 2011.
Cooling counties had a mean temperature change of -.0573C/decade.
Cooling counties had a mean population increase of 39,060.
Cooling counties on average grew by 194% for 1900 to 2011.”
http://sunshinehours.wordpress.com/2012/03/17/county-population-statistics-and-coolingwarming-stations-since-1900/
Higher population growth = warming.
It is disappointing to see this arguing over atmospheric temperature when we should be measuring Earth’s heat budget. Atmospheric temperature is not a metric for heat content. Averaging this incorrect metric only makes things worse.
Is there any chance that someone somewhere will actually measure the heat content of the air in Kilo Joules per Kilogram, the correct metric if there is concern about ‘trapped heat’? If the humidity (which is normally reported at the same time as the air temperature) is known then calculating the enthalpy and therefore the heat content is simply done.
I have a feeling the tropical mid tropospheric ‘hot spot’ is not there as the enthalpy of the air is high therefore it takes a lot more heat energy to raise the atmospheric temperature and the GCMs are not programmed for enthalpy.
This is relatively basic physics, but seems to be too complex for climate scientists – or perhaps like the air-conditioning in the congressional meeting room, they are using it because it is useful even if they know it is wrong – and certainly fools people, even on this blog few are calling them on it.