I’ve been away from WUWT this weekend for recovery from a cold plus family time as we have visitors, so I’m just now getting back to regular posting. Recently on the web there has been a lot of activity and discussions around the issue of the dropping of climatic weather stations aka “the march of the thermometers” as Joe D’Aleo and I reported in this compendium report on issues with surface temperature records.
Most of the station dropout issue covered in that report is based on the hard work of E. M. Smith, aka “chiefio“, who has been aggressively working through the data bias issues that develop when thermometers have been dropped from the Global Historical Climate Network. My contribution to the study of the dropout issue was essentially zero, as I focused on contributing what I’ve been studying for the past three years, the USHCN. USHCN has had a few station dropout issues, mostly due to closure, but nothing compared to the magnitude of what has happened in the GHCN.
That said, the GHCN station dropout Smith has been working on is a significant event, going from an inventory of 7000 stations worldwide to about 1000 now, and with lopsided spatial coverage of the globe. According to Smith, there’s also been an affinity for retaining airport stations over other kinds of stations. His count shows 92% of GHCN stations in the USA are sited at airports, with about 41% worldwide.
The dropout issue has been known for quite some time. Here’s a video that WUWT contributor John Goetz made in March 2008 that shows the global station dropout issue over time. You might want to hit the pause button at time 1:06 to see what recent global inventory looks like.
The question that is being debated is how that dropout affects the outcome of absolutes, averages, and trends. Some say that while the data bias issues show up in absolutes and averaging, it doesn’t effect trends at all when anomaly methods are applied.
Over at Lucia’s Blackboard blog there have been a couple of posts on the issue that raise some questions on methods. I’d like to thank both Lucia Liljegren and Zeke Hausfather for exploring the issue in an “open source” way. All the methods and code used have been posted there at Lucia’s blog which enables a number of people to have a look at and replicate the issue independently. That’s good.
E.M Smith at “chiefio” has completed a very detailed response to the issues raised there and elsewhere. You can read his essay here.
His essay is lengthy, I recommend giving yourself more than a few minutes to take it all in.
Joe D’Aleo and I will have more to say on this issue also.
FYI. Check out
http://diggingintheclay.blogspot.com/2010/03/of-missing-temperatures-and-filled-in.html
particularly the last graph
A “march” can go in either direction. What we have is the Retreat of the Thermometers. The enquiring mind wonders whether it was a tactical retreat.
[taunting of Anthony lends nothing to the conversation ~ ctm]
I’ve completed USHCN original raw data vs USHCN version 2 revised raw data
blink charts for Iowa. –
http://www.rockyhigh66.org/stuff/USHCN_revisions_iowa.htm
As with Illinois and Wisconsin, the majority of stations had their early temperatures lowered, which artificially increases the warming trend.
These revisions in the base data seem as large as those Dr Hansen did with GISS, but while the GISS homogenized data was in-your-face alteration that you could compare with the original raw data, USHCN version 2 pretends it IS the raw data.
Any use of these revised numbers for climate research is a waste of time.
Here are the Illinois and Wisconsin charts –
http://www.rockyhigh66.org/stuff/USHCN_revisions.htm
http://www.rockyhigh66.org/stuff/USHCN_revisions_wisconsin.htm
These are all the USHCN stations in the three states, no cherry picking.
E.M.Smith does not like anomalies, and likes to do his analysis with absolute temperatures, In that world, the “march of the thermometers” towards the Equator, or wherever, may have cause a real temperature bias.
But the climate scientists do it differently. They do two things that prevent that bias. One is the use of anomalies. That is, you form the global mean by averaging differences of station temps from their a local mean over a fixed period. If you are only looking at temperatures relative to those means, it scarcely matters whether stations being dropped are hot or cold. It only matters whether they are rising relative to that local long term mean.
The other main protection is gridding. The global average is not an average of stations. It’s an average of grid cell averages, which you can see on the GISS plots. The grid average is an average of stations in the grid. So removing a station affects the global average only by its effect on its grid average. If the station was colder than the grid average (and you weren’t using anomalies), then there might be a small effect on the global average due to that differential. But cold stations live in cold grids, and are about equally likely to be colder or warmer than their grid mean.
There’s no point in advancing an argument about a simple global average of station’s temperatures. No-one uses it. That is the real spherical cow.
One wonders if the public at large, especially those who have become AGW believers, realize that the average global temperatures (based on earth weather stations data) reported by CRU at UEA, the IPCC and others is only a very small sampling of actual surface temperatures throughout the world, and that at least 99.9% of the earth’s surface temperatures (which includes ocean surface temperatures) are not adequately represented in the data sets reported by the total of the world-wide network of earth weather stations.
Such an insignificant sampling of temperatures becomes an issue on a planet that has experienced surface temperature extremes as high as +57.7 deg C (+135.86 deg F)[1] and as low as -89 deg C (-128.2 deg F)[2]. That’s a peak to trough record setting temperature range of 146.7 deg C (264.06 deg F). The usual annual temperature range on the earth is -40 deg C (-40 deg F) to +40 deg C (+104 deg F), which is a peak to trough temperature range of 80 deg C (144 deg F). These record setting extremes and the usual annual ranges should put the Warmist’s alarmism over a +0.6 deg C average global temperature rise in the past century (as claimed by the IPCC) into perspective.
[1] The hottest air temperature ever recorded was 57.7 °C (135.9 °F) at Al ‘Aziziyah, Libya, on September 13 1922. [Source:Wikipedia]
[2]The coldest air temperature ever recorded on Earth is −89.2 °C (−129 °F), at Vostok Station, Antarctica on 21 July 1983. [Source:Wikipedia]
My biggest concern is the change from manual to automated measurement of temperature. Having seen how real people behave taking real measurements, I know that there is a significant change going from manual to automatic measurements.
But with less thermometers doesn’t the margin of error increase statistically
Even if the thermometer readings are correct and that there is a trend of increasing temperature, so what? It doesn’t mean it’s due solely to AGW, if at all.
Nick Stokes (02:22:28) :
As E.M. Smith’s post has just explained – he is looking at the SHAPE of the data that GIStemp has to try to cope with. He is doing this PRIOR to working out HOW WELL GIStemp copes with these biases.
It is worth taking the time to read properly, especially if you THINK you know what he has been saying.
Another Ian (01:34:38) : Thanks for the sign post. Glad you like it!
Re so many thermometers being located at airports where the jet exhaust (among other heat sources) can increase the readings. The AGW alarmists would have us believe that the increase in jet travel contributes to an increase in global temperature. It seems that the planes are directly contributing to a false, higher reading on the thermometers by blasting them. So I guess I can agree that planes are involved.
Anthony,
you and D’Aleo wrote:
“It can be shown that they [i.e. NCDC] systematically and purposefully, country by country, removed higher-latitude, higher-altitude and rural locations, all of which had a tendency to be cooler.”
If it “can be shown”, why wasn’t it shown in your “report”? E. M. Smith claimed recently that he only “speculated” about the bias, and that he took “no position on motivation”, because he couldn’t “see inside folks minds”.
But it is you who signed the SPPI report. You should take some responsibility for your [snip] you refer us to E. M. Smith, who seems to change his mind about the existence of the warming bias.
You wrote:
“The thermometers were marched towards the tropics, the sea, and airports near bigger cities. These data were then used to determine the global average temperature and to initialize climate models. Interestingly, the very same stations that have been deleted from the world climate network were retained for computing the average-temperature base periods, further increasing the bias towards overstatement of warming by NOAA.”
Do you think this statement is supported by the “very detailed response” of Smith? What about Roy Spencer’s results then? Are you going to dismiss it, like Smith, as another “hypothetical cow”?
OT
The ex-editor of the Guardian has brazenly called for “climate change” to be a matter of belief and faith:
http://www.guardian.co.uk/commentisfree/2010/mar/07/climate-change-inertia-prophet#start-of-comments
“the plain fact is that we surely need a prophet, not yet another committee. We need one passionate, persuasive scientist who can connect and convince – not because he preaches apocalypse in gory detail, but in simple, overwhelming terms. We need to be taught to believe by a true believer in a world where belief is the fatal, missing ingredient.”
Though I do like the “gory detail” pun …
Can anyone tell me if stations in UK and Europe have been checked for siting as has been done in USA? I would be happy to tour round and find them if I can get a clue to their siting.
Also, can anyone tell me why the Tips & Notes tab on this site doesn’t work for me?
[Reply: on my browser I can’t see Tips & Notes unless I expand the screen. ~dbs, mod.]
It seems to me that there is an interesting potential bias in GisTemp due to reducing station numbers. I read Hansen at al 1999 the other day and it conatins some interesting information on this.
On page 25 it states – “For most of the twentieth century, 55-60% of the U.S. stations are rural (population less than 10,000), about 20% are small town (population less than 50,000), and 20-25% are urban. However, in the final (near-real time) year of data, before USHCN data are available, the urban proportion of stations jumps to about 50- 55%.”
GisTemp adjusts for UHI by making the trend for urban stations the same as the trend for nearby rural stations. Figure 3 seems to imply that this is done by making prior years warmer rather than recent years cooler. (That seem to be like saying that the temperature at an urban station 100 years ago is as if it had the same city as today. It would seem to make more sense to adjust todays temperatures down. i.e. as if the city wasn’t there today.
Anyway, we can see from Figure 1 (b) that the total number of stations has dropped rapidly from a peak of about 6000 in 1970-ish.
GisTemp calculates the temperature for a 2 deg x 2 deg grid box (about 200 km square) by averageing all stations within 1200 km with each station weighted by its distance fron the grid centre. Then an anomoly is calculated for the grid point.
Now it seems to me that, whilst the trends of both rural and urban stations might be the same, the changing proportions can effect the overall trend.
Consider two sets of stations contributing to a grid point:
Station set A: Rural. 75% of total affecting grid point in 1900, 50% in 2000. Temp in 1900 is 10 deg C. Temperature in 2000 is 10 deg C. Adjustments are zero.
Station set B: Urban. 25% of total affecting grid point in 1900, 50% in 2000. Measured temp in 1900 is 10 deg C. Measured temp in 2000 is 12 deg C due to UHI. UHI adjustment raised temperature in 1900 to match rural trend. i.e. adjusted temp for 1900 is also 12 deg C (because rural trend is 0).
Then calculate the grid point temperature in 1900 and 2000 (assuming equal distances from drid point).
1900: Temp = 10 * .75 + 12 * .25 = 10.5
2000: Temp = 10 * .5 + 12 * .5 = 11
Haven’t I just created a 0.5 deg per century temperature trend even though the trends of adjusted temperatures for both rural and urban stations were zero?
Nick Stokes (02:22:28) :
You seem to be missing the point, Nick. Creating anomalies and grid averages does not necessarily eliminate bias. It depends significantly on how you do both of those procedures. EM has described that quite clearly, but you seem either to not understand that or, at least, you do not acknowledge the point.
More importantly, EM is doing what GHCN and GISSTEMP should have done: eliminate bias in the raw data first, then compute anomalies and gridded averages.
And, oh, one more thing: GHCN and GISSTEMP should have made all their raw data and adjustment/ conversion logic and code publicly available so people (scientists and others) could examine whether what they did is reasonable or not.
While he has not completed the global analysis yet, the early returns (e.g., Pacific Basin) suggest the “warming trend” of the last ~30 years is an “anomaly” of the raw temperature data machinations. One hopes he submits this work to a review by unbiased climate scientists (if there is such a thing) and statisticians to get independent judgment as to whether what he has done holds up to serious — not pejorative — scrutiny.
Kinda like science is supposed to be done…
Re Nick Stokes (02:22:28) :
I understand that E.M. Smith and others disagree with you and maintain that the anomalies of the dropped stations in a grid are, or can be affected on a time specific bases.
But I would like to leave that alone for the moment. I have see in public literature the claim made that the world mean temperature has moved from this, pick a number, to this, pick a higher number, and so for P.R. purposes it appears some times the mean is used.
But my real question (until the rest is resolved, and I do not think it it yet resolved) is, why use a method where the mean is distorted? Are we incapable of both?
Tips & Notes page opens ok but I can find no way of putting any text into it. Am using IE8
“twawki (03:01:33) :
But with less thermometers doesn’t the margin of error increase statistically”
Yes, it does but it not a linear effect. Going from 7000 to 1000 does increase your random error by 7 fold only 2.6. Depending on the level of significance you were shooting for and the error range of the data itself, you may go from an error of say 0.11 °C to 0.31°C. this is just for random errors in the data and does not address bias issues.
Re Mike McMillan (02:02:16) :
Your blink charts are very impressive and warrants more commentary. (a great deal more) The lowering of the past appears the strongest.
Can you give a quick summary of how you know this is USHCN original raw data vs USHCN version 2 revised raw data, and what GISS does with this data?
Re: vjones (Mar 8 03:30),
He is doing this PRIOR to working out HOW WELL GIStemp copes with these biases.
Well, I kmow what the report has been saying. Summary point 5:
There are two issues here:
1. What has E.M Smith been saying.
2. What have others been saying based on their understand of what E.M Smith has been saying.
Even if Smith has been reasonable it does not follow that those who have been basing claims on his work/their understanding of his work have also been reasonable.
Anthony’s article goes some way to addressing point 1 but not point 2, though as he says, a response will be forthcoming. And that will be interesting to read.
Toyota,
You do realize don’t you, that you don’t need a thermometer at every square inch of the planet to get regional (or grid) averages, don’t you?
Oh yeah, the average temperature of the planet can increase, without ever breaking a daily record.
Just my 2 cents worth could it be that the loss of stations is a result of the cold war ending?? SAC (Strategic Air Command) had bases and weather stations all over the world but mainly focused on the northern hemisphere. With the fall of the USSR we no longer needed super accurate weather charts to predict things like fallout patterns from a global thermonuclear war.