I’ve been away from WUWT this weekend for recovery from a cold plus family time as we have visitors, so I’m just now getting back to regular posting. Recently on the web there has been a lot of activity and discussions around the issue of the dropping of climatic weather stations aka “the march of the thermometers” as Joe D’Aleo and I reported in this compendium report on issues with surface temperature records.
Most of the station dropout issue covered in that report is based on the hard work of E. M. Smith, aka “chiefio“, who has been aggressively working through the data bias issues that develop when thermometers have been dropped from the Global Historical Climate Network. My contribution to the study of the dropout issue was essentially zero, as I focused on contributing what I’ve been studying for the past three years, the USHCN. USHCN has had a few station dropout issues, mostly due to closure, but nothing compared to the magnitude of what has happened in the GHCN.
That said, the GHCN station dropout Smith has been working on is a significant event, going from an inventory of 7000 stations worldwide to about 1000 now, and with lopsided spatial coverage of the globe. According to Smith, there’s also been an affinity for retaining airport stations over other kinds of stations. His count shows 92% of GHCN stations in the USA are sited at airports, with about 41% worldwide.
The dropout issue has been known for quite some time. Here’s a video that WUWT contributor John Goetz made in March 2008 that shows the global station dropout issue over time. You might want to hit the pause button at time 1:06 to see what recent global inventory looks like.
The question that is being debated is how that dropout affects the outcome of absolutes, averages, and trends. Some say that while the data bias issues show up in absolutes and averaging, it doesn’t effect trends at all when anomaly methods are applied.
Over at Lucia’s Blackboard blog there have been a couple of posts on the issue that raise some questions on methods. I’d like to thank both Lucia Liljegren and Zeke Hausfather for exploring the issue in an “open source” way. All the methods and code used have been posted there at Lucia’s blog which enables a number of people to have a look at and replicate the issue independently. That’s good.
E.M Smith at “chiefio” has completed a very detailed response to the issues raised there and elsewhere. You can read his essay here.
His essay is lengthy, I recommend giving yourself more than a few minutes to take it all in.
Joe D’Aleo and I will have more to say on this issue also.
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
It could be that the temperatures are absolutely stable and anomalies show heating because of peculiar redistributions of heat, (PDO, ENSO etc. etc) as seen in http://nsidc.org/images/arcticseaicenews/20100303_Figure4.png .
Sorry, this makes little sense. My only excuse is it is close to my bedtime 🙂 .
I meant that the heat input output is absolutely stable over the planet but anomalies show heating because of peculiar redistributions of heat, (PDO, ENSO etc. etc) as seen in http://nsidc.org/images/arcticseaicenews/20100303_Figure4.png .
Re: Frank K. (Mar 8 06:32),
As to the ideal gas law, the Charles Law part was worked out in the eighteenth century by scientists using Celsius. But yes, in thermo it’s more natural to use Kelvin.
My point was that a lot of thermo is about heat fluxes and changes in heat content. That’s the case in atmospheric science. And you can measure that with K, C, F or anomalies.
David A (04:27:37) :
Re Mike McMillan (02:02:16) :
Your blink charts are very impressive and warrants more commentary. (a great deal more) The lowering of the past appears the strongest.
Can you give a quick summary of how you know this is USHCN original raw data vs USHCN version 2 revised raw data, and what GISS does with this data?
Last summer (’09) I downloaded all the Raw and all the Homogenized charts from
http://data.giss.nasa.gov/gistemp/station_data/
for the states of Illinois, Wisconsin, and Iowa. I made raw/homogenized blink charts, and have been posting them on surfacestations.org’s gallery as the stations were surveyed. Since I had the old raw charts when the “new improved” raw charts came out, I downloaded the new ones and made the raw/new-raw blinks.
What I’ve noticed while gluing the charts together was that the temperature peak in the 1930’s was nearly always lowered, and the 1998 peak was often adjusted lower, but Not lower than the 1930’s peak.
I’m guessing that across the USHCN, that adjustment results in 1998 being hotter than the 1934 record, but no longer so high that subsequent years couldn’t beat it. After all, you can’t have global warming if you can’t break old records.
The new GISS homogenized charts that I’ve examined show even more warming, but the difference between new raw and homogenized charts is much less than between the original raw and homogenized.
The removal of the original USHCN raw charts makes it much more difficult to expose the computer-induced warming that’s going on.
HAS (11:01:40) :
But if they can’t use “Adjustments”, “homogeneousing”, “Anomolies” and “Gridding” they wouldn’t be able to “Hide the Decline” with their Computer Programs.
carrot eater (10:54:41) : – “Now, some parts of the earth were possibly undersampled”
Like no thermometers at all even though they are still there and still providing valuable data?
carrot eater (10:31:57) : And this is exactly what we’re doing here – seeing how much a temperature has risen or dropped over time.
You’re wrong, we are looking at the raw data, and how it can bias any further analysis, even extracting your beloved ‘anomaly’.
“The question that is being debated is how that dropout affects the outcome of absolutes, averages, and trends.”
Wouldn’t one easy test be to just use the period when there are up to 7000 stations and then calculate the average over that time?
After that you could simulate a station drop out during the exact same period and calculate averages again.
That would give a good hint imo what the station drop out may do to the meassured global temperatures.
Re: PeterB in Indainapolis (Mar 8 11:43),
Yes, removing thermometers that are colder than the grid cell average, as I said, will make the grid cellaverage temperature colder. Mountain sites are likely to be in that category. But, as I also said, there are two protections against bias. Gridding doesn’t work so well in that case, but anomalies will.
I am doing a lot of reading on the Younger-Dryas Impact Event, which – like the dinosaur killer of 65 million BC – seems to have killed all the megafauna in North America. It is still in dispute as to whether that is real or not (I think it is very likely true).
1989 seems to have been hit by an extinction event, with a lesser one just recently.
I love the term, “The Great Dying of the Thermometers.” It fits in well with what else I am into.
In an airport on the way back to NYC at the moment, so I probably can’t contribute as much to this post as I should.
A few quick points:
1) The strong correlation between temps reconstructed based on only “dropped” and only continuous stations pre-1992 should lay to rest any claims of fraud or manipulation on the part of NOAA/NCDC. The fact that “dropped” stations show a greater warmer trend on average than continuous stations is actually consistent with a drop-out of colder thermometers, since colder places tend to have a greater warming trend on average.
Pre-cutoff and post-cutoff stations: http://rankexploits.com/musings/wp-content/uploads/2010/03/Picture-98.png
Post-cutoff minus pre-cutoff: http://i81.photobucket.com/albums/j237/hausfath/Picture170.png
2) The whole notion of station “dropping” is something of a misnomer, since those stations were never actively reporting. GHCN was put together in the early 1990s from station data collected via various projects; only the 1300 or so Global Surface Network stations ever provided regular monthly updates via CLIMAT reports. However, GHCN version 3 is being developed as we speak, and should hopefully be released next year. It will collect station records from those stations not in the GSN system that had contributed data to GHCN version 2, as well as new stations established in the interim.
3) Anomalies are essential to properly calculating trends in global temps. The methods used (SAM, CAM, RSM, and FDM) each have their small differences (see Chad’s posts related to them, and Lucia’s recent analysis of the RSM used by GISS). However, all anomaly methods give almost the same trend globally; the major difference is in dealing with fractional station records.
4) The importance of gridding depends largely on how big and representative a set of temp data you are looking at. For GHCN raw data globally, its actually not that important, since GHCN by its nature attempts to select a geographically distributed and proportionate set of stations. If you were to add in all the USHCN stations to the GHCN network, the effects of gridding would be much more obvious.
Here are two graphs I made over at Lucia’s place (see the posts there for the details of the CAM and gridding processes used):
Gridded vs. non-gridded global anomalies via GHCN v2.mean (raw): http://i81.photobucket.com/albums/j237/hausfath/Picture169.png
Gridded vs. non-gridded absolute temps via GHCN v2.mean (raw):
http://i81.photobucket.com/albums/j237/hausfath/Picture168.png
If you plot the absolute temps with the number of stations available, you can see obvious step-changes in absolute temps associated with changes in station number:
http://i81.photobucket.com/albums/j237/hausfath/Picture166.png
However, anomalies appear largely insensitive to these changes.
Re Nick Stokes (13:04:02) :
Chiefio has also maintained in other posts that all of the cold temperatures are in the baseline period, though, so removing them from the later record but not from the baseline does create a warm bias regardless of whether you use anomalies and gridding or not.
I didn’t do that last post very well–didn’t indent. Nick’s comments are the first paragraph, my reply is the second. Sorry about that.
[ Fixed. “Friendly -mod.” ]
geo
#5 No one has proven AGW in the first place, so temperatures mean nothing.
Figure out everything else first, then see if any trace gasses have any effect.
Re: Nick Stokes (Mar 8 02:22),
Thanks Nick, well put.
There are a few things that people need to realize. Every ounce of effort you expend on problems that are not problems is a waste of effort.
Here are a list of non problems:
1. Station drop out ( except WRT overall certainty if the station count
goes too low
2. Rounding ( please see Lucias site)
3. Thermometer accuracy ( please see the law of large numbers)
4. using (Tmax+Tmin)/2 (go download data from CRN and see for yourself)
5. The notion that “averaging” results in the loss of data.
6. The selection of “base periods”
7. The colors used in charts.
here are a list of Open questions ( not problems, but open questions)
1. What is the provenance of the data being used.
2. What adjustments are made and what are the exact calculations.
3. Is there UHI contamination in the signal
4. Does microsite bias matter? how much
5. How should the uncertainty due to spatial coverage be computed?
6. What is the optimal method for station combining and area averaging
( see romanM)
5&6 are methodological questions not specific to climate science.
1&2 should have been addressed long ago. They are record keeping
and statistical questions.
3&4: these questions touch on climate science
without 1 &2 and 5&6 being put to bed, I would question
work on 3&4. By question I mean hold open.
So, just to set priorities. also. 5&6 may depend upon the characteristics of 1&2. understanding the quality of your data and the various kinds of holes.
What do you think Nick?
All this mathematical finagling is dodgy, and is in the process of being revealed as mere cleverity
And so certain Climate Scientists ought to dwell on what Francis Bacon has to say about the uses of mathematics;
mathematics …. ought only to give definiteness to natural philosophy, not to generate or give it birth. From a natural philosophy pure and unmixed, better things are to be expected.
……. inquiries into nature have the best result when they begin with physics and end in mathematics.
Nick Stokes (13:04:02) :
Re: PeterB in Indainapolis (Mar 8 11:43),
Yes, removing thermometers that are colder than the grid cell average, as I said, will make the grid cellaverage temperature colder. Mountain sites are likely to be in that category. But, as I also said, there are two protections against bias. Gridding doesn’t work so well in that case, but anomalies will.
Do you believe what you write????
Hypothetical three stations in same grid;
1. ave temp 25
2. ave temp 26
3. ave temp 27 = ave ann temp in grid =26
remove station 1
ave ann temp = 26.5
Nick Stokes – the other thing one has to remember with regard to the way GHCN uses gridcells is the fact that the temperature numbers are adjusted based upon the temperatures from any other stations within 1200 kilometers. This means that the gridcell containing Boston, Massachusetts, is affected by the temperature of the gridcell containing Atlanta, Georgia. While there would be a mitigating factor if there were temperatures from more Northernly gridcells, the higher latitude sites are decreasing.
Which means the lower (and warmer) latitude sites are effecting the higher latitude sites more than the reverse. If you ever check an anomaly map, the areas with the greatest warming anomalies have been the higher latitudes over the past decade.
Re: Jerker Andersson (Mar 8 13:03),
in fact a couple people have done this test over at Lucias.
1. Zeke
2. Lucia
3. the CCC folks.
They all confirm what the basic math suggests. The methods are robust to station removal. I know it seems counter intuitive. Back in 2008 when this issue was first raised I thought the same thing. but guys at CA quickly smacked me down. ( thanks guys)dropping cold stations would change everything. But it doesnt. Lets see if I can do a very simple example.
heck maybe I can even find the comment in CA where I did it.
Ok Lets make two stations: one hot; one cold. and lets make them
cool over time: ready?
10,9,8,7,6,5,4,3,2,1
20,19,18,17,16,15,14,13,12,11
Now lets do the average in RAW TEMP:
15,14,13,12,11,10,9,8,7,6
Ok notice something? ( look at the rate )
Now Lets lose some data! pretend Phil jones is in charge of things
and we lose the cooler.
10,9,8,7,6,5, 4
20,19,18,17,16,15,14,13,12,11
Now average:
15,14,13,12,11,10,9,13,12,11
DANG! if I work in raw temps and lose stations my average gets messed up.
Spherical cow to the rescue. lets use the dreaded anomaly.
10,9,8,7,6,5,4,3,2,1
20,19,18,17,16,15,14,13,12,11
base period for series 1 {6,5,4} average = 5
base period for series 2 {16,15,14} average = 15
Anomalize
5,4,3,2,1,0,-1,-2,-3,-4
5,4,3,2,1,0,-1,-2,-3,-4
Now, have Dr Jones lose data
5,4,3,2,1,0,
5,4,3,2,1,0,-1,-2,-3,-4
Average the anomalies.
you can do that math
example for you
For those of you who actually think temperature is a good metric of energy, try this experiment that many women the world over do every day. (As a single parent I did too.)
1) Pre-heat an oven to 200-250ºC.
2) Without protection, place the item you want to cook in the oven.
See! No damage!
Now carry out the following experiment.
Place your hand in the stream of 100ºC steam venting from the pot on the hotplate.
These experiments must be performed in this order as the delay after the visit to the burns unit if performed in the wrong order may render the results invalid.
DaveE.
Re Nick Stokes (02:22:28), having read E.M. Smith’s article, I see now that Stokes is referring to the “hypothetical cow” of how GisTemp is supposedly produced, whereas Smith discusses the “real cow” of how it’s actually produced. Smith hasn’t completely dissected this process, but is far enough along to see that:
1) The input data are biased
2) Homogenization, correction, etc. are performed on the input data before anomaly calculation
Later steps in the GisTemp process may or may not offset the second problem, and that’s what he’s still looking at.
Fair summary?
Re: Steve Keohane (Mar 8 12:30),
Steve.
Just to help you understand anomalies.
5,5,5,4,4,4,5,5,5
Now lets pick a base period {4,4,4} average = 4
Anomalize!
1,1,1,0,0,0,1,1,1
Note I picked a cool base. See how the pattern of the data doesnt
change. the early trend is zero, mid trend is zero, late trend is 0.
the overall trend from start to finish is the same too!
Now pick a different base the first period:
0,0,0,-1,-1,-1,0,0,0
What do you see? anomalies change nothing. and picking a cool period
isn’t any great evil deed. I can always shift to a warm baseline. It’s just
addition.
Re: carrot eater (Mar 8 07:50),
Indeed you would. Which is exactly the point of the analysis of Zeke, clear climate code, and Tamino. Before the time of the station number drop, the global trends calculated from the dropped stations and the surviving stations are the same.
Actually I think Zeke’s analysis showed a minor difference
I have just watched a film called ‘The End Of The Line’, about overfishing round the world. Researchers were puzzled that although fishermen in all areas reported lower catches, the annual total was going up, which made no sense. They traced the anomaly to China where they found that the Chinese government were paying their officials based on the amount of fish caught, so these officials were making the figures up! Human nature.
The CRU, NOAA and GISS were formed to measure global warming. If there is no global warming then they cease to have a purpose, their functions could be absorbed elsewhere [eg. CRU in to the UK Met Office, and NOAA in to the real NASA to augment the satellite data.]
We of course expect the people in these organisations to be honorable, and to dutifully report global cooling if it occurs, even if it means them losing their jobs!
[……but, human nature!]
Re: Medic1532 (Mar 8 04:46),
Nick stokes does a fair explanation of why the number of stations dropped?
Nick?
Near-surface air temperature is but one of the met. records. Why not use the others?
Would it be useful to know if the ‘global average’ windspeed has changed? Wind direction? And then how about rainfall? Cloud-cover? Insolation?
It seems such a ‘no-brainer’ to use these other indicators to see if the climate is changing, so can any climatologist tell me why they are not being used?