A Smoldering Gun From Nashville, TN
Guest post by Basil Copeland
The hits just keep on coming. About the same time that Willis Eschenbach revealed “The Smoking Gun at Darwin Zero,” The UK’s Met Office released a “subset” of the HadCRUT3 data set used to monitor global temperatures. I grabbed a copy of “the subset” and then began looking for a location near me (I live in central Arkansas) that had a long and generally complete station record that I could compare to a “homogenized” set of data for the same station from the GISTemp data set. I quickly, and more or less randomly, decided to take a closer look at the data for Nashville, TN. In the HadCRUT3 subset, this is “72730” in the folder “72.” A direct link to the homogenized GISTemp data used is here. After transforming the row data to column data (see the end of the post for a “bleg” about this), the first thing I did was plot the differences between the two series:
The GISTemp homogeneity adjustment looks a little hockey-stickish, and induces an upward trend by reducing older historical temperatures more than recent historical temperatures. This has the effect of turning what is a negative trend in the HadCRUT3 data into a positive trend in the GISTemp version:
So what would appear to be a general cooling trend over the past ~130 years at this location when using the unadjusted HadCRUT3 data, becomes a warming trend when the homogeneity adjustment is supplied.
“There is nothing to see here, move along.” I do not buy that. Whether or not the homogeneity adjustment is warranted, it has an effect that calls into question just how much the earth has in fact warmed over the past 120-150 years (the period covered, roughly, by GISTemp and HadCRUT3). There has to be a better, more “robust” way of measuring temperature trends, that is not so sensitive that it turns negative trends into positive trends (which we’ve seen it do twice how, first with Darwin Zero, and now here with Nashville). I believe there is.
Temperature Data: Pasteurized versus Homogenized
In a recent series of posts, here, here, and with Anthony here, I’ve been promoting a method of analyzing temperature data that reveals the full range of natural climate variability. Metaphorically, this strikes me as trying to make a case for “pasteurizing” the data, rather than “homogenizing” it. In homogenization, the object is to “mix things up” so that it is “the same throughout.” When milk is homogenized, this prevents the cream from rising to the top, thus preventing us from seeing the “natural variability” that is in milk. But with temperature data, I want very much to see the natural variability in the data. And I cannot see that with linear trends fitted through homogenized data. It may be a hokey analogy, but I want my data pasteurized – as clean as it can be – but not homogenized so that I cannot see the true and full range of natural climate variability.
I believe that the only way to truly do this is by analyzing, or studying, how differences in the temperature data vary over time. And they do not simply vary in a constant direction. As everybody knows, temperatures sometimes trend upwards, and at other times downward. The method of studying how differences in the temperature data allows us to see this far more clearly than simply fitting trend lines to undifferenced data. In fact, it can prevent us from reaching the wrong conclusion, as in fitting a positive trend when the real trend has been negative. To demonstrate this, here is a plot of monthly seasonal differences for the GISTemp version of the Nashville, TN data set:
Pay close attention as I describe what we’re seeing here. First, “sd” means “seasonal differences” (not “standard deviation”). That is, it is the year to year variation in each monthly observation, for example October 2009 compared to October 2008. Next, the “trend” is the result of smoothing with Hodrick-Prescott smoothing (lamnda = 14,400). The type of smoothing here is not as critical as is the decision to smooth the seasonal differences. If a reader prefers a different smoothing algorithm, have at at it. Just make sure you apply it to the seasonal differences, and that it not change the overall mean of the series. I.e., the mean of the seasonal differences, for GISTemp’s Nashville, TN data set, is -0.012647, whether smoothed or not. The smoothing simply helps us to see, a little more clearly, the regularity of warming and cooling trends over time. Now note clearly the sign of the mean seasonal difference: it is negative. Even in the GISTemp series, Nashville, TN has spent more time cooling (imagine here periods where the blue line in the chart above is below zero) than it has warming over the last ~130 years.
How can that be? Well, the method of analyzing differences is less sensitive – I.e. more “robust” — than fitting trend lines through the undifferenced data. “Step” type adjustments as we see with homogeneity adjustments only affect a single data point in the differenced series, but affect every data point (before or after it is applied) in the undifferenced series. We can see the effect of the GISTemp homogeneity adjustments here by comparing the previous figure with the following:
Here, in the HadCRUT3 series, the mean seasonal difference is more negative, -0.014863 versus -0.012647. The GISTemp adjustments increases the average seasonal difference by 0.002216, making it less negative, but not enough so that the result becomes positive. In both cases we still come to the conclusion that “on the average” monthly seasonal differences in temperatures in Nashville have been negative over the last ~130 years.
An Important Caveat
So have we actually shown that, at least for Nashville, TN, there has been no net warming over the past ~130 years? No, not necessarily. The average monthly seasonal difference has indeed been negative over the past 130 years. But it may have been becoming “less negative.” Since I have more confidence, at this point, in the integrity of the HadCRUT3 data, than the GISTemp data, I’ll discuss this solely in the context of the HadCRUT3 data. In both the “original data” and in the blue “trend” shown in the above figure, there is a slight upward trend over the past ~130 years:
Here, I’m only showing the fit relative to the smoothed (trend) data. (It is, however, exactly the same as the fit to the original, or unsmoothed, data.) Whereas the average seasonal difference for the HadCRUT3 data here was -0.014863, from the fit through the data it was only -0.007714 at the end of series (October 2009). Still cooling, but less so, and in that sense one could argue that there has been some “warming.” And overall – I.e. if a similar kind of analysis is applied to all of the stations in the HadCRUT3 data set (or “subset”) – I will not be surprised if there is not some evidence for warming. But that has never really be the issue. The issue has always been (a) how much warming, and (b) where has it come from?
I suggest that the above chart showing the fit through the smooth helps define the challenges we face in these issues. First, the light gray line depicts the range of natural climate variability on decadal time scales. This much – and it is very much of the data – is completely natural, and cannot be attributed to any kind of anthropogenic influence, whether UHI, land use/land cover changes, or, heaven forbid, greenhouse gases. If there is any anthropogenic impact here, it is in the blue line, what is in effect a trend in the trend. But even that is far from certain, for before we can conclude that, we have to rule out natural climate variability on centennial time scales. And we simply cannot do that with the instrumental temperature record, because it isn’t long enough. I hate to admit that, because it means either that we accept the depth of our ignorance here, or we look for answers in proxy data. And we’ve seen the mess that has been made of things in trying to rely on proxy data. I think we have to accept the depth of our ignorance, for now, and admit that we do not really have a clue about what might have caused the kind of upward drift we see in the blue trend line in the preceding figure. Of course, that means putting a hold on any radical socioeconomic transformations based on the notion that we know what in truth we do not know.
Basil, I think we need to be more certain about the raw data. You suggest that you do not think using raw data is reasonable and that it is not necessary, if I read you correctly.
Basil 6:28:57 “For all the talk about going back to the “raw” data, I don’t think that is where the problem begins. From my work with US data (I do some consulting work where I have occasion to look at the truly “raw” data occasionally), NOAA does some “quality” control right off the bat in reading from the hand written daily records. I doubt that any systematic “warming bias” is introduced at that point.”
I just can’t agree. NOAA’s “‘quality’ control right off the bat” must be checked by truthful citizens/scientists — perhaps at a sample of stations to verify that the raw data is not already cooked. Why would you “doubt” that any systematic “warming bias” is/has been introduced when this is the most significant scientific scandal of our time plus one that already has cost us billions, if not trillions — perhaps even quadrillions — of dollars during the last 10 years? Do you think these people are going to let this go easily?
“I have taken the difference between tmax and tmin for each month and averaged them over the year.” – oldgifford
Surely you need to take the average of tmax and tmin for each month (ie. sum not difference) and then average? Otherwise you get a graph of temperature range not temperature amplitude.
Of course if tmax-tmin does decline over time, what that might show is the the development of an urban heat island effect, as warm nights is one normal result of urbanisation.
None of this matters. The power-hungry in Denmark are going to make Global Governance happen through climate change means, unless we stop them with something close to a revolution.
They’re literally daring us at this point.
Whoops, inverted my data, still working on it.
“From another thread thiss seems an extremely important analysis:
http://www.gilestro.tk/2009/lots-of-smoke-hardly-any-gun-do-climatologists-falsify-data/
The adjustments over the whole GHCN series add up to
NO BIAS” – bill
An interesting analysis, worrying for our “smoking guns” if true. But is he really analysing “raw” data, as I thought they were only releasing the adjusted stuff? If the data was this available, why have they been working so hard to avoid releasing it? I’m also unclear how you analyse an adjustment in terms of its effect on trend.
Would You Like Your Temperature Data Homogenized, or Pasteurized?
I’d like it explained before Congress.
I prefer my data shaken, not stirred.
Basil 6 44 43
Couldn’t agree more. I have now read some thirty studies on UHi and looked at the Real Climate and IPCC take on this.
As far as I can see they really downplay the effect of UHI as they claim that averaged over the whole globe it is negligible (which is true) However that misses the point that UHI is very important in urban areas which covers an increasingly large percentage of the temperature database.
I think we shoud be supplied raw data then UHi applied individually according to the circumstances of the urbanisation. Clearly a station set in a large park is going to be affected completely differently to a station at a busy airport or in a city centre that 100 years ago was a field at edge of town.
I believe the Met office adds in UHI from 1975, but despite my asking have never told me what factor they use-perhaps it is buried in their web site somewhere.
Some of the claims made in the studies of UHI do in my opinion sometimes overstate the case. A station set in a large city will see warming up to a point then the additional urbanisation is likely to mean the warmth is felt over a wider area rather than become more concentrated.
However certain conditions -like clear still nights- will undoubtedly create a greater uhi effect than in other circumstances like windy weather.
All in all though the uhi effect is noticeable and not really factored in to anythig like the degree it should be (pun intended!)
Tonyb
on above post of mine
The code does not translate to wordpress very well
Any red marked line:
single quote, replace with single quote from keyboard – its a comment so you can delete!
question mark replace with double quote from keyboard
double quote replace with double quote from keyboard
If you ever end up with a correctly formatted column of monthly data you will need to remove all error indicators.
Mark the data column
on [data] tab select filter
on first line of column click the down arrow
deselect all (click the ticked select all box)
look through offered numbers for error indicators “-” “-9999.9” etc.
click the boxes associated with the error indicators
press[ok]
only the data in error is now shown
mark it all and press be careful of the first box as this is patially obscured by the arrow [delete]
Turn off filter by clicking filter in the ribbon again
Data is now clean
I data shows temp*10
the adjacent (right) to the first temp type
= [click temperature to left to enter cell]/10
mark all column including this cell to last row containing temperature
from [home] tab click fill then select fill down
This column now contains correctly scaled temperature but the cell contents are actually formulae.
with column marked copy column [ctrl]+c
right click first cell of this column and select paste special
select values only then ok it
The column now contains actual values and can therfore be copied to another sheet with dates in decimal format i.e. year+(month-1)/12. Note that excel does not like true dates before jan 1st 1900
As an old instrumentation guy, I am always uncomfortable with folks attempting to describe how they use real world temperature measurements to determine trends of only a degree or two over a century. The surface temperature record begins with instruments delivered by horse drawn wagon with calibration accuracy of only a degree or two. Modern air temperature measurement equipment typically has a calibration accuracy of plus or minus 0.5 degrees. Essentially no calibration checks are done over time. Worse is the siting issue.
For sea surface temperature reading, picture some poor fellow on the deck of a rolling wooden sailing vessel attempting to read, by the light of a swaying oil lamp, a thermometer he just pulled out of a bucket of sea water. That’s how it was done a century or so ago. Land temperature is even stranger.
Water is pretty much water but land changes day by day. Trees grow, buildings come and go, measuring station locations have to be changed. What may have started as instrument accuracy of 0.5 degrees is impacted by changes in its local environment. The WUWT temperature site survey shows just how poorly the resultant readings may reflect actual local temperatures and any trends noticed in their data. Having instruments with ten times the internal accuracy would not improve the accuracy of our air temperature readings.
The best any instrument guy would claim would be a trend that shows up as three times the calibration accuracy, and many would demand ten times. And this accuracy must include the error associated with poor siting and instrument reading interpolation. Suggesting that simply smoothing the data will increase its accuracy a logical fallacy. This would be based on the assumption that all errors in measurement are simply random noise. That we are attempting to adjust for instrument changes and UHI should make it obvious that there is much more going on than random noise.
In my judgment, we have enough evidence in the form of freeze/thaw date changes along with changes in plant and animal life ranges to know some amount of warming has occurred since we have bothered to take records. I do not believe, however, that our long (and short) term temperature measurement and estimation methods provide sufficient accuracy to quantify it.
Now, some of you may be thinking about the sophisticated signal analysis done on radio signals. We are able to dig out signals buried deep in noise. You might wonder why if we can do that, why that same technique would not work for temperature measurements. The difference, of course, is that for those radio signals, we know the exact nature of the noise and the nature of the highly repetitive signals we are working on. We use algorithms that are designed to find that specific repetitive signal in the presence of noise. Think for the moment what that means for temperature trends.
With temperature trends, we are looking for a non-repetitive signal in “noise” that we cannot reasonably characterize. The result, of course, is that if we build a signal processing algorithm that looks for a specific trend, it will probably find it. That appears to have been the case with the recent tree ring episode. Eliminate the data sets that do not match an expected pattern from the past, claiming the remaining data sets have proven to be accurate by that standard, and then claim that some non-temperature factor destroyed their accuracy after the matching date range so the later values are inaccurate. That is enough to make any engineer or technician shudder.
Anyway that’s my opinion looking at this discussion from the perspective of an instrumentation tech, which I was many years ago.
Just imagine what that poor bloke standing there in an oil skin rain coat on the deck of his 19th century sailing ship would think if you told him that some time in the future his temperature reading would be interpreted to the nearest one thousandth of a degree!
What do you make of this?
http://www.informationisbeautiful.net/2009/the-climate-deniers-vs-the-consensus/
Nick Stokes (05:05:39) :
It is very hard to tell how skewed that distribution is in that picture, but if you look at the x-axis, you can see how adjustments extend a good deal further to the positive side.
Isn’t the HadCRUT3 data already “adjusted”? If so, it would certainly be interesting to do a similar comparison to data that hasn’t even been touched.
According to NOAA, the largest adjustment in the USHCN data (the US part of the Global Historical Climate Network, which the basis for GISTemp) is time of day adjustments. These adjustments also have the strongest hockey stick shape, especially since the 1950’s.
The black line here is the time of day adjustments. Yellow is the slightly less hockey stick shaped homogenization adjustments. Source with explanations here.
Presumably NOAA has the adjustment data for each station. Thus for the Nashville station, for instance, it should be possible to find out WHEN the time of day adjustment was made, to see how much sense this makes of the adjustment record. Perhaps it accounts for the big mid-60’s adjustment, which might on that grounds be perfectly legitimate.
Time of day adjustments may themselves turn out to be a source of manipulation, but at least they are an adjustment factor that COULD properly move in a systematically warming direction, unlike altitude adjustments (which should tend to cancel each other out over large numbers) or UHI adjustments (which should be downwards).
Quite brilliant fun and useful post – build your own climate change model
http://iowahawk.typepad.com/iowahawk/2009/12/fables-of-the-reconstruction.html
Out of curiosity I downloaded the station data from the Met Office Website for two England stations with long spans of data.
http://www.metoffice.gov.uk/climate/uk/stationdata/
Here are the averages for Durham and Oxford, the slope of the temperature increase varies depending on where you take the snapshot. What we seem to have is a temperature rise over the period of about 2 deg C but apparently starting to downturn. Time will tell.
I live in a small UK village, population about 250, two days ago I drove my mini cooper early evening from our village, the warning alarm came on showing 3 degrees C, 15 minutes later I entered the small town of Mansfield, population 100,000, the temp reading now was 5 degree C, seems the UHI effect is quite significant.
Good piece of work Basil, thanks very much.
The ‘raw data’ is the most revealing, and shows what I think most people already knew – temperature is always going up and down over time. It is interesting to see that during an interglacial, the Earth’s homeostat is successful in keeping the system in balance to a fraction of a degree Celsius. I wish my central heating thermostat was half as good as this.
I suspect that if we could go back and examine the real raw data set used for global temperature trends a similar result would be found. Unless, of course, Nashville doesn’t respond to the higher levels of CO2 like the rest of the world.
The hypothesis of CAGW is brain dead, It’s time to turn off the life support machine before any further harm is done.
Graphs at
http://www.akk.me.uk/Climate_Change.htm
Interesting link there, Plato Says (09:15:47).
It reminded me of IowaHawk’s fascinating nature narrative on the secret life of climate researchers, posted here a while back: click
Seth (07:42:33)
Does anyone know if there has been a large scale attempt to collect all ships logged weather data?
Yes, NOAAs’ National Oceanographic Data Center has the data. see http://www.nodc.noaa.gov/General/NODC-Archive/bt.html about bathythermograph data, specifically.
and http://www.nodc.noaa.gov/General/getdata.html for a list of what data is available (online anyways)
Thom (09:05:19) :
Here is an interesting piece of logic:
“Historically, global warming cycles have lasted 5,000 years. The 800 year CO2 lag only shows that CO2 did not cause the first 16% of warming. The other 4,200 years were likely to have been caused by a CO2 greenhouse effect.”
It is “obvious” that after the original warming (unexplained) that CO2 causes 4,200 years of warming? How? Also, if you cannot account for a natural process, how do you discount it?
And the piece about reliable temperature records? Well, ‘detailed filters’ seems to be what this post was about, no? I think there is enough evidence that the record is shaky to cast doubt on the claims of ‘unequivocal’ evidence of ‘unprecedented’ and ‘catastrophic’ warming. Doomsaying is not helpful to the science underlying the case for man made global warming, and neither is the constant stream of media sensationalism about the aforementioned doomsmanship. Yes, I made that word up.
Thom (09:05:19) :
Also precious is this assertion:
“And we haven’t even gotten to the stage where the oceans warm up.”
10,000ish years is not long enough to warm the oceans? Instead it is going to happen in the next 100? Huh? Are we in the magical 1% of the interglacial where previous natural processes do not apply?
You can fool 1% of the people 100% of the time, 100% of the people 1% of the time, but not 100% of the people 100% of the time.
Bad data is worse than no data!
At least with no data you know what you don’t know!
Larry
Let me get this straight: HadCRUT3 is based on real temperature measurements (with likely UHI effects), but has “corrections” and “homogenizations” in it already — probably including extension of measured data into unmeasured areas of the earth, correct?
Does GISTemp use HadCRUT3 and make more corrections to it? Or do GISTemp and HadCRUT share a root data set?
I would sure like a map to all of this. Maybe a little flow chart showing the Global Warming creation process.