NOTE: Please note that part 2 is now online, please see it here.
I recently plotted all four global temperature metrics (GISS, HadCRUT, UAH, RSS) to illustrate the magnitude of the global temperature drop we’ve seen in the last 12 months. At the end of that post, I mentioned that I’d like to get all 4 metrics plotted side-by-side for comparison, rather than individually.
Of course I have more ideas than time these days to collate such things, but sympathetic reader Earle Williams voluntarily came to my rescue by collating them and providing a nice data set for me in an Excel spreadsheet last night.
The biggest problem of course is what to do with 4 different data sets that have different time spans. The simplest answer, at least for a side by side comparison is to set their time scales to be the same. Satellite Microwave Sounder Unit (MSU) data from the University of Alabama, Huntsville (UAH), and Remote Sensing Systems (RSS) of Santa Rosa, CA only go back to 1979. So the January 1979-January 2008 period is what we’ll concentrate on for this exercise as it very nearly makes up a 30 year climate period. Yes, I know some may call this an arbitrary starting point, but it the only possible point that allows comparison between land-ocean data-sets and the satellite data-sets.
Here is the first graph, the raw anomaly data as it was published this month by all the above listed sources:
Here is the source data file for this plot and subsequent plots.
I also plotted a magnified view to show the detail of the lat 12 months with notations added to illustrate the depth of the anomaly over the past 12 months.
March 2005 to January 2008, magnified view – click for larger image
I was particularly impressed with the agreement of the 4 metrics during the 1998 El Niño year as well as our current 2008 La Niña year.
I also ran a smoothed plot to eliminate some of the noise and to make the trends a bit more visible. For this I used a 1 year (12 month) average.
Again there is good agreement in 1998 and in 2008. Suggesting that all 4 metrics picked up the ENSO event quite well.
The difference between these metrics is of course the source data, but more importantly, two are measured by satellite (UAH, RSS) and two are land-ocean surface temperature measurements (GISS, HadCRUT). I have been critical of the surface temperature measurements due to the number of non-compliant weather stations I’ve discovered in the United States Historical Climatology Network (USHCN) from my www.surfacestations.org project.
One of the first comments from my last post on the 4 global temperature metrics came from Jeff in Seattle who said:
Seems like GISS is the odd man out and should be discarded as an “adjustment”.
Looking at the difference in the 4 times series graphs, differences were apparent, but I didn’t have time to study it more right then. This post today is my follow up to that examination.
Over on Climate Audit, there’s been quite a bit of discussion about the global representivity of the GISS data-set due to all of the adjustments that seem to have been applied to the data at locations that don’t seem to need any adjustments to compensate for things like urban heat islands. Places like Cedarville, CA and Tingo Maria, Peru both illustrate some of the oddities with the adjustment methodology used by NASA GISS. One of the issues being discussed is the application of city nightlights (used as a measure of urbanization near the station) as a proxy for UHI adjustments to be applied to cities in the USA. Some investigation has suggested that the method may not work as well as one might expect. There’s also been the issue of whether of not stations classified as rural are truly rural.
So with all of this discussion, and with this newly collated data-set handed to me today, it gave me an idea. I had never seen a histogram comparison done on all four data-sets simultaneously.
Doing so would show how well the cool and warm anomalies are distributed within the data. If there is a good balance to the distribution, one would expect that the measurement system is doing a good job of capturing the natural variance. If the distribution of the histogram is skewed significantly in either the negative or positive, it would provide clues into what bias issues might remain in the data.
Of course since we have a rising temperature trend since 1979, I would expect all 4 metrics to be more distributed on the positive side of the histogram as a given. But the real test is how well they match. All four metrics correlate well in the time series graphs above, so I would expect some correlation to be present in the histogram as well. The histograms you see below were created from the raw data from 1979-2008. No smoothing or adjustments of any kind were made to the data. The “unadjusted” data in this source data file were used: 4metrics_temp_anomalies.txt
First we have the satellite data-set from UAH:
University of Alabama, Huntsville (UAH) Microwave Sounder Data 1979-2008 – click for larger image
The UAH data above looks well distributed between cool and warm anomaly. A slight warm bias, but to be expected with the positive trend since 1979.
Next we have the satellite data-set from RSS:
Remote Sensing Systems (RSS) Microwave Sounder Data 1979-2008 – click for larger image
At first I was surprised at the agreement between UAH and RSS in the percentages of warm and cool, but then I realized that these data-sets both came from the same instrument on the spacecraft and the only difference is methodology in preparation by the two groups UAH and RSS. So it makes sense that there would be some agreement in the histograms.
Here we have the land-ocean surface data-set from HadCRUT:
Hadley Climate Research Unit Temperature data 1979-2008 – click for larger image
Here, we see a much more lopsided distribution in the histogram. Part of this has to do with the positive trend, but other things like UHI, microsite issues with weather station placement, and adjustments to the temperature records all figure in.
Finally we have the GISS land-ocean surface data-set:
NASA Goddard Institute for Space Studies data 1979-2008 – click for larger image
I was surprised to learn that only 5% of the GISS data-set was on the cool side of zero, while a whopping 95% was on the warm side. Even with a rising temperature trend, this seems excessive.
When the distribution of data is so lopsided, it suggests that there may be problems with it, especially since there appears to be a 50% greater distribution on the cooler side in the HadCRUT data-set.
Interestingly, like with the satellite data sets that use the same sensor on the spacecraft, both GISS and HadCRUT use many of the same temperature stations around the world. There is quite a bit of data source overlap between the two. But, to see such a difference suggests to me that in this case (unlike the satellite data) differences in preparation lead to significant differences in the final data-set.
It also suggests to me that satellite temperature data is a more representative global temperature metric than manually measured land-ocean temperature data-sets because there is a more unified and homogeneous measurement system, less potential bias, no urban heat island issues, no need of maintaining individual temperature stations, fewer final adjustments, and a much faster acquisition of the data.
One of the things that has been pointed out to me by Joe D’Aleo of ICECAP is that GISS uses a different base period than the other data-sets, The next task is to plot these with data adjusted to the same base period. That should come in a day or two.
UPDATE1: I’ve decided to make this a 3 part series, as additional interest has been generated by commenters in looking at the data in more ways. Stay tuned for parts 2 and 3 and we’ll examine this is more detail.
UPDATE2: I had mentioned that I’d be looking at this in more detail in parts 2, and 3. However it appears many have missed seeing that portion of the original post and are saying that I’ve done an incomplete job of presenting all the information. I would agree for part1, but that is what parts 2 and 3 were to be about.
Since I’m currently unable to spend more time to put parts 2 and 3 together due to travel and other obligations, I’m putting the post back on the shelf (archived) to revisit again later when I can do more work on it, including show plots for adjusted base periods.
The post will be restored then along with the next part so that people have the benefit of seeing plots and histograms done on both ways. In part 3 I’ll summarize 1 and 2.
In the meantime, poster Basil has done some work on this of interest which you can see here.
UPDATE3: Part 2 is now online, please see it here.







Here’s my graphing effort. The file both.xls has both Hadley and GISS data plotted. I’ve divided GISS by 100 and subtracted 0.1 to get an apples-to-apples comparison. Download the file both.zip and unzip. Read the notes on the “README” tab for a short summary of what I’ve done. The file will be temporarily available at my webspace by clicking this link.
Since about 2002, GISS has trended higher than RSS. The standard deviation of the residuals of the last 5 years is 0.085. The standard deviation from 1979 to 2001 is 0.142. In the last 5 years, RSS and GISS more closely agree than at any comparable time since the satellites went online.
This graph shows the 5 year moving standard deviation of the residuals.
http://cce.890m.com/gissvsrssstdev.jpg
The increasing agreement that you see starting around 1998 coincides with both the end of the ’98 el nino, which probably enhances the difference between the two methods, and the AMSU instruments going online. One of the AMSU channels is beginning to malfunction, so this probably won’t last forever.
[…] Antony Watts: So with all of this discussion, and with this newly collated data-set handed to me today, it gave me an idea. I had never seen a histogram comparison done on all four data-sets simultaneously. […]
A little disappointed that you haven’t picked up on the fairly basic observation that all these data series are displayed as anomalies …. but against means from different reference time periods (eg.GISS is referenced 1951-1980 whilst HADCRU is referenced to 1961-1990). So any direct comparison is meaningless. ie. “I was particularly impressed with the agreement of the 4 metrics during the 1998 El Niño year as well as our current 2008 La Niña year.”
Its a shame because there are clearly some good points … but you need to get the basics right.
For the Reply to Atmoz– “REPLY: Thanks for doing that, the real question though is why does GISS use a different base period than the other data sets?”
There are some reasons for GISS to use 1951-80 as a base period. First, Hansen and Lebedeff (http://pubs.giss.nasa.gov/abstracts/1987/Hansen_Lebedeff.html ), the first paper of GISS surface temperature analysis, used it, and they don’t want to change the base period at the end of every decade. Also 1951-80 was a time of good global coverage of the observations. 1951-80 was also the period that the global temperature didn’t change much ( http://data.giss.nasa.gov/gistemp/graphs/ ). They don’t want to take a period with a rapid change as a base period.
Basil–
I looked up Cochrane-Orcutt estimation. How do you determine the correlation coefficient for the noise when you do this? (Could you point me to a clear reference? I’m tempted to just do a mindless estimate using data since 1880, but I’d like to know how to do this the correct, official way.)
cce,
You are confusing a stabilizing trend with a converging one. Your graphic of the moving standard deviation isn’t measuring convergence. It is measuring noise in the residuals. if you don’t believe me try padding another 24 months of data onto your residual: 8 months at .04, then 8 months at .05, then 8 months at .06, a syntheic diverging trend of .01 deg C per year. Look at your moving standard deviation and tell me what you conclude about divergence from your metric.
Kind regards,
Earle
REPLY: Thanks Earle, I appreciate the positive input.
Anthony,
The baselines are arbitrary. I wouldn’t worry too much about this. When Gavin did his validation of Hansen, the code anomaly baseline as “perpetual 1958”; the GISS baseline is some particular set of 30 years. He didn’t rebaseline. (The only way to do it is to pick a string of years sometime after computations began and set them together.)
Steve Mc and I did set things to the same baseline (and in comments on some blogs, a few pro-AGW people sniped at us for doing so.)
I thought the flatness with a hint of bimodal of one of those GISS histograms was interesting. Though that may have meant nothing other than there is not enough data to get a nice smooth histogram.
> The real question here though is: why does GISS uses
> a different baseline period than the other 3 metrics?
It doesn’t matter. So please stop wasting your time on a cosmetic distraction and spend it doing some real work. Put it this way, does it matter if one of the datasets is in Fahrenheit, another in Celsius, and another in Kelvin? Kelvin is Celsius + 273.12. GISS is HADCRU + 0.1. Big deal. As long as we can convert back and forth, it doesn’t matter. As noted in my post on Feb 29, I subtracted 0.1 from GISS and it tracks HADCRU reasonably well until the late 1990’s when plotted.
However, it seems obvious from the graph that the difference in readings is larger post-1998 than pre-1998.
This difference requires data analysis, which it seems to me has been accurately done by the surface stations project
I confess that I am highly suspicious of the GISS data, which – like Dr. Hansen, seems to have fallen off the trolley. My suspicion is that the data has been deliberately manipulated into a higher-than-reasonable upward “creep” across the board.
I have no proof, this is only a hunch. Hansen’s ravings have become so irrational, even his own staff at NASA is starting to shake their heads in wonder.
I wonder if there are others who share my suspicions, or have I simply fallen victim to the dark side of the force.
Just asking.
Are there now enough good rural sites to do a study, surely this would indicate whether UHI has a significant effect on US average temps, that would be a start.
[…] UAH, RSS) data, and data set anomalies, including why GISS (NASA) is so different from the rest. A look at temperature anomalies for all 4 global metrics: Part 1 « Watts Up With That? It’s starting to look like some deliberate falsification has been going […]
[…] is another site.. A look at temperature anomalies for all 4 global metrics: Part 1 « Watts Up With That? Thus we play the fools with the time, and the spirits of the wise sit in the clouds and mock […]
[…] is another site.. A look at temperature anomalies for all 4 global metrics: Part 1 « Watts Up With That? Note the rather sharp drop depicted in this. Thus we play the fools with the time, and the […]
Hi Folks!
Windows vista is also causing lots of boot problems, so I often get questions like this:
I have a Dell Dimension, which won’t boot to Windows (Vista), and none the repair options work:
Startup repair: Reports repair fail due to problem with registry
System Restore: Reports no restore points available
Windows Complete PC Restore: Reports no backups available
Windows Memory Diagnostic Tool: No memory problems
Command Prompt.
Can’t think of any appropriate command to use here.
So I booted with the system DVD (as one would with XP) but the upgrade
option has been greyed don’t want to do a new install. I want to restore existing
installation.
What should I do?
————————————————————
So here is the answer:
You can’t do a ‘repair install’ because you need to launch the Vista DVD
from within Windows, not, as you have been doing, booting straight from the
DVD; that is why the ‘upgrade’ is greyed out.
If you cannot launch Vista and none of the repair options will work a new
install is the only other option.
To save problems in future it is actually a good idea to image the hard
drive, using something like True Image. What I do is install operating system, download all updates, check system I working okay for a day or two, activate system, then image the drive/partition. Any time I get a problem I can re-image the drive/partition quickly and be up and running without much trouble. And minor fixes are done by using any registry fix tool, there are plenty of them on the market today.
Regards,
Carl