NOTE: Please note that part 2 is now online, please see it here.
I recently plotted all four global temperature metrics (GISS, HadCRUT, UAH, RSS) to illustrate the magnitude of the global temperature drop we’ve seen in the last 12 months. At the end of that post, I mentioned that I’d like to get all 4 metrics plotted side-by-side for comparison, rather than individually.
Of course I have more ideas than time these days to collate such things, but sympathetic reader Earle Williams voluntarily came to my rescue by collating them and providing a nice data set for me in an Excel spreadsheet last night.
The biggest problem of course is what to do with 4 different data sets that have different time spans. The simplest answer, at least for a side by side comparison is to set their time scales to be the same. Satellite Microwave Sounder Unit (MSU) data from the University of Alabama, Huntsville (UAH), and Remote Sensing Systems (RSS) of Santa Rosa, CA only go back to 1979. So the January 1979-January 2008 period is what we’ll concentrate on for this exercise as it very nearly makes up a 30 year climate period. Yes, I know some may call this an arbitrary starting point, but it the only possible point that allows comparison between land-ocean data-sets and the satellite data-sets.
Here is the first graph, the raw anomaly data as it was published this month by all the above listed sources:
Here is the source data file for this plot and subsequent plots.
I also plotted a magnified view to show the detail of the lat 12 months with notations added to illustrate the depth of the anomaly over the past 12 months.
March 2005 to January 2008, magnified view – click for larger image
I was particularly impressed with the agreement of the 4 metrics during the 1998 El Niño year as well as our current 2008 La Niña year.
I also ran a smoothed plot to eliminate some of the noise and to make the trends a bit more visible. For this I used a 1 year (12 month) average.
Again there is good agreement in 1998 and in 2008. Suggesting that all 4 metrics picked up the ENSO event quite well.
The difference between these metrics is of course the source data, but more importantly, two are measured by satellite (UAH, RSS) and two are land-ocean surface temperature measurements (GISS, HadCRUT). I have been critical of the surface temperature measurements due to the number of non-compliant weather stations I’ve discovered in the United States Historical Climatology Network (USHCN) from my www.surfacestations.org project.
One of the first comments from my last post on the 4 global temperature metrics came from Jeff in Seattle who said:
Seems like GISS is the odd man out and should be discarded as an “adjustment”.
Looking at the difference in the 4 times series graphs, differences were apparent, but I didn’t have time to study it more right then. This post today is my follow up to that examination.
Over on Climate Audit, there’s been quite a bit of discussion about the global representivity of the GISS data-set due to all of the adjustments that seem to have been applied to the data at locations that don’t seem to need any adjustments to compensate for things like urban heat islands. Places like Cedarville, CA and Tingo Maria, Peru both illustrate some of the oddities with the adjustment methodology used by NASA GISS. One of the issues being discussed is the application of city nightlights (used as a measure of urbanization near the station) as a proxy for UHI adjustments to be applied to cities in the USA. Some investigation has suggested that the method may not work as well as one might expect. There’s also been the issue of whether of not stations classified as rural are truly rural.
So with all of this discussion, and with this newly collated data-set handed to me today, it gave me an idea. I had never seen a histogram comparison done on all four data-sets simultaneously.
Doing so would show how well the cool and warm anomalies are distributed within the data. If there is a good balance to the distribution, one would expect that the measurement system is doing a good job of capturing the natural variance. If the distribution of the histogram is skewed significantly in either the negative or positive, it would provide clues into what bias issues might remain in the data.
Of course since we have a rising temperature trend since 1979, I would expect all 4 metrics to be more distributed on the positive side of the histogram as a given. But the real test is how well they match. All four metrics correlate well in the time series graphs above, so I would expect some correlation to be present in the histogram as well. The histograms you see below were created from the raw data from 1979-2008. No smoothing or adjustments of any kind were made to the data. The “unadjusted” data in this source data file were used: 4metrics_temp_anomalies.txt
First we have the satellite data-set from UAH:
University of Alabama, Huntsville (UAH) Microwave Sounder Data 1979-2008 – click for larger image
The UAH data above looks well distributed between cool and warm anomaly. A slight warm bias, but to be expected with the positive trend since 1979.
Next we have the satellite data-set from RSS:
Remote Sensing Systems (RSS) Microwave Sounder Data 1979-2008 – click for larger image
At first I was surprised at the agreement between UAH and RSS in the percentages of warm and cool, but then I realized that these data-sets both came from the same instrument on the spacecraft and the only difference is methodology in preparation by the two groups UAH and RSS. So it makes sense that there would be some agreement in the histograms.
Here we have the land-ocean surface data-set from HadCRUT:
Hadley Climate Research Unit Temperature data 1979-2008 – click for larger image
Here, we see a much more lopsided distribution in the histogram. Part of this has to do with the positive trend, but other things like UHI, microsite issues with weather station placement, and adjustments to the temperature records all figure in.
Finally we have the GISS land-ocean surface data-set:
NASA Goddard Institute for Space Studies data 1979-2008 – click for larger image
I was surprised to learn that only 5% of the GISS data-set was on the cool side of zero, while a whopping 95% was on the warm side. Even with a rising temperature trend, this seems excessive.
When the distribution of data is so lopsided, it suggests that there may be problems with it, especially since there appears to be a 50% greater distribution on the cooler side in the HadCRUT data-set.
Interestingly, like with the satellite data sets that use the same sensor on the spacecraft, both GISS and HadCRUT use many of the same temperature stations around the world. There is quite a bit of data source overlap between the two. But, to see such a difference suggests to me that in this case (unlike the satellite data) differences in preparation lead to significant differences in the final data-set.
It also suggests to me that satellite temperature data is a more representative global temperature metric than manually measured land-ocean temperature data-sets because there is a more unified and homogeneous measurement system, less potential bias, no urban heat island issues, no need of maintaining individual temperature stations, fewer final adjustments, and a much faster acquisition of the data.
One of the things that has been pointed out to me by Joe D’Aleo of ICECAP is that GISS uses a different base period than the other data-sets, The next task is to plot these with data adjusted to the same base period. That should come in a day or two.
UPDATE1: I’ve decided to make this a 3 part series, as additional interest has been generated by commenters in looking at the data in more ways. Stay tuned for parts 2 and 3 and we’ll examine this is more detail.
UPDATE2: I had mentioned that I’d be looking at this in more detail in parts 2, and 3. However it appears many have missed seeing that portion of the original post and are saying that I’ve done an incomplete job of presenting all the information. I would agree for part1, but that is what parts 2 and 3 were to be about.
Since I’m currently unable to spend more time to put parts 2 and 3 together due to travel and other obligations, I’m putting the post back on the shelf (archived) to revisit again later when I can do more work on it, including show plots for adjusted base periods.
The post will be restored then along with the next part so that people have the benefit of seeing plots and histograms done on both ways. In part 3 I’ll summarize 1 and 2.
In the meantime, poster Basil has done some work on this of interest which you can see here.
UPDATE3: Part 2 is now online, please see it here.
Thanks for the shout out Anthony. The data I used to collate all four indices were from the links provided under each graph in Anthony’s post here:
http://wattsupwiththat.wordpress.com/2008/02/19/january-2008-4-sources-say-globally-cooler-in-the-past-12-months/
The GISS data was gathered on Feb 14, 2008. The other three were downloaded on February 25, 2008. If you go to the GISS data here ( http://data.giss.nasa.gov/gistemp/tabledata/GLB.Ts+dSST.txt ) you will see the GISS in table form, not conducive to plotting by month in Excel. I manually rearranged the GISS data into a single column with each monthly value per row, then joined that with the other three temperature indices, which are nicely organized into columns.
I agree that for direct comparison of all four it will be helpful to see them centered around the 1979-2008 mean. I disagree with Atmoz in that any histogram analysis are likely to be wrong. Conclusions based upon the histogram midpoint are not wrong, per se, as that is what the data present because two of the series have a much longer history. Conclusions regarding the shape of the histograms are entirely appropriate regardless of where the histograms are centered. Such analysis is facilitated when they histograms are centered about a common point.
I don’t find it surprising at all that the histrograms look as they do, because the surface readings are indeed shifted upwards relative to the satellite data. That’s what you would expect of two normally distributed random variables with different means. However, that doesn’t necessarily say that there is a data quality issue. That just says that one data set is biased upwards relative to another.
The question is not whether the surface temperature readings are higher than those of the sats (they obviously are), the question is whether the difference between the sats and surface temps are changing over time….and it appears to me that they are not (at least not significantly).
I compared the average satellite and average surface numbers, then took the 12 month moving average of both the satellite and surface readings. Although MA for surface is certainly above that of the satellite, the delta between the surface and satellite MA’s does not show any kind of significant trend (the slope of the regression line for the delta of the MA’s is 6.5E-05)
Although I would tend to have more faith in the satellite data, I don’t think that the histogram views says anything significant about the data quality.
Anthony, as Raven has said, you should make it clear that the reference period for all the four metrics is the same.
Since you are plotting anomalies, the reference period on which anomalies are computed, needs to be the same.
For istance, HadCRUT reference period is usually 61-90, for GISS is 51-80, whereas for satellite based estimation is more recent.
REPLY: Yes, agreed and all coming in part two in detail, I’ll make some notations to this post though.
Excellent work here Anthony. The data seems to support our feeling that GISS is nothing but loaded data.
Looking forward to parts 2 and 3…..
REPLY: It is premature to conclude that, a further examination is needed to be know what the differences are. Hence, parts 2 and 3.
BTW, I know it’s impossible for the RSS and UAH data, but I’d like to see “normal” be the entire 20th century. 1901-2000. That should get at least one complete PDO cycle in the “average”.
Matt
At my blog I have created a graph with all the histograms combined – I think it makes the relationships rather clearer.
REPLY: This is worth a read, folks. Thanks -Anthony
“It is premature to conclude that, a further examination is needed to be know what the differences are”
Fair enough. I suspect the GISS data is using a time period for the “Average” that has more of the negative phase of the PDO in it. Hence, all measurements based on that average are warmer. All the numbers seem to track each other well, with the difference being only where they are above or below “normal”.
Anthony:
I emailed NASA/GISS about their choice of reporting period, and the short answer was: “We use it because we’ve always used it”.
Adding this factor would compensate for the differences between satellite and surface measurements which Atmoz noted.
Hold it. How is that going to compensate for the differences? If surface temps are higher and we are supposed to reduce [sic] troposphere temps by a factor of 1.2, isn’t that just going to drive the differences even further apart?
OTOH, that would sure as heck confirm the degree of microsite error and its impact on the delta!
I get the impression I may have this backwards, but I can’t see how.
By the way, 1928 and 1969 are zero-anomaly years for GISS.
For zero-anomaly, GISS, for some reason unexplained, uses the mean of 1951-1980.
That figure is: 14C (or 57.2F )
Anthony, on a somewhat related issue, a month or so back, someone at CA did histograms of the amount of warming in USHCN sites for each of the CRN ratings.
Two things jumped out at me.
One was how skewed the distribution for sites with poor ratings was to the warming side. Which clearly showed the bulk of the warming was coming from problem sites.
The other was that the good sites showed a normal (or much less skewed) distribution with the mean and mode showing only slight warming. If I recall correctly, less than 0.2C.
So we’re basing entire economies off an arbitrary anomaly value during a notoriously cool period in the 20th century. Wonderful.
Look at it this way. No matter which base period one chooses, GISTEMP shows a series of ascending peaks post-1998. Each peak is higher than the last and higher than the El Nino peak. The higher highs (to borrow stock chart terminology) enable a certain someone to make certain claims about changes in global temperature, claims that are not supported by the other three studies.
While these differences in anomalies may not appear to be significant from a scientific or statistical perspective, it strikes me that they will be very significant when viewed from public relations and public policy perspectives.
What’s up with GISTEMP?
Harold Vance, you hit the nail on the head. I’m so sick of hearing that science says that x year was the nth warmest EVER, when the claim is clearly not scientific, becuase it isn’t independently verified! HadCrut falsifies GISS, GISS falsifies HadCrut, one or both must be totally wrong, but the media act as if GISS is the only surface network out there (and the scarcely even mention the sattelites!).
By the way, might be interesting to include balloons. But any idea why they end in 2005?
http://cdiac.ornl.gov/ftp/trends/temp/angell/global.dat
http://cdiac.ornl.gov/ftp/trends/temp/sterin/global.dat
Actually, there’s HadAT, to:
http://hadobs.metoffice.com/hadat/hadat2/hadat2_monthly_global_mean.txt
I realize part of the point is the recent drop, but it would be interesting to see, anyway.
The smoothed 12 month average graph makes this dip look a lot like 1998. Do you think this might be relevant, or is it an artifact of massaging the data?
Thanks.
William R says “The question is not whether the surface temperature readings are higher than those of the sats (they obviously are), the question is whether the difference between the sats and surface temps are changing over time….and it appears to me that they are not (at least not significantly).”
However, it seems obvious from the graph that the difference in readings is larger post-1998 than pre-1998.
This difference requires data analysis, which it seems to me has been accurately done by the surface stations project.
What’s up with GISTEMP?
Well . . . you mean besides 2005 and 2007? #B^1
I downloaded GISS and HADCRUT3GL data to do a comparison. I originally started with the Hadley data. Then I took the GISS data; divided by 100 and subtracted 0.1 and it sort of tracked Hadley data OK, until the late 1990’s. I saw the same thing that Harold Vance saw.
To go further, I originally plotted a 12 month moving boxcar. Both Hadley and GISS showed a “sawtooth” pattern from about 2001 onwards. I wondered if there was anything special about 12 month moving averages. I went back to square 1, and plotted a “1 month moving average”, 2 month moving average, etc, etc, until I got to the 25 month moving average, by which time I was getting bleary eyed. Anyhow, I noticed that with a 19 or 20 month moving average, the “sawtooth” pattern almost entirely disappeared on the Hadley graph, reappearing as I went on to 25 months. Does the Hadley graph indicate a 19 or 20 month periodicity somewhere? I know of one natural event with that approximate time length, but I really want to stay away from Velikovsky with a 10-foot-pole.
The 19 and 20 month moving averages really emphasized the difference between the 2 datasets. GISS showed a “checkmark”. I.e. a small decline, followed by a larger rise, while Hadley showed a “round top”. Both graphs started declining in mid/late 2007. But the differences do make me suspicious that one of them has to be badly broken.
So Anthony, I have this obsession that the satellite data shows no significant warming between Jan 1979 and Dec 1997. So I took your data and compared GISS and UAH data over two time periods using histograms – Jan1979 to Dec 1997 and Jan 1998 to present. The results:
GISS Jan 1979 to Dec 1997 Cumulative % of -ve values to 0 anomaly, was 8%
UAH Jan 1979 to Dec 1997 Cumulative % of -ve values to 0 anomaly, was 54%
GISS from Jan 1998 to Jan 2007 0.55 positive anomaly Cumulative % =63%
UAH from Jan 1998 to Jan 2007 0.55 positive anomaly cumulative % = 93%
1 The satellite data shows no significant warming or cooling for 19 years – explain that with the CO2 hypothesis.
2 The satellite data does not show the same amount of warming post 1998 as GISS.
I personally like any hypothesis other than that of CO2. The Canadian government has just allotted 254 Million dollars for CO2 geo-sequestration in Alberta. Looks like a colossal waste of money!
GISS seems to have this prejudice towards excess warmth. (Hadcrut, if you do the same analysis, not so much)
I think you already proved, no chiseled in concrete, that the GISS data, and some other data, is wrong and even deliberately falsified (though you would never make such an accusation!) It may already be too late to mitigate the humanitarian disaster which could arise if temperature is really going down!
Our politically correct prognisticators have made ignorance of history a virtue; you’ve shown they also ignore the present. It surprises me you address them as colleagues.
Can you please overlay CO2 concentrations onto the temperature graphs. Seems to me the essence of the Global Warming issue is the relationship between CO2 in the atmosphere and global temperature. Personally, I would like the warmists to stare at a credible display of verifiable global temperatures vrs a display of verifiable CO2 concentrations so that I can posit the question: How is it possible that there is an inverse relationship? See, I only have about 30 seconds of their attention before they flit off to wherever they go so I need something simple.
Thanks,
Larry
There’s a scenario that I seldom see mentioned, since the debate on global climate change is so polarized:
What if greenhouse gases are indeed heating up the climate, while at the same time, particulate matter (aka global dimming) and decreased sun activity are cooling down the climate. We could be in a period where global warming is being masked by other factors…
http://newsprism.wordpress.com/2008/02/14/faith-and-skepticism-in-environmental-politics/
Rev, To be clear, the GISS data you have in that 4-way link is unadjusted, the way you labelled it, right? (It does NOT match the adjusted data I linked to earlier, which continues to Jan 2008).
But isn’t unadjusted GISS data the same thing as adjusted NOAA data? (I.e, bumped up 0.29C with the 2005 and 2007 leading the charge.)
REPLY: 4 way link? I’m confused by this entire question.
I like your article on newsprism, it was interesting. What’s fun is requiring the gorebal warming skeptics(with whom I agree) to now apply first principles of scientific data, evidence and the scientific method itself, to the claim of the existence of god, per Rush, Sean, Anne and their ilk.
Suddenly the scientific method is flawed, there’s no logic, no reason just the “faith” card which only represents an individual’s lack of full commitment to their religion – it’s no different than the gorebal warming religion – just in different clothing.
LOL! It’s laughter at its best; entertaining and amusing.
There are 3 major differences between hadcru and giss.
1. giss estimate the polar region, hadcru do not.
2. hadcru use different stations
3. hadcru use a different method for grids that are a mix of land and sea
and the base peroid is different