Guest essay by David Dohbro
Recently WUWT published my comparison of several different land-based and satellite-based global surface temperature anomaly (GSTA) datasets (1, and data sets’ descriptions and references therein). However, my analysis was critiqued for not using the same baseline period for each data set. Namely, HadCRUT4 refers to the official WMO period 1961-1990, while NCDC and GISS instead use the periods 1901-2000 and 1951-1980, respectively, which results in higher positive temperature anomalies for these two estimates. In addition, UAH references to 1981-2010 while RSS’ output is calculated relative to a 1979–1998 average.
Paul Homewood assisted me in re-analyzing the data by showing how to adjust a GSTA data set to the same baseline/reference period, for which I am very thankful. In addition, it needs to be restated here that this is not an exercise to determine which dataset is better than the other, it is merely assessing how (well) each compare and if there are temporal trends in the difference between each data set.
Here UAH’s data set is used as a reference. Any other data set could have been used as a reference, but since the satellite-based datasets start in the year 1979, whereas the land-based datasets in some cases go back to the year 1850, comparison of the different types of data sets prior 1979 is impossible. As mentioned, UAH references their GSTAs to the 1981-2010 period. All other 4 datasets’ GSTA were first adjusted to that same period for each monthly GSTA. Then the UAH monthly GSTAs were subtracted from the corresponding monthly GSTAs of each other dataset. From that, a 12-month running average (12-mra) was calculated (Figure 1). A value of 0 means that the UAH data and the other dataset are similar, a value >0 means the other dataset report a higher monthly GSTA compared to UAH and vice versa. The 12-mra data was analyzed for normality (Kolmogorov-Smirnov test, Anderson-Darling test, and Lilliefors-van Soest test), and regression analyses (none-parametric: Spearman Rank, parametric: linear-regression to determine slope) were performed on the data for trend analyzes (table 1). Figure 1 shows the linear-regression lines’ slope and R-square value only.
Figure 1.
None of the data was normally distributed (Table 1), hence none-parametric regression analyses was conducted to assesses how well the relationship between difference and time can be described using a monotonic function. It follows from figure 1, that the land-based GSTA datasets compared here follow the same general trend over time; increasing positive divergence with UAH, whereas the other satellite-based GSTA dataset (RSS) shows an opposite and even stronger trend.
Table 1: Statistics of the 12-mra from the difference between several GSTA data sets and the UAH’s GSTA data set.
However, the 12-mra of the difference between UAH and NCDC, when normalized to 1981-2010, is not increasing statistically significant indicating that the difference between these two datasets to date has not increased over time (linear regression slope is also close to 0). The 12-mra of the difference between GISS, HadCRUT4 and UAH, respectively, is, however, increasing with almost identical slopes (figure 1, table 1).
This means that if this trend continues, these two datasets will in the future report monthly GSTA that are increasingly higher than UAH’s. RSS on the other hand will in the future report monthly GSTA that are increasingly lower than UAH’s and that of the three land-based data sets. Albeit the fact that RSS and UAH use data retrieved from the same systems; the National Oceanic and Atmospheric Administration satellites microwave (and Advanced) sounding unit (MSU, and AMSU), the trend observed here suggests that computational and/or other data-treatment differences may play a role. In addition, this disagreement is likely a problem related to under-constraining such that one or both of the measurements is (are) biased relative to the true state of the measurand (2).
In conclusion, all five GSTA datasets analyzed here show an average GSTA over the past 35 years of between 0.01°C to 0.42°C above their respective baseline period temperature, depending data set (1). The land-based data sets report in all most all cases monthly GSTA that are higher than the satellite based GSTAs (1). In addition, the land-based monthly GSTAs analyzed here (GISS, NCDC, HadCRUT4) are increasingly positively diverging from the two satellite-based GSTAs used (RSS, UAH) over time, whereas the two-satellite-based data sets are increasingly negatively diverging from each other. These trends, and especially the opposing trend with RSS, warrant more in-depth analyses, continued scrutiny and attention.
References
(2) Mears, C. A., F. J. Wentz and P. W. Thorne, (2012) Assessing the Value of Microwave Sounding Unit-Radiosonde Comparisons in Ascertaining Errors in Climate Data Records of Tropospheric Temperatures, J. Geophys. Res., 117(D19), D19103, doi:10.1029/2012JD017710.
Why not just compare the “average temperatures”? That way you don’t need a “baseline”, you just need a graph. Do you happen to know the “average temperature” in any of the many years in your pretty graphs? I’m guessing not, since it seems to be top secret. What is the “average temperature”? Such a simple question which no one can answer.
And so the data sets say it is either warming or cooling. I’m an educated man but I’ve had a few and a relatively simple conclusion in lay terms would be appreciated.
As we finish up our new Version 6 of the UAH dataset, it looks like our anomalies in the 2nd half of the satellite record will be slightly cooler, somewhat more like the RSS dataset….but we are talking small adjustments here…hundredths of a deg. C.
UAH Global Temperature Update for May, 2014: +0.33 deg. C
June 10th, 2014 by Roy W. Spencer, Ph. D.
Adjustments just keep on coming.
I would be interested in providing my data set which is based off NCDC Gsod data with no adjustments and not homogenized, purely an anomaly based on measurements alone. You can can follow the link in my name, and there’s a email link there.
Why do you think that comparing the tropospheric temperature using an estimation based on microwave radiative physics to an INDEX that is compiled by average the temperature of water and the air temp at 2meters will tell you ANYTHING.
Here is a hint.
Compare UAH over land with air temps over land.
here is what you will find. They differ.
compare the temp at 1018hPa with the temp at 456hPA they differ.
You shouldnt think they would be the same.
“compare the temp at 1018hPa with the temp at 456hPA they differ”
Is that the difference Steve? I thought he difference was less.
Steven Mosher says:
June 12, 2014 at 9:35 pm
correct Steve, but isn’t he just trying to compare trends, in which case – generally, for a warming or cooling world, we might expect to see similar trends in the differing datasets?
I must confess, I’m finding it hard to see any purpose in this analysis, especially when the surface datasets share station data?
Averages:
On a 25C day someone pours half a cup of water at 0C and half a cup at 50C. Why should I complain? after all it was just an ‘average’ of 25C
I meant on me
Dougmanxx
Of course you can calculate an average temperature, but each data set will give a different answer as they all use different methods to measure it – and the satellites are measuring the entire lower troposphere (10km / 6.2 miles) not the surface anyway so the figures would be meaningless. For instance the average surface temperature is usually quoted as somewhere between 14c and 15c whereas the lower troposphere is around -30c. The reason the lower troposphere is used as a proxy is that it is the only figure that can meaningfully be measured across the entire globe – whereas the surface temperature requires a myriad of randomly spaced temperature stations – and we all know the problems with those!!
Then, of course, even the “surface” measured is different. A temperature station in Breckenridge, Colorado will be measuring the temperature nearly 2 miles above one in Miami, so does an “average” of those two have any meaning? The closest to a meaningful average is probably the sea surface as, of course, it will always be at sea level. Personally I think it is the only one that really matters as the land temperatures will never be able to diverge very much from the sea surface temperature without “weather” working to even the two out.
As we are trying to assess “global warming” then the point is to measure how much these figures change over time, in other words their “anomaly” from a fixed base period. Ultimately, as most living things are in the lower 3,000m / 10,000 feet / 2 miles or so then an average at that level is the only one that really matters to us and the rest of the “biosphere”. But, as I state above, how long is a piece of string?
The satellite LT temps aren’t surface temps, but since they’re mostly unfiddled, they do give us a reality check on the amount of adulteration in the other records. 1998 shows an inordinate amount of anti-Beamon-effect adjustment, for example.
TLM says:
June 13, 2014 at 2:19 am
You miss the point. Virtually every graph posted on this site, and every other blog devoted to talking about “Global Warming/Climate Change/Global Climate Disruption/etcetcetc…” all use “anomaly”. Believe it or not, I understand the purpose of this. But it serves a secondary unacknowledged purpose: it conceals the “average temperature” that was used to calculate the “anomaly”. Ultimately, not one single person can ever tell me what the “average temperature” was in any year of any graph they ever post. Not one. They don’t know. I would argue, that if you ONLY know an “anomaly” you don’t “know” anything. We should require that the “average temperature” that is used to calculate an “anomaly” is also included with that “anomaly” so we can see how that “data” changes over time. Because it is, and it does. It’s the magic of “adjustment” and “homogenization” that change 80 year old “temperature data”. At least if we force people to include all of the “average temperature data” we can then see, in a very simple and easy to understand format, how the past “data” is being changed. I’m going to keep asking, until someone can give me an answer.
** I get that satellite data is going to be different, but the above graphs are chock full of GISS and HADCRut. I also understand comparing the two (satellite v. surface) is problematic, but my question still remains unanswered.
Go here, I have both.
Steven Mosher:
Because their absolute temperatures are so different, all this would tell you they measure very different things. It’s surprising to me they agree as well as they do.
You expect a different slope because, for example, you don’t expect to see minimum temperatures as strongly affected in satellite measures as near-Earth measurements…the water vapor feedback is mostly in surface boundary layer.
On another thread the question about why El Niño affects satellite more than surface temperatures was raised. Roy Spencer gave this answer:
Dougmanxx, while GISTEMP and HADCRUT attempt to measure the same thing, the average temperature over the surface of the Earth, there are some key differences between the two series that leads to divergences. I’m including several of these differences (there may be others that I’m not remembering) along with a short discussion of “why this matters”. To fully explicate these would take separate blog posts (which I’m not going to do, but maybe somebody else will).
One is the difference in gridding. HadCrut uses a 5°x5° grid whereas GISTEMP has a more complicated gridding scheme. By itself this should yield essentially the same estimate for very long term trend, but because there are quasi-periodic processes with periods from 2-60 years, and because these processes have non-uniform spatial pattern, then different averaging schemes will filter these quasi-periodic processes in different ways (leading to different amplitudes and phases for the remnant of these processes that “makes it through” the global average).
A second is different spatial coverage. There are two effects from this. One is the well-known “polar amplification”. Much of the missing area is near the poles, so not including this biases the trend low. GISTEMP attempts to do a better coverage, so it should yield results closer to the actual temperature from this effect alone. HADCRUT famously replaces missing 5°x5° cells with the global average, which is a very poor approximation over land (but works for ocean, since it has almost no polar amplification).
A third is HADCRUT doesn’t just average over all cells (but GISTEMP does). Rather it computes separate average temperatures for the Northern versus Southern Hemispheres, then averages these together. If there are missing cells, this reduces the error associated with a changing geographical bias over time (in early data, much of the Southern Hemisphere was missing over land), and the associated bias in temperature trend that arises do to polar amplification.
Just curious – are there any analysis of “total heat energy” in the global atmosphere? Would be pretty complex as you need to consider the percentages of all the GHG and temperatures, but really isn’t it ultimately about whether or not this is going up or down the golden egg?
That last post was for Dougmanxx
Carrick says:
June 13, 2014 at 5:46 am
Completely immaterial. They compute an “anomaly” from an “average temperature”. What is it? No one wants to say, because they continually change it. I’m not sure about you, but in my world the weather in the past isn’t continually changing. My question is so simple, you completely miss the point. No one can tell me an “average temperature” for ONE station, during ONE month, of ONE year. And even so, they use some kind of “average temperature” for their “global average”, but no one can tell me what it is, for say…. 1963. No. One. You see, if you know the “average temperature” as calculated in June of 2014, you can then compare it to what they use in June of 2015. Then that simple little bit of “data” will tell you what the “adjustments” are doing. So. Do you know ANY “average temperature”? I doubt it. My questions isn’t about trends. It’s not about anything other than: “What was the average temperature you used to make your calculation?” There is one. So why not tell everyone what it is?
Mi Cro says:
June 13, 2014 at 6:25 am
Excellent. From what you’ve posted, I see I’m not being a loon. Actual temperature data makes a difference. I’m guessing what the major products use to calculate their averages won’t look anything like what you’ve posted. And I KNOW they change the “averages”. I’m still going to ask the question, and I’m going to bookmark and read your site. Looks very interesting, than you!
You’re welcome.
I was looking for some specific things, that the temp of the average was, how many samples were used, what were the averages of only the measurements, and if I could see any changes to how much it cools at night.
I also process each station’s anomaly one at a time, and then aggregate those together.
Steven Mosher says:
June 12, 2014 at 9:35 pm
++++++
Great post Mr. Mosher. The kind of critical thinking I love from you. It was clear and cogent and perfect for discussion and debating of the subject matter.
I give you an A+
A good followup to your previous post. I’m not sure the analysis really tells us anything of real consequence, but I find the differences between the datasets interesting, so thank you. A couple of comments:
1) The difference between UAH and RSS seems to have much higher variability than any UAH vs land-based dataset (Based on the eyeball test). This is counter-intuitive since UAH and RSS use the same base data. I wonder why there is such variability.
2) You state, “This means that if this trend continues, these two datasets will in the future report monthly GSTA that are increasingly higher than UAH’s.” Based on eyeball analysis, it appears that the majority of the positive trend between UAH and the land based datasets is due to the period from 1980-1993, less than half the total period of the analysis. After that period the trend appears flat (It would be interesting to see the actual numbers to verify this), so a continuation of the most recent trend would not cause the land-based datasets to report monthly GSTA that are increasingly higher than UAH’s. Of course, it’s a dangerous thing to try to use previous trends to predict the future. Perhaps the flat trend of the past 2 decades is indicative of improvements to the surface stations?
As to an absolute temperature, I recall that Steve Milloy used to publish a running temperature for Earth on his Junk Science website, but pulled the feature for reasons that I forget. Here’s one advantage of using an anomaly. Suppose that you are collecting temperatures from a bunch of thermometers in order to calculate an average, but you don’t control the calibration of the instruments. Your confidence in reporting an actual temperature is quite low. But you do have confidence that the physical characteristics of the thermometers are not changing very much at all (Don’t ask me why.). Such a situation allows you to report that, for example, it is one degree warmer or cooler than last year, while at the same time having no clue what the actual temperature is.
Correct me if I’m wrong, but I seem to remember reading that GISS and HadCRUT use satellite measurements for their ocean temperature anomalies, at least since 1979.
From Steven Mosher on June 12, 2014 at 9:35 pm:
http://woodfortrees.org/plot/best/from:1979/to:2014/normalise/plot/uah-land/from:1979/to:2014/normalise
Gee, looks pretty close in general trends to me.
Except BEST has that 2010 major discrepancy. From the raw data:
2009.92 0.5962010 1.135
2010.08 1.086
2010.17 0.859
2010.25 -1.035
2010.33 1.098
#Data ends
Clearly 2010.25 has the polarity flipped. Such an obvious error, yet computers did the processing and there were no ham-fisted manual changes to the output, no hard-coded exceptions, thus obviously it cannot be a singular mistake. There must be a systemic programming issue which allowed it to happen.
Well to cut out the error I’ll just use to:2010, slap on a 13-month running mean for smoothing, and see what is the difference. I’ll add in RSS land as well.
Pretty close actually, lots of curve matching. But with RSS, something is very troubling. As is clearly seen, from 2005 RSS still shows “the pause”, BEST diverges to show ongoing warming.
Since UAH is currently noted for showing more warming than RSS recently and has a major revision coming that will bring it closer to RSS, it seems there’s a good chance BEST will be diverging from UAH as it now does from RSS, indicating a potential flaw in BEST starting in recent years.
David Dohbro
Re: “Albeit the fact that RSS and UAH use data retrieved from the same systems; the National Oceanic and Atmospheric Administration satellites microwave (and Advanced) sounding unit (MSU, and AMSU), ”
Please check on which satellites are being used by each. See Dr. Roy Spencer:
On the Divergence Between the UAH and RSS Global Temperature Records July 7, 2011
Dougmanxx: Of course the absolute temperatures are know, how can you otherwise calculate an anomaly?
For example, GISS states “Best estimate for absolute global mean for 1951-1980 is 14.0 deg-C or 57.2 deg-F”. source: http://data.giss.nasa.gov/gistemp/tabledata_v3/GLB.Ts+dSST.txt
Jimmy: thanks for the positive feedback. No this analysis is not ground breaking or a total shocker. It simple shows us how these global temperature datasets “behave” over time compared to each other. If several start to diverge from each other then it warrants attention, so keeping a watchfull eye on them this way, so to say, is a good and necessary thing. We can go in the nitty gritty and detail exactly where things may have started to change for an in-depth better understanding, but just like global temperatures it’s the longer term pattern that matters IMHO.
Anthony: thanks for posting! much appreciated!
david dohbro says:
June 13, 2014 at 11:05 am
The “best estimate for a global mean” is immaterial to answer my question. I see the table is made to make it as difficult as possible to figure out an “average temperature” for any one year. Why do you suppose that is? Have they kept any of these tables from previous incarnations? I mean, the “anomalies” change on a monthly basis, so shouldn’t this table also change? I’m going to take a screen shot for posterity’s sake anyway.
Want to see something fascinating? Go here: http://data.giss.nasa.gov/gistemp/tabledata_v2/GLB.Ts+dSST.txt
Amazing that from version to version the past is cooler. I never knew yesterday’s weather was so variable! The temp in 1928 was 13.84 C…. or it was 13.80 C. It all depends which “data” you use! Wheeeeeee. Temperatures are so much FUN.
What an absolute farce.
David L. Hagen,
“my UAH cohort and boss John Christy, who does the detailed matching between satellites, is pretty convinced that the RSS data is undergoing spurious cooling because RSS is still using the old NOAA-15 satellite which has a decaying orbit, to which they are then applying a diurnal cycle drift correction based upon a climate model, which does not quite match reality.” Roy Spencer
Maybe. And maybe not. The problem seems to come from a behavior change of UAH from 2005 on land.
http://oi57.tinypic.com/2nvgjmq.jpg
Fairly obvious that whether or not the earth is warming or not or how fast or not depends on who gets to “…torture and molest…” the data.
From phi on June 13, 2014 at 12:42 pm:
Images You’ll Also Enjoy
Anime naked girl and a sparkling sitting Gelert?
You really need different image accounts to separate your “respectable” persona from your “private side”.
kadaka (KD Knoebel),
This is not my account and I think you do not understand how works tinypic. That said, I’ll probably stop boring WUWT readers, it does not seem to have much use.
From phi on June 14, 2014 at 12:52 am:
It’s the same “Images You’ll Also Enjoy” on different browsers that do not share cookies or bookmarks, thus they are tied to the pic at your link and not to my browsing history. The images are not visually similar. All those URLs at the site contain “pic=2nvgj##” which sure looks like a unique account identifier.
If that is your graph but not your account, get your own account. If that is not your graph, replicate and post elsewhere.
I was actually wondering about the data sources used for the graph.
Your problem was you posted late, after attention was on other articles, thus the potential for a response was low.
Maybe you guys should use the average energy content of the atmosphere as the measurement. I guess this requires humidity measurements as well? I bet in 20 years you would have a great data set to argue about. Meanwhile I take the temperature anomaly as a rough indicator of global warming. The oceans heat content sure seems a more better indicator.