Guest post by Clive Best
Perhaps like me you have wondered why “global warming” is always measured using temperature “anomalies” rather than by directly measuring the absolute temperatures ?
Why can’t we simply average the surface station data together to get one global temperature for the Earth each year ? The main argument to work with anomalies (quoting from the CRU website) is: ”Stations on land are at different elevations, and different countries estimate average monthly temperatures using different methods and formulae. To avoid biases that could result from these problems, monthly average temperatures are reduced to anomalies from the period with best coverage (1961-90)….” In other words although measuring an average temperature is “biased”, measuring an average anomaly (deltaT) is not. Each monthly station anomaly is actually the difference between the measured monthly temperature and so-called “normal” monthly values. In the case of Hadley Cru the normal values are the 12 monthly averages from 1961 to 1990.
The basic assumption is that global warming is a universal, location independent phenomenon which can be measured by averaging all station anomalies wherever they might be distributed. Underlying all this of course is the belief that CO2 forcing and hence warming is everywhere the same. In principal this also implies that global warming could be measured by just one station alone. How reasonable is this assumption and could the anomalies themselves depend on the way the monthly “normals” are derived?
Despite temperatures in Tibet being far lower than say the Canary Islands at similar latitudes, local average temperatures for each place on Earth must exist. The temperature anomalies are themselves calculated using an area-weighted yearly average over a 5×5 degree (lat,lon) grid. Exactly the same calculation can be made for the temperature measurements in the same 5×5 grid which then reflect the average surface temperature over the Earth’s topography. In fact the assumption that it is possible to measure a globally averaged temperature “anomaly” or DT also implies that there must be a globally averaged surface temperature relative to which this anomaly refers. The result calculated in this way for the CRUTEM3 data is shown below:
Fig1: Globally averaged temperatures based on CRUTEM3 Station Data
So why is this never shown ?
The main reason for this I believe is that averaged temperatures highlight something different about the station data. They instead reflect an evolving bias in the geographic sampling of the station data used over the last 160 years. To look into this I have been working with all station data available here and adapting the PERL programs kindly included. The two figures below show the location of stations used dating from 1860 compared to all stations.
Fig 2: Location of all stations in the Hadley Cru set. Stations with long time series are marked with slightly larger red dots.
Fig 3: Stations with data back before 1860
Note how in Figure 1 there is a step rise in temperatures for both hemispheres around 1952. This coincides with a sudden expansion in included land station data as shown below. Only after this time does the data properly cover the warmer tropical regions, although there still remain gaps in some areas. The average temperature rises because gaps for grid points in tropical areas are now filled. (There is no allowance made in the averaging for empty grid points neither for average anomalies nor temperatures). The conclusion is that systematic problems due to poor geographic coverage of stations affects average temperature measurements prior to around 1950.
Fig 4: Percentage of points on a 5×5 degree grid with at least one station. 30 % is roughly the land surface of Earth
Can empty grid points similarly affect the anomalies? The argument against this, as discussed above, is that we measure just the changes in temperature and these should be independent of any location bias i.e. CO2 concentrations rise the same everywhere ! However it is still possible that the monthly averaging itself introduces biases. To look into this I calculated a new set of monthly normals and then recalculated all the global anomalies. The new monthly normals are calculated by taking the monthly averages of all the stations within the same (lat,lon) grid point. These represent the local means of monthly temperatures over the full period, and each station then contributes to its near neighbours. The anomalies are area-weighted and averaged in the same way as before. The new results are shown below and compared to the standard CRUTEM3 result.
Fig5: Comparison of standard CRUTEM3 anomalies(BLACK) and anomalies calculated using monthly normals averaged per grid point rather than averaged per station (BLUE).
The anomalies are significantly warmer for early years (before about 1920), changing the apparent trend. Therefore systematic errors due to the normalisation method for temperature anomalies are of the order of 0.4 degrees in the 19th century. The origin of these errors is due to the poor geographic coverage in early station data and the method used to normalise the monthly dependences. Using monthly normals averaged per lat,lon grid point instead of per station causes the resultant temperature anomalies to be warmer before 1920. Early stations are concentrated in Europe and North America, with poor coverage in Africa and the tropics. After about 1920 these systematic effects disappear. My conclusion is that anomaly measurements before 1920 are unreliable, while those after 1920 are reliable and independent of normalisation method. This reduces evidence of AGW since 1850 from a quoted 0.8 +- 0.1 degrees to about 0.4 +- 0.2 degrees
Note: You can view all the station data through a single interface here or in 3 time slices starting here. Click on a station to see the data. Drag a rectangle to zoom in.
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Suggestion: “the great dying of the thermometers” made that the remaining station time series correlated higher with latitude-region time series. Many dissident stations were dropped and the remaining stations showed a more homogeneous result. Because I found a 27 sigma effect of mean correlations before and after, I’m quite sure that this happened. As a second suggestion I propose to compute the (missing data) variance-covariance matrix of regional time series. If these cover the whole earth, the true variance of the global series can be estimated as the mean covariance in the matrix. I found for 1850-2010 a true variance of 0.0828. This determines the play-room for all temperature change in this period.
@Bart “FTA: “The basic assumption is that global warming is a universal,
location independent phenomenon which can be measured by averaging all
station anomalies wherever they might be distributed.”
I see no basis for such an assumption. The change in temperature should
be proportional to the local temperature, which in turn is dependent on
latitude, wind currents, and local heat capacity, at the very least.”
There is no basis for such an assumption! It could be that the Earth is covered by mini “el Ninos” and local temperatures rise and fall over decades. Supposing that North America rises by 2 degrees while simultaneously a similar area in North Africa falls in temperature by 2 degrees, but globally the temperature remains static. North America is covered in weather stations whereas there are just a handfull in the Sahara. The algorithm used to calculate the global anomaly will then result in a net global increase – simply due to the bias in spatial sampling.
“I have recommended doing a fit to a spherical harmonic expansion instead of all this ad-hoc area averaging. At least doing that, there is a little relief from the sampling theorem (Shannon-Nyquist is a sufficient condition for being able to retrieve a signal from sampled data, but not strictly necessary). Nick Stokes claims to have done it and seen little difference, but he was not forthcoming with the details of precisely what he did.”
Yes this has occured to me as well. What I was imaging was to derive a “normal” spatial temperature distribution. It would need to be a harmonic function of lat and lon. Then we could fit this function to the best station data we have say between 1961 to 1990. The assumption would be that the function remains the same but the amplitude changes. Then the function would be sampled each year at fixed locations. – Out pops the time dependence of the amplitude. Of course this again assumes that “warming” is a global phenomenon and not a local phenomenon.
Nylo says: February 7, 2012 at 11:22 pm “And more to the point, we don’t care what the global temperature is, but how it is changing. And for that, the anomaly is all the information you need.”
Wrong. If you research phase changes like water vapour to rain or rain to ice, you need temperatures, not anomalies. If you research proxies like the growth of trees, or isotope-derived temperatures, you need temperatures, not anomalies, to construct calibrations.
Wrong again. There is also a mathematical problem. One can take a single station record and calculate its 1961-1990 average and so derive an anomaly. Next, you can do this to N stations in a grid cell, then simply combine the anomalies. OTOH, if you average the temperatures on the N stations in the grid cell, derive the 1961-1990 term and subtract it, you’ll get a different answer. The difference will flow through as you construct a global average.
Wrong again. It becomes more complicated when you find you are more interested in energy than temperature and their relationship by equations that can involve raising to powers of 4. The sequence in which you do the conversion affects the outcome. The conversion of an anomaly temperature to watts per sq m would seem rather meaningless.
Wrong again. Even more complicated is the estimation of confidence. Are you justified in assessing errors when you have missing values in the 1961-1990 period? After all, different people might infill by using different methods. The resultant anomalies would be different and their error bars would be in different places.
And so on ad nauseam. You’ll find it’s best to use math and physics in the conventional way.
excellent article, serious science. Reworking these these sloppily produced (and probably rigged) temperature datasets is essential.
The marked difference you note brings the land data much closer to the general form of the SST data in that earlier period
http://i41.tinypic.com/2s8k9ih.png
Slightly off topic but I hope smeone can help;
Q1 I heard / read somewhere that the three peak temperatures in a month where used as a basemark for whether that month was warmer or cooler then the norm, is this correct?
Q2 if the answer to 1 is yes what do you get if tou use the three coolest temperatures?
That’s a coincidence that’s exactly the same warming I got from just using the Cambridge UK data.
0.4 degrees C.
Clive , could you post your revised dataset (and preferably your modified version of the perl script that produces it) ?
I’m doing some work analysing SST and other land data , it would be very interesting to pass this version of the data through the same processing. It could be quite illuminating.
You could call this new time series the “Best land surface temperature record” if no one’s thought of that yet 😉
Kudos.
Just a quick note on the concept of a “normal” temperature for the earth. A norm is an established standard of what should be. Human body temperature (by long observation and experimentation) should be about 98.6 DF according to certain instruments used certain ways; a large deviation from that norm could kill you. We all know what our normal health looks like from how it feels to breathe to our bowel movements; we have established those parameters throughout our life and experience. Other norms exist or are established in other areas of life.
However: There is no norm for the temperature of the earth, nor for the stock market, nor for human population, nor for crop production, etc., etc. There are, to be sure, averages; but they may or may not be helpful (what, for example, is the Dow Jones stock average from its inception to the present?). Even if the average temperature for the earth’s surface could be reliably established–and I hope it can, someday–what use will be made of it? Scientific, or political? Whatever that average figure may turn out to be, let’s not mistake it for a norm; just because a current temperature can be determined, don’t say it should be exactly that temperature and must stay there. It won’t.
“The basic assumption is that global warming is a universal, location independent phenomenon which can be measured by averaging all station anomalies wherever they might be distributed.”
Nonsense. No-one has ever said anything remotely resembling this.
“Underlying all this of course is the belief that CO2 forcing and hence warming is everywhere the same.”
Pure nonsense. Again, such a statement has never appeared anywhere in the climate-related literature.
“In principal this also implies that global warming could be measured by just one station alone.”
Utter rubbish. Seriously, what is your intention in peddling such complete fiction? You must know that what you are saying is completely untrue.
Hi Clive,
Thank you for all your hard work. No opinion, as a proper understanding would require considerably more time.
I was always comfortable with the concept of the anomaly, but much less so with the validity of Surface Temperature (ST) data, since the satellite Lower Troposphere (LT) data showed considerably less warming.
In 2008, I informally compared ~30 years of Hadcrut3 ST with UAH LT, and observed an incremental warming in the ST of about 0.2C, or 0.07C per decade.
http://icecap.us/images/uploads/CO2vsTMacRae.pdf
Whether this apparent ST “warming bias” can be extended back in time is perhaps a matter of subjective opinion.
Anthony Watts and his team has shown that location errors, etc. in ST measurement stations have contributed to a warming bias in the USA ST dataset. Whether due to poor ST measurement location, urban sprawl (UHI), land use change, and/or other causes, there does seem to be a warming bias in the ST data. Furthermore, there is no reason to believe this warming bias started in 1979, when the satellites were launched. The ST warming bias can probably be extended back to 1940, or even earlier.
You may find it helpful to run your own comparison of Hadcrut3 to UAH, to help calibrate your studies on ST.
Best regards, Allan
P.S.
When temperatures cooled in 2008, I made the following observation, again based on unadjusted Hadcrut3 ST and UAH LT:
“The best data shows no significant warming since ~1940. The lack of significant warming is evident in UAH Lower Troposphere temperature data from ~1980 to end April 2008, and Hadcrut3 Surface Temperature data from ~1940 to ~1980.”.
See the first graph at
http://www.iberica2000.org/Es/Articulo.asp?Id=3774
I know, it’s a complicated subject and I may be wrong. However, compared to the IPCC, my predictive track record is looking pretty good to date.
P.P.S.
On another subject, major conclusions have made from CO2 and temperature data from ice cores. I accept that CO2 lags temperature in the ice core data, the lag being in the order of ~600-800 years (from memory) on a long time cycle. What I do question is the reliance on these numbers as absolute values as oppose to relative values. From the ice core data, we have concluded that pre-industrial atmospheric CO2 levels were about 275 ppm, and have now increased about 30% due to combustion of fossil fuels. Actual CO2 measurements at Mauna Loa started in 1958 at about 315 ppm, and now exceed 390 ppm. Earlier data on thousands of CO2 measurements complied by the late Ernst Beck suggest that even higher CO2 levels were measured worldwide, but Ernst’s data has been dismissed, apparently brushed aside with little thought, because it disagreed with the modern paradigm. While Ernst may be wrong, I doubt that his data has been given a fair examination. I suggest that brushing aside thousands of contradictory data points because they do not fit one’s paradigm is poor science practice.
The problem with using anomalies is that the average person does not understand that the anomaly figure is representative of a difference from a baseline number. Unless one is aware of what baseline is being used, the anomaly carries no weight as changing baselines changes the anomaly. I believe that those using anomalies for their argument are fully aware of this ignorance by the average person and try to capitalize on that to win them over. I always look at anomalies and percentages with the following caveat: figures lie and liars figure.
Nylo says:
February 7, 2012 at 11:22 pm
Actual temperature is not used because you cannot assume that the reading of one thermometer in one location is a valid temperature representative of a large area. Not even in the surrounding kilometer. Local conditions affect the temperature too much, so the absolute data is quite meaningless. However, the temperature anomaly CAN be assumed to be about the same in a large area. Because local conditions change in space, but remain quite the same in time. So if one spot is 2C higher than normal, it is quite reasonable to assume that, whatever the temperature is 1 kilometer further, it will be about 2C higher than normal as well for the corresponding location.
This is the biggest load of Warmist B**lsh*t ever, so much actual data is being lost through this stupid process it is unbeleivable “Scientists” would use it. This part in particular “So if one spot is 2C higher than normal, it is quite reasonable to assume that, whatever the temperature is 1 kilometer further, it will be about 2C higher than normal as well for the corresponding location.”
It completely ignores wind direction, Blocking, Pressure, Humidity, Elevation, wind off the Sea.
I am sure that other posters can name quite a few other affects that make this an “averaging” too far.
The Temperatures are the Temperatures and reflect what the locality experiences.
More on Beck, etc.
Note the alleged Siple data 83 year time shift – amazing!
http://hidethedecline.eu/pages/posts/co2-carbon-dioxide-concentration-history-of-71.php
The well known graph for CO2 is based on Ice core data (”Siple”) and direct measurements from Hawaii (Mauna Loa). The Siple data ended with a CO2 concentration of 330 ppm in 1883. 330 ppm CO2 in 1883 is way to high, 330 ppm was first reached by Mauna Loa data around 1960-70. The two graphs (Siple and Mauna Loa) was then united by moving Siple data 83 years forward in time. The argument to do this was, that the atmospheric content of the ice was around 83 years older than the ice. So rather “fresh” atmospheric air should be able to travel down in the snow and ice corresponding to the 83 year old ice? This is perhaps 50 meters down or probably more. And then the fresh air is locked in the 83 year old ice. So a good ventilation down 83 year old ice, and then the ice closes. This hypothesis is still debated – but the classic Siple-Mauna Loa CO2 graph is used widely as solid fact.
Bart (on February 8, 2012 at 1:56 am)
Imagine a data sampling system (time domain) where we stop taking readings whenever temperature drops below a certain level. Who could claim that such a system has a design which addresses the issue of aliasing?
Yet the spatial coverage of the surface record does something similar: we sample at convenient locations. That means not at the poles, not at the tops of mountain ranges, and so forth.
If we had started off with a design to avoid aliasing, we would have considered the properties of the signal at the outset. And used this to determine the sampling regime.
The issue is treated as a statistical problem, and studiously ignores the question of “quality” attaching to the sampling regime in the hope that it should be averaged-out. Even that is not guaranteed.
I know we are in agreement Bart, but I take a harder line on the issue of “sufficient or necessary”. This point relates (IMO) to the availability of other information which could assist us in designing a sampling regime. But isn’t that the heart of the issue – we don’t have any.
Clive Best says:
>>
@Robert Clemenzi
I am using exactly the same procedure as the Hadley CRU team. I am actually using and extending their software. The averaging they use is very simple. The monthly temperature for each cell in the 5×5 degree grid is the simple average of all stations lying within the cell. They use your first equation.
However I like your second equation better – so I will try and use that one and see what the result is !
>>
Taking an average of the T^4 value implies you are trying to take the average of the grey-body radiation. If you want to look at radiation do so, and take the average. Don’t look at temperature.
There is a lot of climate that depends on T not T^4 , such an average is not a better indication of average temperature.
@Dave
Good work – I like it very much.
Dave. Your 30 day average is off-set from your daily values. You are using monthly mean and attributing it to the end of the period not the middle. I would guess by that error that you will also be using a running mean. That is one god-awful frequency filter. Have a look at using a gaussian filter. It’s also a sliding average and can be done in much the same way just by adding a weighting factor.
What bothers me about anomalies is that the raw temperatures are rarely shown. When the monthly highs and lows are shown, it is clear that the highs increase only slightly, with the rise in the lows being the main change. During a warming phase there is an increase in average temperatures, but it only means normal summers, slightly milder winters and higher low temperatures at night, which is good for growing plants.
Even more so, we should measure the energy given the change in temperature..in other words the true measure has to be figuring out if the measured energy is increasing in the lower trop, which would indicate trapping. a parcel of saturated air at 80 degrees contains far more energy than a parcel of dry air at -20. It take very little energy change to raise arctic temps quite a bit and its more than offset by a 1 degree fall in the tropics. If we can quantify that, we will simply see “global warming” is merely distortions of existing temp patterns against the normals, and there is no net change in the energy budget and hence no global warming, just natural oscillations back and forth!
“The argument against this, as discussed above, is that we measure just the changes in temperature and these should be independent of any location bias i.e. CO2 concentrations rise the same everywhere ! ”
As far as I’ve been told the met office measures the temperatures and then calculates the changes. And you only need a gadget for measuring CO2 to know CO2 concentration doesn’t rise the same everywhere but fluctuate wildly between cities and rural places fields and forests and vice versa.
How can citing not be an issue? The most coverage today is amongst the EU countries high/free way systems. That introduce problems with more roads and far wider roads since and more stations concentrated around these road system compared to before 1990 and especially since 1960’s.
John, I agree. There are many techniques from a spatial stats perspectives that could improve the temperature record and the modelling. Unfortunately, most of them would require the raw station data, which seems to have been a bridge that has been burned.
@Clive Best
Perhaps like me you have wondered why “global warming” is always measured using temperature “anomalies” rather than by directly measuring the absolute temperatures ?
No.
In science there is a difference between accuracy and precision. Let me give an example. If 10 of 10 bowshots hit the centre of the target the accuracy is high and the precision is high. If 10 of 10 bowshots hit right from the centre of the target exact on the first ring, the accuracy is bad, but the precision is still high. That means that the process is perfect; there is only a precise offset, maybe from a constant floating wind into the right direction from the viewpoint of the bowman. This can be solved by a calibration, if needed.
If you have two temperature tables, maybe from different latitudes and there is an offset of a precise value, this means that the anomaly is equal for both latitudes. This means in a conclusion that the cause of the anomaly is independent from the latitude.
If you would measure absolute temperatures in Kelvin [K], it becomes clear that the values must be different on different latitudes. This tells you only that the measured temperatures are all precise and accurate, but you cannot conclude anything about the cause of the anomaly.
Why can’t we simply average the surface station data together to get one global temperature for the Earth each year ?
Sure you can do this, but I don’t see any scientific sense in ‘One global’ temperature for a calendar year’, because you flat the high frequency anomaly values from twelve month.
If you look at the monthly UAH satellite data , there are about 6 to 7 soft peaks per calendar year for GL, NH, SH and TR temperatures.
Now, the point is that already the solar tide function of the synodic system of Mercury and Earth effects the Sun 6.3040 times a year [period = 2 * ((1/0.24085) – (1/1.00002))], and this function is recognizable in the UAH GL temperature anomalies. Moreover the anomalies of the sealevel satellite data, where the (doubtful) linear increase of 3.2 mm per year is subtracted from the original data, show about 6.4 peaks per year [16 in 2.5 years] (red curve) nearly phase coherent to the solar tide frequency of Mercury and Earth.
That sealevel spikes are phase coherent to the UHA GL temperature spikes is not a surprise, because the volume of the heated global oceans is greater then at cold phases. But it surprises that these high frequency functions are in phase with the solar tide functions, leaded by the synodic couple of Mercury and Earth.
Because a frequency analysis (for precise, but not accurate data) is possible, and especially for original high frequency data of month, without the absolute accuracy of a global temperature, which can lead to the solar/terrestrial physics, it can be understood that an absolute “Temperature of The Year” has not really a scientific value.
V.
Sorry,
Correction links in above posting:
http://www.volker-doormann.org/images/uah_temp_4_r.gif
http://www.volker-doormann.org/images/sealevel_vs_abcn.gif
thanx
V.
Figure #1 and # 2
So all we really know well is America, Western Europe and Eastern Australia????
From that we get “Global” anomalies???
Nylo – “So if one spot is 2C higher than normal, it is quite reasonable to assume that, whatever the temperature is 1 kilometer further, it will be about 2C higher than normal as well for the corresponding location.”
Easy to disprove, take any region, the size of the area used in the grids, which way more than 1km, and then see on any day how much each location differs from its average. Living in the San Francisco Bay Area such a statement definitely does not apply locally, temperatures can range from ~12C to 38C within less than 30 miles, I’ve seen changes of around 10C across 3 miles. It’s hard to see how with such ranges a delta from normal would have the same value.
@J.Fischer
To try and answer your point: The way the annual global temperature anomaly is calculated assumes a global phenomenon because it uses a simple annual average af all (area weighted) individual station anomalies. This value is then presented to politicians as hard evidence that the Earth has been warming for the last 160 years as a result of AGW.
Suppose the Earth’s climate is actually driven by a series of regional decadal oscillations such as PDO AMO etc. The geographic spread of stations is such that the averaging will bias the global average to those regions with lots of stations. So if North America rises by 2 degrees while the Sahara falls 2 degrees then the net result will be strong positive. That is my main point. The result assumes global phenomena and rules out local phenomena.
@P.Solar
I have put the new normalised data for the annual temperature anomalies here. The 3 columns are 1) Global 2) North Hemisphere 3)South Hemisphere. You can also view the subroutine that calculates it here. If you use the Hadley PERL scripts then the “Field” passed to the routine should be Temperatures (not anomalies). Remember to respect MET Office copyrights.
@Allan MacRae
I agree that there is a tendency in the AGW mainstream to ignore data which doesn’t quite fit the story. I also find the whole paleoclimate debate fascinating. What really causes Ice Ages ? Why have they been coincident with minima of orbital eccentricity for 1 million years ? I have spent a long time thinking about this.. The next Ice age will begin in ~2000 years time. Can global warming save us ?
@P. Solar, Robert Clemenzi
I just did the T**4 averaging and the results can be seen here It is quite possible there could be a mistake as I did this in a hurry but the values are massively weighted to the warm zones and high radiative terms.
@P.Solar Correction:
You can also view the subroutine that calculates it here