From the told ya so department, comes this recently presented paper at the European Geosciences Union meeting.
Authors Steirou and Koutsoyiannis, after taking homogenization errors into account find global warming over the past century was only about one-half [0.42°C] of that claimed by the IPCC [0.7-0.8°C].
Here’s the part I really like: of 67% of the weather stations examined, questionable adjustments were made to raw data that resulted in:
“increased positive trends, decreased negative trends, or changed negative trends to positive,” whereas “the expected proportions would be 1/2 (50%).”
And…
“homogenation practices used until today are mainly statistical, not well justified by experiments, and are rarely supported by metadata. It can be argued that they often lead to false results: natural features of hydroclimatic times series are regarded as errors and are adjusted.”
The paper abstract and my helpful visualization on homogenization of data follows:
Investigation of methods for hydroclimatic data homogenization
Steirou, E., and D. Koutsoyiannis, Investigation of methods for hydroclimatic data homogenization, European Geosciences Union General Assembly 2012, Geophysical Research Abstracts, Vol. 14, Vienna, 956-1, European Geosciences Union, 2012.
We investigate the methods used for the adjustment of inhomogeneities of temperature time series covering the last 100 years. Based on a systematic study of scientific literature, we classify and evaluate the observed inhomogeneities in historical and modern time series, as well as their adjustment methods. It turns out that these methods are mainly statistical, not well justified by experiments and are rarely supported by metadata. In many of the cases studied the proposed corrections are not even statistically significant.
From the global database GHCN-Monthly Version 2, we examine all stations containing both raw and adjusted data that satisfy certain criteria of continuity and distribution over the globe. In the United States of America, because of the large number of available stations, stations were chosen after a suitable sampling. In total we analyzed 181 stations globally. For these stations we calculated the differences between the adjusted and non-adjusted linear 100-year trends. It was found that in the two thirds of the cases, the homogenization procedure increased the positive or decreased the negative temperature trends.
One of the most common homogenization methods, ‘SNHT for single shifts’, was applied to synthetic time series with selected statistical characteristics, occasionally with offsets. The method was satisfactory when applied to independent data normally distributed, but not in data with long-term persistence.
The above results cast some doubts in the use of homogenization procedures and tend to indicate that the global temperature increase during the last century is between 0.4°C and 0.7°C, where these two values are the estimates derived from raw and adjusted data, respectively.
Conclusions
1. Homogenization is necessary to remove errors introduced in climatic time
series.
2. Homogenization practices used until today are mainly statistical, not well
justified by experiments and are rarely supported by metadata. It can be
argued that they often lead to false results: natural features of hydroclimatic
time series are regarded errors and are adjusted.
3. While homogenization is expected to increase or decrease the existing
multiyear trends in equal proportions, the fact is that in 2/3 of the cases the
trends increased after homogenization.
4. The above results cast some doubts in the use of homogenization procedures
and tend to indicate that the global temperature increase during the
last century is smaller than 0.7-0.8°C.
5. A new approach of the homogenization procedure is needed, based on
experiments, metadata and better comprehension of the stochastic
characteristics of hydroclimatic time series.
Full text:- Presentation at EGU meeting PPT as PDF (1071 KB)
- Abstract (35 KB)
h/t to “The Hockey Schtick” and Indur Goklany
UPDATE: The uncredited source of this on the Hockey Schtick was actually Marcel Crok’s blog here: Koutsoyiannis: temperature rise probably smaller than 0.8°C
Here’s a way to visualize the homogenization process. Think of it like measuring water pollution. Here’s a simple visual table of CRN station quality ratings and what they might look like as water pollution turbidity levels, rated as 1 to 5 from best to worst turbidity:
In homogenization the data is weighted against the nearby neighbors within a radius. And so a station might start out as a “1” data wise, might end up getting polluted with the data of nearby stations and end up as a new value, say weighted at “2.5”. Even single stations can affect many other stations in the GISS and NOAA data homogenization methods carried out on US surface temperature data here and here.
In the map above, applying a homogenization smoothing, weighting stations by distance nearby the stations with question marks, what would you imagine the values (of turbidity) of them would be? And, how close would these two values be for the east coast station in question and the west coast station in question? Each would be closer to a smoothed center average value based on the neighboring stations.
UPDATE: Steve McIntyre concurs in a new post, writing:
Finally, when reference information from nearby stations was used, artifacts at neighbor stations tend to cause adjustment errors: the “bad neighbor” problem. In this case, after adjustment, climate signals became more similar at nearby stations even when the average bias over the whole network was not reduced.
D. J. Hawkins says:
………………………………………………………………………………………….
Since the temperature in Walmart at the thermometer display area is likely to be, in fact, uniform, your method has some merit. However, in the real world, it’s unlikely that stations separated by even as little as 5-10 miles see absolutely identical conditions. If it’s 75F at “A” and 71F at “B”, the “real” temperature at both of them isn’t likely to be 73F.
———————————————————————————-
I share the same concern. But there are elegant (and sometimes useful) methods (interpolation, kriging etc) for estimating values on a partially sampled gradient. Some better than others. None are perfect.
As George Box famously said: All models are wrong. Some are useful.
Steve Keohane says:
July 17, 2012 at 7:27 am
……
Put the sensor of your digital thermometer in large enclosed water container, read it at sunset for the day- and at sunrise for the night-time averages.
Johanus says:
July 17, 2012 at 6:49 am
Here’s an example of how you yourself can use homogenization to help guarantee the next thermometer you buy will be more accurate.
Go to a place that sells cheap thermometers (Walmart etc). Normally there will be 5 or 10 instruments on display of various brands. You will immediate notice that they are all predicting different temperatures. Maybe some will read in the mid or low 70′s, some in the high 70′s. There will always be a maverick or two with readings way of into the impossible range.
Which thermometer, if any, should you buy?
Well, it is likely that there are several instsruments in the bunch reporting fairly accurately. Best way to find the most accurate thermometer is to whip out your pocket calculator, add up all the temps and divide by the number of thermometers. (Throw away any obviously bogus readings first, such as a thermometer reading zero.) The resulting average value is most likely to be closest to the “real” temperature.
That is how homogenization works, on a small scale.
That might work when there are 10 thermometers in the same location. Try doing that with your 10 thermometers spread across 10 different stores with varying store temperatures. The thermometer you purchase in that situation will likely be no more accurate than a random selection.
Temperature varies with location so it’s absurd to suggest that one thermometer is incorrect simply because it reads a different temperature to another thermometer at a different location. Unless there is a known fault with a particular instrument, homogenization is more likely to introduce errors than to correct any.
There’s some sort of Climate Change Uncertainty Principle that the super high temps are reality and happening but not where they’re being measured, and a cat.
In title… Homogenization… one “o” is missing.
[REPLY: Fixed. Thanks. -REP]
This probably sounds like a broken record by now. But I’ve been claiming for about four years that half of the warming trend is due to “adjustments”. And a portion of the remainder is due to coming out of an LIA. This again reinforces the climate sensitivity numbers that were produced by Lindzen and Spencer. The climate sensitivity is somewhere between .5C and 1.2C per CO2 doubling.
Steven Mosher says:
July 17, 2012 at 7:52 am
Anthony.
Of course I’ll bring this up today at our BEST meeting. Dr. K is an old favorite. That said, since we dont use homogenized data but use raw data instead I’m not sure what the point will be
————————————————————————————
Steve McIntyre explains what the point is.
It is pretty amazing how lame the BEST guys deal with severe issues.
Their UHI paper is just not plausible, but they don’t care.
BEST is not compatble with ocean temperature data but they don’t care.
Nor is it compatible with satellite data, who cares ?
And if this would only affect homogenized datasets (McIntyre thinks otherwise), BEST would be an outlier and the the whole point of BEST of confirming other data sets falls apart. And that’s the point where Mosher is not sure what the point is…
How are Steirou and Koutsoyiannis regarded by the warmists? Are they regarded as sceptics, or what?
D J Hawkins says:
I think that’s a straw man. Nobody homogenizes temperatures like that. Of course different locations, even quite nearby, have different absolute temperatures. But if you see the temperature record at one location suddenly change relative to all the other nearby locations, it maybe suggests that something has changed in the way the temperature is being recorded, or in the location of the thermometer. If the data are being used to try to understand climate, it makes sense to try to correct for such problems.
The Fundamental Premise, that “Homogenization is necessary to remove errors introduced in climatic time series,” is intellectually dishonest.
It presumes a priori knowledge of what “error-free” data would look like, and totally disregards the physical fact of natural variability and its contribution to the uncertainty of whatever ‘trends’ are later inferred.
Either ‘smoothing’ OR ‘trending’ can be performed on a raw data set, but to perform BOTH is wrong and inevitably produces misleading results.
Between this, UHI, micro-site contamination, solar changes (both TSI and the Svensmark affect), oceanic cycles (PDP, AMO, etc.)
Not a lot of warming left for CO2 to be responsible for.
Careful Zeke, realclimatescientists tried the same sort of attacks before against Koutsoyiannis and got their ears pinned back.
http://climateaudit.org/2008/07/29/koutsoyiannis-et-al-2008-on-the-credibility-of-climate-predictions/
He published an updated version in 2009 or 2010 to shut them up. I’m just sayin’
@Nigel Harris
> But if you see the temperature record at one location suddenly change
> relative to all the other nearby locations, it maybe suggests that something
> has changed in the way the temperature is being recorded, or in the location
> of the thermometer. If the data are being used to try to understand climate,
> it makes sense to try to correct for such problems.
But what if the anomaly at that one location is legimitate (due to local precipitation or whatever). How can you justify “correcting” the observation? The expected value for the temperature in the region must account for all the temps, plus or minus the trend. Else you can’t correctly balance the incoming and outgoing radiation.
See my post above for an example of such a “legitimate” anomaly:
http://wattsupwiththat.com/2012/07/17/new-paper-blames-about-half-of-global-warming-on-weather-station-data-homgenization/#comment-1034593
Since UAH satellite temperatures show an increase of ~0.4 deg over the past 30 years then ALL warming over the past century must have been since the 1970 – i.e just about the time the CO2 effect would be expected to become distinguishable from natural variability.
This paper does nothing to debunk the CO2 effect. On the contrary it suggests that TSI, Svensmmark, PDO and other natural effects are negligible over time. Note Satellite readings are not contaminated by UHI.
Johanus says:
July 17, 2012 at 6:49 am
Skeptikal says:
July 17, 2012 at 6:24 am
*The data doesn’t need homogenization. If one location is hotter or colder than a neighbouring location, that’s weather. Raw data is the only data that’s worth anything. Once you bend the data out of shape, it becomes worthless.*
You’re wrong. The data does need some kind homogenization to correct for inaccurate or poorly situated instruments. We also need it to be able to summarize the weather over larger regions to make predictions and comparisons.
Here’s an example of how you yourself can use homogenization to help guarantee the next thermometer you buy will be more accurate.
…
That is how homogenization works, on a small scale.
Skeptickal is right here. The process you describe is basic QC statistics and applies to the instrument(s) you purchase. You are not planning to purchase the entire lot, including the mavericks, and then employ them all to generate analytical quality data.
Also, consider what happens if you do not have even a clue as to which data instruments were the good ones and which were the bad ones (e.g. GHCN) – or you got all your new instruments out of their packages and droppped them all before individually marking the “bad” instruments, and in fact did not know in which direction the “bad” instruments were biased. No “correction” you could possibly apply is more than guesswork. Also, since you do not have clue as to which instruments are the good and which are not, your “correction” should be applied in both directions weighted possibly by the fact that when you strolled out of the store you knew there were two bad ones in the lot. The outcome of the correction is precisely as good as your guess limited by the quality and specific biases of the worst instruments. If you cannot identify which one are bad and how they err, then …
As to whether homogenization is necessary at all, suppose you are interested in detecting trends, which is in fact the major point of the AGW argument – the trend in the global average temperature employed as a measure of climate. Whether your instruments are “good” or “bad”, if they respond essentially uniformly to temperature changes – this assumes that the instrument error is simply related shifted scales on the instrument rather than problems with the sensor itself – then you can look at the trends from each instrument without data hamburgerization, and still estimate a regional trend, one with less potential error than you would have after homogenization. There’s no need for the actual data from your instruments to be “adjusted.” It should be left strictly alone. Nor do you even need to estimate the global average temperature.
@West, Mosher
According to this
http://www.ipcc.ch/ipccreports/tar/wg1/345.htm
the IPCC shows 3.5°C ECS from equlibrium to equilibrium (of temperature and CO2) for a doubling of CO2 in 70 years. The lag is hundreds of years. IPCC-TCR (from equlibrium to end of doubling after 70 years) is 2°C.
Since CO2 was already increasing in the year 1900, the first calculation of West goes from CO2 increasing to CO2 increasing, which gives the same temperature difference as from equilibrium to equlibrium (3.5°C according to IPCC for doubling, thus years 70 to 140).
Therefore I see no need to consider any lag and think 1.3°C is the West-ECS.
(P.S. I assume that IPCC-ECS is the same thing as Mosher-ECR)
Steve M makes a brief comment on this here:
http://climateaudit.org/2012/07/17/station-homogenization-as-a-statistical-procedure/
Steven Mosher says:
No, I calculated the TCR @ur momisugly 1.3 & took a stab @ur momisugly ECR (minus feedbacks) by taking out about a decade of CO2 increase. Admittedly, not a “climate science” approved method, but for some reason I think around 10 years is plenty enough time to realize the temperature difference from a change in forcing, considering we see temperature changes daily and seasonally from changes in forcing with short lags (see Willis’ post); and whether feedbacks are positive, negative or neutral is still a very open question in my mind. Why should I accept (on faith?) an extremely long lag and net positive feedbacks when I can’t observe that in the real world and have no substantial evidence for it? It’d be like believing in the North American Wood Ape (Bigfoot). Anyway, taking your method of 1.5-2x the ECR would be from 1.95 to 2.6 IF all the warming is due to CO2 increase.
Only one of the three sentences cited from the “peer reviewed paper”, can actually be found in the abstract, which is lightly reviewed, the others are from the talk, which is not reviewed. Anthony get your facts right! A longer response can be found on my blog:
http://variable-variability.blogspot.com/2012/07/investigation-of-methods-for.html
In this community it does not seem to be well known how homogenization is actually performed, thus I also link an introductory post on homogenization:
http://variable-variability.blogspot.com/2012/01/homogenization-of-monthly-and-annual.html
Alexej Buergin says:
temperature difference as from equilibrium to equlibrium
Excellent point!
The temperature in 1900 is in equilibrium [as much?] as the temperature in 2000 and [as much?] as the temperature will be in 2100. Hmmm.
John Finn “Note Satellite readings are not contaminated by UHI.”
Are we sure? NASA has no trouble finding huge amounts of UHI from their satellites.
“Summer land surface temperature of cities in the Northeast were an average of 7 °C to 9 °C (13°F to 16 °F) warmer than surrounding rural areas over a three year period, the new research shows.”
http://www.nasa.gov/topics/earth/features/heat-island-sprawl.html
http://www.nasa.gov/images/content/505071main_buffalo_surfacetemp_nolabels.jpg
If one is buying a thermometer, and expects any degree of accuracy at all, it would be wise to check its calibration at two points bracketing the temperature range of interest.
After all, a broken clock is right twice a day.
cd_uk says: “Victoria
You have me at an advantage I can’t view the Venema et al., 2012 paper only the abstract. Although the abstract does allude to the metric their using to ascertain “better” performance. From the abstract:”
Venema et al., 2012 is published in a open access journal, so you can read it:
http://www.clim-past.net/8/89/2012/
I thought, that open-access was important here given the amount of rubbish and confusion being spread by these kind of blogs.
“To be honest I don’t know enough about this to make a judgement but it does stand to reason if you’re adjusting data, in order to account for irregular superficial effects such as experimental error, then these should have no influence, or nominally so, on the final result: as many pushed up as pushed down. …unless (and as stated), there is good experimental reasons for the net gain. What is this reason?”
You guys are afraid that the urban heat island pollutes the global climate signal. Homogenization aims to remove such artifacts and if we only had the UHI homogenization would make the temperature trend smaller. That is, it would have an influence. Otherwise, one would not need to homogenize data to compute a global mean temperature.
In practice homogenization makes the temperature trend stronger. This is because temperatures in the past were too high. In the 19th century many measurement were performed at North facing walls, especially in summer the rising of setting sun would still burn on these instruments. Consequently these values were too high and homogenization makes the lower again. Similarly, the screens used in the first half of the 20th century were open to the North and to the bottom. This produced too high temperatures on days with little wind and strong sun as the soil would heat up and radiate at the thermometer. These too high temperatures are reduced by homogenization. In the US you have similar problems with the time of observation bias and with the transition from Stevenson screens to automatic weather stations.
The size of the corrections is determined by comparison with neighboring stations, but not by averaging as Anthony keeps on writing wrongly. The key ideas are explained on my blog:
http://variable-variability.blogspot.com/2012/01/homogenization-of-monthly-and-annual.html
Steven Mosher says:
July 17, 2012 at 8:07 am
John West.
You just calculated the transient climate response. ( TCR) at 1.6.
the ECR ( equillibrium Climate response) is anywhere from 1.5 to 2x higher.
so if you calculate a TCR ( what you did) then you better multiply by 2…
Giving you 3.2 for a climate sensitivity. (ECR)
>>>>>>>>>>>>>>>
You cannot make this statement unless you know to some degree of precision what the time constant is, which you do not. Further, even if you knew the time constant for one particular forcing, you must also know the time constants for all other forcings that are active to the point of still being significant, what their sign is, and how far along each is in terms of a total of 5 time constants. There are WAY too many factors all in place at the same time, we know the time constant of pretty much none of them, let alone which ones are at the beginning of the cycle and which ones are at the end.
REPLY: I was of the impression that it was “in press” but I’ve change the wording to reflect that. Hopefully we’ll know more soon. – Anthony
I had not seen the reply before my previous answer. Thank you for correcting this factual error. Maybe you should write less posts per day. Almost any post on a topic I am knowledgeable about contains factual errors. Your explanation of how homogenization works could not have been more wrong. Why not take some time to study the topic? One post a day is also nice.
REPLY: Thank you for your opinion, reading your blog, clearly you wish to prevent opinion and discourse, unless its yours. See what Willis says below. See also what statistician Steve McIntyre has to say about it. Be sure of yourself before making personal accusations – Anthony