An investigation of USHCN station siting issues using a cleaned dataset

While BEST is making a public relations train wreck for themselves by touting preliminary conclusions with no data analysis paper in place yet to support their announcements, there have been other things going on in the world of surface data integrity and uncertainty. This particular paper is something I have been assisting with (by providing metadata and images, plus being part of the review team) since May of last year. Regular WUWT readers may recognize the photo on the cover as being the USHCN station sited over a tombstone in Hanksville, UT photographed by volunteer Juan Slayton.

What is interesting about it is that the author is the president of a private company, WeatherSource who describes themselves as:

Weather Source is the premier provider of high quality historical and real-time digital weather information for 10,000’s of locations across the US and around the world.

What is unique about this company is that they provide weather data to high profile, high risk clients that use weather data in determining risk, processes, and likely outcomes. These clients turn to this company because they provide a superior product; surface data cleaned and deburred of many of the problems that exist in the NCDC data set.

I saw a presentation from them last year, and it was the first time we had ever met. It was quite some meeting, because for the first time, after seeing our station photographs, they had visual confirmation of the problems they surmised from examining data. It was a meeting of the minds, data and metadata.

Here’s what they say about their cleaned weather data:

Weather Source maintains a one-of-a-kind database of weather information built from a combination of multiple datasets merged into one comprehensive “super” database of weather information unlike anything else on the market today. By merging together numerous different datasets, voids in one dataset are often filled by one of the other datasets, resulting in database that has higher data densities and is more complete than any of the original datasets. In addition all candidate data are quality controlled and corrected where possible before being accepted into the Weather Source database. The end result is a “super” database that is more complete and more accurate than any other weather database in existence – period!

It is this “super” database that is at the core of all Weather Service products and services. You can rest assured knowing that the products and services obtained through Weather Source are the best available.

They say this about historical data:

Comprehensive and robust are the two words which best describe the Weather Source Historical Observations Database.

Comprehensive because our database is composed of base data from numerous sources. From these we create merged datasets that are more complete than any single source. This allows us to provide you with virtually any weather information you may need.

Robust because our system of data quality control and cleaning methods ensure that the data you need is accurate and of the highest quality.

As an example, here is a difference comparison of their cleaned data set with a GHCN station:

The processes they use are not unlike that of BEST; the emphasis is on solving problems with the historical data gathered by the NOAA volunteer operated Cooperative Observer Network (COOP). The COOP network also has a smaller subset, the U.S. Historical Climatology Network (USHCN) which is a hand picked set of stations from researchers at the National Climatic Data Center (NCDC) based on station record length and moves:

USHCN stations were chosen using a number of criteria including length of record, percent of missing data, number of station moves and other station changes that may affect data homogeneity, and resulting network spatial coverage.

The problem is that until the surfacestations.org project came along, they never actually looked at the measurement environment of the stations, nor did they even bother to tell the volunteer operators of those stations that they were special, so that they would perform an extra measure of due diligence in data gathering and ensuring that the stations met the most basic of siting rules, such as the NOAA 100 foot rule:

Temperature sensor siting: The sensor should be mounted 5 feet +/- 1 foot above the ground. The ground over which the shelter [radiation] is located should be typical of the surrounding area. A level, open clearing is desirable so the thermometers are freely ventilated by air flow. Do not install the sensor on a steep slope or in a sheltered hollow unless it is typical of the area or unless data from that type of site are desired. When possible, the shelter should be no closer than four times the height of any obstruction (tree, fence, building, etc.). The sensor should be at least 100 feet from any paved or concrete surface.

As many know and NOAA concedes in their own publications on the issue, about 1 in 10 weather stations in the USHCN meet the 100 foot rule. That is the basis of the criticisms I have been making for sometime – that siting has introduced unaccounted for biases in the data.

WeatherSource is a private company catering to business using a special data cleaning process that removes many, but not all of the data integrity issues and biases that have been noted. I’ve seen a detailed flowchart and input/output comparisons on their processes and have confidence that they are addressing issues in a way that provides a superior data set. However, being a business, they have to keep their advantage close, and I have agreed not to disclose their processes which are their legal intellectual property.

The data cleaning techniques of the BEST team made known to me when I met with them in February 2011 were similar, but they promised to use publicly available data and to make the process fully transparent and replicable, as shown on their web page:

So, knowing that BEST had independently identified surface data issues that needed attention, and because there was some overlap in the ideas pioneered by the privately held intellectual properties of WeatherSource with the open source promise of BEST, you can imagine my excitement at the prospect.

Unfortunately, on March 31st in full public view before Congress BEST fumbled the ball, and in football parlance there was a turnover to the opposing “team”. BEST released some potentially buggy and admittedly incomplete preliminary results (which I’ve seen the basis for) with no transparency or replicability whatsoever. BEST doesn’t even have a paper submitted yet.

However, the WeatherSource team has produced a paper, and they have it online today. While it has not peer reviewed by a journal, it has been reviewed by a number of professional people,  and of course with this introduction on WUWT, is about to be reviewed worldwide in the largest review venue it could possibly get.

It should be noted, that the data analysis was done entirely by Mr. Gibbas of WeatherSource. My role was limited to providing metadata, photographs from our surfacestation project, and editing support.

Here’s the full paper:

An Investigation of Temperature Trends from weather station observations representing various locations across Utah (21.5mb PDF) by Mark Gibbas

Bear in mind, because this is a private company, and because small companies must hold their IP rights close to maintain competitive advantages, the data cleaning method can be discussed in general terms, but they cannot provide the code for review without compromising their IP rights. However, the source data used in this paper can be made available on request, as WeatherSource has tentatively offered access to it provided that the end user is doing research, does not use it for commercial gain, and does not republish it in its original form. Use the WUWT contact page if you would like to request a copy of the WeatherSource cleaned data after first agreeing to the caveats listed. Anonymous requests will be ignored. The metadata list of stations is available here: Utah-stations-metadata (PDF) Utah-stations-metadata (Excel .xls)

Here are some excerpts:

BACKGROUND

However, in this study done on a much smaller scale, our goal was to examine the long-term observational temperature records for weather stations scattered across the state of Utah, and attempt to identify local influences acting upon them. For instance, it is well known that urban areas can exhibit a sharp warming trend due to the influence of expanding urban infrastructure. Less well known are other influences upon temperature such as those associated with modern agriculture.

It is not until these local-scale influences are properly quantified that one can begin to infer fluctuations or trends that may be occurring on larger-scales, such as state, region, country, or global. This investigation is an important step towards that ultimate goal—but more importantly, shows that this goal is still difficult to divine given the uncertainty and methodology currently employed to extract a climate change signal from the surface temperature record.

…

APPROACH

To better understand the various influences on the observed temperature record, this study identified and examined numerous weather stations with long observational histories from urban, agricultural and natural wilderness or low impact locations (a low impact location is one that has experienced minimal alterations to its natural state) from around the state of Utah.

Initially, weather data from the National Climatic Data Center (NCDC) U.S. Historical Climatology Network version 2 (USHCNv2 2009)6 were to be used for this study. However, numerous data issues were discovered within the USHCN and as a result this data set was abandoned. Instead, the study used data from the original NCDC U.S. Cooperative Summary of the Day Data (referred to as the DSI-3200 dataset), that was cleaned via a comprehensive regression based data cleaning methodology. A brief explanation of the issues found within the USHCN will follow.

Within the cleaned DSI-3200 dataset, the study identified weather stations within Utah that had at least 50 years of data. Using tools such as Google Earth and demographic information based on the US census, these weather station locations were

ranked in terms of how urban and how agricultural they were. The ranking was done on a scale of 1 to 5 with 5 being the highest. Low impact stations were identified as locations that had low urban and low agricultural rankings.

Based on the urban and agricultural indices, the weather stations were divided into three groups representing urban, agricultural and low impact locations.

…

CAUSES OF TEMPERATURE OBSERVATION VARIANCE

One would assume that if you had two weather stations that were about 30 miles apart in nearly identical elevation and terrain and both recorded temperature consistently for fifty years, the time-series of these stations should correlate nearly perfectly and that both stations would show the same general trend. However, in practice we find that such stations often have significant differences in their observations even though they were both subject to the same large-scale weather patterns. But how could this be? The answer most often is related to changes that occur at the weather stations sites that influence the temperatures that are recorded.

…

Having established that station changes can artificially compromise the capture of real

weather trends, an obvious question has to be: So what good are all the station data if each station is subject to false trends due to station changes? When considering very small numbers of stations this is indeed a problem, but when more stations are averaged as a group, the effects of random station changes largely cancel out. The reason for this is related to the statistical principal that random actions (station changes) average out to zero whereas non-random actions (urban growth) do not.

Thus, to more accurately determine the trend of the urban, agricultural, and low impact

groups, we must average the members into a composite trend for each group. Chart 5 provides a view of the average trends from each group. JJA refers to the summer season months of June, July and August. DJF refers to the winter season months of December, January, and February. TMax refers to the trend of the daily maximum temperature and TMin refers to the same for the daily minimum temperature. The vertical axis is the trend in degrees per year.

In studying chart 5, we note the following:

• The trend in the minimum temperature is more significant than that of the maximum temperature. A thorough discussion as to why this is the case would be quite involved, so for now we’ll simply state that the dominance of the minimum temperature trend is as expected.
• The urban group of stations exhibits the strongest trend, followed by the agricultural group, then the low impact group.
• The difference between the agricultural group to that of the low impact group is more distinct in the summer than in the winter.
• The trend of the low impact minimum temperature is nearly the same in both the summer and winter seasons, whereas the trend of minimum temperature for the urban and agricultural groups is stronger in the summer.

Chart 6 provides a view of the aggregated summer and winter trend for the three groups in degrees Fahrenheit per decade. The urban value of 0.42F is on par with the published estimates of 0.36F to 0.44F per decade for current global warming.

The net trend for the agricultural group is 0.27F per decade which is 37% less than the urban group. The low impact group has trend of 0.20F per decade which is 52% lower than the urban group.

CONCLUSIONS

This study has established that there are significant differences in temperature trends

between weather stations sited at urban, agricultural, and low impact locations. In addition, it has been shown that urban stations exhibit significantly higher trends, which are created by numerous urban factors that are not related to global CO2 levels. Evidence has also been provided that agricultural locations experience a warming trend related to increases in humidity. Given that both urban and agricultural environments likely have warming trend factors that are unrelated to CO2 based warming, it makes sense that to get a more accurate determination of CO2 based global warming, station observations from low impact locations need to be given much greater weight. Currently that is not the case since the vast majority of weather observations originate at airports, which by design are near urban areas.

…

DAILY TEMPERATURE CLEAN METHOD

The method to clean the DSI-3200 data used in this study utilizes regression estimates from neighboring stations to validate, correct and fill missing values as needed to produce a high quality continuous time-series of daily temperature data. The cleaning method does not attempt to correct for urban warming, discontinuities or any other external influences upon the recorded temperature. Key features of the cleaning method include:

• Adjustment of morning, afternoon and midnight reporting times to produce working time-series arrays that are consistent across all stations used in the cleaning process.
• All regressions are based on seasonal time windows.
• Two types of regressions are done for each neighbor to target pair; a seasonal regression based on three years of data centered on a date of interest, and a seasonal regression based on all data. If the three year regression is stable, this value is preferred; otherwise the long term seasonal regression value is used.
• Regression estimates from all neighbors are collect into a distribution, which is analyzed and compared to the original value from the target station. If the target station original value is confirmed based on statistical tests, it is retained. Otherwise if it is found to be suspect it is replaced with the regression estimate from the station with the greatest correlation to the target station. In all cases, missing values within the target time-series are filled using the estimates from the highest correlating neighboring station.

The method outlined above has been found to produce values that are most consistent with the original time-series. This has been determined by qualifying accurate values then artificially removing these values and running the degraded time-series through the clean method. The resulting filled values are then compared to the original qualified values that were removed. These tests have routinely produced replacement values that when compared to the original values, have an R2 or 0.985 or greater.

– end of excerpts

One issue that was noted was how a formerly remote and rural USHCN station, the Snake Creek Powerhouse, had become surrounded by an Olympic sports center and later an irrigated golf course. It was made a special case study #2.

From case#2

The next plot provides the addition of the NCDC USHCN adjusted data. Unfortunately, the adjustment does not correct the issues with the Snake Creek site and in fact degrades the quality of the data further. While the trend for the Snake Creek station is suspect to begin with, the Clean DSI-3200 data set shows an upward trend of 0.246 or roughly a quarter degree Fahrenheit per decade. In contrast the USHCN adjusted data show a cooling trend of -0.05 or roughly -0.5 degrees Fahrenheit per decade.

Clearly this adjustment is wrong; otherwise this location in UT would be experiencing some of the most extreme cooling on the planet.

The adjustment applied to Snake Creek can only be regarded as alarming, and results in the following conclusions:

1. It is suspected that the adjustments are the result of a simple automated method

that applies a linear adjustment to the data such that the end portions of the linear

adjustment can have results that are unrealistic.

2. It is apparent that there is a lack of review and quality checking of the results.

…

end excerpt

While Utah is most certainly not the world, this paper does demonstrate what can be learned by doing a detailed look at station metadata in concert with temperature data cleaning and analysis. As noted in the paper, like BEST’s early presentation, it has not tried to correct for UHI or for humidity (moist enthalpy) which tends to effect night time temperatures the most, particularly in irrigated agricultural areas. Human experience knows this effect in southwest USA dry deserts, which can be very cold at night due to low humidity versus the Southeast USA, where humidity makes for warm summer nights. The question is how much are night-time temperatures being elevated by the addition of humidity from irrigation?

A study of California’s central valley in 2006 by Dr. John Christy of UAH did in fact do a small scale study in the California Central Valley where he took extreme care to hand code in reams of metadata in order to discover this effect. See:

Christy on irrigation and regional temperature effects

Until we get a method in place to look at issues like this and UHI, the surface record is still filled with uncertainties.

However, I find it encouraging that a private company that provides weather data where risk management is involved, where being right provides economic reward and being wrong means economic failure, has seen and dealt with the issues of NOAA COOP station biases, at least partially. Much more is needed.

The mere fact that company exists for the express purpose to create and sell corrected and cleaned data from our government surface temperature record, and they they have clients that need it, speaks volumes.

Details on this paper from Mark Gibbas is here at SPPI with a full download available