Quality Control of pre-1948 Cooperative Observer Network Data

I stumbled across this paper “Quality Control of pre-1948 Cooperative Observer Network Data” on the AMS website while looking for something else. I know the work of many of the people who authored this paper, and thus I have reason to believe it is well done for the issues they were attempting to address. However, one thing that was not addressed, and really needs to be, is a site bias timeline. While it will be difficult to go back and find photos of weather station siting for the entire network, there does exist one resource already in NCDC’s hand that could be utilized for the task: B44 forms. These are site surveys, completed by USWB, and later NWS personell that set up these COOP stations. Each B44 includes a site sketch to scale, such as this one from a station in California:

Putting together these B44 forms over the life of a station would enable the creation of a timeline that shows building and road proximity biases that may exist, which are just as important as catching the errors that creep into the human transcription of the data.

Unfortunately, the B44 forms are not available to the public because of NCDC’s privacy concerns for observers. I can appreciate that. However that just doesn’t hold up anymore because NCDC provides all the tools needed on their Metadata MMS website now to identify the location of the observer, including a Google image map and accurate lat/lon, plus on their “free data” section of the NCDC website, you can download the B91 station originals. These are the daily record of observations done in the observer’s own hand, and have the name of the observer written on them in most cases. They are the source of the US climate data and the focus of the QC paper named above.

A sample of the B91 Record of Climatological Observations Form – click for larger image

Here is another B91 form, done by a private observer I happen to know in Livermore, CA His name is listed at the bottom. Put his last name, “Livermore”, and “weather” together in this Google search, and you get all the details you need to proceed. You can even see his picture courtesy of the NWS COOP awards page here.

It is now an easy task to locate anyone and any address given a name, Google, public records, and mapping programs like Google Earth. Since the “privacy issue” no longer exists due to this volume of info available on individual observers, perhaps it is time for NCDC to free the B44 forms.

Here are some excerpts from the paper:

The National Weather Service’s (NWS) cooperative observer network (COOP) is the core climate network of the U.S. In operation since the late 19th century, it consists primarily of volunteer observers using standard equipment provided by the NWS. The typical suite of elements, observed daily, include precipitation, maximum temperature, minimum temperature, snowfall, and snow depth. Some stations report only precipitation variables. A few stations observe other variables such as pan evaporation and soil temperature.

These observations were routinely digitized beginning with 1948. Although there have been occasional projects to retroactively digitize selected data (e.g. Kunkel, et.al. 1998), most pre-1948 observations remained available only on paper or microfiche. This

has recently changed. The U.S. Congress has provided funding to the National Climatic Data Center (NCDC) for the Climate Database Modernization Program (CDMP 2001). The goal of CDMP is to convert data only available in hard-copy form to computerized formats.

The pre-1948 COOP data was one of the first data sets chosen for this conversion.

The authors have undertaken a project to quality control this data set. This paper describes the QC procedures and discusses certain aspects of the data set.

And also:

There are a number of potential sources of errors in the data set. Some primary examples include observer errors in reading the instruments, observer errors in writing the observations on the form, liquid separation in the thermometers, and legibility of the forms. There are also a number of potential issues with continuity of the data for each station due to changes in instrumentation, observing practices, and exposure.

The primary purpose of the QC for this project was to identify the largest errors in individual values, particularly those that might affect analyses of extreme events.