NCDC's USHCN2 paper: some progress, some duck and cover

The fact that my work is mentioned by NCDC at all is a small miracle, even if it is “muted”, as Roger says. However, I’m pleased to get a mention and I express my thanks to Matt Menne for doing so.  Unfortunately they ducked the issue of the long term site bias contribution and UHI to the surface record. But, we’ll address that later – Anthony

From Roger Pielke Sr.’s Climate Science website

Comments On The New Paper “The United States Historical Climatology Network Monthly Temperature Data – Version 2 By Menne Et Al 2009

There is a new paper on the latest version of the United States Historical Climatololgy Network (USHCN). This data is used to monitor and report on surface air temperature trends in the United States. The paper is

Matthew J. Menne, Claude N. Williams, Jr. and Russell S. Vose, 2009: The United States Historical Climatology Network Monthly Temperature Data – Version 2.(PDF)  Bulletin of the American Meteorological Society (in press). [url for a copy of the paper added thanks and h/t to Steve McIntyre and RomanM on Climate Audit].

The abstract reads

“In support of climate monitoring and assessments, NOAA’s National Climatic Data Center has developed an improved version of the U.S. Historical Climatology Network temperature dataset (U.S. HCN version 2). In this paper, the U.S. HCN version 2 temperature data are described in detail, with a focus on the quality-assured data sources and the systematic bias adjustments. The bias adjustments are discussed in the context of their impact on U.S. temperature trends from 1895-2007 and in terms of the differences between version 2 and its widely used predecessor (now referred to as U.S. HCN version 1). Evidence suggests that the collective impact of changes in observation practice at U.S. HCN stations is systematic and of the same order of magnitude as the background climate signal. For this reason, bias adjustments are essential to reducing the uncertainty in U.S. climate trends. The largest biases in the HCN are shown to be associated with changes to the time of observation and with the widespread changeover from liquid-in-glass thermometers to the maximum minimum temperature sensor (MMTS). With respect to version 1, version 2 trends in maximum temperatures are similar while minimum temperature trends are somewhat smaller because of an apparent over correction in version 1 for the MMTS instrument change, and because of the systematic impact of undocumented station changes, which were not addressed version 1.”

I was invited to review this paper, and to the authors credit, they did make some adjustments to their paper in their revision. Unfortunately, however, they did not adequately discuss a number of remaining bias and uncertainty issues with the U.S. HCN version 2 data.

The United States Historical Climatology Network Monthly Temperature Data – Version 2 still contains significant biases.

My second review of their paper is reproduced below.

Review By Roger A. Pielke Sr. of Menne et al 2009.

Dear Melissa and Chet

I have reviewed the responses to the reviews of the Menne et al paper, and, while they are clearly excellent scientists, and have provided further useful information, unfortunately, they still did not adequately respond to several of the issues that have been raised. I have summarized these issues below:

1. With respect to the degree of uncertainty associated with the homogenization procedure, they misunderstood the comment. The issue is that in the creation of each adjustment [time-of-observation bias, change of instrument], there is a regression relationship that is used to create these adjustments. These regression relationships have an r-squared associated with them as well as a standard deviation. These deviations arise from the adjustment regression evaluation. These values need to be provided (standard deviations, r-squared) for each formula that they use.

Their statement that

“Based on this assessment, the uncertainty in the U.S. average temperature anomaly in the homogenized (version 2) dataset is small for any given year but contributes to an uncertainty to the trends of about (0.004°C)”

is not the correct (complete) uncertainty analysis.

2.

i) With respect to their recognition of the pivotal work of Anthony Watt, while they are clear on this contribution in their response; i.e.

“Nevertheless, we have now also added a citation acknowledging the work of Anthony Watts whose web site is mentioned by the reviewer. Note that we have met personally with Mr. Watts to discuss our homogenization approach and his considerable efforts in documenting the siting characteristics of the HCN are to be commended. Moreover, it would seem that the impetus for modernizing the HCN has come largely as a reaction to his work. “

the text itself is much more muted on this. The above text should, appropriately, be added to the paper.

Also, the authors bypassed the need to provide the existing photographic documentation (as a url) for each site used in their study. They can clearly link in their paper to the website

http://www.surfacestations.org/ for this documentation. Ignoring this source of information in their paper is inappropriate.

ii) On the authors’ response that

“Moreover, it does not necessarily follow that poorly sited stations will experience trends that disagree with well-sited stations simply as a function of microclimate differences, especially during intervals in which both sites are stable. Conversely, the trends between two well-sited stations may differ because of minor changes to the local environment or even because of meso-scale changes to the environment of one or both stations..”

they are making an unsubstantiated assumption on the “stability” of well-sited and poorly-sited stations. What documentation do they have that determines when “both sites are stable”? As has been clearly shown on Anthony Watt’s website, it is unlikely that any of the poorly sited locations have time invariant microclimates.

Indeed, despite their claim that

“We have documented the impact of station changes in the HCN on calculations of U.S. temperature trends and argue that homogenized data are the only way to estimate the climate signal at the surface (which can be important in normals calculations etc) for the full historical record “

is not correct. Without photographs of each site (which now exists for many of them), they have not adequately documented each station.

iii) The authors are misunderstanding the significance of the Lin et al paper. They state

“Moreover, the homogenized HCN minimum temperature data can be thought of as a fixed network (fixed in both location and height). Therefore, the mix of station heights can be viewed as constant throughout the period of record and therefore as providing estimates of a fixed sampling network albeit at 1.5 and 2m (not at the 9m for which differences in trends were found in Oklahoma). Therefore, these referenced papers do not add uncertainty to the HCN minimum temperature trends per se. “

First, as clearly documented on the Anthony Watts website, many of the observing sites are not at the same height above the ground (i.e. not at 1.5m or 2m). Thus, particularly for the minimum temperatures, which vary more with height near the ground, the height matters in patching all of the data together to create long term temperature trends. Even more significant is that the trend will be different if the measurements are at different heights. For example, if there has been overall long term warming in the lower atmosphere, the trends of the minimum temperature at 2m will be significantly larger than when it is measured at 4m (or other higher level). Including minimum temperature trends together will result in an overstatement of the actual warming.

The authors need to discuss this issue. Preliminary analyses have suggested that this warm bias can overstate the reported warming trend by tenths of a degree C.

iv) While the authors seek to exclude themselves from attribution; i.e.

“Our goal is not to attribute the cause of temperature trends in the U.S. HCN, but to produce time series that are more generally free of artificial bias.”

they need to include a discussion of land use/land cover change effects on long term temperature trends, which now has a rich literature. The authors are correct that there are biases associated with non-climatic and microclimate effects in the immediate vicinity of the observation sites (which they refer to as “artificial bias”), and real effects such as local and regional landscape change. However, they need to discuss this issue more completely than they do in their paper, since, as I am sure the Editors are aware, this data is being used to promote the perspective that the radiative effect of the well-mixed greenhouse gases (i.e. “global warming”) is the predominate reason for the positive temperature trends in the USA.

iv) The neglect of using a complementary data analysis (the NARR) because it only begins in 1979 is not appropriate. The more recent years in the HCN analyses would provide an effective cross-comparison. Also, even if the NARR does not separate maximum and minimum temperatures, the comparison could still be completed using the mean temperature trends.

Their statement that

” Given these complications, we argue that a general comparison of the HCN trends to one of the reanalysis products is inappropriate for this manuscript (which is already long by BAMS standards)”

therefore, is not supportable as part of any assessment of the robustness of the trends that they compute. The length issue is clearly not a justifiable reason to exclude this analysis.

In summary, the authors should include the following:

1. In their section “Bias caused by changes to the time of observation”

the regression relationship used in

“…the predictive skill of the Karl et al. (1986) approach to estimating the TOB was confirmed using hourly data from 500 stations over the period 1965-2001 (whereas the approach was originally developed using data from 79 stations over the period 1957-64)”

should be explicitly included with the value of explained variance (i.e. the r-squared value) and standard deviation, rather than referring the reader to an earlier paper. This uncertainty in the adjustment process has been neglected in presenting the trend values with its +/- values.

2. In their section “Bias associated with other changes in observation practice”

the same need to present the regression relationship that is used to adjust the temperatures due to instrument changes; i.e. from

“Quayle et al. (1991) concluded that this transition led to an average drop in maximum temperatures of about 0.4°C and to an average rise in minimum temperatures of 0.3°C for sites with no coincident station relocation.”

What is the r-squared and the standard deviation from which these “averages” were obtained?

3. With respect to “Bias associated with urbanization and nonstandard siting”,

as discussed earlier in this e-mail, the link to the photographs for each site needs to be included and citation to Anthony Watt’s work on this subject more appropriately highlighted.

On the application of “In contrast, no specific urban correction is applied in HCN version 2″, this conclusion conflicts with quite a number of urban-rural studies. They assume “that adjustments for undocumented changepoints in version 2 appear to account for much of the changes addressed by the Karl et al. (1988) UHI correction used in version 1.”

The use of text that concludes that this adjustment process “appear” to account for the urban correction of Karl et al (1988) indicates even some uneasiness by the authors on this issue. They need more text as to why they assume their adjustment can accommodate such urban effects. Moreover, the urban correction in Karl et al is also based on a regression assessment with an explained variance and standard deviation; the same data Karl used should be applied to ascertain if the new “undocumented changepoint adjustment” can reproduce the Karl et al results.

The authors clearly recognize this limitation also in their paragraph that starts with

“It is important to note, however, that while the pairwise algorithm uses a trend identification process to discriminate between gradual and sudden changes, trend inhomogenieties in the HCN are not actually removed with a trend adjustment..”

and ends with

“This makes it difficult to robustly identify the true interval of a trend inhomogeneity (Menne and Williams 2008).”

Yet, despite this clear serious limitation of the ability to quantify long term temperature trends in tenths of a degree C with uncertainties, they present such precise quantitative trends; e.g.

“0.071°and 0.077°C dec-1, respectively” (on page 15).

They also write that

“…there appears to be little evidence of a positive bias in HCN trends caused by the UHI or other local changes”

which ignores detailed local studies that clearly show positive temperature biases; e.g.

Brooks, Ashley Victoria. M.S., Purdue University, May, 2007. Assessment of the Spatiotemporal Impacts of Land Use Land Cover Change on the Historical Climate Network Temperature Trends in Indiana.

Christy, J.R., W.B. Norris, K. Redmond, and K.P. Gallo, 2006, Methodology and results of calculating Central California surface temperature trends: Evidence of human-induced climate change?, J. Climate, 19, 548-563.

Hale, R. C., K. P. Gallo, and T. R. Loveland (2008), Influences of specific land use/land cover conversions on climatological normals of near-surface temperature, J. Geophys. Res., 113, D14113, doi:10.1029/2007JD009548.

4. On the claim that

“However, from a climate change perspective, the primary concern is not so much the absolute measurement bias of a particular site, but rather the changes in that bias over time, which the TOB and pairwise adjustments effectively address (Vose et al. 2003; Menne and Williams 2008) subject to the sensitivity of the changepoint tests themselves.”

this is a circular argument. While I agree it is the changes in bias over time that matter most, without an independent assessment, there is no way for the authors to objectively conclude that their adjustment procedure captures these changes of bias in time.

Their statement that

“Instead, the impact of station changes and non-standard instrument exposure on temperature trends must be determined via a systematic evaluation of the observations themselves (Peterson 2006).”

is fundamentally incomplete. The assessment of the impact “of station changes and non-standard instrument exposure on temperature trends” must be assessed from the actual station location and its changes over time! To rely on the observations to extract this information is clearly circular reasoning.

As a result of these issues, their section “Temperature trends in U.S. HCN” overstate the confidence that should be given to the quantitative values of the trends and the statistical uncertainty in their values.

If this paper is published, the issues raised in this review need to be more objectively and completely presented. It should not be accepted until they do this.

I would be glad to provide further elaboration on the subjects I have presented in this review of their revised paper, if requested.

Best Regards

Roger A. Pielke Sr.