Maybe they need a statistical analysis class

19 09 2007

From Slashdot.org The Wall Street Journal has a sobering piece describing the research of
medical scholar John Ioannidis, who showed that in many peer-reviewed research
papers ‘most
published research findings are wrong
.’ The article continues: ‘These flawed
findings, for the most part, stem not from fraud or formal misconduct, but from
more mundane misbehavior: miscalculation, poor study design or self-serving data
analysis. [...] To root out mistakes, scientists rely on each other to be
vigilant. Even so, findings too rarely are checked by others or independently
replicated. Retractions, while more common, are still relatively infrequent.
Findings that have been refuted can linger in the scientific literature for
years to be cited unwittingly by other researchers, compounding the errors.’





Grilling the Data

19 09 2007

data_grill.gif

9/29/07 UPDATE: We are still waiting on Mr. Steve Bloom to answer this question: “Why is positive bias imparted in USHCN adjustments?”

He incorrectly asserts that he has been “banned” from this blog. Not true. Once he answers this question, that answer along with whatever else he has to say after that will be posted here. Otherwise we’ll continue to wait.

What say you, Mr. Bloom?
——————————————-
Given what NASA GISS has recently done with posting a change to the data methodology on the heels of an error which was embarrasing to them, (see Raising Walhalla) I think this review of a relevant paper might bear some examination:

An Introduced Warming Bias in the USHCN Temperature Database Reference

Balling Jr., R.C. and Idso, C.D. 2002. Analysis of adjustments to the United States Historical Climatology Network (USHCN) temperature database. Geophysical Research Letters 10.1029/2002GL014825.

Abstract http://www.agu.org/pubs/crossref/2002/2002GL014825.shtml and the full paper Download file

What was done:
The authors examined and compared trends among six different temperature databases for the coterminous United States over the period 1930-2000 and/or 1979-2000.

What was learned:
For the period 1930-2000, the RAW or unadjusted USHCN time series revealed a linear cooling of 0.05°C per decade that is statistically significant at the 0.05 level of confidence. The FILNET USHCN time series, on the other hand – which contains adjustments to the RAW dataset designed to deal with biases believed to be introduced by variations in time of observation, the changeover to the new Maximum/Minimum Temperature System (MMTS), station history (including other types of instrument adjustments) and an interpolation scheme for estimating missing data from nearby highly-correlated station records – exhibited an insignificant warming of 0.01°C per decade.

Most interestingly, the difference between the two trends (FILNET-RAW) shows “a nearly monotonic, and highly statistically significant, increase of over 0.05°C per decade.” With respect to the 1979-2000 period, the authors say that “even at this relatively short time scale, the difference between the RAW and FILNET trends is highly significant (0.0001 level of confidence).” Over both time periods, they also find that “the trends in the unadjusted temperature records [RAW] are not different from the trends of the independent satellite-based lower-tropospheric temperature record or from the trend of the balloon-based near-surface measurements.”

What it means:
In the words of the authors, the adjustments that are being made to the raw USHCN temperature data “are producing a statistically significant, but spurious, warming trend in the USHCN temperature database.” In fact, they note that “the adjustments to the RAW record result in a significant warming signal in the record that approximates the widely-publicized 0.50°C increase in global temperatures over the past century.” It would thus appear that in this particular case of “data-doctoring,” the cure is worse than the disease. In fact, it would appear that the cure IS the disease.

From the paper: Our analyses of this difference are in complete agreement with Hansen et al. [2001]
and reveal that virtually all of this difference can be traced to the adjustment for the time of observation bias. Hansen et al. [2001] and Karl et al. [1986]

The reviewer notes: “Our prescription for wellness? Withhold the host of medications being given and the patient’s fever will subside.”

Originally from CO2Science