A Lazy Rainy Metadata Sunday

Guest post by Steven Mosher

I’m feeling lazy today, after a few months of writing code around the metadata question, I have a post to write up about 230 land stations that are located in the ocean. But, I’m feeling lazy today, tired from doing basic quality checks on the data the “team” uses routinely and hopeful that NCDC will answer my mails about the corrections I think need to be made.

As we approach the anniversary of Climategate, with our book well stocked in the remainder bin and it’s pages lining bird cages in the offices at UEA,  I’m going to donate some excerpts to the WUWT community. Thanks for all your support. Thank you more for your criticism. And thank you to Anthony for hosting people with different views.

Steve McIntyre’s recent series of posts on Dr. Jones seminal 1990 paper on UHI, brought back to the foreground a comment made in the mails by Kevin Trenberth that seem particularly applicable to my current work on checking the accuracy of metadata in GHCN. And Judith Curry’s recent posts on climate science “dogma”, a term I don’t think is helpful to address the issues at hand, brought up the variety of methods that the “Team” either used or contemplating using in response to data requests from McIntyre and others. “We can say they are lazy.”

The issue, as Tom and I wrote in the book, is not the science. The mere fact that some small group of scientists expressed less than wholesome ideas about sharing data, the mere fact that they cast aspersions on those requesting the data or thought about casting aspersions, does not in and of itself establish or refute any scientific fact. The world is still warming, despite their all too human behavior. Unfortunately, the public seems less inclined to have the same high regard they once had for climate scientists. The Minnesota satirists have yanked the team off their pedestal and there is no answer to that form of humor. That first laugh was the last laugh. And no, this

does not count as humor despite the pedigree of the writers. You do not answer the satire of midwesterners with self parody.

As we review the statements of the team below, I need to add this: It is neither fruitful to make too much of the comments or to brush them aside as inconsequential. They don’t change the science, but they do show that some would do well to examine their attitudes and beliefs, especially their attitudes towards citizens who want to check the numbers for themselves.

Excerpt from Climategate: the Crutape letters

Climate Audit again is on the radar of Phil Jones of CRU and Kevin Trenberth of UCAR. Ben Santer, a former CRU employee, is also alarmed at what is transpiring. Inspired by McIntyre’s work, Doug Keenan is reviewing the Jones 1990 case and notices some discrepancies in the work of Jones’ co author Wang, a scientist from China. Construing Wang’s work as academic fraud, (a charge later dismissed by Wang’s university), Keenan starts writing letters to Jones. The case is more interesting for the attitudes and preconceptions it reveals about Jones, Trenberth and others than it is about the questionable work of Wang. Trenberth on April 21 to Jones:

I am sure you know that this is not about the science. It is an attack to undermine the science in some way.  In that regard I don’t think you can ignore it all, as Mike suggests as one option, but the response should try to somehow label these guys and lazy and incompetent and unable to do the huge amount of work it takes to construct such a database.  Indeed technology and data handling capabilities have evolved and not everything was saved.  So my feeble suggestion is to indeed cast aspersions on their motives and throw in some counter rhetoric.  Labeling them as lazy with nothing better to do seems like a good thing to do.  How about “I tried to get some data from McIntyre from his 1990 paper, but I was unable because he doesn’t have such a paper because he has not done any constructive work!” :

There is no evidence at all that McIntyre has ever wanted to ‘undermine the science.’ He’s really just asking for the tools to do what they should have done themselves—check their numbers.

Rather than answer Keenan’s rather minor issue (which the university will eventually do), Trenberth suggests a personal attack, suggests impugning their motives. Rather than provide the data, Trenberth suggests a false attack on the abilities of those requesting the data. Ironically, the community reading on the internet has far more capability as a whole than Jones at managing large datasets and documents. Many of the engineers, scientists and academics that regularly visit Climate Audit could give lessons on the subject. Trenberth’s attack on McIntyre and his readers is reminiscent of the early attacks on bloggers by the mainstream media who tried to label them as “pajamas media.” What Trenberth failed to realize is that they were an army of Davids ready to tackle that large database work and in the end provide detailed documentation that NOAA would actually request for its own papers.

Mann weighs in with his views and he politicizes the problem and fails to understand that the story has a life of its own, independent of politics and the Main Stream Media:

So they are simply hoping to blow this up to something that looks like a legitimate controversy. The last thing you want to do is help them by feeding the fire.  Best thing is to ignore them completely. They no longer have their friends in power here in the U.S., and the media has become entirely unsympathetic to the rants of the contrarians at least in the U.S.–the Wall Street Journal editorial page are about the only place they can broadcast their disinformation

Jones also explains some of the confusion about the actual sources of the data. In CRU FOIA responses, CRU has argued that they should not have to supply the data they used because it is available from GHCN and NCAR. Jones reveals that this is not the case:

As for the other request, I don’t have the information on the sources of all the sites used in the CRUTEM3 database. We are adding in new datasets regularly (all of NZ from Jim Renwick recently) , but we don’t keep a source code for each station. Almost all sites have multiple sources and only a few sites have single sources. I know things roughly by country and could reconstruct it, but it would take a while.  GHCN and NCAR don’t have source codes either. It does all come from the NMSs – well mostly, but some from scientists.

As has been pointed out earlier, the construction of a global temperature index is largely a book keeping job. Yet Jones seems singularly ill-equipped to keep a handle on the data or accurate records of where data comes from. His breezy assertion that he ‘could probably reconstruct’ the database record could not be more sharply in contrast to McIntyre’s style—or accepted best practice in data handling and management circles. McIntyre’s work in the mining industry auditing reports is focused on the issue of provenance.

Santer’s solution to this “problem” of Climate Audit readers request for data is not a careful exposition of what data sources were used but rather this, from April 25th:

I looked at some of the stuff on the Climate Audit web site. I’d really like to talk to a few of these “Auditors” in a dark alley.

Jones’s best thought at clearing up this confusion in data sources and data provenance is to deliberately confuse the matter more. He writes:

Ben,

Thanks for the thoughts. I’m in Geneva at the moment, so have a bit of time to think. Possibly I’ll get the raw data from GHCN and do some work to replace our adjusted data with these, then make the Raw (i.e. as transmitted by the NMSs). This will annoy them more, so may inflame the situation.

Reviewing the ideas the scientists have for dealing with data requests we have the following: Lie about the dedication and work habits of the requester, ignore requests, beat requesters up in a dark alley, and deliberately confuse the provenance of the data. Later they will add obstruction of the FOIA process to their bags of tricks, demonstrating that they should never in the future be trusted with data that is fundamental to our understanding of the climate. Releasing the data and letting the “auditors” discredit themselves never occurs to them.

(pgs 99-100)