A Lazy Rainy Metadata Sunday

Today's weather in Northern California at Anthony's weather station at www.bidwellranchcam.com

Guest post by Steven Mosher

I’m feeling lazy today, after a few months of writing code around the metadata question, I have a post to write up about 230 land stations that are located in the ocean. But, I’m feeling lazy today, tired from doing basic quality checks on the data the “team” uses routinely and hopeful that NCDC will answer my mails about the corrections I think need to be made.

As we approach the anniversary of Climategate, with our book well stocked in the remainder bin and it’s pages lining bird cages in the offices at UEA,  I’m going to donate some excerpts to the WUWT community. Thanks for all your support. Thank you more for your criticism. And thank you to Anthony for hosting people with different views.

Steve McIntyre’s recent series of posts on Dr. Jones seminal 1990 paper on UHI, brought back to the foreground a comment made in the mails by Kevin Trenberth that seem particularly applicable to my current work on checking the accuracy of metadata in GHCN. And Judith Curry’s recent posts on climate science “dogma”, a term I don’t think is helpful to address the issues at hand, brought up the variety of methods that the “Team” either used or contemplating using in response to data requests from McIntyre and others. “We can say they are lazy.”

The issue, as Tom and I wrote in the book, is not the science. The mere fact that some small group of scientists expressed less than wholesome ideas about sharing data, the mere fact that they cast aspersions on those requesting the data or thought about casting aspersions, does not in and of itself establish or refute any scientific fact. The world is still warming, despite their all too human behavior. Unfortunately, the public seems less inclined to have the same high regard they once had for climate scientists. The Minnesota satirists have yanked the team off their pedestal and there is no answer to that form of humor. That first laugh was the last laugh. And no, this

does not count as humor despite the pedigree of the writers. You do not answer the satire of midwesterners with self parody.

As we review the statements of the team below, I need to add this: It is neither fruitful to make too much of the comments or to brush them aside as inconsequential. They don’t change the science, but they do show that some would do well to examine their attitudes and beliefs, especially their attitudes towards citizens who want to check the numbers for themselves.

Excerpt from Climategate: the Crutape letters

Climate Audit again is on the radar of Phil Jones of CRU and Kevin Trenberth of UCAR. Ben Santer, a former CRU employee, is also alarmed at what is transpiring. Inspired by McIntyre’s work, Doug Keenan is reviewing the Jones 1990 case and notices some discrepancies in the work of Jones’ co author Wang, a scientist from China. Construing Wang’s work as academic fraud, (a charge later dismissed by Wang’s university), Keenan starts writing letters to Jones. The case is more interesting for the attitudes and preconceptions it reveals about Jones, Trenberth and others than it is about the questionable work of Wang. Trenberth on April 21 to Jones:

I am sure you know that this is not about the science. It is an attack to undermine the science in some way.  In that regard I don’t think you can ignore it all, as Mike suggests as one option, but the response should try to somehow label these guys and lazy and incompetent and unable to do the huge amount of work it takes to construct such a database.  Indeed technology and data handling capabilities have evolved and not everything was saved.  So my feeble suggestion is to indeed cast aspersions on their motives and throw in some counter rhetoric.  Labeling them as lazy with nothing better to do seems like a good thing to do.  How about “I tried to get some data from McIntyre from his 1990 paper, but I was unable because he doesn’t have such a paper because he has not done any constructive work!” :

There is no evidence at all that McIntyre has ever wanted to ‘undermine the science.’ He’s really just asking for the tools to do what they should have done themselves—check their numbers.

Rather than answer Keenan’s rather minor issue (which the university will eventually do), Trenberth suggests a personal attack, suggests impugning their motives. Rather than provide the data, Trenberth suggests a false attack on the abilities of those requesting the data. Ironically, the community reading on the internet has far more capability as a whole than Jones at managing large datasets and documents. Many of the engineers, scientists and academics that regularly visit Climate Audit could give lessons on the subject. Trenberth’s attack on McIntyre and his readers is reminiscent of the early attacks on bloggers by the mainstream media who tried to label them as “pajamas media.” What Trenberth failed to realize is that they were an army of Davids ready to tackle that large database work and in the end provide detailed documentation that NOAA would actually request for its own papers.

Mann weighs in with his views and he politicizes the problem and fails to understand that the story has a life of its own, independent of politics and the Main Stream Media:

So they are simply hoping to blow this up to something that looks like a legitimate controversy. The last thing you want to do is help them by feeding the fire.  Best thing is to ignore them completely. They no longer have their friends in power here in the U.S., and the media has become entirely unsympathetic to the rants of the contrarians at least in the U.S.–the Wall Street Journal editorial page are about the only place they can broadcast their disinformation

Jones also explains some of the confusion about the actual sources of the data. In CRU FOIA responses, CRU has argued that they should not have to supply the data they used because it is available from GHCN and NCAR. Jones reveals that this is not the case:

As for the other request, I don’t have the information on the sources of all the sites used in the CRUTEM3 database. We are adding in new datasets regularly (all of NZ from Jim Renwick recently) , but we don’t keep a source code for each station. Almost all sites have multiple sources and only a few sites have single sources. I know things roughly by country and could reconstruct it, but it would take a while.  GHCN and NCAR don’t have source codes either. It does all come from the NMSs – well mostly, but some from scientists.

As has been pointed out earlier, the construction of a global temperature index is largely a book keeping job. Yet Jones seems singularly ill-equipped to keep a handle on the data or accurate records of where data comes from. His breezy assertion that he ‘could probably reconstruct’ the database record could not be more sharply in contrast to McIntyre’s style—or accepted best practice in data handling and management circles. McIntyre’s work in the mining industry auditing reports is focused on the issue of provenance.

Santer’s solution to this “problem” of Climate Audit readers request for data is not a careful exposition of what data sources were used but rather this, from April 25th:

I looked at some of the stuff on the Climate Audit web site. I’d really like to talk to a few of these “Auditors” in a dark alley.

Jones’s best thought at clearing up this confusion in data sources and data provenance is to deliberately confuse the matter more. He writes:

Ben,

Thanks for the thoughts. I’m in Geneva at the moment, so have a bit of time to think. Possibly I’ll get the raw data from GHCN and do some work to replace our adjusted data with these, then make the Raw (i.e. as transmitted by the NMSs). This will annoy them more, so may inflame the situation.

Reviewing the ideas the scientists have for dealing with data requests we have the following: Lie about the dedication and work habits of the requester, ignore requests, beat requesters up in a dark alley, and deliberately confuse the provenance of the data. Later they will add obstruction of the FOIA process to their bags of tricks, demonstrating that they should never in the future be trusted with data that is fundamental to our understanding of the climate. Releasing the data and letting the “auditors” discredit themselves never occurs to them.

(pgs 99-100)

0 0 votes
Article Rating

Discover more from Watts Up With That?

Subscribe to get the latest posts sent to your email.

60 Comments
Inline Feedbacks
View all comments
November 8, 2010 2:13 am

Sorry Steven,
How can there be a lag time between when CO2 goes up into the atmosphere, and when it starts causing more warmth to stay close to the ground.
As far as we know, CO2 causes extra warmth by backstopping warmth rising from the Earth out to space, much like insulation does in the roof of a house. That effect should be immediate, it either does or it doesn’t. I don’t expect to have to wait a while before my house feels warmer, once I’ve put some insulation in.
What is the mechanism by which CO2 needs to stay in the atmosphere for a while before it starts having its effect? If this were true, the first thing we should notice is that the upper atmosphere should warm first with that warmth spreading downwards as more of it gets backstopped. This hasn’t happened, so there cannot be more warmth “in the pipeline”.
You say that CO2 should enhance the warming during an upwave and cause less cooling during a downwave, and I agree with you, yet we find that the last warm wave only lasted 24 years from 1974 to 1998. Surely this should have lasted longer with all that extra CO2, No?
Sure, GISS may show that 2010 as a whole was the warmest year, but only by extrapolating a few thermometers on land to across the whole of the Arctic, which is sea. Yet we find that none of the other datasets agree with this, and the DMI which uses actual measurements show no extra warming across the Arctic. Even the monthly anomalies on the satellite records haven’t reached the high in 1998, which would surely be a valid expectation if we lived in a warming world.
I’m just not seeing why we in the West have to uproot our lifestyles to the extent that we are being told we will have to. And I’m certainly not seeing how this is so big a problem that we have to brainwash our children into thinking that a chemical which is the fount of all life is “pollution”.

November 8, 2010 2:21 am

Steve, I really appreciate your post here regarding the irrefutable evidence of bad behaviour w.r.t. handling the science, by the key scientists. You’ve collected them together very well and I wish Muir Russell would read them here.
However, I disagree over the science, and the importance of Keenan’s forensics. The whole scientific establishment has bought into UHI growth being negligible w.r.t. global temperature trends, on the strength of Jones’ and Wang’s papers. But I see every indication that their figure of 0.05 deg is out by a factor of 10 or so, that UHI growth’s distortion of temperature trends is not negligible, and that if the UHI growth factor were properly assessed, it would cause the correlation of global temperatures with solar changes to be clearly visible again, thus making the CO2 factor irrelevant.
Anyway, while CO2 rise, at its current concentration, has been asserted to cause warming, there is no evidence in the real world to back up this assertion – whereas there is evidence that CO2 changes lag temperature changes by 800 years or so. I’ve looked for years for this evidence but all I’ve found is brazen assertions. Please correct me if I’m wrong, let me know if you know of any such proof or evidence.

Tenuc
November 8, 2010 2:22 am

Steven Mosher says:
November 7, 2010 at 4:33 pm
“Tenuc:
The harry file is meaningless. Sorry it just is. Even if the code WAS connected to anything that CRU currently do it would still be meaningless.”
Sorry Mosh, but your wrong. What the Harry file shows is that the meta-data sitting behind the temperature measurements was a complete mess (interestingly the same topic of this very post regarding the NCDC meta-data). Indeed it was so bad that Jones was willing to break the law to stop people discovering that the data-set being used to show global warming was unfit for purpose.
No amount of hand-waving about the ‘science still being correct’ can prevent this conclusion – GIGO has to apply.

Dave in Delaware
November 8, 2010 4:36 am

Steven Mosher writes in the main post:
” … does not in and of itself establish or refute any scientific fact. The world is still warming, despite their all too human behavior.”
If I install a Franklin Stove in my basement, and stoke it up, there is a fair chance that I could heat my home that way all winter.
Or I could light a single match, and light sequential matches as each burns out. Even if I double that to two matches, it is doubtful that an observer would feel much difference across the room, let alone the whole house.
And yet the physics is the same in both cases.
Those who focus over much on the stove or matches may completely miss the fact that I also have a functional heat pump that warms the house in winter and cools it in summer.
The question is not whether CO2 has an affect, but rather how much of an affect? Global Warming is real, but may or may not be driven much by additional CO2. When looking for potentially small changes, the accuracy of the temperature data becomes critical to making a determination. When told – Trust us, you need not go to the basement to look for yourselves – I become concerned that there is something to hide. That well intentioned idiot with the matches might just burn the house down by mistake.

JimBrock
November 8, 2010 6:47 am

Are these sites really in the water? Looks like a misfit to me.

kuhnkat
November 8, 2010 11:31 am

Steven Mosher,
I have heard similar statements that Climategate does NOT change the underlying science. This is excellent misdirection. Most Climate Science is NOT Science as it is not falsifiable and DEFINITELY NOT REPEATABLE!!! What you people loosely call Climate Science is not. It is Cargo Cult Science.
Climate Science is based mostly on a series of extremely poor (if not fraudulent) papers like Jones UHI and Mann’s Hockey Stick. It is also based on papers that were NOT ALLOWED TO BE PUBLISHED which would have kept the conversation on a more reasonable level. If there had been an open exchange of ideas and no Cargo Cult Science there would have been no AGW scare and no billions of dollars spent on it.
Even if there were more to the alledged science, I would point out that it very well DOES affect our perception of it. We only know this by its secondary effects. What was the motivation for ALLEDGEDLY honest scientists to completely jump the shark and start the gatekeeping, passing off garbage as reliable scientific research, and LYING about it?? What so corrupted these men so that their first reaction to the slightest honest criticism was to circle the wagons and start breaking laws?? (remember Phil Jones and others only have the statute of limitations to thank for not having to stand trial. Oh, and why would their colleagues in Australia, New Zealand, and possibly other countries fudge their temp records to help??
No, the only reasonable conclusion that can be drawn from Climategate is that these men KNEW their science wasn’t and had to protect it by any means necessary, like smearing honest critics, lying, hiding and possibly destroying data and evidence. Someone who believes in their own work and how solid it is would not dream of putting their careers on the line like that. I believe they already KNEW their jobs would be on the line if there was any inquiry because they KNEW it was Cargo Cult Science.
Finally, since each of those persons are not known to live frugal, low energy lives, eschewing plane travel, limos, heaters and other high energy activities, it would seem that they do not believe that they must be examples of how we should deal with the impending catastrophe they were helping to promote. Like most hypocrites they are not believers in their own lies, but, wish to continue rooting at the trough.
The radiative physics you and other luke-warmers hang onto is only theory. That is, the warming computed under ideal situations may or may not happen as it is only one component of the system. The Cargo Cult Science has twisted so much of physics that it is really ludicrous that you still hang onto their falsified pronouncements. Unless you use adjustments that have never been proven for general use, there IS NO WARMING AS CLAIMED!!! The adjustments were calculated based on specific stations and are smeared over ALL the stations. For the tiny amount of signal you are claiming you would need to survey every station regularly and use much higher resolution equipment with a much higher coverage of the earth to gather data appropriate to the task, and it would still need to include humidity data to be useful.
Actually, I changed my mind. Climategate really doesn’t affect the science. It was Cargo Cult before ClimateGate and it is STILL Cargo Cult!! ClimateGate DID change many people’s perception of Climate Science, appropriately I might add.

Tim Clark
November 8, 2010 11:56 am

Steven mosher says: November 7, 2010 at 6:42 pm
more C02 will warm the planet. This is known physics.
the addition of C02 does not cause immediate warming.
More GHG? you get a warmer planet than you would otherwise, not cooler.
Nothing I’ve seen or read can convince me that the physics I’ve used to build things that work is wrong.

Steven, you’re missing the point. Other posters are saying:
More GHG? More C02 in a closed system will warm the planet. You get a warmer planet than you would otherwise, if the planet did not have internal mechanisms, involving water, whch has served for thousands of years to maintain an equilibrium.

Eric Anderson
November 8, 2010 2:42 pm

Steve Mosher wrote: “others too who want to take issue with the whole GHGs do warm the planet issue.”
This is a strawman, plain and simple. No-one is taking issue with the idea that GHGs warm the planet. That’s what a GHG is — essentially by definition — so the statement is nothing more than a trivial tautology.
What I am hearing some posters question, however, is:
– By how much will x% increase in CO2 over x number of years increase the global temperature? This needs to be answered in the context of the overall climate system, not a lab-based “all things being equal” scenario.
– Given that the temperatures have not been lockstep with CO2 increase over the past century, on what basis can we assume that CO2 was the main cause or even a meaningful contributor to the increase in temperatures over the past century?
– Given the numerous uncovered issues about station coverage, data collection, UHI, infilling, etc., how confident can we even be that there has even been statisically significant warming? And if there was warming, might it be just as readily explained as part of a natural cycle?
– If CO2 is supposed to lead to warmer temperatures, what physical mechanism underlies the “lag” time proposed by some to explain the lack of recent warming? Why is the lag being seen now if it wasn’t seen in earlier decades, such as the 90’s?
That is just a quick toss out of some of the open issues and I probably haven’t done a very good job of articulating these questions, but there are lots of good open questions about CO2’s role as a driver of global temperatures. All of them are part of the science. Our confidence in the answers is necessarily tempered by knowing that the top scientists in the field don’t have answers, and in some cases actively worked to conceal that fact. So when someone questions CO2’s role or the role of man’s emissions, they are typically not questioning the basic physical fact that GHG’s warm the planet, all things being equal — everyone knows that. They are questioning the many significant additional questions that remain — all of which are very much part of the “science.”

November 9, 2010 2:43 pm

The long wait is over.
Come January, Congressman Darrell Issa will begin investigations of the EPA and its regulation of CO2, Michael Mann and how his grant money is used to perpertuate the Hockey Stick/Tree Ring fraud, and Climategate.
I am waiting on the edge of my seat!

Ammonite
November 10, 2010 2:08 am

Eric Anderson says: November 8, 2010 at 2:42 pm
“If CO2 is supposed to lead to warmer temperatures, what physical mechanism underlies the “lag” time proposed by some to explain the lack of recent warming? Why is the lag being seen now if it wasn’t seen in earlier decades, such as the 90′s?”
Hi Eric. I am not entirely sure how other commentators are relating “lag” time and “lack of recent warming”. I see the two as distinct and unrelated. If you heat a pot of water it takes time for the water to reach its final temperature (lag). If a trend is small compared to the volatility of a sequence there will be periods where the sequence is flat or negative (lack of recent warming). If this persists there comes a point where the “trend” must be called into question. We are not there yet. In particular, averaging the temperature in ten year blocks yields a rising sequence and 2010 is shaping to be a very warm year.