The Final Straw

Guest post by Steven Mosher

In Climategate: The Crutape letters we tried to avoid accusing Professor Jones of CRU and UEA of outright fraud. Instead, based on the record found in the emails, we argued a case of noble cause corruption. I enlarged upon that charge at Pajama’s Media . Commenters took me to task for being too soft on Jones. Based on the extant text at that time I would still hold to my case. No skeptic could change my mind. But Phil Jones makes it hard to defend him anymore. On March 1st he testified before Parliament and there he argued that it was standard scientific practice to not share data.  Those who still insist on being generous with him could, I suppose, argue that he has no recollection, but in my mind he is playing with the truth and knows he is playing with the truth.

In 2002 Steve McIntyre had no publications in climate science. He wrote to Jones requesting temperature data. The history of their exchange is detailed in this Climate Audit Post. Jones sent data to McIntyre along with the following mail:

Dear Steve,

Attached are the two similar files [normup6190, cruwld.dat] to those I sent before which should be for the 1994 version. This is still the current version until the paper appears for the new one. As before the stations with normal values do not get used.

I’ll bear your comments in mind when possibly releasing the station data for the new version (comments wrt annual temperatures as well as the monthly). One problem with this is then deciding how many months are needed to constitute an annual average. With monthly data I can use even one value for a station in a year (for the month concerned), but for annual data I would have to decide on something like 8-11 months being needed for an annual average. With fewer than 12 I then have to decide what to insert for missing data. Problem also applies to the grid box dataset but is slightly less of an issue. I say possibly releasing above, as I don’t want to run into the issues that GHCN have come across with some European countries objecting to data being freely available. I would like to see more countries make their data freely available (and although these monthly averages should be according to GCOS rules for GAA-operational Met. Service.

Cheers

Phil Jones

We should note these things: Jones sent data. That was his practice. Jones is aware of the problems in releasing this data. Jones believes that these monthly averages should be released according to GCOS [WMO resolution 40] rules. In 2002 his practice is to release data to a total unknown with no history of publication. And Jones releases the data to him knowing that there are issues around releasing that data.

In 2004 Warwick Hughes exchanges a series of mails with Jones. In 2000 Jones appears to have a cooperative relationship with Hughes.  In 2004 the record shows the following

Dear Jean Palutikof, Dr P.D. Jones,

I was just reading your web page at; http://www.cru.uea.ac.uk/cru/info/warming/ and wish to access the station by station temperature data, updated through 2001 referred to on your CRU web page; http://www.cru.uea.ac.uk/cru/data/temperature/#datdow as

“Over land regions of the world over 3000 monthly station temperature time series are used.” Where can I download the latest station by station data which is a foundation of Dr Jones et al published papers ? Note, I am not asking about the CRU gridded data which I can see on your web site. Looking forward to your help,

Best wishes,

Warwick Hughes

Warwick,

The station data are not on the CRU web site. If you look at the GHCN page at NCDC, you’ll see they have stopped access and cited WMO Res. 40 for this. To my mind this resolution is supposed to make access free. However, it was hinted at to me a year or two ago that I should also not make the station data available.

The gridded data are there as you know.

I would suggest you take this up with WMO and/or GCOS. I have raised it several times with them and got nowhere.

Cheers

Phil

As Jones points out he believes that WMO Resolution 40 should make access free. Jones also says that he himself has taken up this issue with them. One can presume he took it up because he wanted to give access to data. Further, he knows that there may be agreements that preclude release of the data.

The start of 2005 is a critical point in the story line. Jones had cordial exchanges with Hughes in 2000. Jones shared data with McIntyre in 2002 and in 2004 Jones believed that the data should be shared. In 2005 he has been transformed. In January of 2005, McIntyre published a paper (MM05) critical of Mann. As luck would have it at this time former CRU employee Wigley sent an email to Jones about a flyer he has received that discusses FOIA. At this stage no FOIA have been sent to CRU. But Wigley and Jones are concerned about skeptics.  What ever willingness Jones had to share data is gone. Again, Jones shows a clear understanding of the existence of agreements:

Data is covered by all the agreements we sign with people, so I will be hiding behind them. I’ll be passing any requests onto the person at UEA who has been given a post to deal with them.

At the start of Feb 2005, Jones’ attitude toward data sharing becomes clearer and also contradictory. Some people can get this data in violation of agreements, while others who ask for it using legal means will be thwarted.

Mike,

I presume congratulations are in order – so congrats etc !

Just sent loads of station data to Scott [Rutherford]. Make sure he documents everything better this time ! And don’t leave stuff lying around on ftp sites – you never know who is trawling them. The two MMs  [McIntyre and McKittrick]  have been after the CRU station data for years. If they ever hear there is a Freedom of Information Act now in the UK, I think I’ll delete the file rather than send

to anyone. Does your similar act in the US force you to respond to enquiries within 20 days? – our does ! The UK works on precedents, so the first request will test it. We also have a data protection act, which I will hide behind. Tom Wigley has sent me a worried email when he heard about it – thought people could ask him for his model code. He has retired officially from UEA so he can hide behind that. IPR should be relevant here, but I can see me getting into an argument with someone at UEA who’ll say we must adhere to it !

Two weeks after the publication of MM05, prior to the issuance of any FOIA whatsoever, Jones contemplates destroying data rather than sharing it. But read closely. Jones sends this data to Scott Rutherford. So what’s the standard scientific practice? The data is covered by confidentiality agreements. Jones shared it with McIntyre in 2002, and now shares it with Rutherford in 2005. Jones knows it is covered by agreements and he’s questioned those agreements—except when he finds it convenient to hide behind those agreements. He violates them as he pleases. He shares data as he pleases. And if he is pushed to share it he contemplates destroying it.

On  Feb 21, 2005 Keith Briffa sends Jones a mail with a list of editorials that are critical of Dr. Mann for not releasing data. Jones replies to Warwick Hughes’ request for data that same day:

Warwick,

Hans Teunisson will reply. He’ll tell you which other people should reply.   Hans is “Hans Teunissen”

I should warn you that some data we have we are not supposed top pass on   to others. We can pass on the gridded data – which we do.  Even if WMO agrees, I will still not pass on the data.  We have 25 or so years invested in the work. Why should I make the data available to you, when your aim is to try and find something wrong with it. There is IPR to consider.

You can get similar data from GHCN at NCDC.  Australia isn’t restricted there.

Several European countries are. Basically because, for example, France doesn’t  want the French picking up data on France from Asheville. Meteo France  wants to supply data to the French on France. Same story in most of the  others.

Cheers

Phil

Jones has changed his attitude about the WMO. Prior to the publication of MM05 Jones believed that the WMO guidelines would make the data available. Moreover he argued with WMO that it should be released. Now, Jones changes his tune. He argues that he will not release the data even if the WMO agrees. His concern? Hughes will find something wrong with it.

When it comes to deciding whether to share data or not, standards have nothing to do with the decisions Jones made and he knows that. He knows he shared confidential data with Rutherford while he denied it to McIntyre and Hughes. He knows he regarded the confidentiality of those agreements quixotically. Violating them or hiding behind them on a whim. This was scientific malpractice. Lying about that now is beyond excuse.

April 2005 comes and we turn to another request from McIntyre:  There is a constant refrain amongst AGW defenders that people don’t need to share code and data. They argue that papers do a fine job of explaining the science: They argue that people should write their own code based on description in papers. Here is McIntyre’s request. Note that he has read the paper and tried to emulate the method:

Dear Phil,

In keeping with the spirit of your suggestions to look at some of the other multiproxy publications, I’ve been looking at Jones et al [1998]. The methodology here is obviously more straightforward than MBH98. However, while I have been able to substantially emulate your calculations, I have been unable to do so exactly. The differences are larger in the early periods. Since I have been unable to replicate the results exactly based on available materials, I would appreciate a copy of the actual data set used in Jones et al [1998] as well as the code used in these calculations.

There is an interesting article on replication by Anderson et al., some distinguished economists, here [1]http://research.stlouisfed.org/wp/2005/2005-014.pdf discussing the issue of replication in applied economics and referring favorably to our attempts in respect to MBH98.

Regards, Steve McIntyre

When you cannot replicate a paper based on a description of the data and a description of the method, standard practice is to request materials from the author. McIntyre does that. Jones’ “practice” is revealed in his mail to Mann:

From: Phil Jones <p.jones@xxxxxxxxx.xxx>

To: mann@xxxxxxxxx.xxx

Subject: Fwd: CCNet: DEBUNKING THE “DANGEROUS CLIMATE CHANGE” SCARE

Date: Wed Apr 27 09:06:53 2005

Mike,

Presumably you’ve seen all this – the forwarded email from Tim. I got this email from McIntyre a few days ago. As far as I’m concerned he has the data sent ages ago. I’ll tell him this, but that’s all – no code. If I can find it, it is likely to be hundreds of lines of uncommented fortran ! I recall the program did a lot more that just average the series. I know why he can’t replicate the results early on – it is because there was a variance correction for fewer series.

See you in Bern.

Cheers

Phil

Jones does not argue that code should be withheld because of IPR[Intellectual Property Rights]. It’s withheld because he is not sure he can find it and he suspects that it is a mess. More importantly Jones says he knows why McIntyre cannot replicate the results. Jones does not argue “standard scientific practice” to withhold code; he withholds code because it’s either lost, or sloppy and because it will allow McIntyre to understand exactly how the calculations were done.  This is malpractice.  Today when questioned whether people could replicate his work from the papers he wrote Jones “forgot this mail” and said they could replicate his work. And we should note one last thing. Jones again acknowledges sending data to McIntyre. So, what exactly is Jones’ notion of standard practice? To share or not to share? What the record shows is that Jones shared data and didn’t share data, confidential or not, on a basis that cannot be described as scientific or standard. He did so selectively and prejudicially. Just as he refused data to Hughes to prevent his work being checked he refuses information that McIntyre needs to replicate his published results. At the same time he releases that data to others.

That’s not the end of the story as we all know. In 2007 the first two FOIA were issued to CRU for data. One request for a subset of the data was fulfilled after some delay. The larger request was denied. By 2009 it became clear to McIntyre that the CRU data had also been shared with Webster. When McIntyre requested the very same data that Webster got from Jones, CRU started again with a series of denials again citing confidentiality agreements, inventing the terms of those agreements ex nihilo. Webster could have the data. McIntyre could not.

What the record shows is that Jones had no standard scientific practice of sharing or not sharing data. He had no consistent practice of abiding by or violating confidentiality agreements. He had his chance to sit before Parliament and come clean about the record. He had an opportunity to explain exactly why he took these various contradictory actions over the course of years.  Instead he played with the truth again.  Enough.