Over at Climate Audit there is renewed interest in data availability with McIntyre asking whether journals that don’t guarantee data archiving (The Holocene in this case) should be cited in IPCC reports.
It happened that yesterday Phil Jones of CRU gave a talk at KNMI in De Bilt, The Netherlands, where he also talked about availability of data, in this case the data behind the recently published Crutem4 and Hadcrut4 graphs. The talk itself was pretty neutral, just explaining what had been done to produce these two datasets. However at the end Jones made a statement that is relevant to the long lasting discussions about data availability:
For raw temperature data you have to contact the NMS’s.
NMS’s stands for National Meteorological Services, like the Met Office in the UK or KNMI in The Netherlands. The good news is, as we can also read in his latest Crutem4 paper, that CRU will make all data available. However the bad news is that these data have already been homogenized by the NMS’s and the original data are not available at the Crutem4 webpage.
Jones explained that he and Moberg concluded already in 2003 that the homogenisation of temperature data is best done at the NMS’s. So for Crutem4 whenever possible they used these homogenized data of the NMS’s directly, as can be seen at their webpage.
Now in itself this is a fair approach. If the NMS’s cannot figure out what happened with their stations in the past and how best to control for station moves and instrument changes who else can?
…
We know that in some countries adjustments to raw data determine a large part of the trend. In New Zealand sceptics fight (see also here) with NIWA (the NMS of New Zealand) over the adjustments made to the raw data. The temperature trend in the raw data is only 0,3 degrees per century while the adjusted data show a trend of 1 degree per century. Jones uses the adjusted NIWA data in Crutem4. Later this year the High Court in New Zealand will consider this case.
Jones seemed satisfied with the new situation. Anyone asking him for the raw data in the future will be referred to the NMS’s.
================================================================
Anthony: This is particularly worrisome, because as we’ve seen, metadata for GHCN global stations is very poor to virtually non-existent, and from what we know, GHCN and CRU takes very little metadata into account. While the NMS’s may have a better handle on metadata, given the disparity of quality of met services globally, this pretty much ensures that no individual researcher is going to get their hands on a complete set of all data. Phil Jones is essentially blowing off the issue saying “let them figure it out, not our responsibility”. Whatta guy!
================================================================
Crok continues:
In his future answers to sceptics asking for data he can almost copy this paragraph of Joelle Gergis blowing off McIntyre when he requested some tree ring data from her:
This list allows any researcher who wants to access non publically available records to follow the appropriate protocol of contacting the original authors to obtain the necessary permission to use the record, take the time needed to process the data into a format suitable for data analysis etc, just as we have done. This is commonly referred to as ‘research’.
In the case of Crutem4 the raw data in many cases is also not publicly available and anyone interested has to contact each of the NMS’s trying to get these data. There is no guarantee at all that they will release the data.
Now although Jones was very obstructive to data requests from sceptics in the past, I don’t say that Jones is to blame for the current situation. At least he tried to get permission from the NMS’s to release the data as he promised in this Nature article in 2009, which was also covered in several Climate Audit posts (see here for example). In the Nature article Jones said:
“We’re trying to make them all available,” says Jones. “We’re consulting with all the meteorological services — about 150 members [of the World Meteorological Organization] — and will ask them if they are happy to release the data.” A spokesperson for the Met Office confirmed this, saying “we are happy for CRU to take the lead on this, as they are their data”.
But getting the all-clear from other nations won’t be without its challenges, says Jones, who estimates that it could take several months. In addition, some nations may object if they make money by selling their wind, sunshine and precipitation data.
In his new paper on Crutem4 he reports back on this attempt:
In November 2009, the UK Met Office wrote on our behalf to all NMSs to determine if we could release the versions of their monthly temperature series that we held. Of the about 180 letters, we received 62 positive replies, 5 negative replies, and the remainder did not reply.
These results are worrisome in itself. Almost two-thirds of the NMS’s didn’t even bother to answer to a request concerning one of the most important climate graphs in the world. For these countries Crutem4 uses the GHCN data.
===============================================================
The need for a journal that demands all data, (used and excluded) up front, along with methodology, code, and supplementary material to ensure the work is reproducible, before even considering a paper for review is becoming clearly obvious. – Anthony
h/t to Dr. Roger Pielke Sr.
“The need for a journal that demands all data, (used and excluded) up front, along with methodology, code, and supplementary material to ensure the work is reproducible, before even considering a paper for review is becoming clearly obvious.”
Capable parties only need the data.
I argued many years ago for the need for a climatological equivalent of CERN, the global nuclear physics community.
I suspect very strongly that effective data storage, access and FOI processing would be more effective in such an organisation.
What is critical, however, is that oversight is carried out by non-scientists and that policy making is separated healthily from scientists.
They have managed to duck the issue of ‘raw’ data and ‘availability’ by passing the buck to supplier NMS’s.
This doesn’t help anyone – and moreover, does this imply that any ‘supplied’ data, as in not directly under a researchers/government departments control cannot really be verified?
So, for example, if the nice folk at the Maldives supply met data that shows a massive warming trend – does that simply get taken as read?
The picture I’m getting is that the ability for fudging/manipulating has just been increased multifold…….
worse still, it only takes a few ‘SNAFU’s’ for any and all raw data to suddenly be ‘lost’ forever within supplier NMS’s! wink, wink, nudge, nudge…..
ed said:
“@Peter Miller
I would argue that the use of Climate and Science in a sentence is definitionally not possible.”
How about: “I am going to work in the Climate field, so I won’t need to know any Science”?
Given the issues the Kiwis are having with their NMS, how can Jones et al rely on any of the data that is being provided by them?
It’s all very well coming up with a graph of the data you have been provided but you (Jones) have a duty to test that data to ensure its veracity.
Has Jones ever been asked to justify his trust in the data being provided?
My old science teachers had it dead right. If you don’t show your data and working, you get zero marks. We need to agree on a (preferably common) word,which will immediately convey that the “paper” being used to “support” some rubbish claim is null and void and not to be trusted because of any such data hiding / cheating / dishonesty in its production. We also need to be 100% inflexible in applying that word publicly whenever such a “paper” is referenced. No data, no credibility. No argument.
I would suggest that Anthony could have another page listing all such null papers, except that in climate science it would be a darn sight quicker to list the valid ones.
Steven Mosher so were is Jone’s Chinese UHI data ; last we heard ‘the dog had eaten it ‘
And that is before we get to Jones deleting information and asking others to delete information and of course its view that he does not like given information out becasue others may find something wrong with it , which ironically is what ‘critical review ‘ is supposed to do .
The idea Jones follows good practice is frankly a joke , his long sold out doing science to doing advocacy which he has found far more ‘rewarded’
Nick Stokes largely what is being asked is that these ‘professional scientists’ should meet a mini the standard that is expected of their own students, which is actual rather low given the great claims of urgency and certainty they make .
Is it really OK that Jones and co , self proclaimed leaders of their field , cannot work at a standard which we expect a undergraduate to be able to do to pass their degree course ?
Steven Mosher,
“His choice to use adjusted data is entirely rational.”
No, it isn’t rational. Homogénlisations have a huge unexplained bias (0.5 ° C for the twentieth century).
My apologies to the mods for the length of this. It comprises a long note to myself that will be the basis of a future article. As such it is episodic rather than a narrative, but the gist of it can be readily seen that there has been systematic adjustment of raw historic temperature data (perhaps for good reason) of virtually all the old records. The pdf’s referred to were sent to me privately so do not appear.
—– ——- —
Note to myself Feb 2012
Four 17th and 18th Century Metereological associations
Brief extract from these notes;
“The research papers detailing the systematic correction of early temperature records- which are invariably found to be ‘warm biased’ and subsequently adjusted downwards as they did not agree with the model expectations- are detailed in the contents page of the Camuffo/Jones book
‘ Improved understanding of past climatic variability from early daily European instrumental sources.’ Which was produced as a result of the IMPROVE project.
As well as those detailed here, many of the old records seem to be contained in the book ‘ History and Climate’ by Jones and Briffa. They would be worth investigating further to determine the reasons for the downwards adjustments.”
In my article ‘The Long Slow Thaw I made this note, intending to return to it at some point as it covered a subject of which I knew nothing;
http://judithcurry.com/2011/12/01/the-long-slow-thaw/
Note 4
It is interesting to note that in 1780 the Societas Meteorological Palatine of Mannheim, a standardised network of 40 observing stations, was set up in Germany and other European countries and a small number in the USA, all equipped with comparable instrumentation with standard instructions for use. This came to an end in 1795 with the siege and capture of Mannheim.’
Its subsequent investigation has allowed a number of pieces of the historic jigsaw to fall together for me, by putting this paragraph from one of my earlier articles into context,
‘Frederik became King of Prussia in 1701 and immediately set up a measuring station that became Berlin Tempelhof, one of our oldest records and this started a rash of similar stations that caused Samuel Horsley to comment in 1774: ‘The practice of keeping meteorological journals is of late years becoming very general’.
There was a dramatic upturn in weather station numbers and the period was characterised by attempts in the 17th and 18th Century to obtain consistently derived weather data as part of international collaborations, of which four were of some note and are covered in these brief notes. The establishment of the collaborations was largely driven by royal patronage, an increasing fascination with science, the birth of various scientific institutions such as the Royal Society and the invention and development of a number of related instruments including the barometer and thermometer.
The international collaborations were generally made in an attempt to ensure weather information was derived in a consistent manner with instructions given by the promoters to the various subscribers as to where instruments should be placed, what type should be used and frequency and time of reading.
The first brief international brief efforts were began in Italy 1653/1654 and was called Rete medicea with stations in Italy, France, Austria, Germany and Poland all reporting to agreed parameters, including placing the specified thermometer on an external north facing wall. It ended in 1667.
The second association was set up in 1723 by James Jurin secretary of the Royal Society according to strict parameters and with instruction to fill in all six columns of a journal recording everything from wind speed, temperature, precipitation , pressure, with a section for general weather conditions. It was suggested the thermometer be kept indoors in a north facing unheated room as had been the custom in England( by some) since 1660. There were some 20 stations in England, Europe- including Italy and Austria- North America and India. The journals were sent annually to the Royal Society for collation and still exist. The network ceased in 1735
The third was a French sponsored association which commenced 1776 and was called Societe Royale de Medecine of Paris. It was particularly well organised, had some 50 international and French participants and lasted until 1793 .
The fourth and best organised of the lot was The Societas Meteorologica Palatine of Mannheim which lasted from 1781 to 1792. There were some 39 sites including Greenland most of Europe, Russia, and North America . There were strict parameters about the placing of thermometers outside and at an elevation, and there seems to have been considerable use of screens which were first mentioned in regards to the first association in the mid 17th century.
D Camuffo wrote succinctly of these various associations in his chapter ‘History and correction of long temperature series.’ It is worth reading the section from page 9 to 12 although it is Italian centric and all the details perhaps do not seem to quite match up to original sources.
http://books.google.co.uk/books?id=eEOFQy86zzEC&pg=PA11&lpg=PA11&dq=james+jurin+an+invitation+to+form&source=bl&ots=3Lr0jYCSFq&sig=Bc0xTiWxrlXv_-DjUVdSYGQXhZU&hl=en&sa=X&ei=uwRFT-StFMex0AWnuuSJBA&ved=0CCgQ6AEwAA#v=onepage&q=james%20jurin%20an%20invitation%20to%20form&f=false
Detailed information on each of the Associations
Greater details can be obtained from the series of attached pdf’s which were compiled by the Met office some thirty years ago, which in turn were often derived from the original documents translated from Latin.
1) There are no further details available at present on the Early Italian association as this was identified only after research for the rest of this article.
2) The Royal Society . Attached is the pdf ‘James Jurin 1722’ which is a copy of the original letter from James Jurin of the Royal Society in 1722 asking observers in other countries to participate in an international effort to record weather data in a consistent fashion.
Title; ‘An invitation to form a meteorological system of observations by world-wide
Agreement’
Abstract; ‘But just suppose there were observers in suitable numbers, appropriately distributed over a large area of the earth’s surface; and, ultimately, someone to collate their various diaries, and make notes of the agreements and discrepancies; we should then before long have a Meteorological History covering a number of years, of a kind that could hardly be imagined or even dreamed about today. This would help to discover in what regions the winds originated and their subsequent pattern over the earth.’
The Royal Society lived up to its motto ‘nobody’s word is final’ as their aim was to either prove or disprove this theory, by the use of reliable observation. Edmund Halley had been a particular proponent of the theory of wind.
It can be usefully read in conjunction with the pdf ‘Haverfordwest weather register’ which provides a great deal of further background information, although it concerns itself mainly with the English readings All the weather data from Jurins international association still exists.
3) The French association is well described in the pdf ‘ eighteenth century source of meteorological data’.
‘The Society had over seventy correspondents in 1784 and this might well have been the maximum as a slight decrease occurred in 1786, and in 1793 the Society, like many other learned bodies of the Ancien Regime’, was suppressed by the revolutionary decree which abolished ”
Toutes les academies el socieles lilleraires patentees on dotees par la nation.’
The French are attempting to put these records on line-see http://meteo.academie-medecine.fr/
The Met Office are assisting them. I have attached an email from the archivist at the Met Office at the end of this document describing the current situation.
4) The Societas Palatine of Mannheim was the best organised and most consistent of all, lasting from 1781 to 1795 when it came to an end following the siege of Mannheim during the wide ranging Napoleonic wars.. At its peak it had either 39 stations (Camuffo) or 50 (Kington of CRU) observation stations. The history and the locations of stations is very well described in the fascinating pdf ‘societas meterologic palatina’.
The station locations are also described by the personal email from the Met Office quoting David Parker, who was responsible for setting up CET from 1772. (See foot of this paper under ‘emails’.)
I understand the Mannheim data still exists and indeed is much scrutinised and adjusted by such as Camuffo and Phil Jones separately, and also jointly through an EU funded collaboration between them in 2002 in the IMPROVE project when they examined 7 long European data sets, of which four were from the Mannheim association.
Other associations
There were others who drifted in and out of these organised efforts together with some who formed less formal associations such as within the Scandinavian countries. The records of Stockholm from 1750 and Uppsala 1720 are especially interesting. Uppsala records the great warming of the climate at its start date by noting that mulberry trees were grown in the botanic gardens. By the time Stockholm started reporting thirty years later the warm spell had subsided and the mulberry trees destroyed by the returning cold. Botanic gardens are a good source of background information of periods of warmth and cold due to introductions of new plants.
However, many stations never became part of any association.
Could the existing global records be extended?
A reconstruction of a ‘global’ temperature to 1750 could reasonably be made-although with a strong European bias and large error bars-however there are several stations within the groups, including CET- reckoned to be a good proxy for world temperatures. A reconstruction to 1720 could also be attempted to identify the widespread warming trend at the start of the century but this would need to be augmented by documentary records and could do nothing more than demonstrate the temperature ‘tendency but not the precision’ (Lamb)
General comment on the early historic records
The research papers detailing the systematic correction of early temperature records- which are invariably found to be ‘warm biased’ and subsequently adjusted downwards as they did not agree with the model expectations- are detailed in the contents page of the Camuffo/Jones book
‘ Improved understanding of past climatic variability from early daily European instrumental sources.’ Which was produced as a result of the IMPROVE project.
As well as those detailed here, many of the old records seem to be contained in the book ‘ History and Climate’ by Jones and Briffa. They would be worth investigating further to determine the reasons for the downwards adjustments.
Many of the sites and the instrumental readings of these and many other historic stations are recorded in my web site.
http://climatereason.com/LittleIceAgeThermometers/
There certainly seems scope for an independent review of these historic records as the examination of old records and their subsequent reconstruction seems to be very specialised, conducted by some 20 regulars of which Camuffo and Jones have been to the fore.
How accurate are historic records?
I wrote at some length of the evolution of weather recording and of the inherent problems of recording temperatures, in two articles which can both be accessed from this link;
http://wattsupwiththat.com/2011/05/23/little-ice-age-thermometers-%E2%80%93-history-and-reliability-2/
I have come to believe that there is some case to be made that the older historic readings are more accurate than those from the mid 19th to mid 20th century- especially when consistent standards and methodology have been laid down , as with associations, as instruments were of scientific quality, observers often well trained, and a great deal of prestige attached to scientific accuracy due to the patronage of royalty and the wealthy. In addition, many of the old records have been very thoroughly scrutinised and (over?) evaluated and amended. Having said that there are many inherent problems in temperature readings as described in part two of little ice age thermometers linked above, and bearing in mind their crucial importance to determining policy matters , it is doubtful if a good case can be made for using any instrumental records until modern times-1980’s- with the advent of digital stations and satellites, although each of these methods have their own inherent problems.
tonyb
Is your irony/hyperbole detector busted? Try reading old Dilbert strips for an hour a day for a month. That should help.
Steven Mosher says:
June 1, 2012 at 10:32 pm
,,,its still warmer now than when washington crossed the delware.
I hope so — it’s almost summertime, for gosh sakes…
Dale Hartz says:
June 1, 2012 at 5:44 pm
Why don’t we have a principle that any paper, study or major document paid for by public funds must include all supporting data, codes, and other backup?
_________________________________
We do. It is called the Scientific Method and that is the problem. Jones et al are not following the Scientific Method and the editors of the Scientific Journals should have tossed the papers back at them and told them SHOW YOUR WORK
If you do not want to show the work it is not published and it is a “Trade Secret” but it is NOT SCIENCE
You might know that the Australian Bureau of Meteorology was asked by New Zealand to review their temperature data a couple of years ago. So far, we have not been able to obtain more than a paragraph or so of comment.
Meantime, the Australian BoM has moved through at least 4 phases in the last 20-30 years.
1. A version largely compiled from previous publications, some weather stations and little metadata.
2. A homogenised version, started by Simon Torok as a PhD thesis.
3. A reduced set of about 100 stations in a High Quality data set, again with little metadata for sites in general.
4. A version of about 100 stations, not many overlapping with the HQ set, released a few months ago under the title Acorn-SAT.
For a preliminary look at the latter two of these versions do visit Ken Stewart’s works:
http://kenskingdom.wordpress.com/2012/03/13/near-enough-for-a-sheep-station/
http://kenskingdom.wordpress.com/2012/05/14/acorn-sat-a-preliminary-assessment/
You might then form an opinion, not only about Phil leaving it to the NMSs, but also about the need to recalibrate proxies as applicable in recent papers like Gergis et al 2012 that Steve Mc noted.
I look forward to a Daly-type study that simply selects a limited number of weather stations worldwide, where the metadata of each can reasonably be checked. I still refuse to believe that we need thousands of weather stations for a global picture. Ilarionov at Heartland 2010 showed that Russian records could be captured with good accuracy by just four stations.
Without individual clean bills of health for weather stations used, we will never solve the UHI. But there are many clues on the way, that suggest the homogenized records have NOT got UHI adequately sorted. Have a look here particularly at Ilarionov’s superb presentation
Owen in GA says:
June 1, 2012 at 2:09 pm
I don’t know. We all dump on climate science, but I have heard some disgruntled grumbling from other fields that journals are not doing a good job of making sure the work covered in the articles is reproducible. I think the problems may be systemic in the sciences right now and that has me worried.
_____________________________
I think you’re right to be worried, Owen.
I think it would be helpful if reviewers pre-publication comments and questions were also made available for general consumption.
I don’t think this would compromise anonymity more than it already is. [Such as it is, where the identities are often knowable by authors]. It would be an aid to readers, and also allow them to make a judgement about the quality of the Journal and the reviewers. It might also prompt reviewers to greater diligence.
If public sources of data (such as NOAA) do not include the raw data for public scrutiny, we are just reshuffling second had merchandise and I think are targeting the wrong folks here. Those that have taken the raw data and homogenized it, sanitized it, and then slow cooked it are just doing what researchers do with raw data. Those whom we have payed to collect the raw data from surface stations and to keep it safe and in original condition are the ones who should be investigated. This is a plain and simple fact.
The next president of the United States should force these government funded agencies such as NOAA to account for where the untouched raw data is (or has gone to) and then be held accountable on a yearly basis for a data audit to insure we will not be losing raw data from here on out. This is a reasonable request from every private tax paying citizen. We pay you to keep that raw data in pristine condition. And it appears we must now force you, the government, to show us whether or not you have done that. It is our due diligence.
Has science and the scientific method really changed since I was a physics undergraduate? I remember the huge amount of work that went into producing your results from the lab work. Preparing the data for presentation took far longer than the actual experiment. (1) The method had to be described in detail, (2) it was mandatory to quote the errors in all measurements taken and (3) the method of calculating the errors in the final results also had to be shown.
Miss out any of (1)(2) or (3) and your work would be returned for a re-show, no exceptions. That rigour is what instilled in me respect for the scientific method. What the ‘Team’ is doing to science makes me sick because, in the public’s mind, it tarnishes all science. The sooner these frauds are held to account the better.
Anthony, I have a suggestion.
We need two on-line publications: the Book of Reproducible Results and the Book of Unreproducible Results.
It is hard to keep track which papers are published with no supporting data or methods, or with partial data and incomplete methods (whether by design or incompetence). The point is that there are too many important-to-the-field papers being published reporting results are not reproducible by other researchers.
When one gets into a discussion there are frequent references to papers that are definitely in the Journals, but whose result are not reproducible. This is an affront to science but the authors do no seem to be bothered by this, for whatever reasons they have.
On your links to the right of these blogs there is a list of sites binned according to temperament and content. Would it be possible to create there a link that goes to a list, at least two lists actually, of important-to-the-discussion papers divided into two major groups: those which contain archived data and methods and which can be used to reproduce results, and those which do not. A third list might be papers for which the data is promised or is available on request.
Before allowing debaters to get away with citing-in-argument the result of works that are not reproducible, we can have a quick look-up of the paper by simple CTRL+F for part of the name. If it appears in the list of ‘papers with unreproducible results’ then we can legitimately respond with a comment that the works may be important but that the results are not reproducible and are thus suspect until independently verified.
The list might be in the form of a simple chat so anyone posting on that chat can simply post the link to the paper and its official name. That chat can be searched by anyone clicking on the list.
The motivation here is to take a long term position on this issue: the gradual erosion of scientific norms for the acceptance and publication of papers which show how the researcher got the results must be addressed by the academic community at large. With the entry into the field of Climate by many well-informed non-specialists, we should establish the tools needed to assist their activities.
Given the very wide influence of WUWT, it may have inflience in the long term if a large number of people start insisting in debate that only works which meet the norms of formal science be considered.
Yet researchers are glad to use the Surface Stations dataset, ready or not. I guess some people are more equal than others.
Follow up correction: it would be better to say:
“papers with unreproduced results” not “unreproducible”. There is a big difference, the point being to encourage people to include their data and methods when the publish.
It is quite possible that pressure on major journals like Science and Nature will see them stop the silliness of promoting AGW with reams of papers filled with un-reproduced and un-reproducible results. The effect of the present system is to stifle debate because if results are not reproducible, one can hardly engage in scientific debate about them. That was I believe the whole point of having a data archiving policy in the first place (now in tatters).
Dr. Phil Jones: “This is how I will avoid responding to requests to see my data from now on”
These are not the climate frauds you are seeking. Nothing to see here, move along, move along.
It is interesting that the Iceland Met certainly make homogeneity adjustments to their raw data, and yet GHCN still insist on making further adjustments. Presumably this applies to many other countries.
(I actually have access to the original data here)
http://notalotofpeopleknowthat.wordpress.com/2012/03/14/reykjavik-temperature-adjustments/
There has been enough trouble lately in various areas of science with “scientists” who fabricated results just to fit their pet theories and done enough harm with it.
Not showing the raw data and the methodology in detail to make the work reproducible should be reason enough to remove grants to those scientists. Yes, they may do private work and research but not on public money.
And indeed this should be the criteria to admit or not the papers for IPCC submission.
@Larry Plume P.
I agree completely. This is a basic rule of engagement – you take public money, you show what you did with it. To hide behind ‘we are still working on a new publication’ does not entitle one to the private use of public goods (the data and methods developed with public funds). It is patently obvious that some of this ‘science’ is defective and the authors are either misguided, incompetent or crooked. Either way, they are ruling themselves out of receiving further grants. If you look at the terms of, for example, Department of Energy funding documents, it is quite clear that one is not entitled to hide the methods and data while still producing results that will be accepted.
There is a Data Quality Act in the USA that can be applied to these cases. The methods used to make scientific determinations have to themslves be valid and provably so. “Climate science” is not exempted from complying with the requirements of the Data Quality Act with respect to methods. If you produce works, you have to use methods that are valid. If you do not show your methods then it is obvious your methods cannot be checked for validity. If you method is invalid, or cannot be validated, you can’t be compliant with the (very reasonable) mandated need for due diligence.
It surprises me that so many US gov’t Departments have not demanded authors receiving public grants comply with the Data Quality Act. I guess it is optional for Journals (in spite of published editorial policies that turned out to be ‘flexible’) but it is not, in the case of the expenditure of public funds.