From Kate at Small Dead Animals: No U-turns allowed
Dear Tom,
I find it hard to believe that the British Antarctic Survey would permit the deletion of relevant files for two recent publications or that there aren’t any backups for the deleted data on institutional servers. Would you mind inquiring for me? In the mean time, would you please send me the PP format files that you refer to here for the monthly sea ice data for the 20th century models discussed in your GRL article and the 21st century models referred to in your JGR article.
Regards, Steve McIntyre
Then in July… “Unprecedented” Data Purge At CRU
On Monday, July 27, 2009, as reported in a prior thread, CRU deleted three files pertaining to station data from their public directory ftp.cru.uea.ac.uk/. The next day, on July 28, Phil Jones deleted data from his public file – see screenshot with timestemp in post here, leaving online a variety of files from the 1990s as shown in the following screenshot taken on July 28, 2009.
The Telegraph, today – Climategate: University of East Anglia U-turn in climate change row ….. Leading British scientists at the University of East Anglia, who were accused of manipulating climate change data – dubbed Climategate – have agreed to publish their figures in full….
Now, here comes the other shoe! Hide the Decline!
SCIENTISTS at the University of East Anglia (UEA) have admitted throwing away much of the raw temperature data on which their predictions of global warming are based.
It means that other academics are not able to check basic calculations said to show a long-term rise in temperature over the past 150 years.
The UEA’s Climatic Research Unit (CRU) was forced to reveal the loss following requests for the data under Freedom of Information legislation.
The data were gathered from weather stations around the world and then adjusted to take account of variables in the way they were collected. The revised figures were kept, but the originals — stored on paper and magnetic tape — were dumped to save space when the CRU moved to a new building.”
[…]
In a statement on its website, the CRU said: “We do not hold the original raw data but only the value-added (quality controlled and homogenised) data.”
The CRU is the world’s leading centre for reconstructing past climate and temperatures. Climate change sceptics have long been keen to examine exactly how its data were compiled. That is now impossible.
It is still nothing in the Australian media, except for a short bit on Insiders at 9.30 on Sunday morning. Andrew Bolt of course. All eyes on the opposition. No one here is interested.
So I give way to despair. The ETS legislation will pass, we will reduce our 1.6% of greenhouse gases by 5 %. Our electricity charges will rise by 30%, transport, food, clothing, everything will increase in price, the government will make bucketloads in tax.
So it’s up to our friends in the northern hemisphere to pursue this. It won’t be the first time in history that Australia has been saved by the Yanks.
It restores my faith in humanity to check on progress here at WUWT. Keep up the good fight!
This entire data business is getting to resemble Monty Python’s work more each hour.
Do we know if any repository, anywhere, has an untainted climate data-set.
Has there been too much cross-pollination to assure data integrity?
And snow it goes!
Given the sorry state of the temperature measuring stations, as demonstrated clearly by Anthony, of what use would even the original temperature “measurements” be? GIGO
Does a temperature measurement made at a temperature measuring station before an airport was located at the same site have any value as to comparison with a temperature measurement made at the same station 80 or so years later with the airport operating?
on data are / data is:
The word datum (neuter singular), is Latin for “a given thing” and the plural, data, means “given things.”
Also, data (feminine singular), is Latin for “a given female [or some other feminine thing]”; the plural, datae, means “given females [or other feminine things]”.
If you choose to use a Latin word, you have to get the plural correct. To say “the data is,” is to say “the given woman is,” or suchlike.
Malo malo malo pravo malo malo.
[I’d rather be in adversity in an apple tree than a wicked man with a crooked pole.]
Steve (18:05:43) :
There are the photocopies (in pdf form) from
http://www7.ncdc.noaa.gov/IPS/coop/coop.html for US stations and
The American Meterological Society’s Monthly Weather Review
http://ams.allenpress.com/perlserv/?request=get-archive&issn=1520-0493
Somthing to find: The US Army’s Signal Corps Observers Records
The reason I maintain that Latin plurals (and all irregular plurals in English, for that matter) should be used properly is that precision is important in language.
Is it so hard to learn that the plural of criterion is criteria or criterions, that the plural of apparatus is apparatus or apparatuses (but not apparati), and that the plural of virus is either vira or viruses?
A major cause of the pervasive lack of ability to understand rational arguments, and the lack of understanding of scientific principles, is that (throughout the English-speaking world, seemingly) schoolchildren are not taught to use English with precision.
Those who condone the use of ‘continually’ to mean the same thing as ‘continuously’, ‘simplistic’ to mean ‘simple’, ‘alternate’ to mean ‘alternative’, ‘fulsome’ to mean ‘full’, ‘panic’ or even ‘chaos’ to mean ‘some amount of disorder’, and so on, are robbing English, the language with the richest and broadest vocabulary in history, of meaning, depth, subtlety and nuance.
So, what has this to do with the issue at hand? We notice, throughout the debate on AGW, so many people who actually believe that they can reason logically (and who can even, at times, refer correctly to logical fallacies by Latin name) but cannot actually formulate syllogisms correctly. I surely need not multiply examples, but the most common and (to me) vexing argument—I know that the days are getting hotter, and droughts and bushfires are worsening, therefore catastrophic global warming is true and dooms us all—would not be so uncommonly common if people were taught English properly. (Teaching English properly, for me, of course, means featuring grammar and logic as essential components.)
[with your leave, dear moderators]
I don’t know whether to laugh or cry.
On the one hand – the raw data is gone, and will need to be reassembled (presumably). What sort of scientists are these? Amateurs – I’m so angry.
On the other hand, at least I can claim to all and sundry that there really IS NO evidence of global warming. At least for the time being, until the data is reassembled.
Is this what’s called irony?
http://www.freerepublic.com/focus/f-news/2396502/posts
Clive,
Re: “Inspired by jbrodhead ☺ ☺”
http://photoshare.shaw.ca/image/2/d/8/63987/wxstationburndata-0.jpg
Thanks Clive! That is great!… now I can sleep well… LOL!
Reply: Yay! One of my photo’s. ~ CTM aka jeez
Deadman wrote:
“If you choose to use a Latin word, you have to get the plural correct.”
That’s not so. Fowler states, “Latin plurals sometimes become singular English words (e.g., agenda, stamina) …” As long as it’s OK to employ those words as singulars, it’s OK to do the same for “data.”
Not only is it acceptable to use “data” as a collective singular, using data as a plural word is incorrect because it throws the speaker (including those who use “data are”) into inconsistency with his habitual method of speaking, as Phillip W. pointed out. He wrote: “‘Data’ is naturally and consistently used as a mass noun in conversation: the question is asked how much data an instrument produces, not how many; it is asked how data is archived, not how they are archived; there is talk of less data rather than fewer; and talk of data having units, saying they have a megabyte of data, …” For another example of this usage, look at the post just above this one, where the phrase “the raw data is gone” is used.
Because of this inconsistency with long-established and near-universal usage, and because, as Fowler shows, there is no real rule forbidding “data is,” “data are” will never be accepted–it will always sound odd or even affected. It’s counterproductive to make an issue about it, because the people criticized will not change their habit, but be determined to pay no attention to such criticism in the future. This backlash is what happened 100 years ago after schoolmarm grammarians makd a fetish of not splitting an infinitive, distinguishing between shall and will, etc. They lost the war, by going a bridge too far.
“Those who condone the use of ‘continually’ to mean the same thing as ‘continuously’, ‘simplistic’ to mean ‘simple’, ‘alternate’ to mean ‘alternative’, ‘fulsome’ to mean ‘full’, ‘panic’ or even ‘chaos’ to mean ‘some amount of disorder’, and so on, are robbing English, the language with the richest and broadest vocabulary in history, of meaning, depth, subtlety and nuance.”
I’m with you on those. I made a stink about the misuse of “alternate” in one of WUWT’s headlines about a week ago. And I’ve posted a couple of articles on an instablog on the Seeking Alpha site criticizing dozens of mis-usages by authors on the site. Here are links to them:
http://seekingalpha.com/instablog/259051-roger-knights/34743-common-usage-errors-on-the-seeking-alpha-site
http://seekingalpha.com/instablog/259051-roger-knights/35454-common-usage-errors-on-the-seeking-alpha-site-2
“A major cause of the pervasive lack of ability to understand rational arguments, and the lack of understanding of scientific principles, is that (throughout the English-speaking world, seemingly) schoolchildren are not taught to use English with precision.”
I think that precise thinkers are often “sticklers” in matters of language. But the linkage is fairly loose, and thus not likely causative, but rather correlative. For instance, people who are good at math or other logic-related skills often are poor at writing. Programmers are notoriously awkward in their language skills. Trying to teach them to write with precision would probably just make most of them self-conscious and silent, not fluent, and not any better at their math or programming.
PS: FWIW, dictionaries accept “data is.”
Patrick G (01:01:03) :
No evidence of Global Cooling as well.
Which must delight James Hansen, whose position is like one of those toys that you cannot knock over, all the weight is at the bottom, and it rights itself.
I’ m totally shocked by the University of East Anglia’s handling of this affair. I was once peripherally involved in a case of scientific fraud in which a high profile worker had falsified data. The response by the university was, quite rightly, draconian and we all became under suspicion. Our computers were removed, our records impounded and we were really grilled. The fraud was exposed and, fortunately co-workers were cleared of malpractice.
The “investigation” at UEA is being handled by the “Vice Chancellor for Research”, who is a member of the CRU.
The Vice Chancellor should have immediately suspended the involved parties and mounted an investigation by impartial investigators, including those appointed by grant-giving bodies. As it stands the whole issue has been fudged. Destroying primary data – unbelievable Nobody will believe a word that comes out of this investigation, which, I predict, will say that there were some irregularities, but the message about temperature is basically true.
Another thing that I think is interesting about climategate is that a lot of people seem genuinely disappointed that the Earth isn’t warming as they believed. They should be rejoicing!
>>Dr A Burns (12:40:43) :
>>> Hugo M (04:34:41) :
>>>Seems as if they are busy adjusting surface temperatures downwards.
>>Thanks Hugo. It will be very interesting to see what they do and what explanation is given. One more program “adjustment” is no big deal to these scammers.
I had to look into a dictionary for the exact meaning of “scammer”. Well, that’s a bridge too far. My impression is that many of these people are subjected to group presssure and Festinger’s law of Effort Justification. You also wondered why an obviously intelligent man like Harry hadn’t escalated the hopeless situation he faced? Perhaps he even tried? And what if there wasn’t a nearby stream he could divert in order to clean up this modern equivalent of Augean stables? Leaving open a door for an honourable retreat is always a good idea.
As noted in the email quoted above: HadCRUt is just a minor rehash of the GHCN data from NCDC. This is the same source data used by GIStemp. The reason CRUt, GIStemp and NCDC agree is that they all use the same basic input data and it has been artificially “cooked” by deleting cold thermometers. Whack There and don’t let these guys point in a “Circle of 3” each claiming validation from matching the guy to their left…
http://chiefio.wordpress.com/2009/11/03/ghcn-the-global-analysis/
David Jay (11:31:08) :
RE: vjones (06:34:18) :
“At least GISS start with real data.”
Actually, they start with USHCN adjusted temps and then they re-adjust them to suit
Um, there is a bit more than that. Up until 2007 GIStemp was a merger of GHCN (from NCDC) with USHCN for the USA. Then they stopped using USHCN (when it went to Version 2). Just a week or so ago they put back in the USHCN.v2 series (oddly, just after I published how to do it and showed that leaving it out induced a bit of bias… but I’m sure it was a coincidence 😉
So basically, for all the world OTHER THAN the USA, GIStemp is just a slightly munged GHCN / NCDC much as is HadCRUT. One is really admiring the nuances of the same stew… For THE USA ONLY, GIStemp merges the (now) USHCN.v2 data with the GHCN data and makes a hybrid.
Yes, that is as icky as it sounds and it is what the code does.
KeithGuy (03:01:32) :
Is there such a word as de-homoginized?
I don’t know if it is a word, but the process is what you do to make cheese.
First you curdle the milk to precipiate the proteins and fats (de-homgoinzing the solids from the liquids) then you supply “value added” by letting it sit in a dark cave for a while keeping outsiders away form it, only letting the cheese master approach the “stuff”. After enough fermentation, and “value added”, it is ready for release…
SandyInDerby (08:21:06) :
Craig (06:27:07) :
I have used “is the data” all my life, but there again I was brought up on Scottish English!
You can use the “programmer jargon” cover if you like. As an official Silicon Valley Geek, I can say with authority that use of “the data is” is common and normal. Use of “the data are” will get you looked at as one of those sorts… So if you want to fit in as a Geek and not as an academic, you use “data” as a non-gender English (singular or plural) mass-noun. And it’s OK.
It is, after all, jargon. (And it is NOT latin any more… unless, of course, you wish to enforce full latin syntax on all the OTHER loan words in English…)
I only just noticed the UEA admission of ‘loss’ of the original data was in August 2009, according to Roger Pielke’s blog http://rogerpielkejr.blogspot.com/2009/08/we-lost-original-data.html. Yet the UEA website today states that ‘over 95% of the raw station data has been accessible through the Global Historical Climatology Network for several years’ http://www.uea.ac.uk/mac/comm/media/press/2009/nov/homepagenews/CRUupdate
So was the raw data lost? Or is the current UAE website wrong, and knowingly wrong?
While the mainstream media is bending into pretzels to keep the scandal under the rug, Climategate is already the biggest scientific scandal in history because of the global policy implications.
You don’t need to be a scientist to know that the religion called Climategate is taking on water and is about to sink. Just look at all the faithful rushing to the liferafts. Attack has now turned to defence. Next they will retreat then they will be forced to surrender. Alot of decorated careers will only be remembered for their comical rants.
To sign a petition use this link: http://www.ipetitions.com/petition/NOC_NOW/
It appears that we have reached a stage in climate science where we need to create a new ‘independent’ and ‘transparent’ research institute, publicly funded, operating totally in the public domain, hosted on the internet, visible to all, with all data and methods available to all.
This ‘new climate research institute’ could be just a secure data store, ‘write once, read many’ where independent climate researchers from around the world can use raw original data, processed data, verify the methods of others, and publish their own data and methods.
In essence a climate wiki, without a delete facility.
To start the ball rolling, we need a large server and the first tranche of raw data.
Steve Richards (07:48:13) : To start the ball rolling, we need a large server and the first tranche of raw data.
Um, not that large a server… It will fit easily on a laptop (or a CD) and since you are bandwidth limited on the WAN side, any LAN port from the server will do…
I have GHCN data (several copies) feeding into GIStemp running on a PC with a 400 Mhz AMD chip. You need under 100 MB of storage per copy of the world temperature history, so the 10 GB disk on this old box would hold about 100 copies of the data… Plenty of room for variations from “raw” to “cooked” and steps in between…
And I’d start with the oldest, least “fudged” copy of GHCN input you can find. Then, one country at a time, find the original “really raw” stuff to fill in the the “before” copy…
So ANYONE with an internet spigot and a place to park a PC can get this ball rolling. I’ll provide consulting on GHCN, USHCN, USHCN.v2 and GIStemp (having gotten rather familiar with them all.) “Raw” data will require a distributed effort with folks working each national BOM for a copy of the original data or working the web sites for those with the data already published.
VerseVersari (19:07:21) : So was the raw data lost? Or is the current UAE website wrong, and knowingly wrong?
The raw data was lost at UEA, but a copy had been sent to GHCN, where it supposedly still resides.
This, to me, was a great and valuable statement by UEA. It says that they are 95% identical with GHCN. And GHCN is the NCDC product, so it says that CRUt and NCDC argree not because they are independent validations, but because they use the same cooked input data. (GHCN has recent thermometer deletions that bias the record.)
Finally, GHCN is the vast bulk of all the input to GIStemp. So it too agrees because it is substantially “the same thing”.
This means that when these three point to there left in a circle saying “Sure, you found problems, but I’m just like HIM, so I must be right anyway” they are being deliberately misleading. They ARE NOT INDEPENDENT, they are substantially THE SAME THING.
And this point needs to be made every time someone claims one of these turkeys is “right” because it matches the other one(s).
So yes, UEA lost the data (though Hadley may still have a copy) and yet, yes, the data is 95% identical with GHCN. And it is GHCN that needs the most through “cleaning up”. They have systematically stoped recording data from cold thermometers (at altitude, at latitude, and even with respect to large bodies of water).
The Stick is limp. Long live the limp Stick!