From Kate at Small Dead Animals: No U-turns allowed
Dear Tom,
I find it hard to believe that the British Antarctic Survey would permit the deletion of relevant files for two recent publications or that there aren’t any backups for the deleted data on institutional servers. Would you mind inquiring for me? In the mean time, would you please send me the PP format files that you refer to here for the monthly sea ice data for the 20th century models discussed in your GRL article and the 21st century models referred to in your JGR article.
Regards, Steve McIntyre
Then in July… “Unprecedented” Data Purge At CRU
On Monday, July 27, 2009, as reported in a prior thread, CRU deleted three files pertaining to station data from their public directory ftp.cru.uea.ac.uk/. The next day, on July 28, Phil Jones deleted data from his public file – see screenshot with timestemp in post here, leaving online a variety of files from the 1990s as shown in the following screenshot taken on July 28, 2009.
The Telegraph, today – Climategate: University of East Anglia U-turn in climate change row ….. Leading British scientists at the University of East Anglia, who were accused of manipulating climate change data – dubbed Climategate – have agreed to publish their figures in full….
Now, here comes the other shoe! Hide the Decline!
SCIENTISTS at the University of East Anglia (UEA) have admitted throwing away much of the raw temperature data on which their predictions of global warming are based.
It means that other academics are not able to check basic calculations said to show a long-term rise in temperature over the past 150 years.
The UEA’s Climatic Research Unit (CRU) was forced to reveal the loss following requests for the data under Freedom of Information legislation.
The data were gathered from weather stations around the world and then adjusted to take account of variables in the way they were collected. The revised figures were kept, but the originals — stored on paper and magnetic tape — were dumped to save space when the CRU moved to a new building.”
[…]
In a statement on its website, the CRU said: “We do not hold the original raw data but only the value-added (quality controlled and homogenised) data.”
The CRU is the world’s leading centre for reconstructing past climate and temperatures. Climate change sceptics have long been keen to examine exactly how its data were compiled. That is now impossible.
Not to worry, Gavin has now collected and released all the data and code skeptics should want in a directory on RC. While he makes the point that all the data or code any skeptic should want has always been publicly available, he has now gone to the trouble of compiling this directory. If something you want isn’t there- it will be because you don’t need or should not want it. This is a huge change.
http://www.timesonline.co.uk/tol/news/environment/article6936328.ece
They say they dumped much of the raw data when they moved due to not having enough data storage capacity back then.
Okay, remember HARRY_READ_ME.txt
http://www.devilskitchen.me.uk/2009/11/data-horribilis-harryreadmetxt-file.html
From the HARRY_READ_ME.txt file:
http://www.anenglishmanscastle.com/HARRY_READ_ME.txt
The first line:
“READ ME for Harry’s work on the CRU TS2.1/3.0 datasets, 2006-2009!”
Then later in the file:
“You can’t imagine what this has cost me – to actually allow the operator to assign false
WMO codes!! But what else is there in such situations? Especially when dealing with a ‘Master’
database of dubious provenance (which, er, they all are and always will be).”
and then from near the end of the file:
“OH FUCK THIS. It’s Sunday evening, I’ve worked all weekend, and just when I thought it was done I’m
hitting yet another problem that’s based on the hopeless state of our databases. There is no uniform
data integrity, it’s just a catalogue of issues that continues to grow as they’re found.”
But CRU’s press release says that v3 has fixed all the issues. Okaaay, then.
If you can’t repeat it, it’s not science.
This reminds me of the last scene in the Lord of the Flies. The adults have just turned up late to the party.
The post is misleading because no raw data was deleted by the BAS. It was simply converted to a format before use in the papers and deleting those files created more work for SteveMc. It may be annoying but it does not go against the broad principal of data retention.
I think it is very important to distinguish between this case and the CRU where it appears the original data was deleted making it impossible to anyone to check the HadCRUT series.
The science of prestidigitation – now you see it, now you don’t.
Roll up, roll up, ladies and gentlement, find the data.
Is it under this cup ……… no
Is it under the other cup …….. no
Lets start again. Place your bets.
(shuffle, shuffle)
Did UEA incinerate the paper data?
…In an old oil drum in the parkinglot??
….how many of Al Gore’s polar bears died because of that???
……I hope old Al is choking down his fraudulant $100M words…
Watts Up With That… Thank you for making data and stories available to average, non-scientific persons.
Any recent news of how Alan Carlin has faired at the EPA, since the SS enforced the Obama Climate Agenda?
I’d posted this previously but I’m not sure if I had a response. Does anyone know why the past 8 months of hadcrut3, hadsst2 and crutem3 data have been deleted ?
http://www.cru.uea.ac.uk/cru/data/temperature/hadcrut3gl.txt
Phil Jones WILL lose his job over all this, and the pro-AGW lot will want to do what is now the buzz thing to do – wait for it… ‘Time to move on’. That’s what is said here in the UK now whenever a scandal surfaces. Those involved say, ‘Well, yes we made some mistakes and people have lost their jobs. Now let’s move on’. It’s meant to say that enough has been done to rectify the situation. Of course, in reality no such thing has happened and the problem surfaces again in the future! We’ve had this with social services getting involved in child cruelty cases. We’ve even had judges and social care managers saying, ‘This must never be allowed to happen again’. Of course, it does happen again because not enough is done in the first place to stop it happening again!
Barry Foster (01:42:17) :
As a fellow Brit i can only agree with you…..what shit state they’ve gotten us into.
MSM still head in the sand.
Complaints sent to 3 broadcasters and two letters to newspapers regarding A) Bias and B) Non reporting of an unfolding trainwreck.
Do likewise id suggest.
“value-added”
The spin comes with a good laugh.
As Bill (above) said, this is a scandal of epic proportions.
Surely it is time to have these fellows arrested.
Climategate Quiz on Youtube:
The news site that made the original quiz liked Jeffrey LaPorte’s video enough to post it on their website. Check it out here:
http://us.asiancorrespondent.com/gavin-atkins-shadowlands/quiz-now-on-youtube.htm
Jeffrey LaPorte’s website: http://www.JeffreyLaPorte.com
Twitter: http://tinyurl.com/lrxrnk
Nothing about it on radio4 this morning (uk). Just Milliband saying great things about Copenhagen.I find it hard to believe but it wouldn’t surprise me if ‘they’ did do a legally binding deal.There’s so much cash and Government tied up in this thing that it’s still going to take a fight like the Russians in Stalingrad to kill the monster. I like the NIWA scandal best – cranking down 6/7 stations temperatures right from the start of the dataset in each case by AT LEAST 0.5C. That’s a real clear cut frad IMO. The fact that NIWA won’t address it and keep coming back to subsequent station relocations is a sign they know the game’s up,I think.
The combination of “science” and politics is a deadly one, as we have seen in the last century.
“Don’t you see that the whole aim of Newspeak is to narrow the range of thought?… Has it ever occurred to your, Winston, that by the year 2050, at the very latest, not a single human being will be alive who could understand such a conversation as we are having now?…The whole climate of thought will be different. In fact, there will be no thought, as we understand it now. Orthodoxy means not thinking—not needing to think. Orthodoxy is unconsciousness.”….
“In Oceania at the present day, Science, in the old sense, has almost ceased to exist. In Newspeak there is no word for ‘Science’. The empirical method of thought, on which all the scientific achievements of the past were founded, is opposed to the most fundamental principles of Ingsoc. And even technological progress only happens when its products can in some way be used for the diminution of human liberty.”….
“And if all others accepted the lie which the Party imposed—if all records told the same tale—then the lie passed into history and became truth. ‘Who controls the past’ ran the Party slogan, ‘controls the future: who controls the present controls the past.'”
Great damage appears to have been done to both the current records and the GHCN. Where are all the Observers Reports that are handwritten?
How much of them were destroyed?
The GHCN does not jive with the stations mentions in HARRY_READ_ME as how far back in time the stations are supposed to go.
I have found some early station data in the US from the AMS records, which were taken from early observers, Signal Corps, etc. They are in monthly mean or monthy max/monthly min format.
Sacramento, CA from AMS jounal of 01-01-1884
YEAR(S) JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC ANN
1853 —- —- 59.8 61.0 68.0 77.0 75.0 71.0 76.0 78.0 53.0 —- —-
1854 43.0 52.0 53.6 60.0 62.0 67.0 80.6 69.5 65.0 60.0 55.0 48.0 59.5
1855 43.7 52.5 54.8 51.8 60.2 71.1 72.5 73.0 68.0 63.0 50.6 47.9 59.5
1856 48.0 52.6 57.0 58.8 63.9 71.1 75.1 69.6 70.9 58.0 52.2 46.9 60.1
1857 48.5 50.2 56.4 63.3 65.5 71.9 71.4 71.3 67.9 61.5 53.2 43.9 60.7
1858 45.0 52.2 53.7 59.8 65.2 69.4 70.8 70.6 68.9 59.5 54.2 47.7 59.5
1859 44.9 50.5 51.5 57.1 63.0 74.8 69.1 67.2 65.9 63.3 54.0 44.5 58.7
1860 46.2 49.8 53.3 57.8 58.5 65.6 73.2 73.5 67.6 59.8 53.5 43.5 59.0
1861 47.1 52.2 55.0 60.6 63.7 66.2 73.6 69.7 67.8 59.9 53.6 49.3 60.1
1862 46.4 47.5 53.6 58.0 61.2 69.3 73.2 75.0 70.4 67.6 53.1 50.9 60.2
1863 46.9 48.0 57.6 59.5 67.1 69.1 75.6 70.7 69.0 62.8 52.7 46.4 60.3
1864 49.2 53.6 56.1 62.1 68.5 71.1 74.8 74.7 69.8 64.5 53.5 46.5 62.8
1865 47.4 49.0 53.6 59.3 70.2 73.5 74.0 71.7 68.8 63.1 50.9 50.2 61.0
1866 46.5 53.5 54.2 61.9 63.1 72.2 76.2 76.0 72.2 65.2 53.8 44.1 62.1
1867 48.2 47.8 50.7 59.7 64.4 70.3 73.2 71.7 68.8 62.7 54.8 50.2 59.9
1868 47.0 50.5 55.0 60.1 64.2 69.5 73.8 71.2 68.3 62.0 53.0 46.8 60.1
1869 47.6 49.9 53.6 59.0 64.2 70.8 74.4 71.3 69.9 63.1 54.0 47.0 61.4
1870 48.6 51.1 53.0 57.0 61.0 69.3 71.8 72.6 68.0 63.6 53.4 46.4 59.6
1871 48.3 49.4 56.0 59.2 61.5 71.1 70.2 72.0 67.4 62.2 50.2 45.5 59.6
1872 48.5 53.3 56.8 57.6 67.0 69.2 71.4 73.1 68.8 58.9 51.2 48.7 60.4
1873 52.7 48.2 56.8 60.0 67.9 71.7 73.2 66.3 69.9 61.4 57.5 49.0 60.7
1874 45.7 49.3 52.9 59.5 64.7 70.2 72.8 70.9 70.7 61.7 63.5 47.7 59.8
1875 46.9 52.7 53.7 63.0 68.1 70.6 73.3 72.5 55.7 69.9 56.7 45.0 62.5
1876 48.8 50.2 54.6 59.5 65.7 76.9 74.0 72.8 70.1 63.5 53.3 48.0 61.7
1877 49.1 55.0 59.0 60.2 64.5 72.5 75.0 72.9 72.5 62.9 54.7 45.5 61.2
1878 49.7 51.3 56.7 59.4 65.5 71.8 73.4 73.4 69.0 62.9 55.5 48.6 61.3
1879 45.5 55.0 57.4 60.3 60.2 72.1 71.8 74.7 70.5 61.5 50.9 47.2 60.3
1880 43.5 46.0 48.8 54.6 61.6 66.6 70.9 69.7 68.0 62.1 49.7 44.0 57.5
1881 49.2 53.5 55.5 60.8 64.5 66.1 71.1 68.2 67.8 56.8 50.8 50.3 59.2
1882 45.1 46.3 53.0 55.8 64.0 68.1 73.4 71.9 68.4 58.4 49.5 46.2 59.4
So have they actually deleted the original data or simply put the original data in electronic form and then deleted the paper copies?
Well I’ve just canned google. Bling Bling for me!
No more Adwords $ for you boys.
So the CRU have lost the original temperature data and replaced it with their homoginized, quality controlled, constructed data. Oh dear!
But all is not lost because they can still troll through some DOE documents and reconstruct the original data. Hoorah!
These guys are so clever! I would never have worked out such a brilliant way of getting back to where I started.
Is there such a word as de-homoginized?
Correction: The above was the January 1883 Montly Weather Review from AMS.
Michael (00:50:41) :
My question is how do you “homogenise” temperature data?
Homogenization of data in meteorology and climatology is nothing nefarious per se.
It is simply a procedure to join data from various scribal versions or grafting a modern station record to the record of a nearby discontinued old station or correcting a long record for a UHI with the help of the neighboring rural stations.
Of course, it is best done with the actual knowledge of the issues with the homogenized dataset and the stations involved, i.e., when it’s done by local scientists from the original records and not by an algorithm just averaging points on grids.
Perhaps, a punishment for these crimatologists, while doing their time in prison, would be to manually reconstruct and recover the original data 🙂