Comprehensive network analysis shows Climategate likely to be a leak

This lends cred to WUWT’s previous analysis done by our own Charles the moderator: The CRUtape Letters™, an Alternative Explanation,

Climate-Gate: Leaked

by Lance Levsen, Network Analyst – courtesy of Small Dead Animals

https://i0.wp.com/www.swfwmd.state.fl.us/conservation/indoors/leak.jpg?resize=188%2C187

Introduction

Some time starting in mid November 2009, ten million teletypes all started their deet-ditta-dot chatter reeling off the following headline: “Hackers broke into the University of East Anglia’s Climate Research Unit….”

I hate that. It annoys me because just like everything else about climate-gate it’s been ‘value-added’; simplified and distilled. The contents of FOIA2009.zip demand more attention to this detail and as someone once heard Professor Jones mutter darkly, “The devil is in the details…so average it out monthly using TMax!”

The details of the files tell a story that FOIA2009.zip was compiled internally and most likely released by an internal source.

The contents of the zip file hold one top-level directory, ./FOIA. Inside that it is broken into two main directories, ./mail and ./documents. Inside ./mail are 1073 text files ordered by date. The files are named in order with increasing but not sequential numbers. Each file holds the body and only the body of an email.

In comparison, ./documents is highly disorganized. MS Word documents, FORTRAN, IDL and other computer code, Adobe Acrobat PDF’s and data are sprinkled in the top directory and through several sub-directories. It’s the kind of thing that makes the co-workers disorganized desk look like the spit and polish of a boot camp floor.

What people are missing entirely is that these emails and files tell a story themselves.

The Emails

Proponents of the hacker meme are saying that s/he broke into East Anglia’s network and took emails. Let’s entertain that idea and see where it goes.

There is no such thing as a private email. Collecting all of the incoming and outgoing email is simple in a mail server. Using: Postfix the configuration is always_bcc=<email address>, here are links on configuring the same for Sendmail, and for Exim. Those are the three main mail servers in use in the Unix environment. Two of them, Sendmail and Exim are or were in use as the external mail gateways and internal mail servers at the University of East Anglia (UEA).

When a mail server receives an email for someone@domain.net, it checks that it is authoritative for that domain. This means that a server for domain.net will not accept email for domain.ca. The mail server will usually then run checks on the email for spam, virus, and run other filters. It will then check to see whether to route the email to another server or to drop the email in a users mailbox on that server. In all examples examined in the released emails, the mail gateway forwarded the emails to another server.

The user then has a mail client that s/he uses to read email. Outlook Express, Eudora, Apple Mail, Outlook, Thunderbird, mutt, pine and many more are all mail clients.

Mail clients use one of two methods of reading email. The first is called POP and that stands for Post Office Protocol. A mail client reading email with POP logs into the mail server, downloads the email to the machine running the mail client and will then delete the original email from the users spool file on the mail server.

The second protocol is called IMAP, Internet Message Access Protocol. IMAP works by accessing the mailboxes on the mail server and doing most of the actions there. Nothing is actually downloaded onto the client machine. Only email that is deleted and purged by the mail client is gone. Either protocol allows the user the opportunity to delete the email completely.

Most email clients are setup for reading emails with POP by default and POP is more popular than IMAP for reading email.

The released emails are a gold mine for a system administrator or network administrator to map. While none of the emails released contained headers, several included replies that contained the headers of the original emails. An experienced administrator can create an accurate map of the email topography to and from the CRU over the time period in question, 1998 thru 2009.

Over the course time, UEA’s systems administrators made several changes to the way email flows through their systems. The users also made changes to the way they accessed and sent email.

The Users

Using a fairly simple grep1 we can see that from the start of the time-frame, 1999, until at least 2005 the CRU unit accessed their email on a server called pop.uea.ac.uk. Each user was assigned a username on that server. From the released emails, we can link username to people as such:

In the previously referenced grep comes some more useful information. For instance, we know that Professor Davies was using QUALCOMM Windows Eudora Light Version 3.0.3 (32) in September of 1999. (ref Email: 0937153268.txt). If you look at the README.txt for that version you can see that it requires a POP account and doesn’t support IMAP.

As mentioned previously, POP deletes email on the server usually after it is downloaded. Modern POP clients do have an option to save the email on the server for some number of days, but Eudora Light 3.0.3 did not. We can say that Professor Davies’ emails were definitely removed from the server as soon as “Send/Recv” was finished.

This revelation leaves only two scenarios for the hacker:

  1. Professor Davies’ email was archived on a server and the hacker was able to crack into it, or
  2. Professor Davies kept all of his email from 1999 and he kept his computer when he was promoted to Pro-Vice Chancellor for Research and Knowledge Transfer in 2004 from his position as Dean of the School of Environmental Sciences.

The latter scenario requires that the hacker would have had to know how to break into Prof. Davies’ computer and would have had to get into that computer to retrieve those early emails. If that were true, then the hacker would have had to get into every other uea.ac.uk computer involved to retrieve the emails on those systems. Given that many mail clients use a binary format for email storage and given the number of machines the hacker would have to break into to collect all of the emails, I find this scenario very improbable.

Which means that the mail servers at uea.ac.uk were configured to collect all incoming and outgoing email into a single account. As that account built up, the administrator would naturally want to archive it off to a file server where it could be saved.

This is a simple evolution. You just run a crontab to start a shell-script that will stop the mail server, move the mail spool file into a file somewhere else, nulls the live spool and restart the mail server. The account would reside on the mail server, the file could be on any server.

Alternatively you could use a procmail recipe to process the email as it comes in, but that may be a bit too much processing power for a very busy account.

This also helps to explain the general order of the ./mail directory. Only a computer would be able to reliably export bodies of email into numbered files in the FOIA archive. As the numbers are in order not just numerically but also by date, the logical reasoning is that a computer program is numbering emails as they are processed for storage. This is extremely easy to do with Perl and the Mail::Box modules.

The Email Servers

I’ve created a Dia diagram2 of the network topography regarding email only as demonstrated in the released emails. Here’s a jpeg of it:

CRU's network for email 	  from 1998 thru 2009.
click to enlarge

The first thing that springs to mind is that the admins did a lot of fiddling of their email servers over the course of ten years. 🙂 The second thing is the anomaly. Right in the middle of 2006-2009 there is a Microsoft Exchange Server. Normally, this wouldn’t be that big of an blip except we’ve already demonstrated that the servers at UEA were keeping a copy of all email in and out of the network. Admins familiar with MS Exchange know that it too is a mail server of sorts.

It is my opinion that the MS Exchange server was working in conjunction with ueams2.uea.ac.uk and I base this opinion on the fact that ueams2.uea.ac.uk appears both before and after the MS Exchange Server. It doesn’t change its IP address nor does it change the type of mail server that is installed on it. There is a minor version update from 4.51 to 4.69. You can see Debian’s changelog between the Exim versions here.

I’ve shown that the emails were collected from the servers rather than from the users accounts and workstations, but I haven’t shown which servers were doing the collection. There are two options, the mail gateway or the departmental mail servers.

As demonstrated above, I believe that the numbers of the filenames correspond to the order that the emails were archived. If so, the numbers that are missing, represent other emails not captured in FOIA2009.zip.

I wrote a short Bash program3 to calculate the variances between the numbering system of the email filenames. The result is staggering, that’s a lot of email outside of what was released. Here’s a graph of the variances in order as well as a graph with the variances numerically sorted . Graph info down below.

Variance from Email Number to the 	  last Email Number
click to enlarge
Variances sorted and plotted
click to enlarge

The first graph is a little hard to read, but that’s mostly because the first variance is 8,805,971. To see a little better, just lop off the first variance and rerun gnuplot. For simplicity, that graph is here. The mean of the variances is 402839.36 so the average amount of emails between each released email is 402,839. While not really applicable, but useful, the standard deviation is 736228.56 and you can visualize that from the second graph.

I realize that variance without reference is useless, in this instance the number of days between emails. Here is a grep of the emails with their dates of origin.

I do not see the administrators copying the email at the departmental level, but rather at the mail gateway level. This is logical for a few reasons:

  • The machine name ueams2.uea.ac.uk implies that there are other departmental mail servers with the names like ueams1.uea.ac.uk, (or even ueams.uea.ac.uk), maybe a ueams3.uea.ac.uk. If true, then you would need to copy email from at least one other server with the same scripts. This duplication of effort is non-elegant.
  • There is a second machine that you have to copy emails from and that is the MS Exchange server so you would need a third set of scripts to create a copy of email. Again, this would be unlike an Administrator.
  • Departmental machines can be outside the purview of Administration staff or allow non-Administrative staff access. This is not where you want to be placing copies of emails for the purposes of Institutional protection.
  • As shown with the email number variances, and if they are representative of the email number as it passed through UEA’s email systems, that’s a lot of emails from a departmental mail server and more like an institutional mail gateway.

So given the assumptions listed above, the hacker would have to have access to the gateway mail server and/or the Administration file server where the emails were archived. This machine would most likely be an Administrative file server. It would not be optimal for an Administrator to clutter up a production server open to the Internet with sensitive archives.

The Documents

The ./FOIA/documents directory is a complete mess. There are documents from Professor Hulme, Professor Briffa, the now famous HARRY_READ_ME.txt, and many others. There seems to be no order at all.

One file in particular, ./FOIA/documents/mkhadcrut is only three lines long and contains:

	  tail +13021 hadcrut-1851-1996.dat | head -n 359352 | ./twistglob > hadcrut.dat

	  # nb. 1994- data is already dateline-aligned

	  cat hadcrut-1994-2001.dat >> hadcrut.dat

Pretty simple stuff, get everything in hadcrut-1851-1996.dat starting at the 13021st line. From that get only the first 359352 lines and run that through a program called twistglob in this directory and dump the results into hadcrut.dat. Then dump all of the information in hadcrut-1994-2001.dat into the bottom of hadcrut.dat.

….Except there isn’t a program called twistglob in the ./FOIA/documents/ directory. Nor is there the resultant hadcrut.dat or the source files hadcrut-1851-1996.dat and hadcrut-1994-2001.dat.

This tells me that the collection of files and directories in ./documents isn’t so much a shared directory on a server, but a dump directory for someone who collected all of these files. The originals would be from shared folders, home directories, desktop machines, workstations, profiles and the like.

Remember the reason that the Freedom of Information requests were denied? In email 1106338806.txt, Jan 21, 2005 Professor Phil Jones states that he will be using IPR (Intellectual Property Rights) to shelter the data from Freedom of Information requests. In email 1219239172.txt, on August 20th 2008, Prof. Jones says “The FOI line we’re all using is this. IPCC is exempt from any countries FOI – the skeptics have been told this. Even though we (MOHC, CRU/UEA) possibly hold relevant info the IPCC is not part our remit (mission statement, aims etc) therefore we don’t have an obligation to pass it on.”

Is that why the data files, the result files and the ‘twistglob’ program aren’t in the ./documents directory? I think this is a likely possibility.

If Prof. Jones and the UEA FOI Officer used IPR and the IPCC to shelter certain things from the FOIA then it makes sense that things are missing from the ./documents directory. Secondly it supports the reason that ./documents is in such disarray is that it was a dump folder. A dump folder explicitly used to collect information for the purpose of release pursuant to a FOI request.

Conclusion

I suggest that it isn’t feasible for the emails in their tightly ordered format to have been kept at the departmental level or on the workstations of the parties. I suggest that the contents of ./documents didn’t originate from a single monolithic share, but from a compendium of various sources.

For the hacker to have collected all of this information s/he would have required extraordinary capabilities. The hacker would have to crack an Administrative file server to get to the emails and crack numerous workstations, desktops, and servers to get the documents. The hacker would have to map the complete UEA network to find out who was at what station and what services that station offered. S/he would have had to develop or implement exploits for each machine and operating system without knowing beforehand whether there was anything good on the machine worth collecting.

The only reasonable explanation for the archive being in this state is that the FOI Officer at the University was practising due diligence. The UEA was collecting data that couldn’t be sheltered and they created FOIA2009.zip.

It is most likely that the FOI Officer at the University put it on an anonymous ftp server or that it resided on a shared folder that many people had access to and some curious individual looked at it.

If as some say, this was a targeted crack, then the cracker would have had to have back-doors and access to every machine at UEA and not just the CRU. It simply isn’t reasonable for the FOI Officer to have kept the collection on a CRU system where CRU people had access, but rather used a UEA system.

Occam’s razor concludes that “the simplest explanation or strategy tends to be the best one”. The simplest explanation in this case is that someone at UEA found it and released it to the wild and the release of FOIA2009.zip wasn’t because of some hacker, but because of a leak from UEA by a person with scruples.

Footnotes

1 See file ./popaccounts.txt

2 See file ./email_topography.dia

3 See file ./email_variance.sh

4 See file ./gnuplotcmds

Notes

Graph Information

Graphs created with gnuplot using a simple command file4 for input. I use a stripped down version of the variants_results_verbose.txt file, it’s the same, just stripped of comment and the filenames.. The second graph is a numerically sorted version, $> sort -n ./variance_results.txt > variance_sorted_numerically.txt.

Assigned Network Numbers for UAE from RIPE.NET

RIPE.NET has assigned 139.222.0.0 – 139.222.255.255,193.62.92.0 – 193.62.92.255, and 193.63.195.0 – 193.63.195.255 to the University of East Anglia for Internet IP addresses.

RIPE.NET Admin contact for the University of East Anglia: Peter Andrews, Msc, Bsc (hons) – Head of Networking at University of East Anglia. (Linked In, Peter isn’t in the UEA directory anymore so I assume he is no longer at UEA.)

RIPE.NET Tech Contact for the University of East Anglia: Andrew Paxton

Current Mail Servers at UEA

A dig for the MX record of uea.ac.uk (email servers responsible for the domain uea.ac.uk) results in the following:

	  $> dig mx uea.ac.uk

	  ; <<>> DiG 9.6.1-P2 <<>> mx uea.ac.uk

	  ;; global options: +cmd

	  ;; Got answer:

	  ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 737

	  ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 13, ADDITIONAL: 13

	  ;; QUESTION SECTION:

	  ;uea.ac.uk.			IN	MX

	  ;; ANSWER SECTION:

	  uea.ac.uk.		50935	IN	MX	2 ueamailgate01.uea.ac.uk.

	  uea.ac.uk.		50935	IN	MX	2 ueamailgate02.uea.ac.uk.

The IP addresses for the two UEA email servers are:

ueamailgate01.uea.ac.uk. 28000 IN A 139.222.131.184

ueamailgate02.uea.ac.uk. 28000 IN A 139.222.131.185

Test connections to UEA’s current mailservers:

	  $> telnet ueamailgate01.uea.ac.uk 25

	  Trying 139.222.131.184...

	  Connected to ueamailgate01.uea.ac.uk.

	  Escape character is '^]'.

	  220 ueamailgate01.uea.ac.uk ESMTP Sendmail 8.13.1/8.13.1; Mon, 7 Dec 2009 01:45:42 GMT

	  quit

	  221 2.0.0 ueamailgate01.uea.ac.uk closing connection

	  Connection closed by foreign host.

	  $> telnet ueamailgate02.uea.ac.uk 25

	  Trying 139.222.131.185...

	  Connected to ueamailgate02.uea.ac.uk.

	  Escape character is '^]'.

	  220 ueamailgate02.uea.ac.uk ESMTP Sendmail 8.13.1/8.13.1; Mon, 7 Dec 2009 01:45:49 GMT

	  quit

	  221 2.0.0 ueamailgate02.uea.ac.uk closing connection

About Me

I’ve been a Unix, Windows, OS X and Linux systems and network administrator for 15 years. I’ve compiled, configured, and maintained everything from mail servers to single-signon encrypted authentication systems. I run lines, build machines and tinker with code for fun. You can contact me via: lance@catprint.ca.

Lance Levsen,

December, 2009


Sponsored IT training links:

We offer 100% pass result in first attempt for all kind of IT exams including 70-685 and 70-271. Join 640-460 online course to save a big deal on real exam.


0 0 votes
Article Rating
256 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Molon Labe
December 7, 2009 8:43 am

It’s already been shown that the email filenames are derived from the unix ctime representation of the message send date.

James F. Evans
December 7, 2009 8:44 am

While it shouldn’t matter whether the e-mails and computer programs were “hacked” or leaked because it’s the underlying science that counts — it does matter politically.
Leaked material the result of an anonymous whistleblower (an anonymous whistleblower is contradictory) is more politically powerful.
As the leak would be the result of a troubled conscience from within the heart of the scandal — or is that heart of darkness…

geo
December 7, 2009 8:46 am

I agree that the compilation of this archive was more likely (maybe even *much* more likely) done internally than externally.
However, I don’t see why it follows that makes it more likely to be an internal leak *after* the archive was compiled. A hacker might still have stumbled on it. Or, even not a hacker. . . just someone external who found that file temporarily residing at an open ftp where it was put mistakenly (but with no intention to leak) by someone inside UEA.

Invariant
December 7, 2009 8:48 am

Sure. Sounds more like a undercover climate scientist from KGB or CIA.

Neo
December 7, 2009 8:49 am

The also leaves one other conclusion …
all the e-mails about AR4 that Phil Jones thought he deleted, most likely still exist.

chainpin
December 7, 2009 8:49 am

Wow, that is oustanding!
Very nice analysis, thank you.

Third Party
December 7, 2009 8:52 am

from http://thehill.com/blogs/congress-blog/energy-a-environment/70857-climategate-sparks-luetkemeyer-call-for-investigation-sparks-interest-in-legislation-rep-blaine-luetkemeyer
In 1890’s, Arrhenius built upon Fourier’s assessment of atmospheric properties plotting CO2 and temperature data collected in industrialized England. Arrhenius’ plots and calculations related CO2 and ambient temperatures.
Callendar (1930’s) extended the analysis using long term observations from 200 stations reiterating the relation between CO2 and climate warming.
Keeling (1950’s) began collecting atmospheric CO2 samples at Mauna Loa Observatory, Hawaii which is the most complete record. USGS reports all volcanic activity produces nearly 200-million tons CO2 annually; although much less than human activity production. Mauna Loa, near the Observatory and the world’s most active volcano, had major eruptions in 1950, 1975, and 1984. Atmospheric CO2 levels measured at volcanoes indicate the degree of activity and estimated heat flow from one volcano are reported at140-mW/m2.
Correlating CO2 and temperatures data collected near active volcanoes should be significant but not show a cause and effect relation; however, correlating world-wide data significantly shows CO2 lagging temperature by approximately two years. Arrhenius and Callendar analysis similarly could be significantly biased owing to urban heat-island effects and extensive coal burning at the time, as CO2 is an abundant byproduct of burning.
Apparently, no laboratory control experiment to date, such as in a biodome, has shown CO2 levels influencing ambient temperatures. Tyndall (1861) measured the absorptive characteristics of CO2 followed by more precise measurements by Burch (1970). Absorbance is a measure of the quantity of light (energy) absorbed by a sample (CO2 molecule) and the amount of absorbed energy can be represented as specific heat of a substance. Specific heat of CO2 ranges from 0.791-kJ/kgK at 0-degrees F to 0.871-kJ/kgK at 125-degrees F and average atmospheric concentrations are 0.0306-percent. As revealed, the specific heat of CO2 increases as ambient temperatures increase showing CO2 likely is an ambient temperature buffer.
The atmosphere contains from 4-percent water vapor in the troposphere to 40-percent near the surface. Specific heat of water vapor relatively remains constant at 1.996-kJ/kgK. Water absorbs energy (heat) and evaporates to water vapor. During condensation (precipitation) , latent heat is released to the atmosphere thus increasing ambient temperatures.
Water vapor holds the majority of atmospheric heat and regulates climate and temperature more than any compound. Historically, however, water vapor characteristics as related to climate were much less appreciated, but investigations concerning the significance water vapor plays in global climate-dynamics are just beginning.
Energy not stored in the atmosphere is released into space through radiation. Re-radiation is the emission of previously absorbed radiation by molecules. Specific heat of water vapor and CO2 molecules shows that water vapor reradiates significantly more energy back to the surface and this case further is justified by quantities of each compound.
Thus, this synopsis and other publications suggest that minute variations in atmospheric CO2 concentrations likely results in an insignificant affect on climate; whereas water vapor likely is the significant factor. Nevertheless, this argument easily could be rectified with an appropriate biodome-type control experiment.
BY ehmoran on 12/07/2009 at 10:42

None
December 7, 2009 8:52 am

How can someone who claims to be a unix system admin for 15 years not have realised the filenames were “seconds since epoc” dates ?

wws
December 7, 2009 8:52 am

The weekend cover story claiming that “The Russian KGB did this to discredit Copenhagen!!!” was a hoot.
Glad to see the good work done to knock down that nonsense quickly. Of course, that’ won’t stop the warmists from preaching the “Evil Hacker!” story for the next 20 years.

Sped
December 7, 2009 8:54 am

Nice work! Outstanding forensic type analysis of the info.
On a somewhat related note, Google is still hiding the autosuggest! Here is a thread to complain at, if you are interested:
http://tinyurl.com/yzssbsg which really is:
http://www.google.com/support/forum/p/Web+Search/thread?tid=25112ee0c29cbd01&hl=en&fid=25112ee0c29cbd0100047a24d00f3afd
And here is a recent story on Google handling of “climategate”
http://www.seroundtable.com/archives/021306.html

David Madsen
December 7, 2009 8:57 am

The analysis of the creation of the FOI2009.zip file is very intuitive. However is it plausible that a hacker managed to find the file on an internal FTP server as opposed to someone on the inside releasing it? In my opinion, this analysis doesn’t quite put the nail in the coffin on the hacker scenario.
I personally think that this was either a leak, or a hacker got lucky and stumbled across the FOI2009.zip file.

Viking141
December 7, 2009 8:58 am

Excellent analysis and torpedoes the “illegal hacking” sob-story that we’re being fed, just as I, and Im sure many others, suspected. Nice job!

Neo
December 7, 2009 8:59 am

The very easily could have come from within the CRU …

Lesson 1: Don’t let users put passwords in their signatures. Yep, you got that right: One of the scientists included both on his e-mail signature — which means that anyone receiving an e-mail from this guy had access to his files. This may have been the source of the hack; in fact, some folks have theorized that a recipient of the e-mail was the source of the data dump.

SJones
December 7, 2009 8:59 am

With regard to it being ‘stumbled upon’ – wasn’t it the case that is was released the same day as Steve McIntyre’s FOI appeal was turned down? If so, this was likely no accident.

December 7, 2009 9:00 am

The analysis is important because the dominant media narrative involves an attack by “Russian hackers”, which taints the release with motive.
Witness attempts to build on the “sinister motives” theme…
http://www.desmogblog.com/breaking-impersonators-attempt-access-canadian-government-centre-fo-climate-modeling-and-analysis
Apparently the incident wasn’t worth mentioning until now.

mojo
December 7, 2009 9:00 am

Yes indeedy. Never bought the “evil haxors” theory, Russian , Chinese or Martian. Too much connect time required, too much poking around, likely to set of IDM alarms – not a good prospect for a smash and grab.
Inside job. Said so the first day.

NK
December 7, 2009 9:01 am

Dear Mr Levsen–
Thanks for all this work. Very comprehensive and understandable. You wrote in conclusion:
Occam’s razor concludes that “the simplest explanation or strategy tends to be the best one”. The simplest explanation in this case is that someone at UEA found it and released it to the wild and the release of FOIA2009.zip wasn’t because of some hacker, but because of a leak from UEA by a person with scruples
I agree with the first clause, as that was my conclusion 2 weeks ago reading the file. The second clause is debatable. Once that file was compiled for FOIA purposes, it could be accessed by anyone with clearance, and that circle may have become pretty wide, FOI officials, University lawyers, tech staff etc. Anyone of those people, could have released the FOI file for various reasons. We can conclude with a high of confidence “WHAT” the file was, it due diligence in response to the FOI demand, but who released it and why, can’t be known on this info. I agree with GEO about that.

S
December 7, 2009 9:01 am

The explanation of the email numbers is far simpler than this. The email file numbers are standard UNIX epoch numbers. UNIX counts time as the number of seconds since Jan 1, 1970. Take any email filename and put the number into an epoch calculator such as the one on http://www.unixtimestamp.com/index.php and out pops out the email’s date, to the second, modulo time-zone/daylight savings issues which can cause the number to be off by an integral number of hours. There’s nothing special or deep about those numbers. They’re only a convenient way for whomever made the archive to assign unique filenames to each email, and a somewhat dangerous technique if two emails happen to have been processed in the same second.

Ron de Haan
December 7, 2009 9:04 am

Very good approach and very good arguments based on hard analyses and reasoning.
I am convinced it was a leak from the beginning and it should be clear to the world it was a leak.
It takes the wind out of the sails of those stating it was a common theft which allows then to smear the entire skeptical community as “thieves”.
What’s also important is that the BBC has been sitting on leaked information for weeks without undertaking any action.
I am sure the momentum caused by ClimateGate will continue for months to come as more and more facts and analyses will surface and the truth becomes known by the public.
Most of them already new AGW was a scam. Now they know for shore.
Nothing criminal about that.
Thanks for a job well done.

Rhys Jaggar
December 7, 2009 9:05 am

There is little doubt, however, that Russian figures in the UK gain unauthorised access to contents of PCs which have links to the internet.
I remain 100% confident that Chelsea FC have in the past gained access to documentation stored on my PC and released to no-one and that their performance of the hokey-cokey on the Old Trafford pitch prior to a game with Manchester Utd might be the smoking gun to demonstrate it. The document in question was a spoof Ali C conversation with recently departed Chelsea FC manager, Mr Jose Mourinho……for the record, my PC at the time was in Leeds, Yorkshire, around 200 miles away from Mr Abramovitch’s home. Ditto his football club’s home address.
Not that this is peculiar to Mr Abramovitch’s club. Far more likely that the surveillance comes from others who pass it on or sell it on for money. Mr Murdoch’s tabloid executives no doubt value such approaches, particularly in concert with Her Majesty’s constabularies of various districts…..
I wonder how many journalists are drinking beer in Norwich watering holes trying to identify Deep Throat right now?

December 7, 2009 9:06 am

I think Harry the programmer is the source.

Clive
December 7, 2009 9:06 am

Lance Levsen,
Thanks for the analysis. Not read all, but the documentation is very good as is the summary.
Thank you.
Clive

D. King
December 7, 2009 9:07 am

Neo (08:49:32) :
The also leaves one other conclusion …
all the e-mails about AR4 that Phil Jones thought he deleted, most likely still exist.
Good point.

December 7, 2009 9:09 am

Well done. Many have speculated about a insider leak, but this is as close to proof as one can get without access to the UEA computers’ logs. Even then, few civilian research centers track incoming/outgoing ftp transfers. Fewer still log USB disk connects or transfers. It would be relatively straightforward for an insider to copy FOI2009.zip onto a USB key, visit the local library & upload the file anonymously to the russian ftp site.
However, while I agree the compilation of FOI2009.zip was very probably an inside job, an outsider may have found a copy that the staff had dumped on a local ftp server for transfer to a coworker. Of course a Windows shared folder with guest access allowed would suffice, as well 😉

boballab
December 7, 2009 9:11 am

Interesting.
Here is more food for thought: Remember back when this all started we knew it came from a Russian Proxy server and they posted the link to the Air Vent. Then Gavin starts running around about being hacked and that they tried to upload things from a servit in TURKEY. Now we got this thing up in Canda and the some press reading from the UN’s press release that its the Russin’s because it came froma Russian Proxy server.
Notice how that “hack” into RC from a server in Turkey just dissapeared, like Briffa’s divergence?

Henry chance
December 7, 2009 9:14 am

The warmists are superstitious ..Joe Romm claimed global warming caused an airplane crash before they even found where it went down. They are calling this theft and haven’t investigated how the data came out so well organized.

patrick healy
December 7, 2009 9:16 am

what a tremendous piece of work, lance.
as an amateur it looks quite plausable to me.
now all we are waiting for is the one person at CRU with dignity to show themselves. I can hear the whistle blowing already.
I wonder if Sir ‘wottisname’ will call Lance as an expert witness for his internal enquiry.

AlanG
December 7, 2009 9:17 am

Nice analysis. Definitely an internal job. CRU isn’t a big organization like a bank where security is ultra tight. There would be students running around the place. Some of the IT students might even be given admin rights to look after the servers as part of their learning.
My guess would be someone, either the FOI guy or a student, was asked to do a trial cut for an FOI request. Did a hacker find it on an ftp server or did a student push it out? My guess is the second. Wasn’t there something about procedures at UEA and/or CRU being tightened up in September to control student access?
The IPCC has got the wrong end of the stock completely and are going with the break-in scenario like Watergate. Pathetic:
Climate email theft likened to Watergate break-in:
http://www.theaustralian.com.au/news/climate-email-theft-likened-to-watergate-break-in/story-e6frg6xf-1225807887910

JonesII
December 7, 2009 9:18 am

Then….the whistleblower will probably attend Copenhagen…he/she is already there!…
Let´s get more popcorn!

AndrewWH
December 7, 2009 9:20 am

Now I am wondering if the mail servers have some or all of the original raw data on them. Granted, CD burners have been around a long time, but for ease surely some of the data could have just been emailed to Harry once it was tabulated so he could work on it, especially if the files are not too big. Then it may still be in existence.

bill
December 7, 2009 9:21 am

It was obviously a FOI collection – Emails are not held in clear form on a server A password is required to unscramble them (not very secure). These have all been saved as text by the owner as a result of FOI.
However:
Inside job/ outside job it all falls within the the Computer misuse act. UNLESS you have authorised access to the data:
http://www.opsi.gov.uk/acts/acts1990/UKpga_19900018_en_1.htm
(1) A person is guilty of an offence if—
(a) he causes a computer to perform any function with intent to secure access to any program or data held in any computer;
(b) the access he intends to secure is unauthorised; and
(c) he knows at the time when he causes the computer to perform the function that that is the case.
(2) The intent a person has to have to commit an offence under this section need not be directed at—
(a) any particular program or data;
(b) a program or data of any particular kind; or
(c) a program or data held in any particular computer.
(3) A person guilty of an offence under this section shall be liable on summary conviction to imprisonment for a term not exceeding six months or to a fine not exceeding level 5 on the standard scale or to both.

December 7, 2009 9:21 am

“S/HE” [snip]!!!!!
Brave enough to cock a snook at the entire Warmist conspiracy but you fold like a cheap suit before the Pronominal Equity Police. Sheesh!

hunter
December 7, 2009 9:21 am

boballab makes a great point.
RC/Schmidt are deliberately confusing this issue.
The Russian hack cover is so disengenuous as to be laughable.

PhilW
December 7, 2009 9:22 am

How damaging is “Climategate” to NASA?
http://www.thespacereview.com/article/1522/1

Steve
December 7, 2009 9:27 am

Regardless, the MSM will continue calling this a “hack,” even though that can’t be confirmed and it even seems to be otherwise.

Anand Rajan KD
December 7, 2009 9:28 am

For those suggesting that the FOI2009 Zip was created previously and left on a server, and also for those suggesting that it might have done as part of a FOI request, I have questions:
1) UEA and the Team had no intentions of complying with the requests as the emails show. Why would they have zip files ready, as if to suggest a ‘just in case’ action on their part? They did not want to do it, and believed they had legal cover (however flimsy) to go down that path.
2) Why would anyone put pictures of scientists floating off on ice floes with polar bears as part of a FOI response?
The selection of emails also suggest someone knew the significance and import of what was being talked about in the emails. We must not forget that there was an attempt to upload/link to these files to RealClimate.
FOI2009.zip was created after files were collated, probably from several computers. Files were then probably carried out on a thumb drive, physically.
Assuming UEA does not have a fair idea of who leaked the files, one can also extrapolate that a server/s with the emails is not under camera surveillance. We have to take this into account given the fact that the UK is riddled with cameras.
It would also be nice to believe that it is one of the Team who actually is responsible for the leak.

Editor
December 7, 2009 9:29 am

Google is working just fine for me autosuggesting “climategate” when I type “clim”.

Chris Schoneveld
December 7, 2009 9:30 am

Why was the file first sent to a BBC reporter and not to one of the Telegraph?

crosspatch
December 7, 2009 9:32 am

I believe that the numbers of the filenames correspond to the order that the emails were archived. If so, the numbers that are missing, represent other emails not captured in FOIA2009.zip.

No, the filenames of the emails are the unix epoch time of the email. The missing numbers are elapsed time between the emails. The filenames are the datestamps. I believe this is the date when the file was archived as there is generally a couple of hours difference between that date stamp and the date/time stamp in the headers of the email.
For example:
1123622471.txt
1123622471 is unix time August 9, 2005 21:21:11 GMT
The date/time stamp on that email is:
Date: Tue Aug 9 17:21:11 2005
Notice the hour is different but the minutes/seconds are the same.

Ken
December 7, 2009 9:32 am

Hmmm.
Presumably you think the disclosure was a good thing.
Now you’re explaining how it may have been done by an insider who would have had to have particular access capability — going so far as to compile a pattern that points to very very few possible candidates.
Which means you’re helping narrow down the possibilities of the Source — helping the people that suppressed this release [via FOI]!
Which means you’re making it that much harder for that person to
– Get away with it
– Disclose any additional information of significance
– Etc.
All of which is very bad–from your perspective.
As anyone that’s read a spy novel knows–ALWAYS PROTECT SOURCES & METHODS!!!!!
Recall the Hansen case involving the FBI informant to the Russians — they likely weren’t sure who he was…but they sure didn’t screw up a good thing (for them) by disclosing their inferences either.
An article like this sure makes you look clever, but in the aggregate, why help the people that ‘acted badly’? Let them figure it out, or not–certainly don’t help them! Doing so only discourages others, or the same person, in a similar situation from coming forward.

boballab
December 7, 2009 9:36 am

Here is something else to think on:
Notice: that all the “its the Russians” that did this, All track back to the Frenchmen from the UN waving his hands and saying it was so. It started about 2 days ago in the Daily Mail in the UK and they quoted it as such. Now you get the same story but it is now an “un-named source from the UN”
Notice: there has been no statements from the Local police who are investigating the release from UEA. If they had narrowed it down to foriegn hackers the case would have been turned over to the UK version of the FBI and you would have seen headlines of that fact. Instead as far as the world knows the local police are still investigating. Local cops investigate local crimes not international ones.
Notice: couple of days ago While I was surfing around I saw one article in a UK paper that quoted an official from Scotland Yard that they were now investigating the CRU for FOI violations. Nothing spread from that it has dropped into a pit of silence.
Just some musings from someone that just so happens to have a busted leg and is laid up for awhile at the right time for all this.

Skeptic Tank
December 7, 2009 9:38 am

At this point, I’m so cynical and suspicious of these people, I wouldn’t be surprised if they already have identified the whistle-blower, but they won’t divulge the person’s identity. They want to maintain themselves as the victims.
What if that person were identified and disciplined; fired or even perhaps prosecuted. What a source of information they would be to the public. What a source of testimony they would be in related proceedings and investigative hearings.
Oh, they wouldn’t want that.
This isn’t over. I just hope the person in not in danger.

crosspatch
December 7, 2009 9:39 am

There is a unix time conversion utility here:
http://www.csgnetwork.com/unixds2timecalc.html

December 7, 2009 9:41 am

Third Party (08:52:03) :
The atmosphere contains from 4-percent water vapor in the troposphere to 40-percent near the surface.
Get the numbers straight. At 100% relative humidity at 30C [tropics] the concentration of water vapor is 30 gram/cubic meter. Considering that 1 cubic meter of air at the surface weighs 1234 gram, the water concentration can at most be 30/1234 = 2.4%.
Your [or your source’s] numbers are 10-20 times too high.

Svein
December 7, 2009 9:41 am

Darrell S Kaufman was the moron who had his user id and password in his email signature.

Ed Scott
December 7, 2009 9:42 am

The IPCC is irredeemably (no “hope” for “change”) corrupt. The exposure of the truth is a “crime” in the new world of politicized science. The goal of this new “science” is to lie, cheat and steal with the end justifying the means.
The following four reports, via CNN, are interspersed by commercials, so hang-in.
http://us.cnn.com/video/?/video/world/2009/12/07/sot.climate.ipcc.ebs
The head of the UN climate change panel (old friend “no meat” Pachauri) discusses ‘climategate’ and how it’s trying to undermine the group’s findings.
Scientists are accused of ‘cooking the books’ on climate change. CNN’s Jim Acosta reports.
Danish Prime Minister Lars Lokke Rasmussen starts the UN Conference on Climate Change.
CNN’s John Roberts looks at the effect hacked emails may have on the U.N. Climate Conference (Peter Liss, Phil Jones’ replancement, appears to be more of the same).

kadaka
December 7, 2009 9:47 am

Given the revealed high quality of work at UEA/CRU, I wonder if this FOIA compilation was simply left on an anonymous server, awaiting the permission of management to release the address as compliance with the FOIA request. Then someone stumbled upon it…

December 7, 2009 9:49 am

Great job, Charles. Now, if Babs Boxer can pull back the blinders and see this for what it is.
Unfortunately, these politicians are set in their views because cap and tax is a potentially new revenue stream.

Michael
December 7, 2009 9:52 am

“It is the Sun that determines the climate of the Earth, not CO2, and the Sun is in a natural cycle called a solar minimum, producing less radiation to warm the Earth.”
How to Destroy the U.S. Economy: Regulate Carbon Dioxide
http://canadafreepress.com/index.php/article/17657

maz2
December 7, 2009 9:53 am

Goreacle Report: The Gore Effect.
The Extortionists Duke It Out.
Big State vs Big Criminals.
…-
“The Climate Mafia
Fraudulent Emissions-Trading Schemes Rob German Tax Authorities
The Kyoto Protocol introduced a scheme for trading emissions certificates as a way to help reduce CO2 emissions. German tax authorities are now investigating almost 40 companies that traded certificates for allegedly taking advantage of loopholes in sales tax laws to bilk the taxman out of hundreds of millions of euros.
German Environment Minister Norbert Röttgen has hardly been in office for much more than a month, but he’s already choosing his words for dramatic effect. “It’s about the way we live, and it’s about survival,” Röttgen said last Thursday before the German parliament, the Bundestag, referring to the climate summit beginning Monday in Copenhagen. At the summit, the nations of the world will search for ways to reduce the CO2 emissions behind global warming. One of the tools to be discussed is the trading of emissions certificates.
In Germany, though, it is precisely this instrument that is causing a huge headache for Röttgen, as dozens of tax offices across the country are investigating shady emissions trading companies. All of the companies in question maintain accounts with the German Emissions Trading Authority (DEHSt), an arm of the ministry Röttgen heads. According to DEHSt head Hans-Jürgen Nantke, since September, his agency has “received official requests for assistance in cases relating to suspected sales tax fraud from various regional tax offices and tax investigation offices.”
The authorities are investigating questionable transactions in Germany’s central emissions trading registry. As in any other EU country, public utilities and industrial enterprises maintain accounts in the registry, which provides them with a certain number of emissions credits. Under the 2005 Kyoto Protocol, the emissions of climate-harming carbon dioxide are to be reduced as efficiently as possible with the help of this method of trading emissions certificates. According to this system, companies that invest in new, eco-friendly technologies no longer need all of their certificates and can, in turn, sell them to others — and at a steep price.
Current prices for these certificates are set on exchanges, such as France’s Bluenext, the ECX in London, the EEX in Leipzig and, more recently, the Greenmarket segment of the Munich Stock Exchange. At the moment, the right to emit one ton of CO2 into the atmosphere is worth about €14 ($21). In the first half of 2009, the volume of trade in emissions certificates in Europe alone already amounted to €40 billion.
Going after the Climate Speculators
The high-stakes deals being made with certificates are attracting more and more speculators. While investment banks, such as Goldman Sachs, and US-based hedge funds are speculating on emissions certificates, small trading companies are now trying to bilk tax authorities out of the sales tax that is part of many of these certificate-trading deals. Tax authorities are investigating “an estimated 30 to 40 companies,” says Nantke, adding that some of these companies are based abroad.
Investigators are apparently also interested in bank transactions.” (more)
http://www.spiegel.de/international/business/0,1518,665594,00.html

Michael
December 7, 2009 9:53 am

This is on Climate Depot’s top headlines.
Can a lull in solar activity head off climate change?
http://www.boston.com/business/articles/2009/12/07/can_a_lull_in_solar_activity_head_off_climate_change/

Jerker Andersson
December 7, 2009 9:54 am

Elementary, Dr Watson.
Very interesting analysis and probably accurate.
At this very moment there are one or possibly a few ppl who currently realize what they have done releasing this data.
Will he/she or they ever be revealed or will they reveal themselves? I think they one day will reveal what really happened when they released the data but not until they have retired so it could take decades before it happens unless someone else figure it out first.
I am very sure that this is a fact that is known internally at UEA also. This person or people may know more than what is in that zip file so they may move forward carefully with their internal investigations.
Thre could be more data that is not yet revealed that they do not want to become public.

David Segesta
December 7, 2009 9:58 am

“… a leak from UEA by a person with scruples.”
The possibility that there might be a person at UEA with scruples is indeed encouraging.

Henry chance
December 7, 2009 10:00 am

climate Progress posted a youtube on the “hack” and took it down
http://climateprogress.org/2009/12/07/climate-crock-smacks-the-hack-attack/
Looks like Scotland Yard knows if it was hacked or leaked.

George E. Smith
December 7, 2009 10:02 am

“”” Leif Svalgaard (09:41:36) :
Third Party (08:52:03) :
The atmosphere contains from 4-percent water vapor in the troposphere to 40-percent near the surface.
Get the numbers straight. At 100% relative humidity at 30C [tropics] the concentration of water vapor is 30 gram/cubic meter. Considering that 1 cubic meter of air at the surface weighs 1234 gram, the water concentration can at most be 30/1234 = 2.4%.
Your [or your source’s] numbers are 10-20 times too high. “””
I agree Leif; much too high those numbers; I believe that about 1% is the global average moisture content of the atmosphere; so your 2.4 makes a lot of sense.
I could get to work, and calculate the result; but maybe you already know at your finger tips. It is my conjecture that the atmosphere over the dryest of deserts always contains more H2O molecules than CO2 molecules. That may not be true at Vostok Station when it is having one of its cold spells, but I believe it is true of any tropcal desert.
Can you confirm; or deny ?
George

Marc
December 7, 2009 10:02 am

The email filenames are unix timestamps.
As an example from this post: “In email 1106338806.txt, Jan 21, 2005 Professor Phil Jones states…”
If we run the number portion of the filename through the unix ‘date’ command with the -r option (-u reports the time in UTC), we get:
$ date -ur 1106338806
Fri Jan 21 20:20:06 UTC 2005
The same date as reported in the post.

Michael
December 7, 2009 10:03 am

Saudi Arabia calls for ‘climategate’ investigation
http://www.politico.com/news/stories/1209/30291.html
Saudis at COP15: ‘Climategate’ shakes trust in scientists, independent inquiry needed

Cold Englishman
December 7, 2009 10:03 am

One other small point that is significant toward the case of it being a whisleblower. The file was first planted on the BBC weather man’s desk 2 weeks before it was released worldwide. One has to assume that the leaker thought that the Beeb would pick up on the scoop and publish the story, but when they didn’t, decided to try the Russian server. That’s why it was a leak and not a hack.

Charles. U. Farley
December 7, 2009 10:04 am

Neo (08:49:32) :
The also leaves one other conclusion …
all the e-mails about AR4 that Phil Jones thought he deleted, most likely still exist.
In the other as yet unreleased 100mb perhaps?

Optimist
December 7, 2009 10:13 am

surely it is more likely that the dump was assembled specifically NOT to be released. In other words, while tidying up files in preparation for a possible FOI release, they chose to put any embarrassing material in a separate pile to be deleted so it would not be released and this was the file the whistleblower freed. This was the `to shred’ folder?

Carlo
December 7, 2009 10:13 am

Iimpressive analysis by an expert.
What a great piece of work from Charles the moderator and Lance Levsen, Network Analyst.

Tenuc
December 7, 2009 10:19 am

Good analysis Lance. I agree that a leak is the most likely explanation.
Only a very special sort of hacker would know what to look for and there are very few groups with the necessary tools needed to monitor what’s been going on and assemble the material.
Although less likely, I still think GCHQ or the FBI could pull off this sort of stunt, and I don’t think either of these groups would want a world government. The use of a Russian server to hide behind sounds right up their street too.
I’m sure one day the truth will out.

radun
December 7, 2009 10:23 am

Cold Englishman (10:03:48) :
“The file was first planted on the BBC weather man’s desk 2 weeks before it was released worldwide.”
BBC should be able to trace back the original source of their initial contact.

Claude Harvey
December 7, 2009 10:23 am

After reading this piece and the comments that followed, I don’t think I know much more than before about the true source of the released CRU treasure trove.

Håkan B
December 7, 2009 10:28 am

Considering the mess Harry had to deal with I wouldn’t be surprised if security was a complete mess too. So an insider could be just anyone in the house, eaven if the file was in on a “secured” system, maybe eaven an outsider visiting the institution.

December 7, 2009 10:30 am

George E. Smith (10:02:20) :
I could get to work, and calculate the result; but maybe you already know at your finger tips. It is my conjecture that the atmosphere over the dryest of deserts always contains more H2O molecules than CO2 molecules. That may not be true at Vostok Station when it is having one of its cold spells, but I believe it is true of any tropical desert.
Can you confirm; or deny ?

Tropical deserts are hot and can thus contains more water. The driest desert [except from Antartica] is the Atacama [high altitude, thus cold] where water vapor is about 1 mm or ~0.1% which is more than the 0.04% CO2.
About Vostok: when it is really cold there the CO2 also freezes out…

Jack Green
December 7, 2009 10:30 am

Cold Englishman has the best answer. It’s also possible that the journalist or someone with access to what the journalist had received thought this information needed to go public so he posted it on the Russian web page. I would be talking to the journalist and seeing what he knows. It starts there.

December 7, 2009 10:34 am

There is another possible Source but nobody has mentioned it. Yet IMO it still fits best, and is kindest to all. Though I might still be wrong. Clue 1:Has anyone discovered a single email or document that is irrelevant? Clue 2: Has anyone here actually examined Crop Circles? I mean, given them the kind of mass survey with careful direct attention to insitu evidence details that the Surface Stations project teaches?

December 7, 2009 10:36 am

Lance/CTM – superb analysis and forensics – many thanks.

Robinson
December 7, 2009 10:36 am

In other news, Bishop Hill has posted a section of an email, as he says, it’s astonishing:

The results of this study will show that we can probably say a fair bit about 100 year variability was like with any certainty (i.e. we know with certainty that we know fuck-all).

That’s right, we literally know [self-snip] all!

Paul Vaughan
December 7, 2009 10:42 am

If there is an ally undercover inside leaking valuable info, why draw attention to them?

AnonyMoose
December 7, 2009 10:47 am

So some of the deleted emails still exist in the mail archive. Has anyone announced an FOI request for them, before they get deleted again?

crosspatch
December 7, 2009 10:48 am

” Optimist (10:13:02) : ”
While it seems silly to attempt to guess motivations, your thought occurred to me. If I wanted to make sure something WASN’T found in the course of an FOIA investigation, I might assemble a folder of stuff I wanted deleted in a hurry if I needed to. But if I were a sysadmin who worked there and who also happened to follow places like CA, tAV, and WUWT, I might know that an FOIA inquiry was coming and I might assemble such a file on my own.
The problem with that notion is that I would also have to have a pretty good idea of what code to include on the documents directory. I think it was at least two people working in concert. One with access to the email and one with access and familiarity with the code involved.
So imagine a person like “Harry” who has been working all weekend on something sitting in a lounge area on a tea break complaining to a sysadmin (who also work around the clock in many locations) about what a pile of crap the code is. Maybe something like “if the world only knew how bad this stuff was …” was uttered in frustration. I could see a person being frustrated that a huge amount of tax money and regulation was balanced on what amounts to a kluge.
The way my “Occam’s razor” works, at least two people inside were disgusted with the quality of the work and the covering up, and believed that it would be to the world’s benefit to see the data released to stand on its own merit and get questions answered once and for all.
Oh, and maybe you need to make it look like a ‘hack” in order to cover your tracks somewhat.
What I find most interesting is the attempted RC publishing. My guess would be that there is another email somewhere in the archive that is NOT part of the bundle released in the mail directory that gives a publishing credential that was used at RC. My question would be … what account attempted to publish at RC that morning?

JP
December 7, 2009 10:50 am

Nice bit of forensic work. From the very beginning I thought that the FOI file was either mistakenly put on an unprotected FTP site at UEA, or was created by an insider. There is no way the UEA is going after the whistleblower. The thought of totally exposing CRU and attendent agencies would be too much for the Alarmists to bear. Who knows what would come out. Much easier to play victim.

Editor
December 7, 2009 10:54 am

hidemydecline (09:06:23) :
> I think Harry the programmer is the source.
I don’t. I think Harry is too busy, probably not vindictive? altruistic? undedicated? enough to do it. Also, he would have done a much better job organizing the documents directory. It’s such an eclectic mishmash that at first I thought it might have been all the attachments to the Emails but I quickly found that wasn’t the case.
—–
I completely missed looking at the real mail headers in responses, how could I have missed that? OTOH, I instantly recognized that the mail file names were just unix times. While that’s a glaring flaw in the analysis, everything else looks very good to me.

crosspatch
December 7, 2009 11:00 am

And things like 1134526470.txt would not have inspired a lot of confidence 🙂

December 7, 2009 11:06 am

Was that all the “leaker” has or is a more juicy t-bit to come? Is NASA next?
Tune in tomorrow for another episode of “FOOl THE PEOPLE”.

December 7, 2009 11:07 am
crosspatch
December 7, 2009 11:08 am

Whoever accumulated the emails had access to more than just Phil Jones’ email.

$ more 0926947295.txt
From: Dave Schimel \
To: Shrikant Jagtap \
Subject: RE: CO2
Date: Mon, 17 May 1999 09:21:35 -0600 (MDT)
Cc: franci \, Benjamin Felzer \, Mike Hulme \, schimel@ucar.edu,
wigley@ucar.edu, kittel@ucar.edu, nanr@ucar.edu, Mike MacCracken \

Not sure if the above will format correctly.

Ian
December 7, 2009 11:08 am

Re: Cold Englishman.
I think you need to read the BBC weatherman’s account of this a bit more closely. As I understand it, what he received was the string of emails that related directly to the discussion of his article (which had asked where global warming had gone), not the entire 61.9 MB zip file.
It’s also not clear in his account as to whether the string arrived “out of the blue” (i.e., someone sent him a copy of the text versions extractable from the “Mail” file), or whether he received those emails from someone involved in the exchange.

Nick
December 7, 2009 11:08 am

The really interesting part is that I think the emails are a selection. Someone has done quite a considerable amount of work editing out interesting emails. That’s a lot of work.
What the CRU might be slowly realising is that the hacker/whistle blower has all the emails. If they tell a porkie to the inquiry, or the inquiry comes to an odd conclusion, more emails will be leaked.
Nick

P Gosselin
December 7, 2009 11:09 am

This will lift your spirits!
Lord Monckton goes right down the list of charlatans (he calls them crooks) starting at about 9 minutes… Jones/ Hansen / Santer/ Mann, etc.
http://www.cfact.tv/2009/12/07/lord-monckton-on-climategate-at-the-2nd-international-climate-conference/
Hat Tip: EIKE
Anthony’s work is mentioned quite prominently starting at about 11:56.

Ken Hall
December 7, 2009 11:10 am

So the “Russian FSB/KGB did it” is a tinfoil-hat conspiracy theory then!
Good to know we have conspiracy nut cases running the climate “science” upon which will be levied trillions of dollars of policy.
I suppose they will be dissing the moon landings and claiming that Elvis is alive and well and has an enormous carbon footprint too!
[sarc] No, really. Fan-bloody-tastic! [/sarc]

P Gosselin
December 7, 2009 11:10 am

Anyone watching Monckton’s presentation can only conclude the whole thing is a FRAUD.

P Gosselin
December 7, 2009 11:11 am

I also say they are crooks,
The IPCC is gang of mobsters.
Period no need to discuss the source of e-mails.
The leaker is a hero.

Yarmy
December 7, 2009 11:11 am

Leif Svalgaard (09:41:36) :
Third Party (08:52:03) :
The atmosphere contains from 4-percent water vapor in the troposphere to 40-percent near the surface.
Get the numbers straight. At 100% relative humidity at 30C [tropics] the concentration of water vapor is 30 gram/cubic meter. Considering that 1 cubic meter of air at the surface weighs 1234 gram, the water concentration can at most be 30/1234 = 2.4%.
Your [or your source’s] numbers are 10-20 times too high.

Moreover, what’s any of it got to do with leaked emails?

Stacey
December 7, 2009 11:14 am

Dear Lance
Very good post and thanks.
You said:-
“The only reasonable explanation for the archive being in this state is that the FOI Officer at the University was practising due diligence. The UEA was collecting data that couldn’t be sheltered and they created FOIA2009.zip”
Just playing devil’s advocate. Is it plausible the FOI computer was hacked and the zip file downloaded?

Dave
December 7, 2009 11:17 am

With the EPA announcement about trying to regulate facilities that put out more than 25,000 tons of CO2 per year, I wonder how many people realize what other industries that will affect. Like I just looked up Coors (who is supposed to be environmentally friendly) and they put out over 1 million tons per year:
http://www.molsoncoors.com/responsibility/data/performance
I wonder how people will feel when they’re beer/wine/soda is attacked. I think there’s a ton of industries aside from oil/energy that will be impacted that people don’t realize. This for instance talks about CO2 fire extinguishers use in a variety of industries such as steel and some marine vessels have CO2 fire extinguishers that are gigantic:
http://www.epa.gov/ozone/snap/fire/co2/co2report.html
Oh and in the latest of New Scientist they had an article that Copenhagen wouldn’t cost people too much, yet in the article trying to make it sound like no big deal made it sound like a big deal – airfare doubling in cost and electricity bills going up 15% along with many other expenses rising 1% or more. Also in back of the magazine they had a new phrase called “Ockham’s Broom” which is where scientists sweep inconvenient facts under the rug, yet they made no mention of AGW when introducing the word.

crosspatch
December 7, 2009 11:20 am

“Emails are not held in clear form on a server”
Uhm, yes they are in the majority of cases. Most emails are also transmitted between servers in clear form. SMTP is NOT a secure communications format. NEVER put financial information, for example, in an email unless YOU encrypt the email yourself.
Some email server software will store the mail body in a binary format but by far the majority of SMTP mail transmission is done “in the clear” with mail spools that are also plain text.
Also, most large organizations these days never delete email until a long period of time (several years) has elapsed. This is done to protect themselves from litigation or to allow them to litigate. Your work email belongs to your employer and there may be several other employees that have access to it.
Consider every single email you send/receive though your work account to be visible by others.

Jim Carson
December 7, 2009 11:25 am

I can’t agree that this is a good analysis. The fact that Lance Levsen claims,

the average amount of emails between each released email is 402,839.

tells me that this analysis was hastily made and not even scanned for credulity before publishing.
Anthony, I appreciate the speed of breaking news and the difficulty of staying ahead of it, but this warranted a smell test, or better yet, a call for volunteers to review it.
At minimum, you should introduce these things with a disclaimer that you haven’t verified the results. This blog is starting to look like a bandwagon.

P Gosselin
December 7, 2009 11:28 am
dearieme
December 7, 2009 11:30 am

Jones dunnit. That’s my guess. He’s undergone a quasi-religious conversion to truth-telling and this was his chosen route to distance himself from his corrupt past.
No more fudging the data! No more hiding the decline!
He denies it? What more proof could you want?

Michael
December 7, 2009 11:30 am

[snip – sorry, but schoolbuses are wayyy off topic of this thread]

John Galt
December 7, 2009 11:30 am

BBC: UK Climate Code May Be Scrapped
One of the first victims of the Climategate scandal may by the very computer code that is supposed to track global temperature records.
According to a report by the BBC, a computer software expert says the source code used by the Climatic Research Unit at the University of East Anglia is “below the standard in any commercial software.” Thousands of e-mails and documents were stolen from the CRU, and published on the Internet.
[more] http://www.newsmax.com/insidecover/climategate_code_computer/2009/12/05/294861.html
Original Source: http://news.bbc.co.uk/2/hi/programmes/newsnight/8395514.stm

Michael
December 7, 2009 11:32 am

Moderator, please change “They told them” to “I told them”, thanks.

JustPassing
December 7, 2009 11:34 am

Am I now seeing right?
Google now produces 280,000,000 results for climategate.
WOW

PhilW
December 7, 2009 11:37 am

That Monckton video should be nailed to the top of the WUWT posts

Carrick
December 7, 2009 11:41 am

Followup on crosspatch’s comment:
Here is a simple Unix program for converting UNIX epoch time into “ctime”. The hours will be wrong unless you are in the same time zone as the one in which the times were converted into UNIX epoch time.
#include <stdio.h>
#include <time.h>
int main()
{
time_t tm;
while (scanf(“%lu”, &tm) == 1) { printf(“%s”, ctime(&tm)); }
return 0;
}

Julian in Wales
December 7, 2009 11:41 am

So it looks like an inside job?
I seem to remember that the person who released this information claimed it was only part of what S/he had collected?
If it really was a whistle blower it reasonable to expect that S/he was in a position to collect other valuable files? Lets hope for more instalments!

P Gosselin
December 7, 2009 11:44 am

I’m getting 28.9 million on Google.

Julian in Wales
December 7, 2009 11:48 am

I agee with Nick, if I nwere the whistleblower I would keep some emails in reserve and then let them out after the culprits had been lying their heads off for a while.

AnonyMoose
December 7, 2009 11:48 am

Carrick – There might be a way to convert the timestamps already in your system. Try: date ‘–date=@0826209667’

Invariant
December 7, 2009 11:48 am

John Galt (11:30:35) :BBC: UK Climate Code May Be Scrapped. According to a report by the BBC, a computer software expert says the source code used by the Climatic Research Unit at the University of East Anglia is “below the standard in any commercial software.”
I think it’s an amusing red herring for to scrap the source because it is “below the standard in any commercial software.” Surely this is exactly the mediocre quality kind of source code found in most academic institutions…

Jeremy
December 7, 2009 11:51 am

Dave (11:17:49) : With the EPA announcement about trying to regulate facilities that put out more than 25,000 tons of CO2 per year, I wonder how many people realize what other industries that will affect.
Dave,
It is much more serious than small beer and soda. Energy is FOOD. We get fertilizers by using energy to make nitrogen fertilizers. These increase crop yields enormously – they also allow foods to be grown quickly in climates that natural might not sustain food production due to a short season.
If it gets colder (as it might) and, at the same time, we drive up all the input costs of food production (through taxation) then we are in for a very very rough ride globally. A lot of people will starve. Inflation will be rampant.

ChrisM
December 7, 2009 11:52 am

Sent:
Please remember King Canute was a Dane and he could not stop the tide as you will not stop natural climate change

snopercod
December 7, 2009 11:52 am

Great work, Lance Levsen.

P Gosselin
December 7, 2009 11:52 am

Let’s all join together for a moment of prayer, via internet,
to pray that Joe is right.
http://www.accuweather.com/world-bastardi-europe-blog.asp?partner=accuweather

December 7, 2009 11:53 am

You know, one of those graphs looks like a hockey stick….. That clinches it for me. It WAS a leaker. And we have consensus, so it’s time to move on!!!

P Gosselin
December 7, 2009 11:56 am

Just when you thought it can’t get any wackier.
http://www.spectator.co.uk/essays/all/5592803/the-cias-global-cooling-files.thtml
There was a consensus back then too!
h/t Ice Cap

crosspatch
December 7, 2009 11:59 am

“The hours will be wrong unless you are in the same time zone as the one in which the times were converted into UNIX epoch time.”
Yes, and I found it interesting that the time zone the users were in (GMT +1:00 for most) is several hours different from the time stamp on the files. It is almost as if the files are being archived in Kiev or something.

David Porter
December 7, 2009 12:10 pm

(10:30:19) : Leif Svalgaard
Your comment that “CO2 also freezes out” I believe is impossible above a temperature of around -120 degree centigrade. “Phil” on a previous and earlier thread went into some detail on this issue which ended, I think, with Steve Goddard, a regular poster here, deciding thereafter not to make any further posts.
If I have this wrong I would appreciate your further comment

Gary Hladik
December 7, 2009 12:13 pm

wws (08:52:37) : “The weekend cover story claiming that “The Russian KGB did this to discredit Copenhagen!!!” was a hoot.”
Didn’t Nixon’s lawyers suggest that a “sinister force” had caused that famous 18-minute tape gap? Perhaps it’s still at large… 🙂
Leif Svalgaard (10:30:19) : “About Vostok: when it is really cold there the CO2 also freezes out…”
I thought we had a whole thread here awhile ago that concluded CO2 can’t freeze naturally on Earth, as it does on Mars.
Or did I just fall for a tongue-in-cheek comment? 🙂

Speed
December 7, 2009 12:14 pm

Lance:
Re: Nick (11:08:45)
Manually selecting the subject e-mails (1073 of them) from a universe of hundreds of thousands (a guess) would have taken a lot of time and effort. Assuming that some sort of automated search was used, do you have any way of looking for words that are common to all the e-mails?

Chris Schoneveld
December 7, 2009 12:15 pm

JustPassing (11:34:22) :
“Am I now seeing right? Google now produces 280,000,000 results for climategate.”
No, you should put climategate between quotes, hence “climategate” and then you “only” get 2,770,000, whilst “climate-gate” has only 510,000 and “climate gate” 534,000

tty
December 7, 2009 12:18 pm

Leif Svalgaard (10:30:19) :
“About Vostok: when it is really cold there the CO2 also freezes out”
No, it doesn’t. It would if the atmosphere was all CO2, but at the low partial pressure CO2 is at, it has to be a lot colder before it starts to freeze out.
It’s the same with H2O, at a partial pressure of one atmosphere it starts condensing at 373 K, but it doesn’t rain everywhere on Earth where the temperature is below 373 K, because the partial pressure of water is always much lower than one atmosphere.

Richard
December 7, 2009 12:20 pm

Leif Svalgaard (09:41:36) :
Third Party (08:52:03) : The atmosphere contains from 4-percent water vapor in the troposphere to 40-percent near the surface.
Get the numbers straight. At 100% relative humidity at 30C [tropics] the concentration of water vapor is 30 gram/cubic meter. Considering that 1 cubic meter of air at the surface weighs 1234 gram, the water concentration can at most be 30/1234 = 2.4%.
Your [or your source’s] numbers are 10-20 times too high.

Maybe so. But water accounts for about 60% of the greenhouse effect, while CO2 accounts for 26%.
What bugs me is the claim of atmospheric scientists that “the science is settled” and they repeat the incantations – fundamental physics, greenhouse gas, black body radiation as though fundamental physics proves or supports the assumptions of their models, or the reconstructions of Mann or Briffa.
More heat, more water vapour, more greenhouse heating, positive feedback – QED. Dangerous run-away warming, near the tipping point. Hang on – what about clouds?
What about the lessons from nature – our climatic history? We have always cooled after warming showing our climatic system is a negative feedback system.

December 7, 2009 12:22 pm

Thank you for this analysis!
I have been sitting fuming here in the UK at all this Russian hacker nonsense – and especially the IPCC rep suggesting the Russians wanted to sabotage Copenhagen – it is laughable.
The ‘Russians’ not so long ago had a Vice Chair of IPCC – Yuri Izrael, a planetary ecologist, who famously said in 2007 that global warming was a load of hype and that natural cycles were driving the system. He then received a friendly visit from Vladimir Putin who reminded him that Russia had signed the Kyoto Protocol (easy to forget, since they refused for many years and their Academy of Sciences agreed with Izrael). ‘Yuri’ he probably said, ‘What does the cause matter. We get billions of dollars for the carbon credits – and I know its hard for you, a climate scientist, to understand – I mean, we get this for the carbon dioxide we might have produced if our economy was functional, and so it makes no difference to carbon emissions….but just go along with me for a moment. Suppose it is warming…wouldn’t it make sense to cool it down with a little geo-engineering? Be a good academician and study it for me. Here’s a few million roubles for your new eco-geology institute.’
Prof Izrael reported his results in the last but one issue of Russian Meteorology and Hydrology (I read the abstracts only) – and he suggests using sulphur dioxide dispersed at high altitude.
I don’t think the Russian government and the FSB would want to sabotage Copenhagen – there’s loads of money for them in the pot. Including for Professor Izrael.
And I agree with the Optimist – the UEA material was compiled not for release to FOI but to get rid of it – it is incriminating, for goodness sake! Furthermore, it strikes at the heart of academic integrity – in black and white : massaging of data to fit their expectations, collusion with other independent data compilers to arrive at the same message, making up data, manipulating data to remove the ‘blip’ (read evidence of a cycle that they cannot explain), pressurising the peer-review process, seeking to avoid FOI requests, not sharing data with critics (aka sceptics – when all scientists should be sceptical!), and then seeking to subvert the FOI by dumping data (and telling others to delete emails!).
An amusing video of a US new presenter calling it East Angila (with a long ee) finally made sense – there are TWO UEAs. There is the one I know (with Tim O’Riordan, respected professor of environmental sciences) and Mike Hulme (former head of the Tyndall Centre for Climate Studies) and the labs I have been invited to and shared my own research groups work on climate related computer simulation (back in 2002). These are real scientists. They uphold the virtues of science – truth, sharing of data and methods, peer-review and welcoming of criticism. Then there is the CRU that exists in a cyber-reality where these virtues have long been forgotten – and which we now know as East Angila.

Robuk
December 7, 2009 12:23 pm

crosspatch (10:48:32) :
” Optimist (10:13:02) : ”
While it seems silly to attempt to guess motivations, your thought occurred to me. If I wanted to make sure something WASN’T found in the course of an FOIA investigation, I might assemble a folder of stuff I wanted deleted in a hurry if I needed to.
That`s exactly my thoughts, like Mann`s censored file, hide the evidence.

lance
December 7, 2009 12:24 pm

Thanks all with regards to the filenames. Sigh….(id10t error; PBK&S, lart stick aisle one, please)
I’ve updated the original post on SmallDeadAnimals.
This new information actually reinforces the point that the emails had to have been copied at the mail gateway and not on the departmental servers. Even with the best of NTP time synchronization, you’re going to have some drift. Secondly, if the emails were CC’ed to a user on the hypothetical ueams.uea.ac.uk mailserver and both machines got the email at the same time, which one would write out the file? If you’re outputting files based on a timestamp, you’re only using one machine.
Given that, there is only one place where one machine handles all of the email.
I agree that FOIA2009.zip could have been ‘found’ somewhere. I would say however, that it’s lower on the probability scale because the FOI Officer can’t just leave these things around.
IMO, for it to have been ‘hacked’ and especially a targeted hack involves way too many variables.
Cheers,
lance

David Porter
December 7, 2009 12:27 pm

Gary Hladik (12:13:01) :
Regarding Leif’s 10:30:19 comment, I don’t think you were the only one to be sent down a blind alley.

tallbloke
December 7, 2009 12:28 pm

It makes me laugh when I see media pieces blaming the evil hackers and complaining about them giving ammunition to the skeptics.
“Roaches blame light switch operator in warehouse food theft scandal”

December 7, 2009 12:28 pm

David Porter (12:10:25) :
Your comment that “CO2 also freezes out” I believe is impossible above a temperature of around -120 degree centigrade.
Gary Hladik (12:13:01) :
I thought we had a whole thread here awhile ago that concluded CO2 can’t freeze naturally on Earth, as it does on Mars.
I missed that thread. Perhaps I was just demonstrating some ignorance on this, not taking into account the low pressure of CO2 [especially at the altitude of Vostok]. One learns something new every day.

Bob H.
December 7, 2009 12:29 pm

Here’s a more-or-less official russian point of view:
http://english.pravda.ru/science/earth/30-11-2009/110832-climategate-0

rbateman
December 7, 2009 12:31 pm

Nick (11:08:45) :
Who would spend that much time reading that much email to compile the collection?
Someone who has been watching over the years.
Someone who is watching every response, looking to see who is going to go quietly and who is not.
The Email/Data Sword of Damocles.

tallbloke
December 7, 2009 12:33 pm

It makes me laugh when I see media pieces quoting AGW ‘scientists’ blaming the evil hackers and complaining about them giving ammunition to the skeptics.
“Roaches blame light switch operator in warehouse food theft scandal”

Richard
December 7, 2009 12:34 pm

Chris Schoneveld (12:15:00) :
JustPassing (11:34:22) :
“Am I now seeing right? Google now produces 280,000,000 results for climategate.”
No, you should put climategate between quotes, hence “climategate” and then you “only” get 2,770,000, whilst “climate-gate” has only 510,000 and “climate gate” 534,000

Since climategate is only one word, searching for climategate or “climategate” should have no difference, unless it is erroneously searching for ” “climategate” ” – in other words climategate withing inverted comma’s only.
Google has been very erratic with climategate. I got 310,000,000 hits a couple of days ago. Just now 31,000,000 (Results 1 – 10 of about 31,200,000 for climategate. (0.13 seconds)
Climate-gate has given me 30,900,000, and climate gate 12,900,000

December 7, 2009 12:34 pm

Chris Schoneveld (12:15:00) :
I’ve only ever searched for climategate not “climategate” and have had a pretty much upward trend from about 50k to 31.9m just now on Google.
I’ve only had the strange x10 factor once or twice in the last couple of days.

Richard M
December 7, 2009 12:35 pm

Could it be that the FOI team asked for a list of scientists that were involved in the particular area that the FOI pertained to. A search was then made of the email archives for these scientists. The emails were then weeded of personal items and placed in the FOIA folder.
If this approach was used it would explain quite a bit and fits in with a idea that this was the real FOI that would have been released had it been approved. Could you imagine the look on Steve M’s face had he received all of this information?
My own feeling is one of the members of the FOI team, after seeing what was going on, is likely to be the whistle blower.

WAG
December 7, 2009 12:38 pm

How does this “analysis” square with the fact that several other climate research institutions suffered attempted break-ins from the outside?
http://www.nationalpost.com/news/canada/story.html?id=2300282

Pete
December 7, 2009 12:38 pm

Interesting theory but the same as AGW, not yet proven 😉
Interesting as well to see you closely reproduce the horrific hockey stick graph! Gave me a smile!

PaulH
December 7, 2009 12:38 pm

Good analysis and well presented, but I had to wear my propeller-beanie while reading it. 🙂

acementhead
December 7, 2009 12:44 pm

Leif Svalgaard (10:30:19) :
Great to have you back, but
About Vostok: when it is really cold there the CO2 also freezes out…”
not according to to any phase diagram that I can find(well my conclusion from examining them). Coldest ever recorded temp at Vostok ~184 K which would give a CO2 vapour pressure of ~ 0.3 bar.
http://encyclopedia.airliquide.com/images_encyclopedie/VaporPressureGraph/Carbon_dioxide_Vapor_Pressure.GIF
http://www2.warwick.ac.uk/fac/sci/eng/cmd/research/civil/supercritical/phase_changes_of_co2.png

Leon Brozyna
December 7, 2009 12:45 pm

Whether the data came out due to a leak, a hack, or a little green pixie is really beside the point. The data is damning to the AGW political movement. How it got out is a frivolous debating point they’ll use to try to divert attention from the seriousness of what was revealed.
FWIW, I think it was an inside job just from the selective nature of the emails.

crosspatch
December 7, 2009 12:48 pm

“But water accounts for about 60% of the greenhouse effect, while CO2 accounts for 26%. ”
I find that difficult to believe. CO2 is 0.038% of the atmosphere. The spectrum across which it absorbs IR is much narrower than H2O. In order for CO2 to provide half the total greenhouse warming of Earth, there would have to be more CO2 than water vapor in the atmosphere. CO2 is a much less efficient greenhouse gas than is water vapor.

crosspatch
December 7, 2009 12:50 pm

I meant in order for CO2 to provide half the warming of water vapor, there would need to be more CO2 than water vapor as it is less than half as efficient water vapor in absorbing IR.

JustPassing
December 7, 2009 12:52 pm

I’ve just watched Dr Myles Allen being interviewed on the BBC news, head of the Climate Dynamics group at Oxford, comment on the Medieval warm period as “it might or might not have happened”.
Isn’t he one of the guys the email correspondence was forwarded to, where they discuss below
” but meanwhile it might be appropriate for the Met Office to have a say about this, I might ask Richard Black (BBC) what’s up here?”
I’m still getting 280,000,000 on google search for climategate, maybe China relaxed its internet vetting a little 🙂

A Lovell
December 7, 2009 12:56 pm

Forget ‘hacked’, or even ‘leaked’. I say they were LIBERATED!

Chris S
December 7, 2009 12:56 pm

A case of over analysis I think.
Jones compiled this lot himself as a house keeping exercise resulting from a FOI request.
He intended to conceal them not reveal them should he be forced to comply with FOI.
The folders contain some files that could be embarrassing, but not subject to the FOI request, eg. RulesOfTheGame.pdf.
The directory structures are just what you would expect were you to have a clean-up of sensitive data from more than one workstation to a network drive. The e-mail names are just a time stamp.
Being a network directory, it would have been accessible to many users dependent on network/share policies.
Academic users are rarely aware of security requirements needed to prevent access to networked drives from inside or outside the university.
My guess is another UEA staff member came across the info and copied to a flash drive.
Harder to trace than an upload/download.
Just my tuppence worth;)

Rob R
December 7, 2009 1:01 pm

The whistleblower must have felt rather like Frodo in Mordor.

December 7, 2009 1:02 pm

JustPassing (11:34:22) :
Am I now seeing right?
Google now produces 280,000,000 results for climategate.
WOW

I get precisely 33,500,000
“Results 1 – 10 of about 33,500,000 for climategate. (0.08 seconds) ”
How odd we get these precisely round numbers and still totally different values?

crosspatch
December 7, 2009 1:07 pm

For example Here is a plot of transmittance of two atmospheres. One is pure CO2, the other is a mix of H2O and CO2 at a 5:1 mixture (5x more H2O)
H20 provides about 95% of the greenhouse warming of Earth according to my reading. Assuming for the moment that CO2 accounts for all the remaining 5% (which it doesn’t) and that anthropogenic sources increase the amount of CO2 by about 3.2% of the total amount of CO2, then humans are at a maximum responsible for 0.16% (3.2% of 5%) of the total available greenhouse gas in the atmosphere.

December 7, 2009 1:08 pm

A new blog from Germany with texts in English, one of the blogger is [b]v.Storch[/b]
http://klimazwiebel.blogspot.com/

Robinson
December 7, 2009 1:08 pm

the average amount of emails between each released email is 402,839.
tells me that this analysis was hastily made and not even scanned for credulity before publishing.

Actually, this isn’t unusual. If you’ve got several thousand people all using the mail server (students as well, which is likely), there’s going to be a huge amount of traffic. UEA has something like 15,000 students (undergrads and post-grads), plus staff. You could easily turn over 400k mails in a few weeks.

Carrick
December 7, 2009 1:10 pm

AnonyMouse, on a Mac the command is:
date -j -f “%s” 1140021977
Wed Feb 15 10:46:17 CST 2006
This probably won’t work on other systems though.
The code I gave however should work on mac, windows (with cygwin) and linux.

crosspatch
December 7, 2009 1:14 pm

In other words, CO2 adds a couple of little tiny bumps to the transmittance of the atmosphere at a few specific wavelengths but by far the overall absorbing of IR is done by water vapor. It is like taking a window, pulling a sheer across (CO2) and then closing the drapes (water vapor). Sure, the CO2 ads a little bit but the total is completely swamped by the difference made by the drapes. Having the sheer curtain open or closed makes little difference with the abundance of water vapor.
At the poles, CO2 will play a greater role relative to water vapor but water vapor will still dominate.

December 7, 2009 1:24 pm

P Gosselin (11:52:54) :
Let’s all join together for a moment of prayer, via internet,
to pray that Joe is right.
http://www.accuweather.com/world-bastardi-europe-blog.asp?partner=accuweather

Right now, the detailed long term weather forecast for Copenhagen predicts snow and freezing temperatures from 15/12
http://www.yr.no/place/Denmark/Capital/Copenhagen/detailed_long.html
🙂

PaulH
December 7, 2009 1:31 pm

The author updated his analysis making note of the epoch timestamps issue. The updated article with corrections at the original web site here:
http://www.smalldeadanimals.com/FOIA_Leaked/

December 7, 2009 1:33 pm

crosspatch (11:59:25) :
“The hours will be wrong unless you are in the same time zone as the one in which the times were converted into UNIX epoch time.”
Yes, and I found it interesting that the time zone the users were in (GMT +1:00 for most) is several hours different from the time stamp on the files. It is almost as if the files are being archived in Kiev or something.

GMT+1:00 is central European time (Oslo, Copenhagen, Paris, Madrid…)
One of the RC people in the emails is a Norwegian from Oslo (Rasmus Benestad).

pat
December 7, 2009 1:34 pm

the bbc’s involvement is a STORY in itself and an independent investigation is warranted.
bbc’s weatherman paul hudson was obviously not authenticating a few emails when he spoke of receiving the ‘chain of emails’ on october 12 in response to this article:
9 Oct: What happened to global warming?
http://news.bbc.co.uk/2/hi/science/nature/8299079.stm
on 23rd Nov he begins by saying he’s too busy (?) to write anything about the leaked emails/docs, but says:
‘Climategate’ – CRU hacked into and its implications
I was forwarded the chain of e-mails on the 12th October, which are comments from some of the worlds leading climate scientists written as a direct result of my article ‘whatever happened to global warming’. The e-mails released on the internet as a result of CRU being hacked into are identical to the ones I was forwarded and read at the time and so, as far as l can see, they are authentic.
http://www.bbc.co.uk/blogs/paulhudson/2009/11/climategate-cru-hacked-into-an.shtml
he was then gagged from talking to media :
(hudson is a well-known weatherman in the hull region)
Hull Daily News: BBC weatherman in global warming row
When contacted by the Mail, the weatherman said he was not allowed to comment and asked us to speak to the BBC press office.
http://www.thisishullandeastriding.co.uk/news/BBC-weatherman-global-warming-row/article-1553969-detail/article.html
plus hudson dropped out of the debate on his blog (too busy) and handed the story over to harrabin, etc:
24 Nov: ‘Climategate’ – What next?
I first became aware of the news late last week, but because of my weather and filming commitments couldn’t deal with it myself and so passed the news on to some of my colleagues in the BBC’s environment and science team, including our environment analyst Roger Harrabin who wrote about it on saturday morning, and Newsnight, who covered the story last night.
http://www.bbc.co.uk/blogs/paulhudson/2009/11/climategate-what-next.shtml
did someone at UEA/CRU send hudson a pile of stuff following his ‘where’s the global warming’ piece to show CRU knew there was no global warming and they were in panic mode?
the emails go beyond the 12 october date, (not sure if the documents do too, which someone who has checked could tell us) which could mean the ‘whistleblower’ added more emails after they realised BBC would not report on the stuff Hudson received.
however, on 12 october u get a chain of emails re hudson’s ‘where’s the global warming’ piece, which are important to read again in the context of bbc’s involvement:
East anglia emails – 1255352257.txt
From: Kevin Trenberth
To: Michael Mann
Subject: Re: BBC U-turn on climate
Date: Mon, 12 Oct 2009 08:57:37 -0600
Hi all
Well I have my own article on where the heck is global warming? We are asking that here in
Boulder where we have broken records the past two days for the coldest days on record…
The fact is that we can’t account for the lack of warming at the moment and it is a travesty that we can’t….
Michael Mann wrote:
extremely disappointing to see something like this appear on BBC. its particularly odd, since climate is usually Richard Black’s beat at BBC (and he does a great job). from what I can tell, this guy was formerly a weather person at the Met Office.
We may do something about this on RealClimate, but meanwhile it might be appropriate for the Met Office to have a say about this, I might ask Richard Black what’s up here?….
From: “Narasimha D. Rao”
To: “Stephen H Schneider”
Sent: Sunday, October 11, 2009 10:25:53 AM GMT -08:00 US/Canada Pacific
Subject: BBC U-turn on climate
Steve,
You may be aware of this already. Paul Hudson, BBC’s reporter on climate change, on Friday wrote that there’s been no warming since 1998, and that pacific oscillations will force cooling for the next 20-30 years. It is not outrageously biased in presentation as are other skeptics’ views…
BBC has significant influence on public opinion outside the US.
Do you think this merits an op-ed response in the BBC from a scientist?
Narasimha
http://www.eastangliaemails.com/emails.php?eid=1048&filename=1255352257.txt
another possibility: did hudson receive the material by mistake? was it supposed to go to black or harrabin, for example?
could the programmer look into whether there was any interference in the email file after 12 october which would suggest additional material was added.
i am not technically astute, so any comments on the above would be much appreciated.

M12
December 7, 2009 1:47 pm

Has anyone suggested that the person who may have collected all this information, named the file, and even been part of the leak could be the FOI officer himself?

Bryn
December 7, 2009 1:50 pm

You are all wrong. The dingo did it! [Oz joke]

John Galt
December 7, 2009 1:57 pm

Invariant (11:48:21) :
John Galt (11:30:35) :BBC: UK Climate Code May Be Scrapped. According to a report by the BBC, a computer software expert says the source code used by the Climatic Research Unit at the University of East Anglia is “below the standard in any commercial software.”
I think it’s an amusing red herring for to scrap the source because it is “below the standard in any commercial software.” Surely this is exactly the mediocre quality kind of source code found in most academic institutions…

Unfortunately, the quality is not far off most of what is developed for most large businesses.

Anonymous Coward
December 7, 2009 2:00 pm

GMT+1 is also British Summer time, in effect from March to October. From the example above
1123622471.txt
1123622471 is unix time August 9, 2005 21:21:11 GMT
The date/time stamp on that email is:
Date: Tue Aug 9 17:21:11 2005
This is four hours behind GMT, or (USA) Eastern Daylight Time. It’s the wrong direction for Kiev.
I think Harry did it.

Stacey
December 7, 2009 2:01 pm

Schonveld
If you put climategate in quotes you will get climategate in quotes?
Climategate is producing 31.5million?
Am Imissing something?

acementhead
December 7, 2009 2:05 pm

acementhead (12:44:18) : Your comment is awaiting moderation
OK I won’t waste any more time trying to post here.
[Reply: you begrudge an unpaid volunteer for taking off for less than 90 minutes before approving a comment, out of a 10 – 12 hour day of moderating 300 – 500+ posts, every day, 7 days a week? This site usually posts comments promptly, but sometimes there’s an hour or two delay; climategate and Copenhagen have really increased the traffic. There’s a way to show some appreciation: hit the tip jar. ~dbs, mod.]

John Galt
December 7, 2009 2:06 pm

Carsten Arnholm, Norway (13:02:20) :
JustPassing (11:34:22) :
Am I now seeing right?
Google now produces 280,000,000 results for climategate.
WOW
I get precisely 33,500,000
“Results 1 – 10 of about 33,500,000 for climategate. (0.08 seconds) ”
How odd we get these precisely round numbers and still totally different values?

Everyone should look at their Goolge search settings. Mine is set to filter (because I’m at the office) and only to return English pages (since that’s the only language I am literate in).
Your settings likely affect the number of results.

Dave.
December 7, 2009 2:09 pm

I´d like to clear up one thing. Many people are suggesting that the BBC
had the files before the internet release.
What really happened was that a reporter had written an article
suggesting warming had slowed. (Approx a month before the hack.)
He was sent some emails by CRU decrying his article.
When the hack occurred, many people screamed ¨hoax¨.
However, the reporter confirmed at least some emails
were genuine as these were the ones sent to him
regarding his article, again, approx a month before
the leak.
I too favour the insider theory.
What are odds of this folder,which is full of juicy stuff, accidentally
being uploaded. Especially just before Copenhagen.
I´d look towards the ¨Harry¨ file.Notice the frustration in
the comments. Also notice how it finishes abruptly.
Perhaps the commentor had seen enough, got sacked,
given up etc.

Richard Sharpe
December 7, 2009 2:14 pm

Robinson says:

the average amount of emails between each released email is 402,839.
tells me that this analysis was hastily made and not even scanned for credulity before publishing.

Actually, this isn’t unusual. If you’ve got several thousand people all using the mail server (students as well, which is likely), there’s going to be a huge amount of traffic. UEA has something like 15,000 students (undergrads and post-grads), plus staff. You could easily turn over 400k mails in a few weeks.

There are 86,400 seconds in a day.
Do you want to rethink your remark?
(Since 402,839 is actually the average number of seconds between emails, which is about 4.6 days.)

crosspatch
December 7, 2009 2:19 pm

” Carsten Arnholm, Norway (13:33:15) : ”
I am aware of that but the timestamps generated are consistently 5 hours ahead of GMT no matter what the TZ of the header is.
What is 5 hours ahead of GMT? Russia time zone 4 in winter and time zone 3 in summer.

December 7, 2009 2:20 pm

IPCC boss says of Climategate:the only debate is who is behind [the leak]…
In remarks to AFP, Pachauri said he did not believe that the affair would sway the opinion of people who had carefully weighed the evidence for or against climate change.
“I think people are informed enough to realise that the Fourth Assessment Report is completely objective, totally unbiased and solid in its scientific assessment.
“The only debate is who is behind it, I think we should catch the culprits.”

Mapou
December 7, 2009 2:21 pm

“On a somewhat related note, Google is still hiding the autosuggest!”
Google does evil. Google is evil.

Håkan B
December 7, 2009 2:27 pm

There are several online tools for timestamp to date and vice versa conversion, one is here:
http://ibboard.co.uk/timestamp.php

Chris S
December 7, 2009 2:30 pm

Pat,
I suspect Hudson received an e-mail from a BBC colleague (Black?), that included the chain of text Black had received by e-mail from a UEA scientist . That message, complaining about the BBC’s lapse in editorial standards, contained the comments by Trenberth, Mann and others.
As you can see from most of the e-mails, the chain of text is copied to the next recipient.
When Climategate broke, Hudson misconstrued the significance of the e-mail and wrote a poor and misleading article/blog post. He then disappeared to avoid embarrassment;)

December 7, 2009 2:30 pm

acementhead (12:44:18) :
not according to to any phase diagram that I can find(well my conclusion from examining them). Coldest ever recorded temp at Vostok ~184 K which would give a CO2 vapour pressure of ~ 0.3 bar.

Yeah, I forgot about the low partial pressure bit. But perhaps if it became REALLY, REALLY cold 🙂

crosspatch
December 7, 2009 2:37 pm

The time stamps were not generated by the server. Someone parsed the date headers of the emails, generated a timestamp and renamed the files to be a function of the timestamp. This is because both exim and sendmail name mail files as a function of the message-id. If you have the original filename, you can go back through the logs and determine exactly which machine the files were originally removed from. So you “mask” the original file name by going through the files and renaming them according to the date header. In this case it works because there apparently weren’t two emails in the same second.
Both the filename and the file creation date were change. The creation dates of all the files was set the same which is a date earlier than the last email. Notice the file dates (using ls -1) is in 2008 though some of the emails are from 2009. The file names and dates were changed after the fact to remove a trail.

December 7, 2009 2:48 pm

Invariant (11:48:21) :
John Galt (11:30:35) :BBC: UK Climate Code May Be Scrapped. According to a report by the BBC, a computer software expert says the source code used by the Climatic Research Unit at the University of East Anglia is “below the standard in any commercial software.”

I think it’s an amusing red herring for to scrap the source because it is “below the standard in any commercial software.” Surely this is exactly the mediocre quality kind of source code found in most academic institutions…
————————————————————
So does that mean that crappy, improper mediocre code that doesn’t function properly IS GOOD ENOUGH for academia????

VG
December 7, 2009 2:52 pm

The cat is out of the bag, whatever they try, AGW is dead.. so skeptics don’t fret about current news stories etc. The cru story is self replicating and won’t go away. Even MSM is slowly becoming embedded in it. In the end people will notice that the weather/climate ain’t changing…. In one year, this won’t be an issue it will have changed to environmental stuff but not global warming which is OK

crosspatch
December 7, 2009 2:59 pm

And to be clear about what I am getting at … it appears that someone modified their computer clock to make it 4 or 5 hours later than it was. This is because if I am in 10 different timezones and I parse a date/time string that gives its offset from GMT (-0000 in this case) I will get exactly the same date string output because unix time is seconds elapsed since 0000 01-01-1970 UTC. So if I have 10 computers in 10 different timezones, as long as the timezone is set right and the internal clock is correct, I will get exactly the same timestamp when I parse the date header. My location shouldn’t matter. Now if I intentionally change my computer’s timezone but don’t adjust my clock or if I adjust my clock but don’t adjust the computer’s timezone, the generated datestamp will be off, as is what appears to have happened here.
Let me go through this: Assume an email with a Date: header of February 13, 2009 3:31:30 -0000
If I convert that to unix time on my computer, I will get 1234567890 no matter what my computer’s time is set to. Now if I leave off the -0000 and only parse the date/time, it is going to assume MY timezone and offset the result to obtain GMT. If the command generating the timestamp took the offset into account, there would be no difference between the timestamp and the date. There is a 5 hour difference between the GMT time in the header and the generated date stamp. This means the offset in the email header was not considered and either the computer generating it was … oh, holy crap .. the computer was on the East coast of the US in GMT -0500 and added 5 hours to get what it thought was GMT.
It wasn’t 5 hours EAST of GMT, it was 5 hours WEST of GMT. The computer added 5 hours assuming the date/time was local.

Brazil Tony
December 7, 2009 2:59 pm

Maybe the hacker just entered the system and took a FOIA file that had been prepared. The conclusion that it was a “person with scruples” doesn’t seem to follow from the technical analysis.

December 7, 2009 3:07 pm

The profs in the emails talk about using the FTP server as an intranet. I think it was put there for Mann et al to review, and someone at UEA posted the link knowing it was there.

Derryman
December 7, 2009 3:13 pm

It was obvious when the e-mails were confirmed as genuine that this was not a hack. It would have taken far too long to compile the information and is totally at odds with the rather puerile behavior of normal hackers. Also this was not a file prepared in response to an FOI request. Such requests are by definition very targeted and would never cover such a long time period or such a broad range of topic. I suspect that the FOI name on the directory was a neat piece of misdirecton while the information was being gathered together over a long period of time. No question this was a deliberate inside job, though the mole’s motivation may not have been as pure as is widely assumed here.
Finally do not worry about speculating on the identity of the leaker as I am sure better and more sinister brains than ours ( mostly from Cheltenham) will have been all over the UEA servers.

crosspatch
December 7, 2009 3:24 pm

“this was not a hack”
agreed, but steps apparently WERE taken to cover the tracks of whoever obtained the files. The file names of the emails and the file dates were changed.

xyzlatin
December 7, 2009 3:32 pm

Here is a copy of an email I sent Lance:-
G-Day Lance,
I have just finished reading your article on the web site “Smalldeadanimals” and it seemed to me that you like solving puzzles.
On thing I discovered after reading the emails was that there also was a directory FOIA located on Ben Santers computer in the USA. Phil Jones laughed about having a directory of that name.
This was contained in file 1233249393.txt .
Anyway here is another part of the puzzle. Below is a posting I put up on an Australian blog site, Andrew Bolt, days ago.
Andrew,
I posted here in your blog about a week ago, that the whistle blower had left clues to his or her identity .
Check out the movie “Peer Review 1945” posted on YouTube.
It was posted on YouTube on the 19th November. Time and dates are important here.
I myself was on “The Air Vent” web site reading the posts on the “Open Letter On Climate Legislation” when the link from FOIA to http://ftp.tomcity.ru/incoming/free/FOI2009.zip was up, message 10. That link was posted on the 17 November at 9:57 PM.
When I tried the link it was dead. But later FOIA DATA MIRROR on the 20 November at 12:59 AM posted a new link to http://www.megaupload.com/?d=XD050VKY This link still works.
Now Andrew, you tell me, how did the poster of the movie know what was in the emails before the files were publicly available on the net ?
But there is more. Have a look at the name of the poster at YouTube, “indusieumgresium” made up of two Latin words. Indusieum and gresium” both words have references to botany and biology. Botany, trees, tree rings ??
I have found many meanings but come back to some which suggest a grey protective layer. Also a layer to protect the sorus eg “In fungi and lichens, the sorus is surrounded by an external layer” ( indusieumgresium ).
(George Soros,, another link ??)
Also the poster of “Peer Review 1945” says he / she is 39 years old.
Along with the sub titles in the movie that directly refer to what we now know as “Climategate” I would like to know what the original movie was about and its history. What the dialogue is in the German, and why there is the emphasis on the girl crying.
So go to it.
Chris in Hervey Bay
The clues seem to be in the sub titles. “Hans”, 3 reviewers, who wanted more research done ? etc.
I’ll leave it to you to decide if this clue is worthwhile pursuing.
Chris
Hervey Bay, Australia.

Roger Knights
December 7, 2009 3:55 pm

A Lovell (12:56:14) :
“Forget ‘hacked’, or even ‘leaked’. I say they were LIBERATED!”

Right on! (Serves them right.)

Håkan B
December 7, 2009 4:17 pm

crosspatch (14:59:46) :
Exactly what I wrote here:
http://wattsupwiththat.com/2009/11/23/the-crutape-letters%C2%AE-an-alternate-explanation/
Håkan B (08:09:33) :
I return to the files timestamps. There are quite a lot of files with manipulated timestamps, most 2009-01-01-06:00:00, but also from 1980-01-01 and 2004-01-01, they all carry the “06:00:00″ part, that’s what shows up in my timezone CET. Why would someone set it to 06:00:00, wouldn’t it be easier to set it to 00:00:00, while your finger is ther at the “0″? Maybe he set it to 00:00:00 when he added those files, but he was doing it on a computer with timezone set to US Eastern Time?

December 7, 2009 4:32 pm

Okay, here’s another theory which I haven’t seen suggested yet:
Suppose an insider wanted to extort money from CRU. He assembles an archive of selected emails and other files which he knows will be quite damaging if released. He also assembles one or more additional archives of other emails and files which contain even more damaging information. Then he uploads them to a variety of anonymous FTP servers and prepares “dead man” switches which will distribute the information at his command or upon failure to countermand his instructions by preset dates.
Now he makes his extortion demand upon CRU. They blow it off. So he sends the first archive to BBC, which sits on it but probably notifies CRU. They still can’t believe he’d do something so destructive, and hence stonewall him. He releases the first archive to the public, and the fallout from Climategate is immense. It threatens world leaders and prominent scientists, with ramifications in the trillions of dollars (not to mention the political power grab).
Now he ups his demands to CRU and they cave, since the consequences of releasing a second archive are unthinkable. If he’s done a good job with his dead man switches, he may even live to enjoy his hush money.
Hey, it’s a neat theory and it can be made to fit the facts…

Alan Shore
December 7, 2009 4:40 pm

I don’t understand why you skeptics aren’t more skeptical. Lance’s analysis is poor to say the least. Firstly, as others have noted, he didn’t even realize that the email filenames are Unix timestamps. Secondly, he had nothing to say about the modified timestamp information of all the email text files and some of the documents. Thirdly, he didn’t even consider the idea that in all likelihood the documents were attachments to the emails. So it seems that the mighty skeptics, the dedicated band of contrarians who are going to save us all from the climate change fraud have been taken in by a filename – FOIA.zip.

Dave.
December 7, 2009 5:00 pm

To Chris of Hervey Bay.
The movie is ¨Downfall¨ The girl is Eva Braun crying
because the gig is up for the Nazis.
It´s a German film starring Bruno Ganz as Hitler in
the last days in the bunker before the Russians
capture Berlin.
Question, what led you to this video in the first place?
Cheers.
PS. Greetings from Scarness!

Rathtyen
December 7, 2009 5:05 pm

This is the issue that will give ClimateGate a new and powerful boost. Until now, the Warmists have been able to cloud the issue with the integrity behind “stealing” data. Once it is clear it was a whistleblower, that issue goes away. But also, because this likelihood was so bleeding obvious, the deviousness of CRU in claiming it was hacked, will be exposed.
This pattern of deception will be (surely!) a scandal in itself.

Peter S
December 7, 2009 5:13 pm

Here’s my theory – which I posted a little while ago on ClimateAudit:
I think the route into the CRU server – and directly to Jones’s emails is found in the file: 1248862973 – dated July 29 2009.
Here Mann writes to Jones:
“Santer et al paper still didn’t come through in your followup message. Can you post in on ftp where it can be downloaded?”
Jones replies to Mann:
“See below for instructions […]
file is at http://ftp.cru.xxx.xx.xx
login anonymously with emails as pw
then go to people/philjones
and you should find santeretal2001.pdf”
(my emphasis)
This email is CCed to
Kevin Trenberth
Jim Salinger
j.renwick
b.mullan
Gavin Schmidt
James Annan
Grant Foster
Foster’s address being a ‘tamino’ Hotmail account. Now ‘Tamino’ isn’t the most appealing character on the block for “loathesome” AGW sceptics – and much of what made him so depended upon his hiding behind a pseudonym that many people would have liked to officially ‘out’ him from. Apparently, Hotmail email accounts are fairly easy to crack – with instructions on how to do so openly available via a quick search on the internet. Anyone succeeding in such a crack would have found Jones’ ftp instructions in ‘Tamino’s’ in box.
If Jones did indeed have a folder named “emails” on http://ftp.cru.xxx.xx.xx (as his password suggests), and if that folder was intended to be a password-protected repository for the “loads of emails” Jones claimed to have deleted “2 months ago” (on December 3rd 2008) in response to FOI requests (1228330629), then anyone cracking into Tamino’s Hotmail email account would have come up trumps.

December 7, 2009 5:43 pm

Each of the files in the mail directory contains more than just an email message and is a complete email thread, which provides some additional context.
I definitely agree that the documents directory looks like the response to the FOIA request. The thing about the mail directory, is that the selection criteria seems a little broad for the FOIA and would be more appropriate for an FOIA requesting evidence of fraud, not just one requesting the data. I also don’t know the relevance of the marooned.jpg file, relative to the FOIA request.
There is something fishy about the Holdren references. There’s the one that shows him resort to an idiotic justification when confronted with the loosing end of a logical argument, but other than having Holdren’s name on it has no real significance. Also, the others with his name are from being a cc to one of Mike MacCracken’s emails. Mike and he are buddies with a shared interest in geoengineering and the original messages was with regard to cooling being caused by increased sulfur aerosols, which is what MacCracken wants to convince Holdren to do in order to offset the warming that isn’t occurring. Of course, it’s true that Holdren is not very far removed from the cabal.
It’s odd that this Holdren reference showed up in a bunch of unrelated emails. They were all around the same time. One possibility is that there was a lot of ‘reply to all include previous message’ going on, however, in some cases, the <<< that are prepended to the previous messages don't always add up right.
I've seen other instances of unrelated messages being inserted in the middle of threads, so it's also a possibility that this is just an artifact of the mail archive system and how it retrieves messages. BTW, many corporate environments routinely capture and copy all incoming and outgoing email. Sarbanes-Oxley compliance even requires this (like deleting email really does any good).
George

xyzlatin
December 7, 2009 5:44 pm

To Dave at Scarness,
I do believe the link was in a comment on RealClimate, but not 100% sure. I did go back to get it again but the link was gone, fortunately I remembered the name and located it again by doing a YouTube search.
What you have added above, makes the video more suspicious and warrants more investigation. I have linked many more of the subtitles with the contents of the emails in the file FOI2009 just by doing a quick search in XP, I do have a copy of the zip file.
I guess my main point has always been, how did the poster of “Peer Review 1945” know the contents of FOI2009.zip at least before the 19 November. I got the file (FOI2009) on the 20 November at 5:06 AM (my computers date and time stamp) and we are 18 hours ahead of the West Coast of the US, 14 ahead of the East Coast and 10 infront of GMT.
Chris in Urangan.

crosspatch
December 7, 2009 5:57 pm

” Håkan B (16:17:02) : ”
But the offset isn’t consistent. Whoever did this was clever. It appears that the “hours” value has been varied at random. Sometimes the date is 3 hours off, sometimes 4 sometimes 5 sometimes 6.
That that is only by looking at a handful of the emails. It appears that someone had a nice little script that parsed the date and changed the “hours” value at random but the minutes/seconds always hold.

JM Hanes
December 7, 2009 6:03 pm

From a layman who can’t hope to follow the technical discussion:
Is there a reason for assuming that FOIA2009.zip actually represents an FOIA file of some sort created by/at the university, or related to actual FOIA activity there?
I originally just assumed that it was a name given to the file by the leaker, perhaps as a pointed bit of irony.

December 7, 2009 6:33 pm

My initial suspicion was that someone inside, trusted, but not necessarily part of the cabal, was tasked with assembling the information for a recently denied FOIA request. As a result of doing this, they recognized the deceit going on, and wanted to be on the right side of history by ‘leaking’ the assembled data, along with a few extra tidbits to corroborate what we all already knew was happening. It almost seemed too good to be true.
The initial response by CRU was that they were aware that there was some kind of data theft, which was really just an.admission that the data was real.
I agree that one out of context email message can be misleading, but hundreds of email threads with a common theme definitely means something. Statistically, the email samples are more representative of underlying intent and behavior, than say proxies are for temperature reconstructions.

Editor
December 7, 2009 6:36 pm

Leif Svalgaard (10:30:19) :
> About Vostok: when it is really cold there the CO2 also freezes out…
Oh No! The Ghost of Horrors Past! Not this month! Oh, Whew:
> Yeah, I forgot about the low partial pressure bit. But perhaps if it became REALLY, REALLY cold 🙂
Oh Leif, please don’t do that to me – I couldn’t handle another round of that argument, certainly not with all the other nonsense going on!
My main contribution in settling it was to point out that if the dewpoint is -10°C, then frost will form on objects colder than that, but not on objects warmer than that.
The main links to the second round of that argument (we had settled it once before too) are http://wattsupwiththat.com/2009/06/13/results-lab-experiment-regarding-co2-snow-in-antarctica-at-113%c2%b0f-80-5%c2%b0c-not-possible/ and http://wattsupwiththat.com/2009/06/09/co2-condensation-in-antarctica-at-113f/
It really didn’t help when someone posted a similar claim by a physicist at Argonne, but I chased him down and he fixed it. It’s rather interesting how people (including myself) have no trouble with the frost on the windshield analogy, but had trouble applying the same reasoning to CO2 at Vostok.

December 7, 2009 6:38 pm

The reason some of the hours are off may be because the messages are to and from a wide variety of time zones.

xyzlatin
December 7, 2009 6:44 pm

I made a comment on climate audit or the air vent several days ago on this subject but nobody seemed interested.
However, here are my posts on my own blogsite http://castlecarrie.wordpress.com/
Climategate – was the leak in the USA?
December 6, 2009 at 12:37 pm | In Climategate | Leave a Comment | Edit this post
Tags: Climategate
Climategate:- I wonder why everyone assumes that the leaked files were obtained from the CRU server? It seems strange to me that Prof Jones said they had been broken into three days ago and he had done nothing about it. Did he just throw up a “straw man” to cover himself? Say the files were hacked and hope that people would focus on the hacker and ignore the content of the emails?
The whistleblower did not say the files were obtained from CRU.
Here is a statement by the Prof Jones on email number 1233245601.txt dated 29th January 2009 From Phil Jones to Ben Santor at the Lawrence Livermore National Library in California, and he makes mention how funny that Ben has put information in a directory called FOIA.
So that means that there was a directory in the US named FOIA?
Perhaps the leak has come from a file on Ben’s computer in California?
The released emails as has been pointed out by others, were collated Jan 1st 2009.
> With free wifi in my room, I’ve just seen that M+M have
> submitted a paper to IJC on your H2 statistic – using more
> years, up to 2007. They have also found your PCMDI data –
> laughing at the directory name – FOIA? Also they make up
> statements saying you’ve done this following Obama’s
> statement about openness in government! Anyway you’ll likely
> get this for review, or poor Francis will. Best if both
> Francis and Myles did this. If I get an email from Glenn I’ll
> suggest this.
0.000000 0.000000
climategate – read what the whistleblower actually wrote
December 6, 2009 at 2:12 pm | In Climategate | Leave a Comment | Edit this post
Re Climategate: Listen to what the whistleblower said – and what he/she didn’t say.
It is worthwhile noting that the whistleblower described the file on the http://www.megaupload.com site as
File description: climate audit whistleblower FOIA wuwt FOI2009.zip
In other words, the so called “hacker” used the word “whistleblower” when uploading the files. He also did not say that he got them from CRU although we are assuming he did. See my posting below re the FOIA directory in the US.
The Whistleblower – a US citizen aged 39?
December 8, 2009 at 10:40 am | In Climategate | Leave a Comment | Edit this post
Further to my investigation into who is the whistleblower, I posted a comment some days ago on the climate audit site which no one has followed up.
I believe that there is a connection between an YouTube video posted on 19th November 2009 by a person who joined YouTube on 19th November, calling himself Induseiumgresium and the FOIA release FOI2009.zip.
The video shows a clip from an old movie of Hitler and staff spoken in German. The subtext has many of the words of the emails in them. This was posted a day before the general release on Nov 20th. It beggars belief that this person did not know of the content of the emails.
Furthermore, the pseudonym can be broken down into two words, Indusium Gresium, which (roughly) is a layer which covers the sorus of a leaf. There can also be a “false indusium”. Now the connection here to Soros, one of the leading AGW behind the scenes funder, is another coincidence too good to be true.
I have sent two messages to this person, neither one of which has been answered. The person says he is 39 years old.
See http://www.youtube.com/watch?v=-VRBWLpYCPY
Climategate – Prof Jones strange statement
December 7, 2009 at 10:43 pm | In Climategate | Leave a Comment | Edit this post
Further to my discussion about it being odd that when the story broke, Prof Jones said that the breakin had happened 3 days ago, how could that be true if the BBC reporter got it in early October?
There were emails dated after October. So the person if it was a hacker is supposed to have broken into the system twice?
There is something decidedly smelly about Prof Jones statement.
Also another theory aired is that the file may have been accidentally left in a computer that had some sort of public access to it, and the hacker stumbled across it. In that case, he must have stumbled across it twice accidentally, once for the BBC release, and another for the later release.
0.000000 0.000000

Bonnie
December 7, 2009 6:57 pm

I agree with Optimist that it’s a file of “not to be seen” things, not anything in response to FOIA. Perhaps it was Phil Jones’s file of what to delete, as he promised. Only if the hacker was a skeptic would I go for that. Seems much more like an insider, even one of the scientists themselves who doesn’t want to be offed by Professor Watson, who I believe fully capable of such acts, after seeing him on video.

December 7, 2009 7:09 pm

Ric Werme (18:36:46) :
> “Yeah, I forgot about the low partial pressure bit. But perhaps if it became REALLY, REALLY cold :-)”
Oh Leif, please don’t do that to me – I couldn’t handle another round of that argument, certainly not with all the other nonsense going on!

I don’t see where the problem is. At the low partial pressure there is no liquid phase of CO2 [and I didn’t think about that in my hasty reply], and by REALLY cold, I meant like -200C. As some poster pointed out, we are getting too far off topic [leak or hack?].

Frankp
December 7, 2009 7:41 pm

Whence comes this great river, all this uproar?

Richard
December 7, 2009 7:51 pm

Leif Svalgaard (12:28:36) :
David Porter (12:10:25) :
Your comment that “CO2 also freezes out” I believe is impossible above a temperature of around -120 degree centigrade.
Gary Hladik (12:13:01) :
I thought we had a whole thread here awhile ago that concluded CO2 can’t freeze naturally on Earth, as it does on Mars.

I agree the partial pressure of CO2 would be too low for it to freeze here, 0.0387% of the atmosphere. On Mars whose atmosphere has 95% CO2 and much lower temperatures – yes.
Come to think of it the warmists say if the CO2 goes up by 0.03% we will have crossed the tipping point, look at Venus (which has 97%). But what about Mars? (which has 95%). Maybe the sun has got something to do with it.

Richard
December 7, 2009 8:02 pm

Ric Werme (18:36:46) : Hey that was a really cool experiment. Thanks for the link. Proving that the phase diagram is correct and partial pressure matters.
Not everyone has the wherewithal to do that experiment.

Benjamin
December 7, 2009 8:38 pm

Now THAT’S real investigation! Good job!
And that can only make me wonder how the “investigation” is going for the other side. I can HEAR (of) them all over the place, but I’m just not seeing the Russian Hacker yet! (I have a feeling that if he ever turns up, he’ll have the annoying accent of an English climate professor 🙂

crosspatch
December 7, 2009 8:38 pm

“Perhaps it was Phil Jones’s file of what to delete, as he promised.”
No, it can’t be because not all of the emails are Jones’ but all of the emails DO have a recipient or sender address at UEA.

December 7, 2009 8:40 pm

crosspatch (13:07:21) :
For example Here is a plot of transmittance of two atmospheres. One is pure CO2, the other is a mix of H2O and CO2 at a 5:1 mixture (5x more H2O)

Only relevant for a planet with a surface temp of about 1500K!
Try the band around 15μm for something relevant to Earth.

Kevin Kilty
December 7, 2009 8:41 pm

Third Party (08:52:03) :
from http://thehill.com/blogs/congress-blog/energy-a-environment/70857-climategate-sparks-luetkemeyer-call-for-investigation-sparks-interest-in-legislation-rep-blaine-luetkemeyer
The atmosphere contains from 4-percent water vapor in the troposphere to 40-percent near the surface.

Is this in the original? There is no way the atmosphere contains 40 percent water vapor anywhere. Perhaps someone read 40 0/00 as 40 percent rather than 40 per thousand.

Svein
December 7, 2009 8:45 pm

GMT+1:00 is simply England time in daylight savings mode, i.e. a majority of the year.

crosspatch
December 7, 2009 8:49 pm

“The reason some of the hours are off may be because the messages are to and from a wide variety of time zones.”
Which is why I compared messages with a Date: header in the same timezone. The ones that explicitly had +0000 or -0000 in the date string to be specific. Those are mails that would have originated in GMT.
The “offset” varied from 3 hours to 6 hours in the handful of emails that I looked at. That shouldn’t happen.

Svein
December 7, 2009 9:10 pm

There’s definitely something about this “indusieumgresium” dude. Do a Google search, and you’ll see that he has posted that video to almost every video site on the entire internet (slight exaggeration), all on Nov 19.

AnonyMoose
December 7, 2009 9:45 pm

co2isnotevil (18:38:05) : The reason some of the hours are off may be because the messages are to and from a wide variety of time zones.

When the last part of the time in the email contains a number such as “-0400”, that indicates the timezone. The filename ctime value matches the time sent. We don’t know whether the filenames were created by recently processing the emails, or by an email archive method.
I consider it unlikely that the university’s email server is archiving with the timestamp of when the message was sent, as it is likely that many people sending to the university will happen to send at nearly the same time as others. So the filename timestamps were probably created as part of organizing this batch of emails.

AndyW
December 7, 2009 10:18 pm

I don’t think that a whistleblower would try to hack into the Realclimate website as well, I think that tends to suggest it was a hacker who got in, got the info and then attempted to force it to be shown on realclimate.
Andy

December 7, 2009 10:48 pm

Why I doubt the FOI 2009 inforrmation was put together as part of a FOI release is it contains only incriminating material and nothing else. That’s not what you provide in such a request. You provide useful and useless information that fulfills the request.
I haven’t been involved a FOI release, but I have in a Justice Dept challenged merger. Eventually went all the way to the Supreme Court. Justice requested all files, so we gave them all files. This was in the days when PCs weren’t common and all records were on paper. We just filled about two dozen boxes, they picked them up and xeroxed them and returned them in about 10 days. None of our folks sorted through the files to find incriminating material, to make sure they found it. God forbid! Nor would anyone in their right mind do it for a FOI request.
This material most likely put together by a disaffected IT worker who wanted to torpedo Phil Jones and/or the whole warmist agenda. He certainly waited for the right moment.

crosspatch
December 7, 2009 10:49 pm

The filenames were created from the date, of that I am certain. Because the day of year and minutes and seconds match. The only thing that does NOT match is the hour which seems to vary. This is very clever as it removes any artifact of the person’s own timezone. It would go like this:
Parse the date, there are several date parsing routines around, into an array. Randomly “adjust” the “hour” value. Convert the date to Unix time. The resulting Unix time will result in a date exactly like what we are seeing here. The date, minutes, and seconds will match up but the hour will be off masking any timezone artifact of the “adjuster’s” computer.
If this wasn’t done, say they date was always 5 hours off. This might indicate that the person who renamed the file to a unix date string was using a computer that was set to a timezone 5 hours different from GMT and that is initially what I thought until I looked at additional files and noted that the time varied but ONLY varied in the hour value. The minutes and seconds were always consistent. This makes any consistent bias of the computer’s timezone impossible.
The original file name of the email had to be erased. I just don’t understand why they weren’t numbered sequentially unless MANY emails were renamed this way and only the “pertinent” files selected for inclusion into the zip archive. If they had originally been numbered sequentially, one would know how many were skipped and roughly how many there were in total. This way there is no way to tell and there is no way to tell where it was done.

jallen
December 8, 2009 1:12 am

Slightly different take:
They are the residual emails of a batch which had already been *sanitized* from the CRU systems, in order to illegally prepare an incomplete response for a future (likely successful) FOIA request. The emails in question were *not* going to be provided under a FOIA request.
These are deleted emails from a sanitized batch which were foolishly or purposely archived and/or discovered by an insider or whistleblower (perhaps the sanitizer himself). The insider then had pangs of conscience or an axe to grind and released them surreptitiously.

jallen
December 8, 2009 1:28 am

Also: The leaker may enjoy protection under the UK’s Public Interest Disclosure Act of 1998, which was enacted to protect whistleblowers.

Roger Knights
December 8, 2009 2:43 am

“This material most likely put together by a disaffected IT worker who wanted to torpedo Phil Jones and/or the whole warmist agenda. He certainly waited for the right moment.”
I think it would have been more damaging to have released the material a couple of months earlier, if the intent was to derail Copenhagen and prevent the warmists from getting their ducks in a row by having Clinton globe-trot to wangle deals with India and China, NZ commit it CO2 reduction, etc.

Thomas Jones
December 8, 2009 2:43 am
Chris Schoneveld
December 8, 2009 2:56 am

Richard (12:34:32) :
“Since climategate is only one word, searching for climategate or “climategate” should have no difference,”
Well, try it out and you will see the different outcomes:
Climategate without quotes gives: 37,200,000
“Climategate” gives: 2,670,000

Ian Summerell
December 8, 2009 3:04 am

Inside job, looks like it to me.
I worked in the civil service for six years and there was one room of a computer server. No one in the offices know what it did. I think it was a server to monater our work. The only person allowed in the room had a MI5 pass.
Was it the work of MI5 and they leaked the files?

BobC
December 8, 2009 3:15 am

I have seen it reported that 30-40 Russian students study at EAU. It is not inconceivable that one of them leaked the files compiled by the FOI officer, so the Russian connection is not disproven by this analysis. It is made more likely given the original server in Russia (for the leaked files) being one used by a local university (which a student from that university would have had access to).

Mike
December 8, 2009 4:20 am

It always looked like the data had been gathered by the UEA, and I could never understand where the idea of “hacker” came from. If it is the UEA themselves who have been spreading this rumour, then it is yet more proof of their fraudulent behaviour and one can see why someone on the inside might have got fed up with them and whistelblown.
But, there still is the possibility that the FOI officers PC/data was hacked.
As for the allegations that the Russian state were involved, it just shows the paranoia that is endemic in the UEA!

December 8, 2009 4:58 am

Bobc, as far as I can see the only reason anyone is suggesting a Russian involvement is that the files just happened to be placed on a Russian Server. The whole internet knows the reputation of Russia regarding the web, on a (village) forum I run, we block anyone signing up with a Russian email address because they are notorious for spam.
… or is it, could it be, that the real enemy of the warmers, is the old “cold-war enemy?”

Abendigo
December 8, 2009 5:31 am

Despite the “Evil Hackers” front the CRU is putting on this, they must be frantically searching for the leaker. Any word or insight on this?

Syl
December 8, 2009 6:25 am

Alan Shore (16:40:52) :
“Thirdly, he didn’t even consider the idea that in all likelihood the documents were attachments to the emails. ”
Since the data files are NOT attachments to the emails it seems the mighty warmers haven’t even bothered to check either the emails or the data themselves. Why? Afraid of what you may find perhaps?
Tut Tut

Hangtime55
December 8, 2009 9:54 am

I knew that the emails from the CRU were leaked . In all of the posts i’ve been responding to , I ALWAYS used the ” hacked/leaked ” phrase .
Mr. Paul Hudson, a weather presenter and climate correspondent for the BBC says he was FORWARDED the chain of e-mail material on October 12th , 2009 , more than five weeks before it was ‘ hacked ‘ and made public on the internet so how could Russian hackers be responsible for breaking into the university computers when Hudson was FORWRDED the same data ?
This brings up another question , if Hudson was forwarded the ClimateGate files on October 12, 2009 , could there be other recipients of the same information ?
Why it was said that the hacked/leaked emails came from a Russian Server could be a clue . I am only assuming that around the same time that the ClimateGate files were forwarded to Mr. Paul Hudson of the BBC , other copies could had been sent the same day or before or after Hudson received his copy .
Nevertheless , again I knew the CRU files were leaked and this should be the end to Rajendra Pachauri, chairman of the Intergovernmental Panel on Climate Change (IPCC), chairman of the Intergovernmental Panel on Climate Change (IPCC) sad and pitiful attempt to distract the world from the Conspiracy within the ClimateGate files .
Rajendra Pachauri is taking a page out of George W Bush’s playbook , chasing ‘ imaginary ‘ hackers or terrorists while the plot behind the scenes continue to grow.

Richard
December 8, 2009 11:55 am

Chris Schoneveld (02:56:53) :
Richard (12:34:32) :
“Since climategate is only one word, searching for climategate or “climategate” should have no difference,”
Well, try it out and you will see the different outcomes

This does not change the fact of my statement. It SHOULD have no effect? Comprehende?
if it does thats shows there is something wrong with the search mechanism, which is acting funny with Climategate. 310,000,000 hits sometimes and 30,000,000 at others for example.

Richard
December 8, 2009 11:58 am

PS – Lance Levsen – thats a real neat job, quite convincing that its not a hack.

Richard
December 8, 2009 12:11 pm

crosspatch (12:48:52) :
“But water accounts for about 60% of the greenhouse effect, while CO2 accounts for 26%. ”
I find that difficult to believe. CO2 is 0.038% of the atmosphere. The spectrum across which it absorbs IR is much narrower than H2O. In order for CO2 to provide half the total greenhouse warming of Earth, there would have to be more CO2 than water vapor in the atmosphere. CO2 is a much less efficient greenhouse gas than is water vapor

Your reasoning makes sense. I plucked the figure out of Wikipedia where it says CO2 is 9 – 26% I guess I just copied the 26. So that would make the average 17.5%.
Still this seems big. If water vapour is 10 times more than CO2, and water absorbs radiation over a greater range of the spectrum. Maybe its the infra-red where the CO2 absorbs more. That would account for it being a more effecient greenhouse gas

December 8, 2009 1:58 pm

Richard, Crosspatch,
Here’s a plot of the atmospheric transmittance between the surface and space. The different colors represent which gas is doing most of the absorption. This has been compiled from the complete HITRAN 2008 database. Gas color codes are on the left. The gray line is representative of the average energy flux leaving the planet.
http://www.palisad.com/co2/absorb.gif
Simulations show that about 1/3 of the GHG absorption is CO2 and 2/3 is from water vapor. Overall, about half of the surface radiation is absorbed by the atmosphere.
These are all average values representing a wide range of conditions. If the atmosphere is very dry, the CO2 fraction approaches and can even exceed half of the effect. If the atmosphere is very moist, the CO2 contribution can drop to about 10% of the total effect. For surface covered in heavy clouds, nearly 100% of the surface energy is absorbed by the atmosphere (and clouds), independent of GHG concentrations.
George

Chuck Hakkarinen
December 8, 2009 7:15 pm

crosspatch (09:32:25) :
No, the filenames of the emails are the unix epoch time of the email. The missing numbers are elapsed time between the emails. The filenames are the datestamps. I believe this is the date when the file was archived as there is generally a couple of hours difference between that date stamp and the date/time stamp in the headers of the email.
For example:
1123622471.txt
1123622471 is unix time August 9, 2005 21:21:11 GMT
The date/time stamp on that email is:
Date: Tue Aug 9 17:21:11 2005
Notice the hour is different but the minutes/seconds are the same.
^^^^^^^^^^^^^^^^^
If the unix time is 2100 GMT and the email date/time stamp is 1700, on 9 Aug, then this suggests to me that this particular email was sent from the Eastern (Summer) Time zone of the USA or Canada, which happens to be GMT-4 in August. Having not read any of the emails in the zip file, can someone confirm or refute my interpretation?

Richard
December 8, 2009 9:12 pm

co2isnotevil (13:58:14) : from that graph I would say that Ozone is an even more potent GHG than CH4

Daniel Fierro Sydney
December 9, 2009 6:26 am

Treason. All these politicians who attend Hoaxenhagen should be all sacked and trialed for Treason. Especially Australia’s Kevin Rude.

JMANON
December 9, 2009 9:51 am

I accepted at the start that this had to be an inside job.
But who? someone who happened on a file or directory being complied by someone for some nefarious purpose? or compiled by the leaker?
This analysis has been very insightful but doesn’t really answer who compiled it, when and why.
I have to suspect that this folder was carefully managed for leaking.
Why?
Because, suppose I am a researcher at UEA and I want to hide stuff from FOI disclosure. This is a pretty serious action and I’d try everything else first, as is evidently the implications, nobble the UEA FOI officer, and so on. Only as the position becomes desperate enough would I start deleting stuff but I might choose to simply move any incriminating or sensitive material to a new directory somewhere that can be retrieved when the problem has passed or if necessry to avoid going to jail “Oh, I’ve just found it all. Don’t know how it got there but you know what filing is like.”
But if that were the case, I’d just go through all my material and just move anything and everything suspect over to the hideaway stash. I wouldn’t spend time filtering it.
You might argue that that is what we have but I suggest it is not.
Why? because we only have single copies of emails with multiple replies embedded in them which means only a copy was selected that had the replies. Originals or copies in the inbox with earlier replies don’t seem to appear.
If I was creating a stash it wouldn’t matter that I had collected all versions in there – disc space is not a problem, time is. Time to selectively save and selectively delete. only had the most complete copy.
That suggests to me that someone took care to compile this, someone familiar enough to know what to include and what not and with the time to do it.
I suspect anyone discovering a hideaway folder would probably have sneek preview and then release pretty much all of it as it stood and as quickly as possible so as not to have copies on machines subject to inspection (an insider leaking).
So it still begs the question as to who compiled the folder, under what circumstances and did they go through afterwards and trim out a lot of fat and if so why?
It makes me think that some one was careful and familiar enough and with time enough when selecting emails only to include the latest copy with the most replies.
It raises the question as to the purpose of the folder again and I am not convinced it is simply a hideaway folder for incriminating stuff under FOI in case they need to produce it to stay out of jail, say.
In here we might have found the missing raw data, for example. all safe together.
I don’t know how true duplicate email thing is, I didn’t read all of the emails but the searchable website suggests it is so e.g. search for Phil Jones and we don’t find it there.
I also don’t know how significant this is. I may be reaching.
So the nature of the emails in there requires a bit of thought to create a realistic scenario that accounts not just for what is in there but what is not and while some could be eliminated as obviously private and some as unrelated, why only the copies with the most replies?
If we can aanswer that, we might know something more.
Since its release there are probably few corners as yet unexamined though the implications of some of that stuff has yet to be fully developed but we probably shouldn’t expect more.
On the other hand, if complied with a view to leaking, and carefully compiled by someone with sufficient knowledge to know already what is significant, then it would have to be that much of what is there has been carefully selected and the possibility that whoever did it was familiar enough with the material to appreciate the significance of the individual items does argue that some of the most damaging has yet to be released.
If so then what is to be released will probably be released near the end of Copenhagen and will probably be material that won’t be too voluminous and won’t require too much evaluation for the full import to be appreciated within a day or so.
Of course, I could just be indulging in wishful thinking and hoping for more and more dramatic revelations… and if a carefully planned release by some one with time to sift through the docs and with sufficient knowledge to appreciate all that was contained, then I’d have to suspect a new release soon.

December 9, 2009 4:03 pm

Richard,
Yes, O3 seems more powerful than CH4, but not by a whole lot. Besides, H2O and CO2 account for over 95% of all greenhouse gas absorption of surface energy. The band around CH4 may absorb a little more, but there’s more than just CH4 acting on that part of the spectrum.
George

Bernie
December 9, 2009 4:30 pm

February 15, 1995 Kevin Mitnick was captured because a consultant to the company that previously employed me as MIS Manager noticed unusual activity on his Well account. Analysis showed large zip files being created in his directory and forwarded through the consultant’s account. A trace was set up and he was caught in the act of forwarding another such large zip file containing credit card information.
Around the same time I was asked to scrub our ERP server at another employer, a major software vendor. Hackers dialing in from unsecured sales office modems in the former Warsaw Pact markets were tarballing source of the latest secure OS on our ERP server.
Both cases above mirror the examples that Lance uses to make a case for the insider leak scenario for FOIA.zip and both were professional hack jobs.
The examples neither prove nor disprove either the leak or professional hacking scenario. In my experience they are consistent with the latter at least in the two with which I had some personal experience.
Lance’s analysis of email distribution show that these were few files out of many that were collated probably by a crawler of some kind similar to good old gopher. These files were selectively chosen and collected then zipped up for export in a professional fashion identical to the two proven professional break-ins I described.

Bernie
December 9, 2009 7:47 pm

PS:
For further evidence of the SOP of this hack please note the article by Robert Graham on Nov 24 detailing the use of open proxies
http://erratasec.blogspot.com/2009/11/climate-hack-used-open-proxies.html
A further link contained therein comments that
“At around 6.20am (EST) Nov 17th, somebody hacked into the RC server from an IP address associated with a computer somewhere in Turkey, disabled access from the legitimate users, and uploaded a file FOIA.zip to our server. ”
Using open proxies, disabling access, uploading … pretty standard Professional Hack SOP – not your usual wistleblower pattern.

December 9, 2009 8:36 pm

It’s my understanding that the released data was first downloaded to an RC server, probably through the ftp user and password contained in one of the emails, and announced over an RC blog, with a link to the file on the RC ftp site. This happened about 3-4 weeks after the file was first revealed to a reporter, and subsequently buried as not being newsworthy.
That the transfer to RC originated from Turkey means nothing, as it’s relatively easy to use an anonymous proxy, especially if you already know an account and password.

Hangtime55
December 9, 2009 9:10 pm

HEY BERNIE , IT WASN’T A HACK ….. IT WAS A LEAK FOOL !

Hangtime55
December 9, 2009 9:15 pm

HEY co2isnotevil , IT WASN’T DOWNLOADED , IT WAS UPLOADED OUT OF THE CRU by someone INSIDE !!! Come on man ! Are you with the program???
The Climategate files were FOWARDED to the BBC 5 weeks before the leak at the CRU was detected !!! WAKE UP !

December 9, 2009 9:40 pm

One thing that’s interesting, is that there’s overlap between attachments mentioned in the emails, and the files in the documents directory, for example, the marooned.jpg image and RulesOfTheGame.pdf file. It’s possible that the documents have been collected from the same repository that stored the emails. Raw email messages stored as text files always include binary attachments like jpeg images, as inline strings of printable characters encoding binary data, i.e. base-64 encoding, so it’s clear that attachments have been extracted. Some of the files in the documents directory may be decoded zip, gzip and tar files extracted from other email messages.

December 9, 2009 10:30 pm

Hangtime,
The 5 week time line doesn’t seem right, since the last time stamp on a captured email was Nov 12 of this year, which was only 4 weeks ago, see the message in 1258053464.txt. It’s been public for about 3 weeks, which coincides with when the CRU acknowledged the theft, so if this was the case, then the BBC got something else. So I was wrong, it wasn’t 2-3 weeks, but only about 1 week where the BBC had it first. I had been thinking that the last email was in late October, but I just checked and it was November 12.
Also, I said the file was sent to RC, not CRU, but quickly removed. This was in response to the reference to the ‘hacked RC server’ on Nov 17 that Bernie referenced. The zip file then showed up on the Russian web site and shortly thereafter, it was everywhere.
George

December 10, 2009 5:51 am

Thank you for your sharring!!

Hangtime55
December 10, 2009 9:55 am

co2isnotevil
On November 17th , a anonymous person left a comment at The Air Vent website . This comment read :
” . . . We feel that climate science is, in the current situation, too important to be kept under wraps.We hereby release a random selection of correspondence, code, and documents. Hopefully it will give some insight into the science and the people behind it. This is a limited time offer, download now . . . ”
He allegedly then continued with a link to a Russian anonymous FTP account.
On November 20th , Phil Jones at CRU was advised from administrators of RealClimate website that ‘hackers’ attempted to upload a file from CRU to RealClimate but administrators said they refused to accept it .
How RealClimate knew the files were not only from CRU but were also stolen still wasn’t clear to me until later I read that RealClimate did in fact have the files and told Phil Jones they would send it over to him , which they didn’t . This may account on how RealClimate knew the files were from the CRU.
RealClimate posted the files for a while then took them down off their site .
RealClimates statement :
” . . . We were made aware of the existence of this archive last Tuesday morning when the hackers attempted to upload it to RealClimate, and we notified CRU of their possible security breach later that day . . . ”
This statement from RealClimate was released November 20th , 3 days after the CRU files were leaked/hacked and verifies Jone’s statement of the files being taken ” three or four days ago ” .
The anonymous person then is said to had gone to The Blackboard site and posted its link to the files on the Russian Server . Here the files were examined a little closer and then were discovered to possibly be authenic . The link was then picked-up then by everyone interested and the more notable websites , The ClimateAudit and Watts Up With That . I understand the link on the russian server was up for 48 hours until it no longer worked .
Your research on the dates of the files and the timeline of the transfer of the files could be explained .
Phil Jones was interviewed by Ian Wishart of Investigate magazine ( dated Nov 20,09 ) and in his interview he said :
“ . . . It was a hacker. We were aware of this about three or four days ago that someone had hacked into our system and taken and copied loads of data files and emails . . . ”
This statement is suspicious in itself . Jones said the data was ” taken AND copied ” ? Why wasn’t the files simply ‘ Taken ‘ or ‘ Copied ‘ ?
This tells me that the file , ( FOIA2009 ) ‘ obtained ‘ was possibly a folder that was created beforehand and updated as the ‘ insider ‘ saw fit , thus the word Taken refers to leaked or hacked while the word Copied refers to updated ?
To verify this one would have to compare the ClimateGate file that Paul Hudson from the BBC was forwarded on October 12 , 2009 . again , the word ‘ forwarded ‘ implies to the files being ‘ Copied ‘ , not Taken as Jones had stated to Invesigate Magazine .
This means that the ‘ anonymous person ‘ must be a ‘ insider ‘ within the Climate Research Unit , as he had opportunity to ‘ update ‘ the FOIA2009 folder from within until November 12th , 30 days after Hudson at the BBC possibly had an altermatum to either expose ClimateGate to the public OR the ‘ insider ‘ would leak the data him/her/them selfs .
That’s my assessment so far !

December 10, 2009 10:50 am

Hangtime,
This is relatively close to my understanding. The one difference is that I don’t think that it was RC that put the data on their site. I believe the whistleblower uploaded the file to their site and then linked to it from a post on RC. What possible use would RC have for this zipfile in the first place? The only think I can think of is that it was passed on with a question like, “Gavin, how much damage is the truth going to do to us?”.
George

xyzlatin
December 10, 2009 9:08 pm

Hangtime55, Now that you have been working on times and dates, tell me this, How did “indusieumgresium” get to post “Peer Review 1945” up to YouTube on the 19 november ??

Hangtime55
December 11, 2009 12:45 am

xyzlatin :
I have NO IDEA what “indusieumgresium” is . And to tell you the truth , I don’t think it has anything to do with what we are discussing here . . . but i’ll take a look at it when I can .

xyzlatin
December 11, 2009 6:21 am

Hangtime 55,
Mabe the date didn’t click, 19 November.
The movie clip has subtitles that directly reflect the contents of the emails from the CRU.
My view is that the whistleblower could be “indusieumgresium”
“indusieumgresium” is made up of 2 Latin words, indusieum and gresium.
“indusieumgresium” had to read thru 1079 text files, get the obscure video clip, put in the subtitles, and then post it to YouTube by the 19th. Mission Impossible, unless you know previously what was in the released emails.
Took me 2 days alone to get thru the text files, let alone find the damaging bits.
Go check out the clip “Peer review 1945” Google it. And then you may want to disguss it here.
BTW. I was on the site reading “An open letter climate change legislation” when it all happened. I’m in Australia and it was already the 20th here when it all broke out.

Turboblocke
December 11, 2009 7:53 am

For a sense of the importance of climategate: take a look at Google Insights here:http://www.google.com/insights/search/#q=climategate%2Ctiger%20woods&date=today%201-m&cmpt=q
Now if the e-mails had hit a fire hydrant you might get some interest.

Turboblocke
December 11, 2009 9:26 am

Guys: you are barking up the wrong tree with this insider meme. FOIA is an American term: in the UK it is called the FIA or the FOI Act. Don’t believe me: look at this. http://www.google.com/insights/search/#q=foia%2Cfia%2Cfoi&geo=GB&date=today%2012-m&cmpt=qhttp://www.google.com/insights/search/#q=foia%2Cfia%2Cfoi&geo=GB&date=today%2012-m&cmpt=qhttp://www.google.com/insights/search/#q=foia%2Cfia%2Cfoi&geo=GB&date=today%2012-m&cmpt=qhttp://www.google.com/insights/search/#q=foia%2Cfia%2Cfoi&geo=GB&date=today%2012-m&cmpt=q
That’s Google tendancies for FOIA, FIA and FOI in the UK. There’s no sign of FOIA. It was a hacker from the US. He might just have well signed it “Yankee Doodle Dandy.”

Turboblocke
December 11, 2009 9:28 am

I also note that indusieumgresium claims to be in the USA. Busted!

Hangtime55
December 11, 2009 10:19 am

That graph doesn’t tell me too much , afterall it’s Goggle presenting it .
Before Google took over YouTube last year , YouTube was a good site for finding information for whatever you were looking for . Since Google has taken over , Google has been either deleting videos or you get that old comment ” this video is unavailable at this time “, or ” posting comments on this video cannot be made at this time ” . Google has been ‘ censoring ‘ these files , for example , due to BS like videos having ‘ copyrighted ‘ sound tracks ?
When video’s of interests have a large ‘ hit ‘ count , people usually go to them , primarily because if that video had , say 500,000 hits then it must have been a video of some importance .
I’ve noticed that since Google had taken over , video’s that had , say a 500,000 hit count on them now are posted of having only 4,000 hits ?
Even on Google’s main search engines , there are many keywords , for example ‘ Climategate ‘ that when you type it in , you get no suggestions on what your looking for . You have to know what other keywords are in relation to the piece your searching . For the Layman who wants to get more information on a subject like ClimateGate without knowing anything about the subject is screwed .
A few years ago when Google was assisting Communist China increating a Google search engine , Google had said it will censor its search services in China in order to gain greater access to China’s fast-growing market . Google’s move in China came less than a week after it resisted efforts by the US Department of Justice to make it disclose data on what people were searching for.
So Googles ‘ insights ‘ aren’t that important to me .

xyzlatin
December 11, 2009 11:02 am

As I have pointed out before, in one email Phil Jones lauds Ben Santor for having a directory called FOIA. Ben Santor is in US.

xyzlatin
December 11, 2009 11:06 am

Here is a statement by the Prof Jones on email number 1233245601.txt dated 29th January 2009 From Phil Jones to Ben Santor at the Lawrence Livermore National Library in California, and he makes mention how funny that Ben has put information in a directory called FOIA.
So that means that there was a directory in the US named FOIA?
Perhaps the leak has come from a file on Ben’s computer in California?
The released emails as has been pointed out by others, were collated Jan 1st 2009.
> With free wifi in my room, I’ve just seen that M+M have
> submitted a paper to IJC on your H2 statistic – using more
> years, up to 2007. They have also found your PCMDI data –
> laughing at the directory name – FOIA? Also they make up
> statements saying you’ve done this following Obama’s
> statement about openness in government! Anyway you’ll likely
> get this for review, or poor Francis will. Best if both
> Francis and Myles did this. If I get an email from Glenn I’ll

xyzlatin
December 11, 2009 11:41 am

By the way, I have been searching daily for Climategate on Google, and each day the numbers go down! Several days ago the count was 32,600,000. Now down to 28,200,000.

xyzlatin
December 11, 2009 11:44 am

Phil JOnes mentions a FOIA directory in one email 1233245601.txt dated 29th January 2009

Dave.
December 11, 2009 11:48 am

Turboblocke:WRITES
Turboblocke (09:26:15) :
¨Guys: you are barking up the wrong tree with this insider meme. FOIA is an American term: in the UK it is called the FIA or the FOI Act. Don’t believe me: look at this. http://www.google.com/insighGuys: you are barking up the wrong tree with this insider meme. FOIA is an American term: in the UK it is called the FIA or the FOI Act.¨
So in the UK, What is the acronym for Freedom of Information act.
I would think FOIA is pretty close?
¨I also note that indusieumgresium claims to be in the USA. Busted!¨
That´s if he IS involved and submitted his correct details.

xyzlatin
December 11, 2009 3:02 pm

Still begs the question, how did anyone get the video up on YouTube on the 19th.
There just isn’t enough time.
When the emails hit the fan, it was the 20th here.
Remember FOIA said
This is a limited time offer, download now: http://ftp.tomcity.ru/incoming/free/FOI2009.zip
That link worked for only a matter of hours, and you guys were all asleep, it was posted at 9:57 PM on the 17th.
So indusieumgresium would have to been one of the very lucky few who down loaded the zip file, and then I doubt if there was enough time to get the video clip together.
The next download site that went up, Megaupload, was on the 20 Nov. at 12:59 AM.
Remember also at this time ClimateAudit server went down, see the thread on the Air Vent
” Ok it’s blown wide open
Posted by Jeff Id on November 19, 2009″
But still, with all this going on, indusieumgresium, got the video up on the 19th.
Did anyone check out the Latin meanings of the 2 words, they are all connected with biology, trees and ferns, a grey protective layer, protecting the sorus (George Soros ? ), a Chinese Paperbark Maple, (tree rings, a big problem over the temp. stations in China). and it goes on.
indusieum gresium could not have chosen a better name to describe what was going on.

Hangtime55
December 11, 2009 4:09 pm

You seem to have your ‘ indusieumgresium ‘ issue in order . I’ve never heard of it and I don’t have time to divert my attention from other matters i’ve been on for the last 5 weeks , so good luck with your indeavors .

Svein
December 12, 2009 3:37 am

I saw that video again, by indusieumgresium, and there’s nothing that references anything in the CRU emails. It’s a generic poke at the peer review process in general and is not even specific to climate research. Complete diversion.

Simeon Higgs
December 14, 2009 2:14 am

the good thing if it was leaked is that the UK whistleblower act would protect whoever did it.

Yoron
January 8, 2010 4:47 am

You say “I suggest that it isn’t feasible for the emails in their tightly ordered format to have been kept at the departmental level or on the workstations of the parties. I suggest that the contents of ./documents didn’t originate from a single monolithic share, but from a compendium of various sources.
For the hacker to have collected all of this information s/he would have required extraordinary capabilities. The hacker would have to crack an Administrative file server to get to the emails and crack numerous workstations, desktops, and servers to get the documents.”
It’s a hierarchy of machines. As fast as you are into ‘cgi bin’ http://neworder.box.sk/newsread.php?newsid=1351 you’re home free. And as it is a network, all machines are accessible from the ‘top’. Then it just take time and careful investigation.
and people save their mails you know.
And if FSB (Russia) was involved they sure have the expertize and manpower to sort out and also falsify dates, etc, to their heart desire. We can’t say when they first went into that system/OS . the only thing we know is when they released their ‘loot’. Can you see the difference?
But it was a nice description you made.

Yoron
January 8, 2010 4:55 am

Forgot to add 🙂
And that’s why you see it as ‘disorganized’ “from a compendium of various sources.”
You see in every mastrojka (doll inside a doll inside a ….) the most important thing is to offer clever guys like you possible ‘proof’ of plausibility.
You’ve found yours and drew your conclusion. Another could be that it shows how FSB have worked overtime, sorting and choosing the combination of ‘random’ Email released to bring you to that conclusion?
Hacking is one thing, but sorting the intelligence and creating the right approach for using it takes time.
And there is nothing digital on a hard disk that can’t be manipulated.
Cheers
Yoron.

January 8, 2010 4:30 pm

Yoron,
You seem to be in denial. While it’s certainly true that someone did a sophisticated search to produce the FOIA zip file, the emails are clearly from a central repository. Many organizations have email capture systems for various compliance requirements. The machines these compliance apps run on are the most highly secured systems in a network. Many of the data files are attachments to the released emails, and others could have been attachments to other emails that were not released. It’s quite plausible that all of this was in one place (like deleting emails really matters) and someone with high level access to the CRU’s servers use a simple indexing tool to search through emails and attachments for specifically relevant matters. The most likely scenario was that this was prepared for an FOIA request, and upon review of how damaging that information was, they decided to hide behind disclosure laws to prevent it’s release. Of course, whoever prepared it, also recognized the significance, developed a conscience, and thought it was more reprehehsible to continue to hide the deceit.

Voodoo
January 28, 2010 11:19 pm

Don’t disagree with your analysis, but you’re forgetting national security agencies spying on these guys. CRU was the body of chioce for the UN’s IPCC since around 1999! I’m sure the USA (or anyone else) would think it’s in their interest to get the NSA to crack an MS Exchange server and a few servers to know the truth. Your also forgetting passive inductive intercepts on the hardware. If you serious about getting in, you can.