Comprehensive network analysis shows Climategate likely to be a leak

This lends cred to WUWT’s previous analysis done by our own Charles the moderator: The CRUtape Letters™, an Alternative Explanation,

Climate-Gate: Leaked

by Lance Levsen, Network Analyst – courtesy of Small Dead Animals

http://www.swfwmd.state.fl.us/conservation/indoors/leak.jpg

Introduction

Some time starting in mid November 2009, ten million teletypes all started their deet-ditta-dot chatter reeling off the following headline: “Hackers broke into the University of East Anglia’s Climate Research Unit….”

I hate that. It annoys me because just like everything else about climate-gate it’s been ‘value-added’; simplified and distilled. The contents of FOIA2009.zip demand more attention to this detail and as someone once heard Professor Jones mutter darkly, “The devil is in the details…so average it out monthly using TMax!”

The details of the files tell a story that FOIA2009.zip was compiled internally and most likely released by an internal source.

The contents of the zip file hold one top-level directory, ./FOIA. Inside that it is broken into two main directories, ./mail and ./documents. Inside ./mail are 1073 text files ordered by date. The files are named in order with increasing but not sequential numbers. Each file holds the body and only the body of an email.

In comparison, ./documents is highly disorganized. MS Word documents, FORTRAN, IDL and other computer code, Adobe Acrobat PDF’s and data are sprinkled in the top directory and through several sub-directories. It’s the kind of thing that makes the co-workers disorganized desk look like the spit and polish of a boot camp floor.

What people are missing entirely is that these emails and files tell a story themselves.

The Emails

Proponents of the hacker meme are saying that s/he broke into East Anglia’s network and took emails. Let’s entertain that idea and see where it goes.

There is no such thing as a private email. Collecting all of the incoming and outgoing email is simple in a mail server. Using: Postfix the configuration is always_bcc=<email address>, here are links on configuring the same for Sendmail, and for Exim. Those are the three main mail servers in use in the Unix environment. Two of them, Sendmail and Exim are or were in use as the external mail gateways and internal mail servers at the University of East Anglia (UEA).

When a mail server receives an email for someone@domain.net, it checks that it is authoritative for that domain. This means that a server for domain.net will not accept email for domain.ca. The mail server will usually then run checks on the email for spam, virus, and run other filters. It will then check to see whether to route the email to another server or to drop the email in a users mailbox on that server. In all examples examined in the released emails, the mail gateway forwarded the emails to another server.

The user then has a mail client that s/he uses to read email. Outlook Express, Eudora, Apple Mail, Outlook, Thunderbird, mutt, pine and many more are all mail clients.

Mail clients use one of two methods of reading email. The first is called POP and that stands for Post Office Protocol. A mail client reading email with POP logs into the mail server, downloads the email to the machine running the mail client and will then delete the original email from the users spool file on the mail server.

The second protocol is called IMAP, Internet Message Access Protocol. IMAP works by accessing the mailboxes on the mail server and doing most of the actions there. Nothing is actually downloaded onto the client machine. Only email that is deleted and purged by the mail client is gone. Either protocol allows the user the opportunity to delete the email completely.

Most email clients are setup for reading emails with POP by default and POP is more popular than IMAP for reading email.

The released emails are a gold mine for a system administrator or network administrator to map. While none of the emails released contained headers, several included replies that contained the headers of the original emails. An experienced administrator can create an accurate map of the email topography to and from the CRU over the time period in question, 1998 thru 2009.

Over the course time, UEA’s systems administrators made several changes to the way email flows through their systems. The users also made changes to the way they accessed and sent email.

The Users

Using a fairly simple grep1 we can see that from the start of the time-frame, 1999, until at least 2005 the CRU unit accessed their email on a server called pop.uea.ac.uk. Each user was assigned a username on that server. From the released emails, we can link username to people as such:

In the previously referenced grep comes some more useful information. For instance, we know that Professor Davies was using QUALCOMM Windows Eudora Light Version 3.0.3 (32) in September of 1999. (ref Email: 0937153268.txt). If you look at the README.txt for that version you can see that it requires a POP account and doesn’t support IMAP.

As mentioned previously, POP deletes email on the server usually after it is downloaded. Modern POP clients do have an option to save the email on the server for some number of days, but Eudora Light 3.0.3 did not. We can say that Professor Davies’ emails were definitely removed from the server as soon as “Send/Recv” was finished.

This revelation leaves only two scenarios for the hacker:

  1. Professor Davies’ email was archived on a server and the hacker was able to crack into it, or
  2. Professor Davies kept all of his email from 1999 and he kept his computer when he was promoted to Pro-Vice Chancellor for Research and Knowledge Transfer in 2004 from his position as Dean of the School of Environmental Sciences.

The latter scenario requires that the hacker would have had to know how to break into Prof. Davies’ computer and would have had to get into that computer to retrieve those early emails. If that were true, then the hacker would have had to get into every other uea.ac.uk computer involved to retrieve the emails on those systems. Given that many mail clients use a binary format for email storage and given the number of machines the hacker would have to break into to collect all of the emails, I find this scenario very improbable.

Which means that the mail servers at uea.ac.uk were configured to collect all incoming and outgoing email into a single account. As that account built up, the administrator would naturally want to archive it off to a file server where it could be saved.

This is a simple evolution. You just run a crontab to start a shell-script that will stop the mail server, move the mail spool file into a file somewhere else, nulls the live spool and restart the mail server. The account would reside on the mail server, the file could be on any server.

Alternatively you could use a procmail recipe to process the email as it comes in, but that may be a bit too much processing power for a very busy account.

This also helps to explain the general order of the ./mail directory. Only a computer would be able to reliably export bodies of email into numbered files in the FOIA archive. As the numbers are in order not just numerically but also by date, the logical reasoning is that a computer program is numbering emails as they are processed for storage. This is extremely easy to do with Perl and the Mail::Box modules.

The Email Servers

I’ve created a Dia diagram2 of the network topography regarding email only as demonstrated in the released emails. Here’s a jpeg of it:

CRU's network for email 	  from 1998 thru 2009.

click to enlarge

The first thing that springs to mind is that the admins did a lot of fiddling of their email servers over the course of ten years. 🙂 The second thing is the anomaly. Right in the middle of 2006-2009 there is a Microsoft Exchange Server. Normally, this wouldn’t be that big of an blip except we’ve already demonstrated that the servers at UEA were keeping a copy of all email in and out of the network. Admins familiar with MS Exchange know that it too is a mail server of sorts.

It is my opinion that the MS Exchange server was working in conjunction with ueams2.uea.ac.uk and I base this opinion on the fact that ueams2.uea.ac.uk appears both before and after the MS Exchange Server. It doesn’t change its IP address nor does it change the type of mail server that is installed on it. There is a minor version update from 4.51 to 4.69. You can see Debian’s changelog between the Exim versions here.

I’ve shown that the emails were collected from the servers rather than from the users accounts and workstations, but I haven’t shown which servers were doing the collection. There are two options, the mail gateway or the departmental mail servers.

As demonstrated above, I believe that the numbers of the filenames correspond to the order that the emails were archived. If so, the numbers that are missing, represent other emails not captured in FOIA2009.zip.

I wrote a short Bash program3 to calculate the variances between the numbering system of the email filenames. The result is staggering, that’s a lot of email outside of what was released. Here’s a graph of the variances in order as well as a graph with the variances numerically sorted . Graph info down below.

Variance from Email Number to the 	  last Email Number

click to enlarge

Variances sorted and plotted

click to enlarge

The first graph is a little hard to read, but that’s mostly because the first variance is 8,805,971. To see a little better, just lop off the first variance and rerun gnuplot. For simplicity, that graph is here. The mean of the variances is 402839.36 so the average amount of emails between each released email is 402,839. While not really applicable, but useful, the standard deviation is 736228.56 and you can visualize that from the second graph.

I realize that variance without reference is useless, in this instance the number of days between emails. Here is a grep of the emails with their dates of origin.

I do not see the administrators copying the email at the departmental level, but rather at the mail gateway level. This is logical for a few reasons:

  • The machine name ueams2.uea.ac.uk implies that there are other departmental mail servers with the names like ueams1.uea.ac.uk, (or even ueams.uea.ac.uk), maybe a ueams3.uea.ac.uk. If true, then you would need to copy email from at least one other server with the same scripts. This duplication of effort is non-elegant.
  • There is a second machine that you have to copy emails from and that is the MS Exchange server so you would need a third set of scripts to create a copy of email. Again, this would be unlike an Administrator.
  • Departmental machines can be outside the purview of Administration staff or allow non-Administrative staff access. This is not where you want to be placing copies of emails for the purposes of Institutional protection.
  • As shown with the email number variances, and if they are representative of the email number as it passed through UEA’s email systems, that’s a lot of emails from a departmental mail server and more like an institutional mail gateway.

So given the assumptions listed above, the hacker would have to have access to the gateway mail server and/or the Administration file server where the emails were archived. This machine would most likely be an Administrative file server. It would not be optimal for an Administrator to clutter up a production server open to the Internet with sensitive archives.

The Documents

The ./FOIA/documents directory is a complete mess. There are documents from Professor Hulme, Professor Briffa, the now famous HARRY_READ_ME.txt, and many others. There seems to be no order at all.

One file in particular, ./FOIA/documents/mkhadcrut is only three lines long and contains:

	  tail +13021 hadcrut-1851-1996.dat | head -n 359352 | ./twistglob > hadcrut.dat

	  # nb. 1994- data is already dateline-aligned

	  cat hadcrut-1994-2001.dat >> hadcrut.dat

Pretty simple stuff, get everything in hadcrut-1851-1996.dat starting at the 13021st line. From that get only the first 359352 lines and run that through a program called twistglob in this directory and dump the results into hadcrut.dat. Then dump all of the information in hadcrut-1994-2001.dat into the bottom of hadcrut.dat.

….Except there isn’t a program called twistglob in the ./FOIA/documents/ directory. Nor is there the resultant hadcrut.dat or the source files hadcrut-1851-1996.dat and hadcrut-1994-2001.dat.

This tells me that the collection of files and directories in ./documents isn’t so much a shared directory on a server, but a dump directory for someone who collected all of these files. The originals would be from shared folders, home directories, desktop machines, workstations, profiles and the like.

Remember the reason that the Freedom of Information requests were denied? In email 1106338806.txt, Jan 21, 2005 Professor Phil Jones states that he will be using IPR (Intellectual Property Rights) to shelter the data from Freedom of Information requests. In email 1219239172.txt, on August 20th 2008, Prof. Jones says “The FOI line we’re all using is this. IPCC is exempt from any countries FOI – the skeptics have been told this. Even though we (MOHC, CRU/UEA) possibly hold relevant info the IPCC is not part our remit (mission statement, aims etc) therefore we don’t have an obligation to pass it on.”

Is that why the data files, the result files and the ‘twistglob’ program aren’t in the ./documents directory? I think this is a likely possibility.

If Prof. Jones and the UEA FOI Officer used IPR and the IPCC to shelter certain things from the FOIA then it makes sense that things are missing from the ./documents directory. Secondly it supports the reason that ./documents is in such disarray is that it was a dump folder. A dump folder explicitly used to collect information for the purpose of release pursuant to a FOI request.

Conclusion

I suggest that it isn’t feasible for the emails in their tightly ordered format to have been kept at the departmental level or on the workstations of the parties. I suggest that the contents of ./documents didn’t originate from a single monolithic share, but from a compendium of various sources.

For the hacker to have collected all of this information s/he would have required extraordinary capabilities. The hacker would have to crack an Administrative file server to get to the emails and crack numerous workstations, desktops, and servers to get the documents. The hacker would have to map the complete UEA network to find out who was at what station and what services that station offered. S/he would have had to develop or implement exploits for each machine and operating system without knowing beforehand whether there was anything good on the machine worth collecting.

The only reasonable explanation for the archive being in this state is that the FOI Officer at the University was practising due diligence. The UEA was collecting data that couldn’t be sheltered and they created FOIA2009.zip.

It is most likely that the FOI Officer at the University put it on an anonymous ftp server or that it resided on a shared folder that many people had access to and some curious individual looked at it.

If as some say, this was a targeted crack, then the cracker would have had to have back-doors and access to every machine at UEA and not just the CRU. It simply isn’t reasonable for the FOI Officer to have kept the collection on a CRU system where CRU people had access, but rather used a UEA system.

Occam’s razor concludes that “the simplest explanation or strategy tends to be the best one”. The simplest explanation in this case is that someone at UEA found it and released it to the wild and the release of FOIA2009.zip wasn’t because of some hacker, but because of a leak from UEA by a person with scruples.

Footnotes

1 See file ./popaccounts.txt

2 See file ./email_topography.dia

3 See file ./email_variance.sh

4 See file ./gnuplotcmds

Notes

Graph Information

Graphs created with gnuplot using a simple command file4 for input. I use a stripped down version of the variants_results_verbose.txt file, it’s the same, just stripped of comment and the filenames.. The second graph is a numerically sorted version, $> sort -n ./variance_results.txt > variance_sorted_numerically.txt.

Assigned Network Numbers for UAE from RIPE.NET

RIPE.NET has assigned 139.222.0.0 – 139.222.255.255,193.62.92.0 – 193.62.92.255, and 193.63.195.0 – 193.63.195.255 to the University of East Anglia for Internet IP addresses.

RIPE.NET Admin contact for the University of East Anglia: Peter Andrews, Msc, Bsc (hons) – Head of Networking at University of East Anglia. (Linked In, Peter isn’t in the UEA directory anymore so I assume he is no longer at UEA.)

RIPE.NET Tech Contact for the University of East Anglia: Andrew Paxton

Current Mail Servers at UEA

A dig for the MX record of uea.ac.uk (email servers responsible for the domain uea.ac.uk) results in the following:

	  $> dig mx uea.ac.uk

	  ; <<>> DiG 9.6.1-P2 <<>> mx uea.ac.uk

	  ;; global options: +cmd

	  ;; Got answer:

	  ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 737

	  ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 13, ADDITIONAL: 13

	  ;; QUESTION SECTION:

	  ;uea.ac.uk.			IN	MX

	  ;; ANSWER SECTION:

	  uea.ac.uk.		50935	IN	MX	2 ueamailgate01.uea.ac.uk.

	  uea.ac.uk.		50935	IN	MX	2 ueamailgate02.uea.ac.uk.

The IP addresses for the two UEA email servers are:

ueamailgate01.uea.ac.uk. 28000 IN A 139.222.131.184

ueamailgate02.uea.ac.uk. 28000 IN A 139.222.131.185

Test connections to UEA’s current mailservers:

	  $> telnet ueamailgate01.uea.ac.uk 25

	  Trying 139.222.131.184...

	  Connected to ueamailgate01.uea.ac.uk.

	  Escape character is '^]'.

	  220 ueamailgate01.uea.ac.uk ESMTP Sendmail 8.13.1/8.13.1; Mon, 7 Dec 2009 01:45:42 GMT

	  quit

	  221 2.0.0 ueamailgate01.uea.ac.uk closing connection

	  Connection closed by foreign host.

	  $> telnet ueamailgate02.uea.ac.uk 25

	  Trying 139.222.131.185...

	  Connected to ueamailgate02.uea.ac.uk.

	  Escape character is '^]'.

	  220 ueamailgate02.uea.ac.uk ESMTP Sendmail 8.13.1/8.13.1; Mon, 7 Dec 2009 01:45:49 GMT

	  quit

	  221 2.0.0 ueamailgate02.uea.ac.uk closing connection

About Me

I’ve been a Unix, Windows, OS X and Linux systems and network administrator for 15 years. I’ve compiled, configured, and maintained everything from mail servers to single-signon encrypted authentication systems. I run lines, build machines and tinker with code for fun. You can contact me via: lance@catprint.ca.

Lance Levsen,

December, 2009


Sponsored IT training links:

We offer 100% pass result in first attempt for all kind of IT exams including 70-685 and 70-271. Join 640-460 online course to save a big deal on real exam.


Advertisements

  Subscribe  
newest oldest most voted
Notify of
rokshox

It’s already been shown that the email filenames are derived from the unix ctime representation of the message send date.

James F. Evans

While it shouldn’t matter whether the e-mails and computer programs were “hacked” or leaked because it’s the underlying science that counts — it does matter politically.
Leaked material the result of an anonymous whistleblower (an anonymous whistleblower is contradictory) is more politically powerful.
As the leak would be the result of a troubled conscience from within the heart of the scandal — or is that heart of darkness…

geo

I agree that the compilation of this archive was more likely (maybe even *much* more likely) done internally than externally.
However, I don’t see why it follows that makes it more likely to be an internal leak *after* the archive was compiled. A hacker might still have stumbled on it. Or, even not a hacker. . . just someone external who found that file temporarily residing at an open ftp where it was put mistakenly (but with no intention to leak) by someone inside UEA.

Invariant

Sure. Sounds more like a undercover climate scientist from KGB or CIA.

Neo

The also leaves one other conclusion …
all the e-mails about AR4 that Phil Jones thought he deleted, most likely still exist.

chainpin

Wow, that is oustanding!
Very nice analysis, thank you.

Third Party

from http://thehill.com/blogs/congress-blog/energy-a-environment/70857-climategate-sparks-luetkemeyer-call-for-investigation-sparks-interest-in-legislation-rep-blaine-luetkemeyer
In 1890’s, Arrhenius built upon Fourier’s assessment of atmospheric properties plotting CO2 and temperature data collected in industrialized England. Arrhenius’ plots and calculations related CO2 and ambient temperatures.
Callendar (1930’s) extended the analysis using long term observations from 200 stations reiterating the relation between CO2 and climate warming.
Keeling (1950’s) began collecting atmospheric CO2 samples at Mauna Loa Observatory, Hawaii which is the most complete record. USGS reports all volcanic activity produces nearly 200-million tons CO2 annually; although much less than human activity production. Mauna Loa, near the Observatory and the world’s most active volcano, had major eruptions in 1950, 1975, and 1984. Atmospheric CO2 levels measured at volcanoes indicate the degree of activity and estimated heat flow from one volcano are reported at140-mW/m2.
Correlating CO2 and temperatures data collected near active volcanoes should be significant but not show a cause and effect relation; however, correlating world-wide data significantly shows CO2 lagging temperature by approximately two years. Arrhenius and Callendar analysis similarly could be significantly biased owing to urban heat-island effects and extensive coal burning at the time, as CO2 is an abundant byproduct of burning.
Apparently, no laboratory control experiment to date, such as in a biodome, has shown CO2 levels influencing ambient temperatures. Tyndall (1861) measured the absorptive characteristics of CO2 followed by more precise measurements by Burch (1970). Absorbance is a measure of the quantity of light (energy) absorbed by a sample (CO2 molecule) and the amount of absorbed energy can be represented as specific heat of a substance. Specific heat of CO2 ranges from 0.791-kJ/kgK at 0-degrees F to 0.871-kJ/kgK at 125-degrees F and average atmospheric concentrations are 0.0306-percent. As revealed, the specific heat of CO2 increases as ambient temperatures increase showing CO2 likely is an ambient temperature buffer.
The atmosphere contains from 4-percent water vapor in the troposphere to 40-percent near the surface. Specific heat of water vapor relatively remains constant at 1.996-kJ/kgK. Water absorbs energy (heat) and evaporates to water vapor. During condensation (precipitation) , latent heat is released to the atmosphere thus increasing ambient temperatures.
Water vapor holds the majority of atmospheric heat and regulates climate and temperature more than any compound. Historically, however, water vapor characteristics as related to climate were much less appreciated, but investigations concerning the significance water vapor plays in global climate-dynamics are just beginning.
Energy not stored in the atmosphere is released into space through radiation. Re-radiation is the emission of previously absorbed radiation by molecules. Specific heat of water vapor and CO2 molecules shows that water vapor reradiates significantly more energy back to the surface and this case further is justified by quantities of each compound.
Thus, this synopsis and other publications suggest that minute variations in atmospheric CO2 concentrations likely results in an insignificant affect on climate; whereas water vapor likely is the significant factor. Nevertheless, this argument easily could be rectified with an appropriate biodome-type control experiment.
BY ehmoran on 12/07/2009 at 10:42

None

How can someone who claims to be a unix system admin for 15 years not have realised the filenames were “seconds since epoc” dates ?

wws

The weekend cover story claiming that “The Russian KGB did this to discredit Copenhagen!!!” was a hoot.
Glad to see the good work done to knock down that nonsense quickly. Of course, that’ won’t stop the warmists from preaching the “Evil Hacker!” story for the next 20 years.

Sped

Nice work! Outstanding forensic type analysis of the info.
On a somewhat related note, Google is still hiding the autosuggest! Here is a thread to complain at, if you are interested:
http://tinyurl.com/yzssbsg which really is:
http://www.google.com/support/forum/p/Web+Search/thread?tid=25112ee0c29cbd01&hl=en&fid=25112ee0c29cbd0100047a24d00f3afd
And here is a recent story on Google handling of “climategate”
http://www.seroundtable.com/archives/021306.html

David Madsen

The analysis of the creation of the FOI2009.zip file is very intuitive. However is it plausible that a hacker managed to find the file on an internal FTP server as opposed to someone on the inside releasing it? In my opinion, this analysis doesn’t quite put the nail in the coffin on the hacker scenario.
I personally think that this was either a leak, or a hacker got lucky and stumbled across the FOI2009.zip file.

Viking141

Excellent analysis and torpedoes the “illegal hacking” sob-story that we’re being fed, just as I, and Im sure many others, suspected. Nice job!

Neo

The very easily could have come from within the CRU …

Lesson 1: Don’t let users put passwords in their signatures. Yep, you got that right: One of the scientists included both on his e-mail signature — which means that anyone receiving an e-mail from this guy had access to his files. This may have been the source of the hack; in fact, some folks have theorized that a recipient of the e-mail was the source of the data dump.

SJones

With regard to it being ‘stumbled upon’ – wasn’t it the case that is was released the same day as Steve McIntyre’s FOI appeal was turned down? If so, this was likely no accident.

The analysis is important because the dominant media narrative involves an attack by “Russian hackers”, which taints the release with motive.
Witness attempts to build on the “sinister motives” theme…
http://www.desmogblog.com/breaking-impersonators-attempt-access-canadian-government-centre-fo-climate-modeling-and-analysis
Apparently the incident wasn’t worth mentioning until now.

mojo

Yes indeedy. Never bought the “evil haxors” theory, Russian , Chinese or Martian. Too much connect time required, too much poking around, likely to set of IDM alarms – not a good prospect for a smash and grab.
Inside job. Said so the first day.

NK

Dear Mr Levsen–
Thanks for all this work. Very comprehensive and understandable. You wrote in conclusion:
Occam’s razor concludes that “the simplest explanation or strategy tends to be the best one”. The simplest explanation in this case is that someone at UEA found it and released it to the wild and the release of FOIA2009.zip wasn’t because of some hacker, but because of a leak from UEA by a person with scruples
I agree with the first clause, as that was my conclusion 2 weeks ago reading the file. The second clause is debatable. Once that file was compiled for FOIA purposes, it could be accessed by anyone with clearance, and that circle may have become pretty wide, FOI officials, University lawyers, tech staff etc. Anyone of those people, could have released the FOI file for various reasons. We can conclude with a high of confidence “WHAT” the file was, it due diligence in response to the FOI demand, but who released it and why, can’t be known on this info. I agree with GEO about that.

S

The explanation of the email numbers is far simpler than this. The email file numbers are standard UNIX epoch numbers. UNIX counts time as the number of seconds since Jan 1, 1970. Take any email filename and put the number into an epoch calculator such as the one on http://www.unixtimestamp.com/index.php and out pops out the email’s date, to the second, modulo time-zone/daylight savings issues which can cause the number to be off by an integral number of hours. There’s nothing special or deep about those numbers. They’re only a convenient way for whomever made the archive to assign unique filenames to each email, and a somewhat dangerous technique if two emails happen to have been processed in the same second.

Ron de Haan

Very good approach and very good arguments based on hard analyses and reasoning.
I am convinced it was a leak from the beginning and it should be clear to the world it was a leak.
It takes the wind out of the sails of those stating it was a common theft which allows then to smear the entire skeptical community as “thieves”.
What’s also important is that the BBC has been sitting on leaked information for weeks without undertaking any action.
I am sure the momentum caused by ClimateGate will continue for months to come as more and more facts and analyses will surface and the truth becomes known by the public.
Most of them already new AGW was a scam. Now they know for shore.
Nothing criminal about that.
Thanks for a job well done.

Rhys Jaggar

There is little doubt, however, that Russian figures in the UK gain unauthorised access to contents of PCs which have links to the internet.
I remain 100% confident that Chelsea FC have in the past gained access to documentation stored on my PC and released to no-one and that their performance of the hokey-cokey on the Old Trafford pitch prior to a game with Manchester Utd might be the smoking gun to demonstrate it. The document in question was a spoof Ali C conversation with recently departed Chelsea FC manager, Mr Jose Mourinho……for the record, my PC at the time was in Leeds, Yorkshire, around 200 miles away from Mr Abramovitch’s home. Ditto his football club’s home address.
Not that this is peculiar to Mr Abramovitch’s club. Far more likely that the surveillance comes from others who pass it on or sell it on for money. Mr Murdoch’s tabloid executives no doubt value such approaches, particularly in concert with Her Majesty’s constabularies of various districts…..
I wonder how many journalists are drinking beer in Norwich watering holes trying to identify Deep Throat right now?

I think Harry the programmer is the source.

Clive

Lance Levsen,
Thanks for the analysis. Not read all, but the documentation is very good as is the summary.
Thank you.
Clive

D. King

Neo (08:49:32) :
The also leaves one other conclusion …
all the e-mails about AR4 that Phil Jones thought he deleted, most likely still exist.
Good point.

Gene Zeien

Well done. Many have speculated about a insider leak, but this is as close to proof as one can get without access to the UEA computers’ logs. Even then, few civilian research centers track incoming/outgoing ftp transfers. Fewer still log USB disk connects or transfers. It would be relatively straightforward for an insider to copy FOI2009.zip onto a USB key, visit the local library & upload the file anonymously to the russian ftp site.
However, while I agree the compilation of FOI2009.zip was very probably an inside job, an outsider may have found a copy that the staff had dumped on a local ftp server for transfer to a coworker. Of course a Windows shared folder with guest access allowed would suffice, as well 😉

boballab

Interesting.
Here is more food for thought: Remember back when this all started we knew it came from a Russian Proxy server and they posted the link to the Air Vent. Then Gavin starts running around about being hacked and that they tried to upload things from a servit in TURKEY. Now we got this thing up in Canda and the some press reading from the UN’s press release that its the Russin’s because it came froma Russian Proxy server.
Notice how that “hack” into RC from a server in Turkey just dissapeared, like Briffa’s divergence?

Henry chance

The warmists are superstitious ..Joe Romm claimed global warming caused an airplane crash before they even found where it went down. They are calling this theft and haven’t investigated how the data came out so well organized.

patrick healy

what a tremendous piece of work, lance.
as an amateur it looks quite plausable to me.
now all we are waiting for is the one person at CRU with dignity to show themselves. I can hear the whistle blowing already.
I wonder if Sir ‘wottisname’ will call Lance as an expert witness for his internal enquiry.

AlanG

Nice analysis. Definitely an internal job. CRU isn’t a big organization like a bank where security is ultra tight. There would be students running around the place. Some of the IT students might even be given admin rights to look after the servers as part of their learning.
My guess would be someone, either the FOI guy or a student, was asked to do a trial cut for an FOI request. Did a hacker find it on an ftp server or did a student push it out? My guess is the second. Wasn’t there something about procedures at UEA and/or CRU being tightened up in September to control student access?
The IPCC has got the wrong end of the stock completely and are going with the break-in scenario like Watergate. Pathetic:
Climate email theft likened to Watergate break-in:
http://www.theaustralian.com.au/news/climate-email-theft-likened-to-watergate-break-in/story-e6frg6xf-1225807887910

JonesII

Then….the whistleblower will probably attend Copenhagen…he/she is already there!…
Let´s get more popcorn!

AndrewWH

Now I am wondering if the mail servers have some or all of the original raw data on them. Granted, CD burners have been around a long time, but for ease surely some of the data could have just been emailed to Harry once it was tabulated so he could work on it, especially if the files are not too big. Then it may still be in existence.

bill

It was obviously a FOI collection – Emails are not held in clear form on a server A password is required to unscramble them (not very secure). These have all been saved as text by the owner as a result of FOI.
However:
Inside job/ outside job it all falls within the the Computer misuse act. UNLESS you have authorised access to the data:
http://www.opsi.gov.uk/acts/acts1990/UKpga_19900018_en_1.htm
(1) A person is guilty of an offence if—
(a) he causes a computer to perform any function with intent to secure access to any program or data held in any computer;
(b) the access he intends to secure is unauthorised; and
(c) he knows at the time when he causes the computer to perform the function that that is the case.
(2) The intent a person has to have to commit an offence under this section need not be directed at—
(a) any particular program or data;
(b) a program or data of any particular kind; or
(c) a program or data held in any particular computer.
(3) A person guilty of an offence under this section shall be liable on summary conviction to imprisonment for a term not exceeding six months or to a fine not exceeding level 5 on the standard scale or to both.

“S/HE” [snip]!!!!!
Brave enough to cock a snook at the entire Warmist conspiracy but you fold like a cheap suit before the Pronominal Equity Police. Sheesh!

hunter

boballab makes a great point.
RC/Schmidt are deliberately confusing this issue.
The Russian hack cover is so disengenuous as to be laughable.

PhilW

How damaging is “Climategate” to NASA?
http://www.thespacereview.com/article/1522/1

Steve

Regardless, the MSM will continue calling this a “hack,” even though that can’t be confirmed and it even seems to be otherwise.

Anand Rajan KD

For those suggesting that the FOI2009 Zip was created previously and left on a server, and also for those suggesting that it might have done as part of a FOI request, I have questions:
1) UEA and the Team had no intentions of complying with the requests as the emails show. Why would they have zip files ready, as if to suggest a ‘just in case’ action on their part? They did not want to do it, and believed they had legal cover (however flimsy) to go down that path.
2) Why would anyone put pictures of scientists floating off on ice floes with polar bears as part of a FOI response?
The selection of emails also suggest someone knew the significance and import of what was being talked about in the emails. We must not forget that there was an attempt to upload/link to these files to RealClimate.
FOI2009.zip was created after files were collated, probably from several computers. Files were then probably carried out on a thumb drive, physically.
Assuming UEA does not have a fair idea of who leaked the files, one can also extrapolate that a server/s with the emails is not under camera surveillance. We have to take this into account given the fact that the UK is riddled with cameras.
It would also be nice to believe that it is one of the Team who actually is responsible for the leak.

Google is working just fine for me autosuggesting “climategate” when I type “clim”.

Chris Schoneveld

Why was the file first sent to a BBC reporter and not to one of the Telegraph?

crosspatch

I believe that the numbers of the filenames correspond to the order that the emails were archived. If so, the numbers that are missing, represent other emails not captured in FOIA2009.zip.

No, the filenames of the emails are the unix epoch time of the email. The missing numbers are elapsed time between the emails. The filenames are the datestamps. I believe this is the date when the file was archived as there is generally a couple of hours difference between that date stamp and the date/time stamp in the headers of the email.
For example:
1123622471.txt
1123622471 is unix time August 9, 2005 21:21:11 GMT
The date/time stamp on that email is:
Date: Tue Aug 9 17:21:11 2005
Notice the hour is different but the minutes/seconds are the same.

Ken

Hmmm.
Presumably you think the disclosure was a good thing.
Now you’re explaining how it may have been done by an insider who would have had to have particular access capability — going so far as to compile a pattern that points to very very few possible candidates.
Which means you’re helping narrow down the possibilities of the Source — helping the people that suppressed this release [via FOI]!
Which means you’re making it that much harder for that person to
– Get away with it
– Disclose any additional information of significance
– Etc.
All of which is very bad–from your perspective.
As anyone that’s read a spy novel knows–ALWAYS PROTECT SOURCES & METHODS!!!!!
Recall the Hansen case involving the FBI informant to the Russians — they likely weren’t sure who he was…but they sure didn’t screw up a good thing (for them) by disclosing their inferences either.
An article like this sure makes you look clever, but in the aggregate, why help the people that ‘acted badly’? Let them figure it out, or not–certainly don’t help them! Doing so only discourages others, or the same person, in a similar situation from coming forward.

boballab

Here is something else to think on:
Notice: that all the “its the Russians” that did this, All track back to the Frenchmen from the UN waving his hands and saying it was so. It started about 2 days ago in the Daily Mail in the UK and they quoted it as such. Now you get the same story but it is now an “un-named source from the UN”
Notice: there has been no statements from the Local police who are investigating the release from UEA. If they had narrowed it down to foriegn hackers the case would have been turned over to the UK version of the FBI and you would have seen headlines of that fact. Instead as far as the world knows the local police are still investigating. Local cops investigate local crimes not international ones.
Notice: couple of days ago While I was surfing around I saw one article in a UK paper that quoted an official from Scotland Yard that they were now investigating the CRU for FOI violations. Nothing spread from that it has dropped into a pit of silence.
Just some musings from someone that just so happens to have a busted leg and is laid up for awhile at the right time for all this.

Skeptic Tank

At this point, I’m so cynical and suspicious of these people, I wouldn’t be surprised if they already have identified the whistle-blower, but they won’t divulge the person’s identity. They want to maintain themselves as the victims.
What if that person were identified and disciplined; fired or even perhaps prosecuted. What a source of information they would be to the public. What a source of testimony they would be in related proceedings and investigative hearings.
Oh, they wouldn’t want that.
This isn’t over. I just hope the person in not in danger.

crosspatch

There is a unix time conversion utility here:
http://www.csgnetwork.com/unixds2timecalc.html

Third Party (08:52:03) :
The atmosphere contains from 4-percent water vapor in the troposphere to 40-percent near the surface.
Get the numbers straight. At 100% relative humidity at 30C [tropics] the concentration of water vapor is 30 gram/cubic meter. Considering that 1 cubic meter of air at the surface weighs 1234 gram, the water concentration can at most be 30/1234 = 2.4%.
Your [or your source’s] numbers are 10-20 times too high.

Svein

Darrell S Kaufman was the moron who had his user id and password in his email signature.

Ed Scott

The IPCC is irredeemably (no “hope” for “change”) corrupt. The exposure of the truth is a “crime” in the new world of politicized science. The goal of this new “science” is to lie, cheat and steal with the end justifying the means.
The following four reports, via CNN, are interspersed by commercials, so hang-in.
http://us.cnn.com/video/?/video/world/2009/12/07/sot.climate.ipcc.ebs
The head of the UN climate change panel (old friend “no meat” Pachauri) discusses ‘climategate’ and how it’s trying to undermine the group’s findings.
Scientists are accused of ‘cooking the books’ on climate change. CNN’s Jim Acosta reports.
Danish Prime Minister Lars Lokke Rasmussen starts the UN Conference on Climate Change.
CNN’s John Roberts looks at the effect hacked emails may have on the U.N. Climate Conference (Peter Liss, Phil Jones’ replancement, appears to be more of the same).

kadaka

Given the revealed high quality of work at UEA/CRU, I wonder if this FOIA compilation was simply left on an anonymous server, awaiting the permission of management to release the address as compliance with the FOIA request. Then someone stumbled upon it…

Great job, Charles. Now, if Babs Boxer can pull back the blinders and see this for what it is.
Unfortunately, these politicians are set in their views because cap and tax is a potentially new revenue stream.

Michael

“It is the Sun that determines the climate of the Earth, not CO2, and the Sun is in a natural cycle called a solar minimum, producing less radiation to warm the Earth.”
How to Destroy the U.S. Economy: Regulate Carbon Dioxide
http://canadafreepress.com/index.php/article/17657

maz2

Goreacle Report: The Gore Effect.
The Extortionists Duke It Out.
Big State vs Big Criminals.
…-
“The Climate Mafia
Fraudulent Emissions-Trading Schemes Rob German Tax Authorities
The Kyoto Protocol introduced a scheme for trading emissions certificates as a way to help reduce CO2 emissions. German tax authorities are now investigating almost 40 companies that traded certificates for allegedly taking advantage of loopholes in sales tax laws to bilk the taxman out of hundreds of millions of euros.
German Environment Minister Norbert Röttgen has hardly been in office for much more than a month, but he’s already choosing his words for dramatic effect. “It’s about the way we live, and it’s about survival,” Röttgen said last Thursday before the German parliament, the Bundestag, referring to the climate summit beginning Monday in Copenhagen. At the summit, the nations of the world will search for ways to reduce the CO2 emissions behind global warming. One of the tools to be discussed is the trading of emissions certificates.
In Germany, though, it is precisely this instrument that is causing a huge headache for Röttgen, as dozens of tax offices across the country are investigating shady emissions trading companies. All of the companies in question maintain accounts with the German Emissions Trading Authority (DEHSt), an arm of the ministry Röttgen heads. According to DEHSt head Hans-Jürgen Nantke, since September, his agency has “received official requests for assistance in cases relating to suspected sales tax fraud from various regional tax offices and tax investigation offices.”
The authorities are investigating questionable transactions in Germany’s central emissions trading registry. As in any other EU country, public utilities and industrial enterprises maintain accounts in the registry, which provides them with a certain number of emissions credits. Under the 2005 Kyoto Protocol, the emissions of climate-harming carbon dioxide are to be reduced as efficiently as possible with the help of this method of trading emissions certificates. According to this system, companies that invest in new, eco-friendly technologies no longer need all of their certificates and can, in turn, sell them to others — and at a steep price.
Current prices for these certificates are set on exchanges, such as France’s Bluenext, the ECX in London, the EEX in Leipzig and, more recently, the Greenmarket segment of the Munich Stock Exchange. At the moment, the right to emit one ton of CO2 into the atmosphere is worth about €14 ($21). In the first half of 2009, the volume of trade in emissions certificates in Europe alone already amounted to €40 billion.
Going after the Climate Speculators
The high-stakes deals being made with certificates are attracting more and more speculators. While investment banks, such as Goldman Sachs, and US-based hedge funds are speculating on emissions certificates, small trading companies are now trying to bilk tax authorities out of the sales tax that is part of many of these certificate-trading deals. Tax authorities are investigating “an estimated 30 to 40 companies,” says Nantke, adding that some of these companies are based abroad.
Investigators are apparently also interested in bank transactions.” (more)
http://www.spiegel.de/international/business/0,1518,665594,00.html