There’s an embedded archive file called all.7z which contains thousands of additional emails and files.
The 7zip archiver in which this is stored uses 256 bit AES encryption. It’s a tough nut to crack.
“FOIA” chose this most likely because there are no effective tools for 7zip, while there seem to be many for standard .zip and .RAR files.
From their website: http://www.7-zip.org/7z.html
“7-Zip also supports encryption with AES-256 algorithm. This algorithm uses cipher key with length of 256 bits. To create that key 7-Zip uses derivation function based on SHA-256 hash algorithm. A key derivation function produces a derived key from text password defined by user. For increasing the cost of exhaustive search for passwords 7-Zip uses big number of iterations to produce cipher key from text password.”
The password can be 2047 or 8191 characters long, depending on your operating system.
I’m doubtful this password will be cracked anytime soon, maybe DoD could do it. Chances are that “FOIA” chose a very long password, that could take years to crack by a brute force attack.
“FOIA” is holding this in reserve, making it known that it is there, ready to pull the firing pin. I expect we’ll see it sooner than later as the reaction so far from RC and the Team is continued arrogance.
Julian Williams in Wales has an interesting take:
Maybe the passphrase is so complex to be uncrackable; is that possible? Surely after having sat on this material for two years FOIA would have made a decision how he is going to play this, and it just makes no sense to put most of the material behind a crackable passphrase.
But supposing he then sent the passphrase to Phil Jones and M Mann with a threat; Resign now, get the hell out, otherwise this passphrase goes online to the general public. That is a strategy that might push FOIA’s enemies out without completely disgracing the “scientific community”
Just another way of looking at what might motivate FOIA.
“Open Sesame”?
FOIA has said that he/she/it [delete wichever is inapplicable] has said that they will not be releasing the pass phrase. Implication: it’s guessable; my guess is that we’ll all slap our foreheads when it’s revealed. Also, it will be something that you don’t have to remember; the Internet will “remember” it so you can copy and paste the passphrase from a Google search. Think of something like the first paragraph of the US Consitution. This is a game that keeps the story newsworthy. And it’s fun 🙂
I bet Igor Pavlov has cracked it !
Developers always have a “secret spanner” way-in their own creations !
The UK Met Office, traditionally funded from the Ministry of Defence budget, has the computing power for a brute force crack. Perhaps UK weather forecasts will now begin to show an improvement in accuracy.
To you Brute Forcers out there…you’ll want to include the Cyrillic alphabet in your characters set.
There are no back doors or secret passwords for 7-zip.
Using rarcrack to get the password is a waste of time.
The password for 7-zip is hashed using SHA-256. This hash is then used to encrypt the file. If you are going to do an exhaustive search then you can skip trying every password and instead try every hash. This reduces the search from upto 2047 or 8191 characters to 32 bytes (BYTES not characters). Assuming that testing each possible one takes 1 cycle and that you are using a 100GHz processor and there are 1 million of you trying it, it will take you approximately 36,200,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000 years.
Clearly an exhaustive search is out of the question.
The only hope is that the password is based on a word. For example, if you thought the password was “sunshine” then you might try:
sunshine
5un5h1n3
SunShine
Sunsh1n3
etc.
There are over 1000 different ways to try the word sunshine.
The Oxford English Dictionary lists over 250,000 words so if we take that list and generate 1000 variations for each word then we get 250,000,000 words to try. I haven’t extracted the SHA code from 7-zip yet and benchmarked it, but assuming a desktop computer can test 1000 passwords a second then it would take approximately 70 hours of computing time to test all those possibilities.
If they used a 2 word phrase such as “global warming” then there are 250million squared possibilities. This would take 1 person 2 million years to crack. Unfortunately, there are many ways of combining 2 words, such as:
global warming
globalwarming
global_warming
global+warming
global-warming
etc
Testing each of these would take 2 million years.
In conclusion, the only realistic hope you have of cracking the password is if they have used a single word as the password. At some point in the future I will probably extract the relevent code from 7-zip and “John the Ripper” and attempt to crack the password using a dictionary attack.
I am an analyst/programmer with a long history in computing and agree with the general sense here that if the pass phrase has been constructed properly it is very unlikely for it to be cracked. What I am curious about is since this thread has attracted many with knowledge about encryption is a point about history. There was a time when there was much talk about security organisations insisting on a back door into any encryption method. The UK even went so far as to pass a law that UK citizens must produce passwords on demand. So does anyone know if these things were done or was it all nosense in the first place?
Hi Terry, that is great, but in 2 million years these emails will have far less impact. We are going to need a different approach.
I use BitLocker, available with Windows 7 Ultimate, to encrypt my drives. There is no backdoor. If I forget the password, and I don’t save the key somewhere, the files are unrecoverable.
Re: MikeO
You are thinking of the Clipper Chip. This was to supposed to be used for secure communications and used a key to decrypt the communications. The US government, however, held copies of all the keys so would have been able to listen into the communications.
Another one you might be thinking of is PGP (Pretty Good Privacy). The US government classified this a munitions because of the key length and banned its export from the USA. The authors got around this by publishing a book with all the source code printed in it and legally exporting this. The book was then scanned using an OCR and the program recreated. I think, but I’m not sure, that France also banned the import of encryption.
@TerryS
Intels Knights Corner can do more than 1TFLOPs of double precision calculations, how does that affect the time line to break the encryption.
There is a 10% chance of breaking the password in one tenth of your calculated time frame.
Dishman
I think Dyspeptic Curmudgeon is on the right track, the interesting thing will be matching the released emails (and time stamped context) with those that survived in Michael Mann’s email list, for forensic examination – might be very revealing as Mann seems to outsmart himself sometimes, much to the chagrin of the remaining ragged team of followers !.
Weather [pun] they will suddenly abandon him, if he has left himself open by selectively editing of those remaining emails, will be quite telling. Human trait is to abandon excess baggage to contain a crisis!! (or divert flack elsewhere by proclaiming the target!!)..
Incidently I rather like the discussion on ethical conduct proposed for study by science students taking place now at Judith Curry’s site, The posts by the “defenders of the warming faith” have become quite hysterically comedic as they try to defend the undefendable – bias of the trolls or last tango before Durban!!
Remember that screensaver that was processing signals from outer space. People downloaded it and it ran in spare cycles on their computers. Could someone put together a Massively Multi Something’d project to have 100,000 people download the software and try to break it by brute force.
one possibility of getting into the file is to use cloud computing resources … you can rent an impressive amount of processing power and resources these days.
… or pose it as a challenge to Watson, IBM’s latest supercomputing toy.
I think FOIA may have a plan that I haven’t seen discussed. Whoever has the original e-mail files can crack the code by using those e-mails against the files. If you can match up the original to the encrypted data it is much easier to crack. However this may be a trap. FOIA may be hoping to get them scrambling to figure out what else is in there and in doing so allow more information to “escape”. Or there may be a trojan inside the zip file designed to crack these systems from the inside.
Re: Gary Mount
The SHA hashing and AES encryption both use integer operations, not floating point.
Even if you increase the processing power a 1000 fold you are still looking at a couple of thousand years to try all 2 word combinations. Once you start on 3 word combinations you are looking in the order of 500 trillion years.
That assumes that the password is a modified English word and not a German, French, Russian or even a random selection of characters.
Open Sesame!
Re: Disbeliever says:
There are 244 files in the encrypted all.7z that also appeared in the original Climategate 1.0 version.
You can not take the encrypted file and the unencrypted file and then derive the key from the two of them. Having an unencrypted copy will mean that you don’t have to decrypt an entire archive entry to test the password, just the first 16 bytes.
Hercules?
After all, he slew Cerberus, the multi-headed hellhound who guarded the gates of the Underworld.
If I had the pass-phrase, what would it be worth?
… and to whom?
… and would that be in unmarked US dollars?
Just to confirm, the password given above is correct: “A miracle has happened.” – case sensitive, with the full-stop(/period), without the quotes.
Even assuming the NSA doesn’t have a backdoor in 7zip (which is probably a bad assumption), their algorithms could crack that key in hours or minutes if it is anything based in a language. Brute force isn’t really relevant. Real secure keys use truly random phenomenon like atmospheric changes to generate a one time pad, but that seems unlikely here. Unfortunately we don’t have access to the 50 acres of processors the NSA uses.
Maurizio Morabito (omnologos) says:
November 22, 2011 at 4:32 pm
“The technical details might be revealing on who’s behind FOIA.org. How many interested parties knew of 7z at UEA?t”
7zip is pretty popular. I use it frequently for sending dll and exe files via email. Most email providers, especially big free ones like hotmail, gmail, and yahoo block the transmission of attachments containing executable files for security purposes. Zipping an .exe file doesn’t help as the email providers can and do examine the contents of zip files. They don’t have algorithms to look inside 7zip files so it’s easy to use that instead of plain zip. 7zip also gets hellaciously better compression rates which is icing on the cake. It’s also open source, with an SDK, so no one ever has to pay for it and you can customize it and/or bundle it into applications of your own.
Given its popularity among the more computer literate and everyone in the open source community the answer to your question is that pretty much anyone at UEA might know about it.
Mark Buehner says:
November 23, 2011 at 6:49 am
“Even assuming the NSA doesn’t have a backdoor in 7zip”
That’s a pretty safe assumption since 7zip is open source.