Decrypt encrypted files with OpenSSL?

[Pieter Edelman]

Hello,

I feel quite stupid to ask this, but I can't really figure this one out. I'm doing some disaster planning for my personal data and part of it is to store an off-site version of my backup on Amazon S3. I really like this data to be encrypted on S3 but decrypted on the local side, and for that reason I'm interested in Jets3t Synchronize, which has this feature.
However, I don't want to rely on any specific software for restoring my data and so I am trying to find a way to decrypt the Jets3t encrypted files with a different tool, in this case, OpenSSL (because it is well-supported and most likely to be available when a data recovery might be needed). The problem is, I don't succeed in decrypting files.

Here's my test case:
- I enable encryption in Cockpit, password "test" and with algorithm PBEWITHSHAAND256BITAES-CBC-BC
- I upload a small "test.txt" file using Cockpit
- I download this file again using Amazon's AWS Management Console. Indeed it is an encrpyted file.
- I try to decrypt it with OpenSSL:
> openssl enc -d -aes-256-cbc -in test.txt -out test-decrypted.txt -k test
This gives the error: "bad magic number"

I tried various options to OpenSSL, like "-md sha" (to make sure the key/iv pair is generated with SHA), -nosalt, and -nopad, or combinations of these options. However, I either get the message "bad magic number", an error message or a nonsical decrypted file.

I don't know if there's anything specific to Jets3t or the JCE that prevents OpenSSL from decrypting the file, or that it is an OpenSSL issue. I hope somebody more knowledgeable than me can help me out.

Thanks,
Pieter

[James Murty]

Hi Pieter,

The easiest way to decrypt JetS3t-encrypted data without relying on
specific programs is to use the EncryptionUtil class directly, such as
by using its decrypt() method to wrap an output stream.

To be honest I'm not sure what would be the right OpenSSL incantations
to decrypt one of these files. The "recipe" used by JetS3t to generate
the encryption keys etc is expressed in code in the EncryptionUtil
class, but I didn't find the OpenSSL command analogues for these steps
when I tried some experimentation.

There should be a way to use OpenSSL to generate custom Key and IV
values from a password using the same salt and iteration settings as
those in EncryptionUtils, then apply these to decrypt a
JetS3t-encrypted file.

If you figure out the recipe I would be interested to know, but I'm
afraid I don't have any solid leads.

James

> --
> You received this message because you are subscribed to the Google Groups
> "JetS3t Users" group.
> To post to this group, send email to jets3t...@googlegroups.com.
> To unsubscribe from this group, send email to
> jets3t-users...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/jets3t-users?hl=en.
>

[Pieter Edelman]

Hi James,

Thanks for the pointer to the appropriate class. It took me quite a
while, but I managed to get it working, although it's not very pretty.
It turns out the problem is not so straightforward, for two reasons:

1. Differences in key generation between JCE and OpenSSL.
I have no idea what's the reason for this, but if I use JCE and
OpenSSL both generate the key and IV from the same password and salt,
they results differ (it could be the iteration count which apparently
is always in OpenSSL, but when using 1 as the value in JCE, I still
got different results).
In the list of available encryptions methods in Jets3t though there
are three methods ending in "-OPENSSL" (PBEWithMD5And128BitAES-CBC-
OPENSSL, PBEWithMD5And192BitAES-CBC-OPENSSL and PBEWithMD5And256BitAES-
CBC-OPENSSL). When using one of these, the key and IV are the same.

2. OpenSSL expect salt prefix.
OpenSSL transfers the salt along with the encrpyted message, in the
form of a prefix with string "Salted__" followed by the salt (thus the
file becomes 2 bytes larger). When decrypting, OpenSSL expects the
salt to be present, but Jets3t doesn't write this prefix. The solution
here is to specify the actual key and the IV on the command line, in
which case OpenSSL stops looking for the salt prefix (specifying the
salt with the -S option has no effect).
In commands (assuming that 256 bit keys are used, otherwise substitute
for the appropriate key length in the -aes[...] option):
- First figure out the key and IV, using the Jets3t salt (this will
prompt for the password you used to encrypt the files)
> openssl enc -aes256 -S A40BC834D695F313 -P
- And then decrypt your files
> openssl enc -aes256 -d -K [key from command above] -iv [iv from command above] -in [encrypted file] -out [decrypted file]

Pieter

On Dec 16, 5:21 am, James Murty <jamu...@gmail.com> wrote:
> Hi Pieter,
>
> The easiest way to decrypt JetS3t-encrypted data without relying on
> specific programs is to use the EncryptionUtil class directly, such as
> by using its decrypt() method to wrap an output stream.
>
> To be honest I'm not sure what would be the right OpenSSL incantations
> to decrypt one of these files. The "recipe" used by JetS3t to generate
> the encryption keys etc is expressed in code in the EncryptionUtil
> class, but I didn't find the OpenSSL command analogues for these steps
> when I tried some experimentation.
>
> There should be a way to use OpenSSL to generate custom Key and IV
> values from a password using the same salt and iteration settings as
> those in EncryptionUtils, then apply these to decrypt a
> JetS3t-encrypted file.
>
> If you figure out the recipe I would be interested to know, but I'm
> afraid I don't have any solid leads.
>
> James
>

[James Murty]

Hi Pieter,

Thanks very much for figuring this out and documenting it. I was able
to follow the steps you outlined and I'll repeat them here in detail
in case others find this useful.

Nice work!

To encrypt data with JetS3t, then decrypt it with OpenSSL:

1) Make sure you use one of the *-OPENSSL PBE cipher algorithms that
should be available in JetS3t. To list all the available algorithms on
your system, run the EncrytionUtil class
(org.jets3t.service.security.EncryptionUtil) from the command line
with all the required libraries in your classpath.

Example PBE cipher algorithms that OpenSSL can decrypt (from OSX with
Java 1.6 and BouncyCastle lib):

PBEWITHMD5AND192BITAES-CBC-OPENSSL
PBEWITHMD5AND128BITAES-CBC-OPENSSL
PBEWITHMD5AND256BITAES-CBC-OPENSSL (* this is the algorithm I used
for the example commands below)

2) Download the raw JetS3t-encrypted file from S3 using a tool other
than JetS3t's apps (which will automatically decrypt it). For example,
use Amazon's own console or curl/wget with a temporary signed URL.

3) Use OpenSSL to convert your password into the Key and
Initialization Vector (IV) values necessary to decrypt the file. Here
is an example command, and output, to generate these values for
JetS3t's salt value and the password "password":

$ openssl enc -aes256 -S A40BC834D695F313 -P -k password
salt=A40BC834D695F313
key=E0499E736012D566B475A7224E34902D5B3EDFDD48C83EA741FBE24D4A3880D0
iv =3DF92E1C79A672F40ADA758937A3DD04

4) Provide the Key and IV values generated above to OpenSSL to have it
decrypt the file 'data.enc':

$ cat data.enc | openssl enc -aes256 -d \
-K E0499E736012D566B475A7224E34902D5B3EDFDD48C83EA741FBE24D4A3880D0 \
-iv 3DF92E1C79A672F40ADA758937A3DD04

NOTE)

JetS3t's applications and tools store the cipher algorithm for each
encrypted object in the "jets3t-crypto-alg" metadata item, so it's
easy to tell which algorithm was used later on. This makes it possible
to automate decryption, even if you use different algorithms over
time. However, this won't help if you use different passwords...

[Pieter Edelman]

One small correction: I wrote that the salt predix is 2 bytes, but it
is 16 bytes of course ...