OpenSSL::Cipher decrypt returns 'wrong final block length'
Ram
it to the DB. And then decrypt it again when I need to retrieve and
use it.
Im trying to use the OpenSSL::Cipher library. I have the following
module for encryption/decryption
[code]
require 'openssl'
module AESCrypt
# Decrypts a block of data (encrypted_data) given an encryption key
# and an initialization vector (iv). Keys, iv's, and the data
# returned are all binary strings. Cipher_type should be
# "AES-256-CBC", "AES-256-ECB", or any of the cipher types
# supported by OpenSSL. Pass nil for the iv if the encryption type
# doesn't use iv's (like ECB).
#:return: => String
#:arg: encrypted_data => String
#:arg: key => String
#:arg: iv => String
#:arg: cipher_type => String
def AESCrypt.decrypt(encrypted_data, key, iv, cipher_type="aes-256-
cbc")
aes = OpenSSL::Cipher::Cipher.new(cipher_type)
aes.decrypt
aes.key = key
aes.iv = iv if iv != nil
aes.update(encrypted_data) + aes.final
end
# Encrypts a block of data given an encryption key and an
# initialization vector (iv). Keys, iv's, and the data returned
# are all binary strings. Cipher_type should be "AES-256-CBC",
# "AES-256-ECB", or any of the cipher types supported by OpenSSL.
# Pass nil for the iv if the encryption type doesn't use iv's (like
# ECB).
#:return: => String
#:arg: data => String
#:arg: key => String
#:arg: iv => String
#:arg: cipher_type => String
def AESCrypt.encrypt(data, key, iv, cipher_type="aes-256-cbc")
aes = OpenSSL::Cipher::Cipher.new(cipher_type)
aes.encrypt
aes.key = key
aes.iv = iv if iv != nil
aes.update(data) + aes.final
end
end
[/code]
And here is the model code where I encrypt and decrypt the string.
[code]
def encrypt_cc_pass
return if cc_pass.blank?
self.cc_pass_key = Time.now.to_s
self.cc_pass_iv = Date.today.to_s
self.encrypted_cc_pass = AESCrypt.encrypt(cc_pass, cc_pass_key,
cc_pass_iv)
end
def decrypted_cc_pass
AESCrypt.decrypt(encrypted_cc_pass, cc_pass_key, cc_pass_iv)
end
[/code]
And this is the error I get
wrong final block length in:config/initializers/aes_crypt.rb:20:in
`final'
Anyone know what im doing wrong here?
Also, Im not able to understand the inners of encryption here. Is
there any gem with good documentation or simpler usage?
Michael Graff
You are dabbling in crypto and you don't know what you're doing.
Stop, and find someone who does.
From what I read, you're trying to encrypt credit card data. Before
you leak all the data from your database by using the _current time_
-- one of the weakest possible keys and IV values out there -- talk to
someone, hire a consultant, or at least read 'applied cryptography.'
>> AESCrypt.encrypt("this", "X" * 32, "I" * 32)
=> "\223\335@\233\323d\254u9]\241\351\031M\301\352"
>> AESCrypt.decrypt("\223\335@\233\323d\254u9]\241\351\031M\301\352", "X" * 32, "I" * 32)
=> "this"
Your code works. Your lack of understanding about the algorithm you
are using, the meaning of a key in that context, how to securely
generate one, and how important a secure IV is, has bitten you.
Storing the key, IV, and encrypted data in a database is exactly the
same as storing the unencrypted data. It will stop casual browsing.
If that is what you are trying to do, there are far easier methods,
such as converting the whole thing into base64 or hex or something.
If you believe your code actually adds security...
--Michael
Ram
I do agree with you in that I really have little to no idea of
cryptography. Think I mentioned that in the OP itself. And I simply
left out the complex IV and key for the sake of making this post a
little more readable. Also, I have been reading up a bit and I hear
the same of what you say in that its safer to store the iv and key in
a database of their own. Thanks for that..
Well, the point though is that im getting an error that I dont have a
solution for. And I think im giving bad input to the encrypt and
decrypt methods and thats exactly what im trying to get someone to
bring me up to speed with. Even in your example, im guessing you're
generating a 32 bit something....? Well the real key and iv in my code
are generated like this
Digest::SHA1.hexdigest("whatever").unpack('a2'*32).map{|x| x.hex}.pack
('c'*32)
Now I DID pick this up from the net so im not completely sure what
this does either.
I would appreciate it if you could establish the basics that I need to
know here.
Ive been trying to find relevant ROR specific documentation for the
OpenSSL gem but none that are understandable so far. Will look for
'applied cryptography'.
Thanks for the response.
On Feb 26, 4:16 pm, Michael Graff <skan.gryp...@gmail.com> wrote:
> I'm going to seem rude probably, but it's still true.
>
> You are dabbling in crypto and you don't know what you're doing.
> Stop, and find someone who does.
>
> From what I read, you're trying to encrypt credit card data. Before
> you leak all the data from your database by using the _current time_
> -- one of the weakest possible keys and IV values out there -- talk to
> someone, hire a consultant, or at least read 'applied cryptography.'
>
> >> AESCrypt.encrypt("this", "X" * 32, "I" * 32)
>
> => "\223\335@\233\323d\254u9]\241\351\031M\301\352">> AESCrypt.decrypt("\223\335@\233\323d\254u9]\241\351\031M\301\352", "X" * 32, "I" * 32)
>
> => "this"
>
> Your code works. Your lack of understanding about the algorithm you
> are using, the meaning of a key in that context, how to securely
> generate one, and how important a secure IV is, has bitten you.
>
> Storing the key, IV, and encrypted data in a database is exactly the
> same as storing the unencrypted data. It will stop casual browsing.
> If that is what you are trying to do, there are far easier methods,
> such as converting the whole thing into base64 or hex or something.
> If you believe your code actually adds security...
>
> --Michael
>
Michael Graff
AES-256-CBC means this: AES algorithm. Block size 256-bits.
Cipher-block-chaining.
What this describes is a system where your data is encrypted in a
256-bit block, which GENERALLY means you need a 256-bit key. The
documentation for the Ruby OpenSSL system is, as near as I can tell,
non-existent. However, looking up the openssl library for C will tell
you a lot -- Ruby just calls the C library your system has installed.
In this case, AES-256-CBC requires a 256-bit key and IV. (IV is
"initialization vector" and is generally a random value. Its purpose
is to make the same plaintext encrypt to different cyphertext.)
In my example, I generated 32-character long strings using the format:
"X" * 32. This
will make a string with 32-X's in it. Probably not all that secure,
but works for an example. :)
If you are running on a Unix system with /dev/random, and it has
sufficient "real" randomness, you can use it to generate your IVs, and
in your implementation, keys.
File.open("/dev/random") do |rnd|
key = rnd.read(32)
iv = rnd.read(32)
end
If you are unable to use /dev/random, you could use /dev/urandom, but
understand what the difference is. /dev/random is probably strong
(check with your system docs) and unless random data is available,
will "block" -- not return until there is. /dev/urandom is
pseudo-random, and will return always. However, it will use what
little randomness exists and stretch it out.
Note that your system is a lot like the method the
restful_authentication plugin does. restful_authentication uses a
global, per-application "secret" that is used much like a key, and a
per-user "salt" which is used as an IV. Its only purpose is to make
the password "mypassword" look different across users. The main
difference is that you can get the data back, while
restful_authentication is a one-way thing -- you can't recover a
password from what it stores.
For your purposes, you can probably generate a one-time "secure"
secret key, much like restful_authentication does, and store it in a
global. You would then generate per-item IVs of length 256-bits (32
characters) and store those, along with the encrypted CC info.
This is much more important to do if you have your application on one
server and the database on another. The database itself does not have
all the bits needed to open the data. Unfortunately, most people will
break into your web server, not your database engine.
As for what your digest thing does, it appears to "stretch" the
passphrase out from an ASCII format to something much longer. This is
commonly referred to as "key crunching" and does not make a more
secure key than just padding the value out by duplicating the
passphrase until it is long enough. That is, the digest thing is more
or less the same as using the string:
"whateverwhateverwhateverwhatever". The difference is that you can
look at the readable one and know it is a very bad key to use, while
9755571c6b7df2cdb1dddaaae2b399b7 almost looks random. (It's not.)
--Michael
Ram
Thanks a lot for the crash course! :)
The restful_authentication analogy helped and I get what you mean by
thats one-way.
Now about your suggestion for generating a one-time key and storing it
in a global variable, not having decided on the hosting yet, I was
wondering if its still secure if both the database and the application
were on the SAME server. Or is it best to keep the IV/key on a
separate server?
Oh and yes, the Digest thing.. what you said helped me understand
better. :)
By the way, I tried your test code above on IRB and I dont get the
same encrypted string. Here're the results..
>> AESCrypt.encrypt("this","X" * 32,"I" * 32)
>> AESCrypt.decrypt("\223?@\233?d\254u9]\241?M??","X" * 32,"I" * 32)
OpenSSL::CipherError: wrong final block length
from /Users/fire/Sites/Vinay/ROR/RealApps/fi_rest_auth/config/
initializers/aes_crypt.rb:20:in `final'
from /Users/fire/Sites/Vinay/ROR/RealApps/fi_rest_auth/config/
initializers/aes_crypt.rb:20:in `decrypt'
from (irb):2
Im guessing the ? marks suggest something?
Thanks again for the guidance Michael. Appreciate it :).
Michael Graff
really need to question if you should store CC info at all. What
purpose do you need to store it for? If recurring charges, most
processing centers will allow you to schedule that sort of thing and
not have to store the CC info locally. If logging, can you perhaps
store only the last few digits of the card?
If you are storing it "just in case" but don't expect to use it, I
would use a public key system, such as RSA. In RSA, you keep one of
the keys (the public one) on the server, and it encrypts data. To get
it out, you need the private key. People can hack away but so long as
they don't have the private key, they cannot read your data. (more
hand waving here, it IS possible, but we're entering into the world of
theoretical quantum computing here.)
The numerous times I've had to handle CC data I was explicitly
forbidden to store the numbers (card, expiry date, or CCV) other than
the last 6 digits of the card. I could store anything returned from
the processor though, like auth numbers, status codes, etc. This was
in all our cases sufficient to reverse charges (if needed) or to bill
in a recurring manner.
As for the examples, I think your "irb" is hiding data from you. Try
something like:
x = AESCrypt.encrypt(...)
AESCrypt.decrypt(x, ...)
I think your output is the same as mine, you just have ? marks where I
have some \301 escaped characters.
--Michael
Mike Cook
Hi,
I wonder whether this was resolved.
I'm having a similar issue.
I encrypt using a Flash AES CTS cipher from the library (hurlant). I'm
decrypting in Ruby. When used in Flash for both encrypt and decrypt,
all is joyous. However, when I try to use the openSSL package within
Ruby - the one mentioned above, I am able to decrypt most of the cipher
text. However, the last 17 bytes are garbled.
Lots of Google searching...
This is a test system, and in it, I set the IV to 16 bytes of 0. (I
really don't want to be yelled at here about the lack of security in
such a scheme)...
I get the same 'OpenSSL::CipherError: wrong final block length' error
when calling the 'final' method. 'update' works like a charm. I see
all of the clear test, in 32 byte chunks, up to the last partial block.
Then, I see garbled data in the last partial.
So, I'm guessing either the hurlant package handles the last partial
differently than Ruby's openSSL, or (much much much more likely), I am
doing something obviously wrong.
Could a kind soul please let me know if they have experience and
overcome such a problem?
Thanks,
Mike
--
Posted via http://www.ruby-forum.com/.
Michael Graff
'yell' at you :)
You are probably running into padding issues. Since AES is a block
cypher, each block must be exactly the correct length I suspect.
How are you calling the decrypt code? You're just feeding data into
the encryption system and after you are all done with the data,
calling final? That should work, but I do not have any way to write
code in Flash to test this. If you could have Flash encrypt
something, perhaps 101 bytes or so long (odd to ensure padding
occurs), and send the plaintext, the key, and the encrypted string
(uuencode, hex dump, whatever) I can experiment.
--Michael
Mike Cook
Hi,
Thanks... Definitely appears to be padding issues. We are using Bouncy
Castle CTS AES encrypt. The openSSL appears to be using CBC. I'm no
expert... In looking at the RFC, it appears there is a difference
between how CBC and CBC/CTS handles padding. There is a reference in
the rfc to swapping within the last partial block as part of the
process... not sure how/whether this is implemented.
Below is the output from my irb session. The clear text is supposed to
be:
AAAAAAAABBBBBBBBAAAAAAAABBBBBBBBCCCCCCCCDDDDDDDDCCCCCCCCDDDDDDDDAAAAAAAABBBBBBBBAAAAAAAABBBBBBBBCCCCCCCCDDDDDDDDCCCCCCCCDDDDDDDDXXXYYY
key: 4b1114cc73fed8b5428c3dee60d7773a
Please notice that the first part of the message is decrypted
correctly.... the partial at the end is not.
irb(main):821:0> a= OpenSSL::Cipher::Cipher.new('aes-256-cbc')
=> #<OpenSSL::Cipher::Cipher:0xb79b9378>
irb(main):822:0> a.key=d
=> "4b1114cc73fed8b5428c3dee60d7773a"
irb(main):823:0>
a.iv='00000000000000000000000000000000'.unpack('a2'*32).map{|x|
x.hex}.pack('c'*32)=>
"\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"
irb(main):824:0>
data='TmI9HrNrsMBxSfwApvSaQrLIDsLboNhIW/FawPjNUB0x/G0ZDf+gfk4JaTc/tGxDg1s4mrIRFOoBJemK+txUF0+aPw8bxIgzxmB3gq18aJRoSo5PWqbzS8FCCHrb3leKf4UUNFaIAaVVY1a5ymZ/HMPhwAKbii8x9Uk/S0MxaDofHTluc1E='.unpack('m')[0]
=>
"Nb=\036\263k\260\300qI\374\000\246\364\232B\262\310\016\302\333\240\330H[\361Z\300\370\315P\0351\374m\031\r\377\240~N\ti7?\264lC\203[8\232\262\021\024\352\001%\351\212\372\334T\027O\232?\017\e\304\2103\306`w\202\255|h\224hJ\216OZ\246\363K\301B\bz\333\336W\212\177\205\0244V\210\001\245UcV\271\312f\177\034\303\341\300\002\233\212/1\365I?KC1h:\037\0359nsQ"
irb(main):825:0> s=a.update(data)
=>
"AAAAAAAABBBBBBBBAAAAAAAABBBBBBBBCCCCCCCCDDDDDDDDCCCCCCCCDDDDDDDDAAAAAAAABBBBBBBBAAAAAAAABBBBBBBBCCCCCCCCDDDDDDDD8\300u\003|\200\243jf\bkf\005\251\327\242"
irb(main):826:0> s<<a.final
OpenSSL::Cipher::CipherError: wrong final block length
from (irb):826:in `final'
from (irb):826
from :0
irb(main):827:0>
Mike Cook
As a follow on, I was just reading an interesting article on Wikipedia.
Here's a snippet from it:
CBC ciphertext stealing decryption using a standard CBC interface
1. Dn = Decrypt (K, Cn−1). Decrypt the second to the last ciphertext
block.
2. Cn = Cn || Tail (Dn, B−M). Pad the ciphertext to the nearest
multiple of the block size using the last B−M bits of block cipher
decryption of the second-to-last ciphertext block.
3. Swap the last two ciphertext blocks.
4. Decrypt the ciphertext using the standard CBC mode.
5. Truncate the plaintext to the length of the original ciphertext.
Looks like the above is the mechanism to translate the last couple of
blocks of ciphertext to a form understandable by the CBC interface.
I think I understand points 1, 3, 4, and 5. I'm having a tougher time
with number 2, but I get the basic idea.
Let's say the the message is 134 bytes long. Then, I'd do 134 % 16 and
I get 6. If my block size is 128 bits (16 bytes*8 bits), then I'd
subtract 48 bits (6 bytes * 8 bits). That means 80 bits would be used
as a repeated pattern to pad the end of the message to make it evenly
divisible by 16. So, I'd add in one iteration (10 bytes) in this case,
to bring the message to 144 bytes (evenly divisible by the block size -
16).
Anyway, that's what I'm going to try. Please wish me luck ;-)
Mike
Mike Cook
Well... not what I was hoping for... Below is the Ruby code I wrote to
try to implement what I thought Wikipedia was getting at...
When I try to decipher the next to last block, the result comes back
nil. Therefore, there is nothing to pad to the end of the cipher
string.
The good news is, a colleague of mine pointed me to some c code. I was
experimenting with it last night and I was able to decipher characters
at the end of the string. That implementation called for the Xor of the
final blocks. It's not in a script yet... was working in IRB. Will
post that solution once I've completed it today.
But would really appreciate any comments on whether the Wikipedia
reference is correct, and if so, what the heck is wrong with the way I
tried to implement it.
Thanks!
Mike
require 'openssl'
require 'digest'
#set the cipher text and unpack
ct='TmI9HrNrsMBxSfwApvSaQrLIDsLboNhIW/FawPjNUB0x/G0ZDf+gfk4JaTc/tGxDg1s4mrIRFOoBJemK+txUF0+aPw8bxIgzxmB3gq18aJRoSo5PWqbzS8FCCHrb3leKf4UUNFaIAaVVY1a5ymZ/HMPhwAKbii8x9Uk/S0MxaDofHTluc1E='.unpack('m')[0]
aes=nil
#set the cipher
aes = OpenSSL::Cipher::Cipher.new('aes-256-cbc')
#set it to decrypt mode
aes.decrypt
#set the decryption key
aes.key='4b1114cc73fed8b5428c3dee60d7773a'
#set up the IV
aes.iv='00000000000000000000000000000000'.unpack('a2'*32).map{|x|
x.hex}.pack('c'*32)
#mod the string to find the partial block length
partialBlockLength = ct.length % 16
#this is the index of the cbc decrypted output:
endOfOutput = ct.slice(0..ct.length-16-partialBlockLength).length
#grab the next to last block
section = ct.slice(ct.length-partialBlockLength-16..ct.length -
partialBlockLength-1)
#decipher the next to last block
decihperedLastCompleteBlock = aes.update(section)
#pad the end of the cipher with the last blockSize-partialBlockLength
ct<<decihperedLastCompleteBlock.slice(decihperedLastCompleteBlock.length-(16-partialBlockLength)..decihperedLastCompleteBlock.length-1)
#swap the last two blocks
src=ct.slice(0..endOfOutput) + ct.slice(ct.length-16..ct.length-1) +
ct.slice(ct.length-32..ct.length-17)
aes=nil
#set the cipher
aes = OpenSSL::Cipher::Cipher.new('aes-256-cbc')
#set it to decrypt mode
aes.decrypt
#create the digest
d=Digest::MD5.hexdigest('7R]c\'1z<q\\O%eybO:michael_cook_611')
#set the decryption key
aes.key=d
#set up the IV
aes.iv='00000000000000000000000000000000'.unpack('a2'*32).map{|x|
x.hex}.pack('c'*32)
output = aes.update src
Mike Cook
some code with a colleague and translating it to Ruby. The CTS cipher
is now being deciphered using CBC.
I hope this helps some other folks in Ruby land. Please suggest some
ways to better document the steps.
Take care,
Mike
#key
d='4b1114cc73fed8b5428c3dee60d7773a'
tmpArray=nil
aes=nil
alpha=nil
omega=nil
#cipher text, unpacked from Base64
a='TmI9HrNrsMBxSfwApvSaQrLIDsLboNhIW/FawPjNUB0x/G0ZDf+gfk4JaTc/tGxDg1s4mrIRFOoBJemK+txUF0+aPw8bxIgzxmB3gq18aJRoSo5PWqbzS8FCCHrb3leKf4UUNFaIAaVVY1a5ymZ/HMPhwAKbii8x9Uk/S0MxaDofHTluc1E='.unpack('m')[0]
#compute the extra cipher beyond a 16 byte boundary
partial = a.length % 16
#set up the beginning and end of the CTS blocks we need to work on
alpha = a.length-16-partial
omega = a.length-1
#grab the end of the cipher
c=a.slice(alpha..omega)
#start + partial..end-partial
c<<a.slice(alpha+partial..omega-partial)
#first decrypt the beginning
aes = OpenSSL::Cipher::Cipher.new('aes-256-cbc')
aes.decrypt
aes.key=d
aes.iv='00000000000000000000000000000000'.unpack('a2'*16).map{|x|
x.hex}.pack('c'*16)
firstBlock=aes.update a.slice(0..((a.length/32)*32)-1)
#next decrypt the end
aes = OpenSSL::Cipher::Cipher.new('aes-256-cbc')
aes.decrypt
aes.key=d
aes.iv='00000000000000000000000000000000'.unpack('a2'*16).map{|x|
x.hex}.pack('c'*16)
decrypted3=aes.update c
lastBlock=""
#here we need to Xor the result against the contents of end of the
undecrypted cipher text
(0..partial-1).each {|index| tmpA='';tmpA=
(decrypted3[index]^c[16+index]).chr; lastBlock<<tmpA}
#put a number of bytes of undecrypted c then followed with the last
number of the decrypted c... the number of bytes sliced depends upon the
number of bytes in the partial block at the end of the cipher...
tmpArray =
c.slice(16..16+partial-1)<<decrypted3.slice(partial..decrypted3.length-1)
#now handle the middle
aes = OpenSSL::Cipher::Cipher.new('aes-256-cbc')
aes.decrypt
aes.key= d
aes.iv='00000000000000000000000000000000'.unpack('a2'*16).map{|x|
x.hex}.pack('c'*16)
tmpArray=tmpArray<<tmpArray
middle=aes.update tmpArray
#now take the last block of the decipherable original block
last = a.slice(alpha-16..alpha-1)
midBlock=""
#and ^or it against the middle component
(0..middle.length-1).each {|index|
tmpB='';tmpB<<(middle[index]^last[index]).chr;midBlock<<tmpB}
finalBlock = firstBlock+midBlock+lastBlock