Authenticated encryption with PyCrypto
Daniel
and I'd appreciate feedback on this recipe:
http://code.activestate.com/recipes/576980/
Of course, it does not meet all of the requirements set forth by the
OP in the referenced thread (the pycrypto dependency is a problem),
but it is an attempt to provide a simple interface for performing
strong, password-based encryption. Are there already modules out there
that provide such a simple interface? If there are, they seem to be
hiding somewhere out of Google's view.
I looked at ezPyCrypto, but it seemed to require public and private
keys, which was not convenient in my situation... maybe password-based
encryption is trivial to do with ezPyCrypto as well? In addition to
ezPyCrypto, I looked at Google's keyczar, but despite the claims of
the documentation, the API seemed overly complicated. Is it possible
to have a simple API for an industry-strength encryption module? If
not, is it possible to document that complicated API such that a non-
cryptographer could use it and feel confident that he hadn't made a
critical mistake?
Also, slightly related, is there an easy way to get the sha/md5
deprecation warnings emitted by PyCrypto in Python 2.6 to go away?
~ Daniel
M.-A. Lemburg
> Just got done reading this thread:
>
> http://groups.google.com/group/comp.lang.python/browse_thread/thread/b31a5b5f58084f12/0e09f5f5542812c3
>
> and I'd appreciate feedback on this recipe:
>
> http://code.activestate.com/recipes/576980/
>
> Of course, it does not meet all of the requirements set forth by the
> OP in the referenced thread (the pycrypto dependency is a problem),
> but it is an attempt to provide a simple interface for performing
> strong, password-based encryption. Are there already modules out there
> that provide such a simple interface? If there are, they seem to be
> hiding somewhere out of Google's view.
>
> I looked at ezPyCrypto, but it seemed to require public and private
> keys, which was not convenient in my situation... maybe password-based
> encryption is trivial to do with ezPyCrypto as well? In addition to
> ezPyCrypto, I looked at Google's keyczar, but despite the claims of
> the documentation, the API seemed overly complicated. Is it possible
> to have a simple API for an industry-strength encryption module? If
> not, is it possible to document that complicated API such that a non-
> cryptographer could use it and feel confident that he hadn't made a
> critical mistake?
Yes, it is possible, but whatever you come up with will usually
be bound to just one (or a few) different use cases, e.g. just
look at the different cipher modes there are, the different key
sizes, block sizes (for block ciphers), IV strings, padding, etc.
etc.
Note that your code has a padding bug: the decoder doesn't
undo the padding. You're lucky though, since pickle will only
read as much data as it needs and not complain about the extra
data it finds.
You are also using CBC mode, even though you are really after
ECB mode (your code doesn't use chaining). With ECB mode, you
don't need the IV string.
> Also, slightly related, is there an easy way to get the sha/md5
> deprecation warnings emitted by PyCrypto in Python 2.6 to go away?
Yes: you silence them via the warnings module. I suppose that the
latest version of PyCrypto fixes these warnings.
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source (#1, Jan 26 2010)
>>> Python/Zope Consulting and Support ... http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
________________________________________________________________________
::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
http://www.egenix.com/company/contact/
geremy condra
<snip>
> You are also using CBC mode, even though you are really after
> ECB mode (your code doesn't use chaining). With ECB mode, you
> don't need the IV string.
However, ECB mode is not as secure- the IV is the right way to go
here.
I'd also note that you aren't supposed to use RandomPool anymore,
and that AES-192 is frequently recommended over AES-256 for
new applications due to a number of recent developments in
the cryptanalysis of its key schedule.
Geremy Condra
Paul Rubin
> Of course, it does not meet all of the requirements set forth by the
> OP in the referenced thread (the pycrypto dependency is a problem),
> but it is an attempt to provide a simple interface for performing
> strong, password-based encryption. Are there already modules out there
> that provide such a simple interface? If there are, they seem to be
> hiding somewhere out of Google's view.
http://www.nightsong.com/phr/crypto/p3.py
I need to update it to handle 64-bit OS's and use Python 3.x-style print
statements, but that is pretty trivial.
M.-A. Lemburg
> Daniel wrote:
>> Just got done reading this thread:
>>
>> http://groups.google.com/group/comp.lang.python/browse_thread/thread/b31a5b5f58084f12/0e09f5f5542812c3
>>
>> and I'd appreciate feedback on this recipe:
>>
>> http://code.activestate.com/recipes/576980/
>>
> You are also using CBC mode, even though you are really after
> ECB mode (your code doesn't use chaining). With ECB mode, you
> don't need the IV string.
Sorry. Forget that last comment - your code does use chaining.
It's hidden away in the PyCrypto code for block cipher encoding:
the .encode() method will automatically apply the cipher to
all blocks of 32 byte input data and apply chaining to all
of them (initializing the data area with the IV string).
M.-A. Lemburg
geremy condra wrote:
> On Tue, Jan 26, 2010 at 12:37 PM, M.-A. Lemburg <m...@egenix.com> wrote:
>
> <snip>
>
>> You are also using CBC mode, even though you are really after
>> ECB mode (your code doesn't use chaining). With ECB mode, you
>> don't need the IV string.
>
> However, ECB mode is not as secure- the IV is the right way to go
> here.
Right - I forgot that PyCrypto applies the chaining internally
when being passed data of more than 32 bytes.
> I'd also note that you aren't supposed to use RandomPool anymore,
> and that AES-192 is frequently recommended over AES-256 for
> new applications due to a number of recent developments in
> the cryptanalysis of its key schedule.
Do you have pointers for this ?
I could only find
http://en.wikipedia.org/wiki/Advanced_Encryption_Standard#Security
and
https://cryptolux.org/Block
My reading of their FAQ (https://cryptolux.org/FAQ_on_the_attacks)
is that using AES-128 is the way to go (and it's faster too) - at
least for the time being.
Daniel
> Note that your code has a padding bug: the decoder doesn't
> undo the padding. You're lucky though, since pickle will only
> read as much data as it needs and not complain about the extra
> data it finds.
Doesn't the last line in decrypt() do it?
return data[:-ord(data[-1])]
Given, it's a bit cryptic... no pun intended :)
> > Also, slightly related, is there an easy way to get the sha/md5
> > deprecation warnings emitted by PyCrypto in Python 2.6 to go away?
>
> Yes: you silence them via the warnings module. I suppose that the
> latest version of PyCrypto fixes these warnings.
The version that gets installed by easy_install or pip (2.0.1) emits
those warnings. Is there a more recent version?
Thanks for the feedback.
~ Daniel
M.-A. Lemburg
> On Jan 26, 12:37 pm, "M.-A. Lemburg" <m...@egenix.com> wrote:
>> Note that your code has a padding bug: the decoder doesn't
>> undo the padding. You're lucky though, since pickle will only
>> read as much data as it needs and not complain about the extra
>> data it finds.
>
> Doesn't the last line in decrypt() do it?
>
> return data[:-ord(data[-1])]
>
> Given, it's a bit cryptic... no pun intended :)
That's cryptic indeed... I just found that you're not padding
with zero bytes, but instead with char(pad) where pad is the
number of bytes you add:
pad = AES_BLOCK_SIZE - len(data) % AES_BLOCK_SIZE
data = data + pad * chr(pad)
This code will pad with 16 bytes of chr(16) in case len(data)
is in fact on a block size boundary.
When using pickle, you don't need this, since pickle includes
all necessary length information in the serialized data stream.
I'd just pad with \0 and not worry about the extra bytes
at the end when using pickle to serialize the objects.
It's more important to worry about whether you really
want to unpickle the data or not, since pickle opens
up lots of possibilities of executing code on the decoding
side of the communication channel.
>>> Also, slightly related, is there an easy way to get the sha/md5
>>> deprecation warnings emitted by PyCrypto in Python 2.6 to go away?
>>
>> Yes: you silence them via the warnings module. I suppose that the
>> latest version of PyCrypto fixes these warnings.
>
> The version that gets installed by easy_install or pip (2.0.1) emits
> those warnings. Is there a more recent version?
This is the most recent version:
http://www.dlitz.net/software/pycrypto/
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source (#1, Jan 27 2010)
Daniel
> I'd also note that you aren't supposed to use RandomPool anymore,
OK, I updated the recipe to use os.urandom()
> and that AES-192 is frequently recommended over AES-256 for
> new applications due to a number of recent developments in
> the cryptanalysis of its key schedule.
Changed easily enough. The updated recipe defaults to AES-192.
I also made one other minor tweak: the global constants were moved to
class-level so they can be overridden more easily if needed.
Daniel
Daniel
> Daniel wrote:
> > On Jan 26, 12:37 pm, "M.-A. Lemburg" <m...@egenix.com> wrote:
> >> Note that your code has a padding bug: the decoder doesn't
> >> undo the padding. You're lucky though, since pickle will only
> >> read as much data as it needs and not complain about the extra
> >> data it finds.
>
> > Doesn't the last line in decrypt() do it?
>
> > return data[:-ord(data[-1])]
>
> > Given, it's a bit cryptic... no pun intended :)
>
> That's cryptic indeed... I just found that you're not padding
> with zero bytes, but instead with char(pad) where pad is the
> number of bytes you add:
>
> pad = AES_BLOCK_SIZE - len(data) % AES_BLOCK_SIZE
> data = data + pad * chr(pad)
>
> This code will pad with 16 bytes of chr(16) in case len(data)
> is in fact on a block size boundary.
>
> When using pickle, you don't need this, since pickle includes
> all necessary length information in the serialized data stream.
>
> I'd just pad with \0 and not worry about the extra bytes
> at the end when using pickle to serialize the objects.
I think I'll leave the padding in there since it keeps the encrypt/
decrypt methods usable for non-pickle data.
> It's more important to worry about whether you really
> want to unpickle the data or not, since pickle opens
> up lots of possibilities of executing code on the decoding
> side of the communication channel.
I understand the risks of unpickle. With strong, authenticated
encryption I think it is reasonably safe to send an encrypted pickle
through an untrusted medium (the Internet) and know that it has not
been modified enroute. That is, unless someone has obtained the key,
in which case I have a bigger problem to worry about.
> >>> Also, slightly related, is there an easy way to get the sha/md5
> >>> deprecation warnings emitted by PyCrypto in Python 2.6 to go away?
>
> >> Yes: you silence them via the warnings module. I suppose that the
> >> latest version of PyCrypto fixes these warnings.
>
> > The version that gets installed by easy_install or pip (2.0.1) emits
> > those warnings. Is there a more recent version?
>
> This is the most recent version:
>
> http://www.dlitz.net/software/pycrypto/
Thanks.
Daniel
geremy condra
>
>
> geremy condra wrote:
>> On Tue, Jan 26, 2010 at 12:37 PM, M.-A. Lemburg <m...@egenix.com> wrote:
>>
>> <snip>
>>
>>> You are also using CBC mode, even though you are really after
>>> ECB mode (your code doesn't use chaining). With ECB mode, you
>>> don't need the IV string.
>>
>> However, ECB mode is not as secure- the IV is the right way to go
>> here.
>
> Right - I forgot that PyCrypto applies the chaining internally
> when being passed data of more than 32 bytes.
>
>> new applications due to a number of recent developments in
>> the cryptanalysis of its key schedule.
>
>
> I could only find
> http://en.wikipedia.org/wiki/Advanced_Encryption_Standard#Security
> and
> https://cryptolux.org/Block
>
Sure. The best source is the papers themselves:
http://eprint.iacr.org/2009/374.pdf
http://eprint.iacr.org/2009/317.pdf
http://eprint.iacr.org/2009/242.pdf
If you're interested I can provide more. The other thing
to keep in mind is that several of the new attacks abuse
algebraic properties of the key schedule, and that older
attacks like XSL (eXtended Sparse Linearisation) and
models (like the BES) are useful tools. There's also
been some recent research into using bent functions
for cryptanalysis, including cryptanalysis over
GF(p), which falls into the lap of older models like
the BES, which views AES in terms of GF(2^8),
although I don't seem to be able to find free links
to those papers.
> My reading of their FAQ (https://cryptolux.org/FAQ_on_the_attacks)
> is that using AES-128 is the way to go (and it's faster too) - at
> least for the time being.
In applications where speed is not of the essence, there doesn't
appear to be a good reason to prefer AES-128 to AES-192,
and while the margin is small, recent attacks have consistently
led to results that are 2^10 or 2^12 larger for 192 than for 128.
I also doubt highly that the wells opened by these newer results
are the end of this question- so my advice would be to use 192
where possible, and keep an eye out just in case.
Geremy Condra