Python & SSL
James Stroud
I have been trying to make an https client with python, but it seems
that, to do this, one needs to have the socket module compiled with ssl.
This is not the default. So I have a couple of questions.
1. Where do I specify to compile socket with ssl? I found no
obvious option in configure or setup.py or several other
files I checked.
2. Is there a way to do this without re-compiling all of python?
Also, I have done numerous combinations of searches with ssl, https, &
python as terms, but I haven't found a page outlining the steps to make
a certificate and key that python understands. Has anyone been
successful at this? Did you use openssl? I want to make sure I am doing
this part correctly.
Thanks in advance for any help.
James
--
James Stroud
UCLA-DOE Institute for Genomics and Proteomics
Box 951570
Los Angeles, CA 90095
Benji York
> I have been trying to make an https client with python
You probably don't want to use the standard library for HTTPS; here's a
quote from the socket module docs about SSL:
Warning: This does not do any certificate verification!
I'd recommend M2Crypto instead:
http://wiki.osafoundation.org/bin/view/Projects/MeTooCrypto
--
Benji York
James Stroud
Thank you, I will give M2Crypto a try.
John J. Lee
> I have been trying to make an https client with python, but it seems
What exactly do you mean by "make an https client"?
> that, to do this, one needs to have the socket module compiled with ssl.
> This is not the default. So I have a couple of questions.
>
> 1. Where do I specify to compile socket with ssl? I found no
> obvious option in configure or setup.py or several other
> files I checked.
What OS are you on?
> 2. Is there a way to do this without re-compiling all of python?
Are you sure it's NOT compiled in? But, if it's not compiled, it's
not compiled.
> Also, I have done numerous combinations of searches with ssl, https, &
> python as terms, but I haven't found a page outlining the steps to make
> a certificate and key that python understands. Has anyone been
> successful at this? Did you use openssl? I want to make sure I am doing
> this part correctly.
Since you say "make a certificate", and mention "https client", it
sounds like you want to authenticate yourself to an HTTP server using
an SSL certificate? If so, I don't believe the issue Benji raised is
relevant (that issue is relevant for fetching HTTPS URLs rather than
authenticating yourself to a server using an SSL certificate, I
think).
urllib claims to have support for this in the form of the key_file and
cert_file arguments to Urlopener constructor (UNTESTED):
import urllib
opener = urllib.URLopener(key_file="/path/to/my_key_file",
cert_file="/path/to/my_cert_file")
response = opener.open(url)
I can't claim to know that it actually works, though...
John
James Stroud
> James Stroud <jst...@ucla.edu> writes:
>
>
>>I have been trying to make an https client with python, but it seems
>
>
> What exactly do you mean by "make an https client"?
Something that can communicate with an https server. Fetch web pages,
send POST and GET information securely.
>>that, to do this, one needs to have the socket module compiled with ssl.
>>This is not the default. So I have a couple of questions.
>>
>> 1. Where do I specify to compile socket with ssl? I found no
>> obvious option in configure or setup.py or several other
>> files I checked.
>
>
> What OS are you on?
Linux FC 4 with my self-compiled versions of just about everything.
>
>> 2. Is there a way to do this without re-compiling all of python?
>
>
> Are you sure it's NOT compiled in? But, if it's not compiled, it's
> not compiled.
Its not compiled by default. I think I read this somewhere. I was
thinking of compiling just the socket module rather than installing over
my old version.
>
>>Also, I have done numerous combinations of searches with ssl, https, &
>>python as terms, but I haven't found a page outlining the steps to make
>>a certificate and key that python understands. Has anyone been
>>successful at this? Did you use openssl? I want to make sure I am doing
>>this part correctly.
>
>
> Since you say "make a certificate", and mention "https client", it
> sounds like you want to authenticate yourself to an HTTP server using
> an SSL certificate? If so, I don't believe the issue Benji raised is
> relevant (that issue is relevant for fetching HTTPS URLs rather than
> authenticating yourself to a server using an SSL certificate, I
> think).
>
> urllib claims to have support for this in the form of the key_file and
> cert_file arguments to Urlopener constructor (UNTESTED):
>
> import urllib
> opener = urllib.URLopener(key_file="/path/to/my_key_file",
> cert_file="/path/to/my_cert_file")
> response = opener.open(url)
At this point, authenticating is not an issue, though it would be nice
to know how to do. Mainly, I want to establish a secure connection for
2-way communication via https.
At any rate, I was able to make M2Crypto do what I wanted last night, so
I think I'll bypass the standard library route for now.
John J. Lee
> James Stroud wrote:
> > I have been trying to make an https client with python
>
> You probably don't want to use the standard library for HTTPS; here's a
> quote from the socket module docs about SSL:
>
> Warning: This does not do any certificate verification!
[...]
Of course, remembering that the first thing to ask in response to "is
it secure?" is "against what?", for lots of purposes it just doesn't
matter that it ignores certificates.
John
Sybren Stuvel
> Of course, remembering that the first thing to ask in response to
> "is it secure?" is "against what?", for lots of purposes it just
> doesn't matter that it ignores certificates.
I'm curious. Can you give me an example? AFAIK you need to know who
you're talking to before transmitting sensitive information, otherwise
you could be talking to anybody - and that's just what you wanted to
prevent with the encryption, right?
Sybren
--
The problem with the world is stupidity. Not saying there should be a
capital punishment for stupidity, but why don't we just take the
safety labels off of everything and let the problem solve itself?
Frank Zappa
Edward Elliott
> I'm curious. Can you give me an example? AFAIK you need to know who
> you're talking to before transmitting sensitive information, otherwise
> you could be talking to anybody - and that's just what you wanted to
> prevent with the encryption, right?
Encryption has multiple meanings. In the general sense, it encompasses all
of cryptography and the information security properties crypto provides.
This meaning is rather imprecise and you run into problems using it to
answer questions like yours. I won't encryption this way again in this
post.
In a more specific/technical sense, encryption protects the privacy of
transmitted information, preventing third-party eavesdropping. It makes no
guarantees who's on the other end of your encrypted pipe. Block cipher
modes (CBC-DES, CTR-Rijndael/AES, etc) and asymmetric cipher modes
(RSA-OAEP) are examples.
Integrity guarantees that the data sent is the same as the data received.
Authentication verifies the party on the other end of your pipe. This is
the primary purpose of SSL certs, authenticating web sites to browsers (you
get data encryption too, but that's somewhat less important). Note that it
doesn't verify the party's identity or trustworthiness, only that they know
a particular secret. The assumption is that if the certificate system is
setup correctly, possession of that secret makes them trustworthy (or
rather, you can trust the site because their secret proves that a cert
authority somewhere trusts them in some fashion). Trustworthy for what is
never defined.
If that sounds convoluted or illogical, it is. "Ten Risk of PKI" is a good
intro to why this chain of trust isn't all it's cracked up to be. It's
good reading to understand exactly what you benefits you get from an SSL
connection.
http://www.schneier.com/paper-pki.html
So in a long-winded way, I've answered your question. The short and sweet
of it is, yes, SSL is meant to prevent you from "talking to anybody". In
that it succeeds* -- but only so far as making sure the site you're talking
to paid some money to one of the dozens of cert authorities for a minimal
background check. You've gone from "anybody" to "anybody with a couple
hundred bucks and spare time to blow".
*at least, until the end user completely ignores the warning dialogs and
accepts whatever invalid cert he's presented with.
Of course if you control the server and serve a small clientele who already
trust you, you can have clients import your own cert so they really can be
sure who they're talking to -- as long as your master key remains secret.
Watch out for hackers and disgruntled employees.
The moral of this story is: computer security is an ugly, complex business.
Sybren Stuvel
> Encryption has multiple meanings. In the general sense, it
> encompasses all of cryptography and the information security
> properties crypto provides.
And if you already know who'll get the message, it's secure. I get it
:)
Thanks for the nice read ;-)
> Of course if you control the server and serve a small clientele who
> already trust you, you can have clients import your own cert so they
> really can be sure who they're talking to -- as long as your master
> key remains secret.
I'm used to using it like that. My own graduation project uses SSL to
thoroughly check the identity of both parties, without a web of trust.
> The moral of this story is: computer security is an ugly, complex
> business.
Yup.
John J. Lee
> John J. Lee enlightened us with:
> > Of course, remembering that the first thing to ask in response to
> > "is it secure?" is "against what?", for lots of purposes it just
> > doesn't matter that it ignores certificates.
>
> I'm curious. Can you give me an example? AFAIK you need to know who
> you're talking to before transmitting sensitive information, otherwise
> you could be talking to anybody - and that's just what you wanted to
> prevent with the encryption, right?
If Edward hadn't answered I would have said something along the lines
of what he said too, but more than that I just had in mind situations
where, when fetching a web page, the risk (probability and
consequences) of a man-in-the-middle attack doesn't feature much
higher than the risk of getting hit by a piece of debris from outer
space that day. Surprisingly often, it so happens that the people
setting up the web site used https, even though as a user of the site
I don't really care about the encryption or authentication.
That doesn't mean proper certificate handling wouldn't be good to have
(it would), just that installing m2crypto and finding the right docs
isn't necessarily worth the bother.
BTW, I assume the reason the OP (I forgot who that was) didn't have
https support compiled in was just that they didn't have OpenSSL
installed when they typed ./configure (since the Python build process
on unix uses autoconf). Either that or they installed a system
package to get Python (e.g. an .rpm) and the SSL support is is a
separate package (seems unlikely).
John