SSL certificates broken by default

[Dāvis Mosāns]

When you install Ruby, https won't work, because `OpenSSL::X509::DEFAULT_CERT_FILE` points to incorrect certificate location (C:/Users/Luis/Code/luislavena/knap-build/var/knapsack/software/x64-windows/openssl/1.0.0l/ssl/cert.pem)

And worse it's not documented anywhere. RubyInstaller doesn't mention that it won't work and there aren't anything about this in FAQ either.

Firstly I would suggest that Ruby would use relative path to pem, if that's not possible then hardcode it to something like "C:/Ruby/cert.pem" which is much better than current default path.

Now next thing, need to document, that this can be changed with SSL_CERT_FILE env variable which should point to location of valid CA bundle. Currently I see RubyGems and other applications/gems bundle their own certificates, because there's just no other way.

I see mostly everyone uses certificates provided by Mozilla NSS project from https://mxr.mozilla.org/mozilla-release/source/security/nss/lib/ckfw/builtins/certdata.txt?raw=1 and with CURL's perl script they can be converted to usable PEM format.

So IMO best solution would be, for installer, after Ruby installing, download certificates from Mozilla and convert to PEM format and put in Ruby directory and then optionally if it's installed elsewhere than hardcoded path then set SSL_CERT_FILE env variable.

Also there should be some utility/script which could update certificates. I might make something like this in Ruby, to convert from NSS format to PEM, because I don't like that currently need perl for that.

[Luis Lavena]

Hello Davis,

On Sun, May 4, 2014 at 11:58 PM, Dāvis Mosāns <davi...@gmail.com> wrote:

When you install Ruby, https won't work, because `OpenSSL::X509::DEFAULT_CERT_FILE` points to incorrect certificate location (C:/Users/Luis/Code/luislavena/knap-build/var/knapsack/software/x64-windows/openssl/1.0.0l/ssl/cert.pem)

And worse it's not documented anywhere. RubyInstaller doesn't mention that it won't work and there aren't anything about this in FAQ either.

Firstly I would suggest that Ruby would use relative path to pem, if that's not possible then hardcode it to something like "C:/Ruby/cert.pem" which is much better than current default path.

Now next thing, need to document, that this can be changed with SSL_CERT_FILE env variable which should point to location of valid CA bundle. Currently I see RubyGems and other applications/gems bundle their own certificates, because there's just no other way.

I see mostly everyone uses certificates provided by Mozilla NSS project from https://mxr.mozilla.org/mozilla-release/source/security/nss/lib/ckfw/builtins/certdata.txt?raw=1 and with CURL's perl script they can be converted to usable PEM format.

So IMO best solution would be, for installer, after Ruby installing, download certificates from Mozilla and convert to PEM format and put in Ruby directory and then optionally if it's installed elsewhere than hardcoded path then set SSL_CERT_FILE env variable.

Thank you for taking the time to write this do some investigation. Really sorry you hit this problem, like many others on Windows and the lack of details on the FAQ/Troubleshooting page.

I would like to give a bit more of background on this, so please bear with me.

As you pointed out, OpenSSL extension constant DEFAULT_CERT_FILE points to a path that do not exists on your system. Actually, that path do not exists anywhere but the build machine that created the OpenSSL binaries.

Here is the thing: is not Ruby which sets the value of such constant, but is OpenSSL. It defines its value at compilation time and is not possible to alter it afterwards.

There has been attempts to solve this over the years:

1) Bundle with RubyInstaller a base set of certificates and modify the end-user environment for setting SSL_CERT_FILE on installation.

2) Patch OpenSSL Ruby's extension (which is not the same as OpenSSL source code) to take a relative path to a PEM file.

3) Make OpenSSL use Windows own certificate store, which was left unresponded and unaccepted for almost 2 years and never merged.

I listed those in the order it has been requested to be made, but finally, the reverse order should be performed.

RubyInstaller approach has always been not alter Ruby source code and not to alter it's dependencies or the user environment. This has been a painful point for some newcomers but a praised point for others.

Setting SSL_CERT_FILE on behalf of the user and altering their environment can result in unexpected side effects for other tools that depend on OpenSSL, like curl, wget or other languages that link against it.

Taking the approach of patching Ruby's OpenSSL extension has been suggested before:

https://github.com/oneclick/rubyinstaller/pull/47

As you can read from the comments and the issue on Redmine:

https://bugs.ruby-lang.org/issues/2586

But again, was never fixed into OpenSSL becasue seems was never a problem for OpenSSL developers or users :(

--

Also there should be some utility/script which could update certificates. I might make something like this in Ruby, to convert from NSS format to PEM, because I don't like that currently need perl for that.

I don't mind if we proceed the following way:

1) Document the lack of certificates of default OpenSSL configuration in the Wiki

Wiki is open and you can add content freely as long you don't remove other parts that are important.

2) Add a Ruby or a VBScript/tool to the installer that systematically generates the PEM file using the computer certificate store.

3) Document how to set SSL_CERT_FILE but do not modify that automatically on behalf of the user, at least, not without approval.

Sounds good to you?

Thanks again for taking your time and sharing your thoughts. Please excuse the delay on the response and do not hesitate share any progress you make on this.

Regards,

-- 

Luis Lavena
AREA 17
-
Perfection in design is achieved not when there is nothing more to add,
but rather when there is nothing more to take away.
Antoine de Saint-Exupéry

[Dāvis Mosāns]

That sounds good :) I guess we should ask OpenSSL guys what's up with it.

But I would say Ruby script is way more better than writing something in VBScript. Ruby would be already installed and thus perfectly usable.

By the way another thing, http://rubyinstaller.org is available only over HTTP, thus theoretically you could be MITM'ed and get backdoored Ruby installer, for example with sabotaged OpenSSL :D I don't see any file checksums either.

[Dāvis Mosāns]

I sent email to OpenSSL regarding that patch, info about it can be viewed at http://rt.openssl.org/Ticket/Display.html?user=guest&pass=guest&id=2158

[Luis Lavena]

Checksums are listed in the release email and also available on the downloads page and the downloads on bintray.

Sorry for top posting. Sent from mobile.

--
You received this message because you are subscribed to the Google Groups "RubyInstaller" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rubyinstalle...@googlegroups.com.
To post to this group, send email to rubyin...@googlegroups.com.
Visit this group at http://groups.google.com/group/rubyinstaller.
For more options, visit https://groups.google.com/d/optout.

[Dāvis Mosāns]

oops, didn't saw, sorry.

only md5 aren't safe anymore to use because collisions have been found and can be calculated, take a look at http://www.mscs.dal.ca/~selinger/md5collision/ and http://stackoverflow.com/questions/1999824/whats-the-shortest-pair-of-strings-that-causes-an-md5-collision

I apologize for bugging you so much.

[Dāvis Mosāns]

Stephen Henson from OpenSSL replied about patch, but there haven't been any response from original author Yui NARUSE. Maybe he's busy or not interested pursuing it anymore. So in any case, if someone have time it would be great if could adopt this patch and finally get proper certificate support for Windows.

I think no one before had need for Windows certificate store in OpenSSL, because generally when you're writing application you use OpenSSL only on *nix and for Windows use Cryptography API directly, which is more easier because don't have to compile OpenSSL for windows, especially if using MSVC.

So another way how to go about this, could be implementing usage of Cryptography API in Ruby for Windows instead of OpenSSL. But that's probably huge work and because currently Ruby doesn't have any abstraction layer (OpenSSL leaks directly in ruby's code when using `https` eg.) and many gems depends on this behavior (catching SSL's exceptions) it's probably not practical. But overall it would be nice if Ruby could use other libraries for cryptography not only OpenSSL.


2014-05-11 15:09 GMT+03:00 Stephen Henson via RT <r...@openssl.org>:

    There are a few problems with this patch.

    Firstly it will add all Windows store certificates by default to any OpenSSL
    application which uses X509_STORE_set_default_paths. Existing applications
    might not want to arbitrarily trust the same certificates Windows does. A new
    call would be preferable requiring explicit application support.

    Also AFAICS it takes no account of trust settings. Windows has some
    certificates trusted for code signing, server or client authentication. It
    isn't appropriate (for example) to trust certificates for server authentication
    if Windows only trusts them for client authentication. This should extract
    Windows store trust settings and convert them to OpenSSL form.

    As far as the code is concerned I've not reviewed it in any detail but I think
    this could be a problem:

    x509 = d2i_X509(NULL, (const unsigned char**)&cert->pbCertEncoded, cert->cbCertEncoded);

    It will modify: cert->pbCertEncoded: a temporary variable should be used
    instead.

    Steve.
    --
    Dr Stephen N. Henson. OpenSSL project core developer.

[Luis Lavena]

Hello Davis,

Thank you for your response and follow up and sorry for my delayed response.

Please see comments below.

On Wed, May 28, 2014 at 9:53 AM, Dāvis Mosāns <davi...@gmail.com> wrote:

Stephen Henson from OpenSSL replied about patch, but there haven't been any response from original author Yui NARUSE. Maybe he's busy or not interested pursuing it anymore. So in any case, if someone have time it would be great if could adopt this patch and finally get proper certificate support for Windows.

I recall there was other thread where the same issues described by Dr. Stephen Henson were made and that indicated how inviable this approach was (I remember there was a comment from Yui also about it).

Considering that OpenSSL might not apply this patch in this current state and the lack of a supporter to fully debug and implement, I don't think will be easy to pursue it (either short or long term plan).

I think no one before had need for Windows certificate store in OpenSSL, because generally when you're writing application you use OpenSSL only on *nix and for Windows use Cryptography API directly, which is more easier because don't have to compile OpenSSL for windows, especially if using MSVC.

It would have been great if OpenSSL offered a wrapper around Windows own functionality so it could be used transparently, which is not the case.

We can regret about it all that we want, but OpenSSL has been across multiple platforms since I can remember and we might pay the price about its issues (heartbleed and others)

 

So another way how to go about this, could be implementing usage of Cryptography API in Ruby for Windows instead of OpenSSL. But that's probably huge work and because currently Ruby doesn't have any abstraction layer (OpenSSL leaks directly in ruby's code when using `https` eg.) and many gems depends on this behavior (catching SSL's exceptions) it's probably not practical. But overall it would be nice if Ruby could use other libraries for cryptography not only OpenSSL.

There has been attempts to provide a better and sane cryptography library for Ruby, called Krypt:

https://github.com/krypt/krypt

It has the option of pluggable providers (openssl being one) and is written mostly in Ruby, which is more easy to understand.

But, as usual with good tools, it didn't catch on developers to aim this to replace Ruby's OpenSSL from standard library or any Ruby on Windows developer to help.

I'm by no means an expert on Windows API and even less on cryptography, so I cannot take the challenge and work on this to provide a Windows provider.

Even so, it will be a challenge by itself to replace OpenSSL usage all over libraries and others that depend on it.

None of the RubyInstaller contributors, neither myself, are paid or sponsored to work on RubyInstaller, Ruby on Windows or similar. We do it because we want to fix stuff that affects us.

Life will be much easier if OpenSSL could setup it's constant at compile time that are relative to the library current directory (at runtime) and not at build time (which is the current issue).

If OpenSSL project could accept a patch for that, perhaps using a special flag to the configure script, then we could build a version of OpenSSL that uses it and build Ruby against it.

That could allow us provide a script or something that obtains the certs from Windows and place them in the relative PEM/CRT file.

While it doesn't fix OpenSSL on Windows, it might fix the issue most of the Rubyst hits.

Do you think is a viable compromise?

I'm not sure how much I can contribute to make it happen, and not sure if OpenSSL team will apply/include that patch, but we most likely can do custom builds for that (as long the changes are not too big).

Right now RubyInstaller uses the packages build using knapsack (small DSL to build things):

https://github.com/luislavena/knapsack-recipes/tree/master/openssl

You can search this group for instructions on how to get knap-build and build recipes yourself. For testing the patch, I recommend you build following similar steps but locally, not using that tool until we iron it out the issues.

Sounds like an interesting challenge you would like to participate? :-D

Regards and have a nice weekend.

[Luis Lavena]

Hello,

Missed this message, see my response inline.

On Sun, May 11, 2014 at 4:48 PM, Dāvis Mosāns <davi...@gmail.com> wrote:

oops, didn't saw, sorry.

only md5 aren't safe anymore to use because collisions have been found and can be calculated, take a look at http://www.mscs.dal.ca/~selinger/md5collision/ and http://stackoverflow.com/questions/1999824/whats-the-shortest-pair-of-strings-that-causes-an-md5-collision

Indeed MD5 are not safe anymore, which checksum algorithm do you recommend?

Here is the code that generates it:

https://github.com/oneclick/rubyinstaller/blob/master/recipes/packager/inno.rake#L4-L12

We can change it and replace all the invocations to md5_file with the right one.

Cheers,

[Dāvis Mosāns]

 

It would have been great if OpenSSL offered a wrapper around Windows own functionality so it could be used transparently, which is not the case.

I don't think that would be a good way, because OpenSSL is a cryptography library itself and IMO it doesn't really make sense for it be in just "wrapper" mode in which it itself does nothing. That should be different libraries responsibility who abstracts multiple cryptography libraries (eg. OpenSSL, Windows Crypt, etc.) and provides unified interface, independent from backed cryptography library. I haven't researched about this but I wonder if someone have made such library. Like I said, I've mostly seen that applications does this themselves (maybe because there aren't such abstraction libraries?)

There has been attempts to provide a better and sane cryptography library for Ruby, called Krypt:

https://github.com/krypt/krypt

It has the option of pluggable providers (openssl being one) and is written mostly in Ruby, which is more easy to understand.

But, as usual with good tools, it didn't catch on developers to aim this to replace Ruby's OpenSSL from standard library or any Ruby on Windows developer to help.

I'm by no means an expert on Windows API and even less on cryptography, so I cannot take the challenge and work on this to provide a Windows provider.

Even so, it will be a challenge by itself to replace OpenSSL usage all over libraries and others that depend on it.

Krypt seems nice. I hadn't heard about this before and it looks really promising. My only concern would be about performance, how it compares to fully native implementations. Maybe it's not an issue.

Anyway I think it might be worth contacting with Ruby's developers and ask them what they think about implementing some crypto abstraction in Ruby itself, then for example applications shouldn't have to catch OpenSSL exceptions but just some general crypto exceptions and also crypto backend could be replaced easily without need for any changes in ruby applications. By the way I saw that Ruby is going to Gemify OpenSSL (Feature #9612) which I think is a good step forward. OpenSSL isn't only cryptography library, you may have heard about LibreSSL and then there's also NaCl and probably dozens of other libraries. I think Ruby language shouldn't force users to use one particular crypto library.

 

Life will be much easier if OpenSSL could setup it's constant at compile time that are relative to the library current directory (at runtime) and not at build time (which is the current issue).

If OpenSSL project could accept a patch for that, perhaps using a special flag to the configure script, then we could build a version of OpenSSL that uses it and build Ruby against it.

That could allow us provide a script or something that obtains the certs from Windows and place them in the relative PEM/CRT file.

While it doesn't fix OpenSSL on Windows, it might fix the issue most of the Rubyst hits.

Do you think is a viable compromise?

I'm not sure how much I can contribute to make it happen, and not sure if OpenSSL team will apply/include that patch, but we most likely can do custom builds for that (as long the changes are not too big).

I think this is a good idea and worth going for. But this shouldn't be only option. I think it's worth doing both. So improve Ruby for cryptography abstraction, support for multiple cryptography providers and patch OpenSSL to allow relative certificate location. Also with this Ruby crypto abstraction, applications can be updated one at a time and some still can use OpenSSL directly like they've used to do. Firstly should just migrate Ruby core libraries to use this abstraction.
 

Sounds like an interesting challenge you would like to participate? :-D

It could be interesting indeed. And I wouldn't have any problem doing this if only I would have more free time... I'm really busy with so many things I've to do that simply I don't have time for this in nearest future. I don't know when I'll have time to go for this. So I really hope there are other people not just me and Luis who could do something about this.

[Dāvis Mosāns]

 

Indeed MD5 are not safe anymore, which checksum algorithm do you recommend?

SHA-256 would be very good choice (it's also quite popular), there are no known weaknesses discovered yet (nor theoretical, nor practical). But I also would suggest publishing SHA-512 too. Then for example extra paranoid people could check both hashes. Even if one would get compromised it's really really unlikely that it would be possible to generate same binary matching both hashes. Also then it would be possible to check only SHA-512.


There's also SHA-3, but it's not yet widely available. And it's only advantage over SHA-2 is that it's faster to calculate on hardware.

[Dāvis Mosāns]

By the way I saw that Ruby is going to Gemify OpenSSL (Feature #9612) which I think is a good step forward.

whoops, I gave wrong link, it's this https://bugs.ruby-lang.org/issues/9612

[Gary Ewan Park]

Hello Luis,

Thanks for taking the time to write down the details of this issue!  I am facing a similar, but different issue, that I think you will be able to help with.  Here is that I have:

Firstly, I should point out that I am running Windows (I have tried on both Windows 7 and 8 with same results), and I have used Chocolatey to install Ruby, rather than using RubyInstaller (but I don't think this should be a problem, based on my understanding).

What I am trying to get working is the Flickr plugin for Octopress, which you can find here:

https://github.com/neilk/octopress-flickr

The long and short of it is, this doesn't work.  I have been able to break this down to a single test case.  Namely:

Take the Authentication snippet from here and place it into a ruby file somewhere (i.e. c:\temp\test.rb):

http://hanklords.github.io/flickraw/#label-Authentication

Add in your Flickr API keys, and run:

ruby -rubygems C:\temp\test.rb

This will generate the SSL Error mentioned above (NOTE: I have done this exact test on a friends MacBook Pro and it worked fine).

I have then tried a number of workarounds, including setting the environment variable SSL_CERT_FILE, creating the named folder for luislavena and placing cert.cem there, and nothing works.

Do you have any other suggestions about what I could try, or where I could be going wrong?

Thanks

Gary