Account Options

  1. Sign in
The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Google Groups Home
« Groups Home
forge interactions, ssl and ca_path/ca_file
There are currently too many topics in this group that display first. To make this topic appear first, remove this option from another topic.
There was an error processing your request. Please try again.
flag
  11 messages - Collapse all  -  Translate all to Translated (View all originals)
The group you are posting to is a Usenet group. Messages posted to this group will make your email address visible to anyone on the Internet.
Your reply message has not been sent.
Your post was successful
 
From:
To:
Cc:
Followup To:
Add Cc | Add Followup-to | Edit Subject
Subject:
Validation:
For verification purposes please type the characters you see in the picture below or the numbers you hear by clicking the accessibility icon. Listen and type the numbers you hear
 
Ken Barber  
View profile  
 More options Jun 24 2012, 9:45 pm
From: Ken Barber <k...@puppetlabs.com>
Date: Mon, 25 Jun 2012 02:45:17 +0100
Local: Sun, Jun 24 2012 9:45 pm
Subject: forge interactions, ssl and ca_path/ca_file
I just had an 'aha' moment when trying to make the PMT tool interact
with the forge using SSL.

The problem is with Ruby & OpenSSL and its need for a CA path or file
when I want to use VERIFY_PEER as a mechanism. Since
forge.puppetlabs.com uses a publicly signed certificate, I need to
provide the public CA.

I've managed to solve it on Linux by specifying:

https_object.ca_path = '/etc/ssl/certs'

But this isn't going to cut it cross-platform. I'm thinking there is
at least two viable solutions:

a) We hunt around a users OS looking for the cert directories, since
this is going to be prone to problems, we'd need to deal with edge
cases and possibly provide a knob or fallback behaviour for this
b) We ship our own lists of CA's. Its not enough to just ship only our
CA (GeoTrust) as they might be out of business one day, and we'll have
to deal with that. We'd probably need to ship a larger amount, like
the full list provided by curl or something. This would be something
we would continue to maintain and update indefinitely as well (so when
CA's get revoked, we would have to revoke as well).

Does anyone have any other ideas around a solution to this problem? Or
perhaps a viable alternative? Any help would be greatly appreciated.

ken.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Ken Barber  
View profile   Translate to Translated (View Original)
 More options Jun 25 2012, 8:11 am
From: Ken Barber <k...@puppetlabs.com>
Date: Mon, 25 Jun 2012 13:11:16 +0100
Local: Mon, Jun 25 2012 8:11 am
Subject: Re: forge interactions, ssl and ca_path/ca_file
(responding to puppet-dev)

>>> I've managed to solve it on Linux by specifying:
>>> https_object.ca_path = '/etc/ssl/certs'

> You managed to work around your broken build, I think.

You mean Debian 6's broken build - Lol ... found this using the system
ruby 1.8 from Debian, latest version :-).

> We should use the system certificate set, and *ONLY* the system
> certificate set, since that is the only that that will allow our users
> control over what is or is not trusted.

> Anything where we adjust the default settings is a terrible mistake:
> we are taking out of the hands of our users the right to manage trust.

> (With the obvious exception of our own private CA for internal use. :)

Sure - good point.

> On Linux that usually means installing `ca-certificates` or a
> similarly named package.  On the mac, with the system OpenSSL, that
> comes from KeyChain.  On Windows I don't actually know, but I would
> expect it to integrate with the system trust store.

> The most common place I have run into this with is RVM, either using a
> non-standard OpenSSL (eg: RVM package, or MacPorts), in which case the
> solution is to configure your other OpenSSL correctly also.

Sounds like I'll have to go through the various system Rubies and SSL
variants and found out how broken or weird they are, and find out what
is needed to make them work.

ken.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Ken Barber  
View profile   Translate to Translated (View Original)
 More options Jun 25 2012, 8:23 am
From: Ken Barber <k...@puppetlabs.com>
Date: Mon, 25 Jun 2012 13:23:06 +0100
Local: Mon, Jun 25 2012 8:23 am
Subject: Re: forge interactions, ssl and ca_path/ca_file
What is actually interesting about this topic is that the two common
command line tools for web access uses different methodologies. Curl
uses its own CA's, and wget uses the in-built OpenSSL list it seems.

I had a quick look at other Ruby tools to see what they do ... Octokit
turns off verify ... which is quite sad really, but I imagine this is
due to the hassles involved in making this work properly. I don't want
to take this route.

ken.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Nigel Kersten  
View profile  
 More options Jun 25 2012, 11:29 am
From: Nigel Kersten <ni...@puppetlabs.com>
Date: Mon, 25 Jun 2012 08:29:52 -0700
Local: Mon, Jun 25 2012 11:29 am
Subject: Re: [Puppet-dev] Re: forge interactions, ssl and ca_path/ca_file

On Mon, Jun 25, 2012 at 5:23 AM, Ken Barber <k...@puppetlabs.com> wrote:
> What is actually interesting about this topic is that the two common
> command line tools for web access uses different methodologies. Curl
> uses its own CA's, and wget uses the in-built OpenSSL list it seems.

Curl on OS X at least used to use the system keychain (built-in
version, not compiled versions from MacPorts etc)

> I had a quick look at other Ruby tools to see what they do ... Octokit
> turns off verify ... which is quite sad really, but I imagine this is
> due to the hassles involved in making this work properly. I don't want
> to take this route.

I very much don't want us to be distributing another bundle of certs.

When you're trying to maintain a tightly curated set of trusted root
certs, chasing down individual application cert bundles is immensely
frustrating.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Daniel Pittman  
View profile  
 More options Jun 25 2012, 12:54 pm
From: Daniel Pittman <dan...@puppetlabs.com>
Date: Mon, 25 Jun 2012 09:54:35 -0700
Local: Mon, Jun 25 2012 12:54 pm
Subject: Re: [Puppet-dev] Re: forge interactions, ssl and ca_path/ca_file

On Mon, Jun 25, 2012 at 5:11 AM, Ken Barber <k...@puppetlabs.com> wrote:
> (responding to puppet-dev)

>>>> I've managed to solve it on Linux by specifying:
>>>> https_object.ca_path = '/etc/ssl/certs'

>> You managed to work around your broken build, I think.

> You mean Debian 6's broken build - Lol ... found this using the system
> ruby 1.8 from Debian, latest version :-).

Did you install the `ca-certificates` package?  I never had any
problem with this on Debian, although they might have broken Ruby.
You should report it as an upstream bug if they have. :)

--
Daniel Pittman
⎋ Puppet Labs Developer – http://puppetlabs.com
♲ Made with 100 percent post-consumer electrons


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Josh Cooper  
View profile  
 More options Jun 25 2012, 12:54 pm
From: Josh Cooper <j...@puppetlabs.com>
Date: Mon, 25 Jun 2012 09:54:56 -0700
Local: Mon, Jun 25 2012 12:54 pm
Subject: Re: [Puppet-dev] Re: forge interactions, ssl and ca_path/ca_file

I think you just want to call X509_STORE_set_default_paths[1], which will
load the ca certs from the platform-appropriate trust store. There seems to
be a ruby method for it[2]

As far as Windows, there's an openssl patch to have it use capi (crypto
api) to load the trusted certs[3].

Josh

[1] http://fossies.org/dox/brx.src-1.6.0/x509__d2_8c_source.html#l00065
[2]
https://github.com/ruby/ruby/blob/trunk/ext/openssl/ossl_x509store.c#...
[3]
http://old.nabble.com/Get-root-certificates-from-System-Store-of-Wind...
--
Josh Cooper
Developer, Puppet Labs


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Daniel Pittman  
View profile   Translate to Translated (View Original)
 More options Jun 25 2012, 12:58 pm
From: Daniel Pittman <dan...@puppetlabs.com>
Date: Mon, 25 Jun 2012 09:58:20 -0700
Local: Mon, Jun 25 2012 12:58 pm
Subject: Re: [Puppet-dev] Re: forge interactions, ssl and ca_path/ca_file

That would be great to test - it shouldn't be necessary, since Ruby
OpenSSL should do that by default, but if it fixes Ken's problem we
might have some other latent ... dunno.  Oddity.  Change in semantics
between Ruby versions.  Something stupid like that.

Generally, though, you don't need to *do* anything to get Ruby OpenSSL
to validate commercial certificates.

--
Daniel Pittman
⎋ Puppet Labs Developer – http://puppetlabs.com
♲ Made with 100 percent post-consumer electrons


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Ken Barber  
View profile  
 More options Jun 25 2012, 1:06 pm
From: Ken Barber <k...@puppetlabs.com>
Date: Mon, 25 Jun 2012 18:06:49 +0100
Local: Mon, Jun 25 2012 1:06 pm
Subject: Re: [Puppet-dev] Re: forge interactions, ssl and ca_path/ca_file
Well, I've done an OS survey and found the following paths need to be
set, and in some cases couldn't find a path at all:

Redhat 5.6:
ca_file = /etc/pki/tls/certs/ca-bundle.crt

Debian 6 & 7:
ca_path = /etc/ssl/certs

OpenBSD 4.9:
ca_file = /etc/ssl/cert.pem # doesn't contain GeoTrust, so doesn't work for me

DragonFly BSD:
?? couldn't find it, need more time I presume

SLES 11:
ca_path = /etc/ssl/certs

Solaris 10:
ca_path = /opt/csw/etc/ssl/certs # not very good, as its CSW specific
ca_path = /etc/sfw/openssl/certs/ # is empty ... so I couldn't find
anything in core?

Solars 11:
ca_path = /opt/csw/etc/ssl/certs # not very good, as its CSW specific
ca_path = /etc/sfw/openssl/certs/ # no longer empty, but not the CA's we need

In all cases I had to set something in core with Ruby 1.8.7 btw. The
only case where it 'just worked' was in RVM Ruby 1.8.7 on my Mac. Ruby
1.9.3 with RVM didn't work, probably due to OpenSSL oddities.

I'm looking into Josh's pointers, seems like this should be working I
agree, or the path should be built in somehow somewhere. Here is the
sample stub code I've been testing with so far FWIW:

#!/usr/bin/env ruby

require 'net/https'

proxy_class = Net::HTTP::Proxy(nil, nil)
proxy = proxy_class.new('forge.puppetlabs.com', 443)
proxy.use_ssl = true
proxy.verify_mode = OpenSSL::SSL::VERIFY_PEER
# either: proxy.ca_path = 'whatever'
# or: proxy.ca_file = 'whatever'

response = nil
proxy.start do |http|
  request = Net::HTTP::Get.new('/')
  response = http.request(request)
end

puts response.body

ken.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Ken Barber  
View profile   Translate to Translated (View Original)
 More options Jun 25 2012, 1:10 pm
From: Ken Barber <k...@puppetlabs.com>
Date: Mon, 25 Jun 2012 18:10:16 +0100
Local: Mon, Jun 25 2012 1:10 pm
Subject: Re: [Puppet-dev] Re: forge interactions, ssl and ca_path/ca_file
So this works on Debian Josh - with no need to specify exact path:

require 'openssl'
require 'net/protocol'

cert_store = OpenSSL::X509::Store.new
cert_store.set_default_paths

ctx = OpenSSL::SSL::SSLContext.new
ctx.verify_mode = OpenSSL::SSL::VERIFY_PEER
# Comment this out, it breaks
ctx.cert_store = cert_store
s = TCPSocket.open('forge.puppetlabs.com', '443')
s = OpenSSL::SSL::SSLSocket.new(s, ctx)
s.sync_close = true
s.connect

Before I had to use ctx.ca_path = '/etc/ssl/certs'. I think we are on
to something here.

ken.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Ken Barber  
View profile   Translate to Translated (View Original)
 More options Jun 25 2012, 1:14 pm
From: Ken Barber <k...@puppetlabs.com>
Date: Mon, 25 Jun 2012 18:14:43 +0100
Local: Mon, Jun 25 2012 1:14 pm
Subject: Re: [Puppet-dev] Re: forge interactions, ssl and ca_path/ca_file
Booyah ... and this now works on Debian:

require 'net/https'

cert_store = OpenSSL::X509::Store.new
cert_store.set_default_paths

proxy_class = Net::HTTP::Proxy(nil, nil)
proxy = proxy_class.new('forge.puppetlabs.com', 443)
proxy.use_ssl = true
proxy.verify_mode = OpenSSL::SSL::VERIFY_PEER
proxy.cert_store = cert_store

response = nil
proxy.start do |http|
  request = Net::HTTP::Get.new('/')
  response = http.request(request)
end

puts response.body

Let me try this on all my various VM's now and confirm it.

ken.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Ken Barber  
View profile   Translate to Translated (View Original)
 More options Jun 25 2012, 1:28 pm
From: Ken Barber <k...@puppetlabs.com>
Date: Mon, 25 Jun 2012 18:28:15 +0100
Local: Mon, Jun 25 2012 1:28 pm
Subject: Re: [Puppet-dev] Re: forge interactions, ssl and ca_path/ca_file
So the set_default_paths has had some good success on quite a few
platforms, but not on all.

Good on:

* Redhat 5.6
* Debian 6 & 7
* SLES 11
* Solaris 10 (with CSW ruby)

Doesn't work on:

* DragonFly BSD 2.10
* OpenBSD 4.9
* Windows 2008

I think the general problem here is the lack of a default SSL path on
some platforms, which is either an upstream problem - or some package
I'm lacking (my BSD foo isn't great). This is however a good starting
point and feels like the 'right thing to do'. I'll keep working on the
details of those non-working platforms. Thanks a lot Josh & Daniel for
your help.

ken.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
End of messages
« Back to Discussions « Newer topic     Older topic »