Kerberos authentication

[mze...@vmware.com]

This summer I am working in an environment where we have a well-established Linux + Kerberos + Active Directory infrastructure. We would prefer to re-use that infrastructure rather than maintain a separate private key infrastructure for puppet. So, I am investigating enabling Kerberos authentication for node <-> master authentication. So far it looks like the work breaks up into several pieces:

1 MASTER

1.a Configure Apache2 for Kerberos HTTP authentication

1.b Retrieve the principle from the web request.

1.c Enhance Puppet::Network::AuthStore::Declaration to support Active Directory host principles (e.g. myhost$@EXAMPLE.COM).

2 AGENT

2.a Acquire/build a Kerberos / GSSAPI capable HTTP client library

2.b Integrate it into the puppet agent lifecycle

I'm new to puppet and related technologies so would appreciate any high level guidance/suggestions/etc. that the community might have to offer.

mjz

---

Further details and questions (in no particular order)

* POC master progress - So far I have a very small set of non-production quality edits to the puppet master code that allows GET production/node/host$@EXAMPLE.COM to work as expected when using "curl --negotiate -u:" as a client. See:

https://github.com/mzeren-vmw/puppet/commit/draft_kerberos_baby_steps.

I am just starting to create proper patches for pull requests.

* POC agent progress - I have not yet found a GSSAPI capable http client library, but I have found and successfully tested https://github.com/cbeer/net-ssh-kerberos which depends on git://github.com/zenchild/gssapi.git (MIT) which in turn depends on ffi (LGPLv3). When I get to the client side work I may leverage gssapi to extend Net::HTTP?

* How would Kerberos integrate with MCollective? - I don't really understand the relationship between mcollective and puppet yet. How might Kerberos auth affect mcollective based features or future enhancements?

* Is pulling REMOTE_USER from the environment acceptable? Or do we need a way to get the mod_auth_kerb user into a request header as we do for  X-Client-DN and X-Client-Verify? So far mod_rewrite hasn't been cooperating here...

* Apache mod_auth_kerb only? I am focused on using Apache2's mod_auth_kerb. I don't anticipate getting this to work for webrick, and haven't yet investigated nginx. Would a single webserver solution be acceptable?

* There's a related bug here: https://projects.puppetlabs.com/issues/3958 discussion: https://groups.google.com/forum/#!msg/puppet-dev/ECIvFzj4dKo/ivOqTQEKVeoJ

* Are there any plans to support ruby 2.0? Net::HTTP apparently has basic auth support in 2.0, which might lead to cleaner Kerberos support (but I really haven't looked at this in detail yet.)

[Trevor Vaughan]

If you're going to pursue this, be sure to use Kerberos for encryption as well.

A lot of sensitive data can get passed around by Puppet.

It's a good idea, but I feel that it's going to be quite a daunting task given that you'll need to bind through Apache/Nginx -> Passenger and then figure out how to translate that into a certname portion for Puppet to use.

That said, it would be pretty awesome.

Trevor

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.
To post to this group, send email to puppe...@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-dev?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Trevor Vaughan
Vice President, Onyx Point, Inc
(410) 541-6699
tvau...@onyxpoint.com

-- This account not approved for unencrypted proprietary information --

[Nan Liu]

Adventurous project.

On Tue, Jun 4, 2013 at 5:14 PM, <mze...@vmware.com> wrote:

* How would Kerberos integrate with MCollective? - I don't really understand the relationship between mcollective and puppet yet. How might Kerberos auth affect mcollective based features or future enhancements?

You have the option of PSK, AES, and SSL for MCollective security. I don't believe ActiveMQ supports Kerberos. RabbitMQ added it as a plugin very recently, so I doubt anyone explored using this. You will need to write a kerberos plugin (https://github.com/puppetlabs/marionette-collective/tree/master/plugins/mcollective/security).  I would ask on the MCollective mailing list or ping vulcane on IRC #mcollective.

Nan

[mze...@vmware.com]

On Tuesday, June 4, 2013 7:11:05 PM UTC-7, Nan Liu wrote:

You have the option of PSK, AES, and SSL for MCollective security. I don't believe ActiveMQ supports Kerberos. RabbitMQ added it as a plugin very recently, so I doubt anyone explored using this. You will need to write a kerberos plugin (https://github.com/puppetlabs/marionette-collective/tree/master/plugins/mcollective/security).  I would ask on the MCollective mailing list or ping vulcane on IRC #mcollective.

Nan, can you give me an example operation that uses mcollective and will break if the agent doesn't have a client cert?

[Nan Liu]

It's a plugin system, so TLS is not mandatory, simply the default recommendation. MCollective security is independent from puppet and using the puppet cert is due to convenience, and not because the two product share the same credentials. 

You can find out more at the following link:

http://docs.puppetlabs.com/mcollective/security.html

HTH,

Nan

[Trevor Vaughan]

Qpid natively supports Kerberos and, as far as I can tell, works fine with MCollective.

Trevor

--

You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.
To post to this group, send email to puppe...@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-dev?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Andy Parker]

On Tue, Jun 4, 2013 at 5:14 PM, <mze...@vmware.com> wrote:

This summer I am working in an environment where we have a well-established Linux + Kerberos + Active Directory infrastructure. We would prefer to re-use that infrastructure rather than maintain a separate private key infrastructure for puppet. So, I am investigating enabling Kerberos authentication for node <-> master authentication. So far it looks like the work breaks up into several pieces:

1 MASTER

1.a Configure Apache2 for Kerberos HTTP authentication

1.b Retrieve the principle from the web request.

1.c Enhance Puppet::Network::AuthStore::Declaration to support Active Directory host principles (e.g. myhost$@EXAMPLE.COM).

2 AGENT

2.a Acquire/build a Kerberos / GSSAPI capable HTTP client library

2.b Integrate it into the puppet agent lifecycle

I'm new to puppet and related technologies so would appreciate any high level guidance/suggestions/etc. that the community might have to offer.

mjz

As Trevor said, you'll also need to make sure that the channel remains encrypted. I think that the portions of the code that you are going to be trying to swap out (namely network communication infrastructure) are not currently designed for that. It will probably take a lot of moving things around and creation of some new classes to make something like this slot in cleanly. Also, all of this functionality will need to be optional so that scenarios that don't use kerberos don't take on new dependencies.

 

---

Further details and questions (in no particular order)

* POC master progress - So far I have a very small set of non-production quality edits to the puppet master code that allows GET production/node/host$@EXAMPLE.COM to work as expected when using "curl --negotiate -u:" as a client. See:

https://github.com/mzeren-vmw/puppet/commit/draft_kerberos_baby_steps.

I am just starting to create proper patches for pull requests.

Before you start making pull requests for something this large, I think it would be good to outline the parts that you are changing and what the design will be. This also seems like something that would fit well into the armature process (https://github.com/puppetlabs/armatures).

 

* POC agent progress - I have not yet found a GSSAPI capable http client library, but I have found and successfully tested https://github.com/cbeer/net-ssh-kerberos which depends on git://github.com/zenchild/gssapi.git (MIT) which in turn depends on ffi (LGPLv3). When I get to the client side work I may leverage gssapi to extend Net::HTTP?

I have to admit that I've never tried to do Kerberos over HTTP. A few quick web searches uncovered SPNEGO (http://en.wikipedia.org/wiki/SPNEGO) which looks to be the standard for negotiating various authentication mechanisms over http.

 

* How would Kerberos integrate with MCollective? - I don't really understand the relationship between mcollective and puppet yet. How might Kerberos auth affect mcollective based features or future enhancements?

MCollective and puppet are fairly independent components. MCollective has a plugin (agent in its terms) for controlling puppet, but puppet has very little knowledge, if any, of mcollective. I believe that mco already has a pluggable authentication mechanism, so kerberos might fit into it already.

 

 

* Is pulling REMOTE_USER from the environment acceptable? Or do we need a way to get the mod_auth_kerb user into a request header as we do for  X-Client-DN and X-Client-Verify? So far mod_rewrite hasn't been cooperating here...

I think it depends on how portable across rack implementations that is.

 

* Apache mod_auth_kerb only? I am focused on using Apache2's mod_auth_kerb. I don't anticipate getting this to work for webrick, and haven't yet investigated nginx. Would a single webserver solution be acceptable?

I wouldn't worry about webrick. It is really only acceptable for development setups. As far as multiple webserver setups, we try to stick to plain rack, if possible. When that isn't possible then try to make sure that there is a way of configuring either puppet or the webserver to work together. I believe that historically we have tried to work with apache + passenger and nginx + ??? (that one is a little more fuzzy to me).

 

* There's a related bug here: https://projects.puppetlabs.com/issues/3958 discussion: https://groups.google.com/forum/#!msg/puppet-dev/ECIvFzj4dKo/ivOqTQEKVeoJ

* Are there any plans to support ruby 2.0? Net::HTTP apparently has basic auth support in 2.0, which might lead to cleaner Kerberos support (but I really haven't looked at this in detail yet.)

Ruby 2 should be support as of 3.2 (http://docs.puppetlabs.com/puppet/3/reference/release_notes.html#ruby-20-support)

 

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.
To post to this group, send email to puppe...@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-dev?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

Andrew Parker

an...@puppetlabs.com

Freenode: zaphod42

Twitter: @aparker42

Software Developer

Join us at PuppetConf 2013, August 22-23 in San Francisco - http://bit.ly/pupconf13
Register now and take advantage of the Early Bird discount - save 25%!

[mze...@vmware.com]

Andy, thanks for all your answers.

On Wednesday, June 5, 2013 9:25:38 AM UTC-7, Andy Parker wrote:

As Trevor said, you'll also need to make sure that the channel remains encrypted. I think that the portions of the code that you are going to be trying to swap out (namely network communication infrastructure) are not currently designed for that. It will probably take a lot of moving things around and creation of some new classes to make something like this slot in cleanly. Also, all of this functionality will need to be optional so that scenarios that don't use kerberos don't take on new dependencies.

WRT securing the channel: our current plans are to continue to rely on server certificates for channel encryption. I should amend my prior description by saying that we want to use Kerberos for *agent* authentication. Technically Kerberos provides master auth too, but we will be relying on traditional certificate based SSL server auth as part of securing the channel. It might be possible to use SLL with Kerberos TLS and skip the master certs, but that's not something that we need now.

WRT moving code / backward compatibility: so far the server changes look relatively non-invasive. The client changes seem likely to be more invasive, but I understand the requirement to not regress on any existing behavior/dependencies. My goal is to make it "just work" with minimal to zero configuration. We'll see how achievable that is...

 

Before you start making pull requests for something this large, I think it would be good to outline the parts that you are changing and what the design will be. This also seems like something that would fit well into the armature process (https://github.com/puppetlabs/armatures).

Yes. I want to get through a round of client prototyping first, but I think an armature would help structure this work. When I know a bit more I'll work on a draft.

 

I wouldn't worry about webrick. It is really only acceptable for development setups. As far as multiple webserver setups, we try to stick to plain rack, if possible. When that isn't possible then try to make sure that there is a way of configuring either puppet or the webserver to work together. I believe that historically we have tried to work with apache + passenger and nginx + ??? (that one is a little more fuzzy to me)

OK. That's helpful. I think I'm working within standard apache idioms though there are some details that remain to be worked out. We can discuss this more when I start working up a formal change for this part of the master modifications (POC rest.rb diff here: https://github.com/mzeren-vmw/puppet/commit/draft_kerberos_baby_steps).

[Trevor Vaughan]

Given that you're going to be using client certs for encryption, why would you bother with Kerberos authentication?

The Puppet certificate system exists within a private ecosystem (unless you change that) so you're not really gaining any additional control over your environment.

Are there some fancy features that I might be missing or do those who fear the command line want to do agent revocation via a GUI (at which point use Dogtag or something like it).

Trevor

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.
To post to this group, send email to puppe...@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-dev?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

[Wil Cooley]

On Jun 5, 2013 12:18 PM, "Trevor Vaughan" <tvau...@onyxpoint.com> wrote:
>
> Given that you're going to be using client certs for encryption, why would you bother with Kerberos authentication?

This is what I was wondering as I read this too. One benefit I could see (but at a much smaller scoped project) is the ability to securely auto-sign certs.

If you're already joining a machine to a Kerberos realm, it would be one less step if you could use that trust to provide validation for signing Puppet's client cert. Might be a bigger win for Windows systems, where Kerberos is all but required.

Wil

[Trevor Vaughan]

I'm not sure how you would gain anything on the cert auto-signing.

In both cases, without some additional authorization, the assumption is that DNS and/or DHCP are trusted and that the network is locked down. Without that, you don't want to be auto-registering *anything*.

If you're already joining a machine to a Kerberos realm, then you're probably either doing it at install time using a first layer authorization subsystem (razor type install), or you're hopping on after the fact to register the system, or you're using Puppet to do it.

Even the FreeIPA client registration requires someone with a valid Realm trust to execute it or an embedded password that expires after the first use that is snagged using Kickstart fun (and thus potentially interceptable).

All in all, it's just a hard problem.

--

You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.
To post to this group, send email to puppe...@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-dev?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[mze...@vmware.com]

On Tuesday, June 11, 2013 6:38:59 PM UTC-7, Trevor Vaughan wrote:

If you're already joining a machine to a Kerberos realm, then you're probably either doing it at install time using a first layer authorization subsystem (razor type install), or you're hopping on after the fact to register the system, or you're using Puppet to do it.

Right. The step where a privileged operation adds a machine to the domain is something that already has to happen. By using Kerberos authentication of agents to the puppet master we eliminate a second privileged operation to register the machine to puppet, and also avoid deploying PKI for agents. We still need PKI for puppet masters -- but that infrastructure is already in place.

I hope that makes sense. I think these are important justification points for the armature.

[Erik Dalén]

Seems like a decent alternative would be to just have a second webservice/on top of puppet that allows agents to authenticate with their kerberos token and authorize their SSL certificate request that way. That should be fairly easy to build with just some fiddling with mod auth kerb, apache configs and puppet auth.conf.

--

You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.
To post to this group, send email to puppe...@googlegroups.com.

Visit this group at http://groups.google.com/group/puppet-dev.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Erik Dalén

[Andy Parker]

On Tue, Jun 18, 2013 at 5:36 AM, Erik Dalén <erik.gus...@gmail.com> wrote:

Seems like a decent alternative would be to just have a second webservice/on top of puppet that allows agents to authenticate with their kerberos token and authorize their SSL certificate request that way. That should be fairly easy to build with just some fiddling with mod auth kerb, apache configs and puppet auth.conf.

Pluggable autosign? We were talking about working on that some internally and there is a pull request open right now that would lay the groundwork.

I would say that we are going to be working on it *real soon now*, but you all probably know my track record on making statements about what we will be working on :(

 

On 17 June 2013 23:44, <mze...@vmware.com> wrote:

On Tuesday, June 11, 2013 6:38:59 PM UTC-7, Trevor Vaughan wrote:

If you're already joining a machine to a Kerberos realm, then you're probably either doing it at install time using a first layer authorization subsystem (razor type install), or you're hopping on after the fact to register the system, or you're using Puppet to do it.

Right. The step where a privileged operation adds a machine to the domain is something that already has to happen. By using Kerberos authentication of agents to the puppet master we eliminate a second privileged operation to register the machine to puppet, and also avoid deploying PKI for agents. We still need PKI for puppet masters -- but that infrastructure is already in place.

I hope that makes sense. I think these are important justification points for the armature.

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.
To post to this group, send email to puppe...@googlegroups.com.

Visit this group at http://groups.google.com/group/puppet-dev.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Erik Dalén

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.
To post to this group, send email to puppe...@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-dev.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

[mze...@vmware.com]

On Tuesday, June 18, 2013 5:36:30 AM UTC-7, Erik Dalén wrote:

Seems like a decent alternative would be to just have a second webservice/on top of puppet that allows agents to authenticate with their kerberos token and authorize their SSL certificate request that way. That should be fairly easy to build with just some fiddling with mod auth kerb, apache configs and puppet auth.conf.

In our case the obvious external certificate signing service would be Microsoft Certificate Services. However if we can pull off Kerberos authentication then we do not need to deploy and maintain that service.

[Daniel Pittman]

I have to say, in the presence of functional Kerberos infrastructure,
I wouldn't want to operate a second, shadow, authentication system for
an admin tool. Having two places to maintain security credentials is
much worse than just having one.

--
Daniel Pittman
⎋ Puppet Labs Developer – http://puppetlabs.com
♲ Made with 100 percent post-consumer electrons

[mze...@vmware.com]

On Tuesday, June 18, 2013 8:51:20 AM UTC-7, Andy Parker wrote:

On Tue, Jun 18, 2013 at 5:36 AM, Erik Dalén <erik.gus...@gmail.com> wrote:

Seems like a decent alternative would be to just have a second webservice/on top of puppet that allows agents to authenticate with their kerberos token and authorize their SSL certificate request that way. That should be fairly easy to build with just some fiddling with mod auth kerb, apache configs and puppet auth.conf.

Pluggable autosign? We were talking about working on that some internally and there is a pull request open right now that would lay the groundwork.

While pluggable auto-sign as discussed here: https://github.com/puppetlabs/puppet/pull/1522 sounds very useful, our objective is to separate "authentication secret management" from "configuration management". We would like to avoid having the puppet master have any CA role. We could do that by deploying an external CA, but Kerberos support would allow us to skip that additional deployment and maintenance work for agents that already have Kerberos configured correctly.

I hope that justification makes sense, and again, I'll try to capture it in an armature.

[Erik Dalén]

On 18 June 2013 17:51, Andy Parker <an...@puppetlabs.com> wrote:

On Tue, Jun 18, 2013 at 5:36 AM, Erik Dalén <erik.gus...@gmail.com> wrote:

Seems like a decent alternative would be to just have a second webservice/on top of puppet that allows agents to authenticate with their kerberos token and authorize their SSL certificate request that way. That should be fairly easy to build with just some fiddling with mod auth kerb, apache configs and puppet auth.conf.

Pluggable autosign? We were talking about working on that some internally and there is a pull request open right now that would lay the groundwork.

I would say that we are going to be working on it *real soon now*, but you all probably know my track record on making statements about what we will be working on :(

Hmm, had a look at that pull request. Why not use a indirector for autosign? And then just implement a exec terminus similar to the exec node terminus instead of a autosign_command option?

It would be really useful in my opinion to just expose the autosigning over the REST API anyway. So the autosign.conf can be managed using that (something that foreman_proxy provides otherwise).

But my suggestion here wasn't really managing autosign.conf using kerberos or so. It was that you use mod_authnz_krb in apache on the /.*/certificate_status/{certname} endpoint and just allow everyone in the puppet auth.conf. But only allow kerberos tokens matching their host name in the apache config. That way hosts can manage their own certificate status using a krb authenticated HTTP request. So request a cert in the regular way and then use their already existing krb token to authenticate the CSR.

--
Erik Dalén

[mze...@vmware.com]

On Wednesday, June 19, 2013 3:02:26 AM UTC-7, Erik Dalén wrote:

But my suggestion here wasn't really managing autosign.conf using kerberos or so. It was that you use mod_authnz_krb in apache on the /.*/certificate_status/{certname} endpoint and just allow everyone in the puppet auth.conf. But only allow kerberos tokens matching their host name in the apache config. That way hosts can manage their own certificate status using a krb authenticated HTTP request. So request a cert in the regular way and then use their already existing krb token to authenticate the CSR.

Again, the desire (security requirement) here is that the puppet master not be responsible for issuing certificates to agents. If we were to use agent certificates we would likely deploy MS Certificate Services to issue client certs to the agents. However we would like to avoid that extra infrastructure if we can get puppet agents to authenticate via Kerberos. In the prototyping that I've been working on the agent does not have a certificate.

[Eric Sorenson]

On Jun 19, 2013, at 3:02 AM, Erik Dalén <erik.gus...@gmail.com> wrote:

Hmm, had a look at that pull request. Why not use a indirector for autosign? And then just implement a exec terminus similar to the exec node terminus instead of a autosign_command option?

I don't have an opinion about krb but I thought through the indirection idea and do not think that it's appropriate here.

Dom Cleal put a similar idea on the PR so I commented there: 

https://github.com/puppetlabs/puppet/pull/1522#issuecomment-19724225

Eric Sorenson - eric.s...@puppetlabs.com
#puppet irc: eric0